From 99789a00a14eff626983601164977c4bd70ccbdb Mon Sep 17 00:00:00 2001 From: root Date: Sat, 18 Jul 2026 22:16:09 +0000 Subject: [PATCH] no-mistakes(document): Reframed infra-maintenance contract as deliberate partition --- infrastructure-maintenance.prose.md | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/infrastructure-maintenance.prose.md b/infrastructure-maintenance.prose.md index e0dbf07..96c12bf 100644 --- a/infrastructure-maintenance.prose.md +++ b/infrastructure-maintenance.prose.md @@ -7,10 +7,11 @@ description: > running containers, container restarts with health verification, post-update verification of every critical service (LiteLLM proxy, SearXNG, Zulip, Gitea, PM2 processes, Hermes gateways), and rollback on failure. Consolidates the - raw shell scripts that previously did this piecemeal and closes the Docker - image management gap that infrastructure-update left open (infra-update owns - cluster-wide apt waves; this contract owns the host-level maintenance loop - on the primary host + Docker image lifecycle). Runs Sunday 2am ET. Owner: + raw shell scripts that previously did this piecemeal. This contract owns the + HOST-LEVEL weekly maintenance loop on the primary host plus Docker image + pulls ONLY for .116 and .7, while infrastructure-update owns the FULL-FLEET + cluster-wide wave (apt across the full PVE cluster + CTs/VMs AND its Docker + image Wave 3 across all stacks). Runs Sunday 2am ET. Owner: ops (firstmate secondmate). Blast radius: an unverified image pull can break LiteLLM (all agents lose inference) or SearXNG (search-stack down); a bad apt upgrade can leave the host in a half-upgraded state. Pre-update backup check @@ -34,9 +35,12 @@ version: 1.0.0 ## Scope Primary host is the maintenance host where apt updates apply. Docker image pulls -span the two Docker ecosystems that run critical services. Topology, CT IDs, and -IPs are live-state fields — verify against `infrastructure-control.prose.md` -(the source of truth) and the live system before mutating. +span the two Docker ecosystems that run critical services. infrastructure-update +runs the full-fleet cluster-wide wave (including its Wave 3 Docker pulls across +all stacks/hosts); this contract runs a narrower host-level weekly pull limited +to .116 and .7. Topology, CT IDs, and IPs are live-state fields — verify against +`infrastructure-control.prose.md` (the source of truth) and the live system +before mutating. | Host | IP | Role | Trust | |------|----|------|-------| @@ -189,7 +193,7 @@ Escalation: none | warning | critical | fatal ## Related Contracts -- `infrastructure-update.prose.md` — cluster-wide apt waves across the PVE cluster + CTs/VMs (the build-phase update); this contract handles the host-level maintenance loop and Docker image lifecycle. +- `infrastructure-update.prose.md` — owns the full-fleet cluster-wide wave INCLUDING its Wave 3 Docker image updates across all stacks (SearXNG, Firecrawl, Inference Harness on .116, home stack, audiobookshelf); infrastructure-maintenance is a deliberately narrower host-level weekly pull scoped to .116 and .7. - `infrastructure-monitoring.prose.md` — provides the pre-update health baseline (depends_on). - `infrastructure-control.prose.md` — topology source of truth (CT IDs, IPs, hostnames). - `litellm-health.prose.md` — LiteLLM probe details.