feat(contracts): add infrastructure-maintenance contract #24

Merged

jerome merged 4 commits from fm/infra-maint-contract into master 2026-07-19 01:32:30 +00:00

Conversation 0 Commits 4 Files Changed 4

+312 -2

abiba-bot commented 2026-07-18 22:18:45 +00:00

Owner

Copy Link

Copy Source

Summary

Adds a new prose contract infrastructure-maintenance.prose.md and registers it in contract-registry.yaml. The contract consolidates system-level maintenance previously handled by raw shell scripts into a typed, scheduled responsibility, and establishes a deliberate partition with infrastructure-update:

• infrastructure-update owns the full-fleet cluster-wide wave (apt across the PVE cluster + CTs/VMs, and its existing Wave 3 Docker image updates across all stacks).
• infrastructure-maintenance (new) owns the narrower host-level weekly maintenance loop on the primary host plus Docker image pulls scoped to .116 and .7 only.

Contract scope

1. OS package updates — apt update && apt upgrade -y on the primary host, with a pre-update snapshot/backup check
2. Docker image pulls — pull latest stable tags for LiteLLM, SearXNG, and other running containers on .116/.7
3. Container restarts — restart stacks one at a time with per-stack health verification (120s)
4. Post-update service verification — LiteLLM proxy + MCP gateway, SearXNG, Zulip, Gitea, PM2 processes, Hermes gateways, with a regression check against a pre-update health baseline
5. Rollback on failure — image (pin digest in compose, up -d) / apt (Proxmox snapshot or apt install <pkg>=<old>) / config restore, with escalation

Contract metadata

field	value
name	infrastructure-maintenance
kind	responsibility
category	maintenance
sensitivity	high
owner	ops (firstmate secondmate)
agent	ops
trigger	scheduled, weekly Sunday 2am ET (0 2 * * 0)
escalation	warning→abiba+mumuni, critical→abiba+mumuni, fatal→abiba+mumuni+kwame
circuit_breaker	max_retries 2, window 7200, trip_action escalate_to_fatal
depends_on	infrastructure-monitoring (pre-update health baseline)

Registry updates

• Added infrastructure-maintenance to index.by_category.maintenance, index.by_domain.infrastructure, index.by_owner.ops (new), index.by_trigger.scheduled, index.by_sensitivity.high
• Added ops to the top-level owners list
• Changed infrastructure-update owner abiba → ops (in its contracts: entry and the by_owner index)
• Added the full contracts: entry for infrastructure-maintenance (modeled on infrastructure-update)

Validation

• bash scripts/prose-lint.sh — ✅ PASSED (10 pre-existing warnings, zero new from this change)
• python3 -c "import yaml; yaml.safe_load(open('contract-registry.yaml'))" — ✅ valid
• Frontmatter parses; required ## Maintains section present (responsibility kind)
• Index ↔ contracts consistency verified (every indexed name has a contracts: entry)
• no-mistakes pipeline: intent ✅ rebase ✅ review ✅ (fixed broken rollback syntax) test ✅ document ✅ (reframed as deliberate partition per reviewer) lint ✅ push ✅

Notes

• Frontmatter follows existing convention (kind/name/description/agent/triggers/version only) — category/sensitivity/owner/trigger/escalation/circuit_breaker/depends_on live in the registry, matching infrastructure-update and proxmox-monitor. No schema drift.
• All host/IP values grounded in infrastructure-update and infrastructure-control; every live-state field marked VERIFY-BEFORE-USE per the authoring guide.
• infrastructure-update.prose.md content is unchanged (only its registry owner moved to ops).

## Summary Adds a new prose contract `infrastructure-maintenance.prose.md` and registers it in `contract-registry.yaml`. The contract consolidates system-level maintenance previously handled by raw shell scripts into a typed, scheduled responsibility, and establishes a deliberate partition with `infrastructure-update`: - **`infrastructure-update`** owns the full-fleet cluster-wide wave (apt across the PVE cluster + CTs/VMs, *and* its existing Wave 3 Docker image updates across all stacks). - **`infrastructure-maintenance`** (new) owns the narrower **host-level** weekly maintenance loop on the primary host plus Docker image pulls scoped to `.116` and `.7` only. ## Contract scope 1. **OS package updates** — `apt update && apt upgrade -y` on the primary host, with a pre-update snapshot/backup check 2. **Docker image pulls** — pull latest stable tags for LiteLLM, SearXNG, and other running containers on `.116`/`.7` 3. **Container restarts** — restart stacks one at a time with per-stack health verification (120s) 4. **Post-update service verification** — LiteLLM proxy + MCP gateway, SearXNG, Zulip, Gitea, PM2 processes, Hermes gateways, with a regression check against a pre-update health baseline 5. **Rollback on failure** — image (pin digest in compose, `up -d`) / apt (Proxmox snapshot or `apt install <pkg>=<old>`) / config restore, with escalation ## Contract metadata | field | value | |-------|-------| | name | infrastructure-maintenance | | kind | responsibility | | category | maintenance | | sensitivity | high | | owner | ops (firstmate secondmate) | | agent | ops | | trigger | scheduled, weekly Sunday 2am ET (`0 2 * * 0`) | | escalation | warning→abiba+mumuni, critical→abiba+mumuni, fatal→abiba+mumuni+kwame | | circuit_breaker | max_retries 2, window 7200, trip_action escalate_to_fatal | | depends_on | infrastructure-monitoring (pre-update health baseline) | ## Registry updates - Added `infrastructure-maintenance` to `index.by_category.maintenance`, `index.by_domain.infrastructure`, `index.by_owner.ops` (new), `index.by_trigger.scheduled`, `index.by_sensitivity.high` - Added `ops` to the top-level `owners` list - Changed `infrastructure-update` owner `abiba` → `ops` (in its `contracts:` entry and the `by_owner` index) - Added the full `contracts:` entry for `infrastructure-maintenance` (modeled on `infrastructure-update`) ## Validation - `bash scripts/prose-lint.sh` — ✅ PASSED (10 pre-existing warnings, zero new from this change) - `python3 -c "import yaml; yaml.safe_load(open('contract-registry.yaml'))"` — ✅ valid - Frontmatter parses; required `## Maintains` section present (responsibility kind) - Index ↔ contracts consistency verified (every indexed name has a `contracts:` entry) - no-mistakes pipeline: intent ✅ rebase ✅ review ✅ (fixed broken rollback syntax) test ✅ document ✅ (reframed as deliberate partition per reviewer) lint ✅ push ✅ ## Notes - Frontmatter follows existing convention (`kind`/`name`/`description`/`agent`/`triggers`/`version` only) — `category`/`sensitivity`/`owner`/`trigger`/`escalation`/`circuit_breaker`/`depends_on` live in the registry, matching `infrastructure-update` and `proxmox-monitor`. No schema drift. - All host/IP values grounded in `infrastructure-update` and `infrastructure-control`; every live-state field marked `VERIFY-BEFORE-USE` per the authoring guide. - `infrastructure-update.prose.md` content is **unchanged** (only its registry owner moved to `ops`).

abiba-bot added 3 commits 2026-07-18 22:18:45 +00:00

feat(contracts): add infrastructure-maintenance contract ... 1d027f71f6

New responsibility contract consolidating host-level system maintenance
and Docker image lifecycle management, filling the gap left by
infrastructure-update (which owns cluster-wide apt waves).

Scope:
- OS package updates on primary host with pre-update snapshot/backup check
- Docker image pulls for LiteLLM, SearXNG, and other running containers
- Container restarts with per-stack health verification
- Post-update verification: LiteLLM, SearXNG, Zulip, Gitea, PM2, Hermes gateways
- Rollback on failure (image/apt/config restore) with circuit breaker

Owner: ops (firstmate secondmate). Trigger: weekly Sunday 2am ET.
Escalation: warning/critical->abiba+mumuni, fatal->abiba+mumuni+kwame.
circuit_breaker: max_retries 2, window 7200, trip_action escalate_to_fatal.
depends_on: infrastructure-monitoring (pre-update health baseline).

Registry:
- Add infrastructure-maintenance to by_category.maintenance, by_domain.infrastructure,
  by_owner.ops (new), by_trigger.scheduled, by_sensitivity.high
- Add 'ops' to owners list
- Move infrastructure-update owner abiba -> ops (in contracts entry + by_owner index)

Also adds '## Maintaining this file' section to AGENTS.md per fm-ensure-agents-md.

no-mistakes(review): fix docker rollback command to pin digest then compose up a68879e904

no-mistakes(document): Reframed infra-maintenance contract as deliberate partition 99789a00a1

jerome added 1 commit 2026-07-18 23:12:22 +00:00

Merge branch 'master' into fm/infra-maint-contract 5d2ecbace6

jerome merged commit 51da19d92d into master 2026-07-19 01:32:30 +00:00

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

No items

No labels

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

abiba-bot jerome (Jerome Kwame Tabiri) kagentz-bot kwame (Kwame Tabiri) mumuni-bot (Mumuni 🦅 (Syslog Falcon)) tanko-bot theo

No Assignees

2 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: SyslogSolution/prose-contracts#24