update(infrastructure-update): v1.3.0 — full Docker ecosystem coverage from 2026-09-08 live run

- Add 5 Docker ecosystems: docker-vm .7, CT 116 .116, CT 117 (zulip+jitsi),
  hwpve .11 (Authentik), NetBird VPS 72.61.0.17
- Wave 3: add trove-test, docker-stats, monitoring stack, Zulip (pct exec + compose
  recreate preserves zulip_default network), Jitsi, Authentik, NetBird
- Firecrawl test is now POST /v1/search (GET / returns 404 by design)
- Add Authentik 302 + NetBird dashboard 200 checks
- Document harness-litellm 3-5 min cold start after recreate (verified 2026-09-08)
- Add hwpve + VPS compose files to config backup list; success criteria covers 5 hosts

infrastructure-update.prose.md

@@ -3,7 +3,8 @@ kind: responsibility
 name: infrastructure-update
 description: >
   Autonomous system-wide update contract covering all 5 Proxmox nodes,
-  15+ containers/VMs, and 4 Docker ecosystems. Updates apt packages,
+  15+ containers/VMs, and 5 Docker ecosystems (docker-vm .7, CT 116 .116,
+  CT 117, hwpve .11, NetBird VPS 72.61.0.17). Updates apt packages,
   Docker images, and container stacks in safe waves with health checks
   and automatic rollback on failure.
 agent: abiba
@@ -11,7 +12,7 @@ triggers:
   - on "infra update" command
   - weekly (Sunday 03:00 EDT) via cron
   - on security advisory relay from Mumuni
-version: 1.2.0
+version: 1.3.0
 ---
 
 ## Maintains
@@ -80,7 +81,13 @@ Before ANY update wave:
 | VM 109 (.7) | Home stack (Pulse, Stirling PDF) — JDownloader moved to CT 118 LXC 2026-08-01 | `cd /opt/home_stack && docker compose pull && docker compose up -d` |
 | VM 109 (.7) | Audiobookshelf | `cd /opt/audiobookshelf && docker compose pull && docker compose up -d` |
 | CT 116 (.116) | Inference Harness (LiteLLM, Prometheus, Grafana) | `cd /opt/inference-harness && docker compose pull && docker compose up -d` |
-| CT 117 (zulip, storepve) | Zulip | `docker pull zulip/docker-zulip:latest && docker restart zulip-zulip-1` |
+| CT 117 (storepve) | Zulip | `pct exec 117 -- bash -c 'cd /opt/zulip && docker compose pull && docker compose up -d'` (from storepve; compose recreates on zulip_default network) |
+| CT 117 (storepve) | Jitsi | `pct exec 117 -- bash -c 'cd /opt/jitsi && docker compose pull && docker compose up -d'` (from storepve) |
+| hwpve (.11) | Authentik (server, worker, postgres) | `ssh root@192.168.68.11 'cd /root && docker compose pull && docker compose up -d'` |
+| NetBird VPS (72.61.0.17) | NetBird (server, dashboard, proxy, traefik, crowdsec) | `ssh root@72.61.0.17 'cd /root && docker compose pull && docker compose up -d'` |
+| VM 109 (.7) | Trove test | `cd /opt/trove-test && docker compose pull && docker compose up -d` |
+| VM 109 (.7) | docker-stats | `cd /opt/docker-stats && docker compose pull && docker compose up -d` |
+| CT 116 (.116) | Monitoring (Grafana, Prometheus, Alertmanager, PVE exporter) | `cd /opt/monitoring && docker compose pull && docker compose up -d` |
 
 **Verify after Wave 3:**
 - All containers healthy: `docker ps` on each host
@@ -88,7 +95,10 @@ Before ANY update wave:
 - MCP integration test: `curl localhost:4000/mcp-rest/tools/list -H "Authorization: Bearer $MASTER_KEY"` → 90 tools (23 RA-H OS + 67 GitHub)
 - Zulip test: send test message to #agent-hub
 - Dashboard loading: `curl localhost:3001/` (via CT 116)
-- Firecrawl test: `curl :3002/`
+- Firecrawl test: `curl -X POST http://192.168.68.7:3002/v1/search -H 'Content-Type: application/json' -d '{"query":"health","limit":1}'` → `"success":true` (GET `/` returns 404 by design)
+- Authentik test: `curl http://192.168.68.11:9000/` → 302 redirect to login
+- NetBird test: `curl -s -o /dev/null -w '%{http_code}' https://netbird.sysloggh.net/` → 200
+- harness-litellm cold start: allow 3-5 min after recreate — reports unhealthy and :4000 refuses connections while loading config/DB, then recovers to 200 on its own (verified 2026-09-08)
 - SearXNG test: `curl :8888`
 
 ## Wave 4: Proxmox Kernel Reboot
@@ -158,6 +168,8 @@ Before Wave 1, snapshot these files:
 /opt/search-stack/searxng/docker-compose.yml            (VM 109 .7)
 /opt/home_stack/docker-compose.yml                      (VM 109 .7)
 /opt/audiobookshelf/docker-compose.yml                  (VM 109 .7)
+/root/compose.yml                                       (hwpve .11 — Authentik server/worker/postgres)
+/root/docker-compose.yml                                (NetBird VPS — netbird server/dashboard/proxy, traefik, crowdsec)
 /root/.pi/agent/extensions/config.yaml                  (CT 100 .24)
 /etc/systemd/system/strix-server.service                (amdpve .15 — strix-moe)
 /etc/systemd/system/llama-server.service                (VM 101 .8, VM 103 .110)
@@ -220,7 +232,7 @@ When LiteLLM is upgraded to a version supporting per-key MCP grants:
 
 - [ ] All 5 PVE nodes updated, no reboot-loop
 - [ ] All VMs/CTs running post-update
-- [ ] All Docker containers healthy (VM 109 + CT 116 + CT 117)
+- [ ] All Docker containers healthy (VM 109 + CT 116 + CT 117 + hwpve .11 + NetBird VPS)
 - [ ] LiteLLM inference passing (syslog-auto test)
 - [ ] Zulip server + all 3 agents connected
 - [ ] GPU fleet at full capacity (3/3)