SyslogSolution/prose-contracts
7
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

prose: review, fix, and consolidate infrastructure contracts

Browse Source 
- DELETE infrastructure-monitoring.prose.md (redundant — proxmox-monitor already deployed this)
- FIX infrastructure-control.prose.md: remove /grafana/ nginx route (reverted Jul 2, broke gpu-fleet)
- FIX infrastructure-update.prose.md: wrong CT IDs (122→112 Tanko, 123→114 Mumuni, .19→storepve Zulip)
- FIX infrastructure-update.prose.md: Strix Halo :8080→router health check (firewalled to .116 only)
- FIX proxmox-monitor.prose.md: stale /grafana/ URLs→:3001 (post-revert cleanup)
- ADD disk-gc-threat-response.prose.md: verified run (2026-07-04, reclaimed 35.67GB from kagentz)
- ADD infrastructure-update.prose.md, pi-approval-architecture.prose.md, zulip-approval-fix.prose.md, zulip-self-heal.prose.md
- UPDATE hermes-config, litellm-health/self-heal, pm2-self-heal, zulip-* contracts
- ADD runs/20260704-005804-51f9a9 (disk-gc-threat-response run artifacts)

All contracts verified against live PVE API, nginx config, and firewall state.
This commit is contained in:
root
2026-07-04 21:47:33 +00:00
parent 3077fda0b7
commit 253e19f680
14 changed files with 628 additions and 213 deletions
Download Patch File Download Diff File Expand all files Collapse all files
runs/20260704-005804-51f9a9/receipts/disk-gc-threat-response.jsonl
+1
View File
@@ -0,0 +1 @@
{"node":"disk-gc-threat-response","status":"rendered","fingerprints":{"@atomic":"fleet-18-green-0-amber-0-red"},"semantic_diff":"Initial fleet baseline scan: 18 hosts scanned, all GREEN. Previously resolved CT 105 kagentz from 87%→27%. CT 118 jitsi offline.","cost":{},"prev":null,"timestamp":"2026-07-04T00:58:30Z"}
runs/20260704-005804-51f9a9/vm.log.md
+26
View File
@@ -0,0 +1,26 @@
# run:20260704-005804-51f9a9 disk-gc-threat-response
root: disk-gc-threat-response
1→ [scan] fleet-wide disk scan started
2→ CT 100 abiba (amdpve): 12G/59G 21% GREEN
2→ CT 101 llm-gpu (acerpve): 144G/255G 60% GREEN
2→ CT 102 adguard (acerpve): 5.9G/32G 20% GREEN
2→ CT 103 ocu-llm (ocupve): 48G/186G 27% GREEN
2→ CT 104 authentik (minipve): 4.6G/9.8G 50% GREEN
2→ CT 105 kagentz (amdpve): 16G/59G 27% GREEN (resolved: was 87%, pruned 35.67G)
2→ CT 106 ra-h-os (storepve): 2.1G/20G 11% GREEN
2→ CT 107 pbs (storepve): 1.1G/40G 3% GREEN
2→ CT 108 media (storepve): 19G/196G 10% GREEN
2→ CT 109 docker-vm (storepve): 25G/158G 17% GREEN
2→ CT 110 gitea (minipve): 5.6G/40G 16% GREEN
2→ CT 111 tdunna (amdpve): 16G/63G 26% GREEN
2→ CT 112 tanko (amdpve): 17G/49G 36% GREEN
2→ CT 113 baggy (amdpve): 7.0G/52G 15% GREEN
2→ CT 114 mumuni (minipve): 29G/59G 51% GREEN
2→ CT 115 scottdenya (amdpve): 14G/59G 25% GREEN
2→ CT 116 syslog-api (minipve): 16G/40G 42% GREEN
2→ CT 117 zulip (storepve): 6.1G/60G 11% GREEN
2→ CT 118 jitsi (minipve): OFFLINE — no disk scan
3→ [summary] Fleet: 19 hosts, 18 scanned, 18 GREEN, 0 AMBER, 0 RED, 0 CRITICAL, 1 OFFLINE
3→ [summary] Previously resolved: CT 105 kagentz 87%→27% (35.67GB reclaimed)
---end 2026-07-04T00:58:04Z
runs/20260704-005804-51f9a9/world-model/disk-gc-threat-response/disk-health.md
+52
View File
@@ -0,0 +1,52 @@
# Fleet Disk Health — 2026-07-04 00:58 UTC
## Summary
| Metric | Count |
|--------|-------|
| Total hosts | 19 |
| Scanned | 18 |
| GREEN (<75%) | 18 |
| AMBER (75-84%) | 0 |
| RED (85-94%) | 0 |
| CRITICAL (≥95%) | 0 |
| OFFLINE | 1 (jitsi CT 118) |
## Per-Host Status
| CT | Name | Node | Used | Total | % | Status |
|----|------|------|------|-------|---|--------|
| 100 | abiba | amdpve | 12G | 59G | 21% | 🟢 GREEN |
| 101 | llm-gpu | acerpve | 144G | 255G | 60% | 🟢 GREEN |
| 102 | adguard | acerpve | 5.9G | 32G | 20% | 🟢 GREEN |
| 103 | ocu-llm | ocupve | 48G | 186G | 27% | 🟢 GREEN |
| 104 | authentik | minipve | 4.6G | 9.8G | 50% | 🟢 GREEN |
| 105 | kagentz | amdpve | 16G | 59G | 27% | 🟢 GREEN |
| 106 | ra-h-os | storepve | 2.1G | 20G | 11% | 🟢 GREEN |
| 107 | pbs | storepve | 1.1G | 40G | 3% | 🟢 GREEN |
| 108 | media | storepve | 19G | 196G | 10% | 🟢 GREEN |
| 109 | docker-vm | storepve | 25G | 158G | 17% | 🟢 GREEN |
| 110 | gitea | minipve | 5.6G | 40G | 16% | 🟢 GREEN |
| 111 | tdunna | amdpve | 16G | 63G | 26% | 🟢 GREEN |
| 112 | tanko | amdpve | 17G | 49G | 36% | 🟢 GREEN |
| 113 | baggy | amdpve | 7.0G | 52G | 15% | 🟢 GREEN |
| 114 | mumuni | minipve | 29G | 59G | 51% | 🟢 GREEN |
| 115 | scottdenya | amdpve | 14G | 59G | 25% | 🟢 GREEN |
| 116 | syslog-api | minipve | 16G | 40G | 42% | 🟢 GREEN |
| 117 | zulip | storepve | 6.1G | 60G | 11% | 🟢 GREEN |
| 118 | jitsi | minipve | — | — | — | ⚫ OFFLINE |
## Docker Hosts - GC Status
| Host | Docker present | Last GC | Status |
|------|---------------|---------|--------|
| kagentz (CT 105) | Yes | 2026-07-04 (just now) | 35.67GB reclaimed, 27% |
| syslog-api (CT 116) | Yes | Never | 42%, no action needed |
| docker-vm (CT 109) | Yes | Unknown | 17%, no action needed |
| abiba (CT 100) | Yes | Never | 21%, /var/lib/docker=2.8G |
## Notes
- CT 118 (jitsi) is not running — no disk scan possible. May need `qm start 118` if service is needed.
- CT 114 (mumuni) at 51% — closest to AMBER threshold (75%). Agent memory accumulation likely.
- CT 105 (kagentz) resolved from 87% RED → 27% GREEN. Root cause: docker image bloat. Prevented: GC contract in place.
- All Docker hosts are healthy. Next automated scan in 6 hours.