SyslogSolution/prose-contracts
7
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
169 Commits 42 Branches 0 Tags
35d09ba69a238fe408c42953631f8128d943e223
Commit Graph
5 Commits
Author SHA1 Message Date
root 546ca79849 fix: cron alignment + VM→bare metal drift
PR Pipeline — Authorize → Validate → Review → Merge / auth (pull_request) Successful in 3s
Details
PR Pipeline — Authorize → Validate → Review → Merge / validate (pull_request) Successful in 1s
Details
PR Pipeline — Authorize → Validate → Review → Merge / lint (pull_request) Successful in 3s
Details
PR Pipeline — Authorize → Validate → Review → Merge / ai-review (pull_request) Successful in 1s
Details
PR Pipeline — Authorize → Validate → Review → Merge / gate (pull_request) Successful in 2s
Details 
- pm2-self-heal: cron 30min/daytime → 5min/24×7 (matches contract)
- pm2-self-heal: added gpu-monitor to tracked PM2 processes
- infra-monitor+control: removed daytime-only restriction (now 24×7)
- infrastructure-update: pre-flight VM 101/103 → bare metal .8/.110
- Added @reboot pm2 resurrect for persistence across reboots
- README: zulip-monitor.sh → agent-health-check.py
- Cleaned up commented-out zulip-monitor cron line

Live actions: gpu-monitor-server added to PM2 (persistent across reboots)
 2026-07-09 06:27:48 +00:00
root 253e19f680 prose: review, fix, and consolidate infrastructure contracts 
- DELETE infrastructure-monitoring.prose.md (redundant — proxmox-monitor already deployed this)
- FIX infrastructure-control.prose.md: remove /grafana/ nginx route (reverted Jul 2, broke gpu-fleet)
- FIX infrastructure-update.prose.md: wrong CT IDs (122→112 Tanko, 123→114 Mumuni, .19→storepve Zulip)
- FIX infrastructure-update.prose.md: Strix Halo :8080→router health check (firewalled to .116 only)
- FIX proxmox-monitor.prose.md: stale /grafana/ URLs→:3001 (post-revert cleanup)
- ADD disk-gc-threat-response.prose.md: verified run (2026-07-04, reclaimed 35.67GB from kagentz)
- ADD infrastructure-update.prose.md, pi-approval-architecture.prose.md, zulip-approval-fix.prose.md, zulip-self-heal.prose.md
- UPDATE hermes-config, litellm-health/self-heal, pm2-self-heal, zulip-* contracts
- ADD runs/20260704-005804-51f9a9 (disk-gc-threat-response run artifacts)

All contracts verified against live PVE API, nginx config, and firewall state.
 2026-07-04 21:47:33 +00:00
root 1f90c3dc74 fix(pm2-self-heal): document permanent stuck-detection fix, reset PM2 counter 
Root cause: abiba-zulip extension had STUCK_THRESHOLD_MS=30min.
After 30min of inactivity (overnight), it reported stuck=True, which
triggered the health monitor to restart it — cycling restarts up to 18.

Permanent fixes:
1. STUCK_THRESHOLD_MS raised 30min → 4 hours in extension index.js
2. PM2 restart counter reset: delete+re-add abiba-zulip (was 18, now 0)
3. pm2-self-heal.sh alert threshold documented at 30 (implementation already)
4. Prose contract updated with historical note and threshold rationale
 2026-06-29 16:07:47 +00:00
root 4c6b0410ea fix: pm2-self-heal — self-process guard, never restart abiba-zulip 
- Added --no-color flag to all PM2 commands
- Self-process guard: abiba-zulip is READ-ONLY, never restarted
- Corrected column indices (PID=6, restarts=8, status=9)
- Added pm2-self-heal.sh shell script implementation
 2026-06-26 19:53:55 +00:00
root f340c6301d feat: pm2-self-heal — monitors and restarts critical PM2 processes 
Responsibility contract that checks abiba-zulip and abiba-telegram
every 5 minutes. Restarts stopped/errored processes, escalates if
restart count exceeds 5/hour. Logs to knowledge graph and alerts
owner via Zulip DM.
 2026-06-26 17:44:27 +00:00