SyslogSolution/prose-contracts
7
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity

pm2-self-heal: restore abiba-zulip, retire gpu-watchdog, document systemd for gpu-monitor

Browse Source
This commit is contained in:
root
2026-08-03 22:35:55 +00:00
parent 8f719ca7c7
commit f7218e04c0
1 changed files with 6 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
pm2-self-heal.prose.md
+6 -6
View File
@@ -5,18 +5,18 @@ description: >
Monitors critical PM2 processes (abiba-zulip, abiba-telegram) and
auto-restarts any that are stopped or errored. Logs every action to
the knowledge graph and alerts the owner via Zulip DM on failures.
CRITICAL: Never restart abiba-zulip — it runs this contract.
Abiba-zulip is the live Zulip bridge and may be restarted; alert owner on failure.
---
## Maintains
- abiba-telegram: { status: "online", uptime: string, restarts: number }
- gpu-watchdog: { status: "online", uptime: string, restarts: number }
- gpu-monitor: { status: "online", uptime: string, restarts: number }
- abiba-zulip: { status: "online", uptime: string, restarts: number }
- gpu-monitor: { status: "online", uptime: string, restarts: number } (systemd-managed, PM2-tracked)
- gitea-runner: { status: "online", uptime: string, restarts: number }
- last_check: timestamp
> **Note (2026-07-04):** `abiba-zulip` removed — Zulip extension decommissioned.
> **Status (2026-08-03):** `abiba-zulip` fully restored — live Zulip bridge, heartbeating, monitored. `gpu-monitor` runs via systemd (stable); PM2 tracks it for status reporting only. `gpu-watchdog` retired from PM2.
## Continuity
@@ -48,9 +48,9 @@ description: >
- If status is "online" → pass
- If status is "stopped" or "errored" → apply Rule 1
- If restarts > 5 → alert owner
3. **Check abiba-zulip** (self-process, read-only):
3. **Check abiba-zulip** (live Zulip bridge, heartbeating):
- If status is "online" → pass, log restarts count
- If status is "stopped" or "errored" → **DO NOT RESTART** — alert owner immediately
- If status is "stopped" or "errored" → restart (`pm2 restart abiba-zulip` — fully restored)
- If restarts > 5 in last hour → alert owner with full diagnostics
4. **Log results** — Append to `SyslogSolution/health-logs/pm2/{timestamp}.md` in Gitea (not knowledge graph — hard rule)
5. **Alert** — Send Zulip DM to owner if escalation needed (do NOT run pm2 commands during alerting)