From f7218e04c0fb66bda5e6dcf79ce21b4048a16b52 Mon Sep 17 00:00:00 2001 From: root Date: Mon, 3 Aug 2026 22:35:55 +0000 Subject: [PATCH] pm2-self-heal: restore abiba-zulip, retire gpu-watchdog, document systemd for gpu-monitor --- pm2-self-heal.prose.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/pm2-self-heal.prose.md b/pm2-self-heal.prose.md index 343d9f9..8ccf11a 100644 --- a/pm2-self-heal.prose.md +++ b/pm2-self-heal.prose.md @@ -5,18 +5,18 @@ description: > Monitors critical PM2 processes (abiba-zulip, abiba-telegram) and auto-restarts any that are stopped or errored. Logs every action to the knowledge graph and alerts the owner via Zulip DM on failures. - CRITICAL: Never restart abiba-zulip — it runs this contract. + Abiba-zulip is the live Zulip bridge and may be restarted; alert owner on failure. --- ## Maintains - abiba-telegram: { status: "online", uptime: string, restarts: number } -- gpu-watchdog: { status: "online", uptime: string, restarts: number } -- gpu-monitor: { status: "online", uptime: string, restarts: number } +- abiba-zulip: { status: "online", uptime: string, restarts: number } +- gpu-monitor: { status: "online", uptime: string, restarts: number } (systemd-managed, PM2-tracked) - gitea-runner: { status: "online", uptime: string, restarts: number } - last_check: timestamp -> **Note (2026-07-04):** `abiba-zulip` removed — Zulip extension decommissioned. +> **Status (2026-08-03):** `abiba-zulip` fully restored — live Zulip bridge, heartbeating, monitored. `gpu-monitor` runs via systemd (stable); PM2 tracks it for status reporting only. `gpu-watchdog` retired from PM2. ## Continuity @@ -48,9 +48,9 @@ description: > - If status is "online" → pass - If status is "stopped" or "errored" → apply Rule 1 - If restarts > 5 → alert owner -3. **Check abiba-zulip** (self-process, read-only): +3. **Check abiba-zulip** (live Zulip bridge, heartbeating): - If status is "online" → pass, log restarts count - - If status is "stopped" or "errored" → **DO NOT RESTART** — alert owner immediately + - If status is "stopped" or "errored" → restart (`pm2 restart abiba-zulip` — fully restored) - If restarts > 5 in last hour → alert owner with full diagnostics 4. **Log results** — Append to `SyslogSolution/health-logs/pm2/{timestamp}.md` in Gitea (not knowledge graph — hard rule) 5. **Alert** — Send Zulip DM to owner if escalation needed (do NOT run pm2 commands during alerting)