root
253e19f680
- DELETE infrastructure-monitoring.prose.md (redundant — proxmox-monitor already deployed this) - FIX infrastructure-control.prose.md: remove /grafana/ nginx route (reverted Jul 2, broke gpu-fleet) - FIX infrastructure-update.prose.md: wrong CT IDs (122→112 Tanko, 123→114 Mumuni, .19→storepve Zulip) - FIX infrastructure-update.prose.md: Strix Halo :8080→router health check (firewalled to .116 only) - FIX proxmox-monitor.prose.md: stale /grafana/ URLs→:3001 (post-revert cleanup) - ADD disk-gc-threat-response.prose.md: verified run (2026-07-04, reclaimed 35.67GB from kagentz) - ADD infrastructure-update.prose.md, pi-approval-architecture.prose.md, zulip-approval-fix.prose.md, zulip-self-heal.prose.md - UPDATE hermes-config, litellm-health/self-heal, pm2-self-heal, zulip-* contracts - ADD runs/20260704-005804-51f9a9 (disk-gc-threat-response run artifacts) All contracts verified against live PVE API, nginx config, and firewall state.
3.1 KiB
3.1 KiB
kind, name, status, description
| kind | name | status | description |
|---|---|---|---|
| responsibility | zulip-mention-reliability | retired | RETIRED 2026-07-04 — Zulip extension decommissioned. Abiba now operates exclusively through Telegram (@AbibaBot). This contract is kept for historical reference only. |
⚠️ RETIRED — The pi Zulip extension (
~/.pi/agent/extensions/zulip/) and PM2 process (abiba-zulip) have been decommissioned. All mention/reliability monitoring now happens through Telegram. Hermes agents (Tanko, Mumuni) and Agent Zero (kagentz) continue to use Zulip.
Maintains
-
mention_response_rate: number — % of @mentions that get a response (target: >95%)
-
last_test_result: "pass" | "fail" | "degraded"
-
known_failure_modes: array — History of what went wrong and how it was fixed
-
queue_health: "healthy" | "expired" | "reconnecting"
Postconditions
- test_mention_gets_response == true — Send @Abiba Bot test, confirm response within 30s
- stream_subscription_active == true — Bot is subscribed to #agent-hub and #general chat
- queue_active == true — PM2's event queue is registered and polling
- no_echo_loop == true — Bot doesn't respond to its own messages
- dm_always_works == true — DMs still respond even if streams are broken
Known Failure Modes (from History)
| Failure | Root Cause | Fix |
|---|---|---|
| No stream events | Bot had 0 stream subscriptions | Subscribe via API |
| No @mention detection | botUserId was null (register endpoint doesn't return it) | Fetch from /api/v1/users/me |
| No stream events | zulip-js library sends multipart/form-data instead of form-urlencoded | Use URLSearchParams directly |
| Double responses | Both TUI and PM2 loaded the extension | ZULIP_EXTENSION_ACTIVE guard |
Remediation Rules
Rule 1: Queue Expired
- Detect: Poll returns BAD_EVENT_QUEUE_ID
- Fix:
pm2 restart abiba-zulip - Verify: Check logs for "Connected, queue:"
Rule 2: Subscription Missing
- Detect:
GET /api/v1/users/me/subscriptionsreturns 0 for #agent-hub - Fix: Subscribe via
POST /api/v1/users/me/subscriptions - Verify: Re-check subscriptions
Rule 3: Extension Crashed
- Detect: PM2 status shows "errored" for abiba-zulip
- Fix:
pm2 restart abiba-zulip - Escalate: If restarts >5 in last hour, alert user
Rule 4: No Response to Test Mention
- Detect: Send test @mention, wait 30s, check if response appeared
- Fix: Run diagnostics (check queue, check code, check logs)
- Log: Record failure mode as new entry in Known Failure Modes
Continuity
- On user report: User says "you didn't respond" → run diagnostic
- On test failure: Test mention fails → run remediation
- Retrospective: Every 6 hours — send a test @mention to self-verify
- On PM2 restart: Always run a self-test after restart
Execution
- Self-test: Send
@**Abiba Bot** _selftest_in #agent-hub, wait for response - If no response in 30s: Run diagnostics
- Diagnose: Check queue, subscriptions, PM2 status, logs
- Fix: Apply matching remediation rule
- Retest: Try again
- Log: Record success or new failure mode
- Report: DM user only if escalation needed