root
253e19f680
- DELETE infrastructure-monitoring.prose.md (redundant — proxmox-monitor already deployed this) - FIX infrastructure-control.prose.md: remove /grafana/ nginx route (reverted Jul 2, broke gpu-fleet) - FIX infrastructure-update.prose.md: wrong CT IDs (122→112 Tanko, 123→114 Mumuni, .19→storepve Zulip) - FIX infrastructure-update.prose.md: Strix Halo :8080→router health check (firewalled to .116 only) - FIX proxmox-monitor.prose.md: stale /grafana/ URLs→:3001 (post-revert cleanup) - ADD disk-gc-threat-response.prose.md: verified run (2026-07-04, reclaimed 35.67GB from kagentz) - ADD infrastructure-update.prose.md, pi-approval-architecture.prose.md, zulip-approval-fix.prose.md, zulip-self-heal.prose.md - UPDATE hermes-config, litellm-health/self-heal, pm2-self-heal, zulip-* contracts - ADD runs/20260704-005804-51f9a9 (disk-gc-threat-response run artifacts) All contracts verified against live PVE API, nginx config, and firewall state.
77 lines
3.1 KiB
Markdown
77 lines
3.1 KiB
Markdown
---
|
|
kind: responsibility
|
|
name: zulip-mention-reliability
|
|
status: retired
|
|
description: >
|
|
RETIRED 2026-07-04 — Zulip extension decommissioned. Abiba now operates
|
|
exclusively through Telegram (@AbibaBot). This contract is kept for
|
|
historical reference only.
|
|
---
|
|
|
|
> **⚠️ RETIRED** — The pi Zulip extension (`~/.pi/agent/extensions/zulip/`) and
|
|
> PM2 process (`abiba-zulip`) have been decommissioned. All mention/reliability
|
|
> monitoring now happens through Telegram. Hermes agents (Tanko, Mumuni) and
|
|
> Agent Zero (kagentz) continue to use Zulip.
|
|
|
|
## Maintains
|
|
|
|
- mention_response_rate: number — % of @mentions that get a response (target: >95%)
|
|
- last_test_result: "pass" | "fail" | "degraded"
|
|
- known_failure_modes: array — History of what went wrong and how it was fixed
|
|
- queue_health: "healthy" | "expired" | "reconnecting"
|
|
|
|
**Postconditions**
|
|
- test_mention_gets_response == true — Send @Abiba Bot test, confirm response within 30s
|
|
- stream_subscription_active == true — Bot is subscribed to #agent-hub and #general chat
|
|
- queue_active == true — PM2's event queue is registered and polling
|
|
- no_echo_loop == true — Bot doesn't respond to its own messages
|
|
- dm_always_works == true — DMs still respond even if streams are broken
|
|
|
|
## Known Failure Modes (from History)
|
|
|
|
| Failure | Root Cause | Fix |
|
|
|---|---|---|
|
|
| No stream events | Bot had 0 stream subscriptions | Subscribe via API |
|
|
| No @mention detection | botUserId was null (register endpoint doesn't return it) | Fetch from /api/v1/users/me |
|
|
| No stream events | zulip-js library sends multipart/form-data instead of form-urlencoded | Use URLSearchParams directly |
|
|
| Double responses | Both TUI and PM2 loaded the extension | ZULIP_EXTENSION_ACTIVE guard |
|
|
|
|
## Remediation Rules
|
|
|
|
### Rule 1: Queue Expired
|
|
- Detect: Poll returns BAD_EVENT_QUEUE_ID
|
|
- Fix: `pm2 restart abiba-zulip`
|
|
- Verify: Check logs for "Connected, queue:"
|
|
|
|
### Rule 2: Subscription Missing
|
|
- Detect: `GET /api/v1/users/me/subscriptions` returns 0 for #agent-hub
|
|
- Fix: Subscribe via `POST /api/v1/users/me/subscriptions`
|
|
- Verify: Re-check subscriptions
|
|
|
|
### Rule 3: Extension Crashed
|
|
- Detect: PM2 status shows "errored" for abiba-zulip
|
|
- Fix: `pm2 restart abiba-zulip`
|
|
- Escalate: If restarts >5 in last hour, alert user
|
|
|
|
### Rule 4: No Response to Test Mention
|
|
- Detect: Send test @mention, wait 30s, check if response appeared
|
|
- Fix: Run diagnostics (check queue, check code, check logs)
|
|
- Log: Record failure mode as new entry in Known Failure Modes
|
|
|
|
## Continuity
|
|
|
|
- **On user report**: User says "you didn't respond" → run diagnostic
|
|
- **On test failure**: Test mention fails → run remediation
|
|
- **Retrospective**: Every 6 hours — send a test @mention to self-verify
|
|
- **On PM2 restart**: Always run a self-test after restart
|
|
|
|
## Execution
|
|
|
|
1. **Self-test**: Send `@**Abiba Bot** _selftest_` in #agent-hub, wait for response
|
|
2. **If no response in 30s**: Run diagnostics
|
|
3. **Diagnose**: Check queue, subscriptions, PM2 status, logs
|
|
4. **Fix**: Apply matching remediation rule
|
|
5. **Retest**: Try again
|
|
6. **Log**: Record success or new failure mode
|
|
7. **Report**: DM user only if escalation needed
|