root
2831b06c1e
Monitors whether Abiba Bot responds to @mentions in stream topics. Self-test: sends a test mention, checks for response within 30s. Documents 4 known failure modes with remediation rules. Follows same self-improving pattern as litellm-self-heal.
73 lines
2.8 KiB
Markdown
73 lines
2.8 KiB
Markdown
---
|
|
kind: responsibility
|
|
name: zulip-mention-reliability
|
|
description: >
|
|
Monitors whether Abiba Bot is correctly detecting and responding to
|
|
@mentions in Zulip stream topics. Tests periodically, diagnoses
|
|
failures, and attempts fixes. Same self-improving pattern as
|
|
litellm-self-heal.
|
|
---
|
|
|
|
## Maintains
|
|
|
|
- mention_response_rate: number — % of @mentions that get a response (target: >95%)
|
|
- last_test_result: "pass" | "fail" | "degraded"
|
|
- known_failure_modes: array — History of what went wrong and how it was fixed
|
|
- queue_health: "healthy" | "expired" | "reconnecting"
|
|
|
|
## Success Criteria
|
|
|
|
- test_mention_gets_response == true — Send @Abiba Bot test, confirm response within 30s
|
|
- stream_subscription_active == true — Bot is subscribed to #agent-hub and #general chat
|
|
- queue_active == true — PM2's event queue is registered and polling
|
|
- no_echo_loop == true — Bot doesn't respond to its own messages
|
|
- dm_always_works == true — DMs still respond even if streams are broken
|
|
|
|
## Known Failure Modes (from History)
|
|
|
|
| Failure | Root Cause | Fix |
|
|
|---|---|---|
|
|
| No stream events | Bot had 0 stream subscriptions | Subscribe via API |
|
|
| No @mention detection | botUserId was null (register endpoint doesn't return it) | Fetch from /api/v1/users/me |
|
|
| No stream events | zulip-js library sends multipart/form-data instead of form-urlencoded | Use URLSearchParams directly |
|
|
| Double responses | Both TUI and PM2 loaded the extension | ZULIP_EXTENSION_ACTIVE guard |
|
|
|
|
## Remediation Rules
|
|
|
|
### Rule 1: Queue Expired
|
|
- Detect: Poll returns BAD_EVENT_QUEUE_ID
|
|
- Fix: `pm2 restart abiba-zulip`
|
|
- Verify: Check logs for "Connected, queue:"
|
|
|
|
### Rule 2: Subscription Missing
|
|
- Detect: `GET /api/v1/users/me/subscriptions` returns 0 for #agent-hub
|
|
- Fix: Subscribe via `POST /api/v1/users/me/subscriptions`
|
|
- Verify: Re-check subscriptions
|
|
|
|
### Rule 3: Extension Crashed
|
|
- Detect: PM2 status shows "errored" for abiba-zulip
|
|
- Fix: `pm2 restart abiba-zulip`
|
|
- Escalate: If restarts >5 in last hour, alert user
|
|
|
|
### Rule 4: No Response to Test Mention
|
|
- Detect: Send test @mention, wait 30s, check if response appeared
|
|
- Fix: Run diagnostics (check queue, check code, check logs)
|
|
- Log: Record failure mode as new entry in Known Failure Modes
|
|
|
|
## Continuity
|
|
|
|
- **On user report**: User says "you didn't respond" → run diagnostic
|
|
- **On test failure**: Test mention fails → run remediation
|
|
- **Retrospective**: Every 6 hours — send a test @mention to self-verify
|
|
- **On PM2 restart**: Always run a self-test after restart
|
|
|
|
## Execution
|
|
|
|
1. **Self-test**: Send `@**Abiba Bot** _selftest_` in #agent-hub, wait for response
|
|
2. **If no response in 30s**: Run diagnostics
|
|
3. **Diagnose**: Check queue, subscriptions, PM2 status, logs
|
|
4. **Fix**: Apply matching remediation rule
|
|
5. **Retest**: Try again
|
|
6. **Log**: Record success or new failure mode
|
|
7. **Report**: DM user only if escalation needed
|