Root cause: Workers hung because default model was DeepSeek v4-pro (reasoning model that produces empty content with normal token limits). pi RPC workers waited forever for content that never arrived. Changes: - Circuit breaker (CLOSED→OPEN→HALF_OPEN) around Zulip API calls - Retry with exponential backoff + jitter for transient errors - Queue lifecycle management (idle_queue_timeout, auto re-register on BAD_EVENT_QUEUE_ID) - External supervisor watchdog (restarts router after 3 health check failures) - Busy worker timeout (SIGKILL after 5 min + error DM) - PM2 hardening (max_restarts=100, max_memory_restart=500M) - Crash handlers (uncaughtException + unhandledRejection → reconnect, not die) - Fixed default model: deepseek-v4-pro → syslog-harness/syslog-auto - Enhanced health endpoint with circuit breaker stats Architecture: Research-backed from Zulip event system docs + Node.js resilience patterns. Inline circuit breaker (no dependency). Separate watchdog process (Hermes pattern). Verified: Circuit breaker trips on outage, recovers gracefully. End-to-end DM processed in 4 seconds. Watchdog monitoring every 30s.
Abiba Zulip Gateway — Standalone Service
Replaces the old pi extension (~/.pi/agent/extensions/zulip/).
Instead of injecting messages into pi's session (which dies when pi exits), this runs as an independent Node.js systemd service that:
- Polls Zulip event queue for DMs
- Calls the harness inference API directly (no pi dependency)
- Maintains per-sender conversation memory
- Survives pi shutdown and restart
Architecture
Zulip event queue → poll every 3s → DM detected
→ send typing indicator + placeholder
→ POST /v1/chat/completions with conversation history
→ edit placeholder with final response
Requirements
- Node.js 22+
npm installin this directory
Config
All config via environment variables. Copy abiba-zulip.service to set up as a
systemd service, or run directly:
ZULIP_EMAIL=abiba-bot@chat.sysloggh.net \
ZULIP_API_KEY=your_key \
ZULIP_SITE=https://chat.sysloggh.net \
HARNESS_URL=http://192.168.68.116/v1/chat/completions \
HARNESS_API_KEY=sk-xxx \
HARNESS_MODEL=qwen3.6-35B-A3B \
node dist/index.js
Deploy as a service
# 1. Install deps
cd /opt/abiba-zulip
npm install
# 2. Build
npx tsc
# 3. Install systemd service
cp abiba-zulip.service /etc/systemd/system/
systemctl daemon-reload
systemctl enable abiba-zulip
systemctl start abiba-zulip
# 4. Check status
systemctl status abiba-zulip
journalctl -u abiba-zulip -f
# 5. Health check
curl http://127.0.0.1:9200/health
Development
npm run dev # watch mode (tsc watch)
npm run build # compile
npm start # run compiled