root
0c976ddbf7
- Rate limiting during reconnection (429): honor retry-after headers - Placeholder edit fails silently: fallback to sending as new message - Both fixes deployed to pi Zulip extension
85 lines
4.5 KiB
Markdown
85 lines
4.5 KiB
Markdown
---
|
|
kind: pattern
|
|
name: zulip-adapter-lessons
|
|
description: >
|
|
Lessons learned from building and debugging the pi Zulip extension and
|
|
Hermes Zulip plugin. Documents failure modes, fixes, and patterns that
|
|
apply across all Zulip adapters.
|
|
---
|
|
|
|
## Known Failure Modes & Fixes
|
|
|
|
### 1. Queue Registration Content-Type
|
|
- **Symptom**: Event queue registered but delivers no stream events
|
|
- **Root cause**: `multipart/form-data` (from `isomorphic-form-data` / `form-data` npm pkgs) vs `application/x-www-form-urlencoded`
|
|
- **Fix**: Always use URLSearchParams / form-urlencoded for `/api/v1/register`
|
|
- **Applies to**: pi extension (fixed), Hermes adapter (httpx sends form-urlencoded by default ✅)
|
|
|
|
### 2. Missing bot_user_id for @mention Detection
|
|
- **Symptom**: Bot receives stream messages but doesn't respond to @mentions
|
|
- **Root cause**: `/api/v1/register` doesn't return `user_id`; `mentioned_users` check fails when bot ID is null
|
|
- **Fix**: Call `GET /api/v1/users/me` after registration to get `user_id`
|
|
- **Applies to**: pi extension (fixed), Hermes adapter (fixed via `_resolve_bot_user_id()` ✅)
|
|
|
|
### 3. Zero Stream Subscriptions
|
|
- **Symptom**: Bot can send to streams but never receives stream events
|
|
- **Root cause**: Bot user created with 0 stream subscriptions — only DMs arrive
|
|
- **Fix**: `POST /api/v1/users/me/subscriptions` with required stream names
|
|
- **Applies to**: pi extension (fixed), Hermes adapter (⚠️ needs check)
|
|
|
|
### 4. Stale Errors Never Cleared
|
|
- **Symptom**: Health monitor shows persistent error long after recovery
|
|
- **Root cause**: `last_error` set on failure but never cleared on successful poll
|
|
- **Fix**: Clear `lastError` on every successful poll (not just on reconnect)
|
|
- **Applies to**: pi extension (fixed), Hermes adapter (⚠️ needs check)
|
|
|
|
### 5. Stuck Detection Too Aggressive
|
|
- **Symptom**: Monitor restarts bot during quiet periods (nights/weekends)
|
|
- **Root cause**: 30-min stuck threshold doesn't account for low traffic
|
|
- **Fix**: Raise threshold to 4 hours — or remove stuck detection entirely
|
|
- **Applies to**: pi extension (fixed to 4h), Hermes gateway (uses own idle detection)
|
|
|
|
### 6. Env Var Name Mismatch (ZULIP_URL vs ZULIP_SITE)
|
|
- **Symptom**: Plugin not loading because check_fn returns False
|
|
- **Root cause**: Some deploy configs use `ZULIP_URL`, adapter code expects `ZULIP_SITE`
|
|
- **Fix**: Support both names in check_fn, validate_config, and env_enablement_fn
|
|
- **Applies to**: Hermes adapter (fixed ✅)
|
|
|
|
### 7. Poll Interval Too Aggressive
|
|
- **Symptom**: Queue expires faster than expected, reconnection cycling
|
|
- **Root cause**: 3s poll interval with `dont_block=true` creates many requests
|
|
- **Fix**: Use long-poll (no `dont_block`) with 30s+ server timeout
|
|
- **Applies to**: pi extension (uses long-poll ✅), Hermes adapter (uses long-poll ✅)
|
|
|
|
### 8. A2A Server Address Already In Use
|
|
- **Symptom**: A2A server fails to start with "[Errno 98] address already in use"
|
|
- **Root cause**: Previous process holds port after container restart
|
|
- **Fix**: Kill old process before starting, or use docker restart to clear state
|
|
- **Applies to**: kagentz Agent Zero deployment
|
|
|
|
### 9. Zulip API Rate Limiting During Reconnection
|
|
- **Symptom**: Extension gets 429 errors and can't reconnect — responses vanish
|
|
- **Root cause**: Rapid PM2 restarts cause multiple queue registrations, hitting Zulip's per-IP rate limit
|
|
- **Fix**: Read `retry-after` header and wait before retrying; increase backoff between retries
|
|
- **Applies to**: pi extension (fixed), Hermes adapter (has retry logic but should check retry-after)
|
|
|
|
### 10. Placeholder Edit Fails Silently (Streaming Vanishes)
|
|
- **Symptom**: User sees "Thinking..." placeholder but never gets the actual response
|
|
- **Root cause**: `editMessage` API call fails (e.g., message too old, rate limited) but no fallback
|
|
- **Fix**: If edit fails, send the response as a new message instead
|
|
- **Applies to**: pi extension (fixed), Hermes adapter (verify edit fallback exists)
|
|
|
|
## Deployment Checklist
|
|
|
|
When deploying a new Zulip adapter, verify:
|
|
|
|
- [ ] Queue registered with form-urlencoded content type
|
|
- [ ] Bot user_id fetched from `/api/v1/users/me`
|
|
- [ ] Subscribed to all expected streams (check with `/api/v1/users/me/subscriptions`)
|
|
- [ ] `last_error` cleared on successful poll
|
|
- [ ] Env vars support both `ZULIP_URL` and `ZULIP_SITE`
|
|
- [ ] Stream @mentions detected via `mentioned_users` array
|
|
- [ ] `@all-bots` detected via configurable user_id
|
|
- [ ] Poll uses long-poll (not `dont_block=true` polling)
|
|
- [ ] Stuck/idle detection accounts for quiet periods
|