The truncation notice '[...truncated at Zulip limit]' was appended AFTER
slicing at MAX_ZULIP_MESSAGE (10000), causing the final message to exceed
Zulip's API limit. This fix subtracts the notice length from the slice so
the total stays within bounds.
(cherry picked from commit 19c52a9425)
Replaced manual 'git clone' with password in URL with actions/checkout@v4.
Runner already auto-checkouts the repo - manual clone was redundant.
Also fixed YAML syntax issues in Config validation and No secrets check steps.
Credentials were exposed in git history since initial commit.
(cherry picked from commit aeb79c6286)
Root cause: Workers hung because default model was DeepSeek v4-pro (reasoning
model that produces empty content with normal token limits). pi RPC workers
waited forever for content that never arrived.
Changes:
- Circuit breaker (CLOSED→OPEN→HALF_OPEN) around Zulip API calls
- Retry with exponential backoff + jitter for transient errors
- Queue lifecycle management (idle_queue_timeout, auto re-register on BAD_EVENT_QUEUE_ID)
- External supervisor watchdog (restarts router after 3 health check failures)
- Busy worker timeout (SIGKILL after 5 min + error DM)
- PM2 hardening (max_restarts=100, max_memory_restart=500M)
- Crash handlers (uncaughtException + unhandledRejection → reconnect, not die)
- Fixed default model: deepseek-v4-pro → syslog-harness/syslog-auto
- Enhanced health endpoint with circuit breaker stats
Architecture: Research-backed from Zulip event system docs + Node.js resilience
patterns. Inline circuit breaker (no dependency). Separate watchdog process (Hermes pattern).
Verified: Circuit breaker trips on outage, recovers gracefully. End-to-end DM
processed in 4 seconds. Watchdog monitoring every 30s.
- Added Host column (Abiba: 192.168.68.24, Hermes agents: 192.168.68.123)
- Changed Health Port → Health URL with full http://host:port/health URLs
- Updated Check 1 to reference {{health_url}} instead of {{health_port}}
- Added .gitignore for .agents/ (OpenProse run state)
Adds two improvements from pi Zulip extension lessons:
1. _ensure_stream_subscriptions() — checks at connect time if the bot
is subscribed to the primary stream and general-chat; auto-subscribes
if missing. Without this, bots with 0 subscriptions can't receive
stream events (DMs only).
2. selftest check #9 — verifies stream subscriptions and reports
count + names. Catches the 'zero subscriptions' failure mode that
was a major debugging bottleneck in the pi extension.
Same fix as applied to the pi Zulip extension: last_error_msg and
last_error_at are now cleared on every successful poll cycle, not
just on reconnect. Health monitors no longer show stale errors.
- Full contract verification for pi (TypeScript) and Hermes (Python) Zulip plugins
- Fixed @all-bots user_id (1→20) — now dynamically resolved from Zulip API
- Fixed last_event_id stale display in health endpoint (B2)
- Fixed display_recipient logging (B3)
- Added bot_user_id and all_bots_user_id to health endpoint
- Created OpenProse cross-platform verification contract (docs/contracts/)
- Created Hermes plugin deployment script (scripts/deploy-hermes-zulip.py)
Pi extension: 8/9 checks pass (LLM relay untested — no external DMs)
Hermes adapter: 9/10 checks pass (deployment pending)
Bots with no incoming messages were reconnecting every ~60 seconds
(20 polls × 3s interval). Raised to 500 polls (~25min) so idle bots
don't cycle. Connection recovers immediately on BAD_EVENT_QUEUE_ID
regardless of this counter.
Zulip event queues expire after ~600s of inactivity. The adapter now
detects 400/BAD_EVENT_QUEUE_ID responses and auto-reconnects instead of
silently returning empty results.
A2A server uses Agent Zero's AgentContext.communicate() directly
to process Zulip messages inside the container.
Full chain verified: Zulip DM -> adapter -> A2A -> Agent Zero -> response -> Zulip
Zulip API edits messages via PATCH /api/v1/messages/{message_id}
(POST returns 405 on this endpoint). Also:
- edit_message() now returns SendResult (not bool) for Gateway compatibility
- Clean up duplicate gateway instances before restart
The Gateway streaming interface calls edit_message() with finalize=True
and expects SendResult. Fixed:
1. Added **kwargs to edit_message() signature — silently ignores finalize
2. Changed return type from bool to SendResult for Gateway compatibility
3. Switched from PATCH to POST /api/v1/messages/{id} — Zulip docs use
POST for message editing (PATCH returns 405 Method Not Allowed)
4. Same fixes applied to delete_message() and send()
Discovered during Tanko Gen 4 testing.
Fixes 4 critical reliability issues discovered during Zulip server outage recovery:
1. CLOSE-WAIT socket leak: _reconnect() now creates a fresh httpx.AsyncClient()
when the queue expires, preventing stuck connection pool. Old client is
explicitly aclose()'d.
2. Silent poll loop death: After sustained 502 errors, the poll loop was
returning [] without logging — transport errors are swallowed by _api_call
returning (None, 0). Added heartbeat logging every 5 minutes and silence
detection warnings after 60s of no events.
3. HTTP client stuck pool: Added proactive client recreation after 20 consecutive
empty polls (connection pool exhaustion detection) and periodic refresh every
500 polls (connection reuse limit).
4. Log spam: 502 error responses from Netbird contain full HTML pages — now
truncated to 80 chars with newlines stripped.
Also:
- Support ZULIP_URL env var (in addition to ZULIP_SITE)
- expose silence_seconds, consecutive_empty_polls, client_pool_resets in health stats
- Reduced health callback interval from 600s to 300s
SendResult only accepts: success, message_id, error, raw_response,
retryable. The adapter was passing platform=, chat_id=, and metadata=
which are not valid fields, causing TypeError on every send attempt.
This was the error Tanko was showing in Zulip:
'SendResult.__init__() got an unexpected keyword argument "platform"'
The /register endpoint doesn't always return user_id on all Zulip
server versions. Added _resolve_bot_user_id() that queries the
/users/me endpoint as a fallback. This ensures @mention detection
(is_direct_mention check) works even when register doesn't provide it.
Also bumped to v1.0.3 — cumulative fixes since v1.0.1:
- v1.0.2: BAD_EVENT_QUEUE_ID detection (silent queue expiry)
- v1.0.3: bot user_id resolution (@mention detection)
Root cause of Tanko not responding to DMs after initial connection:
Zulip event queues expire after ~10min of dont_block polling. When the
queue expired, _api_call() returned None for all 400-level errors, which
_fetch_events() treated as 'no events' and returned []. The poll loop
never knew the queue died, so it never reconnected.
Fix:
1. _api_call() now returns (Optional[Dict], status_code) tuple
2. _fetch_events() explicitly checks for 400 status and raises
RuntimeError('BAD_EVENT_QUEUE_ID') for the poll loop to catch
3. All callers updated to unpack the tuple
Now when a queue expires, the poll loop catches the error and
calls _reconnect() to register a fresh queue.
Root cause of 'str' object has no attribute 'value' error during connect():
the adapter passed a raw string 'zulip' to the base class's platform
parameter, which expects a Platform enum instance. When self.name was
accessed (calling self.platform.value.title()), the string had no .value
attribute.
Fix: import Platform from gateway.config and wrap with Platform('zulip').
Gen 3 improvements from build-zulip-plugin contract run:
1. Malformed message resilience
- Per-event try/except in poll loop — one bad event never kills adapter
- malformed_events counter tracked in health stats
- Fulfills Success Criterion: handles_malformed_messages == true
2. Periodic @all-bots refresh
- _all_bots_refresh_forever() background task re-resolves every hour
- Cancelled gracefully on disconnect()
- all_bots_refreshes counter in health stats
3. Health stats callback mechanism
- set_health_callback(callback, interval=600) — register external consumer
- _report_health_if_callback() triggers on dedup cleanup cycle
- Designed for Hermes Gateway to wire to RA-H OS knowledge graph logging
- Cross-contract integration: get_all_bots_user_id() and get_bot_user_id()
exposed for zulip-mention-reliability contract
- Adds message_update handler that periodically edits the Zulip
placeholder with partial response text (every ~2s during streaming)
- Sends a 'Thinking...' placeholder to Zulip immediately on DM receipt
- Detects agent busy state and uses deliverAs: 'steer' so subsequent
Zulip DMs steer the conversation mid-stream (like terminal TUI)
- Adds editMessage() to the queue helper (Zulip PATCH /messages/{id})
- sendMessage() now returns the message ID for editing
- Turn_start / agent_end lifecycle tracks agent busy state
- Cleanup: streaming timer properly cleared on shutdown
- Removed spawnSync subprocess (pi -p --mode json) for Zulip DM processing
- Messages now inject directly into the current pi session via
pi.sendUserMessage(), reusing the already-hot LLM — zero cold-start
- agent_end event handler captures assistant response and relays it to Zulip
- Added pending reply queue for ordered delivery of multiple Zulip messages
- Fixed zulip-js import (CJS default export, not named createClient)
- Added types.d.ts for zulip-js with default export declaration
- Updated package.json with yaml dependency, @earendil-works/pi-coding-agent
- Config supports both nested YAML and flat env-var formats
- Removed all subprocess timeout/buffer error paths (no longer needed)
- docs/GITOPS.md: complete workflow from prototype to production
- .gitea/pull_request_template.md: pre-merge checklist for swarm PR review
- scripts/deploy.sh: tag-based deployment to all 6 CTs with health checks
- scripts/rollback.sh: rollback to previous tag
- .gitea/workflows/ci.yml: CI pipeline (activates when Actions runners available)
- .gitignore: prevent config.yaml and secrets from being committed
- .gitattributes: consistent LF line endings across platforms
Abiba (pi)