SyslogSolution/zulip-platform-plugins
7
0
Fork 0
Code Issues 19 Pull Requests Actions Packages Projects Releases Wiki Activity

feat: DM-First v2 architecture - Tanko live on CT 112
CI / validate (pull_request) Failing after 1s
Details

Browse Source 
Architecture change: DM-First (v2) replaces @mention-in-stream (v1) as primary
agent interaction channel. See ADR-001-dm-topology and ADR-002-dm-routing.

Key changes:
- New DM-first Zulip adapter (adapter_dm.py) deployed on CT 112
  - Registers empty narrow to receive ALL events
  - Demuxes DMs (type: 'private') vs stream events
  - Replies to DMs back in private thread, stream replies to #agent-hub
  - Uses blocking long-poll (dont_block=False) to eliminate rate limiting
- Rewritten ARCHITECTURE.md, CONTEXT.md, PRD.md for v2 DM-First
- New ADRs: ADR-001 (DM Topology), ADR-002 (DM Routing)
- Archived old ADRs: 001, 002, 005 (topic-topology, mention-routing, mention-detection)
- Updated ADRs: 003 (agent-naming), 004 (per-agent-bots), 006 (all-bots-model)
- Updated PI extension (index.ts) with DM-aware patterns
- Tanko agent confirmed responding in both DM and @all-bots in #agent-hub
This commit is contained in:
kagentz-bot
2026-06-21 07:50:30 -04:00
parent b8ae43f652
commit 705c29a4e0
14 changed files with 1552 additions and 260 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/PRD.md
+121 -79
View File
@@ -1,54 +1,55 @@
# Product Requirements Document: Zulip Platform Plugins
**Status:** Draft v1.0
**Date:** 2026-06-18
**Authors:** Agent Zero (via grill-with-design on Tabiri infrastructure)
**Status:** Draft v2.0
**Date:** 2026-06-21
**Authors:** Agent Zero (via Tabiri infrastructure)
---
## Problem Statement
The Sysloggh agent mesh consists of **6 autonomous agents** across **3 distinct frameworks** (Hermes Python, Agent Zero, PI/openclaw TypeScript) deployed on **5 Proxmox CTs**. These agents operate in isolation — there is no unified inter-agent communication channel. Each agent processes tasks independently, and cross-agent coordination requires manual intervention.
The Sysloggh agent mesh consists of **6 autonomous agents** across **3 distinct frameworks** (Hermes Python, Agent Zero, PI TypeScript) deployed on **5 Proxmox CTs**. These agents operate in isolation — there is no unified inter-agent communication channel. Each agent processes tasks independently, and cross-agent coordination requires manual intervention.
The existing Zulip infrastructure (Zulip 12.0-1 on CT 117, Docker-based) provides a mature async messaging platform, but no agent is wired into it. A partial implementation (`zulip-plugin/adapter.py`) exists with event streaming and dedup logic, but was never deployed. The gateway adapter is compiled but not running on any agent CT, and the required Python packages are not installed.
The existing Zulip infrastructure (Zulip 12.0-1 on CT 117, Docker-based) provides a mature async messaging platform with built-in DM and stream support. All 6 agent bot users exist on the server and `#agent-hub` stream is created. Only Tanko (v1 Hermes native Zulip) is operational.
**The gap:** Agents cannot collaboratively route tasks, share context, or coordinate via Zulip threads.
**The gap:** Agents cannot reliably receive user requests or communicate via Zulip DMs.
---
## Solution
A multi-platform **Zulip agent communication layer** consisting of one plugin per agent framework, each running co-located on its agent's CT, connecting its agent to the shared Zulip messaging infrastructure at `chat.sysloggh.net`.
A multi-platform **Zulip agent communication layer** where each agent communicates primarily via **Zulip Direct Messages (DMs)**. Agents also subscribe to `#agent-hub` stream for `@all-bots` broadcasts only.
### Architecture at a Glance
```
┌──────────────────────────────────────────────────────────────────┐
│ Zulip Server (CT 117) │
│ chat.sysloggh.net:443 │
│ Stream: #agent-hub │
│ Bot Users: tanko-bot, mumuni-bot, koonimo-bot, koby-bot, │
│ kagentz-bot, abiba-bot, all-bots │
└────┬──────┬──────┬──────┬──────┬──────┬──────┬──────────────────┘
│ │ │ │ │ │ │
┌────┴┐ ┌───┴──┐ ┌┴────┐ ┌┴────┐ ┌┴────┐ ┌┴────┐ ┌──────────┐
│Tnako│ │Mumuni│ │Kooni│ │Koby │ │Kage │ │Abiba│ │@all-bots │
│Herm │ │Herm │ │Herm │ │Herm │ │A0 │ │PI │ │ (special)│
│CT112│ │CT... │ │CT.. │ │CT.. │ │CT.. │ │CT.. │ │ │
└─────┘ └──────┘ └─────┘ └─────┘ └─────┘ └─────┘ └──────────┘
│ │ │ │ │ │
┌────┴┐ ┌───┴──┐ ┌┴────┐ ┌┴────┐ ┌┴────┐ ┌┴────┐
│Config│ │Config│ │Config│ │Config│ │Config│ │Config│
│tanko │ │mumuni│ │kooni │ │koby │ │kage │ │abiba │
│.yaml │ │.yaml │ │.yaml │ │.yaml │ │.yaml │ │.yaml │
└─────┘ └──────┘ └─────┘ └─────┘ └─────┘ └─────┘
┌─────────────────────────────────────────────────────────────┐
│ Zulip Server (CT 117) │
│ chat.sysloggh.net:443 │
│ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ Each Agent Has Two Channels: │ │
│ │ │ │
│ │ DM (PRIMARY) #agent-hub (BROADCAST) │ │
│ │ ┌──────────────────┐ ┌──────────────────┐ │ │
│ │ │ Direct 1:1 chat │ │ @all-bots only │ │ │
│ │ │ No @mention │ │ Cross-agent msgs │ │ │
│ │ │ No topic parsing │ │ Broadcast only │ │ │
│ │ └──────────────────┘ └──────────────────┘ │ │
│ └─────────────────────────────────────────────────────┘ │
│ │
│ Bot Users: tanko-bot, mumuni-bot, koonimo-bot, koby-bot, │
│ kagentz-bot, abiba-bot │
│ (No dedicated @all-bots bot user — content-based match) │
└─────────────────────────────────────────────────────────────┘
```
### Core Principles
- **DM-first:** Agent-owner communication uses Zulip DMs exclusively — no @mention parsing, no topic routing, no `mentioned_users` dependency
- **#agent-hub for broadcast only:** The `#agent-hub` stream is subscribed solely for detecting `@all-bots` notifications
- **Co-located:** Each plugin runs on the same CT as its agent, uses platform-native interfaces
- **@mention-routed:** Messages reach the correct agent via Zulip @mention detection
- **Owner-gated:** Private topics are restricted per-agent owner via config
- **Owner-gated:** Each bot DM is gated by API key — only the bot's credentials grant access
- **Version-controlled:** All plugin code lives in a Gitea monorepo, pushed to CTs via deploy script
- **Graceful degradation:** Users always receive feedback, even when an agent is unavailable
@@ -58,52 +59,72 @@ A multi-platform **Zulip agent communication layer** consisting of one plugin pe
### Agent Communication
1. As a **human user**, I want to **@mention an agent in any Zulip topic**, so that the agent receives my message and responds in-thread.
1. As a **human user**, I want to **DM an agent's bot directly**, so that the agent receives my message and responds in-thread.
2. As a **human user**, I want to **@mention @all-bots**, so that all 6 agents across all frameworks receive the message.
2. As a **human user**, I want to **mention @all-bots in #agent-hub**, so that all 6 agents across all frameworks receive the broadcast.
3. As an **agent**, I want to **receive messages addressed to me**, so that I can process and respond to user requests.
3. As an **agent**, I want to **receive DMs addressed to me**, so that I can process and respond without parsing @mentions or stream context.
### Private Channels
4. As a **human user**, I want a **private topic per agent** (e.g., `#agent-hub > tanko-private`), so that I can have confidential conversations with a single agent.
5. As a **system administrator**, I want **per-agent owner configuration**, so that only designated users can access each agent's private topic.
4. As an **agent**, I want to **detect @all-bots broadcasts in #agent-hub**, so that I can participate in multi-agent coordination.
### Deployment & Operations
6. As a **system administrator**, I want to **version-control all plugin code in a single Gitea repository**, so that updates are tracked, auditable, and pushable across all CTs.
5. As a **system administrator**, I want to **version-control all plugin code in a single Gitea repository**, so that updates are tracked, auditable, and pushable across all CTs.
7. As a **system administrator**, I want a **deploy script** that pulls the latest plugin code and restarts services on all CTs, so that updates are consistent and repeatable.
6. As a **system administrator**, I want a **deploy script** that pulls the latest plugin code and restarts services on all CTs, so that updates are consistent and repeatable.
### Reliability
8. As a **human user**, I want to see **a friendly error message** if an agent is unavailable, rather than a silent timeout.
7. As a **human user**, I want to see **a friendly error message** if an agent is unavailable, rather than a silent timeout.
9. As a **system administrator**, I want **health check endpoints** on each plugin, so that I can monitor plugin status from the harness or monitoring infra.
8. As a **system administrator**, I want **health check endpoints** on each plugin, so that I can monitor plugin status from the harness or monitoring infra.
---
## Implementation Decisions
## Architecture Decisions
### Platform Plugin Contracts
### DM-First Communication
Each of the 3 frameworks uses its **native plugin system** — not a unified bridge or A2A overlay for intra-agent communication. Cross-framework @all-bots messages flow through Zulip: all 7 bot users are subscribed to the same `#agent-hub` stream, so every bot receives every message. Each bot checks if it's @mentioned, then either processes or ignores.
Each agent communicates via Zulip **Direct Messages**. When a user sends a DM to an agent's bot:
### Framework-Specific Implementation
```
User DMs @tanko-bot
→ Zulip pushes event ONLY to tanko-bot's event queue
→ Event has: {type: 'message', message: {type: 'private', ...}}
→ Bot detects message.type === 'private'
→ No @mention parsing, no topic/stream capture needed
→ Bot processes message.content
→ Bot replies to same DM via:
{type: 'private', to: [user_id], content: 'response'}
```
| Framework | Plugin Type | Interface | Reference Architecture |
|-----------|-------------|-----------|----------------------|
| **Hermes Python** (4 agents) | Hermes Gateway `BasePlatformAdapter` | `on_event(message) → Response` | Existing `zulip-plugin/` in homelab |
| **PI/openclaw TypeScript** (1 agent) | openclaw Extension | Extension API (similar to existing Discord/GChat/Mattermost/Matrix) | `/a0/usr/projects/homelab/openclaw/extensions/` |
| **Agent Zero** (1 agent) | A0 Plugin | Agent Zero plugin system | Custom A0 plugin pattern |
**Advantages over stream-based @mention:**
- Only the recipient bot receives the DM — no broadcast noise to other 5 agents
- No `mentioned_users` field dependency (unreliable in Zulip 12.0)
- No topic routing — DMs are inherently 1:1 threading
- `message.type: 'private'` is trivially detectable
### @mention Detection
### @all-bots Broadcast
- Use Zulip SDK's `mentioned_users` field on message events (ADR-005)
- Each bot checks if its own `user_id` is in the `mentioned_users` array
- For @all-bots: bot checks if dedicated `All Bots` user is in `mentioned_users` (ADR-006)
- @all-bots spans all 7 bots across all frameworks (ADR-012)
`@all-bots` in `#agent-hub` uses **content-based detection** (regex) — no dedicated bot user needed:
```
User posts in #agent-hub > topic:
"@**all-bots** status report please"
→ Each bot receives event via #agent-hub subscription
→ Each bot checks: /@\*\*(?:all-bots|All Bots)\*\*/i.test(content)
→ If match: bot forwards to its agent
→ 6 independent responses appear in the topic
```
### Plugin Architecture
| Framework | Plugin Type | Interface | DM Detection | @all-bots Detection |
|-----------|-------------|-----------|--------------|--------------------|
| **Hermes Python** (4 agents) | Gateway PlatformAdapter | `on_event(message)` | `message.type === 'private'` | Regex on stream events |
| **PI TypeScript** (1 agent) | PI Extension | Extension API | `event.type === 'private'` | Regex on stream events |
| **Agent Zero** (1 agent) | A0 Plugin | A0 plugin system | `message.type === 'private'` | Regex on stream events |
### Config Schema (per-agent `config.yaml`)
@@ -115,11 +136,12 @@ agent:
bot_email: "tanko-bot@chat.sysloggh.net"
bot_api_key: "<api_key>"
owner_email: "jerome@sysloggh.net"
private_topic: "tanko-private"
zulip:
server_url: "https://chat.sysloggh.net"
stream: "agent-hub"
stream: "agent-hub" # Only used for @all-bots broadcasts
client:
type: "sdk" # "sdk" for zulip-js/zulip-python, "rest" for raw API
error_handling:
timeout_seconds: 30
@@ -131,16 +153,38 @@ monitoring:
health_port: 9200
```
### Private Topic ACL
---
- Bot subscribes to its private topic (`tanko-private`) at startup
- Owner is matched via `config.yaml > owner_email` (ADR-011)
- Messages in private topic: only respond if sender email matches owner
- Messages in public topics: respond to @mentions from any sender
## Implementation Decisions
### Platform Plugin Contracts
Each of the 3 frameworks uses its **native plugin system** — not a unified bridge or A2A overlay for intra-agent communication. Cross-framework `@all-bots` messages flow through Zulip: all 6 bots are subscribed to `#agent-hub` stream. Each bot checks for `@all-bots` via content-based regex match, then processes.
### DM Event Processing
```python
# Simplified DM processing logic (all platforms)
def on_event(event):
message = event["message"]
# DM case (primary)
if message["type"] == "private":
user_id = message["sender_id"]
content = message["content"]
agent.process(content, reply_to=lambda resp: send_dm(user_id, resp))
return
# Stream case — only process if @all-bots
if message["type"] == "stream":
if re.search(r'@\*\*(?:all-bots|All Bots)\*\*', message["content"]):
agent.process_broadcast(message)
# Ignore all other stream messages
```
### Error Handling Strategy
- Agent timeout: plugin sends graceful degradation message to Zulip thread (ADR-009)
- Agent timeout: plugin sends graceful degradation message to DM thread (ADR-009)
- Zulip server down: SDK auto-reconnects with exponential backoff
- Invalid message format: plugin logs error, does not crash
- Health endpoint (`/health` on port 9200): returns status, uptime, last_message_time
@@ -148,28 +192,28 @@ monitoring:
### Naming Convention
- Bot users: `{name}-bot` (lowercase, dash separator) — `tanko-bot`, `mumuni-bot`, etc. (ADR-003)
- Private topics: `{name}-private` — `tanko-private`, `mumuni-private`, etc. (ADR-001)
- Display names: capitalize properly — "Tanko Bot", "Mumuni Bot", "All Bots"
- DM address: `@**tanko-bot**` (same format as @mention)
- Display names: capitalize properly — "Tanko Bot", "Mumuni Bot"
---
## Testing Decisions
### Unit Tests
- @mention detection logic: correct extraction of bot from `mentioned_users`
- Private topic ACL: owner matches, non-owner denied
- DM detection: `message.type === 'private'` correctly triggers processing
- @all-bots detection: regex matches `@**all-bots**` and `@**All Bots**` in stream content
- Config parsing: all fields present, graceful defaults
- Error response: timeout triggers correct grace message
### Integration Tests
- Each plugin connects to Zulip and receives/sends test messages
- @all-bots message reaches all 7 bots
- Private topic message only reaches owner agent
- Each plugin receives DM and sends reply via Zulip API
- @all-bots message in #agent-hub reaches all 6 bots
- Non-@all-bots messages in #agent-hub are correctly ignored
- Health endpoint returns 200 with valid JSON
- Plugin survives container restart (Zulip reconnection)
### Deployment Tests
- `deploy.sh` successfully pulls latest and restarts services on all CTs
- Plugin survives container restart (Zulip reconnection)
- Config YAML validation before deployment
---
@@ -187,18 +231,16 @@ monitoring:
---
## Further Notes
### Design Decisions Summary
## Design Decisions Summary
| # | Decision | Rationale |
|---|----------|-----------|
| ADR-001 | Topic-per-agent with `-private` suffix | Clear naming, easy ACL enforcement |
| ADR-002 | @mention-based routing (not topic-based) | User confirmed, flexible for any topic |
| ADR-001 | DM-first topology | DMs are simpler, more reliable, and require no @mention parsing |
| ADR-002 | DM-based routing (not @mention) | `message.type === 'private'` is trivially detectable, no `mentioned_users` dependency |
| ADR-003 | Bot naming: `{name}-bot` | Consistent, no space issues in Zulip API |
| ADR-004 | One bot per agent, co-located on CT | Isolated failures, platform-native |
| ADR-005 | SDK `mentioned_users` field | Zulip API native, always accurate |
| ADR-006 | Dedicated `All Bots` user | Explicit @all-bots, not noisy @everyone |
| ADR-005 | [Archived] @mention via SDK — no longer needed | DM-first eliminates @mention dependency entirely |
| ADR-006 | @all-bots via content-based regex | No dedicated bot user needed, works across Zulip versions |
| ADR-007 | Platform-native plugin contracts | Reliability, platform handles state/retries |
| ADR-008 | Native message formats per platform | No translation layer needed |
| ADR-009 | Graceful degradation + user-visible errors | No silent failures |
@@ -209,11 +251,11 @@ monitoring:
### Prerequisites
1. Zulip server operational at `https://chat.sysloggh.net`
2. Zulip bot users created (7 total: tanko-bot, mumuni-bot, koonimo-bot, koby-bot, kagentz-bot, abiba-bot, all-bots)
2. Zulip bot users created (6 total: tanko-bot, mumuni-bot, koonimo-bot, koby-bot, kagentz-bot, abiba-bot)
3. `#agent-hub` stream created and all bots subscribed
4. Gitea repo `zulip-platform-plugins` created and monorepo pushed
5. Hermes gateway framework verified on Tanko/Mumuni/Koonimo/Koby CTs
6. openclaw extension API verified on Abiba CT
6. PI extension API verified on Abiba CT
7. Agent Zero plugin system verified on Kagentz CT
### Deployment Workflow