zulip-platform-plugins/docs/PRD.md

# Product Requirements Document: Zulip Platform Plugins

**Status:** Draft v2.0
**Date:** 2026-06-21
**Authors:** Agent Zero (via Tabiri infrastructure)

---

## Problem Statement

The Sysloggh agent mesh consists of **6 autonomous agents** across **3 distinct frameworks** (Hermes Python, Agent Zero, PI TypeScript) deployed on **5 Proxmox CTs**. These agents operate in isolation — there is no unified inter-agent communication channel. Each agent processes tasks independently, and cross-agent coordination requires manual intervention.

The existing Zulip infrastructure (Zulip 12.0-1 on CT 117, Docker-based) provides a mature async messaging platform with built-in DM and stream support. All 6 agent bot users exist on the server and `#agent-hub` stream is created. Only Tanko (v1 Hermes native Zulip) is operational.

**The gap:** Agents cannot reliably receive user requests or communicate via Zulip DMs.

---

## Solution

A multi-platform **Zulip agent communication layer** where each agent communicates primarily via **Zulip Direct Messages (DMs)**. Agents also subscribe to `#agent-hub` stream for `@all-bots` broadcasts only.

### Architecture at a Glance

```
┌─────────────────────────────────────────────────────────────┐
│                   Zulip Server (CT 117)                     │
│                   chat.sysloggh.net:443                     │
│                                                             │
│  ┌─────────────────────────────────────────────────────┐   │
│  │            Each Agent Has Two Channels:              │   │
│  │                                                     │   │
│  │  DM (PRIMARY)              #agent-hub (BROADCAST)   │   │
│  │  ┌──────────────────┐     ┌──────────────────┐     │   │
│  │  │ Direct 1:1 chat  │     │ @all-bots only    │     │   │
│  │  │ No @mention      │     │ Cross-agent msgs  │     │   │
│  │  │ No topic parsing │     │ Broadcast only    │     │   │
│  │  └──────────────────┘     └──────────────────┘     │   │
│  └─────────────────────────────────────────────────────┘   │
│                                                             │
│  Bot Users: tanko-bot, mumuni-bot, koonimo-bot, koby-bot,  │
│             kagentz-bot, abiba-bot                          │
│  (No dedicated @all-bots bot user — content-based match)    │
└─────────────────────────────────────────────────────────────┘
```

### Core Principles

- **DM-first:** Agent-owner communication uses Zulip DMs exclusively — no @mention parsing, no topic routing, no `mentioned_users` dependency
- **#agent-hub for broadcast only:** The `#agent-hub` stream is subscribed solely for detecting `@all-bots` notifications
- **Co-located:** Each plugin runs on the same CT as its agent, uses platform-native interfaces
- **Owner-gated:** Each bot DM is gated by API key — only the bot's credentials grant access
- **Version-controlled:** All plugin code lives in a Gitea monorepo, pushed to CTs via deploy script
- **Graceful degradation:** Users always receive feedback, even when an agent is unavailable

---

## User Stories

### Agent Communication

1. As a **human user**, I want to **DM an agent's bot directly**, so that the agent receives my message and responds in-thread.

2. As a **human user**, I want to **mention @all-bots in #agent-hub**, so that all 6 agents across all frameworks receive the broadcast.

3. As an **agent**, I want to **receive DMs addressed to me**, so that I can process and respond without parsing @mentions or stream context.

4. As an **agent**, I want to **detect @all-bots broadcasts in #agent-hub**, so that I can participate in multi-agent coordination.

### Deployment & Operations

5. As a **system administrator**, I want to **version-control all plugin code in a single Gitea repository**, so that updates are tracked, auditable, and pushable across all CTs.

6. As a **system administrator**, I want a **deploy script** that pulls the latest plugin code and restarts services on all CTs, so that updates are consistent and repeatable.

### Reliability

7. As a **human user**, I want to see **a friendly error message** if an agent is unavailable, rather than a silent timeout.

8. As a **system administrator**, I want **health check endpoints** on each plugin, so that I can monitor plugin status from the harness or monitoring infra.

---

## Architecture Decisions

### DM-First Communication

Each agent communicates via Zulip **Direct Messages**. When a user sends a DM to an agent's bot:

```
User DMs @tanko-bot
  → Zulip pushes event ONLY to tanko-bot's event queue
  → Event has: {type: 'message', message: {type: 'private', ...}}
  → Bot detects message.type === 'private'
  → No @mention parsing, no topic/stream capture needed
  → Bot processes message.content
  → Bot replies to same DM via:
    {type: 'private', to: [user_id], content: 'response'}
```

**Advantages over stream-based @mention:**
- Only the recipient bot receives the DM — no broadcast noise to other 5 agents
- No `mentioned_users` field dependency (unreliable in Zulip 12.0)
- No topic routing — DMs are inherently 1:1 threading
- `message.type: 'private'` is trivially detectable

### @all-bots Broadcast

`@all-bots` in `#agent-hub` uses **content-based detection** (regex) — no dedicated bot user needed:

```
User posts in #agent-hub > topic:
  "@**all-bots** status report please"

  → Each bot receives event via #agent-hub subscription
  → Each bot checks: /@\*\*(?:all-bots|All Bots)\*\*/i.test(content)
  → If match: bot forwards to its agent
  → 6 independent responses appear in the topic
```

### Plugin Architecture

| Framework | Plugin Type | Interface | DM Detection | @all-bots Detection |
|-----------|-------------|-----------|--------------|--------------------|
| **Hermes Python** (4 agents) | Gateway PlatformAdapter | `on_event(message)` | `message.type === 'private'` | Regex on stream events |
| **PI TypeScript** (1 agent) | PI Extension | Extension API | `event.type === 'private'` | Regex on stream events |
| **Agent Zero** (1 agent) | A0 Plugin | A0 plugin system | `message.type === 'private'` | Regex on stream events |

### Config Schema (per-agent `config.yaml`)

```yaml
# config.yaml (one per bot, stored alongside plugin)
agent:
  name: "Tanko"
  bot_username: "tanko-bot"
  bot_email: "tanko-bot@chat.sysloggh.net"
  bot_api_key: "<api_key>"
  owner_email: "jerome@sysloggh.net"

zulip:
  server_url: "https://chat.sysloggh.net"
  stream: "agent-hub"          # Only used for @all-bots broadcasts
  client:
    type: "sdk"                # "sdk" for zulip-js/zulip-python, "rest" for raw API

error_handling:
  timeout_seconds: 30
  retry_count: 3
  graceful_message: "{{agent_name}} is processing your request — please wait..."

monitoring:
  health_endpoint_enabled: true
  health_port: 9200
```

---

## Implementation Decisions

### Platform Plugin Contracts

Each of the 3 frameworks uses its **native plugin system** — not a unified bridge or A2A overlay for intra-agent communication. Cross-framework `@all-bots` messages flow through Zulip: all 6 bots are subscribed to `#agent-hub` stream. Each bot checks for `@all-bots` via content-based regex match, then processes.

### DM Event Processing

```python
# Simplified DM processing logic (all platforms)
def on_event(event):
    message = event["message"]

    # DM case (primary)
    if message["type"] == "private":
        user_id = message["sender_id"]
        content = message["content"]
        agent.process(content, reply_to=lambda resp: send_dm(user_id, resp))
        return

    # Stream case — only process if @all-bots
    if message["type"] == "stream":
        if re.search(r'@\*\*(?:all-bots|All Bots)\*\*', message["content"]):
            agent.process_broadcast(message)
        # Ignore all other stream messages
```

### Error Handling Strategy

- Agent timeout: plugin sends graceful degradation message to DM thread (ADR-009)
- Zulip server down: SDK auto-reconnects with exponential backoff
- Invalid message format: plugin logs error, does not crash
- Health endpoint (`/health` on port 9200): returns status, uptime, last_message_time

### Naming Convention

- Bot users: `{name}-bot` (lowercase, dash separator) — `tanko-bot`, `mumuni-bot`, etc. (ADR-003)
- DM address: `@**tanko-bot**` (same format as @mention)
- Display names: capitalize properly — "Tanko Bot", "Mumuni Bot"

---

## Testing Decisions

### Unit Tests
- DM detection: `message.type === 'private'` correctly triggers processing
- @all-bots detection: regex matches `@**all-bots**` and `@**All Bots**` in stream content
- Config parsing: all fields present, graceful defaults
- Error response: timeout triggers correct grace message

### Integration Tests
- Each plugin receives DM and sends reply via Zulip API
- @all-bots message in #agent-hub reaches all 6 bots
- Non-@all-bots messages in #agent-hub are correctly ignored
- Health endpoint returns 200 with valid JSON
- Plugin survives container restart (Zulip reconnection)

### Deployment Tests
- `deploy.sh` successfully pulls latest and restarts services on all CTs
- Config YAML validation before deployment

---

## Out of Scope (v1.0)

- **Multi-realm Zulip:** All agents operate within a single Zulip realm
- **Message attachments:** v1 sends and receives text only (no file uploads)
- **Rich formatting:** v1 uses plain markdown responses from agents
- **A2A as primary transport:** Cross-platform communication routes through Zulip, not direct A2A
- **CI/CD pipeline:** Deployment is script-based (`deploy.sh`), not CI/CD
- **Rate limiting:** Zulip's built-in rate limiting is sufficient for initial use
- **Dashboard/monitoring UI:** Health endpoint data consumed by existing harness infra
- **Message history search:** Zulip's native search handles this

---

## Design Decisions Summary

| # | Decision | Rationale |
|---|----------|-----------|
| ADR-001 | DM-first topology | DMs are simpler, more reliable, and require no @mention parsing |
| ADR-002 | DM-based routing (not @mention) | `message.type === 'private'` is trivially detectable, no `mentioned_users` dependency |
| ADR-003 | Bot naming: `{name}-bot` | Consistent, no space issues in Zulip API |
| ADR-004 | One bot per agent, co-located on CT | Isolated failures, platform-native |
| ADR-005 | [Archived] @mention via SDK — no longer needed | DM-first eliminates @mention dependency entirely |
| ADR-006 | @all-bots via content-based regex | No dedicated bot user needed, works across Zulip versions |
| ADR-007 | Platform-native plugin contracts | Reliability, platform handles state/retries |
| ADR-008 | Native message formats per platform | No translation layer needed |
| ADR-009 | Graceful degradation + user-visible errors | No silent failures |
| ADR-010 | Gitea monorepo + deploy.sh | Single source of truth, versioned |
| ADR-011 | Owner in per-agent config.yaml | Explicit, version-controllable |
| ADR-012 | @all-bots spans all frameworks | Single @all-bots reaches every agent |

### Prerequisites

1. Zulip server operational at `https://chat.sysloggh.net`
2. Zulip bot users created (6 total: tanko-bot, mumuni-bot, koonimo-bot, koby-bot, kagentz-bot, abiba-bot)
3. `#agent-hub` stream created and all bots subscribed
4. Gitea repo `zulip-platform-plugins` created and monorepo pushed
5. Hermes gateway framework verified on Tanko/Mumuni/Koonimo/Koby CTs
6. PI extension API verified on Abiba CT
7. Agent Zero plugin system verified on Kagentz CT

### Deployment Workflow

```
1. Developer pushes plugin changes to Gitea repo
2. Run: ./scripts/deploy.sh
3. deploy.sh SSHes into each CT:
   a. git pull
   b. Install/update dependencies
   c. Restart plugin service (systemd or supervisor)
4. Verify health endpoint on each CT
```

### Risk Mitigation

- **Template bug in Zulip 12.0-1:** Portico-header renders 500 on configs without `REGISTRATION_OPEN` or with Authentik SSO — resolved via template patch `{% if settings.REGISTRATION_OPEN %}` guard
- **Docker container restarts:** Plugin auto-reconnects via Zulip SDK's built-in reconnect
- **API key rotation:** Keys stored in plugin config, updated via Gitea push