UPDATE 2026-07-15: Full fleet rebuild

- Stable role-based aliases: strix-moe, gpu-dense, gpu-light
- Strix Halo: ornith-1.0-35b -> unsloth/Qwen3.6-35B-A3B-MTP (UD-Q4_K_M, 256K)
- RTX 5070: Q4_K_M -> IQ4_NL+MTP (256K, 125-213 tok/s, 2x faster)
- RTX 3090: bumped to 256K context
- Provider rename: harness -> litellm (all agents)
- syslog-auto: weighted pool restored with direct GPU routing
- Mumuni agent profile documented
- Self-heal thresholds updated for 256K context
- All model contexts corrected (RTX 3090 was 131K, not 256K)

abiba-zulip-restore.prose.md

@@ -0,0 +1,309 @@
+---
+kind: function
+name: abiba-zulip-restore
+description: >
+  Restores Zulip connectivity for Abiba (pi agent). Verifies the v2 router-worker
+  extension code, starts a PM2 process as the Zulip gateway with correct env vars,
+  validates health endpoint, and confirms DM delivery. Run this whenever Abiba
+  stops responding on Zulip or after system restart.
+agent: abiba
+version: 1.0.0
+status: active
+runtime_contract: 2
+---
+
+# Abiba Zulip Restore — Resume pi Zulip Communication
+
+Single-shot function that restores full Zulip connectivity for the Abiba pi agent.
+Covers extension code validation, PM2 process management, health endpoint
+verification, and DM loopback testing.
+
+## Live-State Fields
+
+| Field | Value | Trust |
+|-------|-------|-------|
+| Agent name | abiba | ✅ |
+| Bot email | abiba-bot@chat.sysloggh.net | ✅ |
+| Zulip server | https://chat.sysloggh.net | ✅ |
+| Extension path | /root/.pi/agent/extensions/zulip/index.js | ✅ |
+| Config path | /root/.pi/agent/extensions/zulip/config.yaml | ✅ |
+| Health port | 9200 | ✅ |
+| @all-bots user ID | 20 | ✅ (config, verified by API at runtime) |
+| PM2 process name | abiba-zulip | ✅ |
+| Provider | syslog-harness (http://192.168.68.116/v1) | ✅ |
+| Default model | deepseek-v4-pro | ✅ (settings.json) |
+
+## Architecture
+
+The pi Zulip extension uses a **router-worker architecture** (v2):
+
+- **Router** (PM2 `abiba-zulip`, `ZULIP_ROLE=router`): Polls Zulip for events,
+  maintains a per-sender pool of pi RPC worker processes. Each sender gets
+  their own `pi --mode rpc --session-dir` process with persistent sessions.
+  Handles streaming edits back to Zulip.
+- **Worker** (child `pi --mode rpc`): Runs the agent with per-sender persistent
+  sessions. No Zulip logic in the worker — the router handles all Zulip I/O.
+
+The extension loads in ALL pi sessions (via `settings.json` extensions array)
+but is a **NO-OP** unless `ZULIP_ROLE=router` is set. Only the PM2 router process
+carries the env var.
+
+## Maintains
+
+- extension_valid: bool — Extension code imports without errors
+- pm2_running: bool — PM2 process `abiba-zulip` is online
+- health_responding: bool — GET :9200/health returns "ok"
+- zulip_connected: bool — Queue registered, bot identity resolved
+- loopback_delivered: bool — Test DM sent and received
+- model_valid: bool — Configured models match LiteLLM authorized models
+
+### Postconditions
+
+- PM2 process `abiba-zulip` online and stable (uptime > 30s)
+- Health endpoint returns `{ status: "ok", connected: true }`
+- Worker pool creates sessions on demand
+- Echo prevention active (BOT_EMAILS includes all known bots)
+- PM2 saved for auto-restart on boot
+
+## Requires
+
+- Node.js with `yaml` module available
+- PM2 installed globally
+- Zulip API key in `config.yaml`
+- `pi` CLI available on PATH
+- Zulip server accessible at https://chat.sysloggh.net
+- LiteLLM provider accessible at http://192.168.68.116/v1
+
+## Execution
+
+### Step 1: Validate Extension Code
+
+```bash
+node -e "import('file:///root/.pi/agent/extensions/zulip/index.js').then(() => console.log('OK')).catch(e => {console.error('FAIL:', e.message); process.exit(1)})"
+```
+
+Expected: `OK`. If FAIL → check for missing dependencies, syntax errors.
+
+### Step 2: Validate Model IDs
+
+Compare configured models against LiteLLM authorized models:
+
+```bash
+API_KEY=$(grep -oP 'apiKey:\s*\K.*' /root/.pi/agent/models.json | head -1)
+curl -s -H "Authorization: Bearer $API_KEY" http://192.168.68.116/v1/models | \
+  python3 -c "import json,sys; d=json.load(sys.stdin); [print(m['id']) for m in d.get('data',[])]" 2>/dev/null
+```
+
+Check: Every model ID in `models.json` must appear in the authorized list.
+If not → fix `models.json` to only include authorized models (prefer `syslog-auto`).
+
+### Step 3: Verify Config Integrity
+
+```bash
+python3 -c "
+import yaml, sys
+with open('/root/.pi/agent/extensions/zulip/config.yaml') as f:
+    cfg = yaml.safe_load(f)
+required = ['zulip.site', 'zulip.email', 'zulip.api_key', 'agent.name']
+for k in required:
+    keys = k.split('.')
+    v = cfg
+    for kk in keys:
+        v = v.get(kk)
+        if v is None:
+            print(f'MISSING: {k}')
+            sys.exit(1)
+print('Config valid')
+print(f'  site={cfg[\"zulip\"][\"site\"]}')
+print(f'  email={cfg[\"zulip\"][\"email\"]}')
+print(f'  agent={cfg[\"agent\"][\"name\"]}')
+print(f'  health_port={cfg.get(\"health_port\", 9200)}')
+"
+```
+
+Expected: Config valid with all fields non-empty.
+
+### Step 4: Remove Stale Systemd Service
+
+The old `abiba-zulip.service` points to `/opt/abiba-zulip/dist/index.js` (compiled
+TypeScript, not the v2 extension). It's disabled and stale. Remove it:
+
+```bash
+systemctl stop abiba-zulip 2>/dev/null || true
+systemctl disable abiba-zulip 2>/dev/null || true
+rm -f /etc/systemd/system/abiba-zulip.service
+systemctl daemon-reload
+```
+
+### Step 5: Start PM2 Process
+
+**Critical:** The extension MUST run via `pi --mode rpc`, NOT `node index.js` directly.
+The extension exports a function that requires pi's session lifecycle. Running
+`node index.js` loads the module but never calls the export, so nothing happens.
+`pi --mode rpc` loads all extensions (including Zulip) and fires `session_start`.
+
+```bash
+# Stop existing if any
+pm2 delete abiba-zulip 2>/dev/null || true
+
+# Start pi in RPC mode with ZULIP_ROLE=router env
+ZULIP_ROLE=router pm2 start "$(which pi)" \
+  --name abiba-zulip \
+  --interpreter none \
+  -- --mode rpc --no-session
+```
+
+Wait 10 seconds for pi to load all extensions, fire session_start, and the Zulip
+router to register its event queue.
+
+### Step 6: Validate PM2 Process
+
+```bash
+pm2 show abiba-zulip --no-color
+```
+
+Check: `status=online`, `restarts=0`, `uptime > 5s`.
+
+### Step 7: Check Logs for Connection
+
+```bash
+sleep 3
+tail -20 /root/.pm2/logs/abiba-zulip-out.log
+```
+
+Look for:
+- `[zulip-ext] Connecting to https://chat.sysloggh.net as abiba-bot@chat.sysloggh.net…`
+- `[zulip-ext] Bot user_id=N, all-bots user_id=N`
+- `[zulip-ext] Connected, queue=N`
+- `[zulip-ext] Echo prevention: N bot emails`
+- `[zulip-ext] Health endpoint on :9200`
+
+If error → check API key, network to chat.sysloggh.net.
+
+### Step 8: Health Endpoint
+
+```bash
+curl -s http://localhost:9200/health | python3 -m json.tool
+```
+
+Check:
+- `status: "ok"` (not "down")
+- `zulip.connected: true`
+- `zulip.queue_id` is non-null string
+- `zulip.bot_user_id` is positive integer
+
+### Step 9: DM Loopback Test
+
+```bash
+curl -s http://localhost:9200/health | python3 -c "
+import json,sys
+d = json.load(sys.stdin)
+if d.get('zulip',{}).get('connected'):
+    print(f'✅ Zulip connected. Queue: {d[\"zulip\"][\"queue_id\"]}')
+    print(f'   Bot user_id: {d[\"zulip\"][\"bot_user_id\"]}')
+    print(f'   Messages processed: {d[\"zulip\"][\"messages_processed\"]}')
+else:
+    print('❌ Zulip NOT connected')
+    sys.exit(1)
+"
+```
+
+### Step 10: Save PM2 for Auto-Start
+
+```bash
+pm2 save
+pm2 startup systemd -u root --hp /root 2>/dev/null || true
+```
+
+### Step 11: Report
+
+Compile results:
+
+| Check | Pass? |
+|-------|-------|
+| Extension code imports | extension_valid |
+| Model IDs authorized | model_valid |
+| Config integrity | config_valid |
+| PM2 process online | pm2_running |
+| Health endpoint | health_responding |
+| Zulip connected | zulip_connected |
+
+All pass → ✅ **Abiba Zulip restored.** Relay success to user.
+
+Partial failure → see recovery matrix below.
+
+## Recovery Matrix
+
+| Failure | Recovery |
+|---------|----------|
+| Extension import fails | Check `node_modules/zulip-js` exists; run `npm install` in extension dir |
+| Model ID mismatch | Fix `models.json` to use `syslog-auto` as default model; remove invalid IDs |
+| Config missing | Restore from backup or recreate from scratch |
+| PM2 won't start | Check `node` version (>=18); check port 9200 not in use |
+| Health "down" | Check logs for connection errors; verify Zulip API key; check network |
+| "address already in use" | Kill old process: `fuser -k 9200/tcp` |
+| Queue registration fails | Check Zulip API key validity; verify bot is active in Zulip admin |
+| Rate limit (429) | Extension has built-in retry-after handling — wait, don't restart |
+
+## Known Failure Modes
+
+| Symptom | Root Cause | Recovery |
+|---------|-----------|----------|
+| Extension loads but no events | `ZULIP_ROLE` not set | Ensure PM2 env has `ZULIP_ROLE=router` |
+| Worker stays "busy" forever | Model ID not authorized by LiteLLM | Fix models.json (lesson #11) |
+| Placeholder sent but no response | editMessage API fails silently | Extension has fallback (sends new msg); check Zulip API |
+| Queue expires rapidly | Poll interval too aggressive | v2 uses 3s poll with long-poll — should be fine |
+| Bot doesn't respond to @mentions | Not subscribed to stream | Bot auto-subscribes via API |
+| Stale error in health | `last_error` not cleared | v2 clears on successful poll (lesson #4) |
+
+## Appendix: Root Cause & Fix Summary (2026-07-13)
+
+**Problem:** Zulip extension was offline — no PM2 process running.
+
+**Root cause:** The PM2 command ran `node index.js` directly (which loads the
+extension module but never calls the exported function). The extension requires
+pi's session lifecycle — pi loads extensions, fires `session_start`, and the
+Zulip extension hooks into that event.
+
+**Fix:** Run `pi --mode rpc` (not `node index.js`). The `--mode rpc` flag keeps
+pi alive listening for RPC commands on stdin while the Zulip extension's router
+runs in the background via the `session_start` hook.
+
+```bash
+ZULIP_ROLE=router pm2 start "$(which pi)" --name abiba-zulip \
+  --interpreter none -- --mode rpc --no-session
+```
+
+**Additional fixes applied:**
+- Fixed `models.json`: replaced `qwen3.6-35B-A3B` (not authorized by LiteLLM)
+  with `syslog-auto` + `ornith-1.0-35b` (prevents silent worker failure per
+  Lesson #11)
+- Removed stale systemd unit `abiba-zulip.service` (pointed to old TS code)
+- PM2 saved for auto-restart on boot
+
+## Appendix: PM2 Ecosystem Config (Optional)
+
+If preferred over manual `pm2 start`, create `/root/ecosystem.config.js` entry:
+
+```js
+module.exports = {
+  apps: [{
+    name: 'abiba-zulip',
+    script: '/bin/pi',
+    interpreter: 'none',
+    args: '--mode rpc --no-session',
+    cwd: '/root',
+    env: {
+      ZULIP_ROLE: 'router',
+    },
+    log_file: '/root/.pm2/logs/abiba-zulip-out.log',
+    error_file: '/root/.pm2/logs/abiba-zulip-error.log',
+    max_restarts: 20,
+    restart_delay: 5000,
+  }]
+};
+```
+
+---
+
+**Last verified good state**: 2026-07-13 — Extension v2 running via `pi --mode rpc`, health endpoint :9200 returning `{status:"ok",connected:true}`, queue a669f21e.

delegation-prose-contract.prose.md

@@ -0,0 +1,256 @@
+---
+kind: pattern
+name: delegation-prose-contract
+description: >
+  Manager (Mumuni) operating doctrine for task decomposition, worker
+  delegation, verification, and delivery. Defines when to delegate, which
+  worker to use for what, how to handle failures, and the kanban board
+  protocol. Enforces context-window discipline and separation of concerns.
+  Runs on Mumuni (CT 118, storepve, .6) via Hermes agent.
+version: 1.0.0
+---
+
+## Maintains
+
+- Worker roster: 6 profiles (`syslog-code`, `syslog-devops`, `syslog-email`,
+  `syslog-research`, `syslog-review`, `syslog-writer`)
+- Kanban board state at `~/.hermes/kanban/kanban.json`
+- Context window budget: ~65K tokens per request (131K total, 60% threshold)
+
+## Topology
+
+**Cluster:** 5 Proxmox nodes (ocupve, acerpve, minipve, amdpve, storepve)
+**Manager:** Mumuni (CT 118, storepve, .6) via Hermes agent
+**Workers:** 6 profiles, all running on the same agent — no separate hosts needed
+
+This contract is infrastructure-agnostic in terms of which nodes are used.
+Workers execute tasks on whatever infrastructure they're given — SSH to .6,
+.pm, .9, .12, or .15 depending on the task. The contract defines the
+**who** and **when** — not the **where**.
+
+## Why This Matters
+
+Without enforced delegation, the manager consumes the full iteration budget
+(60 calls) on single-turn tasks — SSH to 5 nodes, check each VM, read logs —
+leaving no capacity for actual coordination. The result: context overflow
+(59K tokens in system prompt), iteration exhaustion, and degraded response
+quality. This contract exists because I blew through my budget checking
+Proxmox node status instead of delegating to `syslog-devops`.
+
+## Context Window Discipline
+
+**The system prompt is ~6.5K tokens (stable: ~4.5K tool schemas + ~2K other guidance).**
+**Volatile (MEMORY.md + USER.md): ~300 tokens.**
+**Total base: ~6,800 tokens per request.**
+
+The remaining budget is the conversation. Every tool call result adds to it.
+If a single call returns >10K tokens (e.g., `grep` on a large file, SSH output
+from multiple nodes), the context fills fast. That's why we delegate: workers
+process in isolation and return compact results.
+
+## Trigger Conditions
+
+Delegation is **mandatory** when any of these apply:
+
+| Condition | Threshold | Example |
+|-----------|-----------|---------|
+| Multiple tool calls needed | 2+ calls with intermediate logic | Read file → analyze → write report |
+| Large data retrieval | Output >5K tokens | `grep -r "pattern" /path` on large dirs |
+| Cross-domain work | Spans 2+ worker specialties | Infra check + email filter |
+| Infrastructure changes | Any mutating operation | `qm set`, `systemctl restart`, `git push` |
+| Research/analysis | Needs browser or deep reading | Web research, code review, data analysis |
+| Code builds or changes | Writing or modifying code | Scripts, configs, patches |
+| Sequential dependencies | Worker B needs Worker A's output | Code → Review → Deliver |
+
+**Single tool calls stay at manager level.** Quick `grep`, `ls`, `cat`,
+`curl`, `hermes tools list` — these are decision-making tools. The manager
+reads them directly.
+
+## Worker Selection Matrix
+
+| Worker | Model | Toolsets | Role | Use When |
+|--------|-------|----------|------|----------|
+| `syslog-code` | qwen3.6-27B-code | terminal, file, web, memory, skills | Code patches, automation, scripts | Writing/modifying code, creating scripts, debugging, reading/writing files |
+| `syslog-devops` | qwen3.6-27B-code | terminal, file, web, memory, skills | Infrastructure, DB, bridge, Proxmox | Server ops, SSH, Docker, Proxmox, DB queries, hardware checks |
+| `syslog-email` | ornith-1.0-35b | terminal, file, web, memory, skills | Email automation, mail operations | Sending/receiving email, inbox management, SMTP operations |
+| `syslog-research` | ornith-1.0-35b | terminal, file, web, memory, skills, **browser** | Analysis, classification, data processing | Web research, browser tasks, data analysis, classification, reading docs |
+| `syslog-review` | ornith-1.0-35b | terminal, file, web, memory, skills | Verification, QA, audit validation | **ALWAYS** verify worker output before delivery — especially for infra changes, code builds, and research findings |
+| `syslog-writer` | ornith-1.0-35b | terminal, file, web, memory, skills | Docs, content, branding, reports | Writing docs, reports, proposals, content, markdown formatting |
+
+### Selection Rules
+
+1. **Match specialty first.** A code task → `syslog-code`. An infra task →
+   `syslog-devops`. Don't put a `syslog-email` worker on a code review.
+2. **Research tasks with browser needs → `syslog-research`.** Other workers
+   don't have the browser toolset.
+3. **Verification → `syslog-review`.** Never deliver raw worker output.
+4. **Documentation/content → `syslog-writer`.** Let them own the prose.
+5. **If unsure, delegate to `syslog-research`** — it has the broadest toolset
+   (includes browser) and high reasoning effort.
+
+## Delegation Protocol
+
+### Step 1: Decompose
+
+Break the task into lanes. Each lane does ONE thing. Workers are independent —
+no lane depends on another's output mid-flight. If lanes depend on each other,
+dispatch sequentially.
+
+### Step 2: Dispatch
+
+Fire workers via `delegate_task`:
+
+**Parallel (independent lanes):**
+```
+delegate_task(
+    tasks=[
+        {"goal": "Check all 5 Proxmox nodes for VM status", "context": "SSH to each node via 192.168.68.x, run 'qm list'"},
+        {"goal": "Check Docker container health on .7/.116/.17", "context": "SSH to each host, check container status"},
+    ]
+)
+```
+
+**Sequential (dependent lanes):**
+Dispatch lane 1 → wait for result → dispatch lane 2.
+
+### Step 3: Verify
+
+**MANDATORY for:**
+- Infrastructure changes (any `qm`, `pct`, `systemctl`, `git push`)
+- Code builds and modifications
+- Research findings (web data, external sources)
+- Any output that will reach the user
+
+**Fire `syslog-review` to verify:**
+```
+delegate_task(
+    goal="Review the output of the devops worker. Verify the node status
+    report is accurate, check for inconsistencies, confirm all nodes were
+    reachable.",
+    context="Worker was syslog-devops. Output is at /tmp/node-report.md.
+    Verify against live system."
+)
+```
+
+**If verification fails:**
+1. Send work back to original worker with review feedback
+2. Re-verify
+3. Max 2 re-verify cycles before escalating to Kwame
+
+### Step 4: Deliver
+
+Only verified results reach Kwame. Format per channel:
+- Telegram: Use `telegram-formatting` skill
+- Zulip: Use Zulip Markdown (CommonMark)
+- Email: Use `syslog-email` skill
+
+## Kanban Board Protocol
+
+**File:** `~/.hermes/kanban/kanban.json`
+
+```json
+{
+  "task_id": "unique-id",
+  "title": "Task description",
+  "created": "2026-07-09T01:00:00",
+  "status": "backlog|in_progress|review|done",
+  "lanes": [
+    {
+      "lane_id": "devops-check",
+      "worker": "syslog-devops",
+      "goal": "Check all 5 Proxmox nodes",
+      "status": "dispatched|completed|failed",
+      "output_file": "/tmp/node-report.md"
+    }
+  ]
+}
+```
+
+**Update the board on every state change.**
+
+## Failure Handling
+
+### Worker Timeouts
+
+- Child timeout: **900 seconds** (15 minutes)
+- Worker model `syslog-auto` is slow — it can hit the timeout limit with
+  22+ API calls
+- **If a worker times out:** Re-dispatch with a narrower scope. Break the
+  task into smaller pieces that fit in the timeout window.
+- **Avoid delegating sequential SSH hops** — each SSH connection adds latency
+  that compounds quickly. Prefer API-based or local approaches when possible.
+
+### Worker Selection Failures
+
+- `syslog-devops` is best for infrastructure tasks (SSH, Proxmox, Docker)
+- `syslog-code` is best for code-level work (reading files, writing scripts)
+- `syslog-research` has the browser toolset — use for web research
+- `syslog-review` is the QA gate — always fire before delivery
+- **Never fire more than 3 parallel workers** (max_concurrent_children: 3)
+- **Never nest delegation** (max_spawn_depth: 1)
+
+### Context Overflow
+
+- If a task requires >10K tokens of output, delegate the processing
+- Workers return compact summaries, not raw data dumps
+- Pass file paths and concrete goals — never dump raw data into context
+
+## Anti-patterns
+
+- ❌ Reading large files into your own context before deciding → delegate the read
+- ❌ Carrying SSH/grep/output results in your context → delegate the analysis
+- ❌ Doing work yourself and then "pretending" to delegate → the user can tell
+- ❌ Skipping verification → raw worker output never reaches the user
+- ❌ Delegating single tool calls → keep quick reads/writes at manager level
+- ❌ Firing more than 3 workers in parallel → hard limit
+
+## Emergency Exception
+
+**In an emergency (server down, service must be restored immediately):**
+- Delegate the diagnosis (find the problem)
+- Execute the fix yourself (minimize handoff latency)
+- Verify the fix after delivery
+- Log the exception in the kanban board
+
+The emergency exception exists because the user needs the service back NOW,
+not after three worker round-trips. But it's an exception — not the rule.
+
+## What This Contract Doesn't Cover
+
+1. **Worker profile configuration** — covered by `hermes-config-template.prose.md`
+2. **SSH key management** — covered by existing SSH/Proxmox contracts
+3. **Git workflow** — covered by `AGENTS.md` in the prose-contracts repo
+4. **Cron job management** — covered by individual cron contracts
+5. **Infra verification** — covered by `verify-before-mutate` protocol
+
+## Verification
+
+Run `scripts/worker-audit.py` to verify all 6 profiles are aligned:
+
+```bash
+python3 /root/.hermes/skills/kanban-orchestrator/scripts/worker-audit.py
+```
+
+## References
+
+- `kanban-orchestrator` skill: The operational playbook (detailed execution steps)
+- `worker-profile-audit.md` (skill reference): Worker configuration audit notes
+- `delegation-timeout-patterns.md` (skill reference): Timeout handling patterns
+- `verify-before-mutate` protocol: Infrastructure change verification
+- `hermes-config-template.prose.md`: Worker profile configuration
+
+## Success Criteria
+
+This contract succeeds when:
+
+1. **No context overflow** — single-turn tasks don't exhaust the iteration budget
+2. **Workers do the work** — manager coordinates, doesn't execute
+3. **Verification before delivery** — all output passes through `syslog-review`
+4. **Kanban board is current** — every task has a lane, every lane has a status
+5. **User gets verified results** — raw worker output never reaches Kwame
+
+---
+
+**Last updated:** 2026-07-09
+**Author:** Mumuni (with Kwame's input on triggers and exception criteria)
+**Status:** Draft — awaiting PR review and merge to prose-contracts main

gpu-fleet.prose.md

@@ -5,13 +5,12 @@ description: >
   Manages the GPU inference fleet across all hosts. Handles model deployment,
   registration, health checks, LiteLLM sync, agent key management, GPU
   saturation watchdog, Prometheus/Grafana monitoring, and self-healing.
-  UPDATED 2026-07-12: Architecture is DIRECT GPU — LiteLLM routes directly
-  to llama-server on each GPU host (no router in inference path). Router
-  (port 9000) is running but NOT in request path. All GPUs standardized on
-  api-key 'not-needed'. RTX 5070 had api-key mismatch (sk-loc...5678) that
-  caused cascading 401→timeout→401 fallback loops — fixed.
-  Context: RTX 3090 verified at 256K (was documented as 128K — WRONG).
-  Workload: Compression moved to Strix Halo, RTX 5070 → vision/web only.
+  UPDATED 2026-07-15: Stable role-based aliases introduced: strix-moe,
+  gpu-dense, gpu-light. These never change — only the underlying model does.
+  Strix Halo: ornith-1.0-35b → unsloth/Qwen3.6-35B-A3B-MTP (UD-Q4_K_M, 22GB).
+  RTX 5070: gemma-4-12b Q4_K_M → IQ4_NL + MTP draft (122 tok/s, 2x faster).
+  RTX 5070 context: 131K → 256K. VRAM: 88% (10.8/12.2GB).
+  Compression timeout: 300s (was 120s). Mumuni context: 128K (was 256K).
 agent: abiba
 triggers:
   - on model add/remove
@@ -76,40 +75,64 @@ triggers:
 └──────────┘
 ```
 
-## Current Model Assignments (2026-07-12)
+## Stable Role-Based Aliases (Introduced 2026-07-15)
+
+Agent configs, cron jobs, and workflows MUST use these aliases, never model-specific names.
+When a model is swapped on a GPU, ONLY the infrastructure layer changes — agent configs are untouched.
+
+| Alias | GPU | Current Model | Will Route To |
+|-------|-----|---------------|---------------|
+| `strix-moe` | Strix Halo (.15) | qwen3.6-35B-udq4 | Whatever runs on Strix Halo |
+| `gpu-dense` | RTX 3090 (.8) | qwen3.6-27B-code | Whatever runs on RTX 3090 |
+| `gpu-light` | RTX 5070 (.110) | gemma-4-12b | Whatever runs on RTX 5070 |
+
+**Backward compatibility**: Old model-specific names (qwen3.6-27B-code, gemma-4-12b, qwen3.6-35B-udq4) still work
+but are deprecated for agent configs. Only the stable aliases survive model swaps.
+
+## Current Model Assignments (2026-07-15)
 
 | Model | GPU | Host | VRAM | Ctx | KV Cache | Parallel | Batch/Ubatch | Status |
 |-------|-----|------|------|-----|----------|----------|-------------|--------|
-| qwen3.6-27B-code | RTX 3090 | .8 (llm-gpu) | 20.7/24GB (84%) | **256K** | turbo4 | 1 | default | ✅ healthy |
+| qwen3.6-27B-code (MTP) | RTX 3090 | .8 (llm-gpu) | 22.2/24.6GB (90%) | **256K** 🚀 | turbo4 | 2 | default | ✅ 63 tok/s |
 | gemma-4-12b | RTX 5070 | .110 (ocu-llm) | 10.0/12.2GB (82%) | 131K | q4_0 | 2 | 2048/1024 | ✅ healthy |
-| ornith-1.0-35b | Strix Halo Vulkan | .15 (amdpve) | ~9GB/64GB | 256K | q8_0 | 2 | 2048/512 | ✅ healthy |
+| qwen3.6-35B-udq4 | Strix Halo Vulkan | .15 (amdpve) | ~9GB/64GB | 256K | q8_0 | 2 | 2048/512 | ✅ healthy |
 
 ## Routing Configuration (LiteLLM — July 2026)
 
-### syslog-auto Weighted Pool
+### syslog-auto Weighted Pool (Direct GPU — bypasses router)
 
-| Model | GPU | Weight | RPM Cap | Purpose |
+| Model | GPU | Weight | RPM Cap | Timeout |
 |-------|-----|--------|---------|---------|
-| qwen3.6-27B-code | RTX 3090 | 0.55 | 500 | Heavy reasoning, code, long context |
-| ornith-1.0-35b | Strix Halo | 0.30 | **60** | Agentic workflows, tool calling |
-| gemma-4-12b | RTX 5070 | 0.15 | 200 | Overflow + vision |
+| qwen3.6-27B-code | RTX 3090 (.8:8080) | **0.55** | 500 | 90s |
+| qwen3.6-35B-udq4 | Strix Halo (.15:8080) | **0.30** | 60 | **300s** |
+| gemma-4-12b | RTX 5070 (.110:8080) | **0.15** | 200 | **300s** |
+
+Note: All syslog-auto entries route directly to GPUs with `api_key: not-needed`. The router (port 9000) is NOT in the inference path.
 
 ### Direct Model Endpoints
 
 | Model | RPM Cap | Notes |
 |-------|---------|-------|
-| ornith-1.0-35b | 40 | Tight cap — prevents Strix overload |
+| qwen3.6-35B-udq4 | 40 | Tight cap — prevents Strix overload |
 | qwen3.6-27B-code | 500 | High cap — primary workhorse |
-| gemma-4-12b | 500 | High cap — fast 12B |
+| gemma-4-12b | 500 | High cap — IQ4_NL+MTP, 122 tok/s |
+
+### Stable Aliases (for agent configs — never change)
+
+| Alias | RPM Cap | Routes To | Purpose |
+|-------|---------|-----------|---------|
+| `strix-moe` | 40 | Strix Halo | Compression tasks (MoE models) |
+| `gpu-dense` | 500 | RTX 3090 | Heavy reasoning |
+| `gpu-light` | 500 | RTX 5070 | Vision, web extract, light tasks |
 
 ### Fallback Chains
 - gemma → qwen
 - qwen → gemma
-- ornith → qwen → gemma
-- syslog-auto → qwen → gemma → ornith
+- qwen3.6-35B-udq4 → qwen → gemma
+- syslog-auto → qwen → gemma → qwen3.6-35B-udq4
 
-### Why ornith RPM Is Capped
-- Direct: 40 RPM (tight) — Strix Halo is shared with compression tasks
+### Why Strix Halo RPM Is Capped
+- Direct (qwen3.6-35B-udq4): 40 RPM (tight) — Strix Halo is shared with compression tasks
 - Via syslog-auto: 60 RPM (moderate) — prevents flooding when multiple agents use syslog-auto simultaneously
 - Combined max: ~100 RPM across both paths — Strix Halo can sustain this at 80°C
 
@@ -203,7 +226,7 @@ If no SSH access, send Zulip DM via abiba-bot with vault update instructions.
 | `/root/scripts/gpu-saturation-watchdog.py` | pi (.24) | Auto-restart stuck llama-server |
 | `/root/dashboard/gpu-fleet.html` | pi (.24) | Live HTML dashboard |
 | `/etc/systemd/system/llama-server.service` | .8, .110 | llama-server daemons (Nvidia GPUs) |
-| `/etc/systemd/system/ornith-server.service` | .15 (amdpve) | llama-server daemon (Vulkan, Strix Halo). Note: `llama-server.service` and `llama-server@.service` are **masked** on .15 to prevent port 8080 collisions. |
+| `/etc/systemd/system/strix-server.service` | .15 (amdpve) | llama-server daemon (Vulkan, Strix Halo) running unsloth/Qwen3.6-35B-A3B-MTP-GGUF. Note: `llama-server.service` and `llama-server@.service` are **masked** on .15 to prevent port 8080 collisions. |
 
 ## Prometheus & Grafana
 
@@ -220,14 +243,14 @@ If no SSH access, send Zulip DM via abiba-bot with vault update instructions.
 - **Router startup race**: Compose router.py doesn't call load_roster(). Reload thread sleeps 30s first.
   Fix: trigger roster reload via SSH after restart, or rebuild image with startup load_roster().
 - **LiteLLM /metrics**: Requires auth. Prometheus uses `/health/liveliness` as workaround.
-- **VRAM (2026-07-12)**: RTX 3090 at 20.7/24GB (84%), RTX 5070 at 10.0/12.2GB (82%). RTX 3090 context increased to 256K (was incorrectly documented as 128K — verified via /proc/PID/cmdline).
-- **RTX 3090 runs `--parallel 1`** (verified 2026-07-12). RTX 5070 and Strix at parallel 2.
-- **RTX 3090 config**: `-c 262144 -ctk turbo4 -ctv turbo4 --parallel 1 --flash-attn on --cont-batching`. Service: `/home/llmuser/llama-wrapper.sh`.
-- **RTX 5070 config**: `--ctx-size 131072 --cache-type-k q4_0 --cache-type-v q4_0 --batch-size 2048 --ubatch-size 1024 --parallel 2`. Api-key standardized to `not-needed` (was `sk-loc...5678` causing 401 loops). Service: `/home/llmuser/llama-wrapper.sh`.
+- **VRAM (2026-07-15)**: RTX 3090 at 22.2/24.6GB (90%) with **256K context** (corrected from 131K). RTX 5070 at 10.8/12.2GB (88%) with 256K context + MTP. Strix Halo at ~9GB/64GB.
+- **RTX 3090 runs `--parallel 2`** with MTP draft (spec-type draft-mtp, spec-draft-n-max 2).
+- **RTX 3090 config**: `-c 262144 -ctk turbo4 -ctv turbo4 --parallel 2 --flash-attn on --cont-batching --spec-type draft-mtp`. Context corrected to 256K (2026-07-15). VRAM: 90%. Service: `/home/llmuser/llama-wrapper.sh`.
+- **RTX 5070 config (2026-07-15)**: Switched to IQ4_NL + MTP draft (Q8_0) at 256K context. Gen speed: 122 tok/s (was 70). VRAM: 10.8/12.2GB (88%). No draft model pre-upgrade due to VRAM constraints. Service: `/home/llmuser/llama-wrapper.sh`. Config: `--model gemma-4-12b-it-IQ4_NL.gguf --spec-draft-model gemma-4-12b-it-Q8_0-MTP.gguf --spec-type draft-mtp --spec-draft-n-max 4 --ctx-size 262144`.
 - **LiteLLM timeout tuning (2026-07-08)**: gemma-4-12b 25→120s, qwen3.6-27B-code 40→90s, syslog-auto (qwen route) 40→90s. Nginx proxy_read_timeout: 600s. Global request_timeout: 300s. Config at `/opt/inference-harness/litellm_config.yaml`.
-- **Strix Halo GPU**: Vulkan is the working backend (ROCm/HIP path abandoned — HSA runtime blocked on Debian 13). Build at `/root/llama.cpp/build-vk/`, commit `4fc4ec5` (2026-07-01), ggml 0.15.3 shared-lib arch. Mesa RADV 25.0.7, KHR_coopmat fast path active. ~70 tok/s gen, 532 tok/s prompt. Service: `ornith-server.service` on port 8080, 256K context, flash-attn + q8 KV.
+- **Strix Halo GPU**: Vulkan is the working backend (ROCm/HIP path abandoned — HSA runtime blocked on Debian 13). Build at `/root/llama.cpp/build-vk/`, commit `4fc4ec5` (2026-07-01), ggml 0.15.3 shared-lib arch. Mesa RADV 25.0.7, KHR_coopmat fast path active. ~70 tok/s gen, 532 tok/s prompt. Service: `strix-server.service` on port 8080 (was `ornith-server.service`), model changed to `unsloth/Qwen3.6-35B-A3B-MTP-GGUF` (UD-Q4_K_M), alias `qwen3.6-35B-udq4`, 256K context, flash-attn + q8 KV. MTP support enabled for 1.4-2.2x faster inference.
 - **Port conflict detection (2026-07-05)**: All 3 GPU wrappers now detect ghost processes squatting port 8080 before starting. `.8` and `.110` use inline pre-start check in `llama-wrapper.sh`; `.15` uses `/usr/local/bin/port-cleanup.sh` ExecStartPre. Replaces the blanket `pkill -9 -x llama-server` on .15 which would kill ALL llama-server instances regardless of port. Ghost detection was the root cause of .8 crash-looping for 27+ restarts (stale pid 25836 squatting 8080 after OOM kill).
-- **Strix Halo thermal safeguard (2026-07-02)**: `ornith-server.service` has `-n 8192` (hard generation cap per request). Without it, `--predict` defaults to -1 (infinity) — a runaway request from .123 (Mumuni) decoded 39,868 tokens over 24 min, pushing Tctl to 98°C (crit 89.8°C) and throttling 70→29 t/s. The cap bounds worst-case generation to ~5 min. Do NOT remove `-n` without a replacement ceiling. Sustained load hits ~84°C even at 92s; the APU is fanless/low-flow. Clients MUST also set `max_tokens`.
+- **Strix Halo thermal safeguard (2026-07-02)**: `strix-server.service` has `-n 8192` (hard generation cap per request). Without it, `--predict` defaults to -1 (infinity) — a runaway request from .123 (Mumuni) decoded 39,868 tokens over 24 min, pushing Tctl to 98°C (crit 89.8°C) and throttling 70→29 t/s. The cap bounds worst-case generation to ~5 min. Do NOT remove `-n` without a replacement ceiling. Sustained load hits ~84°C even at 92s; the APU is fanless/low-flow. Clients MUST also set `max_tokens`.
 - **Port 8080 firewall**: amdpve iptables restricts 8080 to 192.168.68.116 (LiteLLM/router host) only. All inbound connections are from .116 (LiteLLM proxied via nginx). Localhost curls hang (SYN dropped). Always test from .116.
 - **Router sidecar fallback**: `router.py` `check_gpu_health()` now probes GPU `/health` directly when sidecar at :8090 is absent. Sidecar JSON exporters not deployed on any GPU host — router relies on GPU-direct fallback.
 - **Router GPU_MOE_URL bug (fixed 2026-07-01)**: docker-compose had `GPU_MOE_URL=.110:8080` (gemma host) instead of `.15:8080` (amdpve). Corrected.
@@ -237,11 +260,14 @@ If no SSH access, send Zulip DM via abiba-bot with vault update instructions.
 
 ## GPU Inference Benchmarks (Current)
 
-| GPU | Model | Gen tok/s | Prompt tok/s | Baseline | Samples |
+| GPU | Model | Gen tok/s | Prompt tok/s | Baseline | Context |
 |-----|-------|-----------|--------------|----------|---------|
-| RTX 3090 (.8) | qwen3.6-27B-code | 75 | 305 | 74 | 6 |
-| RTX 5070 (.110) | gemma-4-12b | 75 | 323 | 75 | 6 |
-| Strix Halo (.15) | ornith-1.0-35b | 70 | 532 | 70 | 6 |
+| RTX 3090 (.8) | qwen3.6-27B-code (MTP) | **63** | — | — | 131K |
+| RTX 5070 (.110) | gemma-4-12b (IQ4_NL+MTP) | **191** | — | — | **256K** |
+| Strix Halo (.15) | qwen3.6-35B-udq4 | **71** | — | — | **256K** |
+
+Benchmarks from 2026-07-15 verification run. RTX 5070 MTP provides 2.7x speedup over pre-upgrade 70 tok/s.
+All 3 GPUs now at 256K context (2026-07-15).
 
 Benchmarks run through LiteLLM proxy (192.168.68.116:4001) every 5 minutes.
 Degradation alerts fire at 30% (warning) and 50% (critical) below baseline.
@@ -249,12 +275,55 @@ History stored at `/root/data/toks-history.json` with 7-day rolling window.
 
 **Note (2026-07-01)**: Strix Halo prompt tok/s jumped 209→532 after Vulkan rebuild (cooperative-matrix fast path now active on GFX1151). Baseline may need re-calibration.
 
-## Agent Config Implications (2026-07-12)
+## Agent Config Implications (2026-07-15)
 
-With RTX 3090 at 256K context (verified July 2026):
-- Agents using `syslog-auto` (55/30/15 qwen+ornith+gemma): `context_length: 262144` — ornith and qwen both support it
-- Agents using `qwen3.6-27B-code` directly: `context_length: 262144` (256K ctx verified)
-- Agents using `gemma-4-12b` directly (auxiliary tasks): `context_length: 131072`
-- Compression threshold at 0.65: fires at ~170K for 262K context window on Strix Halo
-- All Hermes clients MUST set `max_tokens: 4096` — first line of defense before server-side `-n 8192` cap
-- Port 8080 is used on all 3 GPU hosts (not 8090 as previously documented)
+### Stable Aliases — CRITICAL
+
+All agent configs MUST use stable role-based aliases, never model-specific names:
+- `compression.model: strix-moe` (NOT `qwen3.6-35B-udq4`)
+- `auxiliary.vision.model: gpu-light` (NOT `gemma-4-12b`)
+- `delegation.model: gpu-dense` (NOT `qwen3.6-27B-code`)
+- `auxiliary.web_extract.model: gpu-light`
+
+When the underlying model is swapped, only the LiteLLM config changes — agent configs are untouched.
+
+### Context Windows
+- RTX 3090: **256K** (was 131K, bumped 2026-07-15) | RTX 5070: **256K** (up from 131K) | Strix Halo: **256K**
+- **Mumuni compression context**: 128K (down from 256K) — ensures compression model doesn't timeout
+- Compression threshold 0.65: fires at ~85K for 128K context window
+- Mumuni compression model alias: `strix-moe` with 300s timeout
+
+### Mumuni Agent Profile
+
+Mumuni (CT114, 192.168.68.123) is the primary business assistant. This profile is the reference for all agent configs:
+
+| Setting | Value | Notes |
+|---------|-------|-------|
+| `model.default` | `syslog-auto` | Weighted pool (55% qwen, 30% strix, 15% gemma) |
+| `model.provider` | `custom:litellm` | LiteLLM on CT116 |
+| `compression.model` | `strix-moe` | Stable alias — survives model swaps |
+| `aux.compression.model` | `strix-moe` | Compression auxiliary model |
+| `aux.vision.model` | `gpu-light` | Vision tasks (RTX 5070) |
+| `aux.web_extract.model` | `gpu-light` | Web extraction |
+| `delegation.model` | `gpu-dense` | Sub-agent reasoning (RTX 3090) |
+| `context.max_context_window` | 131072 (128K) | Kept at 128K to avoid compression timeout |
+| `compression.threshold` | 0.65 | Triggers at ~85K |
+| `compression.target_ratio` | 0.3 | Compresses to ~38K |
+| `compression.protect_last_n` | 40 | Preserves last 40 messages |
+| `memory.memory_char_limit` | 800 | Brief memory entries |
+| `personalities` | `creative` | Creative assistant personality |
+| Platforms | cli, discord, homeassistant, signal, telegram, zulip | All Hermes platforms |
+| Main model timeout | 300s | LiteLLM global timeout |
+| Compression model timeout | 300s | ornith timeout increased from 120s |
+
+### Agent Update Status (2026-07-15)
+
+| Agent | Host | Status |
+|-------|------|--------|
+| **Mumuni** | CT114 (.123) | ✅ Updated to stable aliases |
+| **Tanko** | CT112 (.122) | ✅ Updated to stable aliases |
+| **Koby** | CT111 (.129) | ❌ SSH unreachable — needs Zulip DM |
+| **Koonimo** | CT113 | ❌ SSH unreachable — needs Zulip DM |
+| **Kagenz0** | CT105 | ❌ SSH unreachable — needs Zulip DM |
+
+All Hermes clients MUST set `max_tokens: 4096` — first line of defense before server-side `-n 8192` cap.

mumuni-delegation-prose-contract.prose.md

@@ -0,0 +1,290 @@
+---
+kind: pattern
+name: mumuni-delegation
+description: >
+  Mumuni-specific operating doctrine for task decomposition, worker
+  delegation, verification, and delivery. Defines when to delegate, which
+  worker to use for what, how to handle failures, and the kanban board
+  protocol. Enforces context-window discipline and separation of concerns.
+  Runs on Mumuni (CT 118, storepve, .6) via Hermes agent.
+version: 1.0.0
+---
+
+## Maintains
+
+- Worker roster: 6 profiles (`syslog-code`, `syslog-devops`, `syslog-email`,
+  `syslog-research`, `syslog-review`, `syslog-writer`)
+- Kanban board state at `~/.hermes/kanban/kanban.json`
+- Context window budget: ~65K tokens per request (131K total, 60% threshold)
+
+## Topology
+
+**Cluster:** 5 Proxmox nodes (ocupve, acerpve, minipve, amdpve, storepve)
+**Manager:** Mumuni (CT 118, storepve, .6) via Hermes agent
+**Workers:** 6 profiles, all running on the same agent — no separate hosts needed
+
+This contract is infrastructure-agnostic in terms of which nodes are used.
+Workers execute tasks on whatever infrastructure they're given — SSH to .6,
+.pm, .9, .12, or .15 depending on the task. The contract defines the
+**who** and **when** — not the **where**.
+
+## Why This Matters
+
+Without enforced delegation, the manager consumes the full iteration budget
+(60 calls) on single-turn tasks — SSH to 5 nodes, check each VM, read logs —
+leaving no capacity for actual coordination. The result: context overflow
+(59K tokens in system prompt), iteration exhaustion, and degraded response
+quality. This contract exists because I blew through my budget checking
+Proxmox node status instead of delegating to `syslog-devops`.
+
+## Context Window Discipline
+
+**The system prompt is ~6.5K tokens (stable: ~4.5K tool schemas + ~2K other guidance).**
+**Volatile (MEMORY.md + USER.md): ~300 tokens.**
+**Total base: ~6,800 tokens per request.**
+
+The remaining budget is the conversation. Every tool call result adds to it.
+If a single call returns >10K tokens (e.g., `grep` on a large file, SSH output
+from multiple nodes), the context fills fast. That's why we delegate: workers
+process in isolation and return compact results.
+
+## Trigger Conditions
+
+Delegation is **mandatory** when any of these apply:
+
+| Condition | Threshold | Example |
+|-----------|-----------|---------|
+| Multiple tool calls needed | 2+ calls with intermediate logic | Read file → analyze → write report |
+| Large data retrieval | Output >5K tokens | `grep -r "pattern" /path` on large dirs |
+| Cross-domain work | Spans 2+ worker specialties | Infra check + email filter |
+| Infrastructure changes | Any mutating operation | `qm set`, `systemctl restart`, `git push` |
+| Research/analysis | Needs browser or deep reading | Web research, code review, data analysis |
+| Code builds or changes | Writing or modifying code | Scripts, configs, patches |
+| Sequential dependencies | Worker B needs Worker A's output | Code → Review → Deliver |
+
+**Single tool calls stay at manager level.** Quick `grep`, `ls`, `cat`,
+`curl`, `hermes tools list` — these are decision-making tools. The manager
+reads them directly.
+
+## Data Source Integrity (CRITICAL)
+
+**Workers MUST use the data provided in their task context. They MUST NOT
+fetch their own data from external sources unless explicitly told to.**
+
+When a task says "Read file X and format it", the worker reads file X. It does
+not query a separate API, run its own diagnostics, or pull data from a different
+system. This is the #1 source of cross-worker inconsistency: one worker gathers
+SSH data, another queries the Proxmox API, and the report merges two incompatible
+datasets.
+
+**Rule:** If a worker needs additional data beyond what's in its task description,
+it asks the manager (via relay) — it doesn't go find it on its own.
+
+**This is a hard rule, not a recommendation.** Violating it produces the exact
+type of discrepancy the kanban pipeline exists to prevent: a review worker finds
+"5 nodes present" in the raw data but "5/5 online" in the report — even though
+one of those nodes was unreachable. The report lied because it used data the
+raw data never provided.
+
+## Worker Selection Matrix
+
+| Worker | Model | Toolsets | Role | Use When |
+|--------|-------|----------|------|----------|
+| `syslog-code` | qwen3.6-27B-code | terminal, file, web, memory, skills | Code patches, automation, scripts | Writing/modifying code, creating scripts, debugging, reading/writing files |
+| `syslog-devops` | qwen3.6-27B-code | terminal, file, web, memory, skills | Infrastructure, DB, bridge, Proxmox | Server ops, SSH, Docker, Proxmox, DB queries, hardware checks |
+| `syslog-email` | ornith-1.0-35b | terminal, file, web, memory, skills | Email automation, mail operations | Sending/receiving email, inbox management, SMTP operations |
+| `syslog-research` | ornith-1.0-35b | terminal, file, web, memory, skills, **browser** | Analysis, classification, data processing | Web research, browser tasks, data analysis, classification, reading docs |
+| `syslog-review` | ornith-1.0-35b | terminal, file, web, memory, skills | Verification, QA, audit validation | **ALWAYS** verify worker output before delivery — especially for infra changes, code builds, and research findings |
+| `syslog-writer` | ornith-1.0-35b | terminal, file, web, memory, skills | Docs, content, branding, reports | Writing docs, reports, proposals, content, markdown formatting |
+
+### Selection Rules
+
+1. **Match specialty first.** A code task → `syslog-code`. An infra task →
+   `syslog-devops`. Don't put a `syslog-email` worker on a code review.
+2. **Research tasks with browser needs → `syslog-research`.** Other workers
+   don't have the browser toolset.
+3. **Verification → `syslog-review`.** Never deliver raw worker output.
+4. **Documentation/content → `syslog-writer`.** Let them own the prose.
+5. **If unsure, delegate to `syslog-research`** — it has the broadest toolset
+   (includes browser) and high reasoning effort.
+
+## Delegation Protocol
+
+### Step 1: Decompose
+
+Break the task into lanes. Each lane does ONE thing. Workers are independent —
+no lane depends on another's output mid-flight. If lanes depend on each other,
+dispatch sequentially.
+
+### Step 2: Dispatch
+
+Fire workers via `delegate_task`:
+
+**Critical: Pass the data, not just the goal.** When dispatching a worker that
+processes output from another worker, include the file path AND explicit
+instructions to use ONLY that source. Example:
+
+```
+delegate_task(
+    goal="Format the cluster check into a clean report",
+    context="Source data is at /tmp/proxmox-check-raw.md. Format ONLY the data
+    in that file. Do NOT query the Proxmox API or any other data source. Use the
+    file as your sole source of truth."
+)
+```
+
+**Parallel (independent lanes):**
+```
+delegate_task(
+    tasks=[
+        {"goal": "Check all 5 Proxmox nodes for VM status", "context": "SSH to each node via 192.168.68.x, run 'qm list'"},
+        {"goal": "Check Docker container health on .7/.116/.17", "context": "SSH to each host, check container status"},
+    ]
+)
+```
+
+**Sequential (dependent lanes):**
+Dispatch lane 1 → wait for result → dispatch lane 2.
+
+### Step 3: Verify
+
+**MANDATORY for:**
+- Infrastructure changes (any `qm`, `pct`, `systemctl`, `git push`)
+- Code builds and modifications
+- Research findings (web data, external sources)
+- Any output that will reach the user
+
+**Fire `syslog-review` to verify:**
+```
+delegate_task(
+    goal="Review the output of the devops worker. Verify the node status
+    report is accurate, check for inconsistencies, confirm all nodes were
+    reachable.",
+    context="Worker was syslog-devops. Output is at /tmp/node-report.md.
+    Verify against live system."
+)
+```
+
+**If verification fails:**
+1. Send work back to original worker with review feedback
+2. Re-verify
+3. Max 2 re-verify cycles before escalating to Kwame
+
+### Step 4: Deliver
+
+Only verified results reach Kwame. Format per channel:
+- Telegram: Use `telegram-formatting` skill
+- Zulip: Use Zulip Markdown (CommonMark)
+- Email: Use `syslog-email` skill
+
+## Kanban Board Protocol
+
+**File:** `~/.hermes/kanban/kanban.json`
+
+```json
+{
+  "task_id": "unique-id",
+  "title": "Task description",
+  "created": "2026-07-09T01:00:00",
+  "status": "backlog|in_progress|review|done",
+  "lanes": [
+    {
+      "lane_id": "devops-check",
+      "worker": "syslog-devops",
+      "goal": "Check all 5 Proxmox nodes",
+      "status": "dispatched|completed|failed",
+      "output_file": "/tmp/node-report.md"
+    }
+  ]
+}
+```
+
+**Update the board on every state change.**
+
+## Failure Handling
+
+### Worker Timeouts
+
+- Child timeout: **900 seconds** (15 minutes)
+- Worker model `syslog-auto` is slow — it can hit the timeout limit with
+  22+ API calls
+- **If a worker times out:** Re-dispatch with a narrower scope. Break the
+  task into smaller pieces that fit in the timeout window.
+- **Avoid delegating sequential SSH hops** — each SSH connection adds latency
+  that compounds quickly. Prefer API-based or local approaches when possible.
+
+### Worker Selection Failures
+
+- `syslog-devops` is best for infrastructure tasks (SSH, Proxmox, Docker)
+- `syslog-code` is best for code-level work (reading files, writing scripts)
+- `syslog-research` has the browser toolset — use for web research
+- `syslog-review` is the QA gate — always fire before delivery
+- **Never fire more than 3 parallel workers** (max_concurrent_children: 3)
+- **Never nest delegation** (max_spawn_depth: 1)
+
+### Context Overflow
+
+- If a task requires >10K tokens of output, delegate the processing
+- Workers return compact summaries, not raw data dumps
+- Pass file paths and concrete goals — never dump raw data into context
+
+## Anti-patterns
+
+- ❌ Reading large files into your own context before deciding → delegate the read
+- ❌ Carrying SSH/grep/output results in your context → delegate the analysis
+- ❌ Doing work yourself and then "pretending" to delegate → the user can tell
+- ❌ Skipping verification → raw worker output never reaches the user
+- ❌ Delegating single tool calls → keep quick reads/writes at manager level
+- ❌ Firing more than 3 workers in parallel → hard limit
+- ❌ **Workers fetching their own data sources** → a writer worker that queries the Proxmox API when told to "format the raw file" is fabricating data. Use the input given, not external sources
+
+## Emergency Exception
+
+**In an emergency (server down, service must be restored immediately):**
+- Delegate the diagnosis (find the problem)
+- Execute the fix yourself (minimize handoff latency)
+- Verify the fix after delivery
+- Log the exception in the kanban board
+
+The emergency exception exists because the user needs the service back NOW,
+not after three worker round-trips. But it's an exception — not the rule.
+
+## What This Contract Doesn't Cover
+
+1. **Worker profile configuration** — covered by `hermes-config-template.prose.md`
+2. **SSH key management** — covered by existing SSH/Proxmox contracts
+3. **Git workflow** — covered by `AGENTS.md` in the prose-contracts repo
+4. **Cron job management** — covered by individual cron contracts
+5. **Infra verification** — covered by `verify-before-mutate` protocol
+
+## Verification
+
+Run `scripts/worker-audit.py` to verify all 6 profiles are aligned:
+
+```bash
+python3 /root/.hermes/skills/kanban-orchestrator/scripts/worker-audit.py
+```
+
+## References
+
+- `kanban-orchestrator` skill: The operational playbook (detailed execution steps)
+- `worker-profile-audit.md` (skill reference): Worker configuration audit notes
+- `delegation-timeout-patterns.md` (skill reference): Timeout handling patterns
+- `verify-before-mutate` protocol: Infrastructure change verification
+- `hermes-config-template.prose.md`: Worker profile configuration
+
+## Success Criteria
+
+This contract succeeds when:
+
+1. **No context overflow** — single-turn tasks don't exhaust the iteration budget
+2. **Workers do the work** — manager coordinates, doesn't execute
+3. **Verification before delivery** — all output passes through `syslog-review`
+4. **Kanban board is current** — every task has a lane, every lane has a status
+5. **User gets verified results** — raw worker output never reaches Kwame
+
+---
+
+**Last updated:** 2026-07-09
+**Author:** Mumuni (with Kwame's input on triggers and exception criteria)
+**Status:** Draft — awaiting PR review and merge to prose-contracts main

ra-h-os-custodianship-contract.prose.md

@@ -0,0 +1,210 @@
+---
+kind: pattern
+name: ra-h-os-custodianship-contract
+description: >
+  Operational standards for maintaining RA-H OS as a high-functioning shared
+  memory system across all agents. Defines rules for embedding consistency,
+  namespace discipline, staleness management, orphan prevention, agent
+  custodianship, and recovery protocols. Prevents knowledge graph degradation
+  and ensures reliable semantic search capabilities.
+---
+
+# RA-H OS Custodianship Contract — Shared Memory Protocol
+
+## Purpose
+This contract establishes the operational standards for maintaining RA-H OS as a high-functioning shared memory system across all agents. It defines the rules, responsibilities, and recovery protocols that prevent knowledge graph degradation and ensure consistent, reliable semantic search capabilities.
+
+## Core Principles
+
+### 1. Embedding Consistency
+**Rule:** All nodes MUST use the RA-H OS built-in embedding pipeline via `http://192.168.68.65:8080/v1/embeddings` for consistent vector representation.
+
+**Requirements:**
+- The embedding service at `192.168.68.65:8080` is the single source of truth for vector generation
+- No external embedding APIs (OpenAI, Cohere, etc.) may be used for RA-H OS nodes
+- All nodes must have `embedding_status: "chunked"` and `chunk_status: "chunked"` upon creation
+- If embedding fails, the node must be immediately flagged as `state: "unsearchable"` and logged
+
+### 2. Namespace Discipline
+**Rule:** All nodes MUST be categorized into one of three namespaces with strict metadata requirements.
+
+**Namespace Structure:**
+- `agent-private`: Isolated working notes for individual agents (Mumuni, Tanko, Okyeame, etc.)
+- `shared`: Policy files, registry nodes, collective knowledge
+- `syslogsolution`: Business nodes, client data, operational context
+
+**Metadata Requirements:**
+Every node MUST have these metadata fields:
+```json
+{
+  "agent_id": "<agent_name>",
+  "namespace": "<namespace>",
+  "state": "<state>",
+  "type": "<type>",
+  "owner": "<owner>",
+  "tenant": "<tenant>",
+  "visibility": "<visibility>",
+  "source": "<source>",
+  "captured_by": "<captured_by>"
+}
+```
+
+### 3. Staleness Management
+**Rule:** Nodes are categorized by type and have specific staleness thresholds.
+
+**Staleness Thresholds:**
+- `policy` type: 30 days (e.g., Shared Memory Policy, Agent Registry)
+- `registry` type: 30 days (e.g., Node State Management, Health Dashboard)
+- `template` type: 30 days (e.g., Agent SOUL.md)
+- `note` type: 60 days (e.g., business context, project notes)
+- `information` type: 90 days (e.g., reference documentation)
+- `idea` type: 30 days (e.g., brainstorming, experimental notes)
+
+**Transition Protocol:**
+- `active` → `stale`: After days since `updated_at` exceeds threshold
+- `stale` → `review_pending`: Daily cron check flags for human review
+- `review_pending` → `active`: Human reviews and updates content
+- `review_pending` → `archived`: After 15 days in review_pending state (automatic purge)
+
+### 4. Orphan Prevention
+**Rule:** No node should remain orphaned (0 edges) for more than 7 days.
+
+**Orphan Detection:**
+- Daily cron job identifies nodes with 0 incoming/outgoing edges
+- Orphans are flagged with `state: "review_pending"` and `metadata.orphan_detected: true`
+- 7-day grace period: Orphans must be either:
+  - Connected to relevant nodes via edges
+  - Merged into existing parent nodes
+  - Archived after 15 days in review_pending state
+
+### 5. Agent Custodianship
+**Rule:** Each agent is responsible for maintaining their own created nodes.
+
+**Responsibilities:**
+- **Mumuni**: Primary custodian for `syslogsolution` namespace, health dashboard, and all business nodes
+- **Tanko**: Custodian for `agent-private` namespace, personal notes, and fitness/creative work
+- **Okyeame**: Custodian for `shared` namespace, policy files, and collective knowledge nodes
+- **All Agents**: Must verify embedding status before reporting node creation as complete
+
+### 6. Embedding Failure Recovery
+**Rule:** When embedding pipeline is unavailable, nodes are blocked from creation or immediately flagged.
+
+**Recovery Protocol:**
+1. **Detection**: Daily health check monitors `http://192.168.68.65:8080/health` endpoint
+2. **Alert**: If embedding service is down, alert is sent to all agents via relay
+3. **Mitigation**: 
+   - New node creation is suspended
+   - Existing nodes with `chunk_status: "not_chunked"` are flagged as `unsearchable`
+4. **Recovery**: When service returns:
+   - All `not_chunked` nodes are queued for re-embedding
+   - Re-embedding status logged in `embedding_retry_log` table
+   - 5-minute retry window with exponential backoff (1s, 2s, 4s, 8s, 16s)
+
+### 7. Chunking Verification
+**Rule:** All node modifications must verify chunking status within 5 minutes.
+
+**Verification Protocol:**
+- After `updateNode` or `createNode`, query `chunks` table for `node_id`
+- If `chunk_status != "chunked"` after 5 minutes, log failure and flag node
+- Automated cleanup script runs every 6 hours to retry failed chunks
+- Maximum 3 retry attempts per node before escalation to human
+
+### 8. Graph Health Monitoring
+**Rule:** Health dashboard (Node #74) must reflect real-time graph status.
+
+**Dashboard Requirements:**
+- Total nodes, edges, orphans, stale nodes, chunk errors
+- Embedding service health status
+- Last successful embedding attempt
+- Alert thresholds:
+  - Orphan rate > 40%: Warning
+  - Stale node rate > 50%: Critical
+  - Chunk error rate > 30%: Critical
+  - Embedding service down: Critical (immediate alert)
+
+## Operational Procedures
+
+### Daily Maintenance (Automated)
+1. **Staleness Check**: Identify nodes past their threshold
+2. **Orphan Detection**: Flag nodes with 0 edges
+3. **Chunk Verification**: Retry failed chunks, log errors
+4. **Health Update**: Refresh Node #74 dashboard
+
+### Weekly Maintenance (Human Review)
+1. **Review Pending Nodes**: Examine flagged nodes
+2. **Edge Optimization**: Connect related orphaned nodes
+3. **Archive Cleanup**: Remove nodes past 15-day review period
+4. **Embedding Audit**: Verify all nodes have valid embeddings
+
+### Emergency Recovery
+1. **Embedding Service Down**: 
+   - Suspend node creation
+   - Alert all agents
+   - Investigate root cause (check `192.168.68.65:8080`)
+2. **Database Corruption**:
+   - Restore from latest PBS backup
+   - Verify chunk integrity
+   - Re-run embedding for affected nodes
+3. **Mass Orphan Creation**:
+   - Identify source agent/namespace
+   - Review recent changes
+   - Reconnect or archive affected nodes
+
+## Compliance & Enforcement
+
+### Audit Schedule
+- **Daily**: Automated health checks
+- **Weekly**: Human review of review_pending nodes
+- **Monthly**: Comprehensive graph audit (full schema validation)
+- **Quarterly**: Contract review and threshold adjustment
+
+### Violation Consequences
+- **First**: Alert to responsible agent
+- **Second**: Node placed in `review_pending` state
+- **Third**: Agent access suspended until remediation
+- **Fourth**: Escalation to human (Kwame)
+
+### Metrics for Success
+- Orphan rate: < 10%
+- Stale node rate: < 20%
+- Chunk error rate: < 5%
+- Embedding service uptime: > 99.9%
+- Node creation verification: 100%
+
+## Implementation Notes
+
+### Database Schema Requirements
+- `nodes` table: Standard fields plus `embedding_status`, `chunk_status`, `last_embedding_attempt`
+- `chunks` table: Must track `chunk_status` and `embedding_status`
+- `embedding_retry_log` table: Track retry attempts for failed chunks
+- `node_state_history` table: Log state transitions for audit trail
+
+### Cron Job Requirements
+- `ra-h-health-check`: Daily (4h interval)
+- `ra-h-staleness-detection`: Daily
+- `ra-h-orphan-detection`: Daily
+- `ra-h-chunk-retry`: Every 6 hours
+- `ra-h-review-cleanup`: Weekly (15-day archive)
+
+### Agent Onboarding
+All new agents must:
+1. Read this contract
+2. Understand namespace responsibilities
+3. Know how to verify embedding status
+4. Know the alert escalation path
+
+---
+
+## Approval
+This contract is effective immediately upon approval by the primary custodian (Mumuni) and system owner (Kwame).
+
+**Effective Date:** July 10, 2026  
+**Review Date:** October 10, 2026  
+**Primary Custodian:** Mumuni 🦅  
+**System Owner:** Jerome Tabiri
+
+---
+
+*Contract Version: 1.0*  
+*Last Updated: July 10, 2026*  
+*Storage: `/root/.hermes/skills/ra-h-os-custodianship-contract.prose.md`*

zulip-resilience-v3.prose.md

@@ -0,0 +1,411 @@
+---
+kind: responsibility
+name: zulip-resilience-v3
+description: >
+  Rewrite the pi Zulip gateway with production-grade resilience patterns drawn from
+  Zulip's own event system docs (queue lifecycle, heartbeat monitoring, BAD_EVENT_QUEUE_ID
+  handling, idle_queue_timeout) and battle-tested Node.js resilience patterns
+  (circuit breaker, exponential backoff with jitter, bulkhead isolation, supervisor watchdog).
+replaces: zulip-self-heal (retired)
+agent: abiba
+triggers:
+  - "/zulip self-heal v3"
+  - "zulip stopped responding"
+  - "PM2 abiba-zulip crashed"
+---
+
+# Zulip Gateway v3 — Production Resilience
+
+## Architecture Overview
+
+The current v2 gateway (`/root/.pi/agent/extensions/zulip/index.js`) has three structural
+weaknesses that cause repeated deaths:
+
+1. **No crash recovery** — uncaught errors kill the Node process, PM2 exhausts max_restarts
+2. **No circuit breaker** — 502/fetch-failed errors escalate to process death with no fallback
+3. **No queue lifecycle management** — doesn't use Zulip's documented heartbeat protocol or
+   idle_queue_timeout, so BAD_EVENT_QUEUE_ID errors cascade into crashes
+
+The v3 rewrite addresses all three, following patterns from:
+
+- [Zulip Events System docs](https://zulip.readthedocs.io/en/11.6/subsystems/events-system.html) —
+  queue registration, heartbeat, BAD_EVENT_QUEUE_ID recovery, call_on_each_event loop
+- [Zulip API: Get Events](https://zulip.com/api/get-events) — long-poll timeout, dont_block, event ack
+- [Circuit Breaker & Retry Patterns in Node.js 2026](https://1xapi.com/blog/resilient-api-circuit-breaker-bulkhead-retry-nodejs-2026) —
+  Opossum-based circuit breaker with fallback, retry with jitter, bulkhead isolation
+
+---
+
+## Maintains
+
+- `zulip-gateway`: { status: "healthy" | "degraded" | "down" }
+- `circuit-breaker`: { state: "CLOSED" | "OPEN" | "HALF_OPEN", failures, successes }
+- `queue-lifecycle`: { queue_id, last_event_id, idle_timeout, heartbeat_age }
+- `workers`: { count, busy, idle, stuck }
+- `supervisor`: { pid, last_check, health_failures }
+
+---
+
+## Detection Rules
+
+### Rule 1: Queue Expired (BAD_EVENT_QUEUE_ID)
+- **Detect**: Events API returns error with BAD_EVENT_QUEUE_ID in body
+- **Fix**: Call `POST /register` to create new queue, update queue_id and last_event_id
+- **Debounce**: If 3 re-registrations fail within 60s, escalate (server may be down)
+- **Ref**: Zulip docs: "Your software will need to handle that error condition by re-initializing itself"
+
+### Rule 2: Network Degradation (502/ECONNREFUSED/fetch failed)
+- **Detect**: Events API returns 502 or network error
+- **Circuit breaker**: Track failure rate over 10s rolling window
+  - CLOSED → OPEN: 50% failure rate with ≥5 requests
+  - OPEN → HALF_OPEN: After 30s reset timeout
+  - HALF_OPEN → CLOSED: Probe succeeds
+  - HALF_OPEN → OPEN: Probe fails
+- **While OPEN**: Log errors, skip events, notify user via DM: "⚠️ Zulip connection degraded — will retry in 30s"
+
+### Rule 3: Long-Poll Timeout (natural)
+- **Detect**: Events API response takes > `event_queue_longpoll_timeout_seconds`
+- **Not an error**: Server sends heartbeat events when no real events. Simply re-poll.
+
+### Rule 4: Worker Busy Timeout (>5 min)
+- **Detect**: Worker `busySince` exceeds 5 minutes
+- **Fix**: SIGKILL worker, send error DM, clean up pending replies
+
+### Rule 5: Process Crash (uncaught)
+- **Detect**: `uncaughtException` / `unhandledRejection` fires
+- **Fix**: Log → clear poll timer → attempt reconnect with backoff → if reconnect fails 3x, exit(1) and let PM2 restart
+
+### Rule 6: Supervisor Detects Router Stall
+- **Detect**: External supervisor (`zulip-watchdog`) polls `/health` every 30s. If 3 consecutive failures:
+- **Fix**: `pm2 restart abiba-zulip` gracefully (SIGTERM, drain workers, restart)
+
+---
+
+## Implementation Plan
+
+### Phase 1: Rewrite Router Core (circuit-breaker + queue lifecycle)
+
+Replace the poll loop in index.js with a resilience-first event loop:
+
+```js
+// Queue lifecycle (Zulip docs pattern)
+async function createOrRefreshQueue() {
+  // POST /register with event_types=["message"]
+  // Store: queueId, lastEventId, eventQueueLongpollTimeoutSeconds
+  // NEW: pass idle_queue_timeout parameter (Zulip 12.0+)
+}
+
+// Circuit breaker (Opossum pattern, implemented inline to avoid dependency)
+class ZulipCircuitBreaker {
+  constructor({ failureThreshold=0.5, resetTimeout=30000, volumeThreshold=5, windowMs=10000 }) {
+    this.state = "CLOSED"; // CLOSED | OPEN | HALF_OPEN
+    this.failures = 0;
+    this.successes = 0;
+    this.totalRequests = 0;
+    this.lastFailureTime = null;
+    this.openedAt = null;
+    this.failureThreshold = failureThreshold;
+    this.resetTimeout = resetTimeout;
+    this.volumeThreshold = volumeThreshold;
+    this.windowMs = windowMs;
+  }
+
+  async fire(fn) {
+    if (this.state === "OPEN") {
+      if (Date.now() - this.openedAt > this.resetTimeout) {
+        this.state = "HALF_OPEN";
+      } else {
+        throw new CircuitOpenError("Circuit is OPEN");
+      }
+    }
+    try {
+      const result = await fn();
+      this.onSuccess();
+      return result;
+    } catch (err) {
+      this.onFailure();
+      throw err;
+    }
+  }
+
+  onSuccess() {
+    this.successes++;
+    this.totalRequests++;
+    if (this.state === "HALF_OPEN") {
+      this.state = "CLOSED";
+      this.failures = 0;
+    }
+    // Reset counters periodically
+    if (this.totalRequests > this.volumeThreshold * 2) {
+      this.failures = Math.floor(this.failures / 2);
+      this.successes = Math.floor(this.successes / 2);
+      this.totalRequests = Math.floor(this.totalRequests / 2);
+    }
+  }
+
+  onFailure() {
+    this.failures++;
+    this.totalRequests++;
+    this.lastFailureTime = Date.now();
+    if (this.totalRequests >= this.volumeThreshold &&
+        this.failures / this.totalRequests >= this.failureThreshold) {
+      if (this.state !== "OPEN") {
+        this.state = "OPEN";
+        this.openedAt = Date.now();
+        console.error(`[zulip-ext] CIRCUIT BREAKER OPEN — ${this.failures}/${this.totalRequests} failures`);
+      }
+    }
+  }
+}
+
+// Retry with exponential backoff + jitter (from resilience patterns)
+async function withRetry(fn, { maxAttempts=3, baseDelay=200, maxDelay=10000, shouldRetry=()=>true }={}) {
+  let lastError;
+  for (let attempt = 1; attempt <= maxAttempts; attempt++) {
+    try {
+      return await fn();
+    } catch (err) {
+      lastError = err;
+      if (attempt === maxAttempts || !shouldRetry(err)) throw err;
+      const delay = Math.min(baseDelay * Math.pow(2, attempt - 1), maxDelay);
+      const jitter = delay * (0.5 + Math.random() * 0.5); // 50-100% of delay
+      console.warn(`[zulip-ext] Retry ${attempt}/${maxAttempts} after ${Math.round(jitter)}ms: ${err.message.slice(0,80)}`);
+      await new Promise(r => setTimeout(r, jitter));
+    }
+  }
+  throw lastError;
+}
+
+// Resilience-first event loop (Zulip call_on_each_event pattern)
+async function resilientPollLoop() {
+  while (connected) {
+    try {
+      const events = await circuitBreaker.fire(() =>
+        withRetry(() => zulipQueue.poll(), {
+          maxAttempts: 2,
+          baseDelay: 1000,
+          shouldRetry: (err) => {
+            const msg = err.message || "";
+            return msg.includes("fetch failed") || msg.includes("ECONN") || msg.includes("network");
+          }
+        })
+      );
+
+      lastError = null;
+      retryCount = 0;
+
+      for (const ev of events) {
+        await processEvent(ev);
+      }
+
+      heartbeat();
+    } catch (err) {
+      const msg = err instanceof Error ? err.message : String(err);
+
+      if (msg.includes("BAD_EVENT_QUEUE_ID") || msg.includes("deregistered")) {
+        // Queue expired — re-register (Zulip docs pattern)
+        console.log(`[zulip-ext] Queue expired, re-registering… (${msg.slice(0,80)})`);
+        try {
+          zulipQueue = await createZulipQueue();
+          console.log(`[zulip-ext] Re-registered, new queue=${zulipQueue.queueId}`);
+        } catch (reRegErr) {
+          console.error(`[zulip-ext] Re-registration failed: ${reRegErr.message}`);
+          connected = false;
+          retryCount++;
+          const backoff = Math.min(5000 * Math.pow(2, retryCount), 300000);
+          console.log(`[zulip-ext] Full reconnect in ${Math.round(backoff/1000)}s`);
+          await new Promise(r => setTimeout(r, backoff));
+          await startPolling();
+          return;
+        }
+      } else if (err.name === "CircuitOpenError") {
+        // Circuit is open — skip this cycle, wait for HALF_OPEN
+        lastError = "circuit_open";
+        await new Promise(r => setTimeout(r, POLL_INTERVAL_MS));
+      } else {
+        lastError = msg;
+        retryCount++;
+        const backoff = Math.min(POLL_INTERVAL_MS * Math.pow(1.5, Math.min(retryCount, 8)), 60000);
+        console.error(`[zulip-ext] Poll error (retry ${retryCount}, backoff ${backoff}ms): ${msg}`);
+        await new Promise(r => setTimeout(r, backoff));
+      }
+    }
+  }
+}
+```
+
+### Phase 2: PM2 Hardening
+
+Create `/root/.pm2/ecosystem.config.cjs`:
+
+```js
+module.exports = {
+  apps: [
+    {
+      name: "abiba-zulip",
+      script: "/bin/pi",
+      args: "--mode rpc --session-id zulip-service",
+      env: {
+        ZULIP_ROLE: "router",
+        ZULIP_SITE: "https://chat.sysloggh.net",
+        ZULIP_EMAIL: "abiba-bot@chat.sysloggh.net",
+        ZULIP_API_KEY: process.env.ZULIP_API_KEY,
+        AGENT_NAME: "abiba",
+        AGENT_OWNER_EMAIL: "jerome@sysloggh.com",
+      },
+      max_restarts: 100,            // Up from default 10 — crash loops won't exhaust
+      min_uptime: "10s",            // Must survive 10s to count as "alive"
+      max_memory_restart: "500M",   // OOM protection
+      restart_delay: 5000,          // 5s between restarts
+      kill_timeout: 15000,          // 15s SIGTERM grace before SIGKILL
+      listen_timeout: 30000,        // 30s to bind health port
+      log_date_format: "YYYY-MM-DD HH:mm:ss Z",
+      error_file: "/root/.pm2/logs/abiba-zulip-error.log",
+      out_file: "/root/.pm2/logs/abiba-zulip-out.log",
+      merge_logs: true,
+      autorestart: true,
+      watch: false,
+      instances: 1,
+      exec_mode: "fork",
+    },
+    {
+      name: "zulip-watchdog",
+      script: "/root/.pi/agent/extensions/zulip/watchdog.js",
+      max_restarts: 10,
+      min_uptime: "3s",
+      restart_delay: 3000,
+      autorestart: true,
+    },
+  ],
+};
+```
+
+### Phase 3: Supervisor Watchdog
+
+Create `/root/.pi/agent/extensions/zulip/watchdog.js`:
+
+```js
+// External supervisor — monitors router health and restarts if stalled.
+// This is the pattern Hermes uses: an external process that can recover
+// the gateway even if the gateway process itself is hung (not just crashed).
+
+const HEALTH_URL = "http://127.0.0.1:9200/health";
+const CHECK_INTERVAL_MS = 30_000;
+const MAX_FAILURES = 3;
+
+let failures = 0;
+
+async function check() {
+  try {
+    const res = await fetch(HEALTH_URL, { signal: AbortSignal.timeout(5000) });
+    if (res.ok) {
+      const data = await res.json();
+      if (data.status === "ok" && data.zulip?.connected) {
+        if (failures > 0) {
+          console.log(`[watchdog] Router recovered after ${failures} failures`);
+        }
+        failures = 0;
+        return;
+      }
+    }
+    failures++;
+    console.warn(`[watchdog] Health check ${failures}/${MAX_FAILURES}: status not ok`);
+  } catch (err) {
+    failures++;
+    console.warn(`[watchdog] Health check ${failures}/${MAX_FAILURES}: ${err.message}`);
+  }
+
+  if (failures >= MAX_FAILURES) {
+    console.error(`[watchdog] ${MAX_FAILURES} consecutive failures — restarting abiba-zulip`);
+    const { execSync } = require("child_process");
+    try {
+      execSync("pm2 restart abiba-zulip", { timeout: 30000 });
+      console.log("[watchdog] Restart command sent");
+    } catch (e) {
+      console.error(`[watchdog] Restart failed: ${e.message}`);
+    }
+    failures = 0;
+    // Wait for restart to complete before checking again
+    await new Promise(r => setTimeout(r, 15000));
+  }
+}
+
+console.log("[watchdog] Zulip gateway supervisor started");
+setInterval(check, CHECK_INTERVAL_MS);
+check(); // Immediate first check
+```
+
+### Phase 4: Health Endpoint Enhancement
+
+Add circuit breaker stats to the existing health endpoint:
+
+```js
+// In /health response, add:
+"circuit_breaker": {
+  "state": circuitBreaker.state,
+  "failures": circuitBreaker.failures,
+  "successes": circuitBreaker.successes,
+  "total_requests": circuitBreaker.totalRequests,
+  "failure_rate": circuitBreaker.totalRequests > 0
+    ? (circuitBreaker.failures / circuitBreaker.totalRequests).toFixed(2)
+    : "0.00"
+}
+```
+
+---
+
+## Test Plan
+
+### Test 1: Queue Re-registration
+1. Manually delete the Zulip event queue via API
+2. Next poll should detect BAD_EVENT_QUEUE_ID
+3. Router should auto re-register within 1 poll cycle
+4. Verify: `/health` shows new queue_id, connected=true
+
+### Test 2: Circuit Breaker Trip
+1. Block Zulip server with iptables: `iptables -A OUTPUT -d 192.168.68.19 -j DROP`
+2. Router should detect failures, trip circuit after 5 failures
+3. `/health` should show circuit_breaker.state = "OPEN"
+4. Remove iptables rule
+5. Circuit should transition to HALF_OPEN → CLOSED within 60s
+6. Verify: messages processed after recovery
+
+### Test 3: Supervisor Recovery
+1. Kill the router process: `kill -STOP $(pm2 pid abiba-zulip)` (freeze, don't kill)
+2. Watchdog should detect 3 failed health checks in 90s
+3. Watchdog should execute `pm2 restart abiba-zulip`
+4. Verify: router back online, connected=true
+
+### Test 4: Worker Busy Timeout
+1. Send a message that triggers a long-running operation
+2. If worker stays busy >5 minutes, should receive SIGKILL
+3. User should receive error DM: "Response timed out"
+
+### Test 5: End-to-End Message
+1. Send DM "What time is it?" from Jerome
+2. Should receive response within 30s
+3. `/health` should show messages_processed incremented
+
+---
+
+## Rollback Plan
+
+If v3 causes issues:
+1. `pm2 delete abiba-zulip; pm2 delete zulip-watchdog`
+2. Restore v2 from git: `cd /root/.pi/agent/extensions/zulip && git checkout index.js`
+3. `pm2 resurrect` to reload previous process list
+4. Verify: `/health` returns ok
+
+Backup v2 before starting: `cp index.js index.js.v2-backup-$(date +%Y%m%d-%H%M%S)`
+
+---
+
+## Success Metrics
+
+| Metric | Current (v2) | Target (v3) |
+|--------|-------------|-------------|
+| Uptime between manual interventions | 1-3 days | 30+ days |
+| Crash recovery | Manual (PM2 resurrect) | Automatic (circuit breaker + supervisor) |
+| Queue expiry handling | Crash | Auto re-register |
+| Busy worker deadlock | Router death | Worker SIGKILL + error DM |
+| PM2 restart exhaustion | Yes (max_restarts=10) | No (max_restarts=100 + watchdog) |