The monitor's Abiba leg parsed the :9200/health payload at the top level
(d.get('connected',False)) while the pi Zulip extension serves connection
state NESTED at zulip.connected / zulip.last_error (verified against the
extension's startHealthServer handler and the live endpoint). PI_CONNECTED
was therefore always False and every run took the DISCONNECTED path,
restarting a healthy bot: pm2 restarts=8 with the process created
2026-09-09T09:35:09Z, four ❌ Abiba verdicts today (04:23/05:35/06:55/09:35
UTC) and zero ✅, while the Zulip server answered HTTP 200 and the bot kept
heartbeating. The watchdog was the fault, not the connection.
Changes (Abiba leg only; every other leg byte-identical):
- Read the real nested shape: zulip.connected, zulip.last_error and
zulip.messages_processed. The old retry_count branch is DROPPED — the
payload exposes no retry counter (the extension keeps retryCount internal
and never serialises it), so the branch is fabricated and cannot stay.
- Fail-safe restart decision: a fetch error (HTTP 000), non-2xx response,
empty/unparseable body, or payload missing a boolean zulip.connected is a
clearly-labelled PROBE FAILURE (🟠 alert + ⚠️ log line naming reason and
HTTP code) and NEVER calls pm2 restart. pm2 restart runs only on
affirmative zulip.connected=false (🔴/❌ path unchanged in wording).
connected=true with last_error keeps the degraded 🟡 warn-no-restart path.
Regression test (new tests/zulip-monitor-abiba.sh, 36 checks): extracts the
real Abiba leg from between the # -- abiba-leg-start/-end markers in the
shipped script and executes it verbatim with stubbed curl/notify/pm2 against
fixtures of the real payload shape — asserts connected=true → no restart,
connected=false → restart, and empty/garbage/missing-key/non-boolean/HTTP
000/HTTP 500 bodies → probe-failure alert with zero restarts. The suite
fails loudly on the pre-fix base and on a marker-intact top-level-parse
variant, so this class of bug cannot return silently.
Prose Contracts — Syslog Solution LLC
Operating contracts for Syslog agents. Each .prose.md file defines a function (callable helper), responsibility (recurring duty with state), or pattern (instantiable knowledge/template) that agents can execute via OpenProse or follow manually.
⚠️ Operating Doctrine — Read Before Acting
Two rules govern how agents interact with this repo and the infrastructure it describes. Internalize both before running or relying on any contract.
1. Verify-before-mutate (the .117 rule)
Contracts are leads to investigate, NOT facts to act on. Live-state fields (IPs, ports, hostnames, credentials, container names, PIDs) drift. Before any mutating operation on infrastructure, verify the current state against the live system. On drift, halt and ask the user which value is correct — do NOT "fix" the live state to match a stale contract.
Origin: a contract stated Zulip CT 117 was at .117; the live IP was .19.
An agent ran pct set without checking pct config first and changed the IP
to the wrong value, breaking Zulip. The staleness was harmless until acted on.
Layer 3 enforcement — the safe-mutate wrapper (deployed on all 5 agents:
Abiba, Tanko, Mumuni, Koby, Koonimo — Tanko runs on DSH/DeepSeek Harness since
2026-08-27 but safe-mutate enforcement still applies). ALL infrastructure mutations MUST go
through safe-mutate. Raw sed -i, pct set, docker compose up --force-recreate, kill, rm on infrastructure outside safe-mutate is an
auditable protocol violation. The wrapper runs a verify command, optionally
checks an --expect pattern, refuses on mismatch, and logs every call to
/root/.safe-mutate/audit.log.
safe-mutate --verify "CMD" [--expect "PATTERN"] --mutate "CMD" [--reason "WHY"]
safe-mutate --verify "CMD" --dry-run --mutate "CMD" # verify + show, no execute
safe-mutate --verify "CMD" --verify-only # record a verification only
Load the verify-before-mutate skill (local pi skill + RA-H shared) for the
full protocol. See knowledge graph node #604 for the post-mortem.
2. Field trust levels
Not all contract content carries the same trust:
| Field type | Trust level | Examples |
|---|---|---|
| Policy | Authoritative — trust the contract | "hardcoded harness keys forbidden", "use api_key_env" |
| Live state | Never trust — always verify | IPs, ports, hostnames, credentials, container names, PIDs |
| Procedure | Trust but adapt | command patterns are right, values may be stale |
Contracts label live-state fields with VERIFY-BEFORE-USE. Treat any such
field as a hint to confirm against the live system, not a fact to apply.
3. Verify-before-fix (contract-native)
Never prose run a fix-contract directly. Run its verifier first, read the
verdict, lint the fixer, then run:
prose run <verifier> # e.g. zulip-platform-verification → verdict
prose lint <fixer> # validate structure without executing
prose preflight <fixer> # check deps/env, no execute
prose run <fixer> # only after verdict + lint pass
kind: responsibility contracts VERIFY and return a verdict without making
changes. kind: function contracts DO things. Forme ### Requires →
### Maintains wiring enforces this at the DAG level: a fixer that requires a
fresh verdict cannot fire until the verifier has run.
How Agents Run These Contracts
Option A: Via OpenProse CLI (preferred)
prose run <contract-name> [param1=value1 param2=value2 ...]
Examples:
# Run a memory audit with default parameters
prose run memory-audit-maintenance
# Run a memory audit with custom threshold
prose run memory-audit-maintenance memory_threshold=90 verify_configs=true
# Configure a new agent with the template
prose run hermes-config-template agent_name=syslog-devops default_model=claude-sonnet-4
# Configure an agent with a different auxiliary model
prose run hermes-config-template agent_name=syslog-code default_model=qwen3.6-27B-code auxiliary_model=gemma-4-12b
Option B: Manual Execution
If prose CLI isn't available, any agent can:
- Fetch the raw contract:
curl -sL https://git.sysloggh.net/SyslogSolution/prose-contracts/raw/branch/master/<filename> - Read the Execution section
- Follow the steps
Example for fetching:
curl -sL https://git.sysloggh.net/SyslogSolution/prose-contracts/raw/branch/master/memory-audit-maintenance.prose.md
Available Contracts
Responsibilities (Recurring Duties — kind: responsibility)
Run on trigger or schedule. Maintain persistent world-model state across runs.
| Contract | Domain | Description |
|---|---|---|
memory-audit-maintenance |
Memory | Audits & reorganizes an agent's native memory (MEMORY.md, USER.md). Categorizes entries, moves rules to skills, verifies configs. |
build-zulip-plugin |
Zulip | Generates and iteratively improves a Hermes Zulip platform plugin. Each run produces a new version. |
zulip-health |
Zulip | Checks Zulip connectivity, message flow, and bot responsiveness. |
zulip-mention-reliability |
Zulip | Diagnoses and fixes @mention detection issues in Zulip. |
zulip-approval-fix |
Zulip | Fixes broken /approve and /deny slash commands for Hermes agents. |
litellm-self-heal |
LiteLLM | Consolidated health check + self-healing for the full nginx → LiteLLM → GPU chain. Verifies 8 containers, 3 GPUs, model inference, and agent keys. Applies 9 remediation rules. (litellm-health merged into this contract 2026-07-09.) |
gpu-fleet |
GPU | Manages the GPU inference fleet: model deployment, registration, health checks, LiteLLM sync. |
gpu-monitor |
GPU | Comprehensive GPU fleet monitor — polls sidecars, router, LiteLLM every 15s, renders SSE dashboard. |
proxmox-monitor |
Infra | Proxmox cluster + Docker monitoring via the existing Grafana/Prometheus stack on CT 116. |
disk-gc-threat-response |
Infra | Fleet-wide disk health scan + garbage collection across 15 CTs + 3 GPU hosts + docker-vm KVM VM. 5-tier threat levels with automated GC and Zulip alerting. Two incidents resolved: kagentz (35.67GB) and amdpve (11.56GB). |
pm2-self-heal |
Ops | Monitors PM2 processes (abiba-zulip, abiba-telegram) and auto-restarts any that are stopped or errored. |
Instantiable Templates (kind: pattern — run with prose run)
Generate configs or perform a transformation on demand. No persistent state.
| Contract | Description | Key Parameters |
|---|---|---|
hermes-config-template |
Standard Hermes config for any Syslog agent. Enforces shared infra (Firecrawl, SearXNG, RA-H OS MCP, LiteLLM) + standardized api_key_env key indirection. |
agent_name (required), default_model, auxiliary_model |
hermes-key-enforcement |
Enforcement contract — all harness/LiteLLM providers MUST use api_key_env, never hardcoded keys. Includes detection query, rotation procedure, violation response. External providers (DeepSeek, OpenAI) exempt. |
(none — doctrine reference) |
Reference Documents (kind: pattern — read for context, not run)
Encode topology, architectural decisions, and lessons. You read them before planning; you don't prose run them.
| Contract | Description |
|---|---|
infrastructure-control |
Full topology and control pattern: 5-node Proxmox cluster, 3 Docker ecosystems, NFS storage, network verification, IP-first configuration doctrine. Live-state fields marked VERIFY-BEFORE-USE. |
pi-approval-architecture |
pi's approval model vs Hermes, available commands, architectural constraints. |
zulip-adapter-lessons |
Failure modes, fixes, and patterns from building the pi Zulip extension and Hermes Zulip plugin. |
Functions (kind: function — callable helpers, no state)
Called on-demand as single-render tools.
| Contract | Description |
|---|---|
litellm-api-keys |
Manages LiteLLM API keys for agent identity. Create, rotate, verify, and list agent keys. References gpu-fleet for current key inventory. |
litellm-health |
⚠️ DEPRECATED — consolidated into litellm-self-heal (2026-07-09). Retained for reference only. |
infrastructure-monitoring |
Target-state for Prometheus + GPU exporters + Grafana. Core stack deployed, GPU exporters NOT live. |
stirling-pdf-agent-access |
Documents the Stirling-PDF API access pattern for agents — global API key, 12 operations, curl examples. Agents use the stirling-pdf-api shared skill for templates. |
hello-world |
Minimal test contract — verifies the OpenProse execution pipeline works. |
Scripts (scripts/)
Companion shell scripts that contracts delegate to.
| File | Description |
|---|---|
daily-infra-report.py |
Generates the daily infrastructure dashboard (HTML email to jerome@sysloggh.com). |
pm2-self-heal.sh |
Shell companion to the pm2-self-heal contract — restarts crashed PM2 processes (runs every 5 min). |
agent-health-check.py |
Consolidated agent health: LiteLLM key validation + GPU port conflict + streaming checks (every 10 min). |
Contract Structure
Kinds determine the section set.
kind: responsibility (recurring duty with persistent state):
## Maintains — World-model state this contract tracks
## Parameters — Tunable inputs (with defaults)
## Requires — External dependencies and preconditions
## Continuity — When to run (triggers & cadence)
## Remembers — What to learn between runs
## Invariants — Inviolable rules for execution
## Execution — Step-by-step instructions
## Example Output — What a good run looks like
kind: function (stateless callable helper):
## Parameters — Tunable inputs (with defaults)
## Returns — Output schema
## Requires — External dependencies
## Execution — Step-by-step instructions
kind: pattern (instantiable knowledge):
Variable — templates use Parameters + Execution; reference docs are prose-driven.
kind: gateway / kind: test — not currently used in this repo.
Adding New Contracts
- Create a
.prose.mdfile following the structure above - Push to this repo
- The contract is immediately available for any agent to
prose run
Repository
- URL: https://git.sysloggh.net/SyslogSolution/prose-contracts
- Branch: master
- Auth: mumuni-bot (write access via PAT)