Compare commits

...
3 Commits
Author SHA1 Message Date
jerome 20cbb96e2d Merge pull request 'Vault cleanup + contract sync (WAL #1316)' (#19) from fix/vault-cleanup-contract-sync into master
PR Pipeline — Authorize → Validate → Review → Merge / auth (push) Successful in 4s
PR Pipeline — Authorize → Validate → Review → Merge / validate (push) Successful in 1s
PR Pipeline — Authorize → Validate → Review → Merge / lint (push) Successful in 1s
PR Pipeline — Authorize → Validate → Review → Merge / ai-review (push) Successful in 1s
PR Pipeline — Authorize → Validate → Review → Merge / gate (push) Successful in 0s
Reviewed-on: #19
2026-07-16 21:24:59 +00:00
root 8215b84f88 merge: resolve conflicts with master (delegation delete + gpu-fleet)
PR Pipeline — Authorize → Validate → Review → Merge / auth (pull_request) Successful in 2s
PR Pipeline — Authorize → Validate → Review → Merge / validate (pull_request) Successful in 1s
PR Pipeline — Authorize → Validate → Review → Merge / lint (pull_request) Successful in 2s
PR Pipeline — Authorize → Validate → Review → Merge / ai-review (pull_request) Successful in 3s
PR Pipeline — Authorize → Validate → Review → Merge / gate (pull_request) Successful in 2s
2026-07-16 21:22:34 +00:00
root b7e23e2592 vault: Infisical cleanup + contract sync (WAL #1316)
PR Pipeline — Authorize → Validate → Review → Merge / auth (pull_request) Successful in 2s
PR Pipeline — Authorize → Validate → Review → Merge / validate (pull_request) Successful in 1s
PR Pipeline — Authorize → Validate → Review → Merge / lint (pull_request) Successful in 2s
PR Pipeline — Authorize → Validate → Review → Merge / ai-review (pull_request) Successful in 2s
PR Pipeline — Authorize → Validate → Review → Merge / gate (pull_request) Successful in 0s
- litellm-api-keys: vault audit, tanko/koby/koonimo migration, key rotation log
- gpu-fleet: ornith-1.0-35b→strix-moe (remaining refs)
- hermes-agent-baseline: 256K all GPUs, CT IPs updated, Shumba retired
- delegation-prose-contract: removed (renamed to mumuni-delegation)
- contract-registry.yaml + cron-prompts-review.md: from feat/contract-registry
2026-07-16 21:17:09 +00:00
6 changed files with 2415 additions and 266 deletions
File diff suppressed because it is too large Load Diff
+612
View File
@@ -0,0 +1,612 @@
# Cron Prompts Review — All 10 Scheduled Contracts
Generated: 2026-07-13 20:59:18 ET
---
## hermes-key-enforcement
**Category:** compliance | **Domain:** hermes-agent | **Owner:** abiba | **Schedule:** 0 6 * * *
```
Contract Enforcement: hermes-key-enforcement
Category: compliance
Domain: hermes-agent
Owner: abiba
Schedule: Daily compliance scan at 6am ET
This is a compliance contract. Verify that the contract enforces the required standards and policies. Report any violations found.
Steps:
1. Load contract from prose-contracts/main (file: hermes-key-enforcement.prose.md)
2. Verify prerequisites (connectivity, tools, deps)
3. Execute contract per SOP
4. Run postconditions from contract registry
5. Generate receipt with status (pass/fail/escalated)
6. If any postcondition fails, escalate per contract escalation tiers
7. Log to ~/.hermes/runs/hermes-key-enforcement/
Postconditions to verify:
[
{
"check": "no plaintext API keys in config",
"verify": "grep -rc 'api_key: sk-' /root/.hermes/config.yaml",
"expect": "0 matches"
},
{
"check": "api_key_env used for harness/litellm providers",
"verify": "grep -c 'api_key_env.*LITELLM_API_KEY' /root/.hermes/config.yaml",
"expect": "count > 0"
}
]
Escalation Tiers:
- INFO: notify nobody → action: log_to_receipt
- WARNING: notify abiba → action: relay_alert
- CRITICAL: notify abiba, mumuni → action: relay_alert + pause
- FATAL: notify abiba, mumuni, kwame → action: relay_alert + pause + human_required
Circuit Breaker:
- Max retries: 3 per 60.0min window
- On trip: escalate_to_fatal
Receipt format: JSON with contract, run_id, timestamp, agent, status, actions_taken, postconditions, drift_alerts, evidence_path
Receipt storage: ~/.hermes/runs/hermes-key-enforcement/receipt-{timestamp}.json
Graph node: Create RA-H OS node for receipt with metadata: type=receipt, contract=hermes-key-enforcement, status=<status>
If the contract has no postconditions defined (e.g., reference/pattern contracts), log that it was loaded and skip execution.
IMPORTANT: If the contract file does not exist in prose-contracts/main, report failure and do NOT hallucinate forward.
```
---
## hermes-config-template
**Category:** compliance | **Domain:** hermes-agent | **Owner:** abiba | **Schedule:** 0 4 * * 1
```
Contract Enforcement: hermes-config-template
Category: compliance
Domain: hermes-agent
Owner: abiba
Schedule: Weekly config drift check Monday at 4am ET
This is a compliance contract. Verify that the contract enforces the required standards and policies. Report any violations found.
Steps:
1. Load contract from prose-contracts/main (file: hermes-config-template.prose.md)
2. Verify prerequisites (connectivity, tools, deps)
3. Execute contract per SOP
4. Run postconditions from contract registry
5. Generate receipt with status (pass/fail/escalated)
6. If any postcondition fails, escalate per contract escalation tiers
7. Log to ~/.hermes/runs/hermes-config-template/
Postconditions to verify:
[
{
"check": "agent config template_version matches template file",
"verify": "grep -q 'template_version' /root/.hermes/config.yaml && diff <(grep 'template_version' /root/.hermes/config.yaml | cut -d: -f2 | xargs) <(grep 'template_version' /root/prose-contracts/hermes-config-template.prose.md | cut -d: -f2 | xargs) && echo match || echo mismatch",
"expect": "match"
},
{
"check": "config file is valid YAML",
"verify": "python3 -c 'import yaml; yaml.safe_load(open(\"/root/.hermes/config.yaml\"))' && echo valid || echo invalid",
"expect": "valid"
}
]
Escalation Tiers:
- INFO: notify nobody → action: log_to_receipt
- WARNING: notify abiba → action: relay_alert
- CRITICAL: notify abiba, mumuni → action: relay_alert
- FATAL: notify abiba, mumuni, kwame → action: relay_alert + human_required
Circuit Breaker:
- Max retries: 3 per 60.0min window
- On trip: escalate_to_fatal
Receipt format: JSON with contract, run_id, timestamp, agent, status, actions_taken, postconditions, drift_alerts, evidence_path
Receipt storage: ~/.hermes/runs/hermes-config-template/receipt-{timestamp}.json
Graph node: Create RA-H OS node for receipt with metadata: type=receipt, contract=hermes-config-template, status=<status>
If the contract has no postconditions defined (e.g., reference/pattern contracts), log that it was loaded and skip execution.
IMPORTANT: If the contract file does not exist in prose-contracts/main, report failure and do NOT hallucinate forward.
```
---
## hermes-agent-baseline
**Category:** compliance | **Domain:** hermes-agent | **Owner:** abiba | **Schedule:** 0 5 * * 1
```
Contract Enforcement: hermes-agent-baseline
Category: compliance
Domain: hermes-agent
Owner: abiba
Schedule: Weekly baseline verification Monday at 5am ET
This is a compliance contract. Verify that the contract enforces the required standards and policies. Report any violations found.
Steps:
1. Load contract from prose-contracts/main (file: hermes-agent-baseline.prose.md)
2. Verify prerequisites (connectivity, tools, deps)
3. Execute contract per SOP
4. Run postconditions from contract registry
5. Generate receipt with status (pass/fail/escalated)
6. If any postcondition fails, escalate per contract escalation tiers
7. Log to ~/.hermes/runs/hermes-agent-baseline/
Postconditions to verify:
[
{
"check": "Hermes agent process running",
"verify": "pgrep -f 'hermes' > /dev/null && echo running || echo stopped",
"expect": "running"
},
{
"check": "agent config file exists and valid YAML",
"verify": "test -f /root/.hermes/config.yaml && python3 -c 'import yaml; yaml.safe_load(open(\"/root/.hermes/config.yaml\"))' && echo valid || echo invalid",
"expect": "valid"
},
{
"check": "no uncommitted changes in hermes directory",
"verify": "cd /root/.hermes && git status --porcelain | wc -l",
"expect": "0"
}
]
Escalation Tiers:
- INFO: notify nobody → action: log_to_receipt
- WARNING: notify abiba → action: relay_alert
- CRITICAL: notify abiba, mumuni → action: relay_alert
- FATAL: notify abiba, mumuni, kwame → action: relay_alert + human_required
Circuit Breaker:
- Max retries: 3 per 60.0min window
- On trip: escalate_to_fatal
Receipt format: JSON with contract, run_id, timestamp, agent, status, actions_taken, postconditions, drift_alerts, evidence_path
Receipt storage: ~/.hermes/runs/hermes-agent-baseline/receipt-{timestamp}.json
Graph node: Create RA-H OS node for receipt with metadata: type=receipt, contract=hermes-agent-baseline, status=<status>
If the contract has no postconditions defined (e.g., reference/pattern contracts), log that it was loaded and skip execution.
IMPORTANT: If the contract file does not exist in prose-contracts/main, report failure and do NOT hallucinate forward.
```
---
## proxmox-monitor
**Category:** monitoring | **Domain:** proxmox | **Owner:** abiba | **Schedule:** */15 * * * *
```
Contract Enforcement: proxmox-monitor
Category: monitoring
Domain: proxmox
Owner: abiba
Schedule: Every 15 minutes
This is a monitoring contract. Execute the monitoring checks defined in the contract. Report any deviations from expected state.
Steps:
1. Load contract from prose-contracts/main (file: proxmox-monitor.prose.md)
2. Verify prerequisites (connectivity, tools, deps)
3. Execute contract per SOP
4. Run postconditions from contract registry
5. Generate receipt with status (pass/fail/escalated)
6. If any postcondition fails, escalate per contract escalation tiers
7. Log to ~/.hermes/runs/proxmox-monitor/
Postconditions to verify:
[
{
"check": "all Proxmox nodes reachable",
"verify": "curl -sf http://192.168.68.10:8006/api2/json/status | jq '.status'",
"expect": "healthy"
},
{
"check": "no VMs in crashed state",
"verify": "pvesh get /nodes -output-format=json | jq '.[] | select(.status==\"Crashed\")'",
"expect": "empty"
},
{
"check": "backups running on schedule",
"verify": "pbs-info --check",
"expect": "last_backup < 24h ago"
}
]
Escalation Tiers:
- INFO: notify nobody → action: log_to_receipt
- WARNING: notify abiba, mumuni → action: relay_alert
- CRITICAL: notify abiba, mumuni → action: relay_alert + trigger_remediation
- FATAL: notify abiba, mumuni, kwame → action: relay_alert + pause + human_required
Circuit Breaker:
- Max retries: 3 per 60.0min window
- On trip: escalate_to_fatal
Receipt format: JSON with contract, run_id, timestamp, agent, status, actions_taken, postconditions, drift_alerts, evidence_path
Receipt storage: ~/.hermes/runs/proxmox-monitor/receipt-{timestamp}.json
Graph node: Create RA-H OS node for receipt with metadata: type=receipt, contract=proxmox-monitor, status=<status>
If the contract has no postconditions defined (e.g., reference/pattern contracts), log that it was loaded and skip execution.
IMPORTANT: If the contract file does not exist in prose-contracts/main, report failure and do NOT hallucinate forward.
```
---
## gpu-monitor
**Category:** monitoring | **Domain:** gpu | **Owner:** abiba | **Schedule:** */15 * * * *
```
Contract Enforcement: gpu-monitor
Category: monitoring
Domain: gpu
Owner: abiba
Schedule: Every 15 minutes — polls all GPU subsystems
This is a monitoring contract. Execute the monitoring checks defined in the contract. Report any deviations from expected state.
Steps:
1. Load contract from prose-contracts/main (file: gpu-monitor.prose.md)
2. Verify prerequisites (connectivity, tools, deps)
3. Execute contract per SOP
4. Run postconditions from contract registry
5. Generate receipt with status (pass/fail/escalated)
6. If any postcondition fails, escalate per contract escalation tiers
7. Log to ~/.hermes/runs/gpu-monitor/
Postconditions to verify:
[
{
"check": "GPU metrics accessible",
"verify": "curl -sf http://localhost:9100/gpu-data",
"expect": "200 OK, populated data"
},
{
"check": "dashboard serving",
"verify": "curl -sf http://localhost:9100/gpu-fleet.html",
"expect": "200 OK, HTML returned"
},
{
"check": "health endpoint responsive",
"verify": "curl -sf http://localhost:9100/health",
"expect": "200 OK"
}
]
Escalation Tiers:
- INFO: notify nobody → action: log_to_receipt
- WARNING: notify abiba, mumuni → action: relay_alert
- CRITICAL: notify abiba, mumuni → action: relay_alert + trigger_remediation
- FATAL: notify abiba, mumuni, kwame → action: relay_alert + pause + human_required
Circuit Breaker:
- Max retries: 3 per 60.0min window
- On trip: escalate_to_fatal
Receipt format: JSON with contract, run_id, timestamp, agent, status, actions_taken, postconditions, drift_alerts, evidence_path
Receipt storage: ~/.hermes/runs/gpu-monitor/receipt-{timestamp}.json
Graph node: Create RA-H OS node for receipt with metadata: type=receipt, contract=gpu-monitor, status=<status>
If the contract has no postconditions defined (e.g., reference/pattern contracts), log that it was loaded and skip execution.
IMPORTANT: If the contract file does not exist in prose-contracts/main, report failure and do NOT hallucinate forward.
```
---
## infrastructure-monitoring
**Category:** monitoring | **Domain:** infrastructure | **Owner:** abiba | **Schedule:** */30 * * * *
```
Contract Enforcement: infrastructure-monitoring
Category: monitoring
Domain: infrastructure
Owner: abiba
Schedule: Every 30 minutes
This is a monitoring contract. Execute the monitoring checks defined in the contract. Report any deviations from expected state.
Steps:
1. Load contract from prose-contracts/main (file: infrastructure-monitoring.prose.md)
2. Verify prerequisites (connectivity, tools, deps)
3. Execute contract per SOP
4. Run postconditions from contract registry
5. Generate receipt with status (pass/fail/escalated)
6. If any postcondition fails, escalate per contract escalation tiers
7. Log to ~/.hermes/runs/infrastructure-monitoring/
Postconditions to verify:
[
{
"check": "Proxmox API reachable",
"verify": "curl -sf http://192.168.68.10:8006/api2/json",
"expect": "200 OK"
},
{
"check": "Zulip API reachable",
"verify": "curl -sf https://chat.sysloggh.net/api/v1/me",
"expect": "200 OK"
},
{
"check": "LiteLLM proxy reachable",
"verify": "curl -sf http://192.168.68.116/litellm/v1/models",
"expect": "200 OK"
},
{
"check": "Gitea API reachable",
"verify": "curl -sf https://git.sysloggh.net/api/v1/version",
"expect": "200 OK"
},
{
"check": "SearXNG reachable",
"verify": "curl -sf http://192.168.68.17:8080",
"expect": "200 OK"
}
]
Escalation Tiers:
- INFO: notify nobody → action: log_to_receipt
- WARNING: notify abiba, mumuni → action: relay_alert
- CRITICAL: notify abiba, mumuni → action: relay_alert + trigger_remediation
- FATAL: notify abiba, mumuni, kwame → action: relay_alert + pause + human_required
Circuit Breaker:
- Max retries: 3 per 60.0min window
- On trip: escalate_to_fatal
Receipt format: JSON with contract, run_id, timestamp, agent, status, actions_taken, postconditions, drift_alerts, evidence_path
Receipt storage: ~/.hermes/runs/infrastructure-monitoring/receipt-{timestamp}.json
Graph node: Create RA-H OS node for receipt with metadata: type=receipt, contract=infrastructure-monitoring, status=<status>
If the contract has no postconditions defined (e.g., reference/pattern contracts), log that it was loaded and skip execution.
IMPORTANT: If the contract file does not exist in prose-contracts/main, report failure and do NOT hallucinate forward.
```
---
## zulip-health
**Category:** monitoring | **Domain:** zulip | **Owner:** abiba | **Schedule:** */15 * * * *
```
Contract Enforcement: zulip-health
Category: monitoring
Domain: zulip
Owner: abiba
Schedule: Every 15 minutes — monitors all Zulip-connected agents
This is a monitoring contract. Execute the monitoring checks defined in the contract. Report any deviations from expected state.
Steps:
1. Load contract from prose-contracts/main (file: zulip-health.prose.md)
2. Verify prerequisites (connectivity, tools, deps)
3. Execute contract per SOP
4. Run postconditions from contract registry
5. Generate receipt with status (pass/fail/escalated)
6. If any postcondition fails, escalate per contract escalation tiers
7. Log to ~/.hermes/runs/zulip-health/
Postconditions to verify:
[
{
"check": "bot registration active",
"verify": "curl -sf https://chat.sysloggh.net/api/v1/me | jq '.user_id'",
"expect": "bot_id present"
},
{
"check": "DM delivery working",
"verify": "curl -sf https://chat.sysloggh.net/api/v1/users/me/is-online",
"expect": "online: true"
}
]
Escalation Tiers:
- INFO: notify nobody → action: log_to_receipt
- WARNING: notify abiba, mumuni → action: relay_alert
- CRITICAL: notify abiba, mumuni → action: relay_alert + trigger_remediation
- FATAL: notify abiba, mumuni, kwame → action: relay_alert + pause + human_required
Circuit Breaker:
- Max retries: 3 per 60.0min window
- On trip: escalate_to_fatal
Receipt format: JSON with contract, run_id, timestamp, agent, status, actions_taken, postconditions, drift_alerts, evidence_path
Receipt storage: ~/.hermes/runs/zulip-health/receipt-{timestamp}.json
Graph node: Create RA-H OS node for receipt with metadata: type=receipt, contract=zulip-health, status=<status>
If the contract has no postconditions defined (e.g., reference/pattern contracts), log that it was loaded and skip execution.
IMPORTANT: If the contract file does not exist in prose-contracts/main, report failure and do NOT hallucinate forward.
```
---
## litellm-health
**Category:** monitoring | **Domain:** litellm | **Owner:** abiba | **Schedule:** */10 * * * *
```
Contract Enforcement: litellm-health
Category: monitoring
Domain: litellm
Owner: abiba
Schedule: Every 10 minutes — LiteLLM proxy health
This is a monitoring contract. Execute the monitoring checks defined in the contract. Report any deviations from expected state.
Steps:
1. Load contract from prose-contracts/main (file: litellm-health.prose.md)
2. Verify prerequisites (connectivity, tools, deps)
3. Execute contract per SOP
4. Run postconditions from contract registry
5. Generate receipt with status (pass/fail/escalated)
6. If any postcondition fails, escalate per contract escalation tiers
7. Log to ~/.hermes/runs/litellm-health/
Postconditions to verify:
[
{
"check": "LiteLLM proxy reachable",
"verify": "curl -sf http://192.168.68.116/litellm/v1/models",
"expect": "200 OK, models returned"
},
{
"check": "router deprecated, nginx routes work",
"verify": "curl -sf https://litellm.sysloggh.net/v1/models",
"expect": "200 OK (via nginx)"
}
]
Escalation Tiers:
- INFO: notify nobody → action: log_to_receipt
- WARNING: notify abiba, mumuni → action: relay_alert
- CRITICAL: notify abiba, mumuni → action: relay_alert + trigger_remediation
- FATAL: notify abiba, mumuni, kwame → action: relay_alert + pause + human_required
Circuit Breaker:
- Max retries: 3 per 60.0min window
- On trip: escalate_to_fatal
Receipt format: JSON with contract, run_id, timestamp, agent, status, actions_taken, postconditions, drift_alerts, evidence_path
Receipt storage: ~/.hermes/runs/litellm-health/receipt-{timestamp}.json
Graph node: Create RA-H OS node for receipt with metadata: type=receipt, contract=litellm-health, status=<status>
If the contract has no postconditions defined (e.g., reference/pattern contracts), log that it was loaded and skip execution.
IMPORTANT: If the contract file does not exist in prose-contracts/main, report failure and do NOT hallucinate forward.
```
---
## memory-audit-maintenance
**Category:** maintenance | **Domain:** memory | **Owner:** mumuni | **Schedule:** 0 3 * * *
```
Contract Enforcement: memory-audit-maintenance
Category: maintenance
Domain: memory
Owner: mumuni
Schedule: Daily at 3am ET
This is a maintenance contract. Execute the maintenance tasks defined in the contract. Report any issues found.
Steps:
1. Load contract from prose-contracts/main (file: memory-audit-maintenance.prose.md)
2. Verify prerequisites (connectivity, tools, deps)
3. Execute contract per SOP
4. Run postconditions from contract registry
5. Generate receipt with status (pass/fail/escalated)
6. If any postcondition fails, escalate per contract escalation tiers
7. Log to ~/.hermes/runs/memory-audit-maintenance/
Postconditions to verify:
[
{
"check": "memory files below 80% capacity",
"verify": "wc -l ~/.hermes/memories/*.md",
"expect": "total lines < threshold"
},
{
"check": "no stale entries",
"verify": "grep -r 'STALE' ~/.hermes/memories/",
"expect": "0 matches"
}
]
Escalation Tiers:
- INFO: notify nobody → action: log_to_receipt
- WARNING: notify mumuni → action: relay_alert
- CRITICAL: notify mumuni, abiba → action: relay_alert
- FATAL: notify mumuni, abiba, kwame → action: relay_alert + human_required
Circuit Breaker:
- Max retries: 3 per 60.0min window
- On trip: escalate_to_fatal
Receipt format: JSON with contract, run_id, timestamp, agent, status, actions_taken, postconditions, drift_alerts, evidence_path
Receipt storage: ~/.hermes/runs/memory-audit-maintenance/receipt-{timestamp}.json
Graph node: Create RA-H OS node for receipt with metadata: type=receipt, contract=memory-audit-maintenance, status=<status>
If the contract has no postconditions defined (e.g., reference/pattern contracts), log that it was loaded and skip execution.
IMPORTANT: If the contract file does not exist in prose-contracts/main, report failure and do NOT hallucinate forward.
```
---
## infrastructure-update
**Category:** maintenance | **Domain:** infrastructure | **Owner:** abiba | **Schedule:** 0 2 * * 0
```
Contract Enforcement: infrastructure-update
Category: maintenance
Domain: infrastructure
Owner: abiba
Schedule: Weekly system updates Sunday at 2am ET
This is a maintenance contract. Execute the maintenance tasks defined in the contract. Report any issues found.
Steps:
1. Load contract from prose-contracts/main (file: infrastructure-update.prose.md)
2. Verify prerequisites (connectivity, tools, deps)
3. Execute contract per SOP
4. Run postconditions from contract registry
5. Generate receipt with status (pass/fail/escalated)
6. If any postcondition fails, escalate per contract escalation tiers
7. Log to ~/.hermes/runs/infrastructure-update/
Postconditions to verify:
[
{
"check": "all services running after update",
"verify": "systemctl list-units --state=running",
"expect": "all critical services"
}
]
Escalation Tiers:
- INFO: notify nobody → action: log_to_receipt
- WARNING: notify abiba → action: relay_alert
- CRITICAL: notify abiba, mumuni → action: relay_alert
- FATAL: notify abiba, mumuni, kwame → action: relay_alert + human_required
Circuit Breaker:
- Max retries: 1 per 120.0min window
- On trip: escalate_to_fatal
Receipt format: JSON with contract, run_id, timestamp, agent, status, actions_taken, postconditions, drift_alerts, evidence_path
Receipt storage: ~/.hermes/runs/infrastructure-update/receipt-{timestamp}.json
Graph node: Create RA-H OS node for receipt with metadata: type=receipt, contract=infrastructure-update, status=<status>
If the contract has no postconditions defined (e.g., reference/pattern contracts), log that it was loaded and skip execution.
IMPORTANT: If the contract file does not exist in prose-contracts/main, report failure and do NOT hallucinate forward.
```
---
*End of review*
-256
View File
@@ -1,256 +0,0 @@
---
kind: pattern
name: delegation-prose-contract
description: >
Manager (Mumuni) operating doctrine for task decomposition, worker
delegation, verification, and delivery. Defines when to delegate, which
worker to use for what, how to handle failures, and the kanban board
protocol. Enforces context-window discipline and separation of concerns.
Runs on Mumuni (CT 118, storepve, .6) via Hermes agent.
version: 1.0.0
---
## Maintains
- Worker roster: 6 profiles (`syslog-code`, `syslog-devops`, `syslog-email`,
`syslog-research`, `syslog-review`, `syslog-writer`)
- Kanban board state at `~/.hermes/kanban/kanban.json`
- Context window budget: ~65K tokens per request (131K total, 60% threshold)
## Topology
**Cluster:** 5 Proxmox nodes (ocupve, acerpve, minipve, amdpve, storepve)
**Manager:** Mumuni (CT 118, storepve, .6) via Hermes agent
**Workers:** 6 profiles, all running on the same agent — no separate hosts needed
This contract is infrastructure-agnostic in terms of which nodes are used.
Workers execute tasks on whatever infrastructure they're given — SSH to .6,
.pm, .9, .12, or .15 depending on the task. The contract defines the
**who** and **when** — not the **where**.
## Why This Matters
Without enforced delegation, the manager consumes the full iteration budget
(60 calls) on single-turn tasks — SSH to 5 nodes, check each VM, read logs —
leaving no capacity for actual coordination. The result: context overflow
(59K tokens in system prompt), iteration exhaustion, and degraded response
quality. This contract exists because I blew through my budget checking
Proxmox node status instead of delegating to `syslog-devops`.
## Context Window Discipline
**The system prompt is ~6.5K tokens (stable: ~4.5K tool schemas + ~2K other guidance).**
**Volatile (MEMORY.md + USER.md): ~300 tokens.**
**Total base: ~6,800 tokens per request.**
The remaining budget is the conversation. Every tool call result adds to it.
If a single call returns >10K tokens (e.g., `grep` on a large file, SSH output
from multiple nodes), the context fills fast. That's why we delegate: workers
process in isolation and return compact results.
## Trigger Conditions
Delegation is **mandatory** when any of these apply:
| Condition | Threshold | Example |
|-----------|-----------|---------|
| Multiple tool calls needed | 2+ calls with intermediate logic | Read file → analyze → write report |
| Large data retrieval | Output >5K tokens | `grep -r "pattern" /path` on large dirs |
| Cross-domain work | Spans 2+ worker specialties | Infra check + email filter |
| Infrastructure changes | Any mutating operation | `qm set`, `systemctl restart`, `git push` |
| Research/analysis | Needs browser or deep reading | Web research, code review, data analysis |
| Code builds or changes | Writing or modifying code | Scripts, configs, patches |
| Sequential dependencies | Worker B needs Worker A's output | Code → Review → Deliver |
**Single tool calls stay at manager level.** Quick `grep`, `ls`, `cat`,
`curl`, `hermes tools list` — these are decision-making tools. The manager
reads them directly.
## Worker Selection Matrix
| Worker | Model | Toolsets | Role | Use When |
|--------|-------|----------|------|----------|
| `syslog-code` | qwen3.6-27B-code | terminal, file, web, memory, skills | Code patches, automation, scripts | Writing/modifying code, creating scripts, debugging, reading/writing files |
| `syslog-devops` | qwen3.6-27B-code | terminal, file, web, memory, skills | Infrastructure, DB, bridge, Proxmox | Server ops, SSH, Docker, Proxmox, DB queries, hardware checks |
| `syslog-email` | strix-moe | terminal, file, web, memory, skills | Email automation, mail operations | Sending/receiving email, inbox management, SMTP operations |
| `syslog-research` | strix-moe | terminal, file, web, memory, skills, **browser** | Analysis, classification, data processing | Web research, browser tasks, data analysis, classification, reading docs |
| `syslog-review` | strix-moe | terminal, file, web, memory, skills | Verification, QA, audit validation | **ALWAYS** verify worker output before delivery — especially for infra changes, code builds, and research findings |
| `syslog-writer` | strix-moe | terminal, file, web, memory, skills | Docs, content, branding, reports | Writing docs, reports, proposals, content, markdown formatting |
### Selection Rules
1. **Match specialty first.** A code task → `syslog-code`. An infra task →
`syslog-devops`. Don't put a `syslog-email` worker on a code review.
2. **Research tasks with browser needs → `syslog-research`.** Other workers
don't have the browser toolset.
3. **Verification → `syslog-review`.** Never deliver raw worker output.
4. **Documentation/content → `syslog-writer`.** Let them own the prose.
5. **If unsure, delegate to `syslog-research`** — it has the broadest toolset
(includes browser) and high reasoning effort.
## Delegation Protocol
### Step 1: Decompose
Break the task into lanes. Each lane does ONE thing. Workers are independent —
no lane depends on another's output mid-flight. If lanes depend on each other,
dispatch sequentially.
### Step 2: Dispatch
Fire workers via `delegate_task`:
**Parallel (independent lanes):**
```
delegate_task(
tasks=[
{"goal": "Check all 5 Proxmox nodes for VM status", "context": "SSH to each node via 192.168.68.x, run 'qm list'"},
{"goal": "Check Docker container health on .7/.116/.17", "context": "SSH to each host, check container status"},
]
)
```
**Sequential (dependent lanes):**
Dispatch lane 1 → wait for result → dispatch lane 2.
### Step 3: Verify
**MANDATORY for:**
- Infrastructure changes (any `qm`, `pct`, `systemctl`, `git push`)
- Code builds and modifications
- Research findings (web data, external sources)
- Any output that will reach the user
**Fire `syslog-review` to verify:**
```
delegate_task(
goal="Review the output of the devops worker. Verify the node status
report is accurate, check for inconsistencies, confirm all nodes were
reachable.",
context="Worker was syslog-devops. Output is at /tmp/node-report.md.
Verify against live system."
)
```
**If verification fails:**
1. Send work back to original worker with review feedback
2. Re-verify
3. Max 2 re-verify cycles before escalating to Kwame
### Step 4: Deliver
Only verified results reach Kwame. Format per channel:
- Telegram: Use `telegram-formatting` skill
- Zulip: Use Zulip Markdown (CommonMark)
- Email: Use `syslog-email` skill
## Kanban Board Protocol
**File:** `~/.hermes/kanban/kanban.json`
```json
{
"task_id": "unique-id",
"title": "Task description",
"created": "2026-07-09T01:00:00",
"status": "backlog|in_progress|review|done",
"lanes": [
{
"lane_id": "devops-check",
"worker": "syslog-devops",
"goal": "Check all 5 Proxmox nodes",
"status": "dispatched|completed|failed",
"output_file": "/tmp/node-report.md"
}
]
}
```
**Update the board on every state change.**
## Failure Handling
### Worker Timeouts
- Child timeout: **900 seconds** (15 minutes)
- Worker model `syslog-auto` is slow — it can hit the timeout limit with
22+ API calls
- **If a worker times out:** Re-dispatch with a narrower scope. Break the
task into smaller pieces that fit in the timeout window.
- **Avoid delegating sequential SSH hops** — each SSH connection adds latency
that compounds quickly. Prefer API-based or local approaches when possible.
### Worker Selection Failures
- `syslog-devops` is best for infrastructure tasks (SSH, Proxmox, Docker)
- `syslog-code` is best for code-level work (reading files, writing scripts)
- `syslog-research` has the browser toolset — use for web research
- `syslog-review` is the QA gate — always fire before delivery
- **Never fire more than 3 parallel workers** (max_concurrent_children: 3)
- **Never nest delegation** (max_spawn_depth: 1)
### Context Overflow
- If a task requires >10K tokens of output, delegate the processing
- Workers return compact summaries, not raw data dumps
- Pass file paths and concrete goals — never dump raw data into context
## Anti-patterns
- ❌ Reading large files into your own context before deciding → delegate the read
- ❌ Carrying SSH/grep/output results in your context → delegate the analysis
- ❌ Doing work yourself and then "pretending" to delegate → the user can tell
- ❌ Skipping verification → raw worker output never reaches the user
- ❌ Delegating single tool calls → keep quick reads/writes at manager level
- ❌ Firing more than 3 workers in parallel → hard limit
## Emergency Exception
**In an emergency (server down, service must be restored immediately):**
- Delegate the diagnosis (find the problem)
- Execute the fix yourself (minimize handoff latency)
- Verify the fix after delivery
- Log the exception in the kanban board
The emergency exception exists because the user needs the service back NOW,
not after three worker round-trips. But it's an exception — not the rule.
## What This Contract Doesn't Cover
1. **Worker profile configuration** — covered by `hermes-config-template.prose.md`
2. **SSH key management** — covered by existing SSH/Proxmox contracts
3. **Git workflow** — covered by `AGENTS.md` in the prose-contracts repo
4. **Cron job management** — covered by individual cron contracts
5. **Infra verification** — covered by `verify-before-mutate` protocol
## Verification
Run `scripts/worker-audit.py` to verify all 6 profiles are aligned:
```bash
python3 /root/.hermes/skills/kanban-orchestrator/scripts/worker-audit.py
```
## References
- `kanban-orchestrator` skill: The operational playbook (detailed execution steps)
- `worker-profile-audit.md` (skill reference): Worker configuration audit notes
- `delegation-timeout-patterns.md` (skill reference): Timeout handling patterns
- `verify-before-mutate` protocol: Infrastructure change verification
- `hermes-config-template.prose.md`: Worker profile configuration
## Success Criteria
This contract succeeds when:
1. **No context overflow** — single-turn tasks don't exhaust the iteration budget
2. **Workers do the work** — manager coordinates, doesn't execute
3. **Verification before delivery** — all output passes through `syslog-review`
4. **Kanban board is current** — every task has a lane, every lane has a status
5. **User gets verified results** — raw worker output never reaches Kwame
---
**Last updated:** 2026-07-09
**Author:** Mumuni (with Kwame's input on triggers and exception criteria)
**Status:** Draft — awaiting PR review and merge to prose-contracts main
+3 -3
View File
@@ -7,7 +7,7 @@ description: >
saturation watchdog, Prometheus/Grafana monitoring, and self-healing. saturation watchdog, Prometheus/Grafana monitoring, and self-healing.
UPDATED 2026-07-15: Stable role-based aliases introduced: strix-moe, UPDATED 2026-07-15: Stable role-based aliases introduced: strix-moe,
gpu-dense, gpu-light. These never change — only the underlying model does. gpu-dense, gpu-light. These never change — only the underlying model does.
Strix Halo: ornith-1.0-35b → unsloth/Qwen3.6-35B-A3B-MTP (UD-Q4_K_M, 22GB). Strix Halo: strix-moe → unsloth/Qwen3.6-35B-A3B-MTP (UD-Q4_K_M, 22GB).
RTX 5070: gemma-4-12b Q4_K_M → IQ4_NL + MTP draft (122 tok/s, 2x faster). RTX 5070: gemma-4-12b Q4_K_M → IQ4_NL + MTP draft (122 tok/s, 2x faster).
RTX 5070 context: 131K → 256K. VRAM: 88% (10.8/12.2GB). RTX 5070 context: 131K → 256K. VRAM: 88% (10.8/12.2GB).
Compression timeout: 300s (was 120s). Mumuni context: 128K (was 256K). Compression timeout: 300s (was 120s). Mumuni context: 128K (was 256K).
@@ -186,7 +186,7 @@ Show full fleet status: GPUs, models, VRAM, context windows, parallel slots, act
3. Check LiteLLM: `curl http://192.168.68.116/health` (expect "I'm alive!") 3. Check LiteLLM: `curl http://192.168.68.116/health` (expect "I'm alive!")
4. Check LiteLLM models: `curl -H "Authorization: Bearer $MASTER_KEY" http://192.168.68.116/v1/models` 4. Check LiteLLM models: `curl -H "Authorization: Bearer $MASTER_KEY" http://192.168.68.116/v1/models`
5. Check LiteLLM timeouts: `grep -n 'timeout:' /opt/inference-harness/litellm_config.yaml` 5. Check LiteLLM timeouts: `grep -n 'timeout:' /opt/inference-harness/litellm_config.yaml`
- gemma-4-12b: 120s, qwen3.6-27B-code: 300s, qwen3.6-35B-udq4/strix-moe: 300s (ornith-1.0-35b does NOT exist — legacy name, do not use) - gemma-4-12b: 120s, qwen3.6-27B-code: 300s, qwen3.6-35B-udq4/strix-moe: 300s (strix-moe does NOT exist — legacy name, do not use)
- global request_timeout: 300s, nginx proxy_read_timeout: 600s - global request_timeout: 300s, nginx proxy_read_timeout: 600s
6. Check AMD metrics: `curl http://192.168.68.15:9400/metrics` (Radeon 8060S, util%, VRAM, temp, power) 6. Check AMD metrics: `curl http://192.168.68.15:9400/metrics` (Radeon 8060S, util%, VRAM, temp, power)
7. Check port conflicts: verify only one llama-server on :8080 per host 7. Check port conflicts: verify only one llama-server on :8080 per host
@@ -314,7 +314,7 @@ Mumuni (CT114, 192.168.68.123) is the primary business assistant. This profile i
| `personalities` | `creative` | Creative assistant personality | | `personalities` | `creative` | Creative assistant personality |
| Platforms | cli, discord, homeassistant, signal, telegram, zulip | All Hermes platforms | | Platforms | cli, discord, homeassistant, signal, telegram, zulip | All Hermes platforms |
| Main model timeout | 300s | LiteLLM global timeout | | Main model timeout | 300s | LiteLLM global timeout |
| Compression model timeout | 300s | ornith timeout increased from 120s | | Compression model timeout | 300s | strix-moe timeout increased from 120s |
### Agent Update Status (2026-07-15) ### Agent Update Status (2026-07-15)
+7 -7
View File
@@ -5,7 +5,7 @@ version: 1.0.0
description: > description: >
Canonical known-good baseline for all Syslog Hermes agents. Captures the exact Canonical known-good baseline for all Syslog Hermes agents. Captures the exact
configuration state, keys, workarounds, and audit procedure. When an agent's configuration state, keys, workarounds, and audit procedure. When an agent's
configuration goes sideways, restore from this baseline. Last verified 2026-07-08. GPU context reduced to 128K on .8/.110, parallel 2 fleet-wide. configuration goes sideways, restore from this baseline. Last verified 2026-07-16. All GPUs 256K context (RTX 3090 .8, RTX 5070 .110, Strix Halo .15). Parallel 1 fleet-wide (Strix Halo handles compression solo).
author: Abiba (pi agent) author: Abiba (pi agent)
--- ---
@@ -26,11 +26,11 @@ done
|-------|-----|------|-----|---------------|------------|----------| |-------|-----|------|-----|---------------|------------|----------|
| Tanko | 112 | amdpve | .122 | `tanko` | Infisical vault | Hermes | | Tanko | 112 | amdpve | .122 | `tanko` | Infisical vault | Hermes |
| Mumuni | 114 | minipve | .123 | `mumuni` | Infisical vault | Hermes | | Mumuni | 114 | minipve | .123 | `mumuni` | Infisical vault | Hermes |
| Koby | 111 | amdpve | srv1079750 | `koby` | Infisical vault | **Hermes** | | Koby | 129 | amdpve | srv1079750 | `koby` | Infisical vault | **Hermes** |
| Koonimo | 113 | amdpve | ? | `koonimo` | Infisical vault | Hermes | | Koonimo | 114 | amdpve | ? | `koonimo` | Infisical vault | Hermes |
| Shumba | — | 192.168.68.119 | N/A | N/A (DeepSeek) | Hermes | | Shumba | — | 192.168.68.119 | N/A | N/A (DeepSeek) | Hermes (RETIRED — CT119 now Infisical vault) |
> **Note**: CT hostnames (tdunna→CT111, baggy→CT113) differ from agent identities (koby, koonimo). > **Note**: CT hostnames (tdunna→CT129, baggy→CT114) differ from agent identities (koby, koonimo).
Access: `pct-run <CT_ID> <command>` — no IPs needed. GPU hosts (.8, .110, .15) use SSH. Access: `pct-run <CT_ID> <command>` — no IPs needed. GPU hosts (.8, .110, .15) use SSH.
Keys are stored in Infisical vault (project=agents, env=production) and injected at Keys are stored in Infisical vault (project=agents, env=production) and injected at
@@ -169,9 +169,9 @@ pct-run <CT> grep -A8 "vision:" /root/.hermes/config.yaml | grep api_key
# Must show both api_key: sk-... and api_key_env: LITELLM_API_KEY # Must show both api_key: sk-... and api_key_env: LITELLM_API_KEY
``` ```
### For Koby (CT 111 / tdunna) ### For Koby (CT 129 / tdunna)
Koby runs Hermes on CT 111 (tdunna). Config files at `/root/.hermes/config.yaml`. Koby runs Hermes on CT 129 (tdunna). Config files at `/root/.hermes/config.yaml`.
Same Hermes pattern as Tanko/Mumuni/Koonimo — see config sections above. Same Hermes pattern as Tanko/Mumuni/Koonimo — see config sections above.
**LiteLLM key**: alias `koby` in LiteLLM DB, injected via `infisical run --` wrapper. **LiteLLM key**: alias `koby` in LiteLLM DB, injected via `infisical run --` wrapper.
+26
View File
@@ -122,7 +122,11 @@ runs as user `jerome`, not systemd root, needs user-scope adaptation).
| koby | .129 | infisical-gateway.sh + service token (migrated 2026-07-16) | KOBY_LITELLM_API_KEY | ✅ vault-backed, Zulip (tanko-bot@) + Telegram | | koby | .129 | infisical-gateway.sh + service token (migrated 2026-07-16) | KOBY_LITELLM_API_KEY | ✅ vault-backed, Zulip (tanko-bot@) + Telegram |
| koonimo | .114 | infisical-gateway.sh + service token (migrated 2026-07-16) | KOONIMO_LITELLM_API_KEY | ✅ vault-backed | | koonimo | .114 | infisical-gateway.sh + service token (migrated 2026-07-16) | KOONIMO_LITELLM_API_KEY | ✅ vault-backed |
> **Baggy = Koonimo (CT 113).** Deleted `BAGGY_LITELLM_API_KEY` from vault 2026-07-16. Only `KOONIMO_LITELLM_API_KEY` exists — one secret per agent. > **Baggy = Koonimo (CT 113).** Deleted `BAGGY_LITELLM_API_KEY` from vault 2026-07-16. Only `KOONIMO_LITELLM_API_KEY` exists — one secret per agent.
<<<<<<< HEAD
| tanko | .122 | infisical-gateway.sh + service token (migrated 2026-07-16) | TANKO_LITELLM_API_KEY | ✅ vault-backed |
=======
| tanko | .122 | **hardcoded in config.yaml** (runs as user jerome, not systemd) | TANKO_LITELLM_API_KEY | ⚠️ TODO: migrate to user-scope wrapper | | tanko | .122 | **hardcoded in config.yaml** (runs as user jerome, not systemd) | TANKO_LITELLM_API_KEY | ⚠️ TODO: migrate to user-scope wrapper |
>>>>>>> origin/master
### Tanko migration (pending) ### Tanko migration (pending)
@@ -183,6 +187,21 @@ TOKEN=$(curl -fsSL -X POST https://vault.sysloggh.net/api/v1/auth/universal-auth
-d '{"environment":"prod","secretValue":"sk-<NEW_KEY>","workspaceId":"<WORKSPACE_ID>","type":"shared"}' -d '{"environment":"prod","secretValue":"sk-<NEW_KEY>","workspaceId":"<WORKSPACE_ID>","type":"shared"}'
# OR via CLI: infisical secrets set --token=$TOKEN --projectId=322fceab... --env=prod ... # OR via CLI: infisical secrets set --token=$TOKEN --projectId=322fceab... --env=prod ...
``` ```
<<<<<<< HEAD
**2026-07-16 UPDATE — vault SYNCED + CLEANED.** The abiba service token (`st.8e848433…`, READ+WRITE)
can write to the vault. All session-13 rotated keys in vault and validate 200 against LiteLLM.
4 stale secrets deprecated, 5 personal creds flagged for separate project. Tanko migrated from
hardcoded keys to `infisical-gateway.sh` wrapper. Koonimo Zulip key restored.
**Machine Identity:** `8ddb9438-74fc-4ab6-bd74-929e7c47b53b` exists in Infisical UI.
Client secret needed to use universal auth for vault writes from automation.
**Service Token Inventory (2026-07-16):**
| Token ID | Name | Permissions | Used By |
|----------|------|-------------|---------|
| `st.8e848433…` | tanko-gateway | READ+WRITE | Abiba, Koonimo |
| `st.353699cd…` | tanko-agent | READ-only | Tanko |
=======
Creation requires the Infisical web UI (https://vault.sysloggh.net) under Project Settings → Creation requires the Infisical web UI (https://vault.sysloggh.net) under Project Settings →
Machine Identities, or an admin API call. **TODO: create `abiba-automation` machine identity Machine Identities, or an admin API call. **TODO: create `abiba-automation` machine identity
and store its credentials in the vault itself (or a root-only file).** and store its credentials in the vault itself (or a root-only file).**
@@ -197,11 +216,18 @@ koonimo `sk-OEK7z26n6E…`) are now in the vault as `MUMUNI_LITELLM_API_KEY` / `
`KOONIMO_LITELLM_API_KEY` and validate 200 against LiteLLM. The vault is the source of truth again. `KOONIMO_LITELLM_API_KEY` and validate 200 against LiteLLM. The vault is the source of truth again.
Creating a dedicated `abiba-automation` machine identity (via UI) is still the proper long-term fix Creating a dedicated `abiba-automation` machine identity (via UI) is still the proper long-term fix
so the shared service token isn't reused across hosts — but it is no longer blocking. so the shared service token isn't reused across hosts — but it is no longer blocking.
>>>>>>> origin/master
## Key Rotation Log ## Key Rotation Log
| Date | Agent | Action | Notes | | Date | Agent | Action | Notes |
|------|-------|--------|-------| |------|-------|--------|-------|
<<<<<<< HEAD
| 2026-07-16 | koonimo | add-zulip | Added KOONIMO_ZULIP_API_KEY to vault (Tt2bUL…). Updated wrapper to inject ZULIP_API_KEY + ZULIP_EMAIL. Restarted gateway → Zulip connected as koonimo-bot@ (bot_id=17). 3 platforms now. |
| 2026-07-16 | tanko | migrate | Migrated from hardcoded config.yaml key to infisical-gateway.sh wrapper + service token st.353699cd… (tanko-agent). Systemd user service updated, hardcoded key drop-ins removed. Verified LITELLM_API_KEY from vault, 3 platforms connected. |
| 2026-07-16 | vault | cleanup | 4 stale secrets deprecated: KAGENZ0_LITELLM_API_KEY, HERMES_OPENROUTER_KEY, LITELLM_API_KEY (generic duplicate), ZULIP_API_KEY (generic duplicate). 5 personal creds flagged for separate project. |
=======
>>>>>>> origin/master
| 2026-07-16 | mumuni | rotate | Old key malformed (sk-_SWAl_Vu_, 47 chars, not LiteLLM format) → 401. Deleted old `mumuni` key (token 15cbca18…), generated fresh (alias `mumuni`, 7 models: syslog-auto, qwen3.6-27B-code, gemma-4-12b, strix-moe, gpu-dense, gpu-light, qwen3.6-35B-udq4). New key sk-OzuWsoX2… written to /root/.hermes/.env (Rule 3/13 fallback). Vault sync PENDING (needs machine identity). WAL #1300. | | 2026-07-16 | mumuni | rotate | Old key malformed (sk-_SWAl_Vu_, 47 chars, not LiteLLM format) → 401. Deleted old `mumuni` key (token 15cbca18…), generated fresh (alias `mumuni`, 7 models: syslog-auto, qwen3.6-27B-code, gemma-4-12b, strix-moe, gpu-dense, gpu-light, qwen3.6-35B-udq4). New key sk-OzuWsoX2… written to /root/.hermes/.env (Rule 3/13 fallback). Vault sync PENDING (needs machine identity). WAL #1300. |
| 2026-07-16 | koby | rotate | Old key sk-6sbCNjz (401, stale in /etc/environment). Deleted old `koby` key, generated fresh (alias `koby`). New key sk-BqRRMboTI… in systemd drop-in `hermes-gateway.service.d/litellm-key.conf` + /etc/environment. Created `hermes-gateway.service` unit (was missing — gateway wasn't persistent) with `--replace`. Verified HTTP 200, Telegram connected. | | 2026-07-16 | koby | rotate | Old key sk-6sbCNjz (401, stale in /etc/environment). Deleted old `koby` key, generated fresh (alias `koby`). New key sk-BqRRMboTI… in systemd drop-in `hermes-gateway.service.d/litellm-key.conf` + /etc/environment. Created `hermes-gateway.service` unit (was missing — gateway wasn't persistent) with `--replace`. Verified HTTP 200, Telegram connected. |
| 2026-07-16 | baggy (koonimo) | rotate | Old key sk-krnw_zGB (401, hardcoded in systemd drop-in). Deleted old `baggy` key, generated fresh (alias `baggy`, metadata agent=koonimo). New key sk-OEK7z26n6E… in drop-in `hermes-gateway.service.d/litellm-key.conf`. CT113 IP changed .113→.114. Verified HTTP 200, Zulip connected. | | 2026-07-16 | baggy (koonimo) | rotate | Old key sk-krnw_zGB (401, hardcoded in systemd drop-in). Deleted old `baggy` key, generated fresh (alias `baggy`, metadata agent=koonimo). New key sk-OEK7z26n6E… in drop-in `hermes-gateway.service.d/litellm-key.conf`. CT113 IP changed .113→.114. Verified HTTP 200, Zulip connected. |