tune: switch compression model from strix-moe to syslog-auto

Relieve Strix Halo pressure by distributing compression across the
syslog-auto weighted pool (55% RTX 3090, 30% Strix Halo, 15% RTX 5070).

Updates:
- compression.model: strix-moe -> syslog-auto
- auxiliary.compression.model: strix-moe -> syslog-auto
- Rule 7: Updated for syslog-auto compression, removed aux prohibition
- Rule 8: Updated GPU workload distribution
- All docs/comments updated to reflect the change

AGENTS.md

@@ -128,10 +128,3 @@ safe-mutate --verify "CMD" [--expect "PATTERN"] --mutate "CMD" [--reason "WHY"]
 Read the [Authoring Guide](docs/AUTHORING-GUIDE.md) before writing any new contract.
 It covers the full process: verify → draft → lint → review → ship, with templates
 and style rules.
-
-## Maintaining this file
-
-Keep this file for knowledge useful to almost every future agent session in this project.
-Do not repeat what the codebase already shows; point to the authoritative file or command instead.
-Prefer rewriting or pruning existing entries over appending new ones.
-When updating this file, preserve this bar for all agents and keep entries concise.

CLAUDE.md

@@ -1 +0,0 @@
-AGENTS.md

contract-registry.yaml

@@ -22,7 +22,6 @@ owners:
 - abiba
 - mumuni
 - kwame
-- ops
 trigger_types:
 - scheduled
 - event_driven
@@ -59,7 +58,6 @@ index:
     - memory-audit-maintenance
     - gpu-fleet
     - infrastructure-update
-    - infrastructure-maintenance
     reference:
     - infrastructure-control
     - ra-h-os-custodianship-contract
@@ -92,7 +90,6 @@ index:
     - infrastructure-control
     - infrastructure-monitoring
     - infrastructure-update
-    - infrastructure-maintenance
     - pm2-self-heal
     - disk-gc-threat-response
     gpu:
@@ -133,6 +130,7 @@ index:
     - build-zulip-plugin
     - stirling-pdf-agent-access
     - gpu-fleet
+    - infrastructure-update
     - infrastructure-control
     - zulip-adapter-lessons
     - pi-approval-architecture
@@ -146,9 +144,6 @@ index:
     - mumuni-delegation
     kwame:
     - hello-world
-    ops:
-    - infrastructure-maintenance
-    - infrastructure-update
   by_trigger:
     scheduled:
     - hermes-key-enforcement
@@ -161,7 +156,6 @@ index:
     - litellm-health
     - memory-audit-maintenance
     - infrastructure-update
-    - infrastructure-maintenance
     event_driven:
     - litellm-self-heal
     - pm2-self-heal
@@ -203,7 +197,6 @@ index:
     - hermes-zulip-plugin
     - build-zulip-plugin
     - infrastructure-update
-    - infrastructure-maintenance
     - ra-h-os-custodianship-contract
     - mumuni-delegation
     normal:
@@ -1263,108 +1256,13 @@ contracts:
   last_run: null
   last_status: null
   drift_alerts: []
-- name: infrastructure-maintenance
-  file: infrastructure-maintenance.prose.md
-  kind: responsibility
-  category: maintenance
-  sensitivity: high
-  status: active
-  owner: ops
-  version: 1.0.0
-  trigger:
-    type: scheduled
-    cadence: 0 2 * * 0
-    description: Weekly host-level maintenance Sunday at 2am ET (replaces infrastructure-update
-      build-phase role; infra-update moves to ops)
-    cron_job_id: null
-  execution:
-    agent: ops
-    timeout: 3600
-    requires:
-    - infrastructure-monitoring run within last 30 minutes (pre-update health baseline)
-    - Proxmox snapshot of primary host OR /tmp backup dir created this run
-    - LiteLLM master key from Infisical vault for health verification
-    protocol:
-    - Load contract from prose-contracts/main
-    - Phase 0 preflight — capture health baseline, backup check, record image baseline
-    - Phase 1 apt update && apt upgrade -y on primary host
-    - Phase 2 docker compose pull for LiteLLM, SearXNG, and other running containers
-    - Phase 3 restart stacks one at a time with per-stack health verification
-    - Phase 4 verify every critical service (LiteLLM, SearXNG, Zulip, Gitea, PM2, Hermes gateways)
-    - On failure — rollback per protocol, escalate, do not loop beyond circuit breaker
-    - Log actions to ~/.hermes/runs/infrastructure-maintenance/
-  verification:
-    postconditions:
-    - check: all critical services running after update
-      verify: 'curl -sf http://192.168.68.116/litellm/v1/models && curl -sf https://chat.sysloggh.net/api/v1/server_settings && curl -sf https://git.sysloggh.net/api/v1/version && curl -sf http://192.168.68.7:8888 && pm2 jlist'
-      expect: all probes 200 OK / processes online
-    - check: no regressions from pre-update health baseline
-      verify: diff Phase 0 health-baseline against Phase 4 results
-      expect: no GREEN service turned RED
-    - check: docker containers on latest stable tags
-      verify: docker inspect --format '{{.Config.Image}}' <container> per service matches image-baseline.pulled_tag
-      expect: all containers running pulled tags
-    - check: APT packages up to date with no held broken packages
-      verify: apt list --upgradable 2>/dev/null | wc -l and apt-get -s upgrade | grep -ci broken
-      expect: upgradable == 0, broken == 0
-    artifact: maintenance run report with phase results and any rollback/escalation
-    verify_commands:
-    - curl -sf http://192.168.68.116/litellm/v1/models
-    - curl -sf http://192.168.68.7:8888
-    - curl -sf https://chat.sysloggh.net/api/v1/server_settings
-    - curl -sf https://git.sysloggh.net/api/v1/version
-    - pm2 jlist
-  receipt:
-    format: json
-    storage: ~/.hermes/runs/infrastructure-maintenance/
-    graph_node: true
-    schema:
-      contract: string
-      run_id: string
-      timestamp: ISO 8601
-      agent: string
-      status: pass|fail|escalated
-      phase: preflight|apt|images|restarts|verify|rollback|done|failed
-      actions_taken: array
-      postconditions: array
-      drift_alerts: array
-      evidence_path: string
-  escalation:
-    info:
-      action: log_to_receipt
-      notify: []
-    warning:
-      action: relay_alert
-      notify:
-      - abiba
-      - mumuni
-    critical:
-      action: relay_alert
-      notify:
-      - abiba
-      - mumuni
-    fatal:
-      action: relay_alert + pause + human_required
-      notify:
-      - abiba
-      - mumuni
-      - kwame
-  circuit_breaker:
-    max_retries: 2
-    window: 7200
-    trip_action: escalate_to_fatal
-  depends_on:
-  - infrastructure-monitoring
-  last_run: null
-  last_status: null
-  drift_alerts: []
 - name: infrastructure-update
   file: infrastructure-update.prose.md
   kind: responsibility
   category: maintenance
   sensitivity: high
   status: active
-  owner: ops
+  owner: abiba
   version: 1.0.0
   trigger:
     type: scheduled

gpu-fleet.prose.md

@@ -9,9 +9,9 @@ description: >
   gpu-dense, gpu-light. These never change — only the underlying model does.
   Strix Halo: strix-moe → unsloth/Qwen3.6-35B-A3B-MTP (UD-Q4_K_M, 22GB).
   RTX 5070: gemma-4-12b Q4_K_M → IQ4_NL + MTP draft (122 tok/s, 2x faster).
-  UPDATED 2026-07-17: Context reduced fleet-wide from 256K to 128K for stability.
-  Strix Halo model swapped to Genesis Hermes V3 APEX (LuffyTheFox, 24GB, uncensored,
+  UPDATED 2026-07-17: Strix Halo model swapped to Genesis Hermes V3 APEX (LuffyTheFox, 24GB, uncensored,
   Hermes agent fine-tune, tensor repair, multimodal with mmproj).
+  RTX 5070 swapped to HauhauCS Gemma4-12B QAT Uncensored Balanced (Q4_K_M, 87 tok/s, 0/465 refusals).
   Instability observed near 100K at 256K. 128K is the stable ceiling.
   For larger context needs → fall back to external providers (deepseek).
   VRAM headroom improved: RTX 3090 ~70%, RTX 5070 ~65%.
@@ -87,20 +87,32 @@ When a model is swapped on a GPU, ONLY the infrastructure layer changes — agen
 | Alias | GPU | Current Model | Will Route To |
 |-------|-----|---------------|---------------|
 | `strix-moe` | Strix Halo (.15) | qwen3.6-35B-udq4 | Whatever runs on Strix Halo |
-| `gpu-dense` | RTX 3090 (.8) | qwen3.6-27B-code | Whatever runs on RTX 3090 |
+| `gpu-dense` | RTX 3090 (.8) | qwen3.6-27B-code (ThinkingCap) | Whatever runs on RTX 3090 |
 | `gpu-light` | RTX 5070 (.110) | gemma-4-12b | Whatever runs on RTX 5070 |
 
 **Backward compatibility**: Old model-specific names (qwen3.6-27B-code, gemma-4-12b, qwen3.6-35B-udq4) still work
 but are deprecated for agent configs. Only the stable aliases survive model swaps.
 
-## Current Model Assignments (2026-07-15)
+## Current Model Assignments (2026-07-17)
 
 | Model | GPU | Host | VRAM | Ctx | KV Cache | Parallel | Batch/Ubatch | Status |
 |-------|-----|------|------|-----|----------|----------|-------------|--------|
-| qwen3.6-27B-code (MTP) | RTX 3090 | .8 (llm-gpu) | ~17/24.6GB (70%) | **128K** | turbo4 | 2 | default | ✅ 63 tok/s |
-| gemma-4-12b | RTX 5070 | .110 (ocu-llm) | ~7.8/12.2GB (65%) | 128K | q4_0 | 2 | 2048/1024 | ✅ healthy |
+| qwen3.6-27B-code (ThinkingCap) | RTX 3090 | .8 (llm-gpu) | ~20.9/24.6GB (85%) | **128K** | turbo4 | 1 | default | ✅ 68 tok/s |
+| gemma-4-12b (HauhauCS QAT) | RTX 5070 | .110 (ocu-llm) | ~10.0/12.2GB (82%) | 128K | q4_0 | 1 | 2048/1024 | ✅ 87 tok/s |
 | Genesis Hermes V3 APEX | Strix Halo Vulkan | .15 (amdpve) | ~10GB/64GB | 128K | q4_0 | 1 | 4096/1024 | ✅ 65 tok/s |
 
+> **RTX 5070 model swap (2026-07-17)**: Switched from `gemma-4-12b-it-IQ4_NL` (Unsloth, 191 tok/s)
+> to `HauhauCS/Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced` (Q4_K_M QAT, 87 tok/s).
+> Trade: 54% slower generation for QAT quality, 0/465 refusals, and agent-optimized tuning.
+> MTP draft also swapped: Q8_0 (444MB) → tuned draft (242MB), saving 200MB VRAM.
+> Role unchanged: gpu-light (vision, web extract, light auxiliary tasks).
+
+> **RTX 3090 model swap (2026-07-17)**: Switched from `Qwopus3.6-27B-v2-MTP-Q4_K_M` (63 tok/s)
+> to `bottlecapai/ThinkingCap-Qwen3.6-27B` (Q4_K_M QAT, 68 tok/s).
+> RL-finetuned: 50% fewer thinking tokens, MMLU-Pro 0.85 vs 0.83 base.
+> Self-spec MTP REQUIRED (crashes without it on turboquant build).
+> Added vision via mmproj (0.9GB). VRAM 85%.
+
 ## Routing Configuration (LiteLLM — July 2026)
 
 ### syslog-auto Weighted Pool (Direct GPU — bypasses router)
@@ -118,16 +130,16 @@ Note: All syslog-auto entries route directly to GPUs with `api_key: not-needed`.
 | Model | RPM Cap | Notes |
 |-------|---------|-------|
 | strix-moe (Hermes V3) | 40 | Tight cap — prevents Strix overload |
-| qwen3.6-27B-code | 500 | High cap — primary workhorse |
-| gemma-4-12b | 500 | High cap — IQ4_NL+MTP, 122 tok/s |
+| qwen3.6-27B-code | 500 | ThinkingCap Q4_K_M + MTP self-spec + vision, 68 tok/s |
+| gemma-4-12b | 500 | HauhauCS QAT Uncensored Balanced + MTP, 87 tok/s |
 
 ### Stable Aliases (for agent configs — never change)
 
 | Alias | RPM Cap | Routes To | Purpose |
 |-------|---------|-----------|---------|
 | `strix-moe` | 40 | Strix Halo | Compression tasks (MoE models) |
-| `gpu-dense` | 500 | RTX 3090 | Heavy reasoning |
-| `gpu-light` | 500 | RTX 5070 | Vision, web extract, light tasks |
+| `gpu-dense` | 500 | RTX 3090 (ThinkingCap) | Heavy reasoning, code gen, delegation |
+| `gpu-light` | 500 | RTX 5070 (HauhauCS QAT) | Vision, web extract, light tasks |
 
 ### Fallback Chains
 - gemma → qwen
@@ -249,8 +261,8 @@ If no SSH access, send Zulip DM via abiba-bot with vault update instructions.
 - **LiteLLM /metrics**: Requires auth. Prometheus uses `/health/liveliness` as workaround.
 - **VRAM (2026-07-15)**: RTX 3090 at ~17/24.6GB (~70%) with **128K context** (reduced from 256K 2026-07-17). RTX 5070 at ~7.8/12.2GB (~65%) with 128K context + MTP. Strix Halo at ~7GB/64GB.
 - **RTX 3090 runs `--parallel 2`** with MTP draft (spec-type draft-mtp, spec-draft-n-max 2).
-- **RTX 3090 config**: `-c 131072 -ctk turbo4 -ctv turbo4 --parallel 2 --flash-attn on --cont-batching --spec-type draft-mtp`. Context reduced to 128K (2026-07-17, was 256K). VRAM: ~70%. Service: `/home/llmuser/llama-wrapper.sh`.
-- **RTX 5070 config (2026-07-15)**: Switched to IQ4_NL + MTP draft (Q8_0) at 128K context. Gen speed: 122 tok/s. VRAM: ~7.8/12.2GB (~65%). Service: `/home/llmuser/llama-wrapper.sh`. Config: `--model gemma-4-12b-it-IQ4_NL.gguf --spec-draft-model gemma-4-12b-it-Q8_0-MTP.gguf --spec-type draft-mtp --spec-draft-n-max 4 --ctx-size 131072`.
+- **RTX 3090 config**: `-c 131072 -ctk turbo4 -ctv turbo4 --parallel 1 --flash-attn on --cont-batching --spec-type draft-mtp --spec-draft-n-max 4`. ThinkingCap Qwen3.6-27B Q4_K_M (15.7GB) + mmproj (0.9GB) + MTP self-spec. VRAM: ~85%. Service: `/home/llmuser/llama-wrapper.sh`. ⚠️ MTP REQUIRED for stability on this turboquant build — model segfaults without `--spec-type draft-mtp`. Outputs reasoning_content (hidden from agent, improves answer quality).
+- **RTX 5070 config (2026-07-17)**: HauhauCS Gemma4-12B QAT Uncensored Balanced (Q4_K_M) + tuned MTP draft (242MB) at 128K context, single slot. Gen speed: 87 tok/s (vs 191 IQ4_NL). VRAM: ~10.0/12.2GB (~82%). Service: `/home/llmuser/llama-wrapper.sh`. Model: `Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced-Q4_K_M.gguf`, MTP: `mtp-gemma-4-12B-it.gguf`, mmproj: `mmproj-Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced-BF16.gguf`. Recommended sampling: temp 0.6, top_k 64, top_p 0.9, min_p 0.05, repeat_penalty 1.1.
 - **LiteLLM timeout tuning (verified 2026-07-16 against `/opt/inference-harness/litellm_config.yaml` on CT 116)**: gemma-4-12b 120s, qwen3.6-27B-code 300s, qwen3.6-35B-udq4 300s, strix-moe 300s, syslog-auto routes all 300s. Nginx proxy_read_timeout: 600s. Global request_timeout: 300s.
 - **Strix Halo GPU**: Vulkan is the working backend (ROCm/HIP path abandoned — HSA runtime blocked on Debian 13). Build at `/root/llama.cpp/build-vk/`, commit `4fc4ec5` (2026-07-01), ggml 0.15.3 shared-lib arch. Mesa RADV 25.0.7, KHR_coopmat fast path active. ~70 tok/s gen, 532 tok/s prompt. Service: `strix-server.service` on port 8080, model: `LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-V3-GGUF` (APEX quant), alias `strix-moe`, 128K context, flash-attn + q4 KV, multimodal (mmproj loaded). Hermes agent fine-tune, tensor repair (SSM layers fixed via SVD), uncensored (0/465 refusals).
 - **Port conflict detection (2026-07-05)**: All 3 GPU wrappers now detect ghost processes squatting port 8080 before starting. `.8` and `.110` use inline pre-start check in `llama-wrapper.sh`; `.15` uses `/usr/local/bin/port-cleanup.sh` ExecStartPre. Replaces the blanket `pkill -9 -x llama-server` on .15 which would kill ALL llama-server instances regardless of port. Ghost detection was the root cause of .8 crash-looping for 27+ restarts (stale pid 25836 squatting 8080 after OOM kill).
@@ -266,8 +278,8 @@ If no SSH access, send Zulip DM via abiba-bot with vault update instructions.
 
 | GPU | Model | Gen tok/s | Prompt tok/s | Baseline | Context |
 |-----|-------|-----------|--------------|----------|---------|
-| RTX 3090 (.8) | qwen3.6-27B-code (MTP) | **63** | — | — | **128K** |
-| RTX 5070 (.110) | gemma-4-12b (IQ4_NL+MTP) | **191** | — | — | **128K** |
+| RTX 3090 (.8) | ThinkingCap-Qwen3.6-27B Q4_K_M | **68** | — | — | **128K** |
+| RTX 5070 (.110) | HauhauCS QAT Uncensored Balanced | **87** | — | — | **128K** |
 | Strix Halo (.15) | Genesis Hermes V3 APEX | **65** | 140 | — | **128K** |
 
 Benchmarks from 2026-07-17. Strix Halo swapped to Genesis Hermes V3 APEX (LuffyTheFox). RTX 5070 MTP provides 2.7x speedup over pre-upgrade 70 tok/s.

hermes-agent-baseline.prose.md

@@ -193,8 +193,6 @@ Key is injected via `infisical run --` wrapper at PM2 startup:
       "models": [
         { "id": "syslog-auto" },
         { "id": "strix-moe" },
-        { "id": "gpu-dense" },
-        { "id": "gpu-light" },
         { "id": "qwen3.6-27B-code" },
         { "id": "gemma-4-12b" }
       ]

hermes-config-template.prose.md

@@ -5,11 +5,14 @@ description: >
   Standard Hermes configuration template for Syslog Solution LLC agents.
   Enforces shared infrastructure setup (Firecrawl, SearXNG, local models,
   RA-H OS MCP) while keeping agent-specific API keys and model choices.
-  UPDATED 2026-07-16: Compression model is the stable alias `strix-moe` (NOT `ornith-1.0-35b`,
+  UPDATED 2026-07-18: Compression model switched to `syslog-auto` (was `strix-moe`)
+  to relieve Strix Halo pressure. syslog-auto distributes compression across the
+  weighted pool (55% RTX 3090, 30% Strix Halo, 15% RTX 5070).
+  UPDATED 2026-07-16: Compression model was the stable alias `strix-moe` (NOT `ornith-1.0-35b`,
   which LiteLLM does not serve). All 3 GPUs verified at 128K (reduced from 256K 2026-07-17 for stability).
   Added Rule 12 (Context-Issue Diagnostic) + Rule 13 (.env fallback enforcement) from the
   2026-07-16 Mumuni root-cause investigation (WAL #1300).
-  UPDATED 2026-07-12: GPU workload redistributed. Compression → Strix Halo. RTX 3090 context verified at 128K. Infisical .env fallback required (Rule 3/13).
+  UPDATED 2026-07-12: GPU workload redistributed. Compression → Strix Halo (later switched to syslog-auto 2026-07-18). RTX 3090 context verified at 128K. Infisical .env fallback required (Rule 3/13).
 ---
 
 ## Maintains
@@ -130,7 +133,9 @@ mcp_servers:
 # ─── Compression ───
 compression:
   enabled: true
-  model: strix-moe               # ⚠️ Must match auxiliary.compression.model. Stable alias (gpu-fleet § Stable Role-Based Aliases). NOT ornith-1.0-35b (LiteLLM does not serve that name).
+  model: syslog-auto              # ⚠️ Switched from strix-moe 2026-07-18 to relieve Strix Halo.
+                                  # syslog-auto distributes across weighted pool (55% RTX 3090,
+                                  # 30% Strix Halo, 15% RTX 5070). All GPUs at 128K.
   provider: harness
   max_context_window: 131072      # MUST match actual GPU capacity. All 3 GPUs are 128K (Jul 17).
   threshold: 0.65                 # Fires at ~170K for 262K window, ~85K for 128K
@@ -145,8 +150,9 @@ compression:
 #   model: gpu-light                 # stable alias (NOT raw "gemma-4-12b")
 #   base_url: http://192.168.68.116/v1
 #   api_key_env: LITELLM_API_KEY
-# Do NOT use syslog-auto for auxiliary tasks — it routes to the primary GPU.
-# gpu-light = RTX 5070 (12B), freeing the Strix Halo for agent reasoning.
+# Compression uses syslog-auto (switched from strix-moe 2026-07-18) to distribute
+# load across the weighted pool and relieve Strix Halo pressure.
+# Vision and web_extract use gpu-light = RTX 5070 (12B).
 # Heavy aux (delegation, x_search) use gpu-dense (RTX 3090) instead.
 # NEVER use raw model names (gemma-4-12b, qwen3.6-27B-code, qwen3.6-35B-udq4)
 # in agent configs — use the stable aliases so model swaps don't break agents.
@@ -166,7 +172,7 @@ auxiliary:
     timeout: 30
   compression:
     provider: harness
-    model: strix-moe              # MUST match compression.model above. Stable alias for Strix Halo.
+    model: syslog-auto             # Switched from strix-moe 2026-07-18. Relieves Strix Halo pressure.
     base_url: http://192.168.68.116/v1   # Rule 5: /v1 NOT /litellm/v1
     api_key_env: LITELLM_API_KEY
     timeout: 300                 # gpu-fleet: 300s for large-history summarization (was 60)
@@ -247,29 +253,30 @@ The following MUST be identical across ALL profiles:
 - Apply to BOTH main config AND all sub-agent profiles
 - For agents needing longer outputs: raise to 8192, but never omit
 
-### Rule 7: Auxiliary Model Consistency (UPDATED 2026-07-16)
-- Vision and web_extract use `gemma-4-12b` (RTX 5070 — 12GB, vision-optimized)
-- Compression uses `strix-moe` (stable alias for Strix Halo — 64GB, 128K ctx, compression-optimized)
-- **`strix-moe` is the only valid compression model name** — LiteLLM does NOT serve `ornith-1.0-35b`
-  (it serves `strix-moe`, `qwen3.6-35B-udq4`, `gpu-dense`, `gpu-light`, `syslog-auto`, `gemma-4-12b`, `qwen3.6-27B-code`). Old configs with `ornith-1.0-35b` cause 403/model-not-found on compression calls.
+### Rule 7: Auxiliary Model Consistency (UPDATED 2026-07-18)
+- Vision and web_extract use `gpu-light` (stable alias, RTX 5070 — 12GB, vision-optimized)
+- Compression now uses `syslog-auto` (switched from `strix-moe` 2026-07-18) to distribute
+  compression load across the weighted pool (55% RTX 3090, 30% Strix Halo, 15% RTX 5070).
+  This relieves Strix Halo pressure while keeping compression functional on all GPUs.
+- **`syslog-auto` is the valid compression model** — LiteLLM serves it as the weighted pool.
+  Old configs with `strix-moe` for compression should be updated to `syslog-auto`.
 - All auxiliary services MUST use identical routing:
   - `base_url: http://192.168.68.116/v1`  (Rule 5: `/v1`, NOT `/litellm/v1`)
   - `api_key_env: LITELLM_API_KEY`
-- **Do NOT use `syslog-auto`** for auxiliary tasks — it routes unpredictably
-- **Compression on Strix Halo**: The strix-moe alias routes to Strix Halo
-  (64GB UMA, 128K context) — the designated compression GPU. This frees the
-  RTX 5070 for vision and web search, and the RTX 3090 for heavy reasoning.
+- **Compression via syslog-auto**: Routes through the weighted pool. Strix Halo still handles
+  ~30% of compression calls (at 60 RPM via pool vs 40 RPM direct), but the bulk (55%)
+  goes to RTX 3090 which has ample spare capacity.
 - The `compression:` block's `model` MUST match `auxiliary: compression: model`
 - The `compression: max_context_window: 131072` MUST match actual GPU capacity (128K)
 
-### Rule 8: GPU Workload Distribution (UPDATED 2026-07-16)
-- **RTX 3090 (24GB, 128K ctx, qwen3.6-27B-code)**: Heavy reasoning, code gen, long conversations
-- **RTX 5070 (12GB, 128K ctx, gemma-4-12b)**: Vision, web search, quick tasks, web_extract (IQ4_NL+MTP, ~65% VRAM at 128K)
-- **Strix Halo (64GB, 128K ctx, strix-moe)**: Context compression, summarization, long docs
+### Rule 8: GPU Workload Distribution (UPDATED 2026-07-18)
+- **RTX 3090 (24GB, 128K ctx, qwen3.6-27B-code)**: Heavy reasoning, code gen, long conversations — also handles ~55% of compression via syslog-auto pool
+- **RTX 5070 (12GB, 128K ctx, gemma-4-12b)**: Vision, web search, quick tasks, web_extract — handles ~15% of compression via syslog-auto pool
+- **Strix Halo (64GB, 128K ctx, Geneis Hermes V3 APEX)**: Agent reasoning, compression (~30% via syslog-auto pool), fallback for other GPUs
 - Agent profiles MUST route auxiliary tasks to the correct GPU:
-  - `auxiliary.vision.model: gemma-4-12b` (RTX 5070)
-  - `auxiliary.web_extract.model: gemma-4-12b` (RTX 5070)
-  - `auxiliary.compression.model: strix-moe` (Strix Halo)
+  - `auxiliary.vision.model: gpu-light` (RTX 5070)
+  - `auxiliary.web_extract.model: gpu-light` (RTX 5070)
+  - `auxiliary.compression.model: syslog-auto` (distributed pool, switched from strix-moe 2026-07-18)
 - Default model (`model.default`) and custom_provider remain `syslog-auto` for auto-routing
 - For 128K context window: `threshold: 0.65` (fires at ~85K tokens)
 - Do NOT use `threshold: 0.25` — this fires at 65K, causing premature context loss
@@ -325,8 +332,7 @@ verify ALL FOUR of these against the live config. They are the only root causes
 
 One-line agent health check (run on the agent host):
 ```bash
-# Use grep -v infisical to avoid matching the bash wrapper that contains the same string
-PID=$(pgrep -f "python -m hermes_cli.main gateway run" | grep -v infisical | head -1)
+PID=$(pgrep -f "python -m hermes_cli.main gateway run" | head -1)
 cat /proc/$PID/environ | tr '\0' '\n' | grep ^LITELLM_API_KEY= | sed 's/=.*/<set>/'
 curl -s -o /dev/null -w 'key_health: %{http_code}\n' -H "Authorization: Bearer $(cat /proc/$PID/environ | tr '\0' '\n' | grep ^LITELLM_API_KEY= | cut -d= -f2)" http://192.168.68.116/v1/models
 ```
@@ -352,19 +358,9 @@ directly (no infisical). Apply with `systemctl daemon-reload && systemctl restar
 The wrapper sources `~/.hermes/.env` then exports `LITELLM_API_KEY="$<AGENT>_LITELLM_API_KEY"`.
 See litellm-api-keys.prose.md § Machine Identity for Vault Writes for vault sync.
 
-**⚠️ Vault empty-key guard:** If the vault stores the secret as an empty string,
-the wrapper will inject an empty key and the gateway will silently get 401 errors
-on all LiteLLM requests (triggering silent DeepSeek fallback). The `.env` fallback
-is present but the vault takes precedence when the secret key exists (even if empty).
-
-**Fix:** The wrapper MUST validate the key length after injection. If LITELLM_API_KEY
-is empty or shorter than 20 chars, log a warning and either fail with a clear error
-message or fall back to the `.env` value before starting the gateway.
-
 **Verification (all agents):**
 ```bash
-# Use grep -v infisical to avoid matching the bash wrapper that contains the same string
-GP=$(pgrep -f "python -m hermes_cli.main gateway run" | grep -v infisical | head -1)
+GP=$(pgrep -f "python -m hermes_cli.main gateway run" | head -1)
 K=$(cat /proc/$GP/environ | tr '\0' '\n' | grep '^LITELLM_API_KEY=' | cut -d= -f2)
 curl -s -o /dev/null -w '%{http_code}' -H "Authorization: Bearer $K" http://192.168.68.116/v1/models  # must be 200
 ```

infrastructure-maintenance.prose.md

@@ -1,200 +0,0 @@
----
-kind: responsibility
-name: infrastructure-maintenance
-description: >
-  Weekly system-level maintenance for the Syslog inference fleet: OS package
-  updates on the primary host, Docker image pulls for LiteLLM/SearXNG and other
-  running containers, container restarts with health verification, post-update
-  verification of every critical service (LiteLLM proxy, SearXNG, Zulip, Gitea,
-  PM2 processes, Hermes gateways), and rollback on failure. Consolidates the
-  raw shell scripts that previously did this piecemeal. This contract owns the
-  HOST-LEVEL weekly maintenance loop on the primary host plus Docker image
-  pulls ONLY for .116 and .7, while infrastructure-update owns the FULL-FLEET
-  cluster-wide wave (apt across the full PVE cluster + CTs/VMs AND its Docker
-  image Wave 3 across all stacks). Runs Sunday 2am ET. Owner:
-  ops (firstmate secondmate). Blast radius: an unverified image pull can break
-  LiteLLM (all agents lose inference) or SearXNG (search-stack down); a bad apt
-  upgrade can leave the host in a half-upgraded state. Pre-update backup check
-  and rollback are mandatory for this reason.
-agent: ops
-triggers:
-  - weekly (Sunday 02:00 ET) via cron
-  - on demand when ops/abiba triggers "infra maintenance"
-version: 1.0.0
----
-
-## Maintains
-
-- maintenance-status: { phase: idle|preflight|apt|images|restarts|verify|rollback|done|failed, host, step, result, timestamp }
-- image-baseline: { service, current_tag, pulled_tag, digest, updated_at } — last known-good image per container
-- apt-state: { upgradable_before, upgradable_after, held_broken, kernel_reboot_required }
-- health-baseline: snapshot of critical-service health captured pre-update (used for regression check post-update)
-- rollback-snapshot: { backup_path, configs, image_digests, timestamp } — restore point created in preflight
-- maintenance-history: array of past runs with phase results and any escalations
-
-## Scope
-
-Primary host is the maintenance host where apt updates apply. Docker image pulls
-span the two Docker ecosystems that run critical services. infrastructure-update
-runs the full-fleet cluster-wide wave (including its Wave 3 Docker pulls across
-all stacks/hosts); this contract runs a narrower host-level weekly pull limited
-to .116 and .7. Topology, CT IDs, and IPs are live-state fields — verify against
-`infrastructure-control.prose.md` (the source of truth) and the live system
-before mutating.
-
-| Host | IP | Role | Trust |
-|------|----|------|-------|
-| CT 116 (syslog-api) | 192.168.68.116 | LiteLLM proxy + Grafana + Prometheus (inference harness) | ⚠️ VERIFY-BEFORE-USE |
-| VM 109 (docker-vm) | 192.168.68.7 | SearXNG + Firecrawl + home stack (Docker host) | ⚠️ VERIFY-BEFORE-USE |
-| CT 117 (zulip) | 192.168.68.19 | Zulip (storepve bridge IP .19) | ⚠️ VERIFY-BEFORE-USE |
-| Gitea | https://git.sysloggh.net | Prose-contracts + agent configs source control | ⚠️ VERIFY-BEFORE-USE |
-| CT 100 (abiba/pi) | 192.168.68.24 | PM2 processes (pi agent harness) | ⚠️ VERIFY-BEFORE-USE |
-
-> "Primary host" for the apt phase is the host the ops agent runs maintenance
-> from. Confirm which host that is against infrastructure-control before
-> running; do not assume. If the ops agent is containerized/CT-based, apt runs
-> inside that CT.
-
-## Requires
-
-- SSH/exec access to CT 116 (.116) and VM 109 (.7) for Docker operations
-- `apt`, `docker`, `docker compose` available on target hosts
-- LiteLLM master key available (Infisical vault, `LITELLM_API_KEY`) for health verification
-- `infrastructure-monitoring` run completed within the last 30 minutes — provides the pre-update health baseline used by the regression check
-- Writable backup directory `/tmp/infra-maintenance-backup-<date>/` on each mutated host
-- Proxmox snapshot of the primary host available (or confirmed not required) before apt phase
-
-## Continuity
-
-- Self-driven: weekly cron `0 2 * * 0` (Sunday 02:00 ET)
-- Also wakes on: explicit "infra maintenance" trigger from ops/abiba
-- Depends on `infrastructure-monitoring` for the pre-update health baseline — do not run if the last monitoring run is stale (>30 min) or RED; abort and escalate instead
-
-## Execution
-
-### Phase 0 — Preflight (snapshot/backup check + health baseline)
-
-1. **Capture health baseline** — run the `infrastructure-monitoring` postcondition checks (LiteLLM, Zulip, Gitea, SearXNG, Proxmox API) and record results as `health-baseline`. If any critical service is already down, **abort**: maintenance must not run on a degraded fleet.
-2. **Backup check** — confirm a Proxmox snapshot of the primary host exists OR `/tmp/infra-maintenance-backup-<date>/` was created this run. Snapshot critical config files into the backup dir:
-   - `/opt/inference-harness/docker-compose.yml`, `/opt/inference-harness/litellm_config.yaml` (CT 116)
-   - `/opt/search-stack/searxng/docker-compose.yml`, `/opt/search-stack/firecrawl-source/docker-compose.yaml` (VM 109)
-3. **Record image baseline** — `docker inspect --format '{{.Image}} {{.Config.Image}}' <container>` for every running container on .116 and .7; store digests in `image-baseline` so rollback can restore them.
-4. **Disk check** — `df -h` on each mutated host; abort if free space <20% (apt upgrade + image pulls need headroom).
-
-### Phase 1 — OS package updates (primary host)
-
-```bash
-# On the primary host only (VERIFY host against infrastructure-control first)
-apt update
-apt upgrade -y
-```
-
-- Capture `apt list --upgradable` before and after → store in `apt-state`.
-- If apt reports held/broken packages (`apt-get -s upgrade | grep -i broken`, or non-zero exit), **stop** — do not force. Record `held_broken` and go to rollback/escalate.
-- If `/var/run/reboot-required` exists after upgrade, flag `kernel_reboot_required: true` in `apt-state` but **do not reboot automatically** — that's a separate coordinated action (see infra-update Wave 4). Note it in the report.
-
-### Phase 2 — Docker image pulls
-
-Pull latest stable tags for every running container. Do NOT pin to `:main`/`:nightly` — use stable tags where the compose file specifies them; otherwise `latest`.
-
-```bash
-# CT 116 (.116) — inference harness
-cd /opt/inference-harness && docker compose pull
-
-# VM 109 (.7) — search + home stacks
-cd /opt/search-stack/searxng && docker compose pull
-cd /opt/search-stack/firecrawl-source && docker compose pull
-# any other running stacks on .7 (home stack, audiobookshelf) — pull per their compose files
-```
-
-- LiteLLM and SearXNG are the two explicitly required pulls; "any other running containers" means every stack with a compose file on .116 and .7.
-- Record pulled tag + digest per service in `image-baseline`.
-
-### Phase 3 — Container restarts with health verification
-
-Restart one stack at a time, verify health before moving to the next. Do not restart everything at once — a failure mid-wave must leave the rest running.
-
-```bash
-# CT 116
-cd /opt/inference-harness && docker compose up -d
-# VM 109
-cd /opt/search-stack/searxng && docker compose up -d
-cd /opt/search-stack/firecrawl-source && docker compose up -d
-```
-
-After each stack comes up, wait for health (max 120s):
-- `docker ps` shows the container `Up` (and `healthy` if a healthcheck is defined)
-- Service-specific probe passes (see Phase 4 probes)
-
-If a stack fails to come up within 120s, **stop the wave** and go to rollback for that stack only; do not proceed to the next.
-
-### Phase 4 — Post-update service verification
-
-After ALL updates (apt + images + restarts), verify every critical service is back up and matches the pre-update baseline. This is the regression gate.
-
-| Service | Probe | Expect |
-|---------|-------|--------|
-| LiteLLM proxy | `curl -sf http://192.168.68.116/litellm/v1/models` | 200 OK, models returned |
-| LiteLLM MCP gateway | `curl -sf http://192.168.68.116:4000/mcp-rest/tools/list -H "Authorization: Bearer $MASTER_KEY"` | 90 tools (23 RA-H OS + 67 GitHub) |
-| SearXNG | `curl -sf http://192.168.68.7:8888` | 200 OK |
-| Zulip | `curl -sf https://chat.sysloggh.net/api/v1/server_settings` | 200 OK |
-| Gitea | `curl -sf https://git.sysloggh.net/api/v1/version` | 200 OK |
-| PM2 processes | `pm2 jlist` (CT 100) | all pi-agent processes `online` |
-| Hermes gateways | SSH to Mumuni CT 114, Tanko CT 112; `systemctl is-active hermes-gateway` | `active` for each |
-
-Regression check: every service that was GREEN in `health-baseline` must still be GREEN. A service that was already RED (and caused a preflight abort) is excluded — but Phase 0 should have aborted before we got here.
-
-## Rollback Protocol
-
-If ANY service in Phase 4 fails to come back up (or regresses vs baseline):
-
-1. **Image rollback** — for the failing stack, restore the previous image:
-   ```bash
-   # Restore from recorded image-baseline digest
-   docker compose down
-   # Pin the service image to the recorded digest in compose, then recreate
-   #   image: <name>@sha256:<previous_digest>
-   docker compose pull && docker compose up -d
-   ```
-2. **APT rollback** — restore the primary host from the Proxmox snapshot taken/confirmed in Phase 0. If no snapshot, `apt install <pkg>=<old_version>` per package using apt history (`/var/log/apt/history.log`).
-3. **Config rollback** — restore configs from `/tmp/infra-maintenance-backup-<date>/`.
-4. **Re-verify** — re-run the Phase 4 probes on the rolled-back service. If still failing, escalate (do not loop — circuit breaker below).
-5. **Escalate** — send a Zulip DM to abiba + mumuni with: failing service, phase, baseline vs current, rollback actions taken, backup path.
-
-## Circuit Breaker
-
-- `max_retries: 2` per failing phase — after 2 rollback attempts on the same service, stop and escalate.
-- `window: 7200` seconds — no more than 2 retries within a 2-hour window.
-- `trip_action: escalate_to_fatal` — when tripped, escalate to fatal (abiba + mumuni + kwame) and pause; a human must clear before the next scheduled run.
-
-## Report
-
-After completion (or on abort), emit a receipt (JSON) to `~/.hermes/runs/infrastructure-maintenance/` and send a Zulip DM summary:
-
-```
-🛠 Infrastructure Maintenance — YYYY-MM-DD
-
-Phase: apt | images | restarts | verify | rollback
-Primary host: <host>
-APT: <N> packages upgraded, <M> held/broken, kernel_reboot_required=<bool>
-Images pulled: LiteLLM <tag>, SearXNG <tag>, <others>
-Services: all GREEN | <service> FAILED (rolled back)
-Baseline regression: none | <details>
-Backup: /tmp/infra-maintenance-backup-YYYYMMDD/
-Escalation: none | warning | critical | fatal
-```
-
-## Verification Postconditions
-
-- All critical services running after update (Phase 4 all GREEN)
-- No regressions from pre-update health baseline (Phase 0 baseline)
-- Docker containers on latest stable tags (`image-baseline.pulled_tag` recorded)
-- APT packages up to date with no held broken packages (`apt-state.held_broken == 0`)
-
-## Related Contracts
-
-- `infrastructure-update.prose.md` — owns the full-fleet cluster-wide wave INCLUDING its Wave 3 Docker image updates across all stacks (SearXNG, Firecrawl, Inference Harness on .116, home stack, audiobookshelf); infrastructure-maintenance is a deliberately narrower host-level weekly pull scoped to .116 and .7.
-- `infrastructure-monitoring.prose.md` — provides the pre-update health baseline (depends_on).
-- `infrastructure-control.prose.md` — topology source of truth (CT IDs, IPs, hostnames).
-- `litellm-health.prose.md` — LiteLLM probe details.
-- `proxmox-monitor.prose.md` — Docker stats + monitoring stack health.

litellm-api-keys.prose.md

@@ -171,7 +171,7 @@ through its agent wrapper.
 - **Survives gateway crash**: the wrapper's `while true` + systemd `Restart=always` revive the gateway. Two-layer defense.
 - **Survives Hermes updates**: systemd drop-in overrides unit file ExecStart — `hermes gateway install` cannot break the vault injection.
 - **Survives reboot**: systemd user service + `loginctl enable-linger` ensures gateway starts at boot without a login session.
-- **Auditable**: `cat /proc/$(pgrep -f 'python.*hermes_cli.main.gateway.run' | grep -v infisical | head -1)/environ` shows all injected keys (note: pipe through grep -v infisical to avoid matching the bash wrapper); `infisical secrets` shows the vault source.
+- **Auditable**: `cat /proc/$(pgrep hermes_cli)/environ` shows all injected keys; `infisical secrets` shows the vault source.
 
 ### Migration status (2026-07-17)
 

scripts/agent-health-check.py

@@ -184,8 +184,8 @@ def check_agents():
             print(f"  ⬜ {name} (CT {ct}): cannot SSH — skip liveness check")
             continue
 
-        # Gateway process (exclude the infisical bash wrapper that contains the same string)
-        pid = ssh(host, "pgrep -f 'hermes_cli.main gateway run' | grep -v infisical | head -1", user=user)
+        # Gateway process
+        pid = ssh(host, "pgrep -f 'hermes_cli.main gateway run' | head -1", user=user)
         if not pid:
             print(f"  ❌ {name}: GATEWAY NOT RUNNING")
             FAIL.append(f"gateway-down:{name}")

zulip-resilience-v3.prose.md

@@ -409,68 +409,3 @@ Backup v2 before starting: `cp index.js index.js.v2-backup-$(date +%Y%m%d-%H%M%S
 | Queue expiry handling | Crash | Auto re-register |
 | Busy worker deadlock | Router death | Worker SIGKILL + error DM |
 | PM2 restart exhaustion | Yes (max_restarts=10) | No (max_restarts=100 + watchdog) |
-
----
-
-## Incident Log — 2026-07-18 Fleet-Wide Audit
-
-### Fleet State After Audit
-
-| Agent | Platform | Zulip State | Issues Found | Fix Applied |
-|-------|----------|-------------|--------------|-------------|
-| **Abiba** | pi (CT 100) | ✅ Connected | API key missing from Infisical injection; poll timeout noise | Added .env fallback; AbortError treated as empty poll (no retry); poll timeout 65s→90s |
-| **Tanko** | Hermes (CT 112) | ✅ Connected | Gateway disconnected since Jul 11; watchdog restart didn't re-establish Zulip | Full gateway restart (kill wrapper, let infisical-gateway.sh respawn) |
-| **Mumuni** | Hermes (CT 114) | ✅ Connected | No issues found | None needed |
-
-### Key Fixes Applied
-
-**1. Abiba — Credential Fallback (L4 Pattern)**
-- Root cause: `zulip.api_key` in config.yaml is `""` (expected from Infisical). Infisical vault `ABIBA_ZULIP_API_KEY` wasn't being injected into the process environment.
-- Fix: Added `.env` file fallback at `/root/.pi/agent/extensions/zulip/.env` with known-working key, sourced before the Infisical `exec`.
-- Lesson: Per L4 from gpu-self-heal, Infisical is not always available — always keep a local `.env` fallback.
-
-**2. Abiba — Poll Timeout Handling**
-- Root cause: Zulip long-poll uses `AbortSignal.timeout(65000)`. Zulip's default `event_queue_longpoll_timeout_seconds` can exceed 65s. When the signal fires, an `AbortError` is thrown and caught by the circuit breaker as a failure.
-- Fix: Caught `AbortError` inside `poll()` and return empty array (no events) instead of throwing. Extended timeout to 90s to match Zulip server default.
-- Reference: [Zulip Events System — long-poll timeout](https://zulip.readthedocs.io/en/11.6/subsystems/events-system.html)
-
-**3. Tanko — Gateway Restart**
-- Root cause: Gateway process was running but Zulip platform stayed in "disconnected" state since Jul 11, 2026. The wrapper script (`infisical-gateway.sh`) restarts on crash but the gateway wasn't re-establishing Zulip on restart.
-- Fix: Killed gateway PID to trigger wrapper restart. New gateway (PID 331991) established Zulip connection successfully.
-
-### Fleet-Wide Zulip Health Metrics (as of 2026-07-18)
-
-| Metric | Value |
-|--------|-------|
-| Zulip server | ✅ HTTP 200 |
-| Agents connected | 3/3 (Abiba, Tanko, Mumuni) |
-| Abiba circuit breaker | CLOSED (0 failures) |
-| Abiba uptime | 2D (post-restart) |
-| Tanko gateway uptime | Ongoing |
-| Mumuni gateway uptime | Ongoing |
-| Watchdog status | ✅ Online (2D uptime) |
-
-### Hermes Agent Zulip Plugin Improvements
-
-Based on the audit, improvements that should be ported to all Hermes Zulip adapters:
-
-1. **Circuit breaker pattern** — Already in Abiba's pi extension. Hermes adapters should add the same CLOSED→OPEN→HALF_OPEN state machine with exponential backoff.
-2. **Credential fallback** — All Hermes agents use Infisical for credentials. Add `.env` local fallback per L4 pattern for `ZULIP_API_KEY`.
-3. **Queue re-registration** — Handle `BAD_EVENT_QUEUE_ID` with automatic re-registration instead of gateway restart.
-4. **Supervisor watchdog** — Hermes uses PM2 which auto-restarts on crash, but has no health-check watchdog. Add lightweight external health checks.
-5. **Streaming** — All agents have `streaming: true` in their zulip config. Verify `edit_message()` is implemented in each adapter.
-
-### Abiba pi Zulip Extension v2 — Implemented Resilience Summary
-
-| Feature | Status | Notes |
-|---------|--------|-------|
-| Circuit breaker | ✅ | CLOSED→OPEN→HALF_OPEN; 50% failure threshold; 30s reset timeout |
-| Retry with jitter | ✅ | 2 attempts, 200ms base, 50-100% jitter |
-| Queue lifecycle | ✅ | 10min idle_queue_timeout; BAD_EVENT_QUEUE_ID handling |
-| Crash prevention | ✅ | uncaughtException + unhandledRejection recovery |
-| Worker timeout | ✅ | 5min busy timeout → SIGKILL + error DM |
-| Health endpoint | ✅ | :9200 with circuit breaker metrics |
-| Echo prevention | ✅ | Dynamic bot user resolution |
-| Poll timeout (AbortError) | ✅ v2.1 | Normal timeout returns [] instead of error |
-| Credential fallback | ✅ v2.1 | .env file before Infisical exec |
-| Provider auto-fix | ✅ | Detects reasoning_content models, switches to compatible |