hermes-config-template: update to v2.1.0 — max_tokens, auxiliary consistency, compression threshold

- Add model.max_tokens: 4096 (thermal safety — Rule 6) - Add context_length: 262144 to template - Add Rule 7: Auxiliary model consistency (gemma-4-12b + base_url + api_key_env) - Add Rule 8: Compression threshold for 256K models (0.65, not 0.25) - Fix compression section: threshold 0.65, max_context_window 262144, protect_last_n 40 - Remove stray api_key_env from compression block (not propagated) - Update agent Keys table to show key aliases instead of redacted values - Add base_url and timeout to all auxiliary service configs - Add auxiliary: compression: section matching main compression config - Update version 2.0.0 → 2.1.0
2026-07-04 17:54:43 -04:00
parent 253e19f680
commit 0a3b62a598
1 changed files with 69 additions and 26 deletions
@@ -10,26 +10,28 @@ description: >

 ## Maintains

- template_version: "2.0.0"
+- template_version: "2.1.0"
 - last_applied: timestamp
 - agents_configured: ["tanko", "mumuni", "abiba", "koby", "koonimo", "kagenz0"]
 - agent_keys: map (see Agent Keys section)
 - infra_endpoints_verified: array

-## Agent Keys (LiteLLM — Current 2026-06-30)
+## Agent Keys (LiteLLM — Current 2026-07-04)

-Each agent has a unique LiteLLM API key. Keys are stored in LiteLLM's PostgreSQL DB.
-The env var `LITELLM_API_KEY` is set in `/etc/environment` on each agent host.
+Each agent has a unique LiteLLM API key (virtual key) generated against the LiteLLM
+PostgreSQL DB via `POST /key/generate` on CT 116. Keys are stored in the DB, not in
+config files. The env var `LITELLM_API_KEY` is set in `/etc/environment` on each agent
+host AND in `~/.hermes/.env` for gateway env propagation.
 Sub-agent profiles inherit auth from the main config — no separate keys needed.

-| Agent | Key | Host | SSH | Sub-Agents |
-|-------|-----|------|-----|-----------|
-| Tanko | `sk-3ZsdWJbbNSo9zSnJN2OsJw` | 192.168.68.122 | jerome@.122 | — |
-| Mumuni | `sk-XY2aUfvy2BIs6kp1ZPh6VA` | 192.168.68.123 | root@.123 | 6 profiles ✱ |
-| Abiba | `sk-kxbPgbnV2Zkn6TKQbAEcEg` | 192.168.68.24 | local | — |
-| Koby | `sk-wb86PvcZXWkjShAIBxrpDg` | ? | Zulip | — |
-| Koonimo | `sk-Prx_vRXYb2VEzDmhPCbNeg` | 192.168.68.114 | Zulip | — |
-| Kagenz0 | `sk-Dh4CDkaHebMLEp8qqq20qA` | ? | Zulip | — |
+| Agent | Key Alias | Host | SSH | Sub-Agents |
+|-------|-----------|------|-----|-----------|
+| Tanko | `tanko-*` | 192.168.68.122 | jerome@.122 | — |
+| Mumuni | `mumuni-jul2026` | 192.168.68.123 | root@.123 | 6 profiles ✱ |
+| Abiba | `abiba-*` | 192.168.68.24 | local | — |
+| Koby | `koby-*` | ? | Zulip | — |
+| Koonimo | `koonimo-*` | 192.168.68.114 | Zulip | — |
+| Kagenz0 | `kagenz0-*` | ? | Zulip | — |

 ✱ Mumuni sub-agents: syslog-code, syslog-devops, syslog-email, syslog-research,
  syslog-review, syslog-writer — all at `/root/.hermes/profiles/<name>/config.yaml`
@@ -85,10 +87,12 @@ work immediately after restart.
 ```yaml
 # ─── Model Selection ───
 model:
-  default: <agent_model>          # e.g., qwen3.6-27B-code, syslog-auto
+  default: <agent_model>          # e.g., ornith-1.0-35b, qwen3.6-27B-code
  provider: harness
  base_url: http://192.168.68.116/v1
-  api_key_env: LITELLM_API_KEY    # Set in /etc/environment
+  api_key_env: LITELLM_API_KEY    # Set in /etc/environment AND ~/.hermes/.env
+  max_tokens: 4096                # ⚠️ CRITICAL: Prevents unbounded generation
+  context_length: 262144          # Must match model's max context window

 fallback_providers:
  provider: deepseek
@@ -117,29 +121,44 @@ mcp_servers:
 # ─── Compression ───
 compression:
  enabled: true
+  model: gemma-4-12b              # ⚠️ Must match auxiliary.compression.model
  provider: harness
-  model: gemma-4-12b
-  api_key_env: LITELLM_API_KEY
-  threshold: 0.5
-  target_ratio: 0.25
-  protect_last_n: 30
-  hygiene_hard_message_limit: 400
+  max_context_window: 262144      # Must match model's actual capacity
+  threshold: 0.65                 # Fires at ~170K for 262K window (not 0.25!)
+  target_ratio: 0.30
+  protect_last_n: 40
+  hygiene_hard_message_limit: 350
+  protect_first_n: 3
+  abort_on_summary_failure: false

-# ─── Auxiliary Tasks ───
+# ─── Auxiliary Tasks (CONSISTENCY RULE) ───
+# All auxiliary services MUST use identical model, base_url, and api_key_env:
+#   model: gemma-4-12b
+#   base_url: http://192.168.68.116/v1
+#   api_key_env: LITELLM_API_KEY
+# Do NOT use syslog-auto for auxiliary tasks — it routes to the primary GPU.
+# gemma-4-12b is a lightweight 12B model on the RTX 5070, freeing the Strix Halo
+# for agent reasoning.
 auxiliary:
  vision:
    provider: harness
    model: gemma-4-12b
+    base_url: http://192.168.68.116/v1
    api_key_env: LITELLM_API_KEY
+    timeout: 60
+    download_timeout: 30
  web_extract:
    provider: harness
    model: gemma-4-12b
+    base_url: http://192.168.68.116/v1
    api_key_env: LITELLM_API_KEY
-  session_search:
+    timeout: 30
+  compression:
    provider: harness
    model: gemma-4-12b
-    max_concurrency: 3
+    base_url: http://192.168.68.116/v1
    api_key_env: LITELLM_API_KEY
+    timeout: 60

 # ─── Custom Provider ───
 custom_providers:
@@ -189,9 +208,33 @@ The following MUST be identical across ALL profiles:
 - This means key rotation only touches ONE file (`/etc/environment`)

 ### Rule 5: Main Config Base URL
- Use direct IP: `http://192.168.68.116/v1`
- NOT the NetBird URL (`litellm.sysloggh.net`) — can cause 502 when NetBird is down
- NOT the old path (`/litellm/v1`) — nginx now routes `/v1` directly
+|- Use direct IP: `http://192.168.68.116/v1`
+|- NOT the NetBird URL (`litellm.sysloggh.net`) — can cause 502 when NetBird is down
+|- NOT the old path (`/litellm/v1`) — nginx now routes `/v1` directly
+
+### Rule 6: max_tokens Is Required (Thermal Safety)
+- **Every Hermes config MUST set `model.max_tokens: 4096`** — this is non-negotiable
+- Prevents unbounded generation that caused the July 2 Strix Halo GPU thermal incident
+- The value flows through: `config → agent.max_tokens → transport build_kwargs → API max_tokens`
+- Even though the server now has `-n 8192` hard cap (set by Abiba), the client cap is the first line of defense
+- Apply to BOTH main config AND all sub-agent profiles
+- For agents needing longer outputs: raise to 8192, but never omit
+
+### Rule 7: Auxiliary Model Consistency
+- All auxiliary services (vision, web_extract, compression) MUST use the same model:
+  - `model: gemma-4-12b`
+  - `base_url: http://192.168.68.116/v1`
+  - `api_key_env: LITELLM_API_KEY`
+- **Do NOT use `syslog-auto`** for auxiliary tasks — it routes to the primary 35B reasoning GPU
+- gemma-4-12b is a lightweight 12B model on the RTX 5070, keeping the Strix Halo free for reasoning
+- The `compression:` block's `model` MUST match `auxiliary: compression: model` — they are two different configs for the same service
+
+### Rule 8: Compression Threshold for 256K Models
+- For 262K context window: `threshold: 0.65` (fires at ~170K tokens)
+- Do NOT use `threshold: 0.25` — this fires at 65K, causing premature context loss
+- Do NOT use `threshold: 0.80` — this delays until 209K, risking the gateway hygiene layer
+- `max_context_window: 262144` MUST match the model's actual capacity
+- See `devops-hermes-compression` skill for full reference

 ## Execution