Contracts: Mumuni context-fix (WAL #1300) — strix-moe, 256K all GPUs, Rule 12/13, machine-identity vault procedure
This commit is contained in:
Abiba
@@ -5,10 +5,12 @@ description: >
|
||||
Standard Hermes configuration template for Syslog Solution LLC agents.
|
||||
Enforces shared infrastructure setup (Firecrawl, SearXNG, local models,
|
||||
RA-H OS MCP) while keeping agent-specific API keys and model choices.
|
||||
UPDATED 2026-07-12: GPU workload redistributed. Compression → Strix Halo (ornith-1.0-35b).
|
||||
RTX 3090 context verified at 256K (was incorrectly documented as 128K).
|
||||
RTX 5070 stays at 131K for vision/web. Infisical .env fallback required (Rule 3).
|
||||
Parallel counts corrected (RTX 3090=1, RTX 5070=2, Strix=2).
|
||||
UPDATED 2026-07-16: Compression model is the stable alias `strix-moe` (NOT `ornith-1.0-35b`,
|
||||
which LiteLLM does not serve). All 3 GPUs verified 256K (RTX 5070 bumped 131K→256K on Jul 15).
|
||||
Added Rule 12 (Context-Issue Diagnostic) + Rule 13 (.env fallback enforcement) from the
|
||||
2026-07-16 Mumuni root-cause investigation (WAL #1300).
|
||||
UPDATED 2026-07-12: GPU workload redistributed. Compression → Strix Halo. RTX 3090 context
|
||||
verified at 256K. Infisical .env fallback required (Rule 3/13).
|
||||
---
|
||||
|
||||
## Maintains
|
||||
@@ -31,7 +33,7 @@ Sub-agent profiles inherit auth from the main config — no separate keys needed
|
||||
| Agent | Key Alias | Host | SSH | Sub-Agents |
|
||||
|-------|-----------|------|-----|-----------|
|
||||
| Tanko | `tanko-*` | 192.168.68.122 | jerome@.122 | — |
|
||||
| Mumuni | `mumuni-jul2026` | 192.168.68.123 | root@.123 | 6 profiles ✱ |
|
||||
| Mumuni | `mumuni` | 192.168.68.123 | root@.123 | 6 profiles ✱ |
|
||||
| Abiba | `abiba-pi` | 192.168.68.24 | local | — |
|
||||
| Koby | `koby` | CT 111 (tdunna) | Zulip | — |
|
||||
| Koonimo | `koonimo` | CT 113 (baggy) | Zulip | — |
|
||||
@@ -129,10 +131,9 @@ mcp_servers:
|
||||
# ─── Compression ───
|
||||
compression:
|
||||
enabled: true
|
||||
model: ornith-1.0-35b # ⚠️ Must match auxiliary.compression.model
|
||||
model: strix-moe # ⚠️ Must match auxiliary.compression.model. Stable alias (gpu-fleet § Stable Role-Based Aliases). NOT ornith-1.0-35b (LiteLLM does not serve that name).
|
||||
provider: harness
|
||||
max_context_window: 262144 # For Strix Halo compression (256K ctx).
|
||||
# Set 131072 if using gemma directly.
|
||||
max_context_window: 262144 # MUST match actual GPU capacity. All 3 GPUs are 256K (Jul 15).
|
||||
threshold: 0.65 # Fires at ~170K for 262K window, ~85K for 128K
|
||||
target_ratio: 0.30
|
||||
protect_last_n: 40
|
||||
@@ -164,10 +165,10 @@ auxiliary:
|
||||
timeout: 30
|
||||
compression:
|
||||
provider: harness
|
||||
model: ornith-1.0-35b
|
||||
base_url: http://192.168.68.116/v1
|
||||
model: strix-moe # MUST match compression.model above. Stable alias for Strix Halo.
|
||||
base_url: http://192.168.68.116/v1 # Rule 5: /v1 NOT /litellm/v1
|
||||
api_key_env: LITELLM_API_KEY
|
||||
timeout: 60
|
||||
timeout: 300 # gpu-fleet: 300s for large-history summarization (was 60)
|
||||
|
||||
# ─── Custom Provider ───
|
||||
custom_providers:
|
||||
@@ -236,23 +237,25 @@ The following MUST be identical across ALL profiles:
|
||||
- Apply to BOTH main config AND all sub-agent profiles
|
||||
- For agents needing longer outputs: raise to 8192, but never omit
|
||||
|
||||
### Rule 7: Auxiliary Model Consistency (UPDATED July 2026)
|
||||
### Rule 7: Auxiliary Model Consistency (UPDATED 2026-07-16)
|
||||
- Vision and web_extract use `gemma-4-12b` (RTX 5070 — 12GB, vision-optimized)
|
||||
- Compression uses `ornith-1.0-35b` (Strix Halo — 64GB, 256K ctx, compression-optimized)
|
||||
- Compression uses `strix-moe` (stable alias for Strix Halo — 64GB, 256K ctx, compression-optimized)
|
||||
- **`strix-moe` is the only valid compression model name** — LiteLLM does NOT serve `ornith-1.0-35b`
|
||||
(it serves `strix-moe`, `qwen3.6-35B-udq4`, `gpu-dense`, `gpu-light`, `syslog-auto`, `gemma-4-12b`, `qwen3.6-27B-code`). Old configs with `ornith-1.0-35b` cause 403/model-not-found on compression calls.
|
||||
- All auxiliary services MUST use identical routing:
|
||||
- `base_url: http://192.168.68.116/v1`
|
||||
- `base_url: http://192.168.68.116/v1` (Rule 5: `/v1`, NOT `/litellm/v1`)
|
||||
- `api_key_env: LITELLM_API_KEY`
|
||||
- **Do NOT use `syslog-auto`** for auxiliary tasks — it routes unpredictably
|
||||
- **Compression moved to Strix Halo (July 2026)**: The ornith-1.0-35b model on Strix Halo
|
||||
(64GB UMA, 256K context, 72.4 tok/s) is the designated compression GPU. This frees the
|
||||
- **Compression on Strix Halo**: The strix-moe alias routes to Strix Halo
|
||||
(64GB UMA, 256K context) — the designated compression GPU. This frees the
|
||||
RTX 5070 for vision and web search, and the RTX 3090 for heavy reasoning.
|
||||
- The `compression:` block's `model` MUST match `auxiliary: compression: model`
|
||||
- The `compression: max_context_window: 262144` MUST match Strix Halo's actual capacity
|
||||
|
||||
### Rule 8: GPU Workload Distribution (July 2026)
|
||||
### Rule 8: GPU Workload Distribution (UPDATED 2026-07-16)
|
||||
- **RTX 3090 (24GB, 256K ctx, qwen3.6-27B-code)**: Heavy reasoning, code gen, long conversations
|
||||
- **RTX 5070 (12GB, 131K ctx, gemma-4-12b)**: Vision, web search, quick tasks, web_extract
|
||||
- **Strix Halo (64GB, 256K ctx, ornith-1.0-35b)**: Context compression, summarization, long docs
|
||||
- **RTX 5070 (12GB, 256K ctx, gemma-4-12b)**: Vision, web search, quick tasks, web_extract (bumped 131K→256K Jul 15; IQ4_NL+MTP, 88% VRAM)
|
||||
- **Strix Halo (64GB, 256K ctx, strix-moe)**: Context compression, summarization, long docs
|
||||
- Agent profiles MUST route auxiliary tasks to the correct GPU:
|
||||
- `auxiliary.vision.model: gemma-4-12b` (RTX 5070)
|
||||
- `auxiliary.web_extract.model: gemma-4-12b` (RTX 5070)
|
||||
@@ -293,6 +296,48 @@ The following MUST be identical across ALL profiles:
|
||||
- A non-existent model ID causes 403 errors that silently break the pi RPC worker
|
||||
(no `agent_end` emitted, worker stays "busy", Zulip messages pile up unprocessed)
|
||||
|
||||
### Rule 12: Context-Issue Diagnostic Checklist (ADDED 2026-07-16, WAL #1300)
|
||||
When an agent shows "context issues" (premature compression, 401s, 504s, DeepSeek fallback),
|
||||
verify ALL FOUR of these against the live config. They are the only root causes found in production:
|
||||
|
||||
1. **max_context_window correct?** — BOTH `compression.max_context_window` AND `context.max_context_window`
|
||||
MUST be `262144` (all GPUs are 256K). A value of `131072` causes premature compression at
|
||||
~83K instead of ~170K. Check: `grep -n max_context_window ~/.hermes/config.yaml`
|
||||
2. **base_url uses /v1 NOT /litellm/v1?** — `custom_providers[0].base_url`, `delegation.base_url`,
|
||||
and ALL `auxiliary.*.base_url` MUST be `http://192.168.68.116/v1` (Rule 5). nginx `/litellm/`
|
||||
has a 60s default timeout → 504 on any inference >60s; `/v1/` has 600s.
|
||||
Check: `grep -n 'litellm/v1' ~/.hermes/config.yaml` (must return NOTHING)
|
||||
3. **LITELLM_API_KEY valid?** — The key must be a real LiteLLM key (`sk-` + 64 hex, 67 chars).
|
||||
Malformed values (e.g. `sk-_SWAl_Vu_…`, 47 chars) return 401 → DeepSeek fallback.
|
||||
Verify: `curl -s -o /dev/null -w '%{http_code}' -H "Authorization: Bearer $LITELLM_API_KEY" http://192.168.68.116/v1/models` (must be 200)
|
||||
4. **custom_providers aligned?** — `model: syslog-auto` (Rule 10), `api_mode: chat_completions`
|
||||
(NOT `responses`). A wrong api_mode causes silent request failures.
|
||||
|
||||
One-line agent health check (run on the agent host):
|
||||
```bash
|
||||
PID=$(pgrep -f "python -m hermes_cli.main gateway run" | head -1)
|
||||
cat /proc/$PID/environ | tr '\0' '\n' | grep ^LITELLM_API_KEY= | sed 's/=.*/<set>/'
|
||||
curl -s -o /dev/null -w 'key_health: %{http_code}\n' -H "Authorization: Bearer $(cat /proc/$PID/environ | tr '\0' '\n' | grep ^LITELLM_API_KEY= | cut -d= -f2)" http://192.168.68.116/v1/models
|
||||
```
|
||||
|
||||
### Rule 13: .env Fallback Enforcement (ADDED 2026-07-16, WAL #1300)
|
||||
- The infisical-gateway.sh wrapper MUST source `~/.hermes/.env` BEFORE exporting `LITELLM_API_KEY`,
|
||||
so the local .env overrides a stale Infisical vault value:
|
||||
```bash
|
||||
-- bash -c "
|
||||
. /root/.hermes/.env # [FALLBACK Rule 3/13] local .env overrides stale vault value
|
||||
export LITELLM_API_KEY=\"\$MUMUNI_LITELLM_API_KEY\"
|
||||
...
|
||||
```
|
||||
- `~/.hermes/.env` MUST contain `MUMUNI_LITELLM_API_KEY=sk-<valid>` (or the agent's equivalent).
|
||||
- This is sanctioned by Rule 3: "Infisical service tokens can expire/404; the .env fallback prevents
|
||||
agents from running without keys."
|
||||
- The systemd unit `hermes-gateway.service` ExecStart MUST be `/root/.hermes/infisical-gateway.sh`
|
||||
(the wrapper), NOT `python -m hermes_cli.main gateway run` directly (direct exec breaks ALL env
|
||||
injection → "No messaging platforms enabled").
|
||||
- To sync the Infisical vault (when write access is available): see `litellm-api-keys.prose.md`
|
||||
§ Machine Identity for Vault Writes.
|
||||
|
||||
## Execution
|
||||
|
||||
1. **Check current config** — Read the target agent's config.yaml
|
||||
|
||||
Reference in New Issue
Block a user