From 6570fd60e7e16e74f6ff4613d1c840c4c80f0a9e Mon Sep 17 00:00:00 2001 From: root Date: Sat, 18 Jul 2026 23:28:07 +0000 Subject: [PATCH] tune: switch compression model from strix-moe to syslog-auto Relieve Strix Halo pressure by distributing compression across the syslog-auto weighted pool (55% RTX 3090, 30% Strix Halo, 15% RTX 5070). Updates: - compression.model: strix-moe -> syslog-auto - auxiliary.compression.model: strix-moe -> syslog-auto - Rule 7: Updated for syslog-auto compression, removed aux prohibition - Rule 8: Updated GPU workload distribution - All docs/comments updated to reflect the change --- hermes-config-template.prose.md | 51 +++++++++++++++++++-------------- 1 file changed, 29 insertions(+), 22 deletions(-) diff --git a/hermes-config-template.prose.md b/hermes-config-template.prose.md index 693026a..a91db3d 100644 --- a/hermes-config-template.prose.md +++ b/hermes-config-template.prose.md @@ -5,11 +5,14 @@ description: > Standard Hermes configuration template for Syslog Solution LLC agents. Enforces shared infrastructure setup (Firecrawl, SearXNG, local models, RA-H OS MCP) while keeping agent-specific API keys and model choices. - UPDATED 2026-07-16: Compression model is the stable alias `strix-moe` (NOT `ornith-1.0-35b`, + UPDATED 2026-07-18: Compression model switched to `syslog-auto` (was `strix-moe`) + to relieve Strix Halo pressure. syslog-auto distributes compression across the + weighted pool (55% RTX 3090, 30% Strix Halo, 15% RTX 5070). + UPDATED 2026-07-16: Compression model was the stable alias `strix-moe` (NOT `ornith-1.0-35b`, which LiteLLM does not serve). All 3 GPUs verified at 128K (reduced from 256K 2026-07-17 for stability). Added Rule 12 (Context-Issue Diagnostic) + Rule 13 (.env fallback enforcement) from the 2026-07-16 Mumuni root-cause investigation (WAL #1300). - UPDATED 2026-07-12: GPU workload redistributed. Compression → Strix Halo. RTX 3090 context verified at 128K. Infisical .env fallback required (Rule 3/13). + UPDATED 2026-07-12: GPU workload redistributed. Compression → Strix Halo (later switched to syslog-auto 2026-07-18). RTX 3090 context verified at 128K. Infisical .env fallback required (Rule 3/13). --- ## Maintains @@ -130,7 +133,9 @@ mcp_servers: # ─── Compression ─── compression: enabled: true - model: strix-moe # ⚠️ Must match auxiliary.compression.model. Stable alias (gpu-fleet § Stable Role-Based Aliases). NOT ornith-1.0-35b (LiteLLM does not serve that name). + model: syslog-auto # ⚠️ Switched from strix-moe 2026-07-18 to relieve Strix Halo. + # syslog-auto distributes across weighted pool (55% RTX 3090, + # 30% Strix Halo, 15% RTX 5070). All GPUs at 128K. provider: harness max_context_window: 131072 # MUST match actual GPU capacity. All 3 GPUs are 128K (Jul 17). threshold: 0.65 # Fires at ~170K for 262K window, ~85K for 128K @@ -145,8 +150,9 @@ compression: # model: gpu-light # stable alias (NOT raw "gemma-4-12b") # base_url: http://192.168.68.116/v1 # api_key_env: LITELLM_API_KEY -# Do NOT use syslog-auto for auxiliary tasks — it routes to the primary GPU. -# gpu-light = RTX 5070 (12B), freeing the Strix Halo for agent reasoning. +# Compression uses syslog-auto (switched from strix-moe 2026-07-18) to distribute +# load across the weighted pool and relieve Strix Halo pressure. +# Vision and web_extract use gpu-light = RTX 5070 (12B). # Heavy aux (delegation, x_search) use gpu-dense (RTX 3090) instead. # NEVER use raw model names (gemma-4-12b, qwen3.6-27B-code, qwen3.6-35B-udq4) # in agent configs — use the stable aliases so model swaps don't break agents. @@ -166,7 +172,7 @@ auxiliary: timeout: 30 compression: provider: harness - model: strix-moe # MUST match compression.model above. Stable alias for Strix Halo. + model: syslog-auto # Switched from strix-moe 2026-07-18. Relieves Strix Halo pressure. base_url: http://192.168.68.116/v1 # Rule 5: /v1 NOT /litellm/v1 api_key_env: LITELLM_API_KEY timeout: 300 # gpu-fleet: 300s for large-history summarization (was 60) @@ -247,29 +253,30 @@ The following MUST be identical across ALL profiles: - Apply to BOTH main config AND all sub-agent profiles - For agents needing longer outputs: raise to 8192, but never omit -### Rule 7: Auxiliary Model Consistency (UPDATED 2026-07-16) -- Vision and web_extract use `gemma-4-12b` (RTX 5070 — 12GB, vision-optimized) -- Compression uses `strix-moe` (stable alias for Strix Halo — 64GB, 128K ctx, compression-optimized) -- **`strix-moe` is the only valid compression model name** — LiteLLM does NOT serve `ornith-1.0-35b` - (it serves `strix-moe`, `qwen3.6-35B-udq4`, `gpu-dense`, `gpu-light`, `syslog-auto`, `gemma-4-12b`, `qwen3.6-27B-code`). Old configs with `ornith-1.0-35b` cause 403/model-not-found on compression calls. +### Rule 7: Auxiliary Model Consistency (UPDATED 2026-07-18) +- Vision and web_extract use `gpu-light` (stable alias, RTX 5070 — 12GB, vision-optimized) +- Compression now uses `syslog-auto` (switched from `strix-moe` 2026-07-18) to distribute + compression load across the weighted pool (55% RTX 3090, 30% Strix Halo, 15% RTX 5070). + This relieves Strix Halo pressure while keeping compression functional on all GPUs. +- **`syslog-auto` is the valid compression model** — LiteLLM serves it as the weighted pool. + Old configs with `strix-moe` for compression should be updated to `syslog-auto`. - All auxiliary services MUST use identical routing: - `base_url: http://192.168.68.116/v1` (Rule 5: `/v1`, NOT `/litellm/v1`) - `api_key_env: LITELLM_API_KEY` -- **Do NOT use `syslog-auto`** for auxiliary tasks — it routes unpredictably -- **Compression on Strix Halo**: The strix-moe alias routes to Strix Halo - (64GB UMA, 128K context) — the designated compression GPU. This frees the - RTX 5070 for vision and web search, and the RTX 3090 for heavy reasoning. +- **Compression via syslog-auto**: Routes through the weighted pool. Strix Halo still handles + ~30% of compression calls (at 60 RPM via pool vs 40 RPM direct), but the bulk (55%) + goes to RTX 3090 which has ample spare capacity. - The `compression:` block's `model` MUST match `auxiliary: compression: model` - The `compression: max_context_window: 131072` MUST match actual GPU capacity (128K) -### Rule 8: GPU Workload Distribution (UPDATED 2026-07-16) -- **RTX 3090 (24GB, 128K ctx, qwen3.6-27B-code)**: Heavy reasoning, code gen, long conversations -- **RTX 5070 (12GB, 128K ctx, gemma-4-12b)**: Vision, web search, quick tasks, web_extract (IQ4_NL+MTP, ~65% VRAM at 128K) -- **Strix Halo (64GB, 128K ctx, strix-moe)**: Context compression, summarization, long docs +### Rule 8: GPU Workload Distribution (UPDATED 2026-07-18) +- **RTX 3090 (24GB, 128K ctx, qwen3.6-27B-code)**: Heavy reasoning, code gen, long conversations — also handles ~55% of compression via syslog-auto pool +- **RTX 5070 (12GB, 128K ctx, gemma-4-12b)**: Vision, web search, quick tasks, web_extract — handles ~15% of compression via syslog-auto pool +- **Strix Halo (64GB, 128K ctx, Geneis Hermes V3 APEX)**: Agent reasoning, compression (~30% via syslog-auto pool), fallback for other GPUs - Agent profiles MUST route auxiliary tasks to the correct GPU: - - `auxiliary.vision.model: gemma-4-12b` (RTX 5070) - - `auxiliary.web_extract.model: gemma-4-12b` (RTX 5070) - - `auxiliary.compression.model: strix-moe` (Strix Halo) + - `auxiliary.vision.model: gpu-light` (RTX 5070) + - `auxiliary.web_extract.model: gpu-light` (RTX 5070) + - `auxiliary.compression.model: syslog-auto` (distributed pool, switched from strix-moe 2026-07-18) - Default model (`model.default`) and custom_provider remain `syslog-auto` for auto-routing - For 128K context window: `threshold: 0.65` (fires at ~85K tokens) - Do NOT use `threshold: 0.25` — this fires at 65K, causing premature context loss -- 2.54.0