diff --git a/gpu-fleet.prose.md b/gpu-fleet.prose.md index 7e938d9..8504677 100644 --- a/gpu-fleet.prose.md +++ b/gpu-fleet.prose.md @@ -125,7 +125,7 @@ Note: All syslog-auto entries route directly to GPUs with `api_key: not-needed`. | Alias | RPM Cap | Routes To | Purpose | |-------|---------|-----------|---------| -| `strix-moe` | 40 | Strix Halo | Compression tasks (MoE models) | +| `strix-moe` | 40 | Strix Halo | Agent reasoning, compression (~30% via syslog-auto pool) (MoE models) | | `gpu-dense` | 500 | RTX 3090 | Heavy reasoning | | `gpu-light` | 500 | RTX 5070 | Vision, web extract, light tasks | @@ -136,7 +136,7 @@ Note: All syslog-auto entries route directly to GPUs with `api_key: not-needed`. - syslog-auto → qwen → gemma → qwen3.6-35B-udq4 ### Why Strix Halo RPM Is Capped -- Direct (strix-moe): 40 RPM (tight) — Strix Halo is shared with compression tasks +- Direct (strix-moe): 40 RPM (tight) — Strix Halo handles agent reasoning + compression via syslog-auto pool - Via syslog-auto: 60 RPM (moderate) — prevents flooding when multiple agents use syslog-auto simultaneously - Combined max: ~100 RPM across both paths — Strix Halo can sustain this at 80°C @@ -284,7 +284,7 @@ History stored at `/root/data/toks-history.json` with 7-day rolling window. ### Stable Aliases — CRITICAL All agent configs MUST use stable role-based aliases, never model-specific names: -- `compression.model: strix-moe` (NOT `qwen3.6-35B-udq4`) +- `compression.model: syslog-auto` — switched from strix-moe 2026-07-18 to distribute across weighted pool (55% RTX 3090, 30% Strix Halo, 15% RTX 5070); relieves Strix Halo thermal pressure - `auxiliary.vision.model: gpu-light` (NOT `gemma-4-12b`) - `delegation.model: gpu-dense` (NOT `qwen3.6-27B-code`) - `auxiliary.web_extract.model: gpu-light` @@ -295,7 +295,7 @@ When the underlying model is swapped, only the LiteLLM config changes — agent - RTX 3090: **128K** (reduced from 256K 2026-07-17) | RTX 5070: **128K** (reduced from 256K) | Strix Halo: **128K** - **All agents**: 128K ceiling — stable margin. For >128K workloads, use external providers (deepseek) - Compression threshold 0.65: fires at ~85K (~43K headroom before 128K ceiling) -- Mumuni compression model alias: `strix-moe` with 300s timeout +- Mumuni compression model alias: `syslog-auto` (switched from strix-moe 2026-07-18) with 300s timeout ### Mumuni Agent Profile @@ -305,8 +305,8 @@ Mumuni (CT114, 192.168.68.123) is the primary business assistant. This profile i |---------|-------|-------| | `model.default` | `syslog-auto` | Weighted pool (55% qwen, 30% strix, 15% gemma) | | `model.provider` | `custom:litellm` | LiteLLM on CT116 | -| `compression.model` | `strix-moe` | Stable alias — survives model swaps | -| `aux.compression.model` | `strix-moe` | Compression auxiliary model | +| `compression.model` | `syslog-auto` | Switched from strix-moe 2026-07-18 — distributes across weighted pool | +| `aux.compression.model` | `syslog-auto` | Compression auxiliary — switched from strix-moe 2026-07-18 | | `aux.vision.model` | `gpu-light` | Vision tasks (RTX 5070) | | `aux.web_extract.model` | `gpu-light` | Web extraction | | `delegation.model` | `gpu-dense` | Sub-agent reasoning (RTX 3090) | @@ -318,7 +318,7 @@ Mumuni (CT114, 192.168.68.123) is the primary business assistant. This profile i | `personalities` | `creative` | Creative assistant personality | | Platforms | cli, discord, homeassistant, signal, telegram, zulip | All Hermes platforms | | Main model timeout | 300s | LiteLLM global timeout | -| Compression model timeout | 300s | strix-moe timeout increased from 120s | +| Compression model timeout | 300s | syslog-auto timeout (switched from strix-moe 2026-07-18) | ### Agent Update Status (2026-07-15) diff --git a/gpu-self-heal.prose.md b/gpu-self-heal.prose.md index 62cf205..66fddf7 100644 --- a/gpu-self-heal.prose.md +++ b/gpu-self-heal.prose.md @@ -46,7 +46,7 @@ depends_on: |-------|-----|------|-------|------|-----|-------|------| | `gpu-dense` | RTX 3090 24GB | ct8 (.8:8080) | ThinkingCap Qwen3.6-27B Q4_K_M + MTP + vision | 21.6/24.6GB (88%) | 128K | 74.9 | Heavy reasoning, code gen | | `gpu-light` | RTX 5070 12GB | ct110 (.110:8080) | HauhauCS Gemma4-12B QAT Q4_K_M + MTP draft | 10.1/12.2GB (83%) | 128K | 169.6 | Vision, web extract, light tasks | -| `strix-moe` | Strix Halo 64GB | ct15 (.15:8080) | Genesis Hermes V3 APEX (LuffyTheFox, 24GB) | ~10/64GB (16%) | 128K | 62.9 | Compression, summarization, long docs | +| `strix-moe` | Strix Halo 64GB | ct15 (.15:8080) | Genesis Hermes V3 APEX (LuffyTheFox, 24GB) | ~10/64GB (16%) | 128K | 62.9 | Agent reasoning, compression (~30% via syslog-auto pool), summarization, long docs | Key notes: - All models use direct GPU routing via LiteLLM (`api_key: not-needed`). Router (port 9000) is deprecated and NOT in the inference path. @@ -157,7 +157,7 @@ Key notes: - **Target distribution**: - RTX 3090 (gpu-dense, 24GB, 74.9 tok/s) → Heavy reasoning, code gen, long conversations (slowest per-token but largest context capacity). Weight: 0.55 (LiteLLM). - RTX 5070 (gpu-light, 12GB, 169.6 tok/s) → Vision/image, web search, lightweight tasks (2.3x faster than 3090 per token). Weight: 0.15 (LiteLLM). - - Strix Halo (strix-moe, 64GB, 62.9 tok/s) → Context compression, summarization, long docs (MoE model). Weight: 0.30 (LiteLLM). + - Strix Halo (strix-moe, 64GB, 62.9 tok/s) → Agent reasoning, compression (~30% via syslog-auto pool), summarization, long docs (MoE model). Weight: 0.30 (LiteLLM). - **Note**: RTX 5070 is the fastest endpoint per token. Route high-volume, low-complexity work there first. - **Fix**: - Alert if any GPU is handling workload outside its designated role diff --git a/hermes-agent-baseline.prose.md b/hermes-agent-baseline.prose.md index 02b9f17..f73d04c 100644 --- a/hermes-agent-baseline.prose.md +++ b/hermes-agent-baseline.prose.md @@ -5,7 +5,7 @@ version: 1.0.0 description: > Canonical known-good baseline for all Syslog Hermes agents. Captures the exact configuration state, keys, workarounds, and audit procedure. When an agent's - configuration goes sideways, restore from this baseline. Last verified 2026-07-16. All GPUs 256K context (RTX 3090 .8, RTX 5070 .110, Strix Halo .15). Parallel 1 fleet-wide (Strix Halo handles compression solo). + configuration goes sideways, restore from this baseline. Last verified 2026-07-16. All GPUs 128K context (RTX 3090 .8, RTX 5070 .110, Strix Halo .15). Parallel 1 fleet-wide (compression via syslog-auto pool — switched from strix-moe 2026-07-18). author: Abiba (pi agent) --- diff --git a/inference-optimization.prose.md b/inference-optimization.prose.md index 91a10e6..f5eb168 100644 --- a/inference-optimization.prose.md +++ b/inference-optimization.prose.md @@ -57,7 +57,7 @@ duration. prefill time at 532 tok/s. Fix context first, routing second. - **Route by task**: ornith for multi-step reasoning only; qwen for code/standard - queries; gemma for compression/auxiliary. Never send simple completion to a + queries; gemma for vision/web extraction; syslog-auto for compression (via weighted pool). Never send simple completion to a 35B MoE. - **Compress aggressively**: threshold at 40% (not 65%) — a 256K window should compact at 102K, not 166K. Target 15% tail (not 30%).