2026-07-12 23:40:13 +00:00
1 changed files with 29 additions and 0 deletions
@@ -84,6 +84,35 @@ triggers:
 | gemma-4-12b | RTX 5070 | .110 (ocu-llm) | 10.0/12.2GB (82%) | 131K | q4_0 | 2 | 2048/1024 | ✅ healthy |
 | ornith-1.0-35b | Strix Halo Vulkan | .15 (amdpve) | ~9GB/64GB | 256K | q8_0 | 2 | 2048/512 | ✅ healthy |

+## Routing Configuration (LiteLLM — July 2026)
+
+### syslog-auto Weighted Pool
+
+| Model | GPU | Weight | RPM Cap | Purpose |
+|-------|-----|--------|---------|---------|
+| qwen3.6-27B-code | RTX 3090 | 0.55 | 500 | Heavy reasoning, code, long context |
+| ornith-1.0-35b | Strix Halo | 0.30 | **60** | Agentic workflows, tool calling |
+| gemma-4-12b | RTX 5070 | 0.15 | 200 | Overflow + vision |
+
+### Direct Model Endpoints
+
+| Model | RPM Cap | Notes |
+|-------|---------|-------|
+| ornith-1.0-35b | 40 | Tight cap — prevents Strix overload |
+| qwen3.6-27B-code | 500 | High cap — primary workhorse |
+| gemma-4-12b | 500 | High cap — fast 12B |
+
+### Fallback Chains
+- gemma → qwen
+- qwen → gemma
+- ornith → qwen → gemma
+- syslog-auto → qwen → gemma → ornith
+
+### Why ornith RPM Is Capped
+- Direct: 40 RPM (tight) — Strix Halo is shared with compression tasks
+- Via syslog-auto: 60 RPM (moderate) — prevents flooding when multiple agents use syslog-auto simultaneously
+- Combined max: ~100 RPM across both paths — Strix Halo can sustain this at 80°C
+
 ## Operations

 ### add-model