feat: add routing configuration with RPM caps (ornith syslog-auto: 40→60)

- Strix Halo ornith RPM increased from 40 to 60 inside syslog-auto pool
- Prevented unbounded ornith traffic when multiple agents use syslog-auto
- Direct ornith remains at RPM=40
- All models verified healthy (ornith 80ms, gemma 110ms, qwen 840ms)
- Added full routing table to gpu-fleet contract
This commit is contained in:
root
2026-07-12 23:38:21 +00:00
parent af41f8f57a
commit 0c298eb9d9
+29
View File
@@ -84,6 +84,35 @@ triggers:
| gemma-4-12b | RTX 5070 | .110 (ocu-llm) | 10.0/12.2GB (82%) | 131K | q4_0 | 2 | 2048/1024 | ✅ healthy |
| ornith-1.0-35b | Strix Halo Vulkan | .15 (amdpve) | ~9GB/64GB | 256K | q8_0 | 2 | 2048/512 | ✅ healthy |
## Routing Configuration (LiteLLM — July 2026)
### syslog-auto Weighted Pool
| Model | GPU | Weight | RPM Cap | Purpose |
|-------|-----|--------|---------|---------|
| qwen3.6-27B-code | RTX 3090 | 0.55 | 500 | Heavy reasoning, code, long context |
| ornith-1.0-35b | Strix Halo | 0.30 | **60** | Agentic workflows, tool calling |
| gemma-4-12b | RTX 5070 | 0.15 | 200 | Overflow + vision |
### Direct Model Endpoints
| Model | RPM Cap | Notes |
|-------|---------|-------|
| ornith-1.0-35b | 40 | Tight cap — prevents Strix overload |
| qwen3.6-27B-code | 500 | High cap — primary workhorse |
| gemma-4-12b | 500 | High cap — fast 12B |
### Fallback Chains
- gemma → qwen
- qwen → gemma
- ornith → qwen → gemma
- syslog-auto → qwen → gemma → ornith
### Why ornith RPM Is Capped
- Direct: 40 RPM (tight) — Strix Halo is shared with compression tasks
- Via syslog-auto: 60 RPM (moderate) — prevents flooding when multiple agents use syslog-auto simultaneously
- Combined max: ~100 RPM across both paths — Strix Halo can sustain this at 80°C
## Operations
### add-model