feat: add routing configuration with RPM caps (ornith syslog-auto: 40→60)
PR Pipeline — Authorize → Validate → Review → Merge / auth (pull_request) Successful in 2s
PR Pipeline — Authorize → Validate → Review → Merge / validate (pull_request) Successful in 3s
PR Pipeline — Authorize → Validate → Review → Merge / lint (pull_request) Successful in 2s
PR Pipeline — Authorize → Validate → Review → Merge / ai-review (pull_request) Successful in 2s
PR Pipeline — Authorize → Validate → Review → Merge / gate (pull_request) Successful in 2s
PR Pipeline — Authorize → Validate → Review → Merge / auth (pull_request) Successful in 2s
PR Pipeline — Authorize → Validate → Review → Merge / validate (pull_request) Successful in 3s
PR Pipeline — Authorize → Validate → Review → Merge / lint (pull_request) Successful in 2s
PR Pipeline — Authorize → Validate → Review → Merge / ai-review (pull_request) Successful in 2s
PR Pipeline — Authorize → Validate → Review → Merge / gate (pull_request) Successful in 2s
- Strix Halo ornith RPM increased from 40 to 60 inside syslog-auto pool - Prevented unbounded ornith traffic when multiple agents use syslog-auto - Direct ornith remains at RPM=40 - All models verified healthy (ornith 80ms, gemma 110ms, qwen 840ms) - Added full routing table to gpu-fleet contract
This commit is contained in:
@@ -84,6 +84,35 @@ triggers:
|
|||||||
| gemma-4-12b | RTX 5070 | .110 (ocu-llm) | 10.0/12.2GB (82%) | 131K | q4_0 | 2 | 2048/1024 | ✅ healthy |
|
| gemma-4-12b | RTX 5070 | .110 (ocu-llm) | 10.0/12.2GB (82%) | 131K | q4_0 | 2 | 2048/1024 | ✅ healthy |
|
||||||
| ornith-1.0-35b | Strix Halo Vulkan | .15 (amdpve) | ~9GB/64GB | 256K | q8_0 | 2 | 2048/512 | ✅ healthy |
|
| ornith-1.0-35b | Strix Halo Vulkan | .15 (amdpve) | ~9GB/64GB | 256K | q8_0 | 2 | 2048/512 | ✅ healthy |
|
||||||
|
|
||||||
|
## Routing Configuration (LiteLLM — July 2026)
|
||||||
|
|
||||||
|
### syslog-auto Weighted Pool
|
||||||
|
|
||||||
|
| Model | GPU | Weight | RPM Cap | Purpose |
|
||||||
|
|-------|-----|--------|---------|---------|
|
||||||
|
| qwen3.6-27B-code | RTX 3090 | 0.55 | 500 | Heavy reasoning, code, long context |
|
||||||
|
| ornith-1.0-35b | Strix Halo | 0.30 | **60** | Agentic workflows, tool calling |
|
||||||
|
| gemma-4-12b | RTX 5070 | 0.15 | 200 | Overflow + vision |
|
||||||
|
|
||||||
|
### Direct Model Endpoints
|
||||||
|
|
||||||
|
| Model | RPM Cap | Notes |
|
||||||
|
|-------|---------|-------|
|
||||||
|
| ornith-1.0-35b | 40 | Tight cap — prevents Strix overload |
|
||||||
|
| qwen3.6-27B-code | 500 | High cap — primary workhorse |
|
||||||
|
| gemma-4-12b | 500 | High cap — fast 12B |
|
||||||
|
|
||||||
|
### Fallback Chains
|
||||||
|
- gemma → qwen
|
||||||
|
- qwen → gemma
|
||||||
|
- ornith → qwen → gemma
|
||||||
|
- syslog-auto → qwen → gemma → ornith
|
||||||
|
|
||||||
|
### Why ornith RPM Is Capped
|
||||||
|
- Direct: 40 RPM (tight) — Strix Halo is shared with compression tasks
|
||||||
|
- Via syslog-auto: 60 RPM (moderate) — prevents flooding when multiple agents use syslog-auto simultaneously
|
||||||
|
- Combined max: ~100 RPM across both paths — Strix Halo can sustain this at 80°C
|
||||||
|
|
||||||
## Operations
|
## Operations
|
||||||
|
|
||||||
### add-model
|
### add-model
|
||||||
|
|||||||
Reference in New Issue
Block a user