docs(litellm-health): single source of truth, gpu-vision alias, litellm-health as live owner #79

Merged
abiba-bot merged 8 commits from fix/litellm-health-drift-20260912 into master 2026-09-12 16:05:28 +00:00
3 changed files with 5 additions and 6 deletions
Showing only changes of commit dc42ecc235 - Show all commits
+1 -1
View File
@@ -88,7 +88,7 @@ prose run memory-audit-maintenance memory_threshold=90 verify_configs=true
prose run hermes-config-template agent_name=syslog-devops default_model=claude-sonnet-4
# Configure an agent with a different auxiliary model
prose run hermes-config-template agent_name=syslog-code default_model=qwen3.6-27B-code auxiliary_model=gemma-4-12b
prose run hermes-config-template agent_name=syslog-code default_model=qwen3.6-27B-code auxiliary_model=gpu-vision
```
### Option B: Manual Execution
+1 -1
View File
@@ -13,7 +13,7 @@ the description:
1. **What system does this contract touch?** Name the hosts, CTs, containers,
and services explicitly. "The inference fleet" is vague. "GPU .8 (RTX 3090,
qwen), .110 (RTX 5070, gemma), .15 (Strix Halo, strix-moe), and LiteLLM on CT
qwen), .110 (RTX 5070, gpu-vision), .15 (Strix Halo, strix-moe), and LiteLLM on CT
116" is specific.
2. **Who runs this contract, and when?** State the agent, the trigger (cron,
+3 -4
View File
@@ -25,7 +25,7 @@ triggers:
## Maintains
- gpu_roster: { models: map, hosts: map } — Single source of truth for all GPU models
- gpu_roster: { models: map, hosts: map } — GPU host/model roster; the authoritative alias/weight/fallback registry is CT 116 `litellm_config.yaml`
- router: { status: "healthy", roster_loaded: bool, models: array }
- litellm: { status: "healthy", keys: array, models: array }
- agent_keys: { agent: api_key } — All agent API keys registered in LiteLLM DB
@@ -97,9 +97,8 @@ is retired and returns 400 `Invalid model name`.
## Routing Configuration (LiteLLM — July 2026)
Single source of truth for models, aliases, rpm caps, weights and fallback chains:
CT 116 `/opt/inference-harness/litellm_config.yaml`. Do not duplicate those values in
contracts — read them there.
Model, alias, rpm/weight and fallback values are owned by CT 116
`/opt/inference-harness/litellm_config.yaml` (see § Stable Role-Based Aliases above).
Note: All syslog-auto entries route directly to GPUs with `api_key: not-needed`. The router (port 9000) was decommissioned 2026-09-11 and is NOT in the inference path.