feat: tighten contracts with GPU fleet topology, model routing, Strix Halo details

litellm-health: added GPU fleet topology table, model routing map,
3-tier GPU checks (reachability, VRAM, temp, test inference)

litellm-self-heal: added GPU fleet reference, rules for GPU
unreachable and model not responding

Infrastructure topology now fully documented:
- amdpve: Strix Halo CPU (35B, 16 threads, 262K ctx, llama-server)
- llm-gpu: RTX 3090 24GB (Dense tier)
- ocu-llm: RTX 5070 12GB (MoE + Light tiers)
This commit is contained in:
root
2026-06-29 04:07:05 +00:00
parent 76fc2595ea
commit aa55634571
2 changed files with 47 additions and 2 deletions
+18
View File
@@ -66,6 +66,14 @@ logged as a `[REPORT]` knowledge graph node for long-term trending.
If a fix required another agent's help (e.g., Authentik restart), a relay
message is sent to the responsible agent with full context.
## GPU Fleet
| Host | IP | Hardware | Models | Engine |
|------|-----|----------|--------|--------|
| amdpve | 192.168.68.15 | Strix Halo CPU | ornith-1.0-35b (16t, 262K ctx) | llama-server :8080 |
| llm-gpu | 192.168.68.8 | RTX 3090 24GB | gemma-4-12b Dense | llama-server :8080 |
| ocu-llm | 192.168.68.110 | RTX 5070 12GB | qwen3.6-27B-code MoE+Light | llama-server :8080 |
## Remediation Rules
### Rule 1: Container Not Healthy
@@ -82,6 +90,16 @@ Detect → immediate escalate (external dependency)
### Rule 4: Docs 404
Detect → fix DOCS_URL env var → verify → log
### Rule 5: GPU Unreachable
Detect → SSH to GPU host fails or model not responding
Fix → restart llama-server on host via SSH
Escalate → immediately (downtime affects inference)
### Rule 6: Model Not Responding
Detect → test prompt via LiteLLM returns non-200
Fix → restart llama-server on GPU host → verify
Escalate → after 2 failed restarts
## Execution
1. Run health check