SyslogSolution/prose-contracts
7
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: LiteLLM contracts — align with 2026-07-08 architecture changes
PR Pipeline — Authorize → Validate → Review → Merge / auth (pull_request) Failing after 14m29s
Details
PR Pipeline — Authorize → Validate → Review → Merge / validate (pull_request) Has been skipped
Details
PR Pipeline — Authorize → Validate → Review → Merge / lint (pull_request) Has been skipped
Details
PR Pipeline — Authorize → Validate → Review → Merge / ai-review (pull_request) Has been skipped
Details
PR Pipeline — Authorize → Validate → Review → Merge / gate (pull_request) Has been skipped
Details

Browse Source 
- litellm-health: v4.0.0 — Router removed from path (nginx→LiteLLM→GPU directly),
  GPU engines corrected (Docker→systemd), timeouts updated (gemma 120s, qwen 90s),
  Strix Halo: CPU→Vulkan, context sizes + parallel slots added
- litellm-self-heal: ornith context 262K→256K, Rule 7 marked DEPRECATED
  (router not in path), GPU topology synced with gpu-fleet
- litellm-api-keys: reference gpu-fleet as source of truth for keys,
  removed qwen3.6-35B-A3B from model list (never deployed)
This commit is contained in:
root
2026-07-09 05:37:16 +00:00
parent a7fedf71c6
commit a2c5a14c74
3 changed files with 55 additions and 40 deletions
Download Patch File Download Diff File Expand all files Collapse all files
litellm-self-heal.prose.md
+12 -11
View File
@@ -6,6 +6,9 @@ note: >
Auto-remediation code was removed from the pi Zulip extension (retired 2026-07-04).
This contract is now manual-only — triggers require explicit user request.
Consider reimplementing as a standalone cron job or prose contract.
Source of truth for GPU topology: gpu-fleet.prose.md
Last verified: 2026-07-09
description: >
Standing responsibility that monitors LiteLLM health and proactively
fixes common issues. Reports every action via Zulip DM and RA-H OS
@@ -73,11 +76,11 @@ message is sent to the responsible agent with full context.
## GPU Fleet
| Host | IP | Hardware | Models | Engine |
|------|-----|----------|--------|--------|
| amdpve | 192.168.68.15 | Strix Halo 64GB UMA | ornith-1.0-35b (16t, 262K ctx) | llama-server :8080 |
| llm-gpu | 192.168.68.8 | RTX 3090 24GB | qwen3.6-27B-code | llama-server :8080 |
| ocu-llm | 192.168.68.110 | RTX 5070 12GB | gemma-4-12b | llama-server :8080 |
| Host | IP | Hardware | Models | Engine | Context |
|------|-----|----------|--------|--------|---------|
| amdpve | 192.168.68.15 | Strix Halo 64GB UMA | ornith-1.0-35b | llama-server :8080 (Vulkan) | 256K |
| llm-gpu | 192.168.68.8 | RTX 3090 24GB | qwen3.6-27B-code | llama-server :8080 (systemd) | 128K |
| ocu-llm | 192.168.68.110 | RTX 5070 12GB | gemma-4-12b | llama-server :8080 (systemd) | 128K |
## Remediation Rules
@@ -105,12 +108,10 @@ Detect → test prompt via LiteLLM returns non-200
Fix → restart llama-server on GPU host → verify
Escalate → after 2 failed restarts
### Rule 7: Router Returns 503 (All GPUs Saturated)
Detect → router logs show QUEUE_TIMEOUT with GPUs idle
Root cause → roster not loaded at startup (empty TIER_MODELS)
Fix → POST /admin/roster/reload (if endpoint available) or restart router
Permanent fix → add load_roster() to router startup code
Escalate → if reload doesn't resolve within 2 min
### Rule 7: Router Returns 503 (All GPUs Saturated) — DEPRECATED
Router is no longer in the request path (2026-07-08). LiteLLM proxies
directly to GPU. This rule is retained for reference but is inactive.
If 503 errors occur, check LiteLLM timeouts and GPU health directly.
### Rule 8: Agent Keys Invalid (401)
Detect → agents report auth errors