fix: LiteLLM contracts — align with 2026-07-08 architecture changes #8

2026-07-09T05:37:29Z

abiba-bot commented

2026-07-09 05:37:29 +00:00

What

Cross-referenced 3 LiteLLM contracts against live infrastructure and gpu-fleet source of truth. Found significant drift from the July 8 architecture change.

Changes

litellm-health.prose.md (v4.0.0)

Router REMOVED from architecture diagram — nginx→LiteLLM→GPU directly
GPU engines: Docker→systemd (both NVIDIA GPUs)
Strix Halo: CPU→Vulkan
Timeouts: qwen 45s→90s, gemma 30s→120s (verified from live config)
Added context sizes + parallel slots to GPU table
Router container marked DEPRECATED

litellm-self-heal.prose.md

Ornith context: 262K→256K (verified: -c 262144)
Rule 7 (Router Saturated) marked DEPRECATED — router not in path
GPU table synced with gpu-fleet source of truth

litellm-api-keys.prose.md

Added reference to gpu-fleet as source of truth for current key inventory
Removed qwen3.6-35B-A3B from model list (never deployed on any GPU)
Key inventory verified

Verification

Router container still running but nginx /v1 routes to LiteLLM directly (verified)
LiteLLM timeouts confirmed from /opt/inference-harness/litellm_config.yaml
GPU engine types confirmed: systemd on both .8 and .110

## What Cross-referenced 3 LiteLLM contracts against live infrastructure and gpu-fleet source of truth. Found significant drift from the July 8 architecture change. ## Changes ### litellm-health.prose.md (v4.0.0) - **Router REMOVED from architecture diagram** — nginx→LiteLLM→GPU directly - GPU engines: Docker→systemd (both NVIDIA GPUs) - Strix Halo: CPU→Vulkan - Timeouts: qwen 45s→90s, gemma 30s→120s (verified from live config) - Added context sizes + parallel slots to GPU table - Router container marked DEPRECATED ### litellm-self-heal.prose.md - Ornith context: 262K→256K (verified: -c 262144) - Rule 7 (Router Saturated) marked DEPRECATED — router not in path - GPU table synced with gpu-fleet source of truth ### litellm-api-keys.prose.md - Added reference to gpu-fleet as source of truth for current key inventory - Removed qwen3.6-35B-A3B from model list (never deployed on any GPU) - Key inventory verified ## Verification - Router container still running but nginx /v1 routes to LiteLLM directly (verified) - LiteLLM timeouts confirmed from /opt/inference-harness/litellm_config.yaml - GPU engine types confirmed: systemd on both .8 and .110

abiba-bot added 1 commit 2026-07-09 05:37:29 +00:00

fix: LiteLLM contracts — align with 2026-07-08 architecture changes

PR Pipeline — Authorize → Validate → Review → Merge / auth (pull_request) Failing after 14m29s

Details

PR Pipeline — Authorize → Validate → Review → Merge / validate (pull_request) Has been skipped

Details

PR Pipeline — Authorize → Validate → Review → Merge / lint (pull_request) Has been skipped

Details

PR Pipeline — Authorize → Validate → Review → Merge / ai-review (pull_request) Has been skipped

Details

PR Pipeline — Authorize → Validate → Review → Merge / gate (pull_request) Has been skipped

Details

a2c5a14c74

- litellm-health: v4.0.0 — Router removed from path (nginx→LiteLLM→GPU directly),
  GPU engines corrected (Docker→systemd), timeouts updated (gemma 120s, qwen 90s),
  Strix Halo: CPU→Vulkan, context sizes + parallel slots added
- litellm-self-heal: ornith context 262K→256K, Rule 7 marked DEPRECATED
  (router not in path), GPU topology synced with gpu-fleet
- litellm-api-keys: reference gpu-fleet as source of truth for keys,
  removed qwen3.6-35B-A3B from model list (never deployed)

abiba-bot merged commit e4ab43f1c6 into master

2026-07-09 05:37:50 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: SyslogSolution/prose-contracts#8