litellm_config.yaml
- client-visible model_list reduced to capability names: syslog-auto, gpu-dense, strix-moe, gpu-vision
- retired qwen3.6-27B-code, qwen3.8-27B-uncensored, qwen3.6-35B-udq4 (and the already-dead gemma-4-12b)
- fallbacks and model_cost re-keyed to the surviving names
- upstream litellm_params.model set to the capability names; the backends ignore the model field
(verified HTTP 200 on all three hosts), so this needs no llama.cpp relaunch and loses no KV warmth
- verified live: /v1/models advertises exactly the four names, each returns 200 through nginx
gpu_roster.yaml
- launch args replaced with the VERIFIED live command lines, read from the running processes via the
PVE guest agent (acerpve VM 101 = .8, ocupve VM 103 = .110) and on amdpve (.15)
- .8: model_path corrected to Qwen3.8-27B-Uncensored-Q4_K_M.gguf, max_concurrent 2 -> 1,
context 262144 -> 131072, full arg list recorded (incl. --parallel 1 and the new
--slot-save-path / --metrics added 2026-09-12)
- .15: model_path corrected to Carnice-Qwen3.6-MoE-35B-A3B-Q4_K_M.gguf, full arg list recorded
- keys renamed to the capability names; hosts.current_model aligned
README.md / dashboard
- README dense entry corrected (1 slot, 131K ctx, actual model file)
- dashboard picker now uses capability names; fixed the "Gemma 4 12B" / "12B VLM" labels (the RTX 5070
serves a 9B Qwen3.5) and the gpu-vision -> gpu-light id mapping
scripts/
- added the three operational monitors as tracked files (they were untracked): gpu-monitor.py,
gpu-self-heal.py, litellm-health-check.sh
- cleared their references to retired model names, which were causing failed calls every benchmark
cycle (150 failed gemma-4-12b calls in the last 7 days); gpu-monitor.py's .110 entry also wrongly
listed .8's model
Intentionally NOT changed
- LITELLM-MIGRATION-PLAN.md: historical planning document (June 14), not a live-state claim
- backups/, graphify-out/, litellm_config.yaml.backup: historical artifacts
- unrelated untracked files (router.py, docker-compose.yml.pre-1991-20260911, nginx/default.conf,
dashboard/gpu-monitor.html, scripts/gitea-logger.sh): out of scope for this change
- Added /metrics/gpu-health endpoint with live health scores (VRAM 40%, temp 30%, load 30%)
- Added /metrics/latency endpoint for dashboard KPIs
- Added GPU_LABELS for human-readable model names
- Dashboard v2: rewired to real data endpoints
- KPI cards: GPUs online, circuit trips, avg latency, req/min, active requests
- Health scores from actual gpu_health_score() function
- Rolling 60-sample history chart (real data, no simulation)
- Status: green/yellow/red based on tripped circuits
- No CDN dependency (pure CSS)
- Auto-refresh every 15s
- nginx: /dashboard/ serves static files with cache headers
- docker-compose: dashboard volume mount
Co-authored-by: Abiba <abiba@sysloggh.com>
mumuni-bot