- Added extra_hosts for auth.sysloggh.net to LiteLLM container
- Fixed DOCS_URL=/docs (was /litellm/docs - path mismatch)
- Added Authentik self-signed cert to CA bundle
- Added nginx auth proxy for token/userinfo endpoints (SSL verify off)
- Changed OIDC token/userinfo endpoints to use nginx internal proxy
- Admin UI serving correctly on :4001/ui/ and /litellm/ui/
- Swagger API docs working at /docs and /litellm/docs
- ReDoc API docs working at /redoc and /litellm/redoc
- OIDC login flow verified working end-to-end
The nginx catch-all location / used proxy_pass $litellm_backend_url/ which
stripped the URI path, causing /openapi.json to return the Swagger UI HTML
page instead of the actual OpenAPI spec JSON. This broke the Swagger UI
rendering with "Unable to render this definition" error.
Fix: Add a dedicated location /openapi.json block before the catch-all /
that preserves the full path, so LiteLLM returns its valid openapi: 3.1.0
spec JSON at the public endpoint.
- Added /metrics/gpu-health endpoint with live health scores (VRAM 40%, temp 30%, load 30%)
- Added /metrics/latency endpoint for dashboard KPIs
- Added GPU_LABELS for human-readable model names
- Dashboard v2: rewired to real data endpoints
- KPI cards: GPUs online, circuit trips, avg latency, req/min, active requests
- Health scores from actual gpu_health_score() function
- Rolling 60-sample history chart (real data, no simulation)
- Status: green/yellow/red based on tripped circuits
- No CDN dependency (pure CSS)
- Auto-refresh every 15s
- nginx: /dashboard/ serves static files with cache headers
- docker-compose: dashboard volume mount
Co-authored-by: Abiba <abiba@sysloggh.com>
router/router.py (+158 lines):
- store_perf_record(): captures queue_ms, inference_ms, prompt_tokens,
completion_tokens, tokens_per_sec per request in Redis
- Per-model, per-reason, per-agent rolling windows (last 200-500)
- /metrics/performance?window=N endpoint with percentiles (p50/p95/p99)
for latency, throughput, and queue time per model/reason/agent
- Queue time now surfaced in routing metadata and routes:recent
- Streaming requests tracked with estimated prompt tokens
nginx/nginx.conf:
- Added /metrics/ proxy pass to router_api
Enables model performance comparison and routing tier validation.
Abiba