---
kind: function
name: litellm-health
description: >
  Verifies the LiteLLM inference stack health. Current architecture (2026-06-30):
  nginx:80 → LiteLLM:4000 → Router:9000(internal) → GPU(llama-server).
  Router runs as internal-only backend behind LiteLLM. GPU monitoring via
  Prometheus/Grafana and fleet dashboard.
  Designed as a reusable contract for any Syslog agent.
---

## Architecture (v3.2.0 — Layered: nginx → LiteLLM → Router → GPU)

```
Request → nginx:80 → LiteLLM:4000 → Router:9000(internal) → GPU(llama-server)
                        │                    │
                  Key validation      Model routing
                  Fallback chains     Slot booking
                  Complexity router   Circuit breakers
                  Budget tracking     Redis-backed
                        │                    │
                  Prometheus ← metrics  ←───┘
                        │
                    Grafana :3001
```

**What changed (v3.1.0 → v3.2.0)**:
- Router is BACK — runs internally on :9000 behind LiteLLM
- nginx routes /v1/ and /admin/ → LiteLLM (not router directly)
- Router :9000 is 127.0.0.1-only, not publicly accessible
- GPU fleet dashboard: http://192.168.68.24:9100
- Grafana: http://192.168.68.116:3001 (GPU dashboards via Prometheus)

## Parameters

- public_url: string — The public LiteLLM URL (default: "https://litellm.sysloggh.net")
- backend_host: string — Internal CT host for container checks (default: "192.168.68.116")
- auth_host: string — Authentik server for OIDC (default: "192.168.68.11")
- gpu_hosts: array — GPU inference hosts (default: ["192.168.68.8", "192.168.68.110", "192.168.68.15"])
- gpu_dashboard_url: string — Fleet dashboard (default: "http://192.168.68.24:9100")
- grafana_url: string — Grafana dashboards (default: "http://192.168.68.116:3001")

## Returns

- overall_status: "healthy" | "degraded" | "down"
- checks: array of { name: string, status: string, detail: string }
- timestamp: string — ISO timestamp of the check run
- duration_ms: number — How long the check took

## Requires

- SSH key access to backend_host for container checks
- Network access to public_url, auth_host, and gpu_dashboard_url
- LiteLLM master key for key management endpoints

## GPU Fleet Topology

| Host | IP | Hardware | Models Served | Engine |
|------|-----|----------|---------------|--------|
| llm-gpu | 192.168.68.8 | NVIDIA RTX 3090 (24 GB) | qwen3.6-27B-code | llama-server Docker |
| ocu-llm | 192.168.68.110 | NVIDIA RTX 5070 (12 GB) | gemma-4-12b | llama-server Docker |
| amdpve | 192.168.68.15 | AMD Strix Halo (CPU) | ornith-1.0-35b (35B) | llama-server bare-metal |

## Model Fallback Chains (LiteLLM)

| Primary | Timeout | Fallback | Timeout |
|---------|---------|----------|---------|
| qwen3.6-27B-code | 45s | gemma-4-12b | 30s |
| gemma-4-12b | 30s | qwen3.6-27B-code | 45s |
| ornith-1.0-35b | 120s | qwen → gemma | — |
| syslog-auto (balanced) | — | qwen → gemma | — |

## Containers on CT 116

| Container | Image | Port | Health Check |
|-----------|-------|------|-------------|
| harness-litellm | berriai/litellm:1.90.0-rc.1 | :4000→:4001 | /health/liveliness |
| harness-router | inference-harness-router | :9000 (127.0.0.1) | /health |
| harness-nginx | nginx:alpine | :80 | HTTP 200 on /health |
| harness-postgres | postgres:16-alpine | :5432 | pg_isready |
| harness-redis | redis:7-alpine | :6379 | PING |
| harness-dashboard | inference-harness-dashboard | :3000 | /health |
| harness-grafana | grafana/grafana | :3000→:3001 | /api/health |
| harness-prometheus | prom/prometheus | :9090 | /-/healthy |

## Execution

1. **Read parameters** — Use provided values or defaults

2. **Check public endpoints**:
   - GET {{public_url}}/ui/ → expect 200 ("LiteLLM Dashboard")
   - GET {{public_url}}/docs → expect 200 ("LiteLLM API - Swagger UI")

3. **Check LiteLLM health (no-auth)**:
   - GET http://{{backend_host}}/litellm/health/liveliness → expect 200

4. **Check backend container health**:
   - SSH to {{backend_host}} → `docker ps` → verify 8 containers healthy
   - Critical: harness-litellm, harness-router, harness-nginx, harness-postgres
   - Monitoring: harness-redis, harness-dashboard, harness-grafana, harness-prometheus

5. **Check router roster loaded**:
   - GET http://{{backend_host}}:9000/health → expect 200
   - GET http://{{backend_host}}:9000/health/unified → expect 3 models
   - If router returns "all GPUs saturated" but GPUs idle: roster not loaded → reload

6. **Check GPU fleet health** (via fleet dashboard):
   - GET {{gpu_dashboard_url}}/gpu-data → expect 200 with GPU metrics JSON
   - Verify GPUs reporting status "healthy"
   - Check alerts array for active warnings/critical

7. **Check model inference via LiteLLM** — Test each model:
   - POST /v1/chat/completions model=gemma-4-12b → expect 200
   - POST /v1/chat/completions model=qwen3.6-27B-code → expect 200
   - POST /v1/chat/completions model=ornith-1.0-35b → expect 200
   - Use master key for auth

8. **Check agent keys**:
   - GET /key/list with master key → verify all 6 agents have keys

9. **Check Grafana**:
   - GET {{grafana_url}}/api/health → expect 200

10. **Compile and report** — Determine overall_status from individual check results