abiba-bot/inference-harness
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

feat(plan): update LiteLLM migration plan for CT 116 deployment with Authentik OIDC + zero-downtime strategy

Browse Source 
- Target deployment host: CT 116 syslog-api (192.168.68.116) on minipve
- DNS split-horizon: Option A /etc/hosts for auth.sysloggh.net → 192.168.68.11
- Add guardrails config (pre-call, post-call, content filter)
- Add custom_sso.py for Authentik OIDC via x-authentik-* headers
- Expand from 3-phase to 4-phase deployment with zero-downtime strategy
- Add Phase 0: Infrastructure Prep (DNS, Postgres, config, custom_sso)
- Add §11 GitOps workflow (branching, conventional commits, deployment flow)
- Add §12 Zero-Downtime Migration Strategy (per-agent 2min, global rollback)
- Update nginx config with Authentik forward auth + router fallback
- Update docker-compose.yml for CT 116 with extra_hosts + postgres
- Add multi-provider config placeholders (OpenAI, Anthropic)

Co-authored-by: Abiba <abiba@sysloggh.net>
This commit is contained in:
kagentz-bot
2026-06-14 08:22:32 -04:00
co-authored by Abiba
parent d901235c03
commit 84e0d163ee
1 changed files with 368 additions and 71 deletions
Download Patch File Download Diff File Expand all files Collapse all files
LITELLM-MIGRATION-PLAN.md
+368 -71
View File
@@ -1,5 +1,9 @@
# LiteLLM Integration Migration Plan
## Syslog Solution LLC — June 13, 2026
## Syslog Solution LLC — June 14, 2026
**Deployment Target:** CT 116 `syslog-api` (192.168.68.116) on minipve — all services co-located.
**DNS Strategy:** Option A — /etc/hosts + Docker extra_hosts for internal resolution of `auth.sysloggh.net` → 192.168.68.11.
**GitOps:** This plan lives in `SyslogSolution/syslog-harness` on Gitea. All changes tracked via git with conventional commits.
---
@@ -56,7 +60,9 @@
## 1. Current State Baseline
### 1.1 Router (`router-fixed.py` — port 9000, deployed on CT 116 / docker-vm)
### 1.1 Router (`router-fixed.py` — port 9000, deployed on CT 116 / syslog-api)
**Deployment Host:** CT 116 `syslog-api` on minipve (192.168.68.12), IP 192.168.68.116, 6GB RAM, 40GB disk. Runs Docker with all harness services co-located on this single host.
| Feature | Implementation |
|---------|---------------|
@@ -79,10 +85,47 @@
| qwen3.6-27B-code (Dense) | 192.168.68.8 | :8080 | :8090 | RTX 3090 | 262K |
| gemma-4-12b (VLM) | 192.168.68.110 | :8080 | :8090 | RTX 5070 | 262K |
### 1.3 Existing LiteLLM Attempt
- `/root/litellm-fix.sh` — previous setup script for docker-vm
### 1.3 Existing LiteLLM POC on CT 116
CT 116 already has a LiteLLM container running (POC, 6 days uptime):
```
harness-litellm | ghcr.io/berriai/litellm:main-stable | 127.0.0.1:8081→4000
harness-redis | redis:7-alpine | 127.0.0.1:6379
harness-router | inference-harness-router | 127.0.0.1:9000
harness-nginx | nginx:alpine | 0.0.0.0:80
harness-dashboard | inference-harness-dashboard | 127.0.0.1:3000
```
- `/opt/litellm/` — previous setup directory on CT 116
- Configured with Postgres, host networking, master key
- **Never productionized** — still in exploratory phase
- Currently bypassed — router routes directly to GPUs
- **Goal: Productionize with two-layer architecture on this same host**
### 1.4 DNS Routing (Split-Horizon)
For OIDC SSO with Authentik, CT 116 must resolve `auth.sysloggh.net` internally:
**Problem:** `auth.sysloggh.net` CNAMEs to `netbird.sysloggh.net` → 72.61.0.17 (public VPS). OIDC auth_request from NGINX would route through the internet back to 192.168.68.11 unnecessarily.
**Solution — Option A: /etc/hosts on CT 116 host:**
```bash
# On CT 116 (syslog-api)
echo "192.168.68.11 auth.sysloggh.net" >> /etc/hosts
```
**Docker containers** also need this resolution — add to docker-compose.yml:
```yaml
services:
nginx:
extra_hosts:
- "auth.sysloggh.net:192.168.68.11"
litellm:
extra_hosts:
- "auth.sysloggh.net:192.168.68.11"
```
**DNS Servers:** CT 116 uses 192.168.68.10 for DNS. AdGuard (192.168.68.11) is the long-term solution for LAN-wide split-horizon DNS.
---
@@ -211,6 +254,36 @@ model_list:
api_base: http://router:9000/v1
api_key: os.environ/ROUTER_API_KEY
# Guardrails: Pre-call and post-call content moderation
guardrails:
- guardrail_name: "input-moderation"
litellm_params:
guardrail: openai_moderation
mode: "pre_call"
- guardrail_name: "output-moderation"
litellm_params:
guardrail: openai_moderation
mode: "post_call"
- guardrail_name: "harmful-content-filter"
litellm_params:
guardrail: litellm_content_filter
mode: "pre_call"
categories:
- category: "harmful_self_harm"
enabled: true
action: "BLOCK"
severity_threshold: "medium"
- category: "harmful_violence"
enabled: true
action: "BLOCK"
severity_threshold: "medium"
- category: "harmful_illegal_weapons"
enabled: true
action: "BLOCK"
severity_threshold: "medium"
litellm_settings:
num_retries: 0 # Disabled — our router handles retry
request_timeout: 600 # Match our 10-min llama-server timeout
@@ -248,90 +321,148 @@ resp.headers["X-Usage-Tokens"] = json.dumps({
---
## 5. Deployment Plan (3 Phases)
## 5. Deployment Plan (4 Phases) — Zero-Downtime Strategy
### Phase 1: Shadow Mode (Week 1) — Zero Risk
### Phase 0: Infrastructure Prep (Current Week) — Zero Downtime
**Goal:** Deploy LiteLLM alongside existing router, test in shadow mode.
**Goal:** Prepare CT 116 infrastructure without affecting running agents.
**Tasks:**
1. **Set up DNS split-horizon on CT 116**
```bash
# On CT 116 host (syslog-api)
echo "192.168.68.11 auth.sysloggh.net" >> /etc/hosts
```
Add to docker-compose.yml (see §7):
```yaml
extra_hosts:
- "auth.sysloggh.net:192.168.68.11"
```
2. **Deploy Postgres container** alongside existing services:
```bash
cd /opt/litellm
# Add postgres to docker-compose.yml
docker compose up -d postgres
```
3. **Replace LiteLLM config** with production config.yaml (see §4.3)
- All 4 models → `http://router:9000/v1`
- Add guardrails (pre-call, post-call, content filter)
- Set `num_retries: 0` (router handles retry)
- Set `request_timeout: 600`
4. **Deploy custom_sso.py** for Authentik OIDC integration
- Mount to LiteLLM container volume
- Reference in config.yaml: `custom_ui_sso_sign_in_handler: custom_sso.custom_ui_sso_sign_in_handler`
5. **Restart LiteLLM container** with new config
```bash
docker compose restart litellm
```
6. **Verify internal routing** — LiteLLM → Router pass-through works:
```bash
curl -X POST http://127.0.0.1:4000/v1/chat/completions \
-H "Authorization: Bearer $LITELLM_MASTER_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"syslog-auto","messages":[{"role":"user","content":"test"}]}'
```
**Verification Checklist:**
- [ ] DNS resolution: `getent hosts auth.sysloggh.net` → 192.168.68.11
- [ ] Postgres container healthy
- [ ] LiteLLM `/health` returns 200
- [ ] LiteLLM → Router pass-through returns valid chat completion
- [ ] GPU health metrics unaffected (watch `/metrics`)
### Phase 1: Shadow Mode (Week 1) — Zero Risk, Zero Downtime
**Goal:** Deploy LiteLLM alongside existing router, test in shadow mode. **Agents continue using :9000 directly.**
```
Agent → LiteLLM (:4000) → Router (:9000) → GPU
(new, testing) (existing, unchanged)
Agent can also directly hit :9000 as fallback
Agent can also directly hit :9000 as fallback (unchanged)
```
**Tasks:**
1. **Deploy Postgres + LiteLLM on docker-vm**
1. **Create virtual keys for test agents** via LiteLLM UI or API:
```bash
cd /opt/litellm
# Apply litellm-fix.sh (already prepared)
docker compose up -d
curl -X POST http://127.0.0.1:4000/key/generate \
-H "Authorization: Bearer $LITELLM_MASTER_KEY" \
-H "Content-Type: application/json" \
-d '{"models":["syslog-auto"],"metadata":{"agent":"test"}}'
```
2. **Create config.yaml** with router as upstream (see §4.3)
3. **Create virtual keys for test agents** via LiteLLM UI
- Mirror existing API_KEYS in LiteLLM's key store
- Set per-key budgets (test with $100 cap)
4. **Verify pass-through works**
2. **Verify pass-through works**
```bash
curl -X POST http://docker-vm:4000/v1/chat/completions \
curl -X POST http://127.0.0.1:4000/v1/chat/completions \
-H "Authorization: Bearer sk-litellm-test-key" \
-H "Content-Type: application/json" \
-d '{"model":"syslog-auto","messages":[{"role":"user","content":"test"}]}'
```
5. **Run 24-hour shadow**: Both :4000 and :9000 active, agents use :9000
3. **Run 24-hour shadow**: Both :4000 and :9000 active, agents use :9000
- Monitor LiteLLM spend logs vs router metrics — confirm parity
- Verify GPU health metrics unaffected
- Check guardrails not generating false positives
### Phase 2: Cutover (Week 2) — Gradual Migration
**Zero-Downtime Guarantee:** Router :9000 remains the primary agent endpoint. LiteLLM :4000 is tested in parallel with no impact on production traffic.
**Goal:** Move agents one-by-one to LiteLLM endpoint.
### Phase 2: Cutover (Week 2) — Gradual Agent Migration, Zero Cumulative Downtime
**Goal:** Move agents one-by-one to LiteLLM endpoint. Each agent is migrated individually — other agents unaffected.
**Tasks:**
1. **Migrate API keys to LiteLLM virtual keys:**
- Create virtual key per agent in LiteLLM UI
- Set model access: `syslog-auto` (default), plus individual GPU models
- Set per-agent budget limits
- Create teams: "Core Agents" (Abiba, Mumuni, Tanko), "Dev Agents" (Kagenz0, Koby, Koonimo)
- Create teams:
- "Core Agents" (Abiba, Mumuni, Tanko) — enterprise tier
- "Dev Agents" (Kagenz0, Koby, Koonimo) — professional tier
2. **Update agent configs:**
- Change `OPENAI_API_BASE` from `http://docker-vm:9000/v1` → `http://docker-vm:4000/v1`
2. **Update agent configs one at a time:**
- Change `OPENAI_API_BASE` from `http://192.168.68.116:9000/v1` → `http://192.168.68.116:4000/v1`
- Replace agent API keys with LiteLLM virtual keys
- Test each agent one at a time
- Test each agent individually — verify routing works
- **Per-agent downtime: <2 minutes**
3. **Migrate admin functions:**
- Key creation/revocation → LiteLLM UI
- Rate limit management → LiteLLM per-key RPM + router hardware RPM (dual enforcement)
- Deprecated key tracking → LiteLLM UI key list
4. **Enable SSO** (optional, Phase 2+):
```yaml
general_settings:
litellm_dashboard_sso: true
sso_provider: "google" # or github, microsoft, keycloak
sso_client_id: os.environ/SSO_CLIENT_ID
sso_client_secret: os.environ/SSO_CLIENT_SECRET
```
4. **Enable SSO via Authentik + custom_sso.py:**
- Authentik OAuth2 provider configured
- NGINX auth_request for /ui/* paths
- Custom UI SSO sign-in handler reads x-authentik-* headers
- Users authenticate with existing Authentik credentials
5. **Keep router :9000 accessible** as emergency fallback for 48 hours
- NGINX configured with router_fallback for 502 errors
- Agents can revert by changing `OPENAI_API_BASE` back to :9000
### Phase 3: Production Hardening (Week 3+) — Optimize
**Zero-Downtime Guarantee:** Each agent has <2 minutes downtime during config update. Router :9000 stays online throughout. Fallback path available for instant rollback.
**Goal:** Lock down, optimize, monitor.
### Phase 3: Production Hardening (Week 3+) — Optimize & Scale
**Goal:** Lock down, optimize, monitor. Prepare for client-facing services.
**Tasks:**
1. **Remove deprecated router endpoints:**
1. **Remove deprecated router endpoints** (only after all agents migrated):
- Drop `/admin/keys/*` — fully migrated to LiteLLM UI
- Drop Phase 0 dual-key logic (LiteLLM handles key rotation)
- Simplify `API_KEYS` to single `ROUTER_API_KEY`
2. **Add LiteLLM observability:**
- Prometheus metrics export
- Slack/email budget alerts
- Slack/email budget alerts via webhook → Hermes
- Daily spend report webhook
3. **Enable LiteLLM caching** (Redis, shared with router):
@@ -343,7 +474,7 @@ Agent can also directly hit :9000 as fallback
cache_ttl: 3600
```
4. **Optional: External model fallbacks**
4. **Add external model fallbacks** for client-facing services:
- Add Anthropic Claude as fallback for code-heavy requests
- Add OpenAI GPT-4o as fallback for reasoning overflow
- LiteLLM's native fallback chains handle this cleanly
@@ -352,9 +483,18 @@ Agent can also directly hit :9000 as fallback
- Keep: routing, slots, health checks, performance recording
- Remove: key management, dual-key logic, admin endpoints
6. **Multi-tenancy setup** for client-facing inference services:
- Organization → Team → User hierarchy per client
- Per-client budget limits and guardrail policies
- Self-service onboarding via Authentik SSO → LiteLLM UI
**Zero-Downtime Guarantee:** All changes are additive — new features added while existing routing continues uninterrupted. Router remains the GPU intelligence layer throughout.
---
## 6. Nginx Configuration
---
## 6. Nginx Configuration (with Authentik OIDC Forward Auth)
The existing nginx config routes `/admin/` → router :9000. This MUST change:
@@ -364,44 +504,77 @@ The existing nginx config routes `/admin/` → router :9000. This MUST change:
# proxy_pass http://127.0.0.1:9000/admin/;
# }
# NEW
# === Authentik auth subrequest endpoint ===
location /authentik/auth {
internal;
# Proxy to Authentik's outpost on acerpve (192.168.68.11)
# Uses internal /etc/hosts resolution: auth.sysloggh.net → 192.168.68.11
proxy_pass https://auth.sysloggh.net/outpost.goauthentik.io/auth/nginx;
proxy_pass_request_body off;
proxy_set_header Content-Length "";
proxy_set_header X-Original-URL $scheme://$http_host$request_uri;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
# === LiteLLM Admin UI — Authentik-protected ===
location /ui/ {
auth_request /authentik/auth;
auth_request_set $auth_user $upstream_http_x_authentik_username;
auth_request_set $auth_email $upstream_http_x_authentik_email;
proxy_set_header X-Authentik-Username $auth_user;
proxy_set_header X-Authentik-Email $auth_email;
proxy_pass http://127.0.0.1:4000/ui/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
# === LiteLLM SSO callback (for OIDC redirect flow) ===
location /sso/callback {
proxy_pass http://127.0.0.1:4000/sso/callback;
proxy_set_header Host $host;
}
# === API endpoint — Bearer token auth (no Authentik) ===
location /v1/ {
# Primary: LiteLLM gateway
proxy_pass http://127.0.0.1:4000/v1/;
proxy_set_header Host $host;
proxy_read_timeout 600s;
# Fallback: direct router (if LiteLLM down)
# error_page 502 = @router_fallback;
error_page 502 = @router_fallback;
}
location @router_fallback {
proxy_pass http://127.0.0.1:9000/v1/;
proxy_set_header Host $host;
}
# === Key management API (needs master_key, not Authentik) ===
location /key/ {
proxy_pass http://127.0.0.1:4000/key/;
proxy_set_header Host $host;
proxy_set_header Authorization $http_authorization;
}
# Keep router metrics accessible (not behind LiteLLM)
location /router/ {
proxy_pass http://127.0.0.1:9000/;
# Rewrite /router/stream → :9000/stream
# Rewrite /router/metrics → :9000/metrics
}
# Health check — combines both layers
location /health {
# Check LiteLLM first, then router
proxy_pass http://127.0.0.1:4000/health;
}
```
---
## 7. Docker Compose (`docker-compose.yml` on docker-vm)
## 7. Docker Compose (`docker-compose.yml` on CT 116)
**Deployment host:** CT 116 `syslog-api` (192.168.68.116) on minipve. All services co-located.
```yaml
services:
@@ -409,15 +582,19 @@ services:
litellm:
image: ghcr.io/berriai/litellm:main-stable
network_mode: "host"
extra_hosts:
- "auth.sysloggh.net:192.168.68.11"
volumes:
- ./config.yaml:/app/config.yaml:ro
- ./custom_sso.py:/app/custom_sso.py:ro
environment:
- LITELLM_MASTER_KEY=${LITELLM_MASTER_KEY}
- UI_USERNAME=admin
- UI_PASSWORD=${UI_PASSWORD}
- DATABASE_URL=postgresql://litellm:${POSTGRES_PASSWORD}@localhost:5432/litellm
- STORE_MODEL_IN_DB=True
- ROUTER_API_KEY=${ROUTER_API_KEY}
- OPENAI_API_KEY=${OPENAI_API_KEY} # Optional: external fallback
- ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY} # Optional: external fallback
- PROXY_BASE_URL=https://litellm.sysloggh.net
command:
- --config
- /app/config.yaml
@@ -445,6 +622,17 @@ services:
retries: 5
restart: unless-stopped
# Nginx also needs auth resolution
nginx:
image: nginx:alpine
extra_hosts:
- "auth.sysloggh.net:192.168.68.11"
volumes:
- ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
ports:
- "80:80"
restart: unless-stopped
# Layer 2: Custom Router (Intelligence & Hardware)
# Already deployed separately — not in this compose file
# The router is managed by the existing harness deployment on CT 116
@@ -486,43 +674,151 @@ volumes:
---
## 10. Migration Commands (Quick Reference)
## 10. Migration Commands (Quick Reference) — CT 116
**Target:** CT 116 `syslog-api` (192.168.68.116), minipve. All services co-located.
```bash
# On docker-vm (CT 116):
# On CT 116 (SSH via minipve: pct exec 116 bash):
# 1. Deploy LiteLLM stack
# === Phase 0: Infrastructure Prep ===
# 0. DNS split-horizon fix
echo "192.168.68.11 auth.sysloggh.net" >> /etc/hosts
getent hosts auth.sysloggh.net # Verify → 192.168.68.11
# 1. Navigate to harness deployment directory
cd /opt/litellm
docker compose down -v # Clean slate
docker compose up -d # Postgres + LiteLLM
# 2. Verify
curl http://localhost:4000/health
curl http://localhost:4000/ui # Admin dashboard
# 2. Deploy Postgres (LiteLLM DB)
docker compose up -d postgres
# 3. Create first virtual key via UI or CLI
docker compose exec litellm litellm-proxy keys create \
--key-alias "abiba-test" \
--models "syslog-auto" \
--max-budget 10.0 \
--team-id "core-agents"
# 3. Replace LiteLLM config with production config.yaml (see §4.3)
# - All models → http://router:9000/v1
# - Guardrails enabled
# - custom_sso.py mounted
# 4. Test end-to-end
curl -X POST http://localhost:4000/v1/chat/completions \
# 4. Restart LiteLLM with new config
docker compose restart litellm
# 5. Verify core health
curl http://127.0.0.1:4000/health # LiteLLM alive
curl http://127.0.0.1:9000/health # Router alive
curl http://127.0.0.1:4000/health/liveliness # LiteLLM readiness
# 6. Test LiteLLM → Router pass-through
curl -X POST http://127.0.0.1:4000/v1/chat/completions \
-H "Authorization: Bearer $LITELLM_MASTER_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"syslog-auto","messages":[{"role":"user","content":"ping"}]}'
# Expected: valid chat completion with GPU routing metadata
# === Phase 1: Shadow Mode ===
# 7. Create virtual key for testing
curl -X POST http://127.0.0.1:4000/key/generate \
-H "Authorization: Bearer $LITELLM_MASTER_KEY" \
-H "Content-Type: application/json" \
-d '{"models":["syslog-auto","qwen3.6-35B-A3B","qwen3.6-27B-code","gemma-4-12b"],"metadata":{"agent":"test"},"max_budget":100}'
# 8. Test with virtual key
curl -X POST http://127.0.0.1:4000/v1/chat/completions \
-H "Authorization: Bearer <virtual-key>" \
-d '{"model":"syslog-auto","messages":[{"role":"user","content":"Hello"}]}'
-H "Content-Type: application/json" \
-d '{"model":"syslog-auto","messages":[{"role":"user","content":"Hello from LiteLLM!"}]}'
# 5. Update nginx (see §6)
# 9. Verify NGINX → Authentik routing (after Phase 2)
curl -I http://127.0.0.1/ui/ # Should return 302 → Authentik login
curl -I http://127.0.0.1/v1/chat/completions # Should proxy to LiteLLM :4000
# 10. Monitor both layers
curl http://127.0.0.1:4000/global/spend/logs # LiteLLM spend
curl http://127.0.0.1:9000/metrics # Router GPU metrics
curl http://127.0.0.1:9000/stream # Router SSE dashboard
# 11. NGINX reload after config changes
nginx -t && nginx -s reload
# 6. Monitor both layers
curl http://localhost:4000/global/spend/logs # LiteLLM spend
curl http://localhost:9000/metrics # Router GPU metrics
curl http://localhost:9000/stream # Router SSE dashboard
# 12. View logs
docker compose logs -f litellm # LiteLLM gateway logs
docker compose logs -f nginx # Reverse proxy logs
```
---
## 11. GitOps Workflow
All harness changes tracked in `SyslogSolution/syslog-harness` on Gitea (http://192.168.68.17:3000). Multiple agents (Abiba, Mumuni, Kagenz0, Agent Zero) collaborate on this repo.
**Branching Strategy:**
- `main` — Production-ready, deployed on CT 116
- `dev/<feature>` — Feature branches for each migration phase
- PR required for main merges (allows cross-agent review)
**Conventional Commits:**
```
feat(router): add X-Usage-Tokens header for LiteLLM cost tracking
fix(docker): add extra_hosts for Authentik internal DNS
chore(plan): update LITELLM-MIGRATION-PLAN.md with Phase 0-3 zero-downtime
refactor(router): remove deprecated /admin/keys/* endpoints (Phase 3)
infra(dns): configure /etc/hosts split-horizon for CT 116
```
**Commit Signatures (for audit trail):**
- Agent commits include source: `Co-authored-by: Abiba <abiba@sysloggh.net>`
- Human commits: `Signed-off-by: Jerome Tabiri <jerome@sysloggh.net>`
**Deployment Flow:**
1. PR opened on Gitea (feature → main)
2. Review by at least one other agent or Jerome
3. Merge to main
4. `git pull` on CT 116 (`/opt/litellm/`)
5. `docker compose up -d` (rolling update)
**Post-Deployment Verification:**
- Check LiteLLM `/health` — 200 OK
- Check Router `/metrics` — GPU health unchanged
- Check Dashboard SSE — streaming live
- Test agent inference — valid chat completion
---
## 12. Zero-Downtime Migration Strategy
**Principle:** Agents must experience **zero cumulative downtime** throughout the migration. Router `:9000` is the safety net — never taken offline until all agents verified on LiteLLM.
**Per-Agent Migration (2 min downtime each):**
1. Create LiteLLM virtual key for agent (no impact on running agent)
2. Update agent config: `OPENAI_API_BASE` → `:4000`, `OPENAI_API_KEY` → virtual key
3. Restart agent (2 min)
4. Verify inference works via LiteLLM → Router → GPU
5. If issue: revert `OPENAI_API_BASE` → `:9000`, restart (30 sec rollback)
**Dual-Path Architecture (throughout migration):**
```
Path A: Agent → LiteLLM :4000 → Router :9000 → GPU (new, for migrated agents)
Path B: Agent → Router :9000 → GPU (legacy, for unmigrated agents)
```
Both paths work simultaneously. Router unchanged in both paths.
**Rollback Procedure (per agent, 30 seconds):**
1. Set `OPENAI_API_BASE=http://192.168.68.116:9000/v1`
2. Set `OPENAI_API_KEY=<original-agent-key>`
3. Restart agent
**Emergency Global Rollback (if LiteLLM catastrophic failure):**
1. NGINX auto-fallback: `error_page 502 = @router_fallback` → traffic routes to :9000
2. Manual: Update all agent configs back to :9000 (scripted)
3. Restart all agents (5 min)
**Health Checks (continuously monitored):**
- `/health` — LiteLLM + Router combined status
- `/router/metrics` — GPU health, circuit breaker, active slots
- `/stream` — SSE dashboard (real-time)
- Redis: `circuit:*` keys for tripped breakers
---
## Appendix A: Router Slim-Down (Phase 3)
After full migration, `router-fixed.py` can be simplified by removing:
@@ -574,4 +870,5 @@ litellm_settings:
---
*Plan drafted: 2026-06-13 by Abiba 🦊⚡*
*Status: Ready for Kwame review*
*Updated: 2026-06-14 by Agent Zero (kagentz-bot) — CT 116 deployment target, DNS split-horizon, Authentik OIDC, Zero-Downtime strategy, GitOps workflow*
*Status: Ready for Phase 0 deployment*