kagentz-botandAbiba
84e0d163ee
- Target deployment host: CT 116 syslog-api (192.168.68.116) on minipve - DNS split-horizon: Option A /etc/hosts for auth.sysloggh.net → 192.168.68.11 - Add guardrails config (pre-call, post-call, content filter) - Add custom_sso.py for Authentik OIDC via x-authentik-* headers - Expand from 3-phase to 4-phase deployment with zero-downtime strategy - Add Phase 0: Infrastructure Prep (DNS, Postgres, config, custom_sso) - Add §11 GitOps workflow (branching, conventional commits, deployment flow) - Add §12 Zero-Downtime Migration Strategy (per-agent 2min, global rollback) - Update nginx config with Authentik forward auth + router fallback - Update docker-compose.yml for CT 116 with extra_hosts + postgres - Add multi-provider config placeholders (OpenAI, Anthropic) Co-authored-by: Abiba <abiba@sysloggh.net>
syslog-harness — Inference API Harness
CT 116 Docker stack for routing local GPU models through a unified OpenAI-compatible API.
Architecture
nginx :80 → router :9000 → GPU backends
├─ qwen3.6-35B-A3B (MoE) @ 192.168.68.15:8080 [2 slots, 262K ctx]
├─ qwen3.6-27B-code (Dense) @ 192.168.68.8:8080 [2 slots, 262K ctx]
└─ gemma-4-12b (VLM) @ 192.168.68.110:8080 [2 slots, 262K ctx]
Total: 6 concurrent slots
LiteLLM :8081 (fallback) | Dashboard :3000 | Redis :6379 (local)
Deploy
cd /opt/inference-harness
docker compose up -d
Endpoints
| URL | Purpose |
|---|---|
/v1/chat/completions |
Inference API (OpenAI-compatible) — API key required |
/v1/models |
Available models |
/ |
Dashboard (GPU health, routing, agents, timeseries) |
Authentication
All /v1/chat/completions requests require a valid API key via Authorization: Bearer <key>. Missing or invalid keys return 401 Unauthorized.
Agent API Keys
| Agent | Key |
|---|---|
| Abiba | sk-syslog-abiba |
| Mumuni | sk-syslog-mumuni |
| Tanko | sk-syslog-tanko |
| Koby | sk-syslog-koby |
| Kagenz0 | sk-syslog-kagenz0 |
| Koonimo | sk-syslog-koonimo |
Routing Tiers
| Tier | Trigger | Priority |
|---|---|---|
| Lightweight | No system prompt, ≤1 turn, ≤100 words | VLM → MoE → Dense |
| Simple Conv | ≤1000 tokens, ≤4 turns | VLM → MoE → Dense |
| Heavy | >4000 tokens OR >8 turns | Dense → MoE → VLM |
| Default | Everything else | MoE → VLM → Dense |
Queue
When all GPUs are saturated, requests enter a polling queue (500ms intervals) instead of returning 503 immediately. Timeout: 30s (configurable via QUEUE_TIMEOUT env or X-Queue-Timeout header).
Models
| GPU | Model | VRAM | Slots | Context | Best For | |-----|-------|------|-------| | Strix Halo | qwen3.6-35B-A3B (MoE) | 65GB | 2 | 262K | General quality | | RTX 3090 | qwen3.6-27B-code (Dense) | 24GB | 2 | 262K | Code, reasoning | | RTX 5070 | gemma-4-12b (VLM) | 12GB | 2 | 262K | Speed, vision |
Maintenance
Automated cron job runs daily at 3:00 AM UTC (/opt/inference-harness/maintenance.sh):
- Cleans Redis timeseries keys >60 days
- Prunes Docker build cache >7 days
- Logs container health and Redis memory
Logs: /var/log/harness-maintenance.log