VLM migration: qwen3.5-9b-vlm → gemma-4-12b across entire harness

- router/router.py: model ID, context window (131K→262K), VRAM comment
- litellm_config.yaml: model name updated
- gpu-router.conf, gpu-router-docker.conf: pool names, comments
- dashboard/dashboard.py: MODELS array refactor for single-point config
- README.md: architecture diagram, model table
This commit is contained in:
Abiba Bot
2026-06-05 21:37:18 +00:00
parent 370f5546dd
commit dace488f93
6 changed files with 36 additions and 31 deletions
+2 -2
View File
@@ -8,7 +8,7 @@ CT 116 Docker stack for routing local GPU models through a unified OpenAI-compat
nginx :80 → router :9000 → GPU backends
├─ qwen3.6-35B-A3B (MoE) @ 192.168.68.15:8080 [2 slots]
├─ qwen3.6-27B-code (Dense) @ 192.168.68.8:8080 [2 slots]
└─ qwen3.5-9b-vlm (VLM) @ 192.168.68.110:8080 [2 slots]
└─ gemma-4-12b (VLM) @ 192.168.68.110:8080 [2 slots]
Total: 6 concurrent slots
LiteLLM :8081 (fallback) | Dashboard :3000 | Redis :6379 (local)
@@ -63,7 +63,7 @@ When all GPUs are saturated, requests enter a polling queue (500ms intervals) in
|-----|-------|------|-------|
| Strix Halo | qwen3.6-35B-A3B (MoE) | 65GB | 2 |
| RTX 3090 | qwen3.6-27B-code (Dense) | 24GB | 2 |
| RTX 5070 | qwen3.5-9b-vlm (VLM) | 12GB | 2 |
| New Backend | gemma-4-12b (VLM) | 7.1GB | 2 |
## Maintenance