From 1e24bc3b9bd04a685db85a1542eff48720d950ef Mon Sep 17 00:00:00 2001 From: Abiba Bot Date: Fri, 5 Jun 2026 23:49:03 +0000 Subject: [PATCH] docs: update model table with context windows, capabilities, GPU labels --- README.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 877a107..c5f42d9 100644 --- a/README.md +++ b/README.md @@ -6,9 +6,9 @@ CT 116 Docker stack for routing local GPU models through a unified OpenAI-compat ``` nginx :80 → router :9000 → GPU backends - ├─ qwen3.6-35B-A3B (MoE) @ 192.168.68.15:8080 [2 slots] - ├─ qwen3.6-27B-code (Dense) @ 192.168.68.8:8080 [2 slots] - └─ gemma-4-12b (VLM) @ 192.168.68.110:8080 [2 slots] + ├─ qwen3.6-35B-A3B (MoE) @ 192.168.68.15:8080 [2 slots, 262K ctx] + ├─ qwen3.6-27B-code (Dense) @ 192.168.68.8:8080 [2 slots, 262K ctx] + └─ gemma-4-12b (VLM) @ 192.168.68.110:8080 [2 slots, 262K ctx] Total: 6 concurrent slots LiteLLM :8081 (fallback) | Dashboard :3000 | Redis :6379 (local) @@ -59,11 +59,11 @@ When all GPUs are saturated, requests enter a polling queue (500ms intervals) in ## Models -| GPU | Model | VRAM | Slots | +| GPU | Model | VRAM | Slots | Context | Best For | |-----|-------|------|-------| -| Strix Halo | qwen3.6-35B-A3B (MoE) | 65GB | 2 | -| RTX 3090 | qwen3.6-27B-code (Dense) | 24GB | 2 | -| New Backend | gemma-4-12b (VLM) | 7.1GB | 2 | +| Strix Halo | qwen3.6-35B-A3B (MoE) | 65GB | 2 | 262K | General quality | +| RTX 3090 | qwen3.6-27B-code (Dense) | 24GB | 2 | 262K | Code, reasoning | +| RTX 5070 | gemma-4-12b (VLM) | 12GB | 2 | 262K | Speed, vision | ## Maintenance