SyslogSolution/syslog-harness
7
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: update model table with context windows, capabilities, GPU labels

Browse Source
This commit is contained in:
Abiba Bot
2026-06-05 23:49:03 +00:00
parent e6a6c30211
commit 1e24bc3b9b
1 changed files with 7 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+7 -7
View File
@@ -6,9 +6,9 @@ CT 116 Docker stack for routing local GPU models through a unified OpenAI-compat
```
nginx :80 → router :9000 → GPU backends
├─ qwen3.6-35B-A3B (MoE) @ 192.168.68.15:8080 [2 slots]
├─ qwen3.6-27B-code (Dense) @ 192.168.68.8:8080 [2 slots]
└─ gemma-4-12b (VLM) @ 192.168.68.110:8080 [2 slots]
├─ qwen3.6-35B-A3B (MoE) @ 192.168.68.15:8080 [2 slots, 262K ctx]
├─ qwen3.6-27B-code (Dense) @ 192.168.68.8:8080 [2 slots, 262K ctx]
└─ gemma-4-12b (VLM) @ 192.168.68.110:8080 [2 slots, 262K ctx]
Total: 6 concurrent slots
LiteLLM :8081 (fallback) | Dashboard :3000 | Redis :6379 (local)
@@ -59,11 +59,11 @@ When all GPUs are saturated, requests enter a polling queue (500ms intervals) in
## Models
| GPU | Model | VRAM | Slots |
| GPU | Model | VRAM | Slots | Context | Best For |
|-----|-------|------|-------|
| Strix Halo | qwen3.6-35B-A3B (MoE) | 65GB | 2 |
| RTX 3090 | qwen3.6-27B-code (Dense) | 24GB | 2 |
| New Backend | gemma-4-12b (VLM) | 7.1GB | 2 |
| Strix Halo | qwen3.6-35B-A3B (MoE) | 65GB | 2 | 262K | General quality |
| RTX 3090 | qwen3.6-27B-code (Dense) | 24GB | 2 | 262K | Code, reasoning |
| RTX 5070 | gemma-4-12b (VLM) | 12GB | 2 | 262K | Speed, vision |
## Maintenance