gpu-fleet: July 8 optimization sweep — parallel 2 fleet-wide, 128K ctx on NVIDIA, LiteLLM timeout fixes
PR Pipeline — Authorize → Validate → Review → Merge / auth (push) Successful in 1s
PR Pipeline — Authorize → Validate → Review → Merge / validate (push) Successful in 2s
PR Pipeline — Authorize → Validate → Review → Merge / lint (push) Successful in 2s
PR Pipeline — Authorize → Validate → Review → Merge / ai-review (push) Successful in 1s
PR Pipeline — Authorize → Validate → Review → Merge / gate (push) Successful in 0s

- All GPUs: --parallel 1 → 2 (6 concurrent slots, was 3)
- .8 RTX 3090: ctx 256K→128K, VRAM 96%→83%, turbo4 KV cache
- .110 RTX 5070: ctx 256K→128K, ubatch 4096→512 (was inverted), VRAM 90%→77%, q4_0 KV
- .15 Strix Halo: parallel 1→2, 256K ctx (41GB free), q8_0 KV, AMD metrics via /sys/class/drm
- LiteLLM: gemma timeout 25→120s, qwen timeout 40→90s, syslog-auto (qwen) 40→90s
- Agent configs: context_length 262144 for syslog-auto, 131072 for direct qwen/gemma
- Updated health-check operation, agent config implications, benchmark table (fixed model↔GPU mapping)
This commit is contained in:
root
2026-07-08 17:25:42 +00:00
parent f6e8791352
commit 776df418ca
5 changed files with 149 additions and 29 deletions
+3 -1
View File
@@ -64,7 +64,7 @@ Before ANY update wave:
**Verify after Wave 2:**
- All VMs/CTs running: check via Proxmox API
- LiteLLM healthy: `curl localhost:4000/health/liveliness` (via CT 116)
- GPU servers responding: check :8080 on VM 101, VM 103; check ornith via router
- GPU servers responding: check :8080 on VM 101, VM 103; check ornith via router (http://192.168.68.116/health/unified — .15:8080 is firewalled to .116 only)
- Zulip agents connected: check Mumuni/Tanko gateway state
- Abiba PM2 processes online: `pm2 status`
@@ -120,6 +120,8 @@ Before Wave 1, snapshot these files:
/opt/home_stack/docker-compose.yml (VM 109 .7)
/opt/audiobookshelf/docker-compose.yml (VM 109 .7)
/root/.pi/agent/extensions/config.yaml (CT 100 .24)
/etc/systemd/system/ornith-server.service (amdpve .15)
/etc/systemd/system/llama-server.service (VM 101 .8, VM 103 .110)
```
Run: `mkdir -p /tmp/infra-update-backup-$(date +%Y%m%d) && rsync -av ...`