SyslogSolution/prose-contracts
7
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

gpu-self-heal: refresh to current fleet baseline and topology #21

Merged
jerome merged 3 commits from feat/gpu-self-heal-refresh-20260718 into master 2026-07-18 08:36:37 +00:00
Conversation 0 Commits 3 Files Changed 2
+119 -83
3 Commits
Author SHA1 Message Date
root bddbb22f03 gpu-self-heal: refresh to current fleet baseline and topology
PR Pipeline — Authorize → Validate → Review → Merge / auth (pull_request) Successful in 4s
Details
PR Pipeline — Authorize → Validate → Review → Merge / validate (pull_request) Successful in 1s
Details
PR Pipeline — Authorize → Validate → Review → Merge / lint (pull_request) Successful in 4s
Details
PR Pipeline — Authorize → Validate → Review → Merge / ai-review (pull_request) Successful in 5s
Details
PR Pipeline — Authorize → Validate → Review → Merge / gate (pull_request) Successful in 1s
Details 
- Synced model assignments to 2026-07-17 swaps (ThinkingCap, HauhauCS QAT, Genesis Hermes V3)
- Added stable role-based aliases from gpu-fleet (gpu-dense, gpu-light, strix-moe)
- Updated benchmark baselines to live values (74.9/169.6/62.9 tok/s)
- Replaced router (port 9000) references with LiteLLM + direct routing
- Replaced Prometheus exporter rule with sidecar health probe
- Updated VRAM thresholds to match operational data (300/300/200 MB/h)
- Added response size limit (1MB) to prevent OOM crashes
- Added Lessons L5 (response size crash) and L6 (stable aliases)
- Removed deprecated Rules 11-12 (router-specific distribution balance)
 2026-07-18 08:07:03 +00:00
root 31ec70ae36 gpu-fleet: RTX 3090 swap to ThinkingCap-Qwen3.6-27B Q4_K_M 
- Model: Qwopus Q4_K_M (16GB, 63 tok/s) → ThinkingCap Q4_K_M (15.7GB, 68 tok/s)
- RL-finetuned: 50% fewer thinking tokens, 0.85 MMLU-Pro (vs 0.83 base)
- Self-spec MTP (n=4) REQUIRED for stability — segfaults without it
- Added vision via mmproj (0.9GB) — new capability for this GPU
- VRAM: 20.9/24.6GB (85%), tighter but stable
- Outputs reasoning_content (hidden from Hermes agent)
 2026-07-17 15:50:50 +00:00
root 5eb6d3bfbd gpu-fleet: RTX 5070 swap to HauhauCS Gemma4-12B QAT Uncensored Balanced 
- Model: IQ4_NL (6.3GB, 191 tok/s) → Q4_K_M QAT (6.9GB, 87 tok/s)
- MTP draft: Q8_0 (444MB) → tuned draft (242MB), saves 200MB VRAM
- mmproj: F16 → BF16 (same size, matched to new model)
- Benefits: 0/465 refusals, agent-optimized tuning, QAT quality
- Trade: 54% slower generation (acceptable for gpu-light role)
- Config: --parallel 1, --ctx-size 131072, single-slot full 128K
- VRAM: 10.0/12.2GB (82%), healthy headroom
 2026-07-17 15:24:13 +00:00