tune: raise Tier 2 threshold 4K→10K tok, 6→10 turns for VLM

More conversations now route to VLM as primary. 9B VLM has 262K
context window and 88 tok/s average — well suited for moderate
conversations. Dense absorbs overflow and heavy reasoning.
This commit is contained in:
Abiba
2026-05-27 00:29:25 +00:00
parent 93d0d3cc4b
commit 621a897bec
+2 -2
View File
@@ -218,8 +218,8 @@ def route(rd, tier):
result = select_best_gpu(fallback, "lightweight_fallback") result = select_best_gpu(fallback, "lightweight_fallback")
if result: return result if result: return result
# TIER 2: Simple conversations — short context, any prompt → VLM first, Dense second # TIER 2: Simple conversations — VLM primary (up to 10K tok), any prompt → VLM first, Dense second
if t <= 4000 and turns <= 6 and "qwen3.5-9b-vlm" in avail: if t <= 10000 and turns <= 10 and "qwen3.5-9b-vlm" in avail:
if not is_gpu_busy("qwen3.5-9b-vlm"): if not is_gpu_busy("qwen3.5-9b-vlm"):
return {"model":"qwen3.5-9b-vlm","reason":"simple_conv"} return {"model":"qwen3.5-9b-vlm","reason":"simple_conv"}
# VLM busy — fall back to Dense, then MoE # VLM busy — fall back to Dense, then MoE