tune: raise Tier 2 threshold 4K→10K tok, 6→10 turns for VLM
More conversations now route to VLM as primary. 9B VLM has 262K context window and 88 tok/s average — well suited for moderate conversations. Dense absorbs overflow and heavy reasoning.
This commit is contained in:
+2
-2
@@ -218,8 +218,8 @@ def route(rd, tier):
|
|||||||
result = select_best_gpu(fallback, "lightweight_fallback")
|
result = select_best_gpu(fallback, "lightweight_fallback")
|
||||||
if result: return result
|
if result: return result
|
||||||
|
|
||||||
# TIER 2: Simple conversations — short context, any prompt → VLM first, Dense second
|
# TIER 2: Simple conversations — VLM primary (up to 10K tok), any prompt → VLM first, Dense second
|
||||||
if t <= 4000 and turns <= 6 and "qwen3.5-9b-vlm" in avail:
|
if t <= 10000 and turns <= 10 and "qwen3.5-9b-vlm" in avail:
|
||||||
if not is_gpu_busy("qwen3.5-9b-vlm"):
|
if not is_gpu_busy("qwen3.5-9b-vlm"):
|
||||||
return {"model":"qwen3.5-9b-vlm","reason":"simple_conv"}
|
return {"model":"qwen3.5-9b-vlm","reason":"simple_conv"}
|
||||||
# VLM busy — fall back to Dense, then MoE
|
# VLM busy — fall back to Dense, then MoE
|
||||||
|
|||||||
Reference in New Issue
Block a user