restructure: routing prioritized by reasoning requirements
Tier 1 (Lightweight): VLM → Dense → MoE ≤500 tok, 1 turn
Tier 2 (Simple): VLM → Dense → MoE ≤15K tok, ≤12 turns (was 10K/10)
Tier 3 (Medium): Dense → VLM → MoE ≤25K tok
Tier 4 (Heavy): MoE → Dense → VLM >25K tok (MoE PRIMARY workhorse)
Tier 5 (Default): MoE → Dense → VLM MoE primary fallback
Target: MoE ~50% (heavy primary), VLM ~25% (raised simple + fallback),
Dense ~25% (medium primary + heavy fallback)
Removed turn limit from Medium tier — Simple tier handles conversational
requests up to 12 turns now.
This commit is contained in:
+9
-9
@@ -218,8 +218,8 @@ def route(rd, tier):
|
||||
result = select_best_gpu(fallback, "lightweight_fallback")
|
||||
if result: return result
|
||||
|
||||
# TIER 2: Simple conversations — VLM primary (up to 10K tok), any prompt → VLM first, Dense second
|
||||
if t <= 10000 and turns <= 10 and "qwen3.5-9b-vlm" in avail:
|
||||
# TIER 2: Simple conversations — VLM primary (up to 15K tok), fastest for moderate chat
|
||||
if t <= 15000 and turns <= 12 and "qwen3.5-9b-vlm" in avail:
|
||||
if not is_gpu_busy("qwen3.5-9b-vlm"):
|
||||
return {"model":"qwen3.5-9b-vlm","reason":"simple_conv"}
|
||||
# VLM busy — fall back to Dense, then MoE
|
||||
@@ -227,20 +227,20 @@ def route(rd, tier):
|
||||
result = select_best_gpu(fallback, "simple_conv_fallback")
|
||||
if result: return result
|
||||
|
||||
# TIER 3: Medium complexity — Dense primary (speed), MoE fallback
|
||||
if t <= 25000 and turns <= 15:
|
||||
# TIER 3: Medium complexity — Dense primary, VLM fallback (quality + speed balance)
|
||||
if t <= 25000:
|
||||
candidates = [m for m in ["qwen3.6-27B-code","qwen3.5-9b-vlm","qwen3.6-35B-A3B"] if m in avail]
|
||||
result = select_best_gpu(candidates, "medium")
|
||||
if result: return result
|
||||
|
||||
# TIER 4: Heavy reasoning — Dense first, VLM second (262K ctx), MoE last (thermal)
|
||||
if t > 25000 or turns > 15:
|
||||
candidates = [m for m in ["qwen3.6-27B-code","qwen3.5-9b-vlm","qwen3.6-35B-A3B"] if m in avail]
|
||||
# TIER 4: Heavy reasoning — MoE primary (workhorse), Dense fallback
|
||||
if t > 25000:
|
||||
candidates = [m for m in ["qwen3.6-35B-A3B","qwen3.6-27B-code","qwen3.5-9b-vlm"] if m in avail]
|
||||
result = select_best_gpu(candidates, "heavy_reasoning")
|
||||
if result: return result
|
||||
|
||||
# TIER 5: Default — balanced distribution: Dense first (speed), MoE second (capacity)
|
||||
candidates = [m for m in ["qwen3.6-27B-code","qwen3.5-9b-vlm","qwen3.6-35B-A3B"] if m in avail]
|
||||
# TIER 5: Default — MoE primary, Dense fallback
|
||||
candidates = [m for m in ["qwen3.6-35B-A3B","qwen3.6-27B-code","qwen3.5-9b-vlm"] if m in avail]
|
||||
result = select_best_gpu(candidates, "default")
|
||||
if result: return result
|
||||
return {"model":avail[0],"reason":"last_resort"}
|
||||
|
||||
Reference in New Issue
Block a user