fix(litellm-health): three-state model verdicts - busy is not a failure #123

Merged

abiba-bot merged 3 commits from fix/litellm-health-busy-vs-down-20260919 into master 2026-09-19 11:57:27 +00:00

Conversation 0 Commits 3 Files Changed 1

+58 -18

3 Commits

Author	SHA1	Message	Date
root	9a2ee6faec	fix(litellm): Remove duplicate 'host healthy' from busy line ... The busy line was rendering as: 'busy (completion timed out after retry; host healthy host healthy (200))' because host_detail already contains 'host healthy (200)' and the prefix also said 'host healthy'. Fixed to: 'busy (completion timed out after retry; host healthy (200))' F1 cosmetic fix from PR #123 verify.	2026-09-19 11:53:32 +00:00
root	a12abbeb14	fix(litellm): Implement busy vs down with proper degraded state ... Three states: - healthy: passed, exit 0 (unchanged) - busy (completion timed out after retry AND host /health answered): ⚠️ DEGRADED line, does NOT fail the run, exit 0 - host unreachable or real fault: ❌, exit 1 (unchanged) Summary now reports degraded count: - All pass, no degraded: '✅ All checks passed' - All pass, 1+ degraded: '✅ All checks passed (1 degraded: gpu-dense)' - Some failed: '❌ Some checks failed' or '❌ Some checks failed (1 degraded: ...)' Host health mapping verified: - gpu-dense -> 192.168.68.8:8080/health - gpu-vision -> 192.168.68.110:8080/health - strix-moe -> 192.168.68.15:8080/health	2026-09-19 11:43:39 +00:00
root	0b9aebca37	fix(litellm): Fix timeout kind reporting + add busy/degraded detection ... 1. TIMEOUT KIND FIX: run_command returns (1, '', 'TIMEOUT') when its own timeout fires. probe_http now checks for this before falling through to 'curl exit <rc>', so a 30s timeout reports 'timeout after 30s' not 'curl exit 1'. 2. BUSY/DEGRADED DETECTION: After both model probes fail, check the model's host health endpoint (e.g. 192.168.68.8:8080/health for gpu-dense). If the host answers 200, report 'busy (completion timed out after retry; host healthy 200)' — do NOT fail the run on that alone. If the host does not answer, that's a real FAIL. 3. RETRY TIMEOUT INCREASED: Single-host retry timeout raised from 45s to 90s. Worst-case prefill on a single-slot .8 host is ~76s (observed 83K-token prompt at 1078 tok/s), so 90s covers it. New line shapes: - Busy: 'gpu-dense: busy (completion timed out after retry; host healthy 200)' - Real failure: 'probe-failed: gpu-dense timeout after 30s then timeout after 90s (2 attempts)'	2026-09-19 11:42:36 +00:00