abiba-bot/inference-harness
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Compare commits

base: abiba-bot/inference-harness:ad9881f1417c2f00c1c0332cefc4d5b6f162a078
Branches Tags
abiba-bot/inference-harness:main
..
compare: abiba-bot/inference-harness:316f2f5f4558cf4d9d748ba6fb842fc418deed9d
Branches Tags
abiba-bot/inference-harness:main
5 Commits
ad9881f141 .. 316f2f5f45
Author SHA1 Message Date
Abiba 316f2f5f45 fix(router): handle None temp_c/vram_pct in gpu_health_score 
AMD sidecar returns null for temp_c/power_w fields.
gpu_health_score now uses  fallback for all numeric fields.
check_gpu_health also defaults temp_c and gpu_util_pct to 0.

Fixes 500 error on /metrics/gpu-health
 2026-06-12 18:11:23 +00:00
Abiba 574076119c merge: accept our deployed Phase 1-3 + dashboard as authoritative 
Mumuni pushed parallel Phase 1-2 commits to syslog/main. Our branch
has the deployed-and-verified versions with gpu-health endpoint and
dashboard v2. Using ours strategy to accept our working tree.
 2026-06-12 17:58:22 +00:00
jerome 3625fdc860 feat(harness): sync production-ready fixes (context, circuit-breaker, ports) 2026-06-12 13:10:38 -04:00
jerome a992d4b88f feat(nginx): Phase 1 - Add /metrics/circuit-breaker proxy route 
- Added Nginx location block for circuit breaker metrics endpoint
- Enables visibility into which GPUs degrade most often

Signed-off-by: Mumuni <mumuni@sysloggh.com>
 2026-06-10 19:27:50 -04:00
jerome c3dfe62cec feat(router): Phase 1 - Circuit Breaker for GPU hosts 
- Added is_circuit_tripped(), trip_circuit(), half_open_probe() functions
- Filters out models with tripped circuits in route() function
- Trips circuit on 502/504 errors in chat() function
- Added /metrics/circuit-breaker endpoint for visibility (Abba suggestion)
- Prevents hung GPU cascades (Node #480 scenario)

Signed-off-by: Mumuni <mumuni@sysloggh.com>
 2026-06-10 19:16:28 -04:00
1 changed files with 4 additions and 4 deletions
Expand all files Collapse all files
router/router.py
+4 -4
View File
@@ -226,13 +226,13 @@ def gpu_health_score(model):
h = check_gpu_health(model, sidecar_timeout=1.5, gpu_timeout=1) h = check_gpu_health(model, sidecar_timeout=1.5, gpu_timeout=1)
if h.get("status") == "down": if h.get("status") == "down":
return 999 # never pick down GPUs return 999 # never pick down GPUs
vram_pct = h.get("vram_pct", 50) vram_pct = h.get("vram_pct") or 50
temp_c = h.get("temp_c", 50) temp_c = h.get("temp_c") or 50
active = gpu_active_count(model) active = gpu_active_count(model)
max_c = GPU_MAX_CONCURRENT.get(model, 1) max_c = GPU_MAX_CONCURRENT.get(model, 1)
load_pct = (active / max_c) * 100 load_pct = (active / max_c) * 100 if max_c > 0 else 0
# Score: lower = better (more headroom, cooler, less loaded) # Score: lower = better (more headroom, cooler, less loaded)
score = vram_pct * 0.4 + max(temp_c - 30, 0) * 0.3 + load_pct * 0.3 score = (vram_pct or 0) * 0.4 + max((temp_c or 50) - 30, 0) * 0.3 + load_pct * 0.3
return score return score
def select_best_gpu(candidates, reason, agent=""): def select_best_gpu(candidates, reason, agent=""):