SyslogSolution/syslog-harness
7
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: SyslogSolution/syslog-harness:3625fdc86076961ac17a464c56784665a25c64da
Branches Tags
SyslogSolution/syslog-harness:main SyslogSolution/syslog-harness:master
..
compare: SyslogSolution/syslog-harness:574076119c0aa125d269f77d959e7be0e31d975b
Branches Tags
SyslogSolution/syslog-harness:main SyslogSolution/syslog-harness:master
7 Commits
3625fdc860 .. 574076119c
Author SHA1 Message Date
Abiba 574076119c merge: accept our deployed Phase 1-3 + dashboard as authoritative 
Mumuni pushed parallel Phase 1-2 commits to syslog/main. Our branch
has the deployed-and-verified versions with gpu-health endpoint and
dashboard v2. Using ours strategy to accept our working tree.
 2026-06-12 17:58:22 +00:00
Abiba ad9881f141 feat(dashboard): live GPU health scoring + real KPIs 
- Added /metrics/gpu-health endpoint with live health scores (VRAM 40%, temp 30%, load 30%)
- Added /metrics/latency endpoint for dashboard KPIs
- Added GPU_LABELS for human-readable model names
- Dashboard v2: rewired to real data endpoints
  - KPI cards: GPUs online, circuit trips, avg latency, req/min, active requests
  - Health scores from actual gpu_health_score() function
  - Rolling 60-sample history chart (real data, no simulation)
  - Status: green/yellow/red based on tripped circuits
  - No CDN dependency (pure CSS)
  - Auto-refresh every 15s
- nginx: /dashboard/ serves static files with cache headers
- docker-compose: dashboard volume mount

Co-authored-by: Abiba <abiba@sysloggh.com>
 2026-06-12 17:57:46 +00:00
Abiba a860a8fd0f feat(router): Phase 3 - Dynamic GPU Weighting via Health Scoring 2026-06-11 01:06:13 +00:00
Abiba fabbe340d6 feat(router): Phase 2 - Atomic session token tracking via Redis Lua script 2026-06-11 00:57:29 +00:00
Abiba 2e24ee5598 feat(router): Phase 1 - Circuit Breaker + /metrics/circuit-breaker endpoint 2026-06-11 00:47:15 +00:00
Abiba 19f7d90cc1 feat(nginx): Phase 1 - add /metrics/circuit-breaker proxy route 2026-06-11 00:39:21 +00:00
Abiba b79af634d7 feat(router): Phase 1 - Actual Circuit Breaker for GPU hosts 
- Added is_circuit_tripped(), trip_circuit(), half_open_probe() functions
- Filters out models with tripped circuits in route() function
- Trips circuit on 502/504 errors and timeouts in chat() function
- Prevents hung GPU cascades (Node #480 scenario)

Approved by Abiba via relay #635

Signed-off-by: Mumuni <mumuni@sysloggh.com>
 2026-06-11 00:29:53 +00:00
4 changed files with 935 additions and 195 deletions
Expand all files Collapse all files
dashboard/dashboard.html
+145
View File
@@ -0,0 +1,145 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8"><meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Inference Harness - Dashboard</title>
<style>
:root{--bg:#0b0f17;--text:#bcc3cd;--panel:rgba(31,41,55,0.7);--border:rgba(75,85,99,0.4)}
body{background:var(--bg);color:var(--text);font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',sans-serif;margin:0;padding:1.5rem}
.glass-panel{background:var(--panel);backdrop-filter:blur(12px);border:1px solid var(--border);border-radius:0.75rem;overflow:hidden}
.stat-value{font-size:28px;font-weight:700;line-height:1.1}
.stat-label{font-size:11px;text-transform:uppercase;letter-spacing:0.6px;color:#64748b}
.health-bar{height:0.5rem;background:#374151;border-radius:0.375rem;overflow:hidden}
.health-fill{height:100%;transition:width .5s ease}
.dot-green{background:#10b981;animation:pulse 2s infinite}
.dot-yellow{background:#f59e0b;animation:pulse 2s infinite}
.dot-red{background:#ef4444;animation:pulse 2s infinite}
@keyframes pulse{0%,100%{opacity:1}50%{opacity:.8}}
.container{max-width:1400px;margin:0 auto}
.grid-5{display:grid;grid-template-columns:repeat(5,1fr);gap:1rem}
.grid-3{display:grid;grid-template-columns:repeat(3,1fr);gap:1rem}
.grid-2{display:grid;grid-template-columns:1fr 1fr;gap:1.5rem}
.mb-6{margin-bottom:1.5rem}
.p-4{padding:1rem}
.text-center{text-align:center}
.font-bold{font-weight:700}
.text-white{color:#fff}
.text-sm{font-size:.875rem}
.text-xs{font-size:.75rem}
.text-gray-400{color:#9ca3af}
.text-gray-500{color:#6b7280}
.flex{display:flex}
.items-center{align-items:center}
.justify-between{justify-content:space-between}
.gap-4{gap:1rem}
.w-3{width:.75rem}.h-3{height:.75rem}.rounded-full{border-radius:9999px}
.bg-blue-600{background:#2563eb}.bg-blue-600:hover{background:#1d4ed8}
.px-4{padding-left:1rem;padding-right:1rem}.py-2{padding-top:.5rem;padding-bottom:.5rem}
.rounded-lg{border-radius:.5rem}.text-white{color:#fff}
.mt-2{margin-top:.5rem}.mb-2{margin-bottom:.5rem}.mb-3{margin-bottom:.75rem}
.border-t{border-top:1px solid #374151}.pt-4{padding-top:1rem}
.text-xl{font-size:1.25rem}.text-2xl{font-size:1.5rem}.text-lg{font-size:1.125rem}
.font-semibold{font-weight:600}.font-normal{font-weight:400}
.text-emerald-400{color:#34d399}.text-amber-400{color:#fbbf24}.text-red-400{color:#f87171}
</style>
</head>
<body>
<div class="container">
<div class="flex justify-between items-center mb-6">
<div><h1 class="text-2xl font-bold text-white">Inference Harness</h1><p class="text-sm text-gray-400">Syslog Solution LLC · Real-time Monitoring</p></div>
<div class="flex items-center gap-4">
<span id="statusBadge" class="flex items-center gap-2"><span id="statusDot" class="w-3 h-3 rounded-full dot-green"></span><span id="statusText" class="text-emerald-400 text-lg font-semibold">healthy</span></span>
<span id="lastUpdate" class="text-sm text-gray-500"></span>
</div>
</div>
<div class="grid-5 mb-6">
<div class="glass-panel p-4 text-center"><p class="stat-value text-white" id="kpiGpus">-</p><p class="stat-label">GPUs Online</p></div>
<div class="glass-panel p-4 text-center"><p class="stat-value text-white" id="kpiTrips">-</p><p class="stat-label">Circuit Trips</p></div>
<div class="glass-panel p-4 text-center"><p class="stat-value text-white" id="kpiLatency">-</p><p class="stat-label">Avg Latency</p></div>
<div class="glass-panel p-4 text-center"><p class="stat-value text-white" id="kpiReqs">-</p><p class="stat-label">Requests/min</p></div>
<div class="glass-panel p-4 text-center"><p class="stat-value text-white" id="kpiActive">-</p><p class="stat-label">Active Requests</p></div>
</div>
<h2 class="text-xl font-semibold text-white mb-3">GPU Health Scoring <span class="text-sm text-gray-400 font-normal">Live: VRAM 40% · Temp 30% · Load 30%</span></h2>
<div id="gpuCards" class="grid-3 mb-6"></div>
<div class="glass-panel p-4 mb-6"><h3 class="text-sm font-semibold text-gray-400 mb-3">Health Score History (60s rolling)</h3><div id="healthChart" style="height:220px"></div></div>
<div class="text-center text-sm text-gray-500 pt-4 border-t"><p>Inference Harness Dashboard · Syslog Solution LLC · Auto-refresh 15s</p></div>
</div>
<script>
const COLORS={'qwen3.6-35B-A3B':'#10b981','qwen3.6-27B-code':'#8b5cf6','gemma-4-12b':'#3b82f6'};
const HISTORY=[]; // rolling 60 sample history for chart
function Q(id){return document.getElementById(id)}
function updateStatus(trips,degraded){
const d=Q('statusDot'),t=Q('statusText');
if(degraded){d.className='w-3 h-3 rounded-full dot-red';t.textContent='degraded';t.className='text-red-400 text-lg font-semibold'}
else if(trips>0){d.className='w-3 h-3 rounded-full dot-yellow';t.textContent='trips:'+trips;t.className='text-amber-400 text-lg font-semibold'}
else{d.className='w-3 h-3 rounded-full dot-green';t.textContent='healthy';t.className='text-emerald-400 text-lg font-semibold'}
}
function fetchAll(){
Promise.all([
fetch('/metrics/gpu-health').then(r=>r.json()),
fetch('/metrics/latency').then(r=>r.json())
]).then(function(_a){var health=_a[0],latency=_a[1];
Q('lastUpdate').textContent=new Date().toLocaleTimeString();
var gpus=health.gpus||[],kpi=health.kpi||{};
// KPIs
Q('kpiGpus').textContent=kpi.gpus_online+'/'+kpi.total_gpus;
Q('kpiTrips').textContent=kpi.total_trips;
Q('kpiLatency').textContent=(latency.avg_ms||0)+'ms';
Q('kpiReqs').textContent=latency.requests_per_min||0;
var active=0;gpus.forEach(function(g){active+=g.active_requests||0});
Q('kpiActive').textContent=active;
// Status
var degraded=gpus.some(function(g){return g.status==='down'||g.circuit_tripped});
updateStatus(kpi.total_trips||0,degraded);
// GPU cards
var html='';
gpus.forEach(function(g,i){
var s=g.health_score||0,clr=s>=70?'#10b981':(s>=40?'#f59e0b':'#ef4444');
var badge='';
if(g.circuit_tripped)badge='<span class="text-xs px-2 py-1 bg-red-900 rounded text-red-300">TRIPPED</span>';
else if(i===0)badge='<span class="text-xs px-2 py-1 bg-emerald-900 rounded text-emerald-300">Best</span>';
html+='<div class="glass-panel p-4"><div class="flex justify-between items-center mb-2"><p class="text-lg font-bold text-white">'+g.label+'</p>'+badge+'</div>'+
'<p class="text-4xl font-bold mb-2" style="color:'+clr+'">'+Math.round(s)+'</p>'+
'<div class="health-bar"><div class="health-fill" style="width:'+s+'%;background:'+clr+'"></div></div>'+
'<div class="flex justify-between mt-2 text-xs text-gray-400"><span>VRAM '+g.vram_pct+'%</span><span>'+g.temp_c+'°C</span><span>Active '+g.active_requests+'/'+g.max_concurrent+'</span><span>Trips '+g.circuit_trip_count+'</span></div></div>';
});
Q('gpuCards').innerHTML=html;
// Rolling history
var now=Date.now();HISTORY.push({ts:now,gpus:gpus.map(function(g){return{id:g.id,score:g.health_score}})});
if(HISTORY.length>60)HISTORY.shift();
renderChart();
}).catch(function(){});
}
function renderChart(){
var W=800,H=220,svg='<svg viewBox="0 0 '+W+' '+H+'" style="width:100%;height:220px" preserveAspectRatio="none">';
// Grid lines
for(var i=0;i<=4;i++){var y=(i/4)*H;svg+='<line x1="0" y1="'+y+'" x2="'+W+'" y2="'+y+'" stroke="#1e293b" stroke-width="1"/>';svg+='<text x="5" y="'+(y+10)+'" font-size="9" fill="#64748b">'+(100-i*25)+'</text>'}
// Time labels
for(var i=0;i<=4;i++){var lx=(i/4)*W,lt=HISTORY.length>0?new Date(HISTORY[Math.floor(i/4*(HISTORY.length-1))].ts).toLocaleTimeString():'';svg+='<text x="'+lx+'" y="'+(H-2)+'" font-size="8" fill="#475569" text-anchor="middle">'+lt+'</text>'}
// Plot lines per GPU
var ids=HISTORY.length>0?HISTORY[0].gpus.map(function(g){return g.id}):[];
ids.forEach(function(id){
var c=COLORS[id]||'#94a3b8',pts='',first=true;
HISTORY.forEach(function(h,i){
var gpu=h.gpus.find(function(g){return g.id===id});
if(!gpu)return;
var x=(i/(HISTORY.length-1||1))*W,y=H-(gpu.score/100*H);
pts+=(first?'M':'L')+x.toFixed(1)+','+y.toFixed(1);first=false;
});
if(pts)svg+='<path d="'+pts+'" fill="none" stroke="'+c+'" stroke-width="2" opacity="0.9"/>';
});
svg+='</svg>';Q('healthChart').innerHTML=svg;
}
fetchAll();setInterval(fetchAll,15000);
</script>
</body>
</html>
docker-compose.yml
+6 -5
View File
@@ -6,7 +6,7 @@ services:
container_name: harness-redis container_name: harness-redis
restart: unless-stopped restart: unless-stopped
ports: ports:
- "6379:6379" - "127.0.0.1:6379:6379"
volumes: volumes:
- redis-data:/data - redis-data:/data
command: redis-server --appendonly yes --maxmemory 256mb --maxmemory-policy allkeys-lru command: redis-server --appendonly yes --maxmemory 256mb --maxmemory-policy allkeys-lru
@@ -21,13 +21,13 @@ services:
container_name: harness-router container_name: harness-router
restart: unless-stopped restart: unless-stopped
ports: ports:
- "0.0.0.0:9000:9000" - "127.0.0.1:9000:9000"
environment: environment:
- REDIS_URL=redis://redis:6379 - REDIS_URL=redis://redis:6379
- GPU_MOE_URL=http://192.168.68.15:8080/v1 - GPU_MOE_URL=http://192.168.68.15:8080/v1
- GPU_DENSE_URL=http://192.168.68.8:8080/v1 - GPU_DENSE_URL=http://192.168.68.8:8080/v1
- GPU_LIGHT_URL=http://192.168.68.110:8080/v1 - GPU_LIGHT_URL=http://192.168.68.110:8080/v1
- API_KEYS={"sk-syslog-local-master-key":{"tier":"enterprise","agent":"admin"},"sk-syslog-abiba":{"tier":"enterprise","agent":"Abiba"},"sk-syslog-mumuni":{"tier":"enterprise","agent":"Mumuni"},"sk-syslog-tanko":{"tier":"enterprise","agent":"Tanko"},"sk-syslog-koby":{"tier":"enterprise","agent":"Koby"},"sk-syslog-kagenz0":{"tier":"enterprise","agent":"Kagenz0"},"sk-syslog-koonimo":{"tier":"enterprise","agent":"Koonimo"},"sk-starter-abc123":{"tier":"starter","agent":"test-starter"},"sk-professional-xyz789":{"tier":"professional","agent":"test-pro"}} - API_KEYS={"sk-syslog-local-master-key":{"tier":"enterprise","agent":"admin","deprecated":true},"sk-9e65b69a67-e54af421c1b09fb8bd4f75dacb38cb64":{"tier":"enterprise","agent":"admin"},"sk-syslog-abiba":{"tier":"enterprise","agent":"Abiba","deprecated":true},"sk-856ffb0bbb-e5aaf78b10054eca608f8fbcbd73a889":{"tier":"enterprise","agent":"Abiba"},"sk-syslog-mumuni":{"tier":"enterprise","agent":"Mumuni","deprecated":true},"sk-b57e6e042e-47573660114f3138c852c47f62da807e":{"tier":"enterprise","agent":"Mumuni"},"sk-syslog-tanko":{"tier":"enterprise","agent":"Tanko","deprecated":true},"sk-620a05e95a-e93d875476b650a4d1137249ead8eaa7":{"tier":"enterprise","agent":"Tanko"},"sk-syslog-koby":{"tier":"enterprise","agent":"Koby","deprecated":true},"sk-eb3e6fc1c0-de1bf2edf35a53cb3749a2400483fdee":{"tier":"enterprise","agent":"Koby"},"sk-syslog-kagenz0":{"tier":"enterprise","agent":"Kagenz0","deprecated":true},"sk-12b66b3392-b548aed9138aeb6f698e8e521650ed9b":{"tier":"enterprise","agent":"Kagenz0"},"sk-syslog-koonimo":{"tier":"enterprise","agent":"Koonimo","deprecated":true},"sk-680d06686c-00ee8bf9dc3c93b276af122d49a14dfe":{"tier":"enterprise","agent":"Koonimo"},"sk-starter-abc123":{"tier":"starter","agent":"test-starter","deprecated":true},"sk-55da55907a-1bd7ff344e26feda50e9ac2219697860":{"tier":"starter","agent":"test-starter"},"sk-professional-xyz789":{"tier":"professional","agent":"test-pro","deprecated":true},"sk-b5159863e6-8df3ae52fb958cfe76cc2888c8c8e676":{"tier":"professional","agent":"test-pro"}}
- ADMIN_KEY=sk-admin-ee09fffd04978b61a1569ac670c68814 - ADMIN_KEY=sk-admin-ee09fffd04978b61a1569ac670c68814
healthcheck: healthcheck:
test: ["CMD", "python3", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:9000/health')"] test: ["CMD", "python3", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:9000/health')"]
@@ -44,7 +44,7 @@ services:
container_name: harness-litellm container_name: harness-litellm
restart: unless-stopped restart: unless-stopped
ports: ports:
- "0.0.0.0:8081:4000" - "127.0.0.1:8081:4000"
volumes: volumes:
- ./litellm_config.yaml:/app/config.yaml - ./litellm_config.yaml:/app/config.yaml
environment: environment:
@@ -68,6 +68,7 @@ services:
- "80:80" - "80:80"
volumes: volumes:
- ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro - ./nginx/nginx.conf:/etc/nginx/nginx.conf:ro
- ./dashboard:/opt/inference-harness/dashboard:ro
healthcheck: healthcheck:
test: ["CMD", "curl", "-f", "http://127.0.0.1/health"] test: ["CMD", "curl", "-f", "http://127.0.0.1/health"]
interval: 30s interval: 30s
@@ -82,7 +83,7 @@ services:
container_name: harness-dashboard container_name: harness-dashboard
restart: unless-stopped restart: unless-stopped
ports: ports:
- "0.0.0.0:3002:3000" - "127.0.0.1:3000:3000"
environment: environment:
- REDIS_URL=redis://redis:6379 - REDIS_URL=redis://redis:6379
- GPU_SIDECARS=192.168.68.15:8090,192.168.68.8:8090,192.168.68.110:8090 - GPU_SIDECARS=192.168.68.15:8090,192.168.68.8:8090,192.168.68.110:8090
nginx/nginx.conf
+18 -3
View File
@@ -31,7 +31,7 @@ http {
# Disable buffering for SSE streams # Disable buffering for SSE streams
proxy_buffering off; proxy_buffering off;
# API — through router # API through router
location /v1/ { location /v1/ {
proxy_pass http://router_api; proxy_pass http://router_api;
proxy_http_version 1.1; proxy_http_version 1.1;
@@ -42,7 +42,14 @@ http {
proxy_read_timeout 600s; proxy_read_timeout 600s;
proxy_buffering off; proxy_buffering off;
} }
location /admin/ { proxy_pass http://router_api; proxy_http_version 1.1; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header Authorization $http_authorization; }
location /admin/ {
proxy_pass http://router_api;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Authorization $http_authorization;
}
# SSE streaming endpoint # SSE streaming endpoint
location /stream { location /stream {
@@ -71,7 +78,15 @@ location /admin/ { proxy_pass http://router_api; proxy_htt
proxy_set_header Authorization $http_authorization; proxy_set_header Authorization $http_authorization;
} }
# Dashboard # Professional Dashboard (Phase 1-3) - Static HTML served via Nginx
location /dashboard/ {
alias /opt/inference-harness/dashboard/;
index dashboard.html;
add_header Cache-Control "public, max-age=3600";
add_header X-Content-Type-Options nosniff;
}
# Legacy Dashboard (root) - Proxy to Flask app
location / { location / {
proxy_pass http://dashboard_ui; proxy_pass http://dashboard_ui;
proxy_http_version 1.1; proxy_http_version 1.1;
router/router.py
+760 -181
View File
File diff suppressed because it is too large Load Diff