SyslogSolution/prose-contracts
7
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: IP drift and container count from live testing
PR Pipeline — Authorize → Validate → Review → Merge / auth (pull_request) Failing after 13m1s
Details
PR Pipeline — Authorize → Validate → Review → Merge / validate (pull_request) Has been skipped
Details
PR Pipeline — Authorize → Validate → Review → Merge / lint (pull_request) Has been skipped
Details
PR Pipeline — Authorize → Validate → Review → Merge / ai-review (pull_request) Has been skipped
Details
PR Pipeline — Authorize → Validate → Review → Merge / gate (pull_request) Has been skipped
Details

Browse Source 
Testing the consolidated litellm-self-heal contract against live infra found:
- IP drift: pi host is .24, not .65 (confused with ra-h-os bridge)
  - Fixed gpu-monitor.prose.md: 192.168.68.65 → 192.168.68.24 (3 locations)
  - Fixed gpu-fleet.prose.md: .65 → .24 in config files + agent keys tables
- Container count: 10 running on CT 116, not 8 as contract stated
  - Added harness-docker-stats + harness-pve-exporter to litellm-self-heal
  - Updated container health check list

Verified: GPU monitor restarted on .24:9100, ornith model inference working,
all 10 containers healthy.
This commit is contained in:
root
2026-07-09 06:23:44 +00:00
parent 4164ee5ed0
commit d829f595b4
3 changed files with 15 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files
gpu-fleet.prose.md
+5 -5
View File
@@ -142,7 +142,7 @@ Show full fleet status: GPUs, models, VRAM, context windows, parallel slots, act
|-------|-----|-----|-----|--------|
| Tanko | 112 | .122 | `sk-CggiHWlamQyShxWC3Hx6uw` | SSH jerome |
| Mumuni | 114 | .123 | `sk-VrqCNlwUgzoNGOpikJ7nwQ` | SSH root |
| Abiba | 100 | .65 | `sk-Qvzi4uYQBhlSK_XstEhcyQ` | local (pi agent) |
| Abiba | 100 | .24 | `sk-Qvzi4uYQBhlSK_XstEhcyQ` | local (pi agent) |
| Tdunna | 111 | ? | `sk-6sbCNjz2T6lTVDBdlNHXsA` | Zulip DM |
| Baggy | 113 | ? | `sk-krnw_zGBwvvL5b7l2t-s-A` | no SSH |
| Kagenz0 | 105 | ? | `sk-Dh4CDkaHebMLEp8qqq20qA` | no SSH |
@@ -159,10 +159,10 @@ If no SSH access, send Zulip DM via abiba-bot.
| `/opt/inference-harness/router/router.py` | CT 116 | Router source (builds via compose) |
| `/etc/nginx/nginx.conf` | CT 116 (nginx container) | Routes /v1→LiteLLM, /dashboard/, /litellm/, /health |
| `/opt/monitoring/prometheus.yml` | CT 116 | Prometheus scrape config (5 targets) |
| `/root/scripts/gpu-monitor-server.py` | pi (.65) | GPU fleet monitor v2.1.0 (with benchmarks) |
| `/root/scripts/gpu_benchmark.py` | pi (.65) | GPU inference benchmark module (tok/s tracking) |
| `/root/scripts/gpu-saturation-watchdog.py` | pi (.65) | Auto-restart stuck llama-server |
| `/root/dashboard/gpu-fleet.html` | pi (.65) | Live HTML dashboard |
| `/root/scripts/gpu-monitor-server.py` | pi (.24) | GPU fleet monitor v2.1.0 (with benchmarks) |
| `/root/scripts/gpu_benchmark.py` | pi (.24) | GPU inference benchmark module (tok/s tracking) |
| `/root/scripts/gpu-saturation-watchdog.py` | pi (.24) | Auto-restart stuck llama-server |
| `/root/dashboard/gpu-fleet.html` | pi (.24) | Live HTML dashboard |
| `/etc/systemd/system/llama-server.service` | .8, .110 | llama-server daemons (Nvidia GPUs) |
| `/etc/systemd/system/ornith-server.service` | .15 (amdpve) | llama-server daemon (Vulkan, Strix Halo). Note: `llama-server.service` and `llama-server@.service` are **masked** on .15 to prevent port 8080 collisions. |