abiba-bot/prose-contracts
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
134 Commits 2 Branches 0 Tags
cd52deda9262d417444dc4c2f793bec50f028eca
Commit Graph
19 Commits
Author SHA1 Message Date
Abiba dc572889f8 contracts: sync to ground truth — ornith-1.0-35b→strix-moe, 256K all GPUs, real LiteLLM timeouts 
Verified on ground 2026-07-16 against CT 116 litellm_config.yaml + GPU hosts:
- AMD host serves qwen3.6-35B-udq4 (LiteLLM alias strix-moe); ornith-1.0-35b does NOT exist
- All 3 GPUs at 256K ctx, parallel 2 (RTX 3090 was listed 128K/parallel 1)
- LiteLLM timeouts: qwen 300s, gemma 120s, strix 300s (were stale 90s/120s)
- Added LiteLLM model surface + key scoping to litellm-self-heal
- Patched health-check script path ref

Files: litellm-self-heal, litellm-health, gpu-fleet, gpu-self-heal,
zulip-adapter-lessons, abiba-zulip-restore, hermes-agent-baseline,
delegation-prose-contract, mumuni-delegation-prose-contract
 2026-07-16 17:02:44 +00:00
Abiba 39e2fa0cfa Contracts: Mumuni context-fix (WAL #1300) — strix-moe, 256K all GPUs, Rule 12/13, machine-identity vault procedure 2026-07-16 13:57:31 +00:00
root 622cf7b176 UPDATE 2026-07-15: Full fleet rebuild 
- Stable role-based aliases: strix-moe, gpu-dense, gpu-light
- Strix Halo: ornith-1.0-35b -> unsloth/Qwen3.6-35B-A3B-MTP (UD-Q4_K_M, 256K)
- RTX 5070: Q4_K_M -> IQ4_NL+MTP (256K, 125-213 tok/s, 2x faster)
- RTX 3090: bumped to 256K context
- Provider rename: harness -> litellm (all agents)
- syslog-auto: weighted pool restored with direct GPU routing
- Mumuni agent profile documented
- Self-heal thresholds updated for 256K context
- All model contexts corrected (RTX 3090 was 131K, not 256K)
 2026-07-15 16:04:21 +00:00
root 0c298eb9d9 feat: add routing configuration with RPM caps (ornith syslog-auto: 40→60) 
- Strix Halo ornith RPM increased from 40 to 60 inside syslog-auto pool
- Prevented unbounded ornith traffic when multiple agents use syslog-auto
- Direct ornith remains at RPM=40
- All models verified healthy (ornith 80ms, gemma 110ms, qwen 840ms)
- Added full routing table to gpu-fleet contract
 2026-07-12 23:38:21 +00:00
root be02b0e843 fix: stale dates, context values, and architecture references 
- gpu-fleet: Updated all context references (128K→256K for RTX 3090).
  Fixed parallel count (2→1 for RTX 3090). Updated architecture diagram
  (router labeled deprecated/not in path). Corrected VRAM values.
  Updated dates: June→July 2026.
- hermes-config-template: Fixed description (removed incorrect '128K on
  NVIDIA' claim). Updated to reflect verified 256K on RTX 3090.
- litellm-self-heal: Updated last-verified date.
- litellm-api-keys: Updated last-verified date.
 2026-07-12 22:54:30 +00:00
root 79d4a73895 docs: lessons learned from 2026-07-12 session 
- gpu-fleet: Corrected architecture (direct GPU, no router in path).
  Updated context values (RTX 3090=256K, not 128K). Added api-key
  standardization requirement.
- gpu-self-heal: Added Lessons Learned section with 5 critical findings:
  L1: API key standardization (RTX 5070 sk-loc...5678 vs not-needed)
  L2: Fallback chain cascading failure loop detection
  L3: Verify running state, not documentation
  L4: Infisical fallback requirement (.env must have uncommented key)
  L5: Zulip event queue can silently die after ~40 reconnects
- litellm-self-heal: Updated status manual-only→deployed, cron schedule
- litellm-api-keys: Added Infisical token expiry warning + .env fallback
- hermes-config-template: Rule 3 updated with .env fallback requirement
 2026-07-12 22:49:39 +00:00
root d829f595b4 fix: IP drift and container count from live testing 
Testing the consolidated litellm-self-heal contract against live infra found:
- IP drift: pi host is .24, not .65 (confused with ra-h-os bridge)
  - Fixed gpu-monitor.prose.md: 192.168.68.65 → 192.168.68.24 (3 locations)
  - Fixed gpu-fleet.prose.md: .65 → .24 in config files + agent keys tables
- Container count: 10 running on CT 116, not 8 as contract stated
  - Added harness-docker-stats + harness-pve-exporter to litellm-self-heal
  - Updated container health check list

Verified: GPU monitor restarted on .24:9100, ornith model inference working,
all 10 containers healthy.
 2026-07-09 06:23:44 +00:00
root 776df418ca gpu-fleet: July 8 optimization sweep — parallel 2 fleet-wide, 128K ctx on NVIDIA, LiteLLM timeout fixes 
- All GPUs: --parallel 1 → 2 (6 concurrent slots, was 3)
- .8 RTX 3090: ctx 256K→128K, VRAM 96%→83%, turbo4 KV cache
- .110 RTX 5070: ctx 256K→128K, ubatch 4096→512 (was inverted), VRAM 90%→77%, q4_0 KV
- .15 Strix Halo: parallel 1→2, 256K ctx (41GB free), q8_0 KV, AMD metrics via /sys/class/drm
- LiteLLM: gemma timeout 25→120s, qwen timeout 40→90s, syslog-auto (qwen) 40→90s
- Agent configs: context_length 262144 for syslog-auto, 131072 for direct qwen/gemma
- Updated health-check operation, agent config implications, benchmark table (fixed model↔GPU mapping)
 2026-07-08 17:25:42 +00:00
root 5aa93117de docs: capture all session changes — port conflict, health check, streaming 
Updated contracts to reflect all infrastructure changes made 2026-07-05/06:

gpu-fleet.prose.md:
  - Port conflict detection on all 3 GPU wrappers
  - Ghost process root cause documented (.8 stale pid 25836)

infrastructure-control.prose.md:
  - Section 7: Agent Health Check (consolidated, non-disruptive)
  - Disabled scripts documented (zulip-watchdog, zulip-monitor)

hermes-agent-baseline.prose.md:
  - Health verification section + GPU port conflict detection
  - Change log updated

zulip-health.prose.md:
  - Streaming support section (edit_message + PATCH API)
 2026-07-06 02:46:41 +00:00
root 35e864bdb7 audit: all 4 Hermes agents verified — no master key, api_key workaround applied 
Agent audit (2026-07-05):
  Tanko    CT 112: sk-CggiHWlamQyShxWC3Hx6uw  — vision+compression patched
  Mumuni   CT 114: sk-VrqCNlwUgzoNGOpikJ7nwQ  — vision patched
  Tdunna   CT 111: sk-6sbCNjz2T6lTVDBdlNHXsA  — vision+compression patched
  Baggy    CT 113: sk-krnw_zGBwvvL5b7l2t-s-A  — vision+compression patched

All 4 agents: zero master key references, dedicated LiteLLM keys,
api_key workaround applied to bypass auxiliary_client bug.
 2026-07-05 23:36:00 +00:00
root c8dcd8b63d fix: Tanko key regenerated, Mumuni key verified from /etc/environment 
Tanko: old key sk-620a05e95a... was from docker-compose router config,
not LiteLLM key DB. Deleted + regenerated via LiteLLM API:
  sk-CggiHWlamQyShxWC3Hx6uw  (alias: tanko, budget: $100)

Mumuni: verified via pct-run 114 — key is sk-VrqCNlwUgzoNGOpikJ7nwQ
(was previously listed as sk-XY2aUfvy2... in contracts)
 2026-07-05 20:18:05 +00:00
root 5c90eb8f6c rename: Koonimo→Baggy, Koby→Tdunna — match hostnames 
Standardize agent names across all contracts and pct-run.sh.
No functional changes — same CT IDs, same keys, same nodes.
 2026-07-05 20:04:22 +00:00
root 0db95a4d1c fix: correct IPs, CT IDs, and key status across all contracts 
hermes-key-enforcement.prose.md:
  - Add IP column to verified agents table
  - Fix Tanko status: was using master key via systemd drop-in (not compliant)
  - Add detection query step 2: check systemd drop-ins for master key leaks
  - Add step 3: verify running process env against dedicated key
  - Mark Mumuni, Koby, Koonimo as Unverified (last check was surface-level only)

gpu-fleet.prose.md:
  - Add CT ID column to agent table for cross-reference
  - Fix Abiba IP .24 → .65 (pi agent runs on RA-H OS host)
  - Update Abiba LiteLLM key to match actual models.json key
  - Set Koonimo IP to unknown (was .114 — incorrect per user)
  - Fix pi-specific paths .24 → .65

hermes-config-template.prose.md:
  - Set Koonimo IP to unknown

infrastructure-control.prose.md:
  - Set Koonimo IP to unknown
  - Fix Koonimo CT reference
 2026-07-05 19:58:14 +00:00
root 263c4080f3 ci: add automated PR validation pipeline + fix regressions 
NEW: Gitea Actions CI pipeline (.gitea/workflows/pr-pipeline.yaml)
- Stage 1 (validate): YAML frontmatter validation for all .prose.md files
- Stage 2 (lint): Structural checks + regression detection
  * /grafana/ nginx route (reverted 2026-07-02)
  * Stale CT IDs (122→112, no 123)
  * Verified: .19/.122/.123 = correct bridge IPs
- Stage 3 (ai-review): LiteLLM-powered diff review against ground truth
- Stage 4 (auto-merge): Gate — safe to merge when all green

NEW: scripts/prose-lint.sh — contract structure + regression checker
NEW: scripts/prose-ai-review.sh — AI review via syslog-auto model

FIXES FOUND BY LINT:
- gpu-fleet.prose.md: removed /grafana/ from nginx routing diagram
- gpu-fleet.prose.md: removed /grafana/ from config file description
- infrastructure-control.prose.md: CT 122→112 in topology diagram
- scripts/prose-lint.sh: updated to not flag .19 (verified correct IP)
 2026-07-04 22:13:51 +00:00
root b15771bfd9 prose: add Strix Halo thermal safeguard (-n 8192 cap) to gpu-fleet 
Runaway request from .123 (Mumuni) decoded 39,868 tokens over 24 min,
pushing Tctl to 98°C (9°C past critical) and throttling 70->29 t/s.
Server-side -n 8192 hard cap now bounds generation. Documented connection
source (all from .116/LiteLLM) and the thermal constraint (fanless APU).
 2026-07-02 03:51:36 +00:00
root a07ff2d96a prose: align all contracts with Vulkan rebuild + Koby key regen 
- gpu-fleet: Koby key updated (sk-lDTsWy7H3T...), config table (ornith service
  + masked services), benchmarks (prompt 209→532 tok/s), ROCm→Vulkan status
- gpu-monitor: topology (sidecars :8090 removed, .15 via router), subsystem
  polling (router /health/unified as source of truth), execution steps,
  alert threshold (sidecar-unreachable downgraded from critical)
 2026-07-02 00:29:10 +00:00
root 49e8d968f5 gpu-fleet: Strix Halo Vulkan rebuild + LiteLLM alignment (2026-07-01) 
- ornith now on Vulkan build 4fc4ec5, 256K ctx, ~70 tok/s
- Fixed GPU_MOE_URL bug in docker-compose (.110→.15)
- Added router sidecar fallback for health-checks
- Updated roster context 4096→262144 + correct model_path
- Documented port 8080 firewall (restricted to .116)
 2026-07-02 00:01:56 +00:00
root 38d2f7d956 feat: update contracts with GPU optimizations, Strix Halo/ROCm status, alert migration 
- gpu-fleet: documented parallel=2/batch=4096 optimizations for RTX 3090/5070
- gpu-fleet: added Strix Halo ROCm status (installed, blocked by HSA runtime on Debian 13)
- gpu-fleet: documented alert migration from DM → #agent-hub stream topics
- gpu-monitor: added Alert Delivery section with stream topic conventions
- All GPU alerts now go to #agent-hub for cross-agent visibility
 2026-07-01 22:25:26 +00:00
root 76fc2595ea feat(gpu-fleet): GPU fleet management prose contract 
Declarative GPU roster replaces hardcoded router.py model configs.
Operations: add-model, remove-model, heal, sync-keys, list.
Documents agent key architecture with LITELLM_API_KEY env var.
Covers all 3 GPU hosts (RTX 5070, RTX 3090, Strix Halo).
 2026-06-28 16:24:40 +00:00