32 Commits

Include renames

Author	SHA1	Message	Date
root	8245716286	Remove all hardcoded credentials from repository ... Audit results (all patterns checked across .md, .prose.md, .sh, .py, .js, .ts, .json, .yaml, .yml, .env): - sk-or-v1 (OpenRouter): 0 occurrences - sk- prefix (20+ chars): 0 occurrences - sk_live: 0 occurrences - Bearer <key>: 0 occurrences - api_key: <value>: 0 occurrences - PASSWORD=: 0 occurrences - TOKEN=: 0 occurrences - SECRET=: 0 occurrences Files changed: - agent-zero-fix-summary.md (removed 2 OpenRouter keys) - agent-zero-openrouter-key.prose.md (removed 1 OpenRouter key) - hermes-key-enforcement.prose.md (removed 1 LiteLLM key, 1 external key) - litellm-api-keys.prose.md (removed 1 LiteLLM key) - litellm-self-heal.prose.md (removed 1 stale key reference) - scripts/agent-health-check.py (INFISICAL_TOKEN now required) - scripts/daily-infra-report.py (EMAIL_PASSWORD now required) - zulip-health.prose.md (TOKEN references annotated)	2026-09-17 06:11:42 +00:00
root	bb17c2120f	no-mistakes(document): Make litellm-health the live owner; dedupe probes	2026-09-12 16:00:56 +00:00
root	baaac9d7c6	no-mistakes(review): delete remaining duplicated timeout and frozen model-list state	2026-09-12 15:43:21 +00:00
root	2f65c38213	no-mistakes(review): delete duplicated config tables, point to CT 116 authority	2026-09-12 15:39:15 +00:00
root	d9eb18c024	no-mistakes(review): align sibling contracts on aliases, crew cap, and monitor key	2026-09-12 15:30:14 +00:00
root	3f07b9bbcc	no-mistakes(review): fix master-key inference and probe-surface drift in sibling contracts	2026-09-12 15:22:02 +00:00
agent-zero	7730cc7c03	feat(litellm): 1.99.1 update coverage, trove agent, nginx /ui /docs path fixes	2026-09-11 15:32:20 -04:00
agent-zero	832b184af6	fix(litellm): upgrade contract refs 1.90.0-rc.1 -> 1.99.1 and correct redis role ... - harness-litellm pinned tag updated to 1.99.1 in infrastructure-control, litellm-health, litellm-self-heal container tables - infrastructure-update MCP per-key limitation note now cites v1.99.1 - harness-redis role corrected: dead 'Router slots, circuit breakers' -> 'LiteLLM cache + rate-limit state' (router decommissioned 2026-09-11) - litellm-self-heal Rule 9 wording clarified for cache/rate-limit only Live verification 2026-09-11: harness-litellm healthy, /health 200/200, 7 models, 46 keys, prisma migrations 127 -> 157.	2026-09-11 15:05:00 -04:00
agent-zero	c26255f5ff	fix(router): decommission legacy GPU router across contracts and fleet scripts ... Router container, image, and config are purged on CT 116 (verified: no container, no image, inference-harness-router:latest removed, :9000 free, 11 containers healthy, 7 models, live syslog-auto completion OK). Updates: - gpu-fleet.prose.md: topology diagram rebuilt without the router tier - gpu-monitor / gpu-self-heal / litellm-health / litellm-self-heal: router steps dropped - infrastructure-control.prose.md: container inventory + litellm row corrected - scripts/daily-infra-report.py, scripts/prose-ai-review.sh: host-scoped checks Verified: 40/40 anchors matched uniquely, 0 U+FFFD across changed files, daily-infra-report.py compiles, no stray harness-router references remain.	2026-09-11 14:20:18 -04:00
abiba-bot	4f59b82404	no-mistakes(document): Refresh stale Mumuni roster docs and registry metadata	2026-09-10 10:26:00 +00:00
abiba-bot	88f27e75ed	no-mistakes(document): Refresh stale health-check version, provenance, and lint evidence	2026-09-10 02:01:20 +00:00
root	bbe9ee533f	no-mistakes(document): Docs updated for .8 unit repoint and abiba env.sh sourcing	2026-09-08 12:18:14 +00:00
root	2dfc3e1530	Merged PR #50: fix/gpu-dense-docs	2026-08-30 12:16:39 +00:00
root	42a1d0bd91	docs: add gitea-logger implementation example to litellm-self-heal	2026-07-28 21:23:04 +00:00
root	c9359e1808	fix: redirect health check logs from knowledge graph to Gitea (hard rule) ... Logs (LITELLM-HEALTH, GPU-SELF-HEAL, PM2) now pushed to SyslogSolution/health-logs instead of creating orphan nodes in the shared knowledge graph. - litellm-self-heal: Phase 4 now calls gitea-logger instead of kg-logger - gpu-self-heal: Reporting section updated to Gitea path - pm2-self-heal: Log step redirected to Gitea - 271 existing orphan nodes remain in graph (no delete tool available)	2026-07-28 21:20:56 +00:00
root	3b6cf44a30	fix: update all Mumuni IP references from .123 to .24 (inside Abiba CT100) ... Mumuni CT114 destroyed. Mumuni now runs inside Abiba CT100 at 192.168.68.24. Updated all contract files and agent-health-check.py.	2026-07-28 08:56:45 +00:00
root	de9adb13cf	fix: agent health check v2 — CT liveness, config validation, wrapper integrity, vault emptiness ... Gaps fixed: - Koby (.129) and Koonimo (.114) now have SSH hosts — no longer skipped - Agent key lookup uses correct {NAME}_LITELLM_API_KEY format - New CT liveness check via pct status on PVE nodes - New config YAML integrity check via yaml.safe_load() - New wrapper/CLI integrity check (hermes wrapper, infisical path, hermes-real) - New vault secret non-emptiness check - Ops escalation: failures produce ALERT lines for cron capture	2026-07-26 12:06:32 +00:00
root	1c44bf1259	contract updates: ornith decommissioned, 256K→128K context, Mumuni Discord disabled ... Change 1: Strix Halo — ornith decommissioned - gpu-fleet: Genesis Hermes V3 APEX → qwen3.6-35B-udq4 throughout - inference-optimization: ornith→strix-moe/qwen3.6-35B-udq4 - gpu-monitor: ornith status → Strix Halo status - infrastructure-control: Strix Halo — ornith → qwen3.6-35B-udq4 - infrastructure-update: ornith→strix-moe via router - proxmox-monitor: Strix Halo LLM (ornith) → (qwen3.6-35B-udq4, strix-moe) Change 2: GPU context 256K→128K fleet-wide - hermes-agent-baseline: frontmatter description updated - litellm-health: GPU Fleet Topology table 256K→128K - litellm-self-heal: GPU Fleet Topology, engine flags, VRAM alert - inference-optimization: compress threshold 256K→128K compact at 85K - gpu-fleet: instability note updated Change 3: Mumuni Discord platform disabled - gpu-fleet: Mumuni platforms: removed discord	2026-07-23 09:00:42 +00:00
Abiba	cd52deda92	contracts: stable alias chain + no-master-key-for-inference rule ... Agent configs aligned to stable aliases (gpu-light/gpu-dense/strix-moe/syslog-auto): - Tanko fixed: base_url :4000->/v1 (nginx), max_context_window 131072->262144, aux.compression gemma->strix-moe, aux gemma->gpu-light - Mumuni aligned: aux gemma->gpu-light, delegation/x_search qwen->gpu-dense - Koby/Koonimo already used gpu-light (validated) - Zero raw model names remain in any agent config No-master-key rule: litellm_proxy_master_key (sk-litellm-...) is admin-only (/key/list, /key/generate). All inference uses agent keys. Health-check model tests use dedicated monitor key (/etc/litellm-monitor.env on CT 116). Fixed: daily-infra-report.py, gpu-self-heal.py, litellm-health-check.sh. Fixed wrong key-scope note: qwen3.6-35B-udq4 IS in scope for baggy/koby/mumuni/ abiba-pi keys (not 'NOT in allowlist' as previously documented).	2026-07-16 17:39:05 +00:00
Abiba	97f1cd77e4	litellm-self-heal: document 2026-07-16 ops sync (script fixes, systemd gpu-monitor, live-key agent check) ... Health-check script: critical-only gpu-fleet alerting, strix-moe model test. gpu-monitor: systemd unit (was bare &), Strix counted in gpu_count, VRAM thresholds raised (93/97 — 256K ctx steady-state ~96% on RTX 3090). agent-health-check.py: reads live LITELLM_API_KEY from gateway env (no hardcoded keys). daily-infra-report.py: stale SYNTHETIC key → env-based master key.	2026-07-16 17:23:37 +00:00
Abiba	dc572889f8	contracts: sync to ground truth — ornith-1.0-35b→strix-moe, 256K all GPUs, real LiteLLM timeouts ... Verified on ground 2026-07-16 against CT 116 litellm_config.yaml + GPU hosts: - AMD host serves qwen3.6-35B-udq4 (LiteLLM alias strix-moe); ornith-1.0-35b does NOT exist - All 3 GPUs at 256K ctx, parallel 2 (RTX 3090 was listed 128K/parallel 1) - LiteLLM timeouts: qwen 300s, gemma 120s, strix 300s (were stale 90s/120s) - Added LiteLLM model surface + key scoping to litellm-self-heal - Patched health-check script path ref Files: litellm-self-heal, litellm-health, gpu-fleet, gpu-self-heal, zulip-adapter-lessons, abiba-zulip-restore, hermes-agent-baseline, delegation-prose-contract, mumuni-delegation-prose-contract	2026-07-16 17:02:44 +00:00
root	af41f8f57a	fix: remaining template YAML and topology table corrections ... - hermes-config-template: Template YAML compression model gemma→ornith. Context guidance updated (qwen=256K, only gemma limited to 131K). - litellm-self-heal: GPU topology table fixed (RTX 3090: 128K→256K, parallel 2→1)	2026-07-12 22:55:24 +00:00
root	be02b0e843	fix: stale dates, context values, and architecture references ... - gpu-fleet: Updated all context references (128K→256K for RTX 3090). Fixed parallel count (2→1 for RTX 3090). Updated architecture diagram (router labeled deprecated/not in path). Corrected VRAM values. Updated dates: June→July 2026. - hermes-config-template: Fixed description (removed incorrect '128K on NVIDIA' claim). Updated to reflect verified 256K on RTX 3090. - litellm-self-heal: Updated last-verified date. - litellm-api-keys: Updated last-verified date.	2026-07-12 22:54:30 +00:00
root	79d4a73895	docs: lessons learned from 2026-07-12 session ... - gpu-fleet: Corrected architecture (direct GPU, no router in path). Updated context values (RTX 3090=256K, not 128K). Added api-key standardization requirement. - gpu-self-heal: Added Lessons Learned section with 5 critical findings: L1: API key standardization (RTX 5070 sk-loc...5678 vs not-needed) L2: Fallback chain cascading failure loop detection L3: Verify running state, not documentation L4: Infisical fallback requirement (.env must have uncommented key) L5: Zulip event queue can silently die after ~40 reconnects - litellm-self-heal: Updated status manual-only→deployed, cron schedule - litellm-api-keys: Added Infisical token expiry warning + .env fallback - hermes-config-template: Rule 3 updated with .env fallback requirement	2026-07-12 22:49:39 +00:00
root	d829f595b4	fix: IP drift and container count from live testing ... Testing the consolidated litellm-self-heal contract against live infra found: - IP drift: pi host is .24, not .65 (confused with ra-h-os bridge) - Fixed gpu-monitor.prose.md: 192.168.68.65 → 192.168.68.24 (3 locations) - Fixed gpu-fleet.prose.md: .65 → .24 in config files + agent keys tables - Container count: 10 running on CT 116, not 8 as contract stated - Added harness-docker-stats + harness-pve-exporter to litellm-self-heal - Updated container health check list Verified: GPU monitor restarted on .24:9100, ornith model inference working, all 10 containers healthy.	2026-07-09 06:23:44 +00:00
root	053d30ce7c	feat: consolidate litellm-health into litellm-self-heal ... - litellm-self-heal: merged health check as §Health Check section, includes full architecture diagram (v4.0.0), GPU topology, timeout tables, container list, and execution flow. Eliminates duplication between the two contracts. - litellm-health: marked DEPRECATED, retained for reference - README.md: updated with consolidated contract, deprecation notice, disk-gc-threat-response, and infrastructure-monitoring status	2026-07-09 06:01:48 +00:00
root	a2c5a14c74	fix: LiteLLM contracts — align with 2026-07-08 architecture changes ... - litellm-health: v4.0.0 — Router removed from path (nginx→LiteLLM→GPU directly), GPU engines corrected (Docker→systemd), timeouts updated (gemma 120s, qwen 90s), Strix Halo: CPU→Vulkan, context sizes + parallel slots added - litellm-self-heal: ornith context 262K→256K, Rule 7 marked DEPRECATED (router not in path), GPU topology synced with gpu-fleet - litellm-api-keys: reference gpu-fleet as source of truth for keys, removed qwen3.6-35B-A3B from model list (never deployed)	2026-07-09 05:37:16 +00:00
root	253e19f680	prose: review, fix, and consolidate infrastructure contracts ... - DELETE infrastructure-monitoring.prose.md (redundant — proxmox-monitor already deployed this) - FIX infrastructure-control.prose.md: remove /grafana/ nginx route (reverted Jul 2, broke gpu-fleet) - FIX infrastructure-update.prose.md: wrong CT IDs (122→112 Tanko, 123→114 Mumuni, .19→storepve Zulip) - FIX infrastructure-update.prose.md: Strix Halo :8080→router health check (firewalled to .116 only) - FIX proxmox-monitor.prose.md: stale /grafana/ URLs→:3001 (post-revert cleanup) - ADD disk-gc-threat-response.prose.md: verified run (2026-07-04, reclaimed 35.67GB from kagentz) - ADD infrastructure-update.prose.md, pi-approval-architecture.prose.md, zulip-approval-fix.prose.md, zulip-self-heal.prose.md - UPDATE hermes-config, litellm-health/self-heal, pm2-self-heal, zulip-* contracts - ADD runs/20260704-005804-51f9a9 (disk-gc-threat-response run artifacts) All contracts verified against live PVE API, nginx config, and firewall state.	2026-07-04 21:47:33 +00:00
Abiba	c1d993fe01	fix: GPU table correction + OpenProse v0.15 alignment + self-heal rules 7-9 ... - hello-world: Ensures -> Maintains (v0.15.0 rename). - zulip-mention-reliability: fold Success Criteria into Maintains postconditions. - litellm-self-heal: swap gemma/qwen GPU hosts to match actual deployment; add Strix Halo 64GB UMA detail; add Rules 7 (roster stuck), 8 (agent keys invalid/401), 9 (stale Redis active counter).	2026-07-02 20:39:17 +00:00
root	aa55634571	feat: tighten contracts with GPU fleet topology, model routing, Strix Halo details ... litellm-health: added GPU fleet topology table, model routing map, 3-tier GPU checks (reachability, VRAM, temp, test inference) litellm-self-heal: added GPU fleet reference, rules for GPU unreachable and model not responding Infrastructure topology now fully documented: - amdpve: Strix Halo CPU (35B, 16 threads, 262K ctx, llama-server) - llm-gpu: RTX 3090 24GB (Dense tier) - ocu-llm: RTX 5070 12GB (MoE + Light tiers)	2026-06-29 04:07:05 +00:00
root	41bd3c5748	feat: add daily/weekly digest reporting to self-heal contract ... - Hourly cycle reports: fix alerts, escalation alerts, 10-cycle all-clear - Daily digest: 24h summary with counts, top actions, uptime - Weekly digest: 7-day summary + KG node for trending - All delivered via Zulip DM to owner	2026-06-26 15:23:21 +00:00
root	fb27c99c3d	feat: litellm-self-heal — proactive remediation responsibility	2026-06-26 14:51:04 +00:00