25 Commits

Include renames

Author	SHA1	Message	Date
root	fb52aff6a0	fix: update Mumuni CT reference to 100 (Hermes gateway on CT 100)	2026-07-28 15:49:51 +00:00
root	3b6cf44a30	fix: update all Mumuni IP references from .123 to .24 (inside Abiba CT100) ... Mumuni CT114 destroyed. Mumuni now runs inside Abiba CT100 at 192.168.68.24. Updated all contract files and agent-health-check.py.	2026-07-28 08:56:45 +00:00
root	179529de71	fix: swap gpu-dense to SmartCode-Fable-5 (ThinkingCap + Fable 5 CoT) ... gpu-dense on RTX 3090 swapped from qwen3.6-27B-code to SmartCode-Fable-5-CoT-Reasoning-QKVO-Qwen-3.6-27B-Distilled (UD-Q4_K_XL, ~17.9GB).\n\nImprovements:\n- ~50% fewer thinking tokens via ThinkingCap finetune\n- Fable 5 CoT distillation for improved coding reasoning\n- Same 27B base, fits RTX 3090 at ~73% VRAM\n- Recommended samplers: temp 0.9, top-p 0.95, top-k 60\n- Context: 128K (fleet standard)	2026-07-27 16:43:29 +00:00
root	0fb54926f6	fix: remove CT 114 (mumuni) — destroyed, moved inside CT 100	2026-07-26 12:36:02 +00:00
root	47aa92bee3	fix: verification protocol findings — CT 118 static IP, .110 llama-server restored ... Verification protocol run 2026-07-26: - CT 118 (jdownloader): DHCP had moved it to .131 post-reboot. Now set to static .20. - RTX 5070 (.110) llama-server: was stopped (disabled unit). Started and verified — gemma-4-12b responding through LiteLLM. - All 6 PVE nodes confirmed at correct IPs. - All 18 CTs confirmed at correct IPs on correct nodes. - All public endpoints responding (meet, chat, git, litellm, vault, auth). - Container counts verified: docker-vm 14 ctrs, CT 116 10 ctrs, VPS 5 ctrs.	2026-07-26 12:20:58 +00:00
root	de9adb13cf	fix: agent health check v2 — CT liveness, config validation, wrapper integrity, vault emptiness ... Gaps fixed: - Koby (.129) and Koonimo (.114) now have SSH hosts — no longer skipped - Agent key lookup uses correct {NAME}_LITELLM_API_KEY format - New CT liveness check via pct status on PVE nodes - New config YAML integrity check via yaml.safe_load() - New wrapper/CLI integrity check (hermes wrapper, infisical path, hermes-real) - New vault secret non-emptiness check - Ops escalation: failures produce ALERT lines for cron capture	2026-07-26 12:06:32 +00:00
root	c380196fab	fix: address review findings ... - Remove kagentz from amdpve node table (belongs on hwepve) - Fix kagentz pct-run table: hwepve (was amdpve) - Fix mumuni pct-run table: hwepve (was minipve) - Fix Zulip recovery command: wrap in single SSH call	2026-07-24 20:11:32 +00:00
root	b17c60f997	fix: contract accuracy updates post fleet-wide reboot ... infrastructure-control.prose.md: - Add hwepve as 6th Proxmox node - Fix Gitea IP: .17 (was .110) - Fix AdGuard IP: .10 on minipve (was .102 on acerpve) - Fix Abiba placement: hwepve (was amdpve) - Fix Mumuni placement: hwepve (was minipve) - Fix Authentik port: add :9000 - Add CT 118 (jdownloader), CT 119 (infisical-vault) - Add last-verified date (2026-07-24) infrastructure-update.prose.md: - Add post-reboot CT sweep procedure - Add Zulip Docker network recovery steps - Add WireGuard tunnel verification scripts/netbird-add-domain.sh: - New script to register domains in Netbird proxy store.db	2026-07-24 20:01:43 +00:00
root	1c44bf1259	contract updates: ornith decommissioned, 256K→128K context, Mumuni Discord disabled ... Change 1: Strix Halo — ornith decommissioned - gpu-fleet: Genesis Hermes V3 APEX → qwen3.6-35B-udq4 throughout - inference-optimization: ornith→strix-moe/qwen3.6-35B-udq4 - gpu-monitor: ornith status → Strix Halo status - infrastructure-control: Strix Halo — ornith → qwen3.6-35B-udq4 - infrastructure-update: ornith→strix-moe via router - proxmox-monitor: Strix Halo LLM (ornith) → (qwen3.6-35B-udq4, strix-moe) Change 2: GPU context 256K→128K fleet-wide - hermes-agent-baseline: frontmatter description updated - litellm-health: GPU Fleet Topology table 256K→128K - litellm-self-heal: GPU Fleet Topology, engine flags, VRAM alert - inference-optimization: compress threshold 256K→128K compact at 85K - gpu-fleet: instability note updated Change 3: Mumuni Discord platform disabled - gpu-fleet: Mumuni platforms: removed discord	2026-07-23 09:00:42 +00:00
root	79d4a73895	docs: lessons learned from 2026-07-12 session ... - gpu-fleet: Corrected architecture (direct GPU, no router in path). Updated context values (RTX 3090=256K, not 128K). Added api-key standardization requirement. - gpu-self-heal: Added Lessons Learned section with 5 critical findings: L1: API key standardization (RTX 5070 sk-loc...5678 vs not-needed) L2: Fallback chain cascading failure loop detection L3: Verify running state, not documentation L4: Infisical fallback requirement (.env must have uncommented key) L5: Zulip event queue can silently die after ~40 reconnects - litellm-self-heal: Updated status manual-only→deployed, cron schedule - litellm-api-keys: Added Infisical token expiry warning + .env fallback - hermes-config-template: Rule 3 updated with .env fallback requirement	2026-07-12 22:49:39 +00:00
root	97977f320c	fix: contract improvements from 2026-07-09 run log review ... - pct-run.sh: add CT 109 note (KVM VM), document decommissioned/migrated CTs - disk-gc-threat-response: add docker-vm (SSH .7), add amdpve Docker scope, remove jitsi (decommissioned), update access matrix with KVM VM section - infrastructure-control: docker-vm container count 11→16, add Trove agents stack - infrastructure-monitoring: deployment status updated (core live, GPU exporters not deployed)	2026-07-09 05:32:00 +00:00
root	5aa93117de	docs: capture all session changes — port conflict, health check, streaming ... Updated contracts to reflect all infrastructure changes made 2026-07-05/06: gpu-fleet.prose.md: - Port conflict detection on all 3 GPU wrappers - Ghost process root cause documented (.8 stale pid 25836) infrastructure-control.prose.md: - Section 7: Agent Health Check (consolidated, non-disruptive) - Disabled scripts documented (zulip-watchdog, zulip-monitor) hermes-agent-baseline.prose.md: - Health verification section + GPU port conflict detection - Change log updated zulip-health.prose.md: - Streaming support section (edit_message + PATCH API)	2026-07-06 02:46:41 +00:00
root	5c90eb8f6c	rename: Koonimo→Baggy, Koby→Tdunna — match hostnames ... Standardize agent names across all contracts and pct-run.sh. No functional changes — same CT IDs, same keys, same nodes.	2026-07-05 20:04:22 +00:00
root	b65116b847	feat: pct-run.sh — run commands in CTs by ID, no IPs needed ... Adds /scripts/pct-run.sh that maps CT IDs to PVE nodes and runs pct exec directly. No hardcoded IPs — Proxmox is the source of truth. Updates infrastructure-control with CT access table using pct-run. Usage examples: pct-run 112 cat /etc/hostname # Tanko pct-run 114 grep LITELLM /etc/environment # Mumuni key check pct-run 116 docker ps # LiteLLM containers	2026-07-05 20:02:20 +00:00
root	0db95a4d1c	fix: correct IPs, CT IDs, and key status across all contracts ... hermes-key-enforcement.prose.md: - Add IP column to verified agents table - Fix Tanko status: was using master key via systemd drop-in (not compliant) - Add detection query step 2: check systemd drop-ins for master key leaks - Add step 3: verify running process env against dedicated key - Mark Mumuni, Koby, Koonimo as Unverified (last check was surface-level only) gpu-fleet.prose.md: - Add CT ID column to agent table for cross-reference - Fix Abiba IP .24 → .65 (pi agent runs on RA-H OS host) - Update Abiba LiteLLM key to match actual models.json key - Set Koonimo IP to unknown (was .114 — incorrect per user) - Fix pi-specific paths .24 → .65 hermes-config-template.prose.md: - Set Koonimo IP to unknown infrastructure-control.prose.md: - Set Koonimo IP to unknown - Fix Koonimo CT reference	2026-07-05 19:58:14 +00:00
root	263c4080f3	ci: add automated PR validation pipeline + fix regressions ... NEW: Gitea Actions CI pipeline (.gitea/workflows/pr-pipeline.yaml) - Stage 1 (validate): YAML frontmatter validation for all .prose.md files - Stage 2 (lint): Structural checks + regression detection * /grafana/ nginx route (reverted 2026-07-02) * Stale CT IDs (122→112, no 123) * Verified: .19/.122/.123 = correct bridge IPs - Stage 3 (ai-review): LiteLLM-powered diff review against ground truth - Stage 4 (auto-merge): Gate — safe to merge when all green NEW: scripts/prose-lint.sh — contract structure + regression checker NEW: scripts/prose-ai-review.sh — AI review via syslog-auto model FIXES FOUND BY LINT: - gpu-fleet.prose.md: removed /grafana/ from nginx routing diagram - gpu-fleet.prose.md: removed /grafana/ from config file description - infrastructure-control.prose.md: CT 122→112 in topology diagram - scripts/prose-lint.sh: updated to not flag .19 (verified correct IP)	2026-07-04 22:13:51 +00:00
root	253e19f680	prose: review, fix, and consolidate infrastructure contracts ... - DELETE infrastructure-monitoring.prose.md (redundant — proxmox-monitor already deployed this) - FIX infrastructure-control.prose.md: remove /grafana/ nginx route (reverted Jul 2, broke gpu-fleet) - FIX infrastructure-update.prose.md: wrong CT IDs (122→112 Tanko, 123→114 Mumuni, .19→storepve Zulip) - FIX infrastructure-update.prose.md: Strix Halo :8080→router health check (firewalled to .116 only) - FIX proxmox-monitor.prose.md: stale /grafana/ URLs→:3001 (post-revert cleanup) - ADD disk-gc-threat-response.prose.md: verified run (2026-07-04, reclaimed 35.67GB from kagentz) - ADD infrastructure-update.prose.md, pi-approval-architecture.prose.md, zulip-approval-fix.prose.md, zulip-self-heal.prose.md - UPDATE hermes-config, litellm-health/self-heal, pm2-self-heal, zulip-* contracts - ADD runs/20260704-005804-51f9a9 (disk-gc-threat-response run artifacts) All contracts verified against live PVE API, nginx config, and firewall state.	2026-07-04 21:47:33 +00:00
Abiba	d31ae0c25a	Verify-before-mutate doctrine: annotate live-state fields, fix stale Zulip IP (.117→.19) ... - Added FIELD TRUST convention to frontmatter (live-state vs policy fields) - Marked credentials/IPs with VERIFY-BEFORE-USE labels - Fixed 3 stale Zulip IP references (.117 → .19) that caused the 2026-07-03 outage - Added verify-before-mutate skill reference - See knowledge graph node #604 for post-mortem	2026-07-03 20:10:45 +00:00
Abiba	ae143d7a88	Add Stirling-PDF agent API access contract + skill. OAuth2 configured but disabled (needs license).	2026-07-03 19:10:22 +00:00
Abiba	8dfbad5415	Remove stale Bento-PDF reference from migration note	2026-07-03 18:20:08 +00:00
Abiba	c1bdbbc491	2026-07-03: Hermes key enforcement contract + Stirling-PDF docs + config standardization ... - hermes-key-enforcement.prose.md: NEW contract enforcing api_key_env for all harness/LiteLLM providers, exempting external providers - hermes-config-template.prose.md: Updated with full fix log (15 total fixes across Koby/Koonimo/Mumuni/Tanko), model:auto detection rule, key-to-agent mapping - infrastructure-control.prose.md: Added Stirling-PDF service details (port 8989, credentials, API key, Swagger URL) replacing Bento-PDF	2026-07-03 18:19:41 +00:00
Abiba	b0c0331727	feat: infrastructure-control network-verification + IP-first doctrine, plus 3 supporting contracts ... - infrastructure-control: add Section 5.3 (Network Verification & Routing) with dual-path checks, NetBird ingress health, DNS drift detection, and routing regression alerts. Add Section 7 (Configuration Doctrine — IP-First) mandating LAN IPs for all configs/agents, URLs reserved for human browser access only. Also enrich access matrix (Mumuni, Koonimo, LiteLLM Admin, Grafana entries), reachability matrix, and CT 116 nginx routing + Prometheus targets. - build-zulip-plugin: fold Success Criteria into Maintains postconditions. - pi-approval-architecture: new reference doc — pi approval model vs Hermes, architectural limits, /approve and /deny stub commands. - zulip-approval-fix: new responsibility — documents the Zulip HTML/prefix bug that broke /approve and /deny, and the one-line fix applied.	2026-07-02 20:39:40 +00:00
root	14bb145300	feat: expand infrastructure-control pattern — Proxmox 5-node cluster, Docker 3-ecosystem, NFS storage, network services ... Full discovery and expansion based on PVE API exploration: Proxmox (5 nodes): - minipve (.12): authentik, gitea, mumuni, syslog-api, jitsi - amdpve (.15): abiba, kagentz, tanko, tdunna, baggy, scottdenya - storepve (.6): docker-vm, ra-h-os, PBS, media, zulip - acerpve (.9): llm-gpu, adguard - ocupve (.5): ocu-llm Docker (3 ecosystems, 22 containers): - docker-vm (.7): Firecrawl, SearXNG, Home stack, Audiobookshelf - CT 116: LiteLLM stack (6 containers) - Netbird (.17): VPN controller + dashboard + crowdsec + traefik Pattern now includes: access matrix, per-section checks/remediations, alert routing, escalation chain, quick-start commands, full CT inventory.	2026-06-27 13:11:55 +00:00
root	e090b27e20	feat: expand infrastructure-control pattern — Proxmox, Docker, storage, network ... Full expansion of the infrastructure-control pattern covering: Layer 1 — Proxmox Cluster (5 nodes) - 4 monitoring rules (offline, resource pressure, stopped VMs/CTs, storage) - Auto-start for 8 critical CTs/VMs on unexpected stop - Node resource threshold alerting Layer 2 — Docker Ecosystems (3 hosts, 22 containers) - 6 monitoring rules across docker-vm, syslog-api, netbird - Auto-restart for critical containers and stacks - Disk pressure auto-prune Layer 3 — Storage Fabric (3.6TB + 7.3TB NFS) - Mount availability monitoring - Growth rate tracking for high-use volumes - PBS backup compliance checking Layer 4 — Network Services (DNS, auth, git, chat, VPN) - Service endpoint monitoring - Escalation to Mumuni for authentik issues Layer 5 — Agent Health (6 agents across 3 platforms) - CT auto-start for stopped agents - Health endpoint verification Full access matrix and priority-based SLA table.	2026-06-27 13:11:15 +00:00
root	acfe16e5e7	feat: infrastructure-control — reference pattern for environment reach ... Maps how OpenProse contracts connect to Gitea, containers, remote agents, and infrastructure. Documents the access matrix, what works today, and what gaps remain for full enforcement.	2026-06-26 18:52:54 +00:00