185 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Abiba	15a826457e	vault: canonical Production Vault Access Process + sync vault + migrate koby/koonimo ... - Vault SYNCED: mumuni/koby/koonimo rotated keys written to Infisical (all validate 200). Vault is source of truth again (was stale since session-13 rotations). - Koby (.129) + Koonimo (.114) migrated from hardcoded systemd drop-ins to infisical-gateway.sh wrapper (live vault injection, .env fallback safety net). 4/5 agents now vault-backed (abiba/mumuni/koby/koonimo). - New § Production Vault Access Process: canonical 7-step pattern, migration status table, key rotation procedure, why-it's-non-fail rationale. - Fixed stale notes: 'vault sync PENDING' (now synced), 'Abiba key IS master key' (now a proper agent key sk-sxbphLvk1OU). - Tanko (user jerome, not systemd) documented as pending migration. - hermes-key-enforcement: cross-ref to canonical process, drop-ins deprecated.	2026-07-16 17:51:53 +00:00
Abiba	cd52deda92	contracts: stable alias chain + no-master-key-for-inference rule ... Agent configs aligned to stable aliases (gpu-light/gpu-dense/strix-moe/syslog-auto): - Tanko fixed: base_url :4000->/v1 (nginx), max_context_window 131072->262144, aux.compression gemma->strix-moe, aux gemma->gpu-light - Mumuni aligned: aux gemma->gpu-light, delegation/x_search qwen->gpu-dense - Koby/Koonimo already used gpu-light (validated) - Zero raw model names remain in any agent config No-master-key rule: litellm_proxy_master_key (sk-litellm-...) is admin-only (/key/list, /key/generate). All inference uses agent keys. Health-check model tests use dedicated monitor key (/etc/litellm-monitor.env on CT 116). Fixed: daily-infra-report.py, gpu-self-heal.py, litellm-health-check.sh. Fixed wrong key-scope note: qwen3.6-35B-udq4 IS in scope for baggy/koby/mumuni/ abiba-pi keys (not 'NOT in allowlist' as previously documented).	2026-07-16 17:39:05 +00:00
Abiba	97f1cd77e4	litellm-self-heal: document 2026-07-16 ops sync (script fixes, systemd gpu-monitor, live-key agent check) ... Health-check script: critical-only gpu-fleet alerting, strix-moe model test. gpu-monitor: systemd unit (was bare &), Strix counted in gpu_count, VRAM thresholds raised (93/97 — 256K ctx steady-state ~96% on RTX 3090). agent-health-check.py: reads live LITELLM_API_KEY from gateway env (no hardcoded keys). daily-infra-report.py: stale SYNTHETIC key → env-based master key.	2026-07-16 17:23:37 +00:00
Abiba	dc572889f8	contracts: sync to ground truth — ornith-1.0-35b→strix-moe, 256K all GPUs, real LiteLLM timeouts ... Verified on ground 2026-07-16 against CT 116 litellm_config.yaml + GPU hosts: - AMD host serves qwen3.6-35B-udq4 (LiteLLM alias strix-moe); ornith-1.0-35b does NOT exist - All 3 GPUs at 256K ctx, parallel 2 (RTX 3090 was listed 128K/parallel 1) - LiteLLM timeouts: qwen 300s, gemma 120s, strix 300s (were stale 90s/120s) - Added LiteLLM model surface + key scoping to litellm-self-heal - Patched health-check script path ref Files: litellm-self-heal, litellm-health, gpu-fleet, gpu-self-heal, zulip-adapter-lessons, abiba-zulip-restore, hermes-agent-baseline, delegation-prose-contract, mumuni-delegation-prose-contract	2026-07-16 17:02:44 +00:00
Abiba	d0feb7881e	Contracts: Rule 13 two key-injection patterns + koby/baggy rotation log (WAL #1300)	2026-07-16 15:10:50 +00:00
Abiba	9fd8c68bd2	Contracts: replace remaining ornith-1.0-35b refs with strix-moe	2026-07-16 13:57:57 +00:00
Abiba	39e2fa0cfa	Contracts: Mumuni context-fix (WAL #1300) — strix-moe, 256K all GPUs, Rule 12/13, machine-identity vault procedure	2026-07-16 13:57:31 +00:00
root	622cf7b176	UPDATE 2026-07-15: Full fleet rebuild ... - Stable role-based aliases: strix-moe, gpu-dense, gpu-light - Strix Halo: ornith-1.0-35b -> unsloth/Qwen3.6-35B-A3B-MTP (UD-Q4_K_M, 256K) - RTX 5070: Q4_K_M -> IQ4_NL+MTP (256K, 125-213 tok/s, 2x faster) - RTX 3090: bumped to 256K context - Provider rename: harness -> litellm (all agents) - syslog-auto: weighted pool restored with direct GPU routing - Mumuni agent profile documented - Self-heal thresholds updated for 256K context - All model contexts corrected (RTX 3090 was 131K, not 256K)	2026-07-15 16:04:21 +00:00
root	30bf42b841	feat: Zulip v3 resilience contract + restore playbook ... - zulip-resilience-v3.prose.md: Production resilience rewrite contract covering circuit breaker, retry with jitter, queue lifecycle management, supervisor watchdog, and PM2 hardening. Research-backed from Zulip event system docs. - abiba-zulip-restore.prose.md: Quick-restore playbook for recovery scenarios.	2026-07-13 21:29:35 +00:00
abiba-bot	190ceb9be4	Merge pull request 'GPU workload redistribution + lessons learned July 2026' (#15) from feat/gpu-workload-compression-jul2026 into master	2026-07-12 23:40:12 +00:00
root	0c298eb9d9	feat: add routing configuration with RPM caps (ornith syslog-auto: 40→60) ... - Strix Halo ornith RPM increased from 40 to 60 inside syslog-auto pool - Prevented unbounded ornith traffic when multiple agents use syslog-auto - Direct ornith remains at RPM=40 - All models verified healthy (ornith 80ms, gemma 110ms, qwen 840ms) - Added full routing table to gpu-fleet contract	2026-07-12 23:38:21 +00:00
root	af41f8f57a	fix: remaining template YAML and topology table corrections ... - hermes-config-template: Template YAML compression model gemma→ornith. Context guidance updated (qwen=256K, only gemma limited to 131K). - litellm-self-heal: GPU topology table fixed (RTX 3090: 128K→256K, parallel 2→1)	2026-07-12 22:55:24 +00:00
root	be02b0e843	fix: stale dates, context values, and architecture references ... - gpu-fleet: Updated all context references (128K→256K for RTX 3090). Fixed parallel count (2→1 for RTX 3090). Updated architecture diagram (router labeled deprecated/not in path). Corrected VRAM values. Updated dates: June→July 2026. - hermes-config-template: Fixed description (removed incorrect '128K on NVIDIA' claim). Updated to reflect verified 256K on RTX 3090. - litellm-self-heal: Updated last-verified date. - litellm-api-keys: Updated last-verified date.	2026-07-12 22:54:30 +00:00
root	79d4a73895	docs: lessons learned from 2026-07-12 session ... - gpu-fleet: Corrected architecture (direct GPU, no router in path). Updated context values (RTX 3090=256K, not 128K). Added api-key standardization requirement. - gpu-self-heal: Added Lessons Learned section with 5 critical findings: L1: API key standardization (RTX 5070 sk-loc...5678 vs not-needed) L2: Fallback chain cascading failure loop detection L3: Verify running state, not documentation L4: Infisical fallback requirement (.env must have uncommented key) L5: Zulip event queue can silently die after ~40 reconnects - litellm-self-heal: Updated status manual-only→deployed, cron schedule - litellm-api-keys: Added Infisical token expiry warning + .env fallback - hermes-config-template: Rule 3 updated with .env fallback requirement	2026-07-12 22:49:39 +00:00
root	19b6db9891	feat: GPU workload redistribution — compression → Strix Halo ... - Move compression model from gemma-4-12b (RTX 5070) to ornith-1.0-35b (Strix Halo) - Add Rule 8: GPU Workload Distribution — per-GPU role assignment - Add Rule 9: Compression Threshold for 256K models - Update Rule 7: Auxiliary Model Consistency with new compression routing - Add gpu-self-heal.prose.md contract with 10 remediation rules - Strix Halo (64GB, 256K, 72.4 tok/s) → compression specialist - RTX 5070 (12GB) → vision/web search specialist - RTX 3090 (24GB, 256K) → heavy reasoning specialist - All rules grilled and confirmed with Kwame 2026-07-12	2026-07-12 22:27:15 +00:00
jerome	22eaaf4254	Merge pull request 'fix: add data source integrity rule to delegation contract' (#14) from feat/data-source-integrity-rule into master ... Reviewed-on: #14	2026-07-12 18:09:10 +00:00
jerome	fdb22948d9	fix: add data source integrity rule to delegation contract ... Workers MUST use provided input data, not fetch external sources. Adds: Data Source Integrity section, dispatch guidance example, anti-pattern entry. Root cause of e2e test discrepancies where writer worker queried Proxmox API instead of using raw data.	2026-07-11 18:51:30 -04:00
jerome	e9968e165d	Merge pull request 'feat: add RA-H OS Custodianship Contract for shared memory protocol' (#13) from feat/ra-h-os-custodianship-contract into master ... Reviewed-on: #13	2026-07-10 23:39:30 +00:00
mumuni-bot	44ba53cf71	fix: add YAML frontmatter to RA-H OS Custodianship Contract ... - Added required frontmatter block with kind=pattern, name, and description - Fixes CI validate step that was blocking the PR pipeline - All other CI checks (lint, ai-review) were skipped due to validate failure	2026-07-10 19:16:30 -04:00
jerome	b96b334283	feat: add RA-H OS Custodianship Contract for shared memory protocol	2026-07-10 09:43:49 -04:00
jerome	dc829bb09d	Merge pull request 'feat: mumuni-delegation-prose-contract — manager operating doctrine for worker delegation' (#12) from feat/delegation-prose-contract into master ... Reviewed-on: #12	2026-07-09 07:43:26 +00:00
jerome	a8354e88bc	rename: delegation-prose-contract → mumuni-delegation-prose-contract ... Contract is Mumuni-specific operating doctrine — different agents have different architectures and don't need this. Renamed file and frontmatter to reflect that.	2026-07-09 03:18:29 -04:00
jerome	98d8ce6772	fix: add topology section to satisfy lint checks ... Added cluster/node topology section referencing all 5 Proxmox nodes and clarifying that the contract is infrastructure-agnostic — defines the who and when, not the where.	2026-07-09 02:53:44 -04:00
jerome	548e421fa1	feat: add delegation-prose-contract pattern — manager operating doctrine for worker delegation ... Defines when to delegate, worker selection matrix, verification protocol, kanban board, and failure handling. Addresses the context overflow and iteration exhaustion issues where the manager was doing all work itself instead of delegating to the 6 worker profiles.	2026-07-09 02:29:42 -04:00
abiba-bot	0134ad8be6	Merge pull request 'fix: cron alignment + VM→bare metal drift' (#11) from fix/cron-alignment-jul2026 into master	2026-07-09 06:28:04 +00:00
root	546ca79849	fix: cron alignment + VM→bare metal drift ... - pm2-self-heal: cron 30min/daytime → 5min/24×7 (matches contract) - pm2-self-heal: added gpu-monitor to tracked PM2 processes - infra-monitor+control: removed daytime-only restriction (now 24×7) - infrastructure-update: pre-flight VM 101/103 → bare metal .8/.110 - Added @reboot pm2 resurrect for persistence across reboots - README: zulip-monitor.sh → agent-health-check.py - Cleaned up commented-out zulip-monitor cron line Live actions: gpu-monitor-server added to PM2 (persistent across reboots)	2026-07-09 06:27:48 +00:00
abiba-bot	c31dbad95f	Merge pull request 'fix: IP drift and container count from live testing' (#10) from fix/testing-drift-jul2026 into master	2026-07-09 06:24:07 +00:00
root	d829f595b4	fix: IP drift and container count from live testing ... Testing the consolidated litellm-self-heal contract against live infra found: - IP drift: pi host is .24, not .65 (confused with ra-h-os bridge) - Fixed gpu-monitor.prose.md: 192.168.68.65 → 192.168.68.24 (3 locations) - Fixed gpu-fleet.prose.md: .65 → .24 in config files + agent keys tables - Container count: 10 running on CT 116, not 8 as contract stated - Added harness-docker-stats + harness-pve-exporter to litellm-self-heal - Updated container health check list Verified: GPU monitor restarted on .24:9100, ornith model inference working, all 10 containers healthy.	2026-07-09 06:23:44 +00:00
abiba-bot	4164ee5ed0	Merge pull request 'feat: consolidate litellm-health into litellm-self-heal' (#9) from feat/consolidate-litellm-contracts into master	2026-07-09 06:02:47 +00:00
root	053d30ce7c	feat: consolidate litellm-health into litellm-self-heal ... - litellm-self-heal: merged health check as §Health Check section, includes full architecture diagram (v4.0.0), GPU topology, timeout tables, container list, and execution flow. Eliminates duplication between the two contracts. - litellm-health: marked DEPRECATED, retained for reference - README.md: updated with consolidated contract, deprecation notice, disk-gc-threat-response, and infrastructure-monitoring status	2026-07-09 06:01:48 +00:00
abiba-bot	e4ab43f1c6	Merge pull request 'fix: LiteLLM contracts — align with 2026-07-08 architecture changes' (#8) from fix/litellm-contracts-jul2026 into master	2026-07-09 05:37:48 +00:00
root	a2c5a14c74	fix: LiteLLM contracts — align with 2026-07-08 architecture changes ... - litellm-health: v4.0.0 — Router removed from path (nginx→LiteLLM→GPU directly), GPU engines corrected (Docker→systemd), timeouts updated (gemma 120s, qwen 90s), Strix Halo: CPU→Vulkan, context sizes + parallel slots added - litellm-self-heal: ornith context 262K→256K, Rule 7 marked DEPRECATED (router not in path), GPU topology synced with gpu-fleet - litellm-api-keys: reference gpu-fleet as source of truth for keys, removed qwen3.6-35B-A3B from model list (never deployed)	2026-07-09 05:37:16 +00:00
abiba-bot	a7fedf71c6	Merge pull request 'fix: contract improvements from 2026-07-09 run log review' (#7) from fix/contract-improvements-jul2026 into master	2026-07-09 05:33:25 +00:00
root	97977f320c	fix: contract improvements from 2026-07-09 run log review ... - pct-run.sh: add CT 109 note (KVM VM), document decommissioned/migrated CTs - disk-gc-threat-response: add docker-vm (SSH .7), add amdpve Docker scope, remove jitsi (decommissioned), update access matrix with KVM VM section - infrastructure-control: docker-vm container count 11→16, add Trove agents stack - infrastructure-monitoring: deployment status updated (core live, GPU exporters not deployed)	2026-07-09 05:32:00 +00:00
abiba-bot	a3786ab289	Merge pull request 'disk-gc: 2026-07-09 run — amdpve Docker bloat resolved (11.56GB)' (#6) from feat/disk-gc-amdpve-2026-07-09 into master	2026-07-09 05:23:30 +00:00
root	15e8998289	disk-gc: 2026-07-09 run — amdpve Docker bloat resolved (11.56GB reclaimed), access matrix documented, pct-run verified for all 15 CTs	2026-07-09 05:20:42 +00:00
root	25c6c76fb3	disk-gc: 2026-07-09 run — amdpve Docker bloat resolved (11.56GB reclaimed), access matrix documented, pct-run verified for all 15 CTs	2026-07-09 05:17:05 +00:00
root	4819247e39	Add zulip-oidc-redirect-fix prose contract ... Fixes Zulip OIDC authentication where redirect_uri used internal IP (192.168.68.19) instead of public domain (chat.sysloggh.net) after server restart. Monkey-patches social_core.strategy.BaseStrategy to use ROOT_DOMAIN_URI for all redirect URI construction.	2026-07-09 00:02:18 +00:00
root	1dc4155d9a	fix(hermes-zulip-plugin): default to master branch, add gateway restart ... - Switched default branch from feat/zulip-streaming to master - Added Step 5: gateway restart with smoke check after plugin install - Plugin changes require restart to take effect for a valid test - Outstanding feature branch merges should be addressed post-install	2026-07-08 17:51:19 +00:00
root	09efbdf4e5	feat: add hermes-zulip-plugin (install & repair) and relocate hermes-zulip-restore to repo ... - hermes-zulip-plugin.prose.md: installs latest zulip-platform plugin from canonical repo, verifies _strip_html fix, sends relay success signal. Distinct from restore — plugin-layer only, no env/gateway/connection checks. - hermes-zulip-restore.prose.md: relocated from /root/ into prose-contracts repo. - Removed duplicate zulip-platform-verification from /root/vault/contracts/.	2026-07-08 17:47:23 +00:00
jerome	4058010e57	Merge pull request 'feat: hermes-zulip-restore contract' (#5) from feat/hermes-zulip-restore into master ... Reviewed-on: #5	2026-07-08 17:38:16 +00:00
root	776df418ca	gpu-fleet: July 8 optimization sweep — parallel 2 fleet-wide, 128K ctx on NVIDIA, LiteLLM timeout fixes ... - All GPUs: --parallel 1 → 2 (6 concurrent slots, was 3) - .8 RTX 3090: ctx 256K→128K, VRAM 96%→83%, turbo4 KV cache - .110 RTX 5070: ctx 256K→128K, ubatch 4096→512 (was inverted), VRAM 90%→77%, q4_0 KV - .15 Strix Halo: parallel 1→2, 256K ctx (41GB free), q8_0 KV, AMD metrics via /sys/class/drm - LiteLLM: gemma timeout 25→120s, qwen timeout 40→90s, syslog-auto (qwen) 40→90s - Agent configs: context_length 262144 for syslog-auto, 131072 for direct qwen/gemma - Updated health-check operation, agent config implications, benchmark table (fixed model↔GPU mapping)	2026-07-08 17:25:42 +00:00
jerome	c0191c9edc	feat: hermes-zulip-restore contract — restore Zulip on any Hermes agent ... Deploys zulip-platform adapter to bundled plugin path, verifies env credentials, restarts gateway, and validates Zulip connection. Covers Mumuni (CT114), Tanko (CT112), Koby (CT111). Includes _strip_html slash-command fix, known failure modes, and owner-aware restart paths.	2026-07-08 03:19:01 -04:00
jerome	f6e8791352	fix(ci): change kind from reference to template in hermes-agent-baseline.prose.md ... The validate job was failing because 'reference' is not an allowed kind. Changed to 'template' since this document serves as a canonical baseline template for restoring agent configurations.	2026-07-07 20:57:48 -04:00
root	b1ef4cbe9b	docs: self-update contract — Gen 6 postconditions, status clarification, state persistence ... - Added stream_subscription_check, dm_response_time_offline_verifiable, known_issues_tracked postconditions - Added 'Status' section clarifying active for Hermes agents only (not retired with pi Zulip) - Added 'State Persistence' section with KG node schema for build-zulip-plugin:state - Execution step 4: always run offline tests (simulate_dm_latency, selftest) - Execution step 5: update KG state node after each generation	2026-07-07 07:18:45 +00:00
kagentz-bot	41ef546d21	Merge pull request 'fix: use IP 192.168.68.17:3000 instead of git.sysloggh.net for CI checkout' (#4) from fix/ci-checkout-ip into master	2026-07-06 22:56:48 +00:00
Agent Zero	8f89603c31	fix: use IP 192.168.68.17:3000 instead of git.sysloggh.net hostname for CI checkout ... The Gitea Actions runner cannot resolve the internal hostname git.sysloggh.net during checkout. Using the direct IP address resolves the DNS resolution issue in CI pipeline steps.	2026-07-06 18:54:12 -04:00
Agent Zero	f1a031b682	fix: Firecrawl health endpoint is / not /health	2026-07-06 18:21:30 -04:00
Agent Zero	f14b2fc04e	infra-update contract: v1.1.0 - fix CT→VM refs, remove deprecated stacks	2026-07-06 18:05:16 -04:00
Agent Zero	09ffbf10aa	infra-update contract: v1.1.0 — fix CT→VM refs, add missing stacks, correct paths	2026-07-06 18:05:16 -04:00