115 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
root	af41f8f57a	fix: remaining template YAML and topology table corrections ... - hermes-config-template: Template YAML compression model gemma→ornith. Context guidance updated (qwen=256K, only gemma limited to 131K). - litellm-self-heal: GPU topology table fixed (RTX 3090: 128K→256K, parallel 2→1)	2026-07-12 22:55:24 +00:00
root	be02b0e843	fix: stale dates, context values, and architecture references ... - gpu-fleet: Updated all context references (128K→256K for RTX 3090). Fixed parallel count (2→1 for RTX 3090). Updated architecture diagram (router labeled deprecated/not in path). Corrected VRAM values. Updated dates: June→July 2026. - hermes-config-template: Fixed description (removed incorrect '128K on NVIDIA' claim). Updated to reflect verified 256K on RTX 3090. - litellm-self-heal: Updated last-verified date. - litellm-api-keys: Updated last-verified date.	2026-07-12 22:54:30 +00:00
root	79d4a73895	docs: lessons learned from 2026-07-12 session ... - gpu-fleet: Corrected architecture (direct GPU, no router in path). Updated context values (RTX 3090=256K, not 128K). Added api-key standardization requirement. - gpu-self-heal: Added Lessons Learned section with 5 critical findings: L1: API key standardization (RTX 5070 sk-loc...5678 vs not-needed) L2: Fallback chain cascading failure loop detection L3: Verify running state, not documentation L4: Infisical fallback requirement (.env must have uncommented key) L5: Zulip event queue can silently die after ~40 reconnects - litellm-self-heal: Updated status manual-only→deployed, cron schedule - litellm-api-keys: Added Infisical token expiry warning + .env fallback - hermes-config-template: Rule 3 updated with .env fallback requirement	2026-07-12 22:49:39 +00:00
root	19b6db9891	feat: GPU workload redistribution — compression → Strix Halo ... - Move compression model from gemma-4-12b (RTX 5070) to ornith-1.0-35b (Strix Halo) - Add Rule 8: GPU Workload Distribution — per-GPU role assignment - Add Rule 9: Compression Threshold for 256K models - Update Rule 7: Auxiliary Model Consistency with new compression routing - Add gpu-self-heal.prose.md contract with 10 remediation rules - Strix Halo (64GB, 256K, 72.4 tok/s) → compression specialist - RTX 5070 (12GB) → vision/web search specialist - RTX 3090 (24GB, 256K) → heavy reasoning specialist - All rules grilled and confirmed with Kwame 2026-07-12	2026-07-12 22:27:15 +00:00
abiba-bot	0134ad8be6	Merge pull request 'fix: cron alignment + VM→bare metal drift' (#11) from fix/cron-alignment-jul2026 into master	2026-07-09 06:28:04 +00:00
root	546ca79849	fix: cron alignment + VM→bare metal drift ... - pm2-self-heal: cron 30min/daytime → 5min/24×7 (matches contract) - pm2-self-heal: added gpu-monitor to tracked PM2 processes - infra-monitor+control: removed daytime-only restriction (now 24×7) - infrastructure-update: pre-flight VM 101/103 → bare metal .8/.110 - Added @reboot pm2 resurrect for persistence across reboots - README: zulip-monitor.sh → agent-health-check.py - Cleaned up commented-out zulip-monitor cron line Live actions: gpu-monitor-server added to PM2 (persistent across reboots)	2026-07-09 06:27:48 +00:00
abiba-bot	c31dbad95f	Merge pull request 'fix: IP drift and container count from live testing' (#10) from fix/testing-drift-jul2026 into master	2026-07-09 06:24:07 +00:00
root	d829f595b4	fix: IP drift and container count from live testing ... Testing the consolidated litellm-self-heal contract against live infra found: - IP drift: pi host is .24, not .65 (confused with ra-h-os bridge) - Fixed gpu-monitor.prose.md: 192.168.68.65 → 192.168.68.24 (3 locations) - Fixed gpu-fleet.prose.md: .65 → .24 in config files + agent keys tables - Container count: 10 running on CT 116, not 8 as contract stated - Added harness-docker-stats + harness-pve-exporter to litellm-self-heal - Updated container health check list Verified: GPU monitor restarted on .24:9100, ornith model inference working, all 10 containers healthy.	2026-07-09 06:23:44 +00:00
abiba-bot	4164ee5ed0	Merge pull request 'feat: consolidate litellm-health into litellm-self-heal' (#9) from feat/consolidate-litellm-contracts into master	2026-07-09 06:02:47 +00:00
root	053d30ce7c	feat: consolidate litellm-health into litellm-self-heal ... - litellm-self-heal: merged health check as §Health Check section, includes full architecture diagram (v4.0.0), GPU topology, timeout tables, container list, and execution flow. Eliminates duplication between the two contracts. - litellm-health: marked DEPRECATED, retained for reference - README.md: updated with consolidated contract, deprecation notice, disk-gc-threat-response, and infrastructure-monitoring status	2026-07-09 06:01:48 +00:00
abiba-bot	e4ab43f1c6	Merge pull request 'fix: LiteLLM contracts — align with 2026-07-08 architecture changes' (#8) from fix/litellm-contracts-jul2026 into master	2026-07-09 05:37:48 +00:00
root	a2c5a14c74	fix: LiteLLM contracts — align with 2026-07-08 architecture changes ... - litellm-health: v4.0.0 — Router removed from path (nginx→LiteLLM→GPU directly), GPU engines corrected (Docker→systemd), timeouts updated (gemma 120s, qwen 90s), Strix Halo: CPU→Vulkan, context sizes + parallel slots added - litellm-self-heal: ornith context 262K→256K, Rule 7 marked DEPRECATED (router not in path), GPU topology synced with gpu-fleet - litellm-api-keys: reference gpu-fleet as source of truth for keys, removed qwen3.6-35B-A3B from model list (never deployed)	2026-07-09 05:37:16 +00:00
abiba-bot	a7fedf71c6	Merge pull request 'fix: contract improvements from 2026-07-09 run log review' (#7) from fix/contract-improvements-jul2026 into master	2026-07-09 05:33:25 +00:00
root	97977f320c	fix: contract improvements from 2026-07-09 run log review ... - pct-run.sh: add CT 109 note (KVM VM), document decommissioned/migrated CTs - disk-gc-threat-response: add docker-vm (SSH .7), add amdpve Docker scope, remove jitsi (decommissioned), update access matrix with KVM VM section - infrastructure-control: docker-vm container count 11→16, add Trove agents stack - infrastructure-monitoring: deployment status updated (core live, GPU exporters not deployed)	2026-07-09 05:32:00 +00:00
abiba-bot	a3786ab289	Merge pull request 'disk-gc: 2026-07-09 run — amdpve Docker bloat resolved (11.56GB)' (#6) from feat/disk-gc-amdpve-2026-07-09 into master	2026-07-09 05:23:30 +00:00
root	15e8998289	disk-gc: 2026-07-09 run — amdpve Docker bloat resolved (11.56GB reclaimed), access matrix documented, pct-run verified for all 15 CTs	2026-07-09 05:20:42 +00:00
root	25c6c76fb3	disk-gc: 2026-07-09 run — amdpve Docker bloat resolved (11.56GB reclaimed), access matrix documented, pct-run verified for all 15 CTs	2026-07-09 05:17:05 +00:00
root	4819247e39	Add zulip-oidc-redirect-fix prose contract ... Fixes Zulip OIDC authentication where redirect_uri used internal IP (192.168.68.19) instead of public domain (chat.sysloggh.net) after server restart. Monkey-patches social_core.strategy.BaseStrategy to use ROOT_DOMAIN_URI for all redirect URI construction.	2026-07-09 00:02:18 +00:00
root	1dc4155d9a	fix(hermes-zulip-plugin): default to master branch, add gateway restart ... - Switched default branch from feat/zulip-streaming to master - Added Step 5: gateway restart with smoke check after plugin install - Plugin changes require restart to take effect for a valid test - Outstanding feature branch merges should be addressed post-install	2026-07-08 17:51:19 +00:00
root	09efbdf4e5	feat: add hermes-zulip-plugin (install & repair) and relocate hermes-zulip-restore to repo ... - hermes-zulip-plugin.prose.md: installs latest zulip-platform plugin from canonical repo, verifies _strip_html fix, sends relay success signal. Distinct from restore — plugin-layer only, no env/gateway/connection checks. - hermes-zulip-restore.prose.md: relocated from /root/ into prose-contracts repo. - Removed duplicate zulip-platform-verification from /root/vault/contracts/.	2026-07-08 17:47:23 +00:00
jerome	4058010e57	Merge pull request 'feat: hermes-zulip-restore contract' (#5) from feat/hermes-zulip-restore into master ... Reviewed-on: #5	2026-07-08 17:38:16 +00:00
root	776df418ca	gpu-fleet: July 8 optimization sweep — parallel 2 fleet-wide, 128K ctx on NVIDIA, LiteLLM timeout fixes ... - All GPUs: --parallel 1 → 2 (6 concurrent slots, was 3) - .8 RTX 3090: ctx 256K→128K, VRAM 96%→83%, turbo4 KV cache - .110 RTX 5070: ctx 256K→128K, ubatch 4096→512 (was inverted), VRAM 90%→77%, q4_0 KV - .15 Strix Halo: parallel 1→2, 256K ctx (41GB free), q8_0 KV, AMD metrics via /sys/class/drm - LiteLLM: gemma timeout 25→120s, qwen timeout 40→90s, syslog-auto (qwen) 40→90s - Agent configs: context_length 262144 for syslog-auto, 131072 for direct qwen/gemma - Updated health-check operation, agent config implications, benchmark table (fixed model↔GPU mapping)	2026-07-08 17:25:42 +00:00
jerome	c0191c9edc	feat: hermes-zulip-restore contract — restore Zulip on any Hermes agent ... Deploys zulip-platform adapter to bundled plugin path, verifies env credentials, restarts gateway, and validates Zulip connection. Covers Mumuni (CT114), Tanko (CT112), Koby (CT111). Includes _strip_html slash-command fix, known failure modes, and owner-aware restart paths.	2026-07-08 03:19:01 -04:00
jerome	f6e8791352	fix(ci): change kind from reference to template in hermes-agent-baseline.prose.md ... The validate job was failing because 'reference' is not an allowed kind. Changed to 'template' since this document serves as a canonical baseline template for restoring agent configurations.	2026-07-07 20:57:48 -04:00
root	b1ef4cbe9b	docs: self-update contract — Gen 6 postconditions, status clarification, state persistence ... - Added stream_subscription_check, dm_response_time_offline_verifiable, known_issues_tracked postconditions - Added 'Status' section clarifying active for Hermes agents only (not retired with pi Zulip) - Added 'State Persistence' section with KG node schema for build-zulip-plugin:state - Execution step 4: always run offline tests (simulate_dm_latency, selftest) - Execution step 5: update KG state node after each generation	2026-07-07 07:18:45 +00:00
kagentz-bot	41ef546d21	Merge pull request 'fix: use IP 192.168.68.17:3000 instead of git.sysloggh.net for CI checkout' (#4) from fix/ci-checkout-ip into master	2026-07-06 22:56:48 +00:00
Agent Zero	8f89603c31	fix: use IP 192.168.68.17:3000 instead of git.sysloggh.net hostname for CI checkout ... The Gitea Actions runner cannot resolve the internal hostname git.sysloggh.net during checkout. Using the direct IP address resolves the DNS resolution issue in CI pipeline steps.	2026-07-06 18:54:12 -04:00
Agent Zero	f1a031b682	fix: Firecrawl health endpoint is / not /health	2026-07-06 18:21:30 -04:00
Agent Zero	f14b2fc04e	infra-update contract: v1.1.0 - fix CT→VM refs, remove deprecated stacks	2026-07-06 18:05:16 -04:00
Agent Zero	09ffbf10aa	infra-update contract: v1.1.0 — fix CT→VM refs, add missing stacks, correct paths	2026-07-06 18:05:16 -04:00
root	5aa93117de	docs: capture all session changes — port conflict, health check, streaming ... Updated contracts to reflect all infrastructure changes made 2026-07-05/06: gpu-fleet.prose.md: - Port conflict detection on all 3 GPU wrappers - Ghost process root cause documented (.8 stale pid 25836) infrastructure-control.prose.md: - Section 7: Agent Health Check (consolidated, non-disruptive) - Disabled scripts documented (zulip-watchdog, zulip-monitor) hermes-agent-baseline.prose.md: - Health verification section + GPU port conflict detection - Change log updated zulip-health.prose.md: - Streaming support section (edit_message + PATCH API)	2026-07-06 02:46:41 +00:00
root	79855ea9e1	feat: consolidated agent health check — key validation + GPU port conflict + streaming ... Single non-disruptive script replacing 7 scattered Zulip health checks. Checks every 10 min via cron, never restarts anything: - LiteLLM key validation for all 4 Hermes agents - GPU port conflict / ghost process detection - Agent gateway liveness + Zulip streaming status - Recent gateway error count Port conflict detection added to all 3 GPU wrappers: - .8 (qwen): wrapper detects ghost on port 8080 before starting - .110 (gemma): same pattern - .15 (ornith): port-cleanup.sh replaces blanket pkill -x llama-server	2026-07-06 02:23:54 +00:00
root	2ea9cac23a	docs: hermes-agent-baseline — canonical good-state reference ... Single source of truth for all 4 Hermes agents. Captures: - Agent map (CT, IP, key, alias) - Key architecture and config patterns - Mandatory api_key workaround with explanation - Known bug: auxiliary_client ignores api_key_env - Audit procedure (full audit, leak check, key verification) - Systemd patterns - Attachment cache locations	2026-07-05 23:37:56 +00:00
root	35e864bdb7	audit: all 4 Hermes agents verified — no master key, api_key workaround applied ... Agent audit (2026-07-05): Tanko CT 112: sk-CggiHWlamQyShxWC3Hx6uw — vision+compression patched Mumuni CT 114: sk-VrqCNlwUgzoNGOpikJ7nwQ — vision patched Tdunna CT 111: sk-6sbCNjz2T6lTVDBdlNHXsA — vision+compression patched Baggy CT 113: sk-krnw_zGBwvvL5b7l2t-s-A — vision+compression patched All 4 agents: zero master key references, dedicated LiteLLM keys, api_key workaround applied to bypass auxiliary_client bug.	2026-07-05 23:36:00 +00:00
root	53d4290886	docs: document auxiliary_client api_key_env bug + workaround ... _resolve_task_provider_model() in agent/auxiliary_client.py ignores api_key_env for auxiliary tasks (vision, compression). The custom provider path resolves it, but aux tasks take a different code path. Workaround: set api_key directly alongside api_key_env. Applied to Tanko (vision + compression).	2026-07-05 23:17:37 +00:00
root	c8dcd8b63d	fix: Tanko key regenerated, Mumuni key verified from /etc/environment ... Tanko: old key sk-620a05e95a... was from docker-compose router config, not LiteLLM key DB. Deleted + regenerated via LiteLLM API: sk-CggiHWlamQyShxWC3Hx6uw (alias: tanko, budget: $100) Mumuni: verified via pct-run 114 — key is sk-VrqCNlwUgzoNGOpikJ7nwQ (was previously listed as sk-XY2aUfvy2... in contracts)	2026-07-05 20:18:05 +00:00
root	5c90eb8f6c	rename: Koonimo→Baggy, Koby→Tdunna — match hostnames ... Standardize agent names across all contracts and pct-run.sh. No functional changes — same CT IDs, same keys, same nodes.	2026-07-05 20:04:22 +00:00
root	b65116b847	feat: pct-run.sh — run commands in CTs by ID, no IPs needed ... Adds /scripts/pct-run.sh that maps CT IDs to PVE nodes and runs pct exec directly. No hardcoded IPs — Proxmox is the source of truth. Updates infrastructure-control with CT access table using pct-run. Usage examples: pct-run 112 cat /etc/hostname # Tanko pct-run 114 grep LITELLM /etc/environment # Mumuni key check pct-run 116 docker ps # LiteLLM containers	2026-07-05 20:02:20 +00:00
root	0db95a4d1c	fix: correct IPs, CT IDs, and key status across all contracts ... hermes-key-enforcement.prose.md: - Add IP column to verified agents table - Fix Tanko status: was using master key via systemd drop-in (not compliant) - Add detection query step 2: check systemd drop-ins for master key leaks - Add step 3: verify running process env against dedicated key - Mark Mumuni, Koby, Koonimo as Unverified (last check was surface-level only) gpu-fleet.prose.md: - Add CT ID column to agent table for cross-reference - Fix Abiba IP .24 → .65 (pi agent runs on RA-H OS host) - Update Abiba LiteLLM key to match actual models.json key - Set Koonimo IP to unknown (was .114 — incorrect per user) - Fix pi-specific paths .24 → .65 hermes-config-template.prose.md: - Set Koonimo IP to unknown infrastructure-control.prose.md: - Set Koonimo IP to unknown - Fix Koonimo CT reference	2026-07-05 19:58:14 +00:00
root	2d2259035a	Merge branch 'fix/mumuni-review-suggestions': tested changes ... - Recursive detection query catches sub-profile violations (verified) - safe-mutate cross-reference added to Violation Response (verified)	2026-07-05 08:50:02 +00:00
root	7ce6cad01f	fix: incorporate Mumuni review — recursive detection query + safe-mutate cross-reference ... Test results: - Test #1: Recursive grep finds sub-profile violations old query missed (2/2 vs 1/2) - Test #2: safe-mutate step correctly positioned in Violation Response, YAML valid	2026-07-05 08:49:58 +00:00
root	0147ac2abb	Revert "fix: incorporate Mumuni review — recursive detection query + safe-mutate cross-reference" ... This reverts commit c4ab31e56b.	2026-07-05 08:49:23 +00:00
root	c4ab31e56b	fix: incorporate Mumuni review — recursive detection query + safe-mutate cross-reference	2026-07-05 08:46:32 +00:00
root	9cf6304e23	fix: remove hardcoded LiteLLM defaults — rely on Gitea secrets	2026-07-04 23:33:34 +00:00
root	91ec7ea3d2	Merge PR #1: docs: update key enforcement — all agents named keys, systemd fix, CI pipeline	2026-07-04 23:32:19 +00:00
root	e59f59d776	fix: ai-review gracefully skips when LiteLLM secrets not configured	2026-07-04 23:30:25 +00:00
root	889f4e4dd1	docs: update key enforcement — all agents named keys, systemd fix, CI pipeline ... - All 4 Hermes agents now have dedicated named LiteLLM keys (tanko, mumuni, koby, koonimo) - Documented systemd drop-in pattern fix (litellm-key.conf) - Added one-step rotation procedure - Added CI pipeline and branch protection documentation - Removed old mumuni keys, generated fresh keys for koby/koonimo	2026-07-04 23:28:59 +00:00
root	cd7d038f91	ci: verify branch protection — abiba-bot push should work	2026-07-04 23:15:56 +00:00
root	b3ab3a6850	fix: remove all Gitea expression syntax from ai-review script	2026-07-04 23:09:29 +00:00
root	105844cf7b	fix: ai-review script — handle Gitea env vars + push event diffs	2026-07-04 23:08:19 +00:00