271 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
root	2eb901debf	fix: correct zulip-health Tanko probe to use CT 115 (amdpve) for SSH access ... Fixed SSH key gap issue by probing from CT 115 (amdpve) instead of CT 117 (storepve). Changed from 'ps aux' to 'systemctl is-active dsh-web' + localhost curl probe. Fixes false DEGRADED status from SSH authentication failure.	2026-09-08 09:39:40 +00:00
root	801de0a25c	fix: correct proxmox-monitor probe endpoints to use localhost for exporters ... Docker Stats and PVE Exporter are bound to 127.0.0.1 (localhost-only), not 0.0.0.0. They must be probed from .116 via SSH. Prometheus and Grafana remain 0.0.0.0 (LAN-reachable). Fixes false alarm from remote probe showing connection-refused (by design for localhost binds).	2026-09-08 08:41:21 +00:00
root	36ae464f59	fix: probe alignment across multiple monitoring contracts ... - infra-monitoring: Added Router+LiteLLM+PVE API probes with expected HTTP codes (200/401/500) - gpu-monitor: Added explicit check-health section with Router+LiteLLM probe commands - proxmox-monitor: Added check-health section for Prometheus/Grafana/exporters (all bound to 0.0.0.0) - zulip-health: Fixed A2A port from :8001 to :50080 and added auth-gated expectation (401) Closes: infra-monitoring-probe-alignment, gpu-monitor-probe-alignment, proxmox-monitor-probe-alignment, zulip-health-probe-alignment	2026-09-08 07:20:41 +00:00
root	a3e97ce72b	fix: add Router+LiteLLM+PVE API probes to infrastructure-monitoring check-health ... - Added Router health probe (http://192.168.68.116/health) — expected 200 - Added LiteLLM health probe (http://192.168.68.116/litellm/health) — expected 200 - Added PVE API probe (https://192.168.68.116:8006/api2/json) — 401 expected for unauthenticated - Clarified that 401 means API is up, 000 means unreachable, 500 means API down Closes: infra-monitoring-probe-alignment	2026-09-08 07:20:41 +00:00
root	cc5fe0991c	Move Zulip bot creds from inline to env/vault reference ... The abiba-bot@chat.sysloggh.net:KEY is now sourced from /etc/litellm-monitor.env as ZULIP_USER and ZULIP_BOT_KEY.	2026-09-08 06:18:21 +00:00
mumuni-bot	dc78604360	Merge pull request 'feat: litellm-client-timeouts — standard client timeout/retry policy for all agents' (#61) from fix/litellm-client-timeouts into master	2026-09-08 06:14:03 +00:00
mumuni-bot	7bbf148778	feat: litellm-client-timeouts contract — standard client timeout/retry policy ... Root cause of the 2026-09-06 incident: 157 client-abandoned 408s against a backend that was succeeding at 20-70s/call once clients stopped giving up. Measured grounding: syslog-auto 28.8s avg / 25.4s TTFT over 888 calls; nginx already allows 600s (Rule 5); the gap was entirely client-side. Values: >=300s primary path, >=120s gpu-dense delegation, 2 retries with 15s/45s backoff, identified probes at 30s/hourly, batch jobs chunked + day-scheduled. Verified live: Prometheus metrics, 0.56s syslog-auto probe, gpu-fleet /health/unified topology.	2026-09-08 06:07:43 +00:00
mumuni-bot	3a25c7cce5	Merge pull request 'fix: restore agent-health-check AGENTS roster + land missed Tanko-DSH rows' (#60) from fix/tanko-dsh-reland into master	2026-09-07 07:34:34 +00:00
mumuni-bot	0aa0ea4906	fix: restore agent-health-check AGENTS roster + land missed Tanko-DSH rows ... 2dfc3e1 (PR #50's out-of-band squash) overwrote agent-health-check.py with a version whose AGENTS dict was empty — the health check silently skipped every agent since. Restore the pre-stomp roster verbatim: tanko (runtime: dsh), abiba, koby, koonimo. Also land two Tanko-DSH rows from PR #52 that the earlier merge missed: - hermes-zulip-plugin live-state table (plugin retired on CT112) - zulip-self-heal restart-services table (restart via DSH service, not hermes gateway restart) Mumuni rows from the branch were NOT restored: they carry pre-migration CT100/.24 data, superseded by PRs #54-56 (Mumuni now on kagentz CT105/.14). Closes the reland of PR #52's substance; the stale branch head stays closed.	2026-09-07 07:31:34 +00:00
mumuni-bot	19821ed6b5	Merge pull request 'fix: remove duplicated ## Execution block in infrastructure-monitoring' (#59) from fix/infra-monitoring-dedup into master	2026-09-07 07:12:19 +00:00
mumuni-bot	403fbcdd9f	fix: remove duplicated ## Execution block in infrastructure-monitoring ... PR #50's content was squash-merged out-of-band (2dfc3e1, 2026-08-30) while the PR itself stayed open, double-applying the check-health/Execution section. This drops the second copy; keeps one canonical Execution -> check-health -> Phases 1-4 -> Verification Commands flow.	2026-09-07 07:08:49 +00:00
mumuni-bot	0753f38cf9	Merge pull request 'fix: rename agent-zero fix-summary out of contract scan path' (#58) from fix/agent-zero-summary-not-a-contract into master	2026-09-03 05:15:58 +00:00
mumuni-bot	274596fdd1	fix: rename agent-zero fix-summary out of contract scan path ... agent-zero-fix-summary.prose.md has no YAML frontmatter (kind/name/description), so CI validate fails on every master push that touches it (runs 226-228). It is a dated session fix-log, not a contract - the durable knowledge already lives in agent-zero-openrouter-key.prose.md (kind: function, referenced from the summary). Rename to .md so validate/lint (which scan only *.prose.md) stop rejecting master; matches repo root docs like cron-prompts-review.md. Verified live: local validate repro PASS (36 files), prose-lint PASS at baseline 15 warnings, no new warnings. Intentionally NOT changed: no content edits, no other files, no contract frontmatter added to non-contract docs.	2026-09-03 05:02:41 +00:00
mumuni-bot	8c4df63db4	docs: add critical fix for .env.clobbered-by-new-image ... - Root cause: Agent Zero loaded key from .env.clobbered-by-new-image - Not main .env file! - Both files must be updated when changing OpenRouter key - Added to prose contract for future reference	2026-09-01 21:24:06 +00:00
mumuni-bot	79a1d22c99	docs: note that full container restart was required for key change ... - run_ui process caches API keys in memory - supervisorctl restart run_ui was insufficient - Full container restart (docker restart agent-zero) required	2026-09-01 20:37:11 +00:00
mumuni-bot	782831f548	docs: add vault sync status to agent-zero key contract ... - Note that vault sync is pending (service token not on kagentz) - Key is stored in /home/hermes/syslog/agent-zero-keys.env as fallback	2026-09-01 20:33:05 +00:00
mumuni-bot	143dd3f16b	docs: add Agent Zero fix summary ... - Documents root cause analysis (OpenRouter 401 + Telegram conflicts) - Details fixes applied (key update, telegram plugin disable, restart) - Includes current state verification - Provides next steps and prevention measures	2026-09-01 18:01:11 +00:00
mumuni-bot	11076ad174	feat: add Agent Zero OpenRouter integration to litellm-api-keys ... - Added Agent Zero (kagentz .14) to .env fallback inventory - Documented direct OpenRouter API access (not via LiteLLM proxy) - Included key prefix, user ID, model config, and rotation procedure - Cross-referenced with agent-zero-openrouter-key.prose.md	2026-09-01 17:59:02 +00:00
mumuni-bot	b079c02d0c	feat: add agent-zero-openrouter-key contract ... - Documents OpenRouter API key for Agent Zero Docker container - Key: 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 - User: user_2rt9lCqcd5d7Vk1t18DHsvWdPTT - Storage: Infisical vault (project=agents) + /a0/usr/.env fallback - Model: moonshotai/kimi-k3 (Cost Efficient preset) - Fixed 401 error from 2026-09-01 (old key belonged to different user)	2026-09-01 17:58:02 +00:00
abiba-bot	09065e7dee	Merge pull request 'Fix zulip-monitor.sh: Remove email spam, use Zulip #agent-hub only' (#57) from fix/zulip-monitor-email-fix into master	2026-08-30 12:24:51 +00:00
root	e8b9f990b2	Fix zulip-monitor.sh LOG variable scope bug ... Move LOG, TIMESTAMP, and ISSUES declarations from inside notify() to script scope so they are available throughout the script.	2026-08-30 12:24:06 +00:00
root	2dfc3e1530	Merged PR #50: fix/gpu-dense-docs	2026-08-30 12:16:39 +00:00
root	79af0ae7a3	Merged PR #52: fix/tanko-runtime	2026-08-30 12:16:28 +00:00
mumuni-bot	30c821469b	Merge pull request 'fix: repoint Mumuni refs in Normal-sensitivity contracts to kagentz CT105/.14' (#56) from fix/mumuni-normal-contracts into master	2026-08-30 01:15:56 +00:00
mumuni-bot	3dcbbf1d76	ci: retrigger pipeline (validate job failed in 2s on clean tree; local repro of validation logic = 0 failures)	2026-08-30 01:15:08 +00:00
mumuni-bot	aac4c7eac3	fix: repoint Mumuni refs in Normal-sensitivity contracts to kagentz CT105/.14 ... Third and final PR of the post-migration contract sweep (ref PRs #54, #55). Normal-sensitivity contracts per AGENTS.md (any registered agent): - gpu-fleet: key roster row, Mumuni Agent Profile header + status table (thermal-safeguard incident note left as historical record) - infrastructure-update: apt patch table row; Hermes config path block corrected to /home/hermes/.hermes + system unit /etc/systemd/system/ hermes-gateway.service (was /root/.hermes + user unit) - infrastructure-maintenance: Hermes gateway health-check target - inference-optimization: apply-agent-compression host + config_path All values verified live on kagentz 2026-08-29 (hostname, IP, user, systemd unit, /home/hermes/.hermes). Abiba-owned refs (.24 pi agent, GPU monitor :9100) and CRITICAL files intentionally untouched - flagged to Abiba via relay #739. Refs PR #54, #55; relay #738/#739.	2026-08-30 01:12:22 +00:00
mumuni-bot	031ad814a0	Merge pull request 'fix: correct Mumuni gateway lifecycle path (no sudo on kagentz) + 2 leftover CT100 mentions' (#55) from fix/restart-cmd-and-leftovers into master	2026-08-30 01:08:13 +00:00
mumuni-bot	ca39fead74	fix: correct Mumuni gateway lifecycle path (no sudo on kagentz) + 2 leftover CT100 mentions ... Post-merge verification of PR #54 found: 1. Lifecycle commands in zulip-health/zulip-self-heal used 'sudo systemctl' for hermes-gateway - sudo is NOT installed on kagentz (no sudoers, no polkit user rules). Correct path: root SSH invocation (root@192.168.68.14), matching how the Proxmox host actually manages the unit. 2. hermes-zulip-restore.prose.md line 5 + zulip-resilience-v3 line 423 still said 'Mumuni CT100' in prose - repointed. Found via independent review + live runtime check (whoami, which sudo, journalctl, systemctl show). Refs PR #54, relay #738/#739.	2026-08-30 01:06:30 +00:00
mumuni-bot	9b280060b7	Merge pull request 'fix: repoint Mumuni refs from CT100/.24 to kagentz CT105/.14 post-migration' (#54) from fix/mumuni-kagentz-repoint into master	2026-08-29 21:44:37 +00:00
mumuni-bot	e898048baf	fix: repoint Mumuni refs from CT100/.24 to kagentz CT105/.14 post-migration ... Mumuni migrated off Abiba CT100 (192.168.68.24, purged) to dedicated CT kagentz (192.168.68.14) on minipve. Verified live on kagentz: - Gateway: systemd unit hermes-gateway.service (User=hermes), active - Hermes home: /home/hermes/.hermes - gateway_state.json present at ~/.hermes/gateway_state.json Updates Mumuni rows/references in 9 contracts. Abiba-owned refs (.24 pi agent, GPU monitor, infrastructure-control CRITICAL file) and historical runs/ logs intentionally left unchanged. Per relay #738 follow-ups. Refs #735-#738.	2026-08-29 21:41:07 +00:00
abiba-bot	b01469ba18	Merge pull request 'fix: standardize Hermes contract base URLs to authenticated path' (#53) from fix/hermes-key-enforcement-update into master	2026-08-28 17:16:18 +00:00
root	994ae1b7ac	fix: standardize base URLs to authenticated /litellm/v1 path in remaining contracts ... - Update hermes-agent-baseline.prose.md: change /v1 → /litellm/v1 in all base_url references - Update hermes-key-enforcement.prose.md: change workaround section base_url to /litellm/v1 - Ensures all contracts consistently use the authenticated LiteLLM path - DeepSeek harness exemption remains documented for external providers	2026-08-28 15:32:24 +00:00
abiba-bot	b835986d44	Merge pull request 'Add check-health section with live probes to infrastructure-monitoring contract' (#51) from fix/infra-check-health into master ... Merged by captain approval 2026-08-22: add check-health section with live probes to infrastructure-monitoring contract	2026-08-23 00:01:39 +00:00
root	d6ad016ac9	Add check-health section with live probes to infrastructure-monitoring contract ... Fixes the canned 'API UNREACHABLE (HTTP 000)' reporting. Adds an explicit check-health execution section (Zulip POST, pm2, GPU exporters, Prometheus, Grafana, LiteLLM probes) with a hard RUN LIVE, NEVER ECHO rule, mirroring the gpu-monitor contract pattern. Captain priority 2026-08-22.	2026-08-22 23:59:28 +00:00
mumuni-bot	c3306e87e4	Merge pull request 'fix: ship pm2-self-heal crash-loop guard + align gpu-dense docs to Qwen3.8-27B' (#49) from fix/pm2-guard-gpu-doc into master	2026-08-17 02:34:49 +00:00
mumuni-bot	20fe5adcbc	fix: ship pm2-self-heal crash-loop guard + align gpu-dense docs to Qwen3.8-27B ... - pm2-self-heal.sh: restart abiba-telegram when TEL_RESTARTS > 1000 even if 'online' (catches quiet crash-loops like the 10k-restarts spoton incident); alert includes count. - pm2-self-heal.prose.md: document the crash-loop guard under Rule 2. - gpu-fleet.prose.md / gpu-self-heal.prose.md: gpu-dense is Qwen3.8-27B-Uncensored-Q4_K_M (~16.8GB, alias qwen3.6-27B-code for LiteLLM routing), not SmartCode-Fable-5 (verified live on .8:8080 Aug 16).	2026-08-17 02:32:52 +00:00
mumuni-bot	82eae77cc5	Merge pull request 'docs: reflect hwepve removal from Tabiri cluster (5 nodes + standalone London node)' (#48) from fix/hwepve-london-migration into master	2026-08-17 02:24:44 +00:00
Agent Zero	8b00a4beea	docs: Mumuni now inside Abiba CT 100 in zulip-resilience-v3 status table	2026-08-15 20:13:29 -04:00
Agent Zero	19a67c6815	fix: align mumuni ct field to 100 in daily-infra-report.py	2026-08-15 20:12:09 -04:00
kagentz-bot	44f7008302	docs: reflect hwepve removal from Tabiri cluster (5 nodes + standalone London node) ... - Remove hwepve from cluster member lists/diagrams in infrastructure-control, proxmox-monitor, infrastructure-update, litellm-health, mumuni-delegation - CTs 100 (abiba) and 105 (kagentz) moved to minipve; CT 114 (mumuni) no longer exists (Mumuni runs inside Abiba CT 100) - Add standalone London role + NetBird routing peer description for hwepve - Update scripts: agent-health-check.py, pct-run.sh, prose-ai-review.sh - Update disk-gc CT access table, hermes baselines/restore host references	2026-08-15 20:03:02 -04:00
mumuni-bot	a179164f1f	Merge pull request 'Ship: Koby external-agent ruling (fix api_key_env hygiene + docs)' (#46) from ship/koby-external-agent-fix into master ... merge	2026-08-13 00:33:54 +00:00
mumuni-bot	c4626c3512	Merge pull request 'fix: redirect pm2/zulip health-check logging to Gitea, never graph (hard rule)' (#47) from fix/health-logs-not-graph into master ... merge	2026-08-13 00:33:33 +00:00
mumuni-bot	b799d46596	fix: redirect pm2/zulip health-check logging to Gitea, never graph (hard rule) ... - pm2-self-heal.prose.md: description said 'logs every action to the knowledge graph', contradicting its own execution step (line ~60) that says '(not knowledge graph - hard rule)'. Align description to Gitea. - zulip-self-heal.prose.md: Reporting section said 'every cycle produces a knowledge graph node'. Contract is RETIRED; remove graph-node directive. - Both now consistently log to SyslogSolution/health-logs, never the graph.	2026-08-13 00:09:40 +00:00
root	3abf784538	Fix no-mistakes warnings: remove prose from fence, reorder/renumber rules, and add Koby exception to Rule 10	2026-08-11 18:16:12 +00:00
root	d73084d481	Trivial: trigger no-mistakes re-run (all fixes applied)	2026-08-11 18:04:31 +00:00
root	9f0a940cee	Fix no-mistakes warnings: prose out of YAML fence, Rule 16 rename, table cell fix	2026-08-11 18:02:08 +00:00
root	5c3ba31b17	Ship: Koby external-agent ruling (Resolve conflict and merge master)	2026-08-11 17:48:36 +00:00
root	d0144d6db6	Ship: Koby external-agent ruling (fix api_key_env hygiene + docs)	2026-08-11 17:22:25 +00:00
jerome	1d4f6c8ebb	Merge pull request 'Ship: enforcement-rules-reality-fix (captain directive 2026-08-09)' (#45) from ship/enforcement-rules-reality-fix into master ... Reviewed-on: #45	2026-08-10 16:38:05 +00:00
root	38a32f8b32	Ship: enforcement-rules-reality-fix (captain directive 2026-08-09)	2026-08-10 13:28:53 +00:00