Abiba
d0feb7881e
Contracts: Rule 13 two key-injection patterns + koby/baggy rotation log (WAL #1300 )
2026-07-16 15:10:50 +00:00
Abiba
9fd8c68bd2
Contracts: replace remaining ornith-1.0-35b refs with strix-moe
2026-07-16 13:57:57 +00:00
Abiba
39e2fa0cfa
Contracts: Mumuni context-fix (WAL #1300 ) — strix-moe, 256K all GPUs, Rule 12/13, machine-identity vault procedure
2026-07-16 13:57:31 +00:00
root
622cf7b176
UPDATE 2026-07-15: Full fleet rebuild
...
- Stable role-based aliases: strix-moe, gpu-dense, gpu-light
- Strix Halo: ornith-1.0-35b -> unsloth/Qwen3.6-35B-A3B-MTP (UD-Q4_K_M, 256K)
- RTX 5070: Q4_K_M -> IQ4_NL+MTP (256K, 125-213 tok/s, 2x faster)
- RTX 3090: bumped to 256K context
- Provider rename: harness -> litellm (all agents)
- syslog-auto: weighted pool restored with direct GPU routing
- Mumuni agent profile documented
- Self-heal thresholds updated for 256K context
- All model contexts corrected (RTX 3090 was 131K, not 256K)
2026-07-15 16:04:21 +00:00
root
30bf42b841
feat: Zulip v3 resilience contract + restore playbook
...
- zulip-resilience-v3.prose.md: Production resilience rewrite contract covering
circuit breaker, retry with jitter, queue lifecycle management, supervisor
watchdog, and PM2 hardening. Research-backed from Zulip event system docs.
- abiba-zulip-restore.prose.md: Quick-restore playbook for recovery scenarios.
2026-07-13 21:29:35 +00:00
abiba-bot
190ceb9be4
Merge pull request 'GPU workload redistribution + lessons learned July 2026' ( #15 ) from feat/gpu-workload-compression-jul2026 into master
2026-07-12 23:40:12 +00:00
root
0c298eb9d9
feat: add routing configuration with RPM caps (ornith syslog-auto: 40→60)
...
- Strix Halo ornith RPM increased from 40 to 60 inside syslog-auto pool
- Prevented unbounded ornith traffic when multiple agents use syslog-auto
- Direct ornith remains at RPM=40
- All models verified healthy (ornith 80ms, gemma 110ms, qwen 840ms)
- Added full routing table to gpu-fleet contract
2026-07-12 23:38:21 +00:00
root
af41f8f57a
fix: remaining template YAML and topology table corrections
...
- hermes-config-template: Template YAML compression model gemma→ornith.
Context guidance updated (qwen=256K, only gemma limited to 131K).
- litellm-self-heal: GPU topology table fixed (RTX 3090: 128K→256K, parallel 2→1)
2026-07-12 22:55:24 +00:00
root
be02b0e843
fix: stale dates, context values, and architecture references
...
- gpu-fleet: Updated all context references (128K→256K for RTX 3090).
Fixed parallel count (2→1 for RTX 3090). Updated architecture diagram
(router labeled deprecated/not in path). Corrected VRAM values.
Updated dates: June→July 2026.
- hermes-config-template: Fixed description (removed incorrect '128K on
NVIDIA' claim). Updated to reflect verified 256K on RTX 3090.
- litellm-self-heal: Updated last-verified date.
- litellm-api-keys: Updated last-verified date.
2026-07-12 22:54:30 +00:00
root
79d4a73895
docs: lessons learned from 2026-07-12 session
...
- gpu-fleet: Corrected architecture (direct GPU, no router in path).
Updated context values (RTX 3090=256K, not 128K). Added api-key
standardization requirement.
- gpu-self-heal: Added Lessons Learned section with 5 critical findings:
L1: API key standardization (RTX 5070 sk-loc...5678 vs not-needed)
L2: Fallback chain cascading failure loop detection
L3: Verify running state, not documentation
L4: Infisical fallback requirement (.env must have uncommented key)
L5: Zulip event queue can silently die after ~40 reconnects
- litellm-self-heal: Updated status manual-only→deployed, cron schedule
- litellm-api-keys: Added Infisical token expiry warning + .env fallback
- hermes-config-template: Rule 3 updated with .env fallback requirement
2026-07-12 22:49:39 +00:00
root
19b6db9891
feat: GPU workload redistribution — compression → Strix Halo
...
- Move compression model from gemma-4-12b (RTX 5070) to ornith-1.0-35b (Strix Halo)
- Add Rule 8: GPU Workload Distribution — per-GPU role assignment
- Add Rule 9: Compression Threshold for 256K models
- Update Rule 7: Auxiliary Model Consistency with new compression routing
- Add gpu-self-heal.prose.md contract with 10 remediation rules
- Strix Halo (64GB, 256K, 72.4 tok/s) → compression specialist
- RTX 5070 (12GB) → vision/web search specialist
- RTX 3090 (24GB, 256K) → heavy reasoning specialist
- All rules grilled and confirmed with Kwame 2026-07-12
2026-07-12 22:27:15 +00:00
jerome
22eaaf4254
Merge pull request 'fix: add data source integrity rule to delegation contract' ( #14 ) from feat/data-source-integrity-rule into master
...
Reviewed-on: SyslogSolution/prose-contracts#14
2026-07-12 18:09:10 +00:00
jerome
fdb22948d9
fix: add data source integrity rule to delegation contract
...
Workers MUST use provided input data, not fetch external sources.
Adds: Data Source Integrity section, dispatch guidance example,
anti-pattern entry. Root cause of e2e test discrepancies where
writer worker queried Proxmox API instead of using raw data.
2026-07-11 18:51:30 -04:00
jerome
e9968e165d
Merge pull request 'feat: add RA-H OS Custodianship Contract for shared memory protocol' ( #13 ) from feat/ra-h-os-custodianship-contract into master
...
Reviewed-on: SyslogSolution/prose-contracts#13
2026-07-10 23:39:30 +00:00
mumuni-bot
44ba53cf71
fix: add YAML frontmatter to RA-H OS Custodianship Contract
...
- Added required frontmatter block with kind=pattern, name, and description
- Fixes CI validate step that was blocking the PR pipeline
- All other CI checks (lint, ai-review) were skipped due to validate failure
2026-07-10 19:16:30 -04:00
jerome
b96b334283
feat: add RA-H OS Custodianship Contract for shared memory protocol
2026-07-10 09:43:49 -04:00
jerome
dc829bb09d
Merge pull request 'feat: mumuni-delegation-prose-contract — manager operating doctrine for worker delegation' ( #12 ) from feat/delegation-prose-contract into master
...
Reviewed-on: SyslogSolution/prose-contracts#12
2026-07-09 07:43:26 +00:00
jerome
a8354e88bc
rename: delegation-prose-contract → mumuni-delegation-prose-contract
...
Contract is Mumuni-specific operating doctrine — different agents have
different architectures and don't need this. Renamed file and frontmatter
to reflect that.
2026-07-09 03:18:29 -04:00
jerome
98d8ce6772
fix: add topology section to satisfy lint checks
...
Added cluster/node topology section referencing all 5 Proxmox nodes and
clarifying that the contract is infrastructure-agnostic — defines the who
and when, not the where.
2026-07-09 02:53:44 -04:00
jerome
548e421fa1
feat: add delegation-prose-contract pattern — manager operating doctrine for worker delegation
...
Defines when to delegate, worker selection matrix, verification protocol,
kanban board, and failure handling. Addresses the context overflow and
iteration exhaustion issues where the manager was doing all work itself
instead of delegating to the 6 worker profiles.
2026-07-09 02:29:42 -04:00
abiba-bot
0134ad8be6
Merge pull request 'fix: cron alignment + VM→bare metal drift' ( #11 ) from fix/cron-alignment-jul2026 into master
2026-07-09 06:28:04 +00:00
root
546ca79849
fix: cron alignment + VM→bare metal drift
...
- pm2-self-heal: cron 30min/daytime → 5min/24×7 (matches contract)
- pm2-self-heal: added gpu-monitor to tracked PM2 processes
- infra-monitor+control: removed daytime-only restriction (now 24×7)
- infrastructure-update: pre-flight VM 101/103 → bare metal .8/.110
- Added @reboot pm2 resurrect for persistence across reboots
- README: zulip-monitor.sh → agent-health-check.py
- Cleaned up commented-out zulip-monitor cron line
Live actions: gpu-monitor-server added to PM2 (persistent across reboots)
2026-07-09 06:27:48 +00:00
abiba-bot
c31dbad95f
Merge pull request 'fix: IP drift and container count from live testing' ( #10 ) from fix/testing-drift-jul2026 into master
2026-07-09 06:24:07 +00:00
root
d829f595b4
fix: IP drift and container count from live testing
...
Testing the consolidated litellm-self-heal contract against live infra found:
- IP drift: pi host is .24, not .65 (confused with ra-h-os bridge)
- Fixed gpu-monitor.prose.md: 192.168.68.65 → 192.168.68.24 (3 locations)
- Fixed gpu-fleet.prose.md: .65 → .24 in config files + agent keys tables
- Container count: 10 running on CT 116, not 8 as contract stated
- Added harness-docker-stats + harness-pve-exporter to litellm-self-heal
- Updated container health check list
Verified: GPU monitor restarted on .24:9100, ornith model inference working,
all 10 containers healthy.
2026-07-09 06:23:44 +00:00
abiba-bot
4164ee5ed0
Merge pull request 'feat: consolidate litellm-health into litellm-self-heal' ( #9 ) from feat/consolidate-litellm-contracts into master
2026-07-09 06:02:47 +00:00
root
053d30ce7c
feat: consolidate litellm-health into litellm-self-heal
...
- litellm-self-heal: merged health check as §Health Check section,
includes full architecture diagram (v4.0.0), GPU topology, timeout
tables, container list, and execution flow. Eliminates duplication
between the two contracts.
- litellm-health: marked DEPRECATED, retained for reference
- README.md: updated with consolidated contract, deprecation notice,
disk-gc-threat-response, and infrastructure-monitoring status
2026-07-09 06:01:48 +00:00
abiba-bot
e4ab43f1c6
Merge pull request 'fix: LiteLLM contracts — align with 2026-07-08 architecture changes' ( #8 ) from fix/litellm-contracts-jul2026 into master
2026-07-09 05:37:48 +00:00
root
a2c5a14c74
fix: LiteLLM contracts — align with 2026-07-08 architecture changes
...
- litellm-health: v4.0.0 — Router removed from path (nginx→LiteLLM→GPU directly),
GPU engines corrected (Docker→systemd), timeouts updated (gemma 120s, qwen 90s),
Strix Halo: CPU→Vulkan, context sizes + parallel slots added
- litellm-self-heal: ornith context 262K→256K, Rule 7 marked DEPRECATED
(router not in path), GPU topology synced with gpu-fleet
- litellm-api-keys: reference gpu-fleet as source of truth for keys,
removed qwen3.6-35B-A3B from model list (never deployed)
2026-07-09 05:37:16 +00:00
abiba-bot
a7fedf71c6
Merge pull request 'fix: contract improvements from 2026-07-09 run log review' ( #7 ) from fix/contract-improvements-jul2026 into master
2026-07-09 05:33:25 +00:00
root
97977f320c
fix: contract improvements from 2026-07-09 run log review
...
- pct-run.sh: add CT 109 note (KVM VM), document decommissioned/migrated CTs
- disk-gc-threat-response: add docker-vm (SSH .7), add amdpve Docker scope,
remove jitsi (decommissioned), update access matrix with KVM VM section
- infrastructure-control: docker-vm container count 11→16, add Trove agents stack
- infrastructure-monitoring: deployment status updated (core live, GPU exporters not deployed)
2026-07-09 05:32:00 +00:00
abiba-bot
a3786ab289
Merge pull request 'disk-gc: 2026-07-09 run — amdpve Docker bloat resolved (11.56GB)' ( #6 ) from feat/disk-gc-amdpve-2026-07-09 into master
2026-07-09 05:23:30 +00:00
root
15e8998289
disk-gc: 2026-07-09 run — amdpve Docker bloat resolved (11.56GB reclaimed), access matrix documented, pct-run verified for all 15 CTs
2026-07-09 05:20:42 +00:00
root
25c6c76fb3
disk-gc: 2026-07-09 run — amdpve Docker bloat resolved (11.56GB reclaimed), access matrix documented, pct-run verified for all 15 CTs
2026-07-09 05:17:05 +00:00
root
4819247e39
Add zulip-oidc-redirect-fix prose contract
...
Fixes Zulip OIDC authentication where redirect_uri used internal IP
(192.168.68.19) instead of public domain (chat.sysloggh.net) after
server restart. Monkey-patches social_core.strategy.BaseStrategy
to use ROOT_DOMAIN_URI for all redirect URI construction.
2026-07-09 00:02:18 +00:00
root
1dc4155d9a
fix(hermes-zulip-plugin): default to master branch, add gateway restart
...
- Switched default branch from feat/zulip-streaming to master
- Added Step 5: gateway restart with smoke check after plugin install
- Plugin changes require restart to take effect for a valid test
- Outstanding feature branch merges should be addressed post-install
2026-07-08 17:51:19 +00:00
root
09efbdf4e5
feat: add hermes-zulip-plugin (install & repair) and relocate hermes-zulip-restore to repo
...
- hermes-zulip-plugin.prose.md: installs latest zulip-platform plugin from
canonical repo, verifies _strip_html fix, sends relay success signal.
Distinct from restore — plugin-layer only, no env/gateway/connection checks.
- hermes-zulip-restore.prose.md: relocated from /root/ into prose-contracts repo.
- Removed duplicate zulip-platform-verification from /root/vault/contracts/.
2026-07-08 17:47:23 +00:00
jerome
4058010e57
Merge pull request 'feat: hermes-zulip-restore contract' ( #5 ) from feat/hermes-zulip-restore into master
...
Reviewed-on: SyslogSolution/prose-contracts#5
2026-07-08 17:38:16 +00:00
root
776df418ca
gpu-fleet: July 8 optimization sweep — parallel 2 fleet-wide, 128K ctx on NVIDIA, LiteLLM timeout fixes
...
- All GPUs: --parallel 1 → 2 (6 concurrent slots, was 3)
- .8 RTX 3090: ctx 256K→128K, VRAM 96%→83%, turbo4 KV cache
- .110 RTX 5070: ctx 256K→128K, ubatch 4096→512 (was inverted), VRAM 90%→77%, q4_0 KV
- .15 Strix Halo: parallel 1→2, 256K ctx (41GB free), q8_0 KV, AMD metrics via /sys/class/drm
- LiteLLM: gemma timeout 25→120s, qwen timeout 40→90s, syslog-auto (qwen) 40→90s
- Agent configs: context_length 262144 for syslog-auto, 131072 for direct qwen/gemma
- Updated health-check operation, agent config implications, benchmark table (fixed model↔GPU mapping)
2026-07-08 17:25:42 +00:00
jerome
c0191c9edc
feat: hermes-zulip-restore contract — restore Zulip on any Hermes agent
...
Deploys zulip-platform adapter to bundled plugin path, verifies
env credentials, restarts gateway, and validates Zulip connection.
Covers Mumuni (CT114), Tanko (CT112), Koby (CT111).
Includes _strip_html slash-command fix, known failure modes,
and owner-aware restart paths.
2026-07-08 03:19:01 -04:00
jerome
f6e8791352
fix(ci): change kind from reference to template in hermes-agent-baseline.prose.md
...
The validate job was failing because 'reference' is not an allowed kind.
Changed to 'template' since this document serves as a canonical baseline
template for restoring agent configurations.
2026-07-07 20:57:48 -04:00
root
b1ef4cbe9b
docs: self-update contract — Gen 6 postconditions, status clarification, state persistence
...
- Added stream_subscription_check, dm_response_time_offline_verifiable, known_issues_tracked postconditions
- Added 'Status' section clarifying active for Hermes agents only (not retired with pi Zulip)
- Added 'State Persistence' section with KG node schema for build-zulip-plugin:state
- Execution step 4: always run offline tests (simulate_dm_latency, selftest)
- Execution step 5: update KG state node after each generation
2026-07-07 07:18:45 +00:00
kagentz-bot
41ef546d21
Merge pull request 'fix: use IP 192.168.68.17:3000 instead of git.sysloggh.net for CI checkout' ( #4 ) from fix/ci-checkout-ip into master
2026-07-06 22:56:48 +00:00
Agent Zero
8f89603c31
fix: use IP 192.168.68.17:3000 instead of git.sysloggh.net hostname for CI checkout
...
The Gitea Actions runner cannot resolve the internal hostname
git.sysloggh.net during checkout. Using the direct IP address
resolves the DNS resolution issue in CI pipeline steps.
2026-07-06 18:54:12 -04:00
Agent Zero
f1a031b682
fix: Firecrawl health endpoint is / not /health
2026-07-06 18:21:30 -04:00
Agent Zero
f14b2fc04e
infra-update contract: v1.1.0 - fix CT→VM refs, remove deprecated stacks
2026-07-06 18:05:16 -04:00
Agent Zero
09ffbf10aa
infra-update contract: v1.1.0 — fix CT→VM refs, add missing stacks, correct paths
2026-07-06 18:05:16 -04:00
root
5aa93117de
docs: capture all session changes — port conflict, health check, streaming
...
Updated contracts to reflect all infrastructure changes made 2026-07-05/06:
gpu-fleet.prose.md:
- Port conflict detection on all 3 GPU wrappers
- Ghost process root cause documented (.8 stale pid 25836)
infrastructure-control.prose.md:
- Section 7: Agent Health Check (consolidated, non-disruptive)
- Disabled scripts documented (zulip-watchdog, zulip-monitor)
hermes-agent-baseline.prose.md:
- Health verification section + GPU port conflict detection
- Change log updated
zulip-health.prose.md:
- Streaming support section (edit_message + PATCH API)
2026-07-06 02:46:41 +00:00
root
79855ea9e1
feat: consolidated agent health check — key validation + GPU port conflict + streaming
...
Single non-disruptive script replacing 7 scattered Zulip health checks.
Checks every 10 min via cron, never restarts anything:
- LiteLLM key validation for all 4 Hermes agents
- GPU port conflict / ghost process detection
- Agent gateway liveness + Zulip streaming status
- Recent gateway error count
Port conflict detection added to all 3 GPU wrappers:
- .8 (qwen): wrapper detects ghost on port 8080 before starting
- .110 (gemma): same pattern
- .15 (ornith): port-cleanup.sh replaces blanket pkill -x llama-server
2026-07-06 02:23:54 +00:00
root
2ea9cac23a
docs: hermes-agent-baseline — canonical good-state reference
...
Single source of truth for all 4 Hermes agents. Captures:
- Agent map (CT, IP, key, alias)
- Key architecture and config patterns
- Mandatory api_key workaround with explanation
- Known bug: auxiliary_client ignores api_key_env
- Audit procedure (full audit, leak check, key verification)
- Systemd patterns
- Attachment cache locations
2026-07-05 23:37:56 +00:00
root
35e864bdb7
audit: all 4 Hermes agents verified — no master key, api_key workaround applied
...
Agent audit (2026-07-05):
Tanko CT 112: sk-CggiHWlamQyShxWC3Hx6uw — vision+compression patched
Mumuni CT 114: sk-VrqCNlwUgzoNGOpikJ7nwQ — vision patched
Tdunna CT 111: sk-6sbCNjz2T6lTVDBdlNHXsA — vision+compression patched
Baggy CT 113: sk-krnw_zGBwvvL5b7l2t-s-A — vision+compression patched
All 4 agents: zero master key references, dedicated LiteLLM keys,
api_key workaround applied to bypass auxiliary_client bug.
2026-07-05 23:36:00 +00:00