Commit Graph
6 Commits
Author SHA1 Message Date
Abiba 17f4c79fe4 vault: sync BAGGY_LITELLM_API_KEY = KOONIMO_LITELLM_API_KEY (same agent CT113)
PR Pipeline — Authorize → Validate → Review → Merge / auth (pull_request) Successful in 4s
PR Pipeline — Authorize → Validate → Review → Merge / validate (pull_request) Successful in 3s
PR Pipeline — Authorize → Validate → Review → Merge / lint (pull_request) Successful in 2s
PR Pipeline — Authorize → Validate → Review → Merge / ai-review (pull_request) Successful in 3s
PR Pipeline — Authorize → Validate → Review → Merge / gate (pull_request) Successful in 0s
Baggy = Koonimo (CT 113). The vault had two divergent keys:
- BAGGY_LITELLM_API_KEY = sk-QT-kDt8Szo... (stale, 401)
- KOONIMO_LITELLM_API_KEY = sk-OEK7z26n6... (valid, 200, session-13 rotation)

Synced BAGGY to match KOONIMO (the valid key).
Added contract note: both secrets MUST mirror each other.
This was the root cause of Koonimo's 401 after migration —
the ghost process had BAGGY in its env instead of KOONIMO.
2026-07-16 19:22:48 +00:00
Abiba a9caf5b216 vault: fix koby platform connectivity + document migration lessons
PR Pipeline — Authorize → Validate → Review → Merge / auth (pull_request) Successful in 2s
PR Pipeline — Authorize → Validate → Review → Merge / validate (pull_request) Successful in 1s
PR Pipeline — Authorize → Validate → Review → Merge / lint (pull_request) Successful in 1s
PR Pipeline — Authorize → Validate → Review → Merge / ai-review (pull_request) Successful in 1s
PR Pipeline — Authorize → Validate → Review → Merge / gate (pull_request) Successful in 0s
Koby was broken after migration — Zulip credentials lost (only in process memory),
Telegram token overwritten (only existed in old .env.bak-20260603).

Fix: Koby shares Tanko's Zulip bot (tanko-bot@, TANKO_ZULIP_API_KEY in vault).
Telegram token recovered from .env.bak-20260603. Both platforms now connected.

Added Koby migration lessons to contract (back up .env before migration,
inject ALL platform env vars, cat /proc/pid/environ before killing old gateway).
Added KOBY_ZULIP_API_KEY to vault (shares Tanko's key).
2026-07-16 18:19:27 +00:00
Abiba 15a826457e vault: canonical Production Vault Access Process + sync vault + migrate koby/koonimo
PR Pipeline — Authorize → Validate → Review → Merge / auth (pull_request) Successful in 5s
PR Pipeline — Authorize → Validate → Review → Merge / validate (pull_request) Successful in 1s
PR Pipeline — Authorize → Validate → Review → Merge / lint (pull_request) Successful in 2s
PR Pipeline — Authorize → Validate → Review → Merge / ai-review (pull_request) Successful in 3s
PR Pipeline — Authorize → Validate → Review → Merge / gate (pull_request) Successful in 0s
- Vault SYNCED: mumuni/koby/koonimo rotated keys written to Infisical (all validate 200).
  Vault is source of truth again (was stale since session-13 rotations).
- Koby (.129) + Koonimo (.114) migrated from hardcoded systemd drop-ins to
  infisical-gateway.sh wrapper (live vault injection, .env fallback safety net).
  4/5 agents now vault-backed (abiba/mumuni/koby/koonimo).
- New § Production Vault Access Process: canonical 7-step pattern, migration status
  table, key rotation procedure, why-it's-non-fail rationale.
- Fixed stale notes: 'vault sync PENDING' (now synced), 'Abiba key IS master key'
  (now a proper agent key sk-sxbphLvk1OU).
- Tanko (user jerome, not systemd) documented as pending migration.
- hermes-key-enforcement: cross-ref to canonical process, drop-ins deprecated.
2026-07-16 17:51:53 +00:00
Abiba cd52deda92 contracts: stable alias chain + no-master-key-for-inference rule
PR Pipeline — Authorize → Validate → Review → Merge / auth (pull_request) Successful in 2s
PR Pipeline — Authorize → Validate → Review → Merge / validate (pull_request) Successful in 2s
PR Pipeline — Authorize → Validate → Review → Merge / lint (pull_request) Successful in 3s
PR Pipeline — Authorize → Validate → Review → Merge / ai-review (pull_request) Successful in 3s
PR Pipeline — Authorize → Validate → Review → Merge / gate (pull_request) Successful in 0s
Agent configs aligned to stable aliases (gpu-light/gpu-dense/strix-moe/syslog-auto):
- Tanko fixed: base_url :4000->/v1 (nginx), max_context_window 131072->262144,
  aux.compression gemma->strix-moe, aux gemma->gpu-light
- Mumuni aligned: aux gemma->gpu-light, delegation/x_search qwen->gpu-dense
- Koby/Koonimo already used gpu-light (validated)
- Zero raw model names remain in any agent config

No-master-key rule: litellm_proxy_master_key (sk-litellm-...) is admin-only
(/key/list, /key/generate). All inference uses agent keys. Health-check model
tests use dedicated monitor key (/etc/litellm-monitor.env on CT 116).
Fixed: daily-infra-report.py, gpu-self-heal.py, litellm-health-check.sh.

Fixed wrong key-scope note: qwen3.6-35B-udq4 IS in scope for baggy/koby/mumuni/
abiba-pi keys (not 'NOT in allowlist' as previously documented).
2026-07-16 17:39:05 +00:00
Abiba 97f1cd77e4 litellm-self-heal: document 2026-07-16 ops sync (script fixes, systemd gpu-monitor, live-key agent check)
PR Pipeline — Authorize → Validate → Review → Merge / auth (pull_request) Successful in 5s
PR Pipeline — Authorize → Validate → Review → Merge / validate (pull_request) Successful in 1s
PR Pipeline — Authorize → Validate → Review → Merge / lint (pull_request) Successful in 2s
PR Pipeline — Authorize → Validate → Review → Merge / ai-review (pull_request) Successful in 1s
PR Pipeline — Authorize → Validate → Review → Merge / gate (pull_request) Successful in 1s
Health-check script: critical-only gpu-fleet alerting, strix-moe model test.
gpu-monitor: systemd unit (was bare &), Strix counted in gpu_count, VRAM
thresholds raised (93/97 — 256K ctx steady-state ~96% on RTX 3090).
agent-health-check.py: reads live LITELLM_API_KEY from gateway env (no hardcoded keys).
daily-infra-report.py: stale SYNTHETIC key → env-based master key.
2026-07-16 17:23:37 +00:00
Abiba dc572889f8 contracts: sync to ground truth — ornith-1.0-35b→strix-moe, 256K all GPUs, real LiteLLM timeouts
Verified on ground 2026-07-16 against CT 116 litellm_config.yaml + GPU hosts:
- AMD host serves qwen3.6-35B-udq4 (LiteLLM alias strix-moe); ornith-1.0-35b does NOT exist
- All 3 GPUs at 256K ctx, parallel 2 (RTX 3090 was listed 128K/parallel 1)
- LiteLLM timeouts: qwen 300s, gemma 120s, strix 300s (were stale 90s/120s)
- Added LiteLLM model surface + key scoping to litellm-self-heal
- Patched health-check script path ref

Files: litellm-self-heal, litellm-health, gpu-fleet, gpu-self-heal,
zulip-adapter-lessons, abiba-zulip-restore, hermes-agent-baseline,
delegation-prose-contract, mumuni-delegation-prose-contract
2026-07-16 17:02:44 +00:00