GPU workload redistribution + lessons learned July 2026 #15

2026-07-12T23:39:52Z

abiba-bot commented

2026-07-12 23:39:52 +00:00

Compression to Strix Halo. Context corrected. 5 lessons. RPM caps added.

abiba-bot added 5 commits 2026-07-12 23:39:52 +00:00

feat: GPU workload redistribution — compression → Strix Halo 19b6db9891

- Move compression model from gemma-4-12b (RTX 5070) to ornith-1.0-35b (Strix Halo)
- Add Rule 8: GPU Workload Distribution — per-GPU role assignment
- Add Rule 9: Compression Threshold for 256K models
- Update Rule 7: Auxiliary Model Consistency with new compression routing
- Add gpu-self-heal.prose.md contract with 10 remediation rules
- Strix Halo (64GB, 256K, 72.4 tok/s) → compression specialist
- RTX 5070 (12GB) → vision/web search specialist
- RTX 3090 (24GB, 256K) → heavy reasoning specialist
- All rules grilled and confirmed with Kwame 2026-07-12

docs: lessons learned from 2026-07-12 session 79d4a73895

- gpu-fleet: Corrected architecture (direct GPU, no router in path).
  Updated context values (RTX 3090=256K, not 128K). Added api-key
  standardization requirement.
- gpu-self-heal: Added Lessons Learned section with 5 critical findings:
  L1: API key standardization (RTX 5070 sk-loc...5678 vs not-needed)
  L2: Fallback chain cascading failure loop detection
  L3: Verify running state, not documentation
  L4: Infisical fallback requirement (.env must have uncommented key)
  L5: Zulip event queue can silently die after ~40 reconnects
- litellm-self-heal: Updated status manual-only→deployed, cron schedule
- litellm-api-keys: Added Infisical token expiry warning + .env fallback
- hermes-config-template: Rule 3 updated with .env fallback requirement

fix: stale dates, context values, and architecture references be02b0e843

- gpu-fleet: Updated all context references (128K→256K for RTX 3090).
  Fixed parallel count (2→1 for RTX 3090). Updated architecture diagram
  (router labeled deprecated/not in path). Corrected VRAM values.
  Updated dates: June→July 2026.
- hermes-config-template: Fixed description (removed incorrect '128K on
  NVIDIA' claim). Updated to reflect verified 256K on RTX 3090.
- litellm-self-heal: Updated last-verified date.
- litellm-api-keys: Updated last-verified date.

fix: remaining template YAML and topology table corrections af41f8f57a

- hermes-config-template: Template YAML compression model gemma→ornith.
  Context guidance updated (qwen=256K, only gemma limited to 131K).
- litellm-self-heal: GPU topology table fixed (RTX 3090: 128K→256K, parallel 2→1)

feat: add routing configuration with RPM caps (ornith syslog-auto: 40→60)

PR Pipeline — Authorize → Validate → Review → Merge / auth (pull_request) Successful in 2s

Details

PR Pipeline — Authorize → Validate → Review → Merge / validate (pull_request) Successful in 3s

Details

PR Pipeline — Authorize → Validate → Review → Merge / lint (pull_request) Successful in 2s

Details

PR Pipeline — Authorize → Validate → Review → Merge / ai-review (pull_request) Successful in 2s

Details

PR Pipeline — Authorize → Validate → Review → Merge / gate (pull_request) Successful in 2s

Details

0c298eb9d9

- Strix Halo ornith RPM increased from 40 to 60 inside syslog-auto pool
- Prevented unbounded ornith traffic when multiple agents use syslog-auto
- Direct ornith remains at RPM=40
- All models verified healthy (ornith 80ms, gemma 110ms, qwen 840ms)
- Added full routing table to gpu-fleet contract

abiba-bot merged commit 190ceb9be4 into master

2026-07-12 23:40:13 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: SyslogSolution/prose-contracts#15