Abiba 6efd5ff51c feat: context-aware routing + compaction signals ...

- Added GPU_CONTEXT map (MoE 131K, VLM 131K, Dense 65K)
- Heavy tier now prefers MoE/VLM (131K) over Dense (65K) for large requests
- Response headers: X-Context-Remaining, X-Context-Model
- Routing data includes context_remaining field
- Agents can use this to trigger compaction when nearing limits

2026-05-19 21:13:56 +00:00

..

Dockerfile

Initial commit: CT 116 inference harness — nginx, LiteLLM, router, dashboard, Redis

2026-05-16 18:51:50 +00:00

requirements.txt

Initial commit: CT 116 inference harness — nginx, LiteLLM, router, dashboard, Redis

2026-05-16 18:51:50 +00:00

router.py

feat: context-aware routing + compaction signals

2026-05-19 21:13:56 +00:00