abiba-bot/inference-harness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
16 Commits 1 Branch 0 Tags
b7882b2434f2d558acf3fc3906772920089fa3aa
T
Clone
Open with VS Code Open with VSCodium Open with Intellij IDEA
Download ZIP Download TAR.GZ Download BUNDLE
Abiba b7882b2434 fix: reduce 27B Dense context to 192K to free VRAM 
RTX 3090 was at 94.9% VRAM at 262K context. Reduced to 192K (196608),
freeing ~2.4GB. VRAM now at 85% with room for active inference.
2026-05-25 00:31:40 +00:00
dashboard
May 19, 2026: Full harness update
2026-05-19 15:03:47 +00:00
nginx
May 19, 2026: Full harness update
2026-05-19 15:03:47 +00:00
router
fix: reduce 27B Dense context to 192K to free VRAM
2026-05-25 00:31:40 +00:00
.gitignore
May 19, 2026: Full harness update
2026-05-19 15:03:47 +00:00
docker-compose.yml
fix: non-blocking GPU health checks + 256K turboquant context upgrade
2026-05-23 05:57:13 +00:00
docker-compose.yml.bak
May 19, 2026: Full harness update
2026-05-19 15:03:47 +00:00
litellm_config.yaml
May 19, 2026: Full harness update
2026-05-19 15:03:47 +00:00
maintenance.sh
May 19, 2026: Full harness update
2026-05-19 15:03:47 +00:00
S
Description
SyslogAI Inference Harness — 3-GPU router, dashboard, LiteLLM proxy
371 KiB
Languages
Python 97.6%
Shell 1.9%
Dockerfile 0.5%