diff --git a/inference-optimization.prose.md b/inference-optimization.prose.md index 58c8760..91a10e6 100644 --- a/inference-optimization.prose.md +++ b/inference-optimization.prose.md @@ -1,6 +1,10 @@ --- name: inference-optimization kind: responsibility +description: > + Optimizes the full Syslog inference stack — LiteLLM routing weights, GPU model + assignments, agent context management, and prompt caching — to reduce response + times to sub-15s average. All GPUs now at 128K context (stable ceiling). id: 067NC6KP02RG60S50M40E30928 ---