Latest posts
For thirteen days, one model on Cloudflare’s Workers AI answered very long prompts from other customers’ leftover state, with a 200 status. What the engine’s source shows about the gap between a system that passes its checks and one seen to be correct.
How to judge a config change on a live LLM server, when the before-and-after comparison itself is lying to you: the restart, the traffic and the yardstick, and the one setup that sidesteps them.
In SGLang’s default configuration, chunked prefill doesn’t let running replies through while a long prompt is processed. Here’s the flag that does, and what it costs.
