Streaming is a protocol, not a vibe

A streamed completion is a sequence of small events over one long-lived HTTP response, typically server-sent events: lines prefixed with data, each carrying a fragment, terminated by a sentinel. Nothing guarantees that a network chunk contains a whole event, so the parser has to buffer and split on delimiters, and if you write that loop assuming one chunk equals one message, it works locally and fails under load.

The infrastructure between you and the user has opinions. Compression middleware, CDN buffering and some serverless platforms will hold the whole response and deliver it at once, turning your carefully streamed feature back into a five-second wait with extra complexity. Test streaming through your actual deployment, not against localhost, because this is the single most common reason a stream stops streaming in production.

Streaming also changes your error model. The status code arrives before the content, so a failure halfway through is a truncated success rather than an HTTP error, and your client has to notice a stream that ended without a completion signal and handle it as failure. That case is missing from most implementations, generated or otherwise.