Overview
How might we make generation feel immediate without forcing people to wait for a finished answer?
When replies take longer than a glance
- Chat, coding, and writing tools where responses take seconds and blank waits feel broken.
- Long-form answers where people skim early and may change direction mid-stream.
- Any surface where time-to-first-token matters more than total generation time.
- Products that need interruptibility: stop, edit, or retry before a full wrong answer finishes.
When a complete unit is required
- Tiny deterministic outputs (a single number or yes/no) where streaming adds noise.
- Layouts that must paint as a complete unit (some charts or print-ready pages) until structure is known.
- Accessibility contexts where rapid token updates overwhelm screen readers without a “complete” announcement.
States
Design the whole generation lifecycle, not only the blinking caret.
Waiting
The request is in flight but no tokens yet. A thinking status proves work started without faking the answer.
Streaming
Tokens append in place. Layout stays stable; a caret or live cue marks the growing end.
Interrupted
Stop cancels further tokens. Partial text remains visible and usable.
Complete
Generation finished. The caret clears; copy, share, and follow-ups unlock on the final text.
Failed
The stream errored. Offer retry without losing the prompt or prior turns.
Key UX elements
The parts that must be present for streaming to feel fast and controllable.
Prove work started quickly.
Show a thinking status before tokens arrive. Empty silence after send feels like failure.
Mark the live edge of the reply.
A subtle caret or pulse shows growth without competing with the words.
Let people cancel mid-answer.
A visible Stop control is table stakes when the direction is wrong before the end.
Keep the page from thrashing.
Stream into a stable block. Jumping scroll and reflow on every token breaks reading.
Treat early text as useful.
Allow skim, copy, and redirect before the full answer finishes when the draft is already clear.
Announce the end, not every token.
Clear the live cue and prefer a single “response ready” signal for assistive tech.
Anti-patterns to avoid
No stop or cancel control while tokens stream.
Layout thrash that jumps the page with every token.
Streaming fake progress while the model has not started.
Blocking copy/share until the full answer finishes when partial text is already useful.
How products use it
| Product | Implementation |
|---|---|
| ChatGPT | Token stream in the thread with stop generation during the run. |
| Claude | Streaming replies with interrupt and retry on the same turn. |
| Cursor | Streams code and chat; apply/diff flows wait for stable hunks. |
| Perplexity | Streams the answer while research steps and citations settle. |
Implementation
Copy this prompt to generate a production-ready implementation in Cursor, Claude Code, Lovable, or any AI coding agent.
Generate a production-ready implementation of the "Streaming" AI interface design pattern.
Pattern Definition:Frequently asked questions
What is streaming in AI chat UX?
Streaming shows the model’s output as it is generated, usually token by token, so users see progress immediately instead of staring at a blank wait state until the full answer arrives.
Does streaming make the model faster?
Streaming mainly improves perceived performance. Time to first token feels faster even when total generation time is unchanged. Real speed still depends on model and infrastructure.
Should users be able to stop a stream?
Yes. A visible Stop control is table stakes. Users often know mid-answer that the direction is wrong and should not wait for a full wrong completion.
How should streaming work with screen readers?
Avoid announcing every token. Prefer polite live regions for start/complete, or a “response ready” cue, so assistive tech is not flooded during generation.