
Time to first token (TTFT): how fast an AI starts replying
In short: Time to first token is the delay between sending a request and receiving the first token of the reply, the wait users feel before a stream starts.
Time to first token (TTFT) is the time between sending a request to a language model and receiving the first token of its response. It is the wait a user feels before a streaming reply starts. An NVIDIA benchmarking guide from April 2025 counts request queuing, prompt processing and network latency inside TTFT, so it grows with prompt length.
Time to first token in plain words
It is how long the waiter takes to come back with the first plate. The meal may take a while, but a quick first plate makes the wait feel shorter.
How it works
Before generating anything, the model must process the entire prompt, a phase called prefill. Longer prompts, large documents and many tool definitions take longer. Provider load and network distance add further delay.
Models that think before answering add that time too. Anthropic's latency guide, read in November 2026, points speed-critical applications to its fastest model at low effort.
Example
As an illustration of measuring it with a streaming client:
import time
start = time.perf_counter()
stream = client.messages.stream(model=MODEL, max_tokens=500, messages=msgs)
with stream as s:
for event in s.text_stream:
print("TTFT:", round(time.perf_counter() - start, 2), "s"); break
Where you'll meet it
- Model provider performance pages and third-party benchmarks
- Observability dashboards for AI applications
- Voice agents, where a slow first response breaks conversation
- Service-level objectives for AI features
What it changes for you
For product teams, TTFT shapes how fast a feature feels, so stream responses; the Anthropic guide says streaming improves perceived responsiveness. For developers, trim prompts and pick a faster model or lower reasoning effort on interactive paths. OpenAI's prompt caching guide lists a shorter wait before the response starts as one benefit of caching a long, stable prefix. My view: for voice products, TTFT is the first number to watch.
Time to first token vs total latency
TTFT measures when the reply starts. Total latency measures when it ends. The NVIDIA guide defines end-to-end latency as TTFT plus generation time.
Common mistake
My view: measuring latency only with short test prompts. Production prompts with retrieved documents and tool definitions are often much longer, and TTFT rises with them. Measure on real request sizes.
Related terms
Tokens per second · Prompt caching · Inference · Reasoning model · KV cache
From the archive: Measure voice latency with paired turn events
Measure time to first token on production-sized prompts, because short test prompts flatter it.


