Measure voice latency with paired turn events

About 600 milliseconds looks quick until you ask where the stopwatch started. Pipecat's August 2026 PhoneBench table reports roughly that P95 time-to-first-answer-token for PhoneLLM Alpha 1. Its boundary is request start to first answer token—not the caller's last sound to the bot's first audible reply.

My engineering rule is to store the two events behind every latency number, attached to the same turn. That makes a model comparison useful without quietly promoting it into an end-to-end promise. The implementation below checks those boundaries; it does not benchmark PhoneLLM.

Pipecat exposes several different waits

Pek Pongpaet's local PhoneLLM demonstration prompted this question. A specialised voice model is worth investigating. Before changing models, though, I would inspect the measurement that justified the change. That is editorial judgment, not a claim that I deployed his stack.

Pipecat's metrics guide, checked on 15 September 2026, separates first-byte, first-answer-token, TTS first-audio and text-aggregation timing. Its user-to-bot observer measures another interval. A tool call can finish one first-answer-token measurement and require a second inference after the tool returns. One fast event does not tell you when a useful spoken answer arrived.

Keep those component metrics for diagnosis. Add paired events for the boundary you actually want to improve.

Calculate each voice turn before calculating percentiles

Start with this dependency-free Python example. The timestamps are deliberately synthetic, in nanoseconds on one clock. Save it as turn_latency.py and run python3 turn_latency.py.

def latency_ms(events):
    names = ("speech_end", "llm_start",
             "first_answer", "first_audio_ready")
    times = [events[name] for name in names]
    if any(type(t) is not int or t < 0 for t in times):
        raise ValueError("Expected non-negative integer nanoseconds")
    if times != sorted(times):
        raise ValueError("Events must share one ordered clock")
    speech, request, answer, audio = times
    return {
        "request_to_answer_ms": (answer - request) / 1_000_000,
        "speech_to_audio_ready_ms": (audio - speech) / 1_000_000,
    }

sample = {
    "speech_end": 0,
    "llm_start": 300_000_000,
    "first_answer": 900_000_000,
    "first_audio_ready": 1_400_000_000,
}
print(latency_ms(sample))

The local synthetic check produced 600.0 and 1400.0 milliseconds respectively. It also rejected reversed timestamps and a missing event. These are checks of the calculation and input contract, not measured voice-service performance. Nothing in that output says PhoneLLM takes 1.4 seconds.

For actual collection, attach a session ID and turn ID to each event. Stamp callbacks in the same process with time.monotonic_ns(). Python documents this as a monotonic clock with an undefined reference point: use differences, not calendar interpretations. Store each turn separately, then calculate percentiles over its completed deltas. Adding independently calculated component P95s does not give you a measured end-to-end P95; those slow components may belong to different turns.

Audio ready still stops before the listener

Wire first_audio_ready to a defined output boundary, not the first arbitrary TTS byte, which might precede useful audio. Label it exactly that. Network delivery, buffering and playback remain outside this server-side example. To measure the caller's experience, instrument the relevant client/audio boundary; never subtract a browser clock from a server clock as though their origins matched.

The ordered-event assumption also excludes speculative replies and overlapping turns. Count interrupted, timed-out and incomplete turns separately instead of dropping them silently from your report. Otherwise the nicest latency chart can describe only the calls that survived your filter. Keep task success beside timing: a quick acknowledgement is not a completed booking.

What's in it for you

• Keep the model benchmark useful without mistaking it for a caller-experience measurement.
• Reproduce a small timing check before wiring equivalent events into your own service.
• Find whether the next investigation belongs before inference, inside inference or after the answer token.

Give every latency number a start event, an end event and a turn ID before giving it a target.