TTS Latency: How Time to First Audio Actually Works

A developer sends text to a TTS API and hears speech back in under 200 milliseconds. What happens in those 200 milliseconds is a multi-stage pipeline that most documentation treats as atomic.

Time to First Audio (TTFA) is the single metric that correlates most closely with how responsive a voice agent feels to the person talking to it. Every major TTS provider is optimising for it. But the pipeline from text input to playable audio involves more moving parts than most people realise, and where you measure matters as much as what you measure.

What is Time to First Audio and why does it matter for voice agents

TTFA is the elapsed time between sending a TTS request and receiving the first playable audio sample. It is not the same as total synthesis time (generating the full audio) or time to first byte (which can include empty container headers). For voice agents using streaming synthesis, audio playback starts as soon as the first chunk arrives. The agent can begin speaking while the rest of the sentence is still being generated. That makes TTFA the metric that maps to user-perceived responsiveness, not total generation time.

In June 2026, TTFA figures span a wide range. Cartesia Sonic-3.5 reports roughly 82ms end-to-end TTFA. ElevenLabs Flash v2.5 sits around 75ms model inference latency. Gradium claims 155ms P50. Deepgram Aura-2 runs at about 313ms. The ITU-T G.114 recommendation sets 150ms as the one-way delay target for natural conversation, with 300ms as the outer acceptable bound. Below 100ms is where voice agents start to feel like they are not delaying their responses at all.

How the TTS latency pipeline breaks down

A TTS request goes through four distinct stages before audio reaches the caller. Each stage contributes to TTFA in different ways.

Text preprocessing. The input text is tokenised, normalised (numbers to words, abbreviations expanded), and parsed for SSML tags or prosody instructions. This stage is fast relative to the rest of the pipeline. It usually completes in under 5ms for typical sentence lengths. But complex SSML with multiple breaks, phoneme tags, or multi-language sections can push it past 20ms.

Acoustic model inference. The preprocessed text passes through the model that predicts acoustic features: mel-spectrograms, pitch contours, duration patterns. This is where the architecture matters. Transformer-based models scale quadratically with sequence length. State Space Models (SSMs) like those used in Cartesia Sonic scale linearly. A model with 500M parameters does this in around 30-50ms on a modern GPU. Larger multi-task models can take 100ms or more per forward pass.

Vocoder or generative stage. The acoustic features are converted into raw audio waveforms. HiFi-GAN v2 based vocoders run in roughly 10-15ms for the first chunk. Newer diffusion-based generative decoders produce higher quality audio but add 20-40ms of inference time. This stage is often the biggest contributor to quality differences between providers, because the acoustic model output can look similar across architectures while the decoder step determines how natural the final waveform sounds.

Packetisation and transmission. The generated audio is encoded into the output format (WAV, MP3, Opus), wrapped in the protocol response, and sent over the network. Format choice matters here. Opus encoding adds 5-15ms of compression time. Raw PCM skips encoding but increases payload size. Network round-trip time between the client and the API endpoint adds anywhere from 2ms in the same region to 100ms across continents.

What makes TTFA benchmarks inconsistent

Comparing TTFA across providers is harder than it should be because every benchmark measures something different.

Some providers report model inference latency only, excluding network, preprocessing, and packetisation. Others report end-to-end TTFA from a specific geographic region. Some run their benchmarks on warm caches. A few return WAV headers within milliseconds while the first actual audio sample arrives much later, inflating their time-to-first-byte numbers without corresponding audio.

Gradium published a benchmark in May 2026 measuring TTFA from a single Paris office across multiple providers. Their numbers show Gradium at 155ms P50 with a remarkably narrow 2ms interquartile range. Cartesia Sonic-3 at 188ms P50 but with a 100ms IQR. The wide IQR for Cartesia raises questions about whether the benchmark was hitting cold starts or variable-load nodes. Without standardised measurement conditions across the same region, warm state, format, and payload, the ranking tells you more about whose infrastructure is closest to Paris than whose engine is actually fastest.

The Coval TTS benchmark tracks TTFA continuously from multiple regions and is the closest thing to an independent standard today. Even there, the methodology matters. The format your client uses (Opus versus PCM versus MP3), the audio sample rate, and the geographic distance to the provider’s edge all shift the numbers.

What streaming does to the latency calculation

Streaming TTS changes the equation because it does not wait for full synthesis before sending audio. The model generates one chunk, sends it immediately, and keeps generating the next chunk while playback starts.

This means the TTFA for a streaming endpoint captures only the time to generate and transmit the first chunk, which is typically much shorter than the time to generate the full sentence. A non-streaming endpoint for a 10-second clip would report TTFA equal to total generation time. A streaming endpoint for the same clip might send the first chunk in 150ms and continue streaming while the user hears the first words.

The tradeoff is that streaming shifts complexity to the client. The client has to handle chunked audio playback, buffering, and seamless concatenation. Some providers handle buffering server-side. Others leave it to the SDK. The latency number on the provider’s marketing page rarely tells you which model they used.

How TTFA fits into the voice agent latency budget

For a voice agent processing a user’s speech in real time, TTFA is one component in a longer chain. The STT model transcribes the user’s speech. The LLM generates a response. The TTS model reads that response as audio. Each stage adds its own latency.

A typical production budget in June 2026 looks something like this. STT transcribes in 200-400ms for a 3-second utterance. The LLM generates the first response token in 100-500ms depending on model size and whether speculative decoding is enabled. TTS generates the first audio chunk in 100-300ms on a fast streaming endpoint. The round-trip lands between 400ms and 1200ms. Staying under 700ms consistently requires optimisation at every layer.

The ITU-T G.114 recommendation sets 150ms as the target one-way delay for natural conversation, with 300ms as acceptable. For a voice agent, that 300ms ceiling applies to the whole pipeline, not just TTS. That is why the TTFA leaderboard is not just a curiosity for TTS vendors. It directly constrains which voice agent architectures can stay under the latency budget for natural conversation. A TTS provider at 300ms TTFA eats the entire budget before STT or LLM processing even starts.

What to look for when evaluating TTS latency in production

Run your own measurement from your target region. The provider’s published TTFA is likely measured from their cloud region or a major metropolitan area where they host test infrastructure. If your users are in Southeast Asia and the provider’s nearest endpoint is in Oregon, your real-world TTFA will be different.

Test with your actual audio format. Opus compression, sample rate conversion, and streaming protocol choices all affect measured latency. A provider that publishes benchmark numbers using raw PCM might not deliver the same performance when you request MP3 output.

Measure TTFA on cold and warm runs. Some providers cache voice profiles or model weights after the first request, making the second and subsequent calls significantly faster. If your use case involves sustained sessions with the same voice, warm performance is what matters. If you are handling one-off calls from unknown voices, cold performance is the relevant number.

Consider the provider’s geographic edge distribution. The lowest-latency providers in June 2026 differentiate partly through inference architecture and partly through how close their inference nodes are to major population centres. A 50ms model advantage disappears if the nearest node is an extra 80ms round-trip away.

FAQ

What is the difference between TTFA and total synthesis time?

TTFA measures when the first playable audio arrives. Total synthesis time measures when the entire audio generation completes. For streaming TTS, these are very different numbers because playback starts long before generation finishes. TTFA is the relevant metric for voice agent responsiveness. Total synthesis time matters for batch processing and file generation.

Which TTS providers had the lowest TTFA in June 2026?

Cartesia Sonic-3.5 reported roughly 82ms end-to-end TTFA, the lowest publicly available figure. ElevenLabs Flash v2.5 runs at about 75ms model inference latency. Gradium measured 155ms P50 in independent benchmarks. Deepgram Aura-2 sits around 313ms. Direct comparison is complicated by different measurement methodologies and geographic factors.

Does streaming TTS always have lower latency than non-streaming?

Yes for TTFA, no for total generation time. Streaming TTS sends the first audio chunk as soon as it is generated, so the user hears speech earlier. Non-streaming endpoints wait until the full audio file is synthesised before sending anything. For a voice agent scenario, streaming is always preferable because the user perceives less delay, even though the total compute time is roughly the same.

How does the network affect TTS latency?

Network round-trip time between the client and the API endpoint adds directly to TTFA. A client in Singapore connecting to a provider with no local edge node might see 100ms or more of network latency added to the provider’s engine time. Providers with regional edge inference deploy closer to users and reduce this component significantly. Always test from the region your users will be in.

Why do different TTS latency benchmarks show different results?

Benchmarks differ in measurement methodology, geographic origin, warm versus cold state, output format, and payload size. Some report model inference time only. Others report end-to-end TTFA including network and preprocessing. Without standardised conditions, direct comparisons across providers are unreliable. The most useful benchmark is the one you run yourself with your own audio, from your own region.