What surprised me about TTS API design after years of STT

Switching from speech-to-text to text-to-speech APIs means unlearning a bunch of assumptions about how voice APIs should work. Here are the things that caught me off guard.

Output is harder than input

With STT, the API receives audio and returns text. The hard part is accuracy and latency. The API surface is relatively simple: send audio, get transcription. Streaming adds complexity but the core contract is straightforward.

With TTS, the API receives text and returns audio. That sounds symmetric but it’s not. The output has parameters that don’t exist on the input side. Voice selection, speed, pitch, emotion, SSML tags, sample rate, bitrate, output format. Each one multiplies the design surface and every combination needs to work reliably.

The streaming path is also more complex on the TTS side. With STT, you stream audio in and get partial transcripts back. The contract is append-only. With TTS, you send text and get audio chunks back. But those chunks have timing information - speech marks that tell you which word is being spoken at which millisecond. That data is essential for captions, highlighting, and synchronisation, but it means the API needs to return two parallel streams or a multiplexed one.

SSML is both powerful and painful

Speech Synthesis Markup Language has been around for years and it shows. It works. You can control emphasis, pacing, pitch, pronunciation, and silence with XML tags. But it’s verbose and easy to get wrong.

The trend in newer APIs is toward natural language control instead of XML markup. “Speak this sentence with excitement” instead of wrapping it in <prosody pitch="high"> tags. I think that’s the right direction but SSML still has more granular control for use cases where you need it. Supporting both means maintaining two code paths.

Speech marks are the killer feature nobody talks about

Every TTS API returns word-level timestamps if you ask for them. They’re called speech marks and they tell you exactly when each word starts and ends in the generated audio. This data is incredibly useful - captions, karaoke-style highlighting, aligning audio with video cuts - but most developers I’ve talked to didn’t know it existed.

It’s one of those features that isn’t prominent in documentation because it was added as metadata rather than a product surface. But it’s the kind of thing that, once you know about it, changes how you think about building with TTS.

The pricing problem

TTS pricing varies wildly across providers. Per-character, per-second, per-token. With and without tiers. Voice cloning costs extra. Emotion control costs extra. Streaming sometimes costs the same as non-streaming, sometimes more. Comparing options requires a spreadsheet.

A few years ago, the quality gap between premium and budget TTS was wide enough that you paid up or accepted noticeably worse output. That gap has narrowed significantly. The top models are close enough that pricing is now a primary decision factor, which is a shift from the STT market where a couple of providers clearly lead on quality and everyone else competes on price or niche coverage.

The takeaway

TTS API design is harder than STT API design because the output space is larger and more subjective. That makes choosing a provider more nuanced. It also makes building on one more rewarding when the results are good. The craft matters in a way that it doesn’t on the input side, and I find that motivating.