---
title: I Thought I Knew APIs. Then I Met Audio.
description: "REST APIs felt easy, even with HATEOAS. Streaming audio broke that. The destination is HTTP/2 bidirectional streaming: feedback while you still get audio."
canonical: https://lukeocodes.dev/i-thought-i-knew-apis-then-i-met-audio
type: article
locale: en_GB
published: 2026-08-13T00:00:00.000Z
modified: 2026-08-13T00:00:00.000Z
authors:
  - Luke Oliff
section: Career
tags:
  - Career
  - Developer Experience
  - Voice AI
feed: https://lukeocodes.dev/rss.xml
search: https://lukeocodes.dev/search.json?q={query}
schema:
  - "@context": https://schema.org
    "@graph":
      - "@type": WebSite
        "@id": https://lukeocodes.dev/#website
        url: https://lukeocodes.dev
        name: Luke Oliff
        alternateName: lukeocodes
        inLanguage: en
        publisher:
          "@id": https://lukeocodes.dev/authors/luke#person
        description: Personal blog and writing by Luke Oliff — developer relations, voice AI, developer experience.
        potentialAction:
          - "@type": SearchAction
            target:
              "@type": EntryPoint
              urlTemplate: https://lukeocodes.dev/blog/search?q={search_term_string}
            query-input: required name=search_term_string
      - "@type": Person
        "@id": https://lukeocodes.dev/authors/luke#person
        name: Luke Oliff
        url: https://lukeocodes.dev/authors/luke
        description: Engineer and developer relations leader. Writing about developer relations, AI, and the voice industry, from the UK.
        mainEntityOfPage: https://lukeocodes.dev/authors/luke
        givenName: Luke
        familyName: Oliff
        image: https://lukeocodes.dev/authors/luke.jpg
        jobTitle: Engineer and developer relations leader
        sameAs:
          - https://www.linkedin.com/in/lukeocodes
          - https://github.com/lukeocodes
          - https://bsky.app/profile/lukeocodes.dev
          - https://x.com/lukeocodes
          - https://hashnode.com/@lukeocodes
          - https://dev.to/lukeocodes
        knowsAbout:
          - Developer Relations
          - Developer Experience
          - Voice AI
          - Text-to-Speech
          - Answer Engine Optimisation
          - Search Engine Optimisation
          - Open Source
          - Developer Tooling
          - API Design
          - Speech-to-Text
          - Voice Agents
          - Community Building
        knowsLanguage:
          - English
        nationality: British
        alumniOf:
          - "@type": Organization
            name: The Open University
  - "@context": https://schema.org
    "@graph":
      - "@type": Person
        "@id": https://lukeocodes.dev/authors/luke#person
        name: Luke Oliff
        url: https://lukeocodes.dev/authors/luke
        description: Engineer and developer relations leader. Writing about developer relations, AI, and the voice industry, from the UK.
        mainEntityOfPage: https://lukeocodes.dev/authors/luke
        givenName: Luke
        familyName: Oliff
        image: https://lukeocodes.dev/authors/luke.jpg
        jobTitle: Engineer and developer relations leader
        sameAs:
          - https://www.linkedin.com/in/lukeocodes
          - https://github.com/lukeocodes
          - https://bsky.app/profile/lukeocodes.dev
          - https://x.com/lukeocodes
          - https://hashnode.com/@lukeocodes
          - https://dev.to/lukeocodes
        knowsAbout:
          - Developer Relations
          - Developer Experience
          - Voice AI
          - Text-to-Speech
          - Answer Engine Optimisation
          - Search Engine Optimisation
          - Open Source
          - Developer Tooling
          - API Design
          - Speech-to-Text
          - Voice Agents
          - Community Building
        knowsLanguage:
          - English
        nationality: British
        alumniOf:
          - "@type": Organization
            name: The Open University
      - "@type": BlogPosting
        "@id": https://lukeocodes.dev/i-thought-i-knew-apis-then-i-met-audio#article
        headline: I Thought I Knew APIs. Then I Met Audio.
        alternativeHeadline: I Thought I Knew APIs. Then I Met Audio.
        description: "REST APIs felt easy, even with HATEOAS. Streaming audio broke that. The destination is HTTP/2 bidirectional streaming: feedback while you still get audio."
        datePublished: 2026-08-13T00:00:00.000Z
        dateModified: 2026-08-13T00:00:00.000Z
        url: https://lukeocodes.dev/i-thought-i-knew-apis-then-i-met-audio
        mainEntityOfPage: https://lukeocodes.dev/i-thought-i-knew-apis-then-i-met-audio
        inLanguage: en
        author:
          - "@id": https://lukeocodes.dev/authors/luke#person
        isPartOf:
          "@id": https://lukeocodes.dev/#website
        keywords: Career, Developer Experience, Voice AI
        articleSection: Career
        about:
          - "@type": Thing
            name: Developer Experience
          - "@type": Thing
            name: Voice AI
        abstract: "APIs used to feel easy. I mean that literally, not as nostalgia. Send a request, get an answer, and the transaction closed. Even HATEOAS stayed inside that model: the API handed you the URI for the next step and the whole thing still behaved like furniture."
        speakable:
          "@type": SpeakableSpecification
          cssSelector:
            - .post-tldr
            - h1
        image: https://lukeocodes.dev/blog/i-thought-i-knew-apis-then-i-met-audio/hero.png
        wordCount: 1147
        timeRequired: PT6M
        publisher:
          "@id": https://lukeocodes.dev/authors/luke#person
      - "@type": BreadcrumbList
        itemListElement:
          - "@type": ListItem
            position: 1
            name: Home
            item: https://lukeocodes.dev
          - "@type": ListItem
            position: 2
            name: Writing
            item: https://lukeocodes.dev/blog
          - "@type": ListItem
            position: 3
            name: I Thought I Knew APIs. Then I Met Audio.
            item: https://lukeocodes.dev/i-thought-i-knew-apis-then-i-met-audio
---

[Luke Oliff.](/)

- [Blog](/blog)
- [CV](/cv)
- [Uses](/uses)
- [User manual](/user-manual)
- [Living with ADHD](/living-and-working-with-adhd)
- [RSS](/rss.xml)

[Luke Oliff · Writing](/blog)

# I Thought I Knew APIs. Then I Met Audio.

Aug 2026 · [Career](/blog/category/career) · 6 min read · [Luke Oliff](/authors/luke)

APIs used to feel easy. I mean that literally, not as nostalgia. Send a request, get an answer, and the transaction closed. Even HATEOAS stayed inside that model: the API handed you the URI for the next step and the whole thing still behaved like furniture. The state was visible, the errors were HTTP status codes, and the connection was a detail the client library swallowed.

## What streaming audio did to a calm mental model

The wobble came when I met streaming audio during a week at Deepgram. The connection did not end. It stayed open for minutes, audio arriving in chunks while results flowed back the other way at the same time. No 4xx code to catch. When something broke, the socket just dropped, or the stream went quiet. Years of webhooks at Vonage had never once shown me that shape.

That is a different kind of API, and it felt like the ground had moved.

## The state you cannot see

What made it hard was not just the new lifecycle. It was state you can't see. A JSON payload is readable: field names, values, nesting, all right there. An audio file tells you nothing. Sample rate, bit depth, channel count, encoding all live in binary headers that stay invisible until you run a separate tool.

![A webhook JSON payload beside an ffprobe terminal dump, showing the visible versus invisible state contrast](/blog/i-thought-i-knew-apis-then-i-met-audio/visible-vs-invisible.png)

You can open two audio files in a file manager and they look identical. `ffprobe` disagrees. Most voice AI problems are audio problems, not API problems, and I didn't know that until I held real audio files. The format the phone saved was not the format the API wanted, and nothing in the first error message explained any of it.

## The lesson: HTTP/2 bidirectional streaming

The destination, it turns out, is HTTP/2 true bidirectional streaming. Native to the protocol. One TCP connection carries many streams, and each side signals its own end independently, so you can keep receiving after you stop sending. Full-duplex at the stream level. Usually this rides on gRPC: one stream per RPC, length-prefixed framing, and a bidirectional streaming RPC is precisely send-up-while-receiving-down. gRPC metadata, key/value pairs riding the HTTP/2 headers and trailers, flows along mid-call.

![A bidirectional streaming socket with audio and feedback flowing up while audio, partials, and metadata flow down simultaneously](/blog/i-thought-i-knew-apis-then-i-met-audio/duplex-socket.png)

The real-world proof is mundane in the good way. Cloud Speech-to-Text exposes streaming through gRPC only. Kaldi and Nuance style streaming ASR servers push binary speech up and partial hypotheses with word timestamps, confidence, and utterance info back down, all on the same open stream.

Two superpowers fall out of that, and they're the ones I keep thinking about. You can send feedback while you are still receiving audio. And you can receive metadata mid-stream, without closing anything. Not special JSON on a finished response. Live.

gRPC is the usual carrier but not the only one. WebSocket is browser-native and full-duplex at the message level, but it multiplexes nothing and has no streaming RPC semantics. SSE is strictly one-way, server to client. WebTransport over HTTP/3 is the emerging path for full duplex in the browser, because gRPC-web can't do client streaming or bidirectional streaming. So gRPC when you own both ends, WebSocket or WebTransport when the browser is the client.

## Why this is the shape that pulls me

The specific capability that pulls me is a single gRPC stream carrying a discriminated union: audio bytes interleaved with control, metadata, and feedback messages, in both directions. That lets you deliver PCM mixed with metadata while you keep sending more data upstream. Two-way payloads at the same time. That never happened in request/response, and it is exactly the simultaneous conversation full-duplex AI products need.

Let me be honest about framing. I am not building this right now. It is a technology I keep coming back to, the transport I reach for in my head whenever I think about a full-duplex product. There is a difference between shipping it today and knowing the shape it has to take. The shape is HTTP/2 bidirectional streaming.

## Where it lands

A voice agent is a pipeline: ASR into an LLM into TTS. Each stage streams into the next, and the invisible state is where the failures live. The [Inside the Streaming Cascade Powering Voice AI](/inside-the-streaming-cascade-powering-voice-ai) is how those stages stay connected, and the agent pipeline below has its invisible layer highlighted because it is the same invisible state that broke my REST mental model, applied to a full-duplex conversation.

![A voice agent pipeline of ASR, LLM, and TTS with the invisible state layer highlighted](/blog/i-thought-i-knew-apis-then-i-met-audio/agent-pipeline.png)

Years ago the extent of what I knew was APIs that ended. Now I look toward audio and data streaming over one full-duplex connection, where the model keeps talking and metadata keeps coming back mid-stream. I wrote out the architecture case in [Full-Duplex Voice AI Needs a New Architecture](/full-duplex-voice-ai-new-architecture), and this post is the journey side of the same coin. APIs stopped feeling easy. I don't regret that.

### FAQ

#### Why is HTTP/2 bidirectional streaming the right transport for audio?

Because audio is continuous both ways and HTTP/2 streams handle that natively. One TCP connection carries many streams, and each side ends its own stream independently, so you keep receiving after you stop sending. That full-duplex, stream-level behaviour is what real-time audio and its metadata actually need, and it's the shape gRPC formalises.

#### Is gRPC the only way to do this?

No. gRPC is the usual carrier because it maps cleanly onto the stream, but WebSocket is full-duplex at the message level and WebTransport over HTTP/3 is the emerging browser path, since gRPC-web can't do client streaming or bidirectional streaming. Pick gRPC when you own both ends, WebSocket or WebTransport when the browser is the client.

#### Can you really send feedback while still receiving audio?

Yes. In a bidirectional streaming RPC the two directions are independent streams. You can send a control or feedback message upstream while audio or transcription is still flowing down to you, with nothing closed and no round-trip. That simultaneity is the whole point, and it's impossible in request/response.

#### How is this different from a plain WebSocket?

A WebSocket is full-duplex and browser-native, but it's message-level only. No multiplexing, no streaming RPC semantics, no per-stream ordering guarantees. HTTP/2 bidirectional streaming gives you length-prefixed message framing, many concurrent streams, and a defined request/response shape on a single connection, which is why gRPC builds on it.

#### Where does this fit for full-duplex AI products?

In a full-duplex voice product both parties speak at once, so the transport has to carry audio and metadata in both directions simultaneously. HTTP/2 bidirectional streaming is what that requires. It lets the model keep talking while it still receives input, which is exactly the simultaneous two-way payload full-duplex AI is built on.

[← Newer: Friday fun: the chatbot that killed a farmer's crop, then diagnosed itself](/friday-fun-chatbot-farmer-crops)

[All posts](/blog) →

## Search this site

`GET https://lukeocodes.dev/search.json?q=<words>` returns matching posts as JSON, each with its page URL and its markdown URL. Add `&limit=<1-50>` for more than 10. How it works: [/blog/search](https://lukeocodes.dev/blog/search.md).

```sh
curl "https://lukeocodes.dev/search.json?q=developer+relations"
```

---

© 2026 Luke Oliff. Posts live at the root of this site.

- [GitHub](https://github.com/lukeocodes)
- [LinkedIn](https://www.linkedin.com/in/lukeocodes)
- [X](https://x.com/lukeocodes)
- [Bluesky](https://bsky.app/profile/lukeocodes.dev)
- [Email](mailto:luke@lukeoliff.com)
- [RSS](/rss.xml)

## Cookies

This site sets one analytics cookie, from Google Analytics, and only in a browser that accepts the cookie banner. A client that does not run scripts sets no cookies, so there is nothing to accept or decline here.

[Cookie policy](/cookies)
