---
title: SDKs for Streaming APIs Are Different
description: REST SDK patterns stop working when the connection never closes. SDKs for streaming APIs need reconnect logic, backpressure, and graceful shutdown built in.
canonical: https://lukeocodes.dev/sdks-for-streaming-apis-are-different
type: article
locale: en_GB
published: 2026-04-16T00:00:00.000Z
modified: 2026-04-16T00:00:00.000Z
authors:
  - Luke Oliff
section: Engineering
tags:
  - Engineering
  - API Design
  - Streaming
feed: https://lukeocodes.dev/rss.xml
search: https://lukeocodes.dev/search.json?q={query}
schema:
  - "@context": https://schema.org
    "@graph":
      - "@type": Organization
        "@id": https://ably.com/#organization
        name: Ably
        url: https://ably.com
      - "@type": WebSite
        "@id": https://lukeocodes.dev/#website
        url: https://lukeocodes.dev
        name: Luke Oliff
        alternateName: lukeocodes
        inLanguage: en
        publisher:
          "@id": https://lukeocodes.dev/authors/luke#person
        description: Personal blog and writing by Luke Oliff — developer relations, voice AI, developer experience.
        potentialAction:
          - "@type": SearchAction
            target:
              "@type": EntryPoint
              urlTemplate: https://lukeocodes.dev/blog/search?q={search_term_string}
            query-input: required name=search_term_string
      - "@type": Person
        "@id": https://lukeocodes.dev/authors/luke#person
        name: Luke Oliff
        url: https://lukeocodes.dev/authors/luke
        description: Staff engineer and Staff Tech Lead for Developer Experience at Ably. Writing about developer relations, AI, and the voice industry, from the UK.
        mainEntityOfPage: https://lukeocodes.dev/authors/luke
        givenName: Luke
        familyName: Oliff
        image: https://lukeocodes.dev/authors/luke.jpg
        jobTitle: Staff Tech Lead, Developer Experience
        worksFor:
          "@id": https://ably.com/#organization
        sameAs:
          - https://www.linkedin.com/in/lukeocodes
          - https://github.com/lukeocodes
          - https://bsky.app/profile/lukeocodes.dev
          - https://x.com/lukeocodes
          - https://hashnode.com/@lukeocodes
          - https://dev.to/lukeocodes
        knowsAbout:
          - Developer Relations
          - Developer Experience
          - Voice AI
          - Text-to-Speech
          - Answer Engine Optimisation
          - Search Engine Optimisation
          - Open Source
          - Developer Tooling
          - API Design
          - Speech-to-Text
          - Voice Agents
          - Community Building
        knowsLanguage:
          - English
        nationality: British
        alumniOf:
          - "@type": Organization
            name: The Open University
  - "@context": https://schema.org
    "@graph":
      - "@type": Organization
        "@id": https://ably.com/#organization
        name: Ably
        url: https://ably.com
      - "@type": Person
        "@id": https://lukeocodes.dev/authors/luke#person
        name: Luke Oliff
        url: https://lukeocodes.dev/authors/luke
        description: Staff engineer and Staff Tech Lead for Developer Experience at Ably. Writing about developer relations, AI, and the voice industry, from the UK.
        mainEntityOfPage: https://lukeocodes.dev/authors/luke
        givenName: Luke
        familyName: Oliff
        image: https://lukeocodes.dev/authors/luke.jpg
        jobTitle: Staff Tech Lead, Developer Experience
        worksFor:
          "@id": https://ably.com/#organization
        sameAs:
          - https://www.linkedin.com/in/lukeocodes
          - https://github.com/lukeocodes
          - https://bsky.app/profile/lukeocodes.dev
          - https://x.com/lukeocodes
          - https://hashnode.com/@lukeocodes
          - https://dev.to/lukeocodes
        knowsAbout:
          - Developer Relations
          - Developer Experience
          - Voice AI
          - Text-to-Speech
          - Answer Engine Optimisation
          - Search Engine Optimisation
          - Open Source
          - Developer Tooling
          - API Design
          - Speech-to-Text
          - Voice Agents
          - Community Building
        knowsLanguage:
          - English
        nationality: British
        alumniOf:
          - "@type": Organization
            name: The Open University
      - "@type": BlogPosting
        "@id": https://lukeocodes.dev/sdks-for-streaming-apis-are-different#article
        headline: SDKs for Streaming APIs Are Different
        alternativeHeadline: SDKs for Streaming APIs Are Different
        description: REST SDK patterns stop working when the connection never closes. SDKs for streaming APIs need reconnect logic, backpressure, and graceful shutdown built in.
        datePublished: 2026-04-16T00:00:00.000Z
        dateModified: 2026-04-16T00:00:00.000Z
        url: https://lukeocodes.dev/sdks-for-streaming-apis-are-different
        mainEntityOfPage: https://lukeocodes.dev/sdks-for-streaming-apis-are-different
        inLanguage: en
        author:
          - "@id": https://lukeocodes.dev/authors/luke#person
        isPartOf:
          "@id": https://lukeocodes.dev/#website
        keywords: Engineering, API Design, Streaming
        articleSection: Engineering
        about:
          - "@type": Thing
            name: API Design
        abstract: "Throwback Thursday to when I first realised that everything I knew about SDK design was built on an assumption I had never questioned: every API call ends."
        speakable:
          "@type": SpeakableSpecification
          cssSelector:
            - .post-tldr
            - h1
        image: https://lukeocodes.dev/blog/sdks-for-streaming-apis-are-different/hero.png
        wordCount: 1630
        timeRequired: PT9M
        publisher:
          "@id": https://ably.com/#organization
      - "@type": BreadcrumbList
        itemListElement:
          - "@type": ListItem
            position: 1
            name: Home
            item: https://lukeocodes.dev
          - "@type": ListItem
            position: 2
            name: Writing
            item: https://lukeocodes.dev/blog
          - "@type": ListItem
            position: 3
            name: SDKs for Streaming APIs Are Different
            item: https://lukeocodes.dev/sdks-for-streaming-apis-are-different
---

[Luke Oliff.](/)

- [Blog](/blog)
- [CV](/cv)
- [Uses](/uses)
- [User manual](/user-manual)
- [Living with ADHD](/living-and-working-with-adhd)
- [RSS](/rss.xml)

[Luke Oliff · Writing](/blog)

# SDKs for Streaming APIs Are Different

Apr 2026 · [Engineering](/blog/category/engineering) · 9 min read · [Luke Oliff](/authors/luke)

Throwback Thursday to when I first realised that everything I knew about SDK design was built on an assumption I had never questioned: every API call ends.

REST APIs taught us a pattern that works beautifully for CRUD. You build an HTTP client, wrap it in a function call, handle the response or the error, and move on. The connection opens, data travels, the connection closes, and that is the whole transaction.

Then I started building SDKs for streaming audio APIs.

The connection does not close. It stays open for minutes or hours. Audio arrives in chunks, not responses. Messages flow in both directions simultaneously. And every assumption the REST SDK pattern made about lifecycle, error handling, and resource management turns into a question you have to answer from scratch.

## How REST SDKs handle API calls

A REST SDK is simple because HTTP is simple. The client opens a connection, sends a request, waits for the response, and parses it. Error handling is a status code. Retry is a for loop with a sleep.

```python
class RestClient:
    def __init__(self, base_url, api_key):
        self.base_url = base_url
        self.headers = {"Authorization": f"Bearer {api_key}"}

    def transcribe(self, audio_file):
        resp = requests.post(
            f"{self.base_url}/v1/transcribe",
            headers=self.headers,
            files={"audio": audio_file},
        )
        resp.raise_for_status()
        return resp.json()
```

This works because HTTP gives you a natural transaction boundary. The request starts, the response ends, and between them you have a clean unit of work. If the network drops mid-request, you retry the whole thing. If the server returns a 500, you retry. If the client crashes, no state is lost because the state was in that one request.

None of this survives contact with a streaming API.

## What changes when your API streams audio

A streaming audio connection is a persistent WebSocket. Audio chunks flow from client to server, and transcription results flow from server to client, all on the same connection. The connection can last for the entire duration of a conversation, which could be hours.

This changes everything about the SDK design.

**No natural transaction boundary.** Audio is continuous. You cannot retry "the last second" of audio the way you retry an HTTP request. The server has already received and processed some of it. The client has already sent it and moved on. Reconnection is not about replaying a request, it is about resuming from a known state.

**Two-way data flow.** The SDK is not just sending data, it is receiving data at the same time. Messages arrive asynchronously. The SDK needs to dispatch them to the right handler without blocking the send path.

**The connection is stateful.** The server holds session state, audio context, and model state on your behalf. If the connection drops, that state may be lost. The SDK needs to decide whether to reconnect, resend context, or start fresh.

**No natural error surface.** A dropped WebSocket is not a 4xx or 5xx. It is a close frame with a code, or a timeout, or a reset. The SDK needs to classify the failure and decide whether to reconnect, report an error, or silently recover.

## Pattern 1: Connection lifecycle as a first-class concept

In a REST SDK, the connection lifecycle is invisible. The HTTP client handles it. You never think about opening or closing connections.

In a streaming SDK, the connection lifecycle is the most important thing the SDK manages. It needs a connect method, a disconnect method, and internal state tracking for every state the connection can be in.

```python
class StreamClient:
    def __init__(self, api_key):
        self.api_key = api_key
        self.ws = None
        self.connected = False
        self.closing = False

    async def connect(self, url):
        self.ws = await websockets.connect(
            url,
            extra_headers={"Authorization": f"Token {self.api_key}"},
        )
        self.connected = True
        self.closing = False

    async def disconnect(self):
        self.closing = True
        if self.ws:
            await self.ws.close()
            self.ws = None
            self.connected = False
```

The state machine is not optional. The SDK has to know whether it is connecting, connected, reconnecting, or closed because every method on the SDK behaves differently depending on the state. Send audio during reconnection? Buffer it. Send audio while closing? Drop it.

## Pattern 2: Reconnection with exponential backoff and jitter

The next thing you learn is that WebSocket connections drop. Not if. When. The server restarts, the network blips, the load balancer cycles. The SDK has to reconnect, and it has to do it in a way that does not make things worse.

Exponential backoff with jitter is the standard approach. Start with a small delay, double it each time, cap it, and add random jitter so reconnecting clients spread out instead of hitting in waves.

```python
async def connect_with_backoff(self, url):
    delay = 1
    max_delay = 30
    while not self.closing:
        try:
            await self.connect(url)
            return
        except ConnectionError:
            jitter = random.uniform(-delay / 2, delay / 2)
            await asyncio.sleep(delay + jitter)
            delay = min(delay * 2, max_delay)
```

The REST equivalent of this is a retry loop around `requests.post()`. But the streaming version is different in a critical way: while the SDK is reconnecting, audio is still arriving from the microphone. The SDK needs to buffer that audio, reconnect, and then send the buffered audio without the caller noticing a gap.

That requirement does not exist in REST SDKs.

## Pattern 3: Backpressure from the streaming end

REST SDKs do not think about backpressure. The client sends a request and waits. The server either responds or does not. There is no scenario where the server sends data faster than the client can consume it because the server sends exactly as much as the client asked for.

Streaming is different. Audio can arrive faster than the client can process it, or the network can be slower than the audio source. The SDK has to handle both directions.

```python
class BoundedSender:
    def __init__(self, max_buffer=65536):
        self.max_buffer = max_buffer
        self.buffer = b""

    async def send(self, ws, chunk):
        self.buffer += chunk
        if len(self.buffer) > self.max_buffer:
            self.buffer = self.buffer[-self.max_buffer:]
            await asyncio.sleep(0.05)
        await ws.send(self.buffer)
        self.buffer = b""
```

The key insight is that backpressure operates differently for STT and TTS. For speech-to-text, dropping a few milliseconds of audio is acceptable. The model catches up. For text-to-speech, dropping a chunk creates an audible glitch that the listener notices. The SDK needs to know which direction it is handling and adjust its backpressure strategy accordingly.

REST SDKs never have to make this distinction.

## Pattern 4: Graceful shutdown with drain and finalise

A REST SDK shuts down by closing the HTTP session. There is nothing to finalise. The server does not care that the client disconnected because the transaction was already complete.

A streaming connection needs an orderly shutdown. The client tells the server the stream is done, waits for the server to process the final audio and return any remaining results, and only then closes the WebSocket.

```python
async def graceful_shutdown(self, timeout=5):
    await self.ws.send({"type": "EndOfStream"})
    try:
        async with asyncio.timeout(timeout):
            async for msg in self.ws:
                self._handle_result(msg)
    except asyncio.TimeoutError:
        pass
    finally:
        await self.ws.close()
        self.connected = False
```

If the SDK just closes the WebSocket without the drain phase, the server might be mid-transcription when the connection drops. The transcript cuts off. The user sees half a sentence and assumes the system is broken. The drain phase prevents that.

## What this means for SDK API design

The differences have real consequences for how the SDK surface looks.

A REST SDK exports functions or methods that map to API endpoints. The user calls `client.transcribe(audio)` and gets a result back. Simple.

A streaming SDK exports a stateful object that the user instantiates, connects, and then interacts with through events and callbacks. The user calls `client.connect()`, registers event handlers, streams audio, and handles results as they arrive. It is a fundamentally different programming model.

```python
# REST style
client = RestClient(api_key)
result = client.transcribe("audio.wav")
print(result["transcript"])

# Streaming style
client = StreamClient(api_key)
client.on("transcript", lambda t: print(t))
await client.connect()
await client.stream_mic()  # runs until interrupted
await client.disconnect()
```

The streaming version requires the user to think about connection state, event handling, and lifecycle management. The REST version does not. A good streaming SDK hides as much of that complexity as possible, but some of it is inherent. The connection is stateful. The user has to deal with that.

## FAQ

### When do I need a streaming SDK instead of a REST client?

If your use case involves real-time audio, live transcription, or bidirectional communication, you need a streaming SDK. If you are batch-processing pre-recorded files, a REST client is fine. The dividing line is whether the connection outlives a single request-response cycle.

### Can I build streaming patterns on top of a REST client?

Technically yes, but you will be fighting the abstraction. Polling an endpoint every 100 milliseconds is not streaming, it is polling with extra steps. A WebSocket connection built on a persistent TCP socket is fundamentally different from repeated HTTP requests, and the SDK should reflect that difference.

### What is the hardest part of shipping a streaming SDK?

Connection state management. REST SDKs have one state (ready). Streaming SDKs have connecting, connected, reconnecting, draining, and closed. Every method on the SDK has different behaviour in each state. Getting the state machine wrong causes bugs that are hard to reproduce because they depend on network timing.

### Should I use a streaming SDK in a serverless function?

Probably not. Serverless runtimes have short timeouts and cold starts that do not play well with persistent connections. A REST client or a job-based pattern is better for serverless. Streaming SDKs belong in long-running processes like desktop apps, mobile clients, and backend services.

### Do all streaming APIs use WebSockets?

Most use WebSocket or WebRTC. Some use HTTP/2 server-sent events for one-way streaming. The SDK patterns apply regardless of the underlying transport. The need for connection lifecycle management, reconnection, backpressure, and graceful shutdown is the same whether the transport is WebSocket, SSE, or a custom TCP protocol.

[← Newer: Friday fun: live speech to text from the terminal](/friday-fun-live-speech-to-text-terminal)

[All posts](/blog) →

## Search this site

`GET https://lukeocodes.dev/search.json?q=<words>` returns matching posts as JSON, each with its page URL and its markdown URL. Add `&limit=<1-50>` for more than 10. How it works: [/blog/search](https://lukeocodes.dev/blog/search.md).

```sh
curl "https://lukeocodes.dev/search.json?q=developer+relations"
```

---

© 2026 Luke Oliff. Posts live at the root of this site.

- [GitHub](https://github.com/lukeocodes)
- [LinkedIn](https://www.linkedin.com/in/lukeocodes)
- [X](https://x.com/lukeocodes)
- [Bluesky](https://bsky.app/profile/lukeocodes.dev)
- [Email](mailto:luke@lukeoliff.com)
- [RSS](/rss.xml)

## Cookies

This site sets one analytics cookie, from Google Analytics, and only in a browser that accepts the cookie banner. A client that does not run scripts sets no cookies, so there is nothing to accept or decline here.

[Cookie policy](/cookies)
