Building an AI Voice Cloning Web App with Next.js and Speechify

You can build a working voice cloning web app in Next.js with two server routes and one API call: POST /api/clone takes an audio sample plus consent and returns a voice_id, and POST /api/speak synthesizes text with that id. The clone is usable on the very next request. No GPU, no model files, no fine-tuning queue.

Most people build this the hard way. They pull down an open-source model, rent a GPU, lose an afternoon to CUDA versions, and then find that inference latency makes the whole thing useless for anything a user waits on. The Speechify API does the cloning for you over HTTP, so the interesting part of the app becomes the part you actually care about: the UI, and keeping your key off the client.

This walkthrough builds that app. The finished code is in speechify-ai-demos/next-voice-cloning-app, and you’ll need a Speechify key on a plan that includes cloning. Grab one here if you don’t have it yet.

What voice cloning does in one API call

You POST an audio sample of someone speaking. The API returns a voice_id. Every future synthesis call with that id produces audio in the cloned voice, and the clone persists on your account until you delete it. There is no separate “is the model ready” poll, which is the thing that surprised me most the first time I ran it. The clone is live on the next request.

The sample does the heavy lifting. Ten to thirty seconds of clean, single-speaker speech, no music, minimal background noise. Phone audio works but gives you a phone-quality clone, so this is the one place where the input matters more than anything you do in code. I cover the create, use, delete lifecycle in more depth in the dynamic video narration post if you want the REST-level view first.

The setup

Start an app and add the SDK:

npx create-next-app@latest voice-clone-app --typescript --app
cd voice-clone-app
npm install @speechify/api

Put your key in .env.local. Next.js loads it into the server runtime, and because we only read it inside route handlers, it stays off the client:

SPEECHIFY_API_KEY=your_key_here

That is the whole reason the clone runs in a route handler instead of the browser. The key lives on the server, the browser only ever talks to your own API. Here is the shape of it.

Voice cloning web app

The API key never leaves the server

The browser talks to your own Next.js route handler. That handler holds the Speechify key and makes the upstream call, so the key is never in client JavaScript.

1. Browser Upload + consent User picks a sample and fills in the consent name and email. POST /api/clone
2. same-origin
3. Next.js route handler Holds the key Reads SPEECHIFY_API_KEY from the server environment and calls the SDK. client.voices.create()
4. Bearer key
5. Speechify API Clones the voice Returns a voice_id the app uses for later synthesis. POST /v1/voices

The voice_id travels back to the browser, but the key does not. Every later synthesis call goes back through /api/speak the same way.

Capturing the voice sample in the browser

Two ways in. The simplest is a file input, so the user hands you a .wav or .mp3 they already have:

"use client";
import { useState } from "react";

export function SampleUpload({ onFile }: { onFile: (f: File) => void }) {
  return (
    <input
      type="file"
      accept="audio/*"
      onChange={(e) => {
        const file = e.target.files?.[0];
        if (file) onFile(file);
      }}
    />
  );
}

If you’d rather record in-app, MediaRecorder captures from the mic and gives you a Blob you can treat exactly like an uploaded file:

const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
const recorder = new MediaRecorder(stream);
const chunks: Blob[] = [];

recorder.ondataavailable = (e) => chunks.push(e.data);
recorder.onstop = () => {
  const sample = new File(chunks, "sample.webm", { type: "audio/webm" });
  // send `sample` to your clone route
};

recorder.start();
// ...later
recorder.stop();

Either way you end up with a File. Here is the app on first load, before any of that happens:

The whole app is two steps: clone in step one, synthesize in step two. Step two stays disabled until a clone exists.

Consent is required, not decorative

The clone call needs a consent field: a JSON string with the name and email of the person whose voice you’re cloning. The API rejects the request without it. Don’t stub this out and forget to swap it back. Voice cloning is the most obvious abuse vector in the whole API, and the consent record is the thing that separates legitimate use from someone scraping a podcast.

For local testing, use the bundled spacewalk.wav from the cookbook. It is public-domain NASA audio, so consent is unambiguous. For anything you ship, collect consent from a real form filled in by the real person.

Sample loaded, consent filled, ready to clone. This is the exact state the demo’s Playwright run drives before clicking Clone.

The clone route

The server handler receives the upload as FormData, passes it to client.voices.create, and returns the new id. The key is read from the server environment and never serialized back:

// app/api/clone/route.ts
import { NextResponse } from "next/server";
import { SpeechifyClient, SpeechifyError } from "@speechify/api";

export const runtime = "nodejs";

const client = new SpeechifyClient({ token: process.env.SPEECHIFY_API_KEY });

export async function POST(req: Request) {
  const form = await req.formData();
  const sample = form.get("sample");
  const fullName = form.get("fullName");
  const email = form.get("email");

  if (!(sample instanceof File) || typeof fullName !== "string" || typeof email !== "string") {
    return NextResponse.json({ error: "sample, fullName and email are required" }, { status: 400 });
  }

  try {
    const voice = await client.voices.create({
      name: `clone-${Date.now()}`,
      gender: "male",
      sample,
      consent: JSON.stringify({ fullName, email }),
    });
    return NextResponse.json({ voiceId: voice.id, displayName: voice.display_name });
  } catch (err) {
    if (err instanceof SpeechifyError && err.statusCode === 402) {
      return NextResponse.json(
        { error: "Voice cloning isn't included in your current plan." },
        { status: 402 },
      );
    }
    throw err;
  }
}

Two things earn their place here. The SDK’s sample field accepts a web File directly, so a browser upload goes straight through without you writing it to a temp file first. And that 402 branch is real. Voice cloning is plan-gated, and if your plan doesn’t include it the API returns voice_cloning_not_included. I hit that exact 402 on two of my own keys while building this, so branch on err.statusCode === 402 and show the user something useful instead of a stack trace.

client.voices.create returns { id, display_name, type, ... }. The id is what every later synthesis call needs.

The clone landed and the status line shows the returned voice_id. Step two is now enabled, so the same id can be used to synthesize.

Synthesizing with the cloned voice

Once the client has a voiceId, synthesis is a second route, and it’s the same call you’d make for any catalogue voice. The cloned id just slots in where george would go:

// app/api/speak/route.ts
import { NextResponse } from "next/server";
import { SpeechifyClient } from "@speechify/api";

export const runtime = "nodejs";

const client = new SpeechifyClient({ token: process.env.SPEECHIFY_API_KEY });

export async function POST(req: Request) {
  const { text, voiceId } = await req.json();

  if (typeof text !== "string" || typeof voiceId !== "string") {
    return NextResponse.json({ error: "text and voiceId are required" }, { status: 400 });
  }

  const speech = await client.audio.speech({
    input: text,
    voice_id: voiceId,
    audio_format: "mp3",
    model: "simba-english",
  });

  return NextResponse.json({ audio: speech.audio_data }); // base64 mp3
}

The response comes back with audio_data as base64. On the client, turn it into a playable URL and hand it to an <audio> element:

async function speak(text: string, voiceId: string) {
  const res = await fetch("/api/speak", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ text, voiceId }),
  });
  const { audio } = await res.json();

  const blob = await (await fetch(`data:audio/mpeg;base64,${audio}`)).blob();
  const url = URL.createObjectURL(blob);
  new Audio(url).play();
}

That is the full loop. Upload or record in the browser, clone on the server, synthesize on the server, play back in the browser. If you’ve read the Node.js TTS post, the synthesis half is a call you’ve already made, pointed at your clone.

A real clone, synthesized and playing back. The audio player shows a three-second clip generated from the cloned voice. This screenshot is the end of the demo’s automated Playwright run against the live API.

Cleaning up cloned voices

Clones persist until you delete them, so a multi-tenant app needs a cleanup story. The delete call takes voice_id:

// app/api/voice/route.ts
await client.voices.delete({ voice_id: voiceId });

Delete a voice when a user removes it, and store and reuse the voiceId for returning users rather than re-cloning every session. Re-cloning costs an upload and adds latency for no benefit if the voice already exists.

Where to take it next

The demo runs on simba-english, which pairs with cloned voices. For catalogue voices, simba-3.2 is the current recommendation, though it needs a voice that lists it (the *_32 voices) rather than george. Streaming, speech marks for captions, and SSML all work with a cloned voice_id exactly as they do for any other voice.

The full runnable app is in speechify-ai-demos/next-voice-cloning-app, the REST recipe is in the cookbook, and the parameter surface is in the Speechify docs .

FAQ

Does the API key ever reach the browser in this setup?

No. Both the clone and the synthesis run in route handlers under app/api/, which only execute on the server. The browser talks to your own /api/clone and /api/speak endpoints and never sees SPEECHIFY_API_KEY. next.config.ts also marks @speechify/api as a server-external package, so the SDK is never bundled into client JavaScript.

How do I pass a browser-uploaded file to the SDK?

The SDK’s sample field accepts a web File directly, which is what a multipart FormData upload gives you in a Next.js route handler. Read it with form.get("sample"), confirm it’s a File, and pass it straight into client.voices.create. There’s no need to write the upload to disk or convert it to a stream first.

What happens if voice cloning isn’t on my plan?

The API returns 402 voice_cloning_not_included. The clone route catches it by checking err instanceof SpeechifyError && err.statusCode === 402 and returns a clean 402 to the client. Cloning is a billing-plan entitlement, not something a key’s scopes carry, so a “full access” key on a plan without cloning still gets the 402. Branch on the status code and point the user at pricing.

How long should the voice sample be?

Ten to thirty seconds of clean, single-speaker speech. Under ten seconds and the clone gets unreliable on prosody. Over thirty adds upload time without improving quality. If you record with MediaRecorder, give the user a fixed paragraph to read so you get consistent length and content every time.

Which model should I use to synthesize with a cloned voice?

Use simba-english for an English clone, which is what the demo does. A cloned voice lists the models it supports in the create response, and English clones support simba-english and simba-multilingual. The simba-3.2 model pairs with its own catalogue voices rather than personal clones, so match the model to what the voice actually lists.