Streaming over WebSocket

Streaming is the main way to use utter. You open one WebSocket, push audio frames as you capture them, and read text back while the speaker is still talking.

Connect

wss://api.utter.cc/v1/stream?tier=turbo&language=hi-IN%2Ben-IN&sample_rate=16000

Authenticate the handshake with the same bearer header used everywhere else:

Authorization: Bearer utt_live_…

Query parameters

Parameter	Required	Default	Values
tier	no	standard	ultra, turbo, standard, economy. See Latency tiers.
language	no	hi-IN+en-IN	hi-IN+en-IN, hi-IN, en-IN. See Languages.
sample_rate	no	16000	The rate of the PCM you are about to send, in Hz.

The + in hi-IN+en-IN must be percent encoded as %2B in a query string, or it is decoded as a space. Most HTTP client libraries do this for you when you pass parameters as a dict rather than concatenating a URL by hand.

sample_rate declares what you are sending. It does not ask the server to resample. Send 44.1 kHz audio while claiming 16 kHz and you get a confident, completely wrong transcript. Resample on your side.

Frame format

Audio frames are binary WebSocket messages containing raw PCM:

• float32, samples in the range -1.0 to 1.0
• little endian
• mono, one channel, interleaving is not supported
• at the rate you declared in sample_rate

No WAV header, no container, no compression. A frame is a whole number of samples, 4 bytes each. Frames of 100 ms to 500 ms of audio work well. Very small frames add per message overhead; very large ones add latency the tier cannot make up for.

At 16 kHz, a 320 ms frame is 16000 * 0.320 = 5120 samples, so 20480 bytes.

Ending the stream

Send the text message "eof" when the audio is finished. It is a text frame, not binary, and it is exactly those three characters with no JSON around it.

The server finishes decoding whatever it holds, emits any remaining final events, sends done, and closes. Closing the socket without sending "eof" ends the session, and audio still in flight may not produce a transcript.

Events

Every message from the server is a JSON text frame with a type field.

ready

Sent once, immediately after the handshake. Wait for it before sending audio.

{
  "type": "ready",
  "session_id": "ses_01JD4Z2Q8W6MFB3T7YKX",
  "tier": "turbo",
  "att_context_size": [70, 1],
  "language": "hi-IN+en-IN",
  "sample_rate": 16000
}

If the tier you asked for is not servable on this deployment, the connection is refused rather than quietly downgraded. See Latency tiers.

partial

The current best guess for the segment being spoken. Partials are unstable by design: the text can change, shrink or be rewritten entirely as more audio arrives. Render them, do not store them.

{
  "type": "partial",
  "text": "invoice share kar diya",
  "t": 1.28
}

t is seconds of audio elapsed since the start of the session.

final

A segment has closed and will not change. This is the text to keep.

{
  "type": "final",
  "text": "Invoice share kar diya hai, meeting 5 baje start hoti hai.",
  "start": 0.0,
  "end": 4.62
}

start and end are seconds from the start of the session. Concatenating every final in order gives you the transcript.

done

Sent once after "eof", just before the server closes the socket.

{
  "type": "done",
  "audio_s": 8.52,
  "billed_s": 8.52,
  "tier": "turbo",
  "segments": 2
}

billed_s is the number of seconds metered for this session. See Pricing and credits.

error

Something went wrong. The socket closes after this.

{
  "type": "error",
  "error": {
    "code": "tier_unavailable",
    "message": "Tier 'ultra' is not available on this deployment.",
    "request_id": "req_01JD4Z2Q8W6M"
  }
}

The error object is the same envelope the REST endpoints use. See Errors.

Order

ready → (partial | final)* → done

with error possible at any point, and terminal. There is no guarantee of a partial before a final: a short segment may close without ever producing one.

Python

Full example, streaming a file at realtime pace so the partials arrive the way they would from a live microphone. Requires pip install websockets soundfile numpy. There is no utter package on PyPI, so this uses the ordinary websockets client.

stream.py

import asyncio, json, os, sys, time
from urllib.parse import urlencode

import numpy as np
import soundfile as sf
import websockets

SAMPLE_RATE = 16000
FRAME_MS = 320


def url(tier: str = "turbo", language: str = "hi-IN+en-IN") -> str:
    query = urlencode(
        {"tier": tier, "language": language, "sample_rate": SAMPLE_RATE}
    )
    return f"wss://api.utter.cc/v1/stream?{query}"


async def send_audio(ws, audio: np.ndarray, realtime: bool = True) -> None:
    """Push float32 mono PCM in FRAME_MS frames, then the eof sentinel."""
    step = SAMPLE_RATE * FRAME_MS // 1000
    for i in range(0, len(audio), step):
        frame = audio[i : i + step]
        # .tobytes() on a slice copies, so only this frame goes on the wire.
        await ws.send(frame.tobytes())
        if realtime:
            await asyncio.sleep(len(frame) / SAMPLE_RATE)
    await ws.send("eof")


async def read_events(ws) -> list[str]:
    finals: list[str] = []
    async for raw in ws:
        event = json.loads(raw)
        kind = event["type"]
        if kind == "partial":
            print(f"  … {event['text']}", end="\r", flush=True)
        elif kind == "final":
            print(" " * 78, end="\r")
            print(event["text"])
            finals.append(event["text"])
        elif kind == "done":
            print(
                f"\n{event['audio_s']}s audio, "
                f"billed {event['billed_s']}s on {event['tier']}"
            )
            break
        elif kind == "error":
            raise RuntimeError(f"{event['error']['code']}: {event['error']['message']}")
    return finals


async def transcribe(path: str) -> str:
    audio, sr = sf.read(path, dtype="float32")
    if sr != SAMPLE_RATE:
        raise SystemExit(
            f"{path} is {sr} Hz. Resample it first, for example:\n"
            f"  ffmpeg -i {path} -ac 1 -ar {SAMPLE_RATE} out.wav"
        )
    if audio.ndim > 1:
        audio = audio.mean(axis=1)

    headers = {"Authorization": f"Bearer {os.environ['UTTER_API_KEY']}"}
    async with websockets.connect(url(), additional_headers=headers) as ws:
        ready = json.loads(await ws.recv())
        if ready["type"] != "ready":
            raise RuntimeError(f"expected ready, got {ready}")
        print(f"tier {ready['tier']}, context {ready['att_context_size']}")

        sender = asyncio.create_task(send_audio(ws, audio))
        finals = await read_events(ws)
        await sender
    return " ".join(finals)


if __name__ == "__main__":
    print(asyncio.run(transcribe(sys.argv[1])))

Two things this example is careful about, both of which are easy to get wrong:

• Send and receive concurrently. Reading only after the last frame is sent turns a streaming API into a batch one, and the partials are wasted.
• Send a copy of the frame. NumPy’s .tobytes() copies. If you reach for a memoryview or a buffer instead, make sure you are not handing over the whole underlying array.

JavaScript (browser)

Live microphone capture. The browser gives you float32 samples already, at the AudioContext rate, so the only real work is resampling to 16 kHz and keeping the frames small.

Caution

Never put a utt_live_ key in browser code. Have your own server open the connection, or hand the browser a short lived connection token. The example below fetches a wss:// URL from your backend and never sees the key.

mic.html

<button id="start">Start</button>
<button id="stop" disabled>Stop</button>
<p id="final"></p>
<p id="partial" style="opacity:.6"></p>

mic.js

const SAMPLE_RATE = 16000;
const FRAME = 5120; // 320 ms at 16 kHz

let ws, ctx, node, stream;

async function start() {
  // Your server holds the API key and returns a ready to use wss:// URL.
  const { url } = await fetch("/api/utter-session", { method: "POST" }).then(
    (r) => r.json(),
  );

  ws = new WebSocket(url);
  ws.binaryType = "arraybuffer";

  const finalEl = document.getElementById("final");
  const partialEl = document.getElementById("partial");

  ws.onmessage = (ev) => {
    const event = JSON.parse(ev.data);
    switch (event.type) {
      case "ready":
        console.log("ready on tier", event.tier, event.att_context_size);
        break;
      case "partial":
        partialEl.textContent = event.text;
        break;
      case "final":
        finalEl.textContent += " " + event.text;
        partialEl.textContent = "";
        break;
      case "done":
        console.log(`billed ${event.billed_s}s`);
        ws.close();
        break;
      case "error":
        console.error(event.error.code, event.error.message);
        ws.close();
        break;
    }
  };

  await new Promise((res) => (ws.onopen = res));

  stream = await navigator.mediaDevices.getUserMedia({ audio: true });
  ctx = new AudioContext({ sampleRate: SAMPLE_RATE });
  await ctx.audioWorklet.addModule("pcm-worklet.js");

  node = new AudioWorkletNode(ctx, "pcm-frames", {
    processorOptions: { frame: FRAME },
  });
  node.port.onmessage = ({ data }) => {
    // data is a fresh Float32Array; .buffer is safe because the worklet
    // allocated it per frame rather than handing over a view into a pool.
    if (ws.readyState === WebSocket.OPEN) ws.send(data.buffer);
  };

  ctx.createMediaStreamSource(stream).connect(node);
  document.getElementById("stop").disabled = false;
}

function stop() {
  ws?.send("eof"); // text frame, the server flushes and sends `done`
  node?.disconnect();
  ctx?.close();
  stream?.getTracks().forEach((t) => t.stop());
  document.getElementById("stop").disabled = true;
}

document.getElementById("start").onclick = start;
document.getElementById("stop").onclick = stop;

The worklet just accumulates 128 sample render quanta into 320 ms frames:

pcm-worklet.js

class PcmFrames extends AudioWorkletProcessor {
  constructor(options) {
    super();
    this.frame = options.processorOptions.frame;
    this.buf = new Float32Array(this.frame);
    this.n = 0;
  }

  process(inputs) {
    const channel = inputs[0]?.[0];
    if (!channel) return true;

    for (let i = 0; i < channel.length; i++) {
      this.buf[this.n++] = channel[i];
      if (this.n === this.frame) {
        // slice() copies. Posting this.buf itself would send the same
        // memory every time and the transcript would be garbage.
        this.port.postMessage(this.buf.slice());
        this.n = 0;
      }
    }
    return true;
  }
}

registerProcessor("pcm-frames", PcmFrames);

Note

new AudioContext({ sampleRate: 16000 }) asks the browser to resample the mic input for you, and Chrome, Firefox and Safari all honour it. If you need to work against a context locked to the hardware rate, resample the float32 samples in the worklet before framing them, and set sample_rate on the connection to whatever you actually send.

Limits

One session	one WebSocket, one language, one tier
Changing tier or language	open a new connection
Silence	still audio, still metered

To transcribe two speakers on separate microphones, open two sessions.