Realtime Transcription: Build Captions That Handle Partial Results

Start with what the reader sees

A live caption is provisional. Words can arrive over several events, and a completed transcript for one turn may arrive after a later turn. A caption interface needs to distinguish text that is still changing from text that the service has completed.

Choose live transcription for audio arriving from a microphone or stream. For a finished recording, the file transcription workflow is usually the clearer fit. Streaming text from a completed upload does not make it a live audio session.

Commit turns for the selected model

The current OpenAI realtime transcription guide recommends gpt-live-transcribe. For this model, omit audio.input.turn_detection or set it to null. Detect the boundary in your client and commit the buffered audio as described in the guide; don't copy a VAD configuration from a different model without checking compatibility.

The guide uses 24 kHz PCM in its server WebSocket example. Browser audio has a different connection path, so follow the documented transport rather than assuming every client sends the same events.

Reconcile events by turn

Associate incoming deltas and completion events with their item_id. Do not sort completed turns by the time the completion event arrives: completion ordering across different turns isn't guaranteed. Keep a separate order from the conversation items or your own commit sequence.

The following TypeScript is a local state helper, not an API handler. It assumes the caller has already established the correct turn order and passed decoded text. It doesn't handle reconnects or persistence.

type Caption = { text: string; final: boolean };
const captions = new Map<string, Caption>();

function addDelta(id: string, delta: string) {
  const current = captions.get(id);
  if (current?.final) return;
  captions.set(id, { text: (current?.text ?? "") + delta, final: false });
}
function complete(id: string, text: string) {
  captions.set(id, { text, final: true });
}

Design for corrections and gaps

Mark partial captions visually and update them when the final text arrives. If a completion never arrives after a disconnect, show that the turn is incomplete instead of silently presenting it as confirmed.

gpt-live-transcribe does not return word-level timestamps, speaker labels, or transcription confidence scores. If those fields are necessary, check the current compatibility guide and choose an appropriate alternative or build a separate fallback.

Tune latency with representative audio

The transcription delay setting trades earlier partial text against more context for the model. Begin with a setting that fits the experience, then test the same recordings at other supported settings. Exact milliseconds can vary by model configuration.

Include accents, noisy rooms, telephony, names, email addresses, dates, currency, and code-switching. Track empty or truncated turns as well as recognition errors. An apparently small wording mistake can be costly when it changes a delivery address.

Check retention before launch

Decide whether captions or recordings need to be stored at all. If they do, document who can access them, how long they are retained, and how the user can stop the session. Test with synthetic data while your access and retention design is still changing.

Work through an out-of-order example

Suppose the interface has committed two turns: the first says a street name, the second a house number. Keep their display order from the input sequence. If the second turn completes first, update the second row in place; don't move it above the first. When the first completion arrives, replace that row's provisional text with the final transcript.

A reconnect complicates this picture. The application should retain enough state to know which turns it has already displayed, and should label a turn as incomplete if it cannot confirm its final result. Whether an event can be replayed depends on the connection and service workflow. Consult the event documentation for your chosen session rather than assuming all history will be resent.

The map example above deliberately omits ordering and persistence. It also accepts the final text supplied by its caller; that caller needs to validate the event type and route each event to the right session and item. Never merge data from two users just because an item identifier looks similar.

Create a release test that a teammate can repeat

Prepare a short synthetic recording with two turns, a proper name, and a number. Write down the expected final text and the point at which the user pauses. Run it through the complete path and inspect the provisional captions, final captions, display order, and stop control. Add a second recording with room noise or a telephone-quality microphone.

Then simulate a delayed completion and a disconnected client in your test environment. The expected behavior should describe what the reader sees, not only what the logs contain. Keep a record of the selected model, delay setting, transport, and test date so a later model or session change can be compared fairly.

Do not claim that the sample produces speaker labels or timestamps. The current transcription guide says those outputs aren't returned by gpt-live-transcribe; verify the selected alternative if your product depends on them.

Keep the user-facing caption order independent of the arrival order of completion events. During a review, check not only the text but also whether the interface clearly identifies an incomplete turn. Retain the exact event traces needed for a synthetic test under your own retention rules, and avoid assuming that an API connection will replay every missed event after a reconnect.