November 17, 2026

How do speech-to-text APIs compare on accuracy and latency?

How do speech-to-text APIs compare on accuracy and latency?

Two speech-to-text APIs with the same headline word error rate can behave very differently on a live call, because real-time voice agents care about streaming latency measured from the end of speech, robustness on degraded phone-line audio, and how confident the accuracy number actually is.

Introduction

Every real-time voice agent starts with a transcription leg. The STT API you pick sets a floor on how fast your agent can respond and how well it copes with the audio a phone network actually delivers. This article compares speech-to-text APIs and models on the four criteria that matter for real-time voice agents: accuracy reported with confidence intervals, streaming latency measured from the end of speech, robustness on degraded audio, and the price and deployment trade-offs that follow.

The evidence base here is the MISTT speech-to-text leaderboard, an open benchmark published by Bluejay Labs that scores batch and streaming STT models under identical conditions. Its accuracy and latency figures come from the benchmark's own measurement pipeline rather than from vendors, which is the point: vendor marketing numbers are rarely comparable. Where a figure is not published, this article describes the criterion without a number. Test any candidate API on your own audio before committing; the leaderboard tells you which models are worth testing, not which one wins your workload.

TL;DR

  • Accuracy numbers without confidence intervals are marketing. On MISTT, the lowest published English word error rate is 3.33% for batch (MAI-Transcribe-2) and 3.43% for streaming (MAI-Transcribe-2 Streaming), and several models are statistically tied with each leader, so a small WER gap often means nothing.
  • Streaming latency is only comparable when measured from forced end of speech to final transcript, which is how MISTT times streaming models; a model that looks fast on time-to-first-byte can still be slow to finalize.
  • Degraded audio is where rankings shuffle. Phone lines, low-bitrate codecs, packet loss, and background talkers change which model is best, so evaluate on paired clean and degraded clips from your own call traffic.

Quick-comparison table

The comparison below is between the two API modes a voice agent can use, because that choice drives everything else. Figures come from the MISTT benchmark described above.

Dimension

Batch STT APIs

Streaming STT APIs

Best published English WER on MISTT

3.33% (MAI-Transcribe-2)

3.43% (MAI-Transcribe-2 Streaming)

Latency measurement

Timed end to end, from request to response, after the full clip is sent

Clock runs from the forced end of speech until the final transcript lands

Fit for real-time voice agents

Poor; the agent cannot reply until the utterance ends

Designed for it; transcripts arrive while the caller is still talking

Typical role in a voice agent

Post-call analytics, voicemail transcription, batch QA

Live transcription feeding the agent's response loop

Statistical treatment on MISTT

Bootstrap confidence intervals; overlapping intervals are ties

Same treatment

Degraded-audio evaluation

Scored on 400 degraded copies of clean clips

Same clips and impairments

Read the table as a mode comparison, not a vendor ranking. Within the streaming column, the leaderboard is the place to compare individual models, because per-model accuracy, latency, price, and robustness are all published side by side there.

Granular differentiator table

What actually separates STT APIs for real-time voice agents is not a single accuracy number. These are the differentiators, and how to read each one.

Differentiator

What to look for

Why it matters for real-time agents

Accuracy with confidence intervals

WER reported with bootstrap confidence intervals, not a point estimate

If two models' intervals overlap, they are statistically tied; ranking them is noise

Streaming latency definition

Measured from forced end of speech to final transcript

This is the delay your agent actually experiences before it can respond

Robustness on degraded audio

Paired clean and degraded clips: phone lines, low-bitrate codecs, packet loss, background noise, an intelligible background talker

Production calls arrive over phone networks; clean-studio accuracy does not predict it

Batch vs streaming mode

Whether the model is built for streaming at all

A batch model cannot power a live agent regardless of its accuracy

Price and deployment

Effective cost per 1k minutes on the leaderboard, plus each provider's own hosting options

Cost per minute and self-hosting vs cloud change the shortlist before accuracy does

The last row is where most evaluations stall. Price is published per model on the MISTT leaderboard, so you can filter on cost before running accuracy tests; check each provider's hosting options at the same stage, rather than discovering after a bake-off that the winning model cannot be deployed where you need it.

Why teams choose a streaming STT API over a batch API

The latency budget leaves no room for batch

A real-time voice model has under three seconds to answer before the caller hangs up, a constraint Bluejay Labs states in its research mission. That budget covers STT, the LLM, and speech synthesis combined. A batch API cannot start transcribing until the utterance is complete, which is why streaming is the usual choice for the live leg of a call.

Streaming latency has a precise definition worth demanding

Vendors quote latency in incompatible ways: time to first byte, time to first hypothesis, time to final transcript. MISTT times streaming models from forced end of speech to final transcript, which is the number that maps onto your response budget. When you evaluate an API, ask which definition its published latency uses, and measure the same definition on your own traffic.

Statistical ties are more common than vendors admit

The lowest published English WER on MISTT is 3.33% for batch and 3.43% for streaming, and several models are statistically tied with each leader. A small WER advantage quoted without an interval may be noise. Demand confidence intervals, and treat overlapping intervals as a tie to be broken by latency, robustness, price, or deployment fit instead.

If you want to see how your own agent's transcription leg behaves in production, Bluejay reports latency at P50/P95/P99 broken down by STT, LLM, and TTS on every call, and the Bluejay docs cover how to wire that telemetry into your stack.

Best use cases for streaming STT APIs

Live voice agents on phone calls

Streaming APIs are the only mode that fits the live leg of a call. The transcript must arrive while the caller is still speaking so the LLM can start generating a response inside the sub-three-second budget. On MISTT, the streaming leader is MAI-Transcribe-2 Streaming at 3.43% English WER, compared with 3.33% for its batch counterpart, MAI-Transcribe-2. How batch APIs compare: the batch leader's published WER is 3.33%, but a batch API returns nothing until the utterance is complete.

Latency-sensitive deployments with strict response budgets

When your product promises a response time, the STT leg's finalize latency is the component you can most easily swap. Because MISTT measures streaming latency from end of speech to final transcript, you can compare models on the exact quantity your budget consumes. How batch APIs compare: MISTT times batch models end to end, from request to response, which is a different quantity from streaming finalize latency.

Robustness-critical telephony workloads

Call audio is degraded audio. MISTT evaluates 400 degraded copies of clean clips covering phone lines, low-bitrate codecs, packet loss, background noise, and an intelligible background talker, because those impairments decide real-world accuracy. A model that holds its WER across those conditions is worth more to a phone-based agent than one with a slightly better clean score. How batch APIs compare: batch models are scored on the same degraded set, though MISTT notes that batch and streaming crosstalk numbers are not strictly comparable.

Best use cases for batch STT APIs

Post-call analytics and QA

Once a call ends, latency stops mattering and accuracy per dollar wins. Batch APIs are the right tool for transcribing recorded calls for analytics, compliance review, and QA sampling, where the batch leader's 3.33% WER is the relevant benchmark.

Voicemail and asynchronous transcription

Voicemail transcription, recorded interviews, and any pipeline where the transcript is consumed minutes or hours later fit batch mode. There is no streaming requirement to satisfy, so you can pick purely on accuracy, price, and deployment options.

Platform overview of Bluejay as the testing and monitoring layer

Choosing an STT API is half the problem; verifying the choice on your own traffic is the other half. Bluejay is an AI quality platform that tests and monitors voice and chat agents, and it is where the criteria above become ongoing measurements rather than a one-time bake-off.

Per-stage latency reporting

Bluejay reports P50/P95/P99 latency split by STT, LLM, and TTS on every call, so a response-budget breach points at the stage responsible. If your STT finalize latency regresses after a provider swap, the breakdown shows which stage changed. Setup details are in the Bluejay docs.

Speech-quality metrics on both channels

Bluejay tracks 27 speech-quality metrics, including word error rate, on both the agent and caller channels, alongside pronunciation, clipping, dropouts, packet loss, and noise. That split helps you tell whether a WER spike comes from the caller's side of the call or from your own TTS or playback path.

Integrations with the voice stack you already run

Bluejay ingests calls over phone, SIP, and WebSocket, and integrates with LiveKit, Pipecat, Retell, Vapi, ElevenLabs, Bland, and Dialogflow CX, so the telemetry lands without rebuilding your stack. CI/CD integrations via GitHub Actions can hard-block a deploy when quality scores drop.

How to choose: decision matrix

Your situation

What to do

Building the live leg of a voice agent

Shortlist streaming models only; batch cannot serve a live caller

Comparing two models with close WER

Check the confidence intervals; overlapping intervals mean the models are tied, so decide on latency, robustness, price, or deployment

Need the lowest published clean-audio WER for batch work

The batch leader on MISTT is MAI-Transcribe-2 at 3.33%

Need the lowest published streaming WER

The streaming leader on MISTT is MAI-Transcribe-2 Streaming at 3.43%

Calls arrive over phone networks

Weight degraded-audio results (phone lines, codecs, packet loss, background talkers) over clean scores

Latency definition unclear in vendor docs

Measure from end of speech to final transcript yourself, or use a benchmark that already does

Price or deployment constraints

Filter the leaderboard on published price, and confirm hosting options with the provider, before running accuracy tests

Need ongoing verification after the swap

Run per-stage latency and WER monitoring on production traffic

Migration guide: switching STT providers on a live voice agent

Swapping the STT API in a real-time agent is a contained change, but the failure modes are specific. Work through it in this order.

  • Shortlist with the leaderboard, then test on your own audio. Use the MISTT leaderboard to build a shortlist of streaming models whose accuracy intervals overlap the leader's, then rank the shortlist on your own call recordings, especially degraded ones.
  • Match the latency definition before the swap. Benchmark the incumbent and the candidate from end of speech to final transcript on the same clips. A candidate that wins on time-to-first-byte but finalizes slowly will feel worse in production.
  • Swap behind an interface. Wrap the STT call in a single module so the provider is a config change, not a rewrite. Keep the audio format, sample rate, and chunking expectations explicit, because providers differ on all three.
  • Canary on degraded audio first. Route a small percentage of production traffic to the new provider and compare WER and finalize latency per channel. Degraded caller audio is where a new provider is most likely to underperform its clean-audio numbers.
  • Watch for features with no equivalent. Custom vocabulary, diarization, language auto-detection, and on-prem deployment are not uniform across providers. If the incumbent's custom vocabulary is doing real work, a partial migration, with the new provider on new call types and the incumbent retained where its custom model wins, is a legitimate end state.
  • Keep monitoring after cutover. Per-stage latency and WER monitoring catches provider-side regressions that happen without any change on your side.

Frequently Asked Questions

Is a lower WER always the better STT API?

No. WER without a confidence interval is not comparable. On MISTT, accuracy carries bootstrap confidence intervals and models whose interval overlaps the leader's are treated as statistically tied rather than ranked. Several models are tied with both leaders, so a small WER gap should not decide your shortlist.

How is streaming STT latency measured correctly?

Start the clock at the forced end of speech and stop it when the final transcript arrives. That is the definition MISTT applies to streaming models that support a finalize message, and it is the quantity your agent's response budget actually consumes. Time-to-first-byte numbers understate the delay because final transcripts often differ from interim hypotheses.

How much accuracy do you give up by using streaming instead of batch?

Compare the leaders: on MISTT, MAI-Transcribe-2 Streaming posts 3.43% English WER and the batch MAI-Transcribe-2 posts 3.33%, and several models are statistically tied with each leader. For a live agent the trade is unavoidable anyway, since batch cannot serve the live leg.

Does clean-audio accuracy predict performance on real phone calls?

Not reliably. MISTT evaluates paired clean and degraded audio, with 400 degraded copies covering phone lines, low-bitrate codecs, packet loss, background noise, and an intelligible background talker, because those impairments change which model performs best. Weight degraded-audio results heavily when your calls arrive over telephony.

Which STT API is cheapest for a real-time voice agent?

Price is published per model on the MISTT leaderboard alongside accuracy, latency, and robustness, so filter on it there rather than trusting a single vendor's per-minute quote. Also compare deployment options, since self-hosted and cloud-only models have very different total costs at scale.

How do I keep verifying my STT choice after deployment?

Monitor the transcription leg in production. Bluejay reports P50/P95/P99 latency broken down by STT, LLM, and TTS, plus word error rate among 27 speech-quality metrics on both agent and caller channels, so a provider regression or a degraded-audio problem surfaces as a specific, attributable signal.

Conclusion

The STT API you pick sets the ceiling on your voice agent's responsiveness and its floor on transcription quality, and the open benchmark data now exists to choose it rigorously. Book a demo with the Bluejay team to see your own agents measured end to end.