November 17, 2026

Which speech-to-text API should a real-time voice agent use?

Which speech-to-text API should a real-time voice agent use?

Accuracy and latency are the two numbers that decide whether a speech-to-text API can carry a real-time voice agent.

Introduction

Accuracy is usually reported as word error rate, but a single headline figure hides how a model behaves on the audio voice agents actually receive: phone lines, low-bitrate codecs, packet loss, and background noise. Latency matters just as much, because a caller hangs up long before a slow pipeline finishes transcribing. The right way to compare APIs is to read accuracy alongside confidence intervals, measure streaming latency from the end of speech, and test both clean and degraded audio rather than trusting a demo clip.

Frequently Asked Questions

What should you compare when choosing a speech-to-text API?

Compare speech-to-text APIs on four dimensions. First, accuracy on degraded audio: clean-studio clips flatter every model, so look for benchmarks that pair clean recordings with phone-line, noisy, and packet-loss versions of the same audio. Second, statistical honesty: word error rate should come with bootstrap confidence intervals, because models whose intervals overlap the leader are effectively tied, not ranked. Third, streaming latency measured from forced end of speech to final transcript, which is the delay a caller actually feels. Fourth, fit to your deployment: batch and streaming models are different products, and a model that leads one list may not even appear on the other.

Where can you find independent speech-to-text accuracy and latency results?

Bluejay Labs publishes the MISTT speech-to-text leaderboard, which scores 104 models (57 batch, 47 streaming) on 2,999 clean clips plus 400 degraded copies covering phone lines, codecs, packet loss, noise, and a background talker, with accuracy confidence intervals and streaming latency timed from end of speech. Its published lowest English word error rates are 3.33% for a batch model and 3.43% for a streaming model, with several models statistically tied with each leader. Once you have picked a model, Bluejay helps you test and monitor the voice agents built on top of it, so transcription quality is verified against your real calls rather than a benchmark alone.

Conclusion

Judge speech-to-text APIs the way callers experience them: accuracy with confidence intervals, latency from end of speech, and robustness on degraded audio. Then keep evaluating the full agent in production, because a strong transcription score is the starting point, not the finish line.

Once you have a shortlist, sign up for Bluejay to test each candidate inside your full voice agent before you commit.