November 17, 2026

Is there a public benchmark for voice AI agents on booking and support tasks?

Is there a public benchmark for voice AI agents on booking and support tasks?

If you are choosing a voice AI model for real agent work like appointment booking or customer support, public benchmarks do exist, and they are worth reading before any vendor conversation.

Introduction

The most relevant is MIVAS, an open speech-to-speech benchmark from Bluejay Labs that scores model harnesses on stateful, production-shaped voice agent tasks across healthcare, legal, and customer support. It measures what actually matters in deployment: correct tool use, clean handoffs, and the final state of the database after the call, not just whether the model sounded fluent.

Frequently Asked Questions

Is there a public benchmark for voice AI agents on realistic tasks?

Yes. MIVAS (Multi-Industry Voice Agent Simulation Bench) is a public, open source benchmark built specifically for realistic voice agent tasks. Each task places a model harness inside a multi-agent graph with production-length prompts, tools, and isolated state, then deterministic verifiers score tool use, handoffs, and final database state. That means a booking call is judged on whether the appointment actually landed in the system correctly, not on a judge model's opinion of the transcript.

What makes MIVAS useful for buyers?

Two design choices make it useful for buyers. First, reliability is measured over repeated runs rather than a single pass: every case runs five independent times, and pass@1 versus pass^k separates a model that occasionally succeeds from one you can trust. Second, tasks come from industry suites modeled on production voice lines, including a customer support suite alongside healthcare and legal, so the scenarios resemble the calls your agents will actually take. You can compare all 14 model harnesses across 216 tasks on the MIVAS speech-to-speech leaderboard.

Does a public benchmark tell you how your own agent will perform?

A benchmark tells you how models perform in general, though, not how your agent performs on your call flows, your tools, and your phone lines. That gap is where a testing and monitoring platform like Bluejay fits in: it runs automated, repeatable evaluation of your own agents, in CI/CD and in production.

Conclusion

Public benchmarks for realistic voice agent tasks exist, and MIVAS is one example: open source, industry-specific, and scored deterministically on outcomes like tool calls and database state. Use it to shortlist models, then validate your own deployment with continuous testing, because a leaderboard score is a starting point, not a substitute for evaluating your agent on your own traffic.

To move from benchmark results to your own agent, open a Bluejay account and test it on your booking and support flows.