Introducing Bluejay Labs and MIVAS

Every voice AI demo sounds ready. The model greets warmly, answers fluently, and books the appointment. Then it meets a real front desk: a caller who interrupts, a PHI gate that cannot be skipped, a refund that needs exact policy math, and a database that has to be correct after the call ends, not just a transcript that reads well.

Today we are launching Bluejay Labs, our research lab, and its first public benchmark: MIVAS, the Multi-Industry Voice Agent Simulation Bench. Bluejay Labs exists to close the gap between those two worlds, and MIVAS is the measuring stick: public, reproducible scores for how speech-to-speech models perform inside production-shaped voice systems.

Why Bluejay is starting a research lab

The mission of Bluejay Labs is to accelerate the adoption of safe, trustworthy, and reliable conversational intelligence systems. Voice AI is already answering phones in dermatology groups, plaintiff law firms, and national retail support centers — but adoption is throttled by a simple problem: nobody can tell, from a demo, whether a system is dependable enough to put in front of customers.

The lab’s method is to build RL environments and industry benchmarks that steer models toward desirable outcomes: worlds, tools, and tasks with verifiers that train the behavior, and public scores that show whether the systems are getting better. Its datasets are designed for speech-to-speech models specifically — production-shaped voice turns with overlap, tool calls, and real task pressure, not generic text scraped from the web.

“Bluejay Labs exists so voice systems can be used in the world, not only capable in a demo. Safe, trustworthy, and reliable enough to adopt.”

What is MIVAS?

MIVAS (Multi-Industry Voice Agent Simulation Bench) is an indicator of voice AI performance across economic sectors. It places each speech-to-speech model inside a multi-agent graph with production-length prompts, real tools, and isolated database state, then has locked, adaptive callers — Bluejay Digital Humans — place live calls into it. Deterministic verifiers score every conversation on three things: tool use, handoffs, and the final database state.

That last part is the point. A voice agent’s job is not to sound good; it is to leave the business’s systems in the right state. MIVAS treats the conversation the way an operations team would: did the required tools fire with the right arguments, did the call move through the right specialists, and is the database correct at hangup?

How MIVAS works

Each industry pack is a complete fictional company: a specialist agent graph, node-level prompts at production length, domain tools, a seeded database, and 72 locked tasks — 60 base cases and 12 audio variants that replay selected cases under background noise or a degraded signal, balanced evenly across easy, medium, and hard. Every case states who is calling, what they need, which tools must fire, which specialists must take the call, and what the database must look like at hangup.

Every case runs five independent times, each with its own conversation and its own SQLite file. After hangup, three deterministic verifiers score the same call, and the combined pass is the logical AND of all three: a correct write cannot hide a wrong route, and a correct handoff cannot hide a missing tool. Pass@1 measures whether a single run clears that bar; pass⁵ measures whether the same capability holds across all five attempts — the difference between a demo and something you would put on a phone line.

Three industries, three complete companies

MIVAS launches with three worlds, each written from production voice lines we already knew and then fully fictionalized so the environment is versioned, shareable, and safe to publish.

Healthcare: Straus Dermatology

A 160-location, 380-provider dermatology group, with “Robin” as its inbound line. Seven specialist desks sit behind a PHI identity gate: scheduling, coverage, cosmetic quotes, billing, and clinical triage that never reads results or approves a refill. The measurement surface is the work that actually stalls a front desk — several database writes inside a front-desk latency budget.

Legal: Halverson & Reed

A plaintiff-side contingency firm, with “Hal” on intake. Conflict screening runs before any facts of the matter, per ABA Rule 1.18. Practice-area and jurisdiction gates are separate checks, fee figures come only from the booking tool, and every decline is attorney-only. The graph is a simple five-node chain; the difficulty is the compliance work each node requires.

Customer support: Kestrel Electronics

A national electronics retailer with an order-bound identity gate in front of orders, returns, service, and membership — and a deliberately ungated fraud desk, because demanding a ZIP code and card digits from a frightened impersonation-scam victim is the scammer’s own move. Returns math, restocking fees, and delivery cutoffs are pinned in the state API so every spoken figure can be checked.

What the first scores say

Eight model harnesses ran the same locked suites in August 2026 — native speech-to-speech systems from OpenAI, Google, xAI, Amazon, and Alibaba, plus a cascaded STT/LLM/TTS baseline (Deepgram Flux, GPT‑4.1, and ElevenLabs Flash 2.5).

Three findings stand out. First, nobody clears 68% on a single run: the best model completes about two of every three production-shaped tasks, and under half of the hard ones. Second, reliability is the real gap. The leader’s 67.9% pass@1 falls to 50.7% when the same case must pass five times in a row — and a voice agent that succeeds four calls out of five is not one you can put on a phone line. Third, native speech-to-speech has pulled ahead of cascaded stacks at the top end, but the small, cheap tiers land at or below the cascaded baseline, so cost-per-hour and capability have to be traded off deliberately.

Why public scores matter

Teams shipping voice agents currently pick their model by vibes, demos, and provider marketing. Model providers, in turn, optimize for the benchmarks that exist — and until now, none of them scored multi-agent routing, provider-native handoffs, and final database state as one conjunctive job. MIVAS gives buyers evidence and gives providers a target that looks like the actual work.

The bench is fully reproducible: the task suites are locked, the verifiers are deterministic, and the environments, prompts, and harnesses are public in the mivas-bench repository. The same Digital Humans that place calls in MIVAS power simulation testing on the Bluejay platform — the lab and the product share one thesis: you cannot make voice AI trustworthy without measuring it under real task pressure.

This is the start. More industries, more model harnesses, and the datasets and RL environments behind the bench are on the lab’s roadmap — with public scores tracking whether the field is actually getting better.

Explore Bluejay Labs and MIVAS

The lab’s mission, the full leaderboard, methodology, industry environments, and locked task catalog are live on the research site, and the environments can be reproduced from the public repository.