AI model evaluation and the limits of benchmark scores

Connect with Centific to discover what's next in AI.

See where to meet us

Connect with Centific to discover what's next in AI.

Platforms

Data Marketplace

Data Canvas

AI Data Foundry

OneForma

Data Procurement

AI Localization

Expert Network

Join our Expert Network

Build & Train AI

RL Environments

Data Collection & Creation

RLHF & Preference Optimization

Supervised Fine Tuning

Model Safety & Evaluation

Internationalization

Vertical AI

Physical AI

Speech & audio

Healthcare

Explore our full suite of AI platforms, data marketplaces, and expert services designed to build, train, fine-tune, and deploy reliable, production-grade AI systems at scale.

Platforms

Data Marketplace

Data Canvas

AI Data Foundry

OneForma

Data Procurement

AI Localization

Expert Network

Join our Expert Network

Build & Train AI

RL Environments

Data Collection & Creation

RLHF & Preference Optimization

Supervised Fine Tuning

Model Safety & Evaluation

Internationalization

Vertical AI

Physical AI

Speech & audio

Healthcare

Explore our full suite of AI platforms, data marketplaces, and expert services designed to build, train, fine-tune, and deploy reliable, production-grade AI systems at scale.

Book a Demo

Article

AI model evaluation and the limits of benchmark scores

Explore eight Centific AI research papers accepted at NeurIPS 2026 and learn why evaluating AI models requires measuring task completion, safety, reliability, and professional standards.

Published on Nov 12, 2026

•

10 min read time

Topics

AI Evaluation

AI Research

AI Benchmarks

Author(s)

Shamsheer Rahiman

Aastha Ravindrakumar Bisen

Sasa Phanitsombat

Summarize

An AI model can get every answer right and still turn in work that falls short of a professional standard. In one Centific study, the strongest model got every final answer right on about 80% of 179 healthcare tasks. When we graded the full deliverable, it passed just 18% of the time. Which number would you use to decide whether the model was ready for the job?

That question runs through eight studies from Centific Applied AI Research. We looked at what happens when a phone assistant should leave a user’s data alone, when a voice agent has to catch an account number, and when the same multilingual labels are graded against two different answer keys. We also tested a way to build benchmarks around the tasks people do at work.

The research will be presented at NeurIPS 2026 workshops through eight papers that have been accepted.

How to evaluate AI agents

How do you tell whether an AI agent did a good job on your behalf? For a phone assistant, you can check what changed on the device, including whether it left things alone when it should have. For a negotiating agent, you can look at whether the deal worked for both sides. Both examples show why the details of the task matter when you build an evaluation.

Testing AI phone assistants for actions and restraint

We ran Apple’s ~3B on-device model as an assistant on a simulated iPhone and checked the device’s state after each task, instead of trusting the transcript. It passed 10 of 20 tasks.

Fourteen tasks called for action and six for holding back. The assistant passed 9 of the 14 and only 1 of the 6. When a web page told it to delete a user’s reminders, it deleted them with no warning.

We also asked an AI judge to grade the same runs from their transcripts. It misgraded about a third of the tasks, nearly all of them cases where the assistant was supposed to hold back, or to carry a remembered detail into a later action. Checking the device directly showed whether the assistant had left the user’s data alone.

Evaluating AI negotiation agents on deal quality

We built two agent marketplaces. In MarketDeal, agents negotiate a price and settle payment. In SwapShop, they barter with no price at all. Across 140 runs, close rates looked healthy, between 60% and 87% in basic trading. But the share of targets that ended in a deal good for both sides ranged from 13% to 80%. Gemini 3.5 Flash closed 9 of its 15 targets, and only 2 of those deals were good for both sides (Figure 1).

Bar chart of Gemini 3.5 Flash in basic trading: it closed 9 of 15 targets, but only 2 of 15 closed with both sides better off.

Figure 1. One configuration, 15 targets in basic trading.

Behavior also varied a lot between configurations: who captured the surplus, what agents checked and revealed, and where the money went. A model that traded well in one market could falter in the other. Two models from the same family, with the same reputation tool, behaved nothing alike.

In PAVE, the device showed what the transcript could miss. In the marketplaces, the negotiation transcript showed what the final ledger could miss, including what the agents checked, what they revealed and how they reached an agreement. The paper proposes a seven-part rubric for grading that behavior alongside the outcome.

Automating benchmark building for professional work

The phone and marketplace studies each needed tasks that exposed a specific kind of failure, along with clear criteria for grading what happened. Building those tasks and criteria by hand takes time. STRATA tests whether AI agents can help build them for professional work, starting with finance.

STRATA builds professional tasks and their rubrics from a short brief naming a domain and an occupation. It works in two phases, each using multiple agents. TaskGEN creates tasks grounded in publicly sourced references and deliverables, and RubricGEN writes rubrics at three levels: domain, occupation and task.

A benchmark built this way has to be trustworthy, so STRATA checks its own output. Nothing is tuned against the model being evaluated, and every factual value is taken word for word from a verified public source.

On finance, STRATA built 102 benchmark tasks across seven occupations, at a median cost of $11.80 and 32 minutes per task. Two subject-matter experts rated the tasks’ rigor at 4.5 and 5.0, and agreed within one point on 72% of the rating criteria.

How to evaluate voice agents and audio AI

The agent studies asked what you need to check after an AI takes action. With voice and audio, you also need to check what the model picked up along the way. A call completion rate can hide a misheard account number. A test where the speaker’s voice and topic point in the same direction can hide gender bias. And a single pass through a long recording can miss cues that are easier to pick up in shorter segments.

Testing voice agents on phone calls

We tested five voice agents from OpenAI, xAI, Google and Amazon on 3,825 conversations across six worlds (banking, insurance, travel, healthcare, logistics and pathfinding) and 11 conversation types. The best one completed 49% of tasks on the first attempt.

Agents that were strong at completing tasks weren’t always good at holding a conversation, and the reverse was also true. When a caller reads out an account number or other ID over the phone, the five agents misheard about one in six.

If you’re evaluating a voice agent for phone calls, track whether it completes the task and whether it captures critical details such as account numbers. The next study looks at another way a speech model can go wrong. When the speaker’s voice and the topic suggest different answers, which signal does the model follow?

Testing for gender bias in speech AI

When a speaker’s voice and topic agreed, even the worst speech-to-speech model got the speaker’s gender wrong only 2% of the time.

When voice and topic clashed, the same model’s error rate jumped from 2% to 90%. A model might, for example, refer to a man talking about childcare as “she.” To catch that kind of bias, include test cases where the topic cues a stereotype that conflicts with the speaker’s voice.

A clean output voice is no proof that a system is fair. To find this kind of bias, you have to test the cases where the signals disagree.

Improving AI analysis of long audio recordings

Meetings, interviews and calls can run for hours, and models that take them in one pass miss a lot. In our tests, a model read the speakers’ emotions correctly only 27% of the time.

The same model could do much better with the same recording. With AudioLoop, a language model writes code that cuts the recording at each change of speaker, then works through it piece by piece and keeps notes. The model reached 43% (Figure 2), and working from transcripts it scored twice as high at spotting contradictions.

Bar chart comparing emotion recognition in a 2.5-hour recording with the same model: 27% when the whole recording is processed in one pass, versus 43% with AudioLoop.

Figure 2. Same model, same 2.5-hour recording.

If you’re evaluating AI on long meetings, calls or interviews, test whether it does better when it can work through the recording in segments. In this study, changing how the same model worked through the same recording raised its emotion accuracy from 27% to 43%.

How answer keys and grading standards affect AI scores

AudioLoop showed how a model’s score can change when it works through the same recording in a different way. The next two studies show how a score can change even when the work stays the same. In one, we changed the answer key used to grade multilingual labels. In the other, we graded healthcare deliverables against the full professional standard.

How answer keys affect multilingual benchmark scores

On a six-language benchmark for finding personal information in text, experts re-labelled a sample of 2,628 prompts from scratch. On that sample, one annotator's labels scored 91.6 out of 100 against the answer key that shipped with the benchmark.

Against the experts’ key, the same labels scored 86.7 (Figure 3). That is a drop of about 5 points, and 7.5 points in Hindi. A second annotator's labels dropped only 2 points, and the first annotator's lead over the second disappeared.

Bar chart showing the same labels scored against two answer keys: 91.6 out of 100 against the shipped answer key, but 86.7 out of 100 against the expert-rebuilt key.

Figure 3. Scores out of 100 on a six-language benchmark. The gap was 7.5 points in Hindi.

When two annotators disagreed and no one resolved the disagreement, the benchmark almost always kept the first annotator's labels. That gave those labels a built-in advantage when they were graded against the shipped key. If you use a multilingual benchmark, ask how disagreements were resolved and whether the score holds up against an independently built key. The healthcare study asks a different question about grading the same work. What happens when the final answers are only one part of the grade?

Evaluating healthcare AI on complete deliverables

We tested six frontier models on 179 healthcare tasks, each asking for one professional deliverable built from a real document: a clinical guideline, a drug label or a clinical report. The strongest, Claude Sonnet 5, got every final answer right on 79.9% of tasks.

Graded on the full professional standard, it passed only 18.4%, and Gemini 3.1 Pro passed none.

On a subset of 55 tasks, asking GPT-5.6 Sol to cite its sources raised its pass rate on source traceability from 16% to 100%. Its pass rate on the full rubric rose from 2% to 9%. If you’re testing AI for document-based healthcare work, report how often it meets each requirement and how often it meets them all on the same task.

How to design AI evaluations for real-world tasks

A completed task can still contain a mistake that matters. A phone assistant might change a setting it should leave alone. A voice agent might mishear an account number. A healthcare model might give the right answer without showing where it came from. When you build an evaluation, those are the failures worth naming up front. That makes it easier to design test cases that bring them out and check whether your answer key and grading criteria catch them.

Centific can run these benchmarks on your models, build a private benchmark around your own tasks and documents, or review how your existing answer keys were put together.

Eight AI evaluation studies at a glance

Paper	Area	Main finding
PAVE	Agentic · Safety	An on-device assistant did the right thing on 9 of 14 action tasks, but held back on only 1 of 6.
Closing the Deal	Agentic	Close rates looked healthy, but deals good for both sides ranged from 13% to 80%.
STRATA	Agentic	Built 102 finance benchmark tasks at a median $11.80 and 32 minutes each.
DuplexWorld	Audio · Agentic	The best of five voice agents finished 49% of tasks on the first attempt.
Voice or Stereotype?	Audio · Safety	The worst model’s gender error went from 2% to 90% when voice and topic clashed.
AudioLoop	Audio	Emotion accuracy on a 2.5-hour conversation rose from 27% to 43%.
Same Output, Different Gold	Internationalization	The same human labels scored 91.6 or 86.7, depending on the answer key.
Healthcare.pdf	Healthcare	Top models got about four answers in five right but met the full professional standard on 18% at most.

Explore the AI research papers and benchmarks at NeurIPS 2026

Every paper, session time, room and link is on our NeurIPS page. The public benchmarks and datasets from these papers are on our Hugging Face hub.

modular

AI solutions delivered?

Centific offers a plugin-based architecture built to scale your AI with your business, supporting end-to-end reliability and security. Streamline and accelerate deployment—whether on the cloud or at the edge—with a leading frontier AI data foundry.

Start Building

Connect data, models, and people — in one enterprise-ready platform.

Latest Insights

Ideas, insights, and

research from our team

From original research to field-tested perspectives—how leading organizations build, evaluate, and scale AI with confidence.

Explore

Article

AI model evaluation and the limits of benchmark scores

Nov 12, 2026

Announcement

Centific Unveils Major Evolution of Flow Designed to Orchestrate and Govern Multilingual AI

Nov 11, 2026

Research insight

How to evaluate AI personal assistants for task completion and safety

Nov 9, 2026

Connect with Centific

Stay ahead of what’s next

Stay ahead

Updates from the frontier of AI data.

Receive updates on platform improvements, new workflows, evaluation capabilities, data quality enhancements, and best practices for enterprise AI teams.

Book a Demo

Get a live walkthrough

Talk to our team

Careers

See all our open positions

Turn data into AI that works

Book a demo

AI model evaluation and the limits of benchmark scores

How to evaluate AI agents

Testing AI phone assistants for actions and restraint

Evaluating AI negotiation agents on deal quality

Automating benchmark building for professional work

How to evaluate voice agents and audio AI

Testing voice agents on phone calls

Testing for gender bias in speech AI

Improving AI analysis of long audio recordings

How answer keys affect multilingual benchmark scores

Evaluating healthcare AI on complete deliverables

How to design AI evaluations for real-world tasks

Eight AI evaluation studies at a glance

Explore the AI research papers and benchmarks at NeurIPS 2026

Are your ready to get

modular

AI solutions delivered?

Ideas, insights, and

Ideas, insights, and

research from our team

research from our team

Stay ahead of what’s next

Stay ahead

Updates from the frontier of AI data.