Verifying Robot Data at Scale: NVIDIA Cosmos Curator + Cosmos 3 in the Centific Physical AI Stack

Connect with Centific to discover what's next in AI.

See where to meet us

Connect with Centific to discover what's next in AI.

Platforms

Data Marketplace

Data Canvas

AI Data Foundry

OneForma

Data Procurement

AI Localization

Expert Network

Join our Expert Network

Build & Train AI

RL Environments

Data Collection & Creation

RLHF & Preference Optimization

Supervised Fine Tuning

Model Safety & Evaluation

Internationalization

Vertical AI

Physical AI

Speech & audio

Healthcare

Explore our full suite of AI platforms, data marketplaces, and expert services designed to build, train, fine-tune, and deploy reliable, production-grade AI systems at scale.

Platforms

Data Marketplace

Data Canvas

AI Data Foundry

OneForma

Data Procurement

AI Localization

Expert Network

Join our Expert Network

Build & Train AI

RL Environments

Data Collection & Creation

RLHF & Preference Optimization

Supervised Fine Tuning

Model Safety & Evaluation

Internationalization

Vertical AI

Physical AI

Speech & audio

Healthcare

Explore our full suite of AI platforms, data marketplaces, and expert services designed to build, train, fine-tune, and deploy reliable, production-grade AI systems at scale.

Book a Demo

Article

Verifying Robot Data at Scale: NVIDIA Cosmos Curator + Cosmos 3 in the Centific Physical AI Stack

See how Centific combined NVIDIA Cosmos Curator, Cosmos 3, and Nemotron to verify robot training data at scale — lifting annotation accuracy above 70% at a fraction of commercial API costs.

Published on Oct 21, 2026

•

17 min read time

Topics

Physical AI

Robotics Training Data

Data Annotation

NVIDIA Cosmos

Verticals

Physical AI

Author(s)

Leela Krishna

Kriti Banka

Mangesh Damre

Shyam Ganatra

Ravi Garg

Summarize

From the 4% problem to a production pipeline.

In our first post, "How Centific Builds Better Robot Training Data, Powered by NVIDIA Cosmos Curator" we described the 4% problem: in one 2026 study, only 4% of pre-filtered egocentric clips survived full 3D hand-object reconstruction QC, and the economics of robot training data therefore turn on verified hours, not collected hours. That post ended with a design principle to verify and label deeply rather than collect widely and a first hands-on result: NVIDIA Cosmos Curator running end to end on egocentric manipulation footage, with NVIDIA Cosmos 3 Super naming objects and materials a frame-level pipeline cannot resolve. In Part 2 we went deeper into verification itself: where language-only judging of robot data falls short, the numeric layers that catch what it misses, and the directions we are exploring with the NVIDIA Cosmos team.

This closing part is the production story. Over the past two months, Centific Research put Cosmos Curator, the Cosmos 3 model family, and NVIDIA Nemotron 3 Super 3 Super to work on a live engagement: dense annotation and verification of long-horizon egocentric and teleoperation manipulation video for a frontier physical-AI customer, scored against the customer's own gold annotations and two rounds of professional audit. NVIDIA Nemotron 3 Super is a 120B open MoE model activating just 12B parameters to deliver maximum compute efficiency and accuracy for complex multi-agent applications. The headline results: Cosmos 3 Super is the best pre-annotation captioner we have benchmarked with a 28% relative lead over the next-best model on full-row correctness and a Curator-based pre-annotation stage lifted our end-to-end accuracy from the low 50s to above 70% in measured, incremental steps, at a fraction of the cost of commercial video-understanding APIs. Humans still make the final call; the pipeline's job is to make each human hour count for more.

The task: dense annotation at audit grade

The deliverable looks simple and is not. Every video is divided into contiguous segments with no gaps, no overlaps, timestamps to ±0.1 s. Every segment carries a caption of the intended action, a factual description of what actually happened when the two diverge, and a 1–5 execution-quality score. Captions must follow a strict structure (action verb + specific object + location + hand used), name visible details that disambiguate ("place the stainless pan on the left front burner," never "adjust the pan"), stand alone without context from earlier segments, and avoid vague verbs (adjust, moves, reaches) entirely. Pauses are segmented and scored. Retries are separated from first attempts. A professional audit team grades the output against a gold sheet, finding by finding.

At program scale this is a large human annotation task: a single ten-minute egocentric video yields 100–200 segments, and the two audit rounds we received concentrated on exactly the failure modes that exhaust human annotators: missed actions, chained actions folded into one segment, wrong or vague verbs, and object descriptions without distinguishing detail. Our approach is therefore not to replace the human pass but to pre-annotate: the machine drafts segment boundaries, captions, and scores; humans verify, correct, and augment under a standard operating procedure we wrote directly from the two rounds of audit feedback. Every percentage point of draft accuracy is a percentage point of human effort converted from authoring to reviewing. That is the productivity lever on a task priced in reviewer-hours.

Architecture: Curator as the backbone, Cosmos 3 as the brain, NVIDIA Nemotron as the editor

The pipeline follows the patterns in the Cosmos Cookbook recipes and uses Cosmos Curator (v2.4.0, on Cosmos-Xenna/Ray) in the role its own design intends: the dataset-scale backbone (ingest, splitting, score-only quality filters, Cosmos-Embed1 embeddings, semantic deduplication, model serving, and sharding) with our task-specific stages plugged into its stage interface. Six stages matter:

Ingest and normalization (Curator). Decode, transcode, and fixed-stride clip extraction. One field lesson: TransNetV2 shot detection correctly returns zero cuts on continuous single-take egocentric video. Shot detection is the wrong boundary tool for this footage, which is why stage 2 exists.
Kinematic segmentation (ours). Segment boundaries come from a hand-kinematics signal (our NVIDIA CaRe-Ego-based detector) rather than visual cuts: 65.6% boundary recall at ±0.5 s, more than double the best agentic video-understanding configuration we tested (31.1%).
Object registry (Curator models + SAM 3). Cosmos 3 Super proposes candidate object nouns from sampled frames; SAM 3, served through Curator's model registry, using its multi-object tracking contract, verifies which objects are physically present, collapses synonyms by mask overlap, and keeps persistent object identities across the episode. The registry covers roughly four fifths of the objects named in gold annotations and anchors color and identity consistency.
Captioning (Cosmos 3 Super, NVIDIA NIM). The per-segment captioner, constrained by the registry and by an action-frame verb library (next section). Cosmos 3 Super's distinguishing strength is video-level inference: it resolved white granules as salt because it watched them being added to cracked eggs, a label no frame-level crop can justify.
Guideline compliance (Nemotron 3 Super). A Nemotron model runs as the editorial judge inside Curator's caption-quality slot: it rewrites drafts into the required action-first structure, enforces the banned-verb list, checks that hindsight fields describe only observable deviations, and flags rows for priority human review. This is the stage that turned audit findings into executable rules.
Human review and export (SOP + Curator sharding). Reviewers work through the SOP distilled from both audit rounds; accepted episodes are deduplicated, sharded, and delivered with per-row provenance.

Annotation pipeline · 8 stages, 1 human loop

Everything inside the dashed boundary runs as Curator pipeline stages on one 8×H100 node; reviewer corrections feed back into the captioning stage, so each audit round makes the drafts better.

The division of labor is deliberate. Curator is not a dense action annotator and we do not use it as one. Our evaluation of a pure-Curator annotation path scored 34.8% fully-correct rows against 52–56% for our engine. Curator earns its place upstream and around the annotator: ingest, serving, embeddings, dedup, and shards at scale; Cosmos 3 and Nemotron earn theirs inside it.

What we measured

Captioner benchmark: Cosmos 3 Super leads

We benchmarked seven VLM captioner configurations on five production videos against gold annotations, scoring each caption slot independently (did the model get the verb, the object, the location, and the hand right), plus all-3 correct (verb + object + hand in one row) and compliance with the vague-verb rule (share of rows free of banned verbs like adjust, moves, reaches).

Model	Verb	Object	Location	Hand	All-3 correct	Vague-verb compliance
Cosmos 3 Super	.362	.427	.318	.632	.137	.973
Qwen3-VL-32B	.337	.362	.185	.592	.107	.989
Gemini-Robotics-ER-2	.278	.328	.236	.581	.103	.998
Cosmos 3 Nano (previous baseline)	.232	.309	.366	.571	.067	.760
Gemini-3.8-Flash	.221	.303	.208	.566	.065	1.000
Claude Fable 5.1	.198	.284	.107	.541	.044	1.000
Gemma-4-31B	.227	.337	.242	.505	.044	.998

Cosmos 3 Super wins on verb, object, and hand, and leads all-3 correctness by 28% relative over the runner-up. Just as important operationally: moving from Nano to Super pushed vague-verb compliance from .760 to .973, retiring Nano's single worst failure and the largest driver of audit complaints. Absolute all-3 numbers look low because the metric is unforgiving (three independent slots must all match gold exactly); it is the ranking and the deltas that drive engineering decisions.

Segmentation: the kinematic prior beat agentic video understanding

Segmentation is where the two largest audit categories live (missed actions and chained actions), so when agentic video understanding shipped with advertised sub-second cut detection, we tested it against our kinematic boundary detector on the same footage:

Configuration	Segments found	Boundary recall @ ±0.5 s	Media tokens
Static sampling (1 FPS)	18	9.8%	18,876
Agentic	20	13.1%	13,617
Agentic + granularity prompt	63	31.1%	6,671
Our kinematic signal (CaRe-Ego)	—	65.6%	—

We did not adopt it, since the kinematic detector is twice as good on the metric that matters, but two learnings transfer: telling the model the target segment granularity more than doubled recall (13.1% → 31.1%), and agentic processing genuinely cut media tokens ~65% versus static sampling, which matters for any token-metered video work.

The action-frame upgrade: a 2011 idea, implemented with RAG

The newest gain comes from an old paper: "Verb Semantics for Robot Dialog" (Thomas & Jenkins, 2011), which proposes that every verb binds to exactly one action class from a fixed set, each carrying noun slots in the FrameNet sense: lifting an apple and lifting an orange are one frame with a different object in the slot. We implemented exactly that as a retrieval-augmented constraint on Cosmos 3 Super: a library of 48 action verbs mined from production captions (27 of which cover 95% of the gold sheet), from which the model must pick one verb per segment. It never writes a verb outside the list. Objects and hands stay free-form, filled from the clip with the registry's per-video object names.

On a 155-segment gold video, the action-frame configuration resolved 71% of previously flagged incorrect-verb items, against 66% for the unconstrained run. The broader trajectory is the real story: each increment (better model, registry grounding, action frames) has added roughly +5 points. Cosmos 3 Nano started us at ~50% usable-row accuracy; Cosmos 3 Super took it past 65%; the action-frame RAG configuration is now past 70%.

The economics: frontier APIs can do it; self-hosted Cosmos does it economically

To be clear about the alternatives: frontier video APIs are capable here. Gemini's robotics-tuned models sit mid-table in our captioner benchmark, and GPT-6 Astra-class models bring strong video reasoning. The question for a production annotation program processing thousands of video-hours, and re-processing them on every pipeline improvement, is unit cost. Dense annotation is token-hungry: multi-pass processing (segmentation, per-segment captioning, scoring, compliance checks) consumes roughly 4M input tokens and 0.25M output tokens per video-hour at 1 FPS-equivalent sampling, from our measured media-token counts.

Option	List price (Oct 2026)	Est. cost per video-hour	Per 1,000 video-hours	Notes
Cosmos 3 Super + Curator, self-hosted 8×H100 (NVIDIA Brev)	$24.50/cluster-hour	~$6	~$6,000	Flat rate at ~4 video-hours processed per cluster-hour; re-runs and iteration are free within reserved capacity; data never leaves the tenancy
Gemini 3.1 Pro API	$2 / $12 per 1M input/output tokens	~$11	~$11,000	Long-context surcharge ($4/$18 beyond 200K tokens) hits long-horizon videos
GPT-6 Astra API (standard tier)	$10 / $50 per 1M input/output tokens	~$53	~$53,000	Rises to $20/$75 per 1M beyond 272K-token context, common for long episodes

At program scale the self-hosted Cosmos stack runs at roughly half the unit cost of Gemini 3.1 Pro and an eighth of GPT-6 Astra, while scoring above both in our captioner benchmark. The flat-rate economics matter beyond the headline number: our accuracy gains arrive as +5-point increments, each requiring a full corpus re-run. On a metered API every iteration is a new bill; on a reserved 8×H100 box on NVIDIA Brev, iteration is how you spend capacity you already own. Estimates use published list prices as of October 2026 and our measured throughput; your token counts will vary with video length, sampling rate, and pass structure.

Build versus adopt: two weeks instead of six months

There is a second comparison that mattered more than the per-hour math: the cost of the pipeline itself. Before committing to Curator, our Agentic Development team scoped what it would take to build an equivalent distributed video pipeline in-house on Apache Ray. The list grows quickly: a streaming scheduler with GPU packing, serving for a changing roster of vision-language models, quality filters, embeddings, deduplication, sharding, and the operational hardening nobody likes to budget for. The honest estimate came back at about six months end to end. With Cosmos Curator we went to market in under two weeks, and the engineering effort moved to the stages that actually differentiate us.

	Build in-house on Apache Ray (scoped)	Adopt Cosmos Curator (actual)
Time to production	About 6 months end to end	Under 2 weeks
What the team builds	Streaming scheduler, GPU packing, model serving, filters, embeddings, dedup, sharding, plus operational hardening	Only the custom QC and annotation stages
Adding a new VLM	A hand-wired integration each time	A config change in the caption slot
Ongoing maintenance	Carried by our team alone	Shared with NVIDIA and the open-source community

That gap, roughly a factor of twelve in calendar time, is the strongest single argument in this series for building on the open NVIDIA stack.

One honest aside from the segmentation experiment: agentic processing's ~65% media-token reduction would narrow the API gap for token-metered work; it just didn't come with the boundary accuracy our task needs.

Humans stay in the loop, and honesty about where we are

Pre-annotation does not remove the human pass; it changes what humans spend it on. We codified both rounds of audit feedback into a Standard Operating Procedure for reviewers: what to check first (segment coverage and boundaries), the caption structure to enforce, how to score pauses and retries, and when a machine draft is trustworthy enough to verify rather than rewrite. Nemotron 3 Super's compliance stage pre-sorts rows so reviewer attention lands on the segments most likely to be wrong. The effect of a draft that is 70% right rather than 50% right is felt directly in reviewer throughput: more rows are confirmed than authored.

We want to be precise about status, because this is a technical blog and not a victory lap. The solution is not finished. Audit-grade dense annotation of long-horizon manipulation video is a genuinely hard, tedious problem, and our progress has come in measured +5-point increments (model upgrade, registry grounding, action frames), each validated against gold before it ships. At the time of writing the pipeline is above 70% on usable-row accuracy and resolving 71% of flagged verb errors, with per-hand attribution as the next increment; we estimate the remaining gap to target quality in weeks, not quarters. Cosmos Curator and Cosmos 3 Super are certainly helping; they are the reason the increments are cheap to attempt, and we will keep publishing the numbers as they move.

Technical insights

The Curator features that earned their keep

Curator feature	Why it mattered in production
Model registry + caption-stage serving (vLLM/NIM)	One interface serves Cosmos 3 Nano/Super, Qwen, Gemma, Nemotron 3 Super, and fetches SAM 3. Swapping captioners is a config change, which is what made a 7-model benchmark cheap to run
Multi-object tracking contract	Stable per-object identities (a mask cannot silently jump instances), multi-prompt tracking in a single pass, bounded memory via periodic session resets. It fixed three defects our hand-written tracker had
Cosmos-Embed1 embeddings + semantic dedup	Corpus hygiene: near-duplicate and idle-span detection before anything skews a training mixture
Score-only filter mode	Quality signals recorded as metadata without dropping a single span, essential for a verification program, where no filter may discard a client row
Fixed-stride clip extraction	The workhorse unit-maker for serving, together with the honest negative that TransNetV2 shot detection correctly finds zero cuts in single-take egocentric video

The general lesson: Curator's value in this program is as substrate (model access, scale, object identity, and corpus hygiene) while the dense-annotation intelligence lives in the custom stages plugged into it. Used that way, every piece earned its keep; used as a turnkey dense annotator, it scored 20 points below our engine. Know which tool you are holding.

Nano to Super: what the bigger model actually buys

The 8B→32B upgrade was not a uniform lift. It moved the metrics that drive audit outcomes. Fully-correct rows doubled (.067 → .137), usable-row accuracy went from ~50% to 65%+, and vague-verb compliance jumped from .760 to .973, retiring Nano's single worst failure mode. The qualitative difference is video-level inference: Super resolves what a thing is from what happens to it across the clip (the salt-into-cracked-eggs example), and it judged object color more reliably than our pixel-measurement track. Given a registry color it disagreed with, it overrode it and was usually right. Nano keeps a role as the cheap first-pass annotator for bulk triage; on a flat-rate cluster the cost of running Super where quality is graded is throughput, not dollars.

What egocentric video demanded, and what that gives VLA training

Egocentric manipulation footage breaks the standard video-understanding toolkit in specific, fixable ways. There are no cuts, so shot detection returns nothing and boundaries must come from hand kinematics. The camera is a moving head, so hand motion must be measured against the world, not the frame. Getting that right takes SLAM-composed wrist tracks and an explicit camera trajectory; without them, head motion leaks into every downstream signal as phantom hand motion. Object identity must survive minutes of occlusion and re-entry, which is what the SAM 3 registry's persistent IDs provide. And single frames are ambiguous; white granules are only "salt" because the model watched the whole clip.

Each of those fixes is also exactly what VLA training and retargeting consume. Contiguous, kinematically segmented phases with action-first language give a VLA instruction-grade supervision at two grains: one caption per phase, one instruction per episode. The action-frame rows (actor, hand, object, phase, outcome) map directly onto robot skill primitives, and because an action frame is embodiment-agnostic (pick up is one class whether the slot is filled by a human hand, a two-finger gripper, or a 22-DoF hand), it is the natural bridge for retargeting human demonstrations to robot embodiments. World-referenced wrist trajectories plus per-hand attribution (our next increment) supply retargeting's actual inputs: effector pose deltas and grasp states rather than pixels. Verified, densely labeled egocentric hours are not just cleaner data; they are the representation the quality-over-quantity results say moves policy success.

What's next: the Physical AI Data Factory

As part of our forward evaluation, we are adopting workflows from newly open-sourced NVIDIA Cosmos Data Factory, a collection of agent-executable synthetic-data-generation blueprints (Defect Image Generation with Cosmos AnomalyGen, Video Data Augmentation with Cosmos) that run on NVIDIA OSMO. Its contribution to robotics capability is broader than any single workflow: it standardizes data pipelines as agent-executable skills, which makes them reproducible across teams rather than locked in one lab's scripts; its AnomalyGen-class generation attacks the failure-data gap, the defects and edge cases that rarely occur naturally but that inspection and recovery policies must learn from; and its augmentation workflows multiply verified corpora with controlled variation. Joined to the verification pipeline described in this post, the two halves close a loop, generation feeding verification and verification gating generation, into what is, in effect, a Centific data factory powered by NVIDIA Cosmos Data Factory. Our evaluation targets two uses first: augmenting verified teleoperation corpora with controlled environment variation, and generating labeled edge cases for the verification gates themselves.

Our thanks to the NVIDIA teams behind Cosmos Curator, the Cosmos 3 model family, Nemotron, and the CosmosData Factory for building and open-sourcing this stack. An annotation program like this one is only economically possible because the backbone (curation framework, world-model captioners, serving, and now workflow recipes) is open, inspectable, and runs on hardware we control. That is what lets a data company put its engineering effort where it differentiates: verification gates, annotation quality, and the human expertise that no model replaces yet.

A note on Humanoid IQ. The pipeline in this post is one subsystem of Humanoid IQ, Centific's flagship robotics data platform, which runs from data collection through delivery: multi-modal capture (egocentric, egocentric stereo, VR and full-body teleoperation, paired on the same tasks), the verification and dense-annotation stack described here, and a curated task library built for transfer: across embodiments (two-finger grippers to 21- and 22-DoF hands, with human hands as the source), across capture modalities, across twelve industry verticals, and across an outcome mix of roughly 70% success, 10% failure, and 20% failure-with-recovery. The design goal is that one verified, densely annotated episode trains VLAs and world action models alike. Transferability is the product, not a post-processing step.

Explore Centific's Physical AI capabilities at centific.com/physical-ai, or inspect verified Humanoid IQ episodes yourself at robot-iq.centific.com/inspect.

Sources

NVIDIA Cosmos Curator · Cosmos Cookbook · NVIDIA Physical AI Data Factory
Thomas & Jenkins, "Verb Semantics for Robot Dialog," Brown University, 2011
Pricing: Gemini API rates, Oct 2026 · GPT-6 Astra official rates · NVIDIA Brev
Internal: captioner benchmark (7 configurations, 5 videos, 475 gold spans), segmentation experiments, action-frame evaluation (gold video, 155 segments), Curator evaluation reports, September 2026

All benchmark figures are measured on confidential production videos referenced by key only; no client footage or captions appear in this post.

modular

AI solutions delivered?

Centific offers a plugin-based architecture built to scale your AI with your business, supporting end-to-end reliability and security. Streamline and accelerate deployment—whether on the cloud or at the edge—with a leading frontier AI data foundry.

Start Building

Connect data, models, and people — in one enterprise-ready platform.

Latest Insights

Ideas, insights, and

research from our team

From original research to field-tested perspectives—how leading organizations build, evaluate, and scale AI with confidence.

Explore

Article

Verifying Robot Data at Scale: NVIDIA Cosmos Curator + Cosmos 3 in the Centific Physical AI Stack

Oct 21, 2026

Article

The 4% problem: How Centific Builds Better Robot Training Data, Powered by NVIDIA Cosmos Curator

Oct 5, 2026

Press release

Centific Recognized as a Leader in Localization Services

Sep 28, 2026

Connect with Centific

Stay ahead of what’s next

Stay ahead

Updates from the frontier of AI data.

Receive updates on platform improvements, new workflows, evaluation capabilities, data quality enhancements, and best practices for enterprise AI teams.

Book a Demo

Get a live walkthrough

Talk to our team

Careers

See all our open positions

Turn data into AI that works

Book a demo

Verifying Robot Data at Scale: NVIDIA Cosmos Curator + Cosmos 3 in the Centific Physical AI Stack

The task: dense annotation at audit grade

Architecture: Curator as the backbone, Cosmos 3 as the brain, NVIDIA Nemotron as the editor

What we measured

Captioner benchmark: Cosmos 3 Super leads

Segmentation: the kinematic prior beat agentic video understanding

The action-frame upgrade: a 2011 idea, implemented with RAG

The economics: frontier APIs can do it; self-hosted Cosmos does it economically

Build versus adopt: two weeks instead of six months

Humans stay in the loop, and honesty about where we are

Technical insights

The Curator features that earned their keep

Nano to Super: what the bigger model actually buys

What egocentric video demanded, and what that gives VLA training

What's next: the Physical AI Data Factory

Sources

Are your ready to get

modular

AI solutions delivered?

Ideas, insights, and

Ideas, insights, and

research from our team

research from our team

Stay ahead of what’s next

Stay ahead

Updates from the frontier of AI data.