Benchmarks

Evaluation Discipline Is as Important as the Model

In physical AI, the ability to measure task success, degradation and real-world robustness is often as important as the model itself.

Georgian + NewtonX Physical AI Survey, Wave 1 · Reporting base: Buyers, n=85 unless a benchmark states otherwise · July 24 to Aug. 21, 2026

Benchmark H4.1

What Buyers Can Actually Measure

Survey question

How well can you actually measure each of the following for the systems you are closest to?

Audience
All qualified buyers, n=85
In depth
Hypothesis 4 — Evaluation Discipline as the Signal

Four items, each answered on the same four-point scale.

What Buyers Can Actually Measure

20%

of robotics buyers say they can reliably measure whether a change they made actually improved real-world performance — which, in our view, a learning loop depends on

All qualified buyers, n=85

Data for What Buyers Can Actually Measure
CategoryWe measure this well and rely on it
Whether a deployment is succeeding or failing at its task in the real world33%
Whether performance is degrading over time27%
Whether a change to the system actually improved real-world performance20%
How the system handles edge cases and conditions it wasn't built for19%
0%10%20%30%40%Whether a deployment is succeeding orfailing at its task in the real worldWhether a deployment is succeeding or failing at its task in the real world — We measure this well and rely on it: 33%33%Whether performance is degrading over timeWhether performance is degrading over time — We measure this well and rely on it: 27%27%Whether a change to the system actuallyimproved real-world performanceWhether a change to the system actually improved real-world performance — We measure this well and rely on it: 20%20%How the system handles edge cases andconditions it wasn't built forHow the system handles edge cases and conditions it wasn't built for — We measure this well and rely on it: 19%19%

All qualified buyers, n=85

Benchmark H4.2

Metrics Tracked Consistently

Survey question

Which of the following metrics does your organization track consistently today?

Audience
All qualified buyers, n=85
In depth
Hypothesis 4 — Evaluation Discipline as the Signal

Select all that apply, so the shares add to more than 100%. Buyers track 5.4 of these metrics each on average.

Metrics Tracked Consistently

69%

of robotics buyers track uptime and availability consistently, the most-tracked of the eleven metrics we asked about

All qualified buyers, n=85

Data for Metrics Tracked Consistently
Category% of buyers
Uptime / availability69%
Safety incidents65%
Cost per unit of work61%
Throughput58%
Deployment time53%
Autonomous completion rate51%
Payback period45%
Intervention rate42%
Mean time to resolution41%
Support hours per unit / site31%
Recovery success rate26%
None of the above2%
0%20%40%60%80%Uptime / availabilityUptime / availability — % of buyers: 69%69%Safety incidentsSafety incidents — % of buyers: 65%65%Cost per unit of workCost per unit of work — % of buyers: 61%61%ThroughputThroughput — % of buyers: 58%58%Deployment timeDeployment time — % of buyers: 53%53%Autonomous completion rateAutonomous completion rate — % of buyers: 51%51%Payback periodPayback period — % of buyers: 45%45%Intervention rateIntervention rate — % of buyers: 42%42%Mean time to resolutionMean time to resolution — % of buyers: 41%41%Support hours per unit / siteSupport hours per unit / site — % of buyers: 31%31%Recovery success rateRecovery success rate — % of buyers: 26%26%None of the aboveNone of the above — % of buyers: 2%2%

All qualified buyers, n=85