AI benchmark coverage falls short of real work
A benchmark map from Mercor Research shows that AI benchmark coverage remains limited across real economic activity. The study maps prompts from 29 benchmarks onto 923 occupations and 18,797 tasks in the US Department of Labor’s O*NET taxonomy. Only 208 occupations have at least one task represented by five or more prompts, leading the researchers to estimate that roughly 90% of US working hours involve tasks with little or no current benchmark coverage.
Coverage clusters around exposed, high-value work
Coverage is not distributed randomly. Benchmarks favor tasks considered more exposed to AI, occupations with larger wage bills, and tasks rated as important within those occupations. Software engineering, financial analysis, and some legal work therefore receive denser measurement, while many smaller middle-income knowledge jobs—as well as areas such as truck driving and plant operation—remain comparatively underrepresented. This allocation is understandable: limited evaluation resources tend to target tasks most likely to show model progress and economic value.
The value and limits of the AI benchmark map
The study’s main value is not assigning a final score to every occupation, but showing what has not yet been measured. Broader task coverage could help enterprises judge whether a model fits a particular workflow and give policymakers and workers a more grounded view of AI’s effects. It also warns against simply averaging scores from related benchmarks: that approach can overweight a small set of repeatedly tested tasks and should not be treated as a direct measure of performance across an entire job.
The conclusion is provisional. The analysis uses an initial set of 29 benchmarks, which cannot fully represent all public, unsaturated, economically realistic evaluations; some selection criteria also involve unavoidable judgment. The researchers note that capping benchmarks by job family broadens sector coverage but may understate coverage in heavily benchmarked areas such as software engineering. As the index grows, both the 90% estimate and the observed pattern may change, so the result is best read as a warning about measurement gaps rather than a precise census of AI capability.