labor-market-analysis
Latest stories covering labor-market-analysis, with summaries and links to related coverage.
2 articles · page 1 of 1
Connected topics
90% of US working hours are spent on tasks not covered well by AI benchmarks
research
9 min read
Mercor has published a Benchmark Map that measures how well 29 selected AI benchmarks cover tasks across the US labor force. The map uses the Department of Labor’s O*NET taxonomy, which divides the workforce into 923 occupations, 22 job families, and 18,797 tasks. A task is counted as covered when an LLM judge maps at least five benchmark prompts to it; one prompt can map to multiple tasks. Only 208 of 923 occupations have at least one task meeting that threshold, while six of 22 job families have none. Mercor estimates that 90% of US working hours are spent on tasks with little or no coverage in its initial benchmark set, making economy-wide assessments of AI capability difficult. The organization says benchmark coverage disproportionately favors AI-exposed tasks, with O*NET tasks rated exposed to AI about six times more likely to appear than unexposed tasks. Tasks in occupations with larger wage bills and tasks rated most important are also overrepresented. Mercor found that reweighting APEX benchmark scores so each O*NET task counts equally changed model results little for investment banking, law, accounting, and software engineering. Less than one-third of relevant tasks in those four occupations had sufficient APEX coverage. Mercor characterizes its estimates as provisional because its benchmark list is incomplete, selection criteria involve judgment, and excluding saturated benchmarks can understate coverage.
US Job Market Visualizer
tool
1 min read
The US Job Market Visualizer on GitHub visualizes 342 occupations from the Bureau of Labor Statistics, representing 143 million jobs in the US economy. Each rectangle's area corresponds to total employment, with color coding for metrics such as projected growth outlook, median pay, education requirements, and AI exposure.