Does AI Actually Work?

The Benchmark–Production Gap

Frontier models score above human expert baselines on bar-exam questions, medical licensing exams, and graduate-level science benchmarks. In production, the same models fabricate citations, fail multi-step tasks, and require human review that often costs more than the work they save. Both statements are true. The gap between them is the single most important number in the AI economy, and it is a gap, not a contradiction: benchmarks measure capability on well-specified tasks; production demands reliability on ambiguous ones.

Two studies from inside the labs themselves made the gap measurable in early 2026.

Anthropic, March 2026: theoretical vs. observed exposure

Anthropic's Economic Index research distinguished theoretical exposure — the share of tasks in an occupation that frontier models can perform at benchmark level — from observed usage, measured in real Claude conversations. Theoretical exposure spans roughly half of US occupations; observed automation clusters in a narrow band: software, writing, customer support. In most exposed occupations, actual delegation remained a small fraction of what the models could theoretically handle. Anthropic Economic Index, Mar 2026

OpenAI, January 2026: the capability overhang

OpenAI's own report documented a "capability overhang": users systematically under-use what the models can do. Its starkest datapoint — power users invoke extended thinking roughly 7× more than the median paying user — implies the median user has tried the frontier, found it unreliable or opaque for their work, and settled into a shallow usage pattern. The capability exists; the trust does not. OpenAI, Jan 2026

Hallucination: The Reliability Tax

Every independent evaluation finds the same shape: frontier models hallucinate — confidently assert false facts, nonexistent cases, invented citations — at rates that fall with each generation but have never reached zero. Legal databases document hundreds of sanctions against lawyers who filed AI-generated briefs citing phantom precedent. In medicine, studies find clinically significant errors in a meaningful minority of diagnostic and dosing answers, even from models that pass licensing exams. Court dockets · peer-reviewed studies

The structural problem: a language model optimised for plausible continuation has no internal distinction between "true" and "sounds true." Retrieval-augmented systems and citation grounding reduce the rate; they do not eliminate it, and each reduction adds cost and latency. Reliability is not a bug being fixed — it is a tax being paid, forever, on every deployment.

The Coding Evidence: A Natural Experiment That Split

Nowhere is the evidence more contested than in software, the industry's own backyard:

• The optimistic study: GitHub's controlled trial of Copilot found novice developers completed a task ~55% faster with the assistant. Widely cited; narrow task; junior developers. GitHub / arXiv, 2022–23
• The pessimistic study: METR's 2025 randomised trial of experienced developers on their own repositories found AI tools made them 19% slower — while the developers felt 20% faster. Perception and measurement pointed in opposite directions. METR, 2025
• The enterprise cautionary: A major consultancy's internal experiment was paused after AI-generated code introduced defects; several firms report rising code-review burdens as AI output volume grows. Press reports, 2025

The reconciling interpretation: AI coding tools compress the typing part of programming and inflate the verification part. For boilerplate and greenfield tasks, net positive. For complex, existing codebases with correctness requirements, the evidence is genuinely unresolved — and the companies selling the tools are the ones with the best internal data, which they do not publish.

Enterprise ROI: What the Surveys Actually Say

• Consultancy surveys consistently find a majority of enterprises running AI pilots; a much smaller minority report measurable P&L impact. The pilot-to-production conversion rate is the quiet statistic of the era. McKinsey / BCG annual surveys
• The deployments with the cleanest measured returns are the least glamorous: customer-service deflection, translation, summarisation of structured documents, code migration. Gains cluster at 10–20% productivity in narrow, supervised workflows. NBER field studies
• The deployments pitched to boards — autonomous agents replacing whole functions — have the weakest evidence. Independent agent evaluations show multi-step task success rates in the low double digits on realistic benchmarks. Independent agent evals

The Verdict, Split by Domain

Confidence Assessment