#217 – Beth Barnes on the most important graph in AI right now — and the 7-month rule that governs its progress
Published: June 2, 2025
Duration: 3:47:09
AI models today have a 50% chance of successfully completing a task that would take an expert human one hour. Seven months ago, that number was roughly 30 minutes — and seven months before that, 15 minutes. (See graph.)
These are substantial, multi-step tasks requiring sustained focus: building web applications, conducting machine learning research, or solving complex programming challenges.
Today’s guest, Beth Barnes, is CEO of METR (Model Evaluation & Threat Research) — the leading organisation measuring these capabilities.
Links to learn more, video, highlights, and full transcript: https://80k.info/bb
Beth's team has been timing...