Runtimes
Nobody knows how to measure coding agents anymore
On July 8, 2026, OpenAI disavowed SWE-Bench Pro: ~30% broken tasks. It is the third reference instrument declared dead in three generations. Anatomy of a metrology crisis: noise ceiling, contamination, and a harness effect worth a whole model generation.