CheckerBench: 78% Compiled, 32% Were Accepted
A new benchmark had AI agents write reusable static-analysis checkers. 78.25% compiled, 73.65% told the broken revision from the fixed one, 32.30% were accepted. The loss is almost entirely after detection, and the two obvious explanations both test small.