FrontierML
Evaluating AI agents at autonomous empirical ML R&D.
Leaderboard
Each model’s mean normalized score over the 25 graded tasks of 25. On each task 0 is the starting reference and 1 a strong-result reference set by judgment, which no run reached. Ten tasks are private and count only in these means.
Research taste
Four composite measures of how each model went about its research, built from every job, model call and hand-in its runs recorded. Each runs from 0 to a target at 1 and none affects a task score. No model reaches two thirds of any target, and none leads on all four.
How well an agent knows what its work is worth
Whether its changes are new, and whether they lead
How much progress a research dollar buys
How much of its budget became working experiments
compositepast two thirdstarget 1scale ends at 0.7; the target at 1 lies beyond
compositelevelforecastchecksnoisescale ends at 0.7; the target at 1 lies beyond
compositefrequencyintensityfrontierscale ends at 0.7; the target at 1 lies beyond
compositedistancerelativetimelinessscale ends at 0.7; the target at 1 lies beyond
compositeusefocusreliabilityscale runs to the target at 1
Task depth
No run reached its task’s strong-result reference, and on the hardest tasks the best run closed well under half the gap. Every task leaves meaningful research beyond twelve hours.
Every run on each of the public tasks, on one scale. The band is the distance the best run leaves to the strong-result reference; on the median task the best run closed 62% of the gap.
The best runThe other runs