FrontierML

Evaluating AI agents at autonomous empirical ML R&D.

Leaderboard

Each model’s mean normalized score over the 25 graded tasks of 25. On each task 0 is the starting reference and 1 a strong-result reference set by judgment, which no run reached. Ten tasks are private and count only in these means.

ModelScore on each taskstrong resultMean
Claude Fable 5.2
0.523
Claude Opus 5.5
0.399
GPT-6 Astra
0.371
Gemini 4 Carbon
0.353
Grok 4.7
0.115*

A public taskA private taskThe mean* Grok 4.7 ran only the public tasks, so its mean covers those 15Astra is GPT-6 Astra, not GPT-6.2 Astra

Research taste

Four composite measures of how each model went about its research, built from every job, model call and hand-in its runs recorded. Each runs from 0 to a target at 1 and none affects a task score. No model reaches two thirds of any target, and none leads on all four.

noveltymetacognitionefficiencycontrol

How well an agent knows what its work is worth

Whether its changes are new, and whether they lead

How much progress a research dollar buys

How much of its budget became working experiments

Fable 5.20.310.260.530.63
Opus 5.50.350.150.500.59
Astra0.130.200.340.58
Carbon0.190.230.390.54
Grok 4.70.140.040.220.28

compositepast two thirdstarget 1scale ends at 0.7; the target at 1 lies beyond

compositelevelforecastchecksnoisescale ends at 0.7; the target at 1 lies beyond

compositefrequencyintensityfrontierscale ends at 0.7; the target at 1 lies beyond

compositedistancerelativetimelinessscale ends at 0.7; the target at 1 lies beyond

compositeusefocusreliabilityscale runs to the target at 1

Task depth

No run reached its task’s strong-result reference, and on the hardest tasks the best run closed well under half the gap. Every task leaves meaningful research beyond twelve hours.

Every run on each of the public tasks, on one scale. The band is the distance the best run leaves to the strong-result reference; on the median task the best run closed 62% of the gap.

TaskLeft to 1

Judgment

The best runThe other runs