A 27B research agent outscored Claude Opus 4.8 and GPT-5.5 on held-out paper replication by learning how to direct the research while outsourcing the coding.

Huge implication, maybe you do not need one giant model to do everything.

You can train one model to think like the researcher, then let stronger coding models do the implementation underneath it.

Replica, proposed in this paper, a scalable task space for paper replication, gives it a surprisingly simple way to practice that job.

Take a research paper, remove one results figure, and tell the agent: recreate this result by actually running the experiment.

Now the agent has to figure out all the messy stuff papers leave out: what to implement, what to simplify, what experiments to run, and whether the result is believable.

That gives the researchers something they can repeatedly train on, with an automated rubric grading each attempt.

After training on 242 of these tasks, Faraday uses GPT-5.5 as its coding agent but decides what research to do.

On 68 unseen AI-for-science tasks, Faraday scored 0.791 versus 0.748 for Claude Opus 4.8 and 0.729 for GPT-5.5, beating both on 60% of tasks.

And the difference was not just prettier plots.

Faraday was more likely to actually test the mechanism in the paper instead of taking shortcuts that produced the expected-looking answer.

• arxiv. org/abs/2608.13331

Title: "Training AI Scientists to Replicate Research"