New Tsinghua and other Chinese Univ paper finds AI agents can optimize a training plan for hours, but rarely realize when the plan itself is wrong.

Once training starts, an AI agent's first strategy tends to become the strategy it keeps optimizing.

Across 1,338 post-training trajectories, only 74 of 3,557 adjacent training experiments, 2.1%, changed the high-level strategy. Most iterations stayed inside the same approach, tweaking data, hyperparameters, formatting, or bugs.

The agents were not inactive. They could train, evaluate, diagnose failures, and improve checkpoints.

The problem appeared when evidence suggested the whole approach should change.

Adding an experiment journal, skill library, and evaluator lifted GSM8K by 12.6 points and HumanEval by 40.8 points over the Opus 4.6 baseline, but still did not trigger strategy changes.

Human guidance could redirect the plan before training, yet the agent returned to local tuning once training began.

Even 2–8× more inference tokens mostly bought more refinement, with almost no reliable gain on AIME 2025.

For AI R&D agents, the missing mechanism may be an explicit trigger to reopen the strategy itself, not just keep optimizing it.

– arxiv. org/abs/2608.19072

Title: "What is Missing from AI Post-Training AI: An Empirical Analysis"