New research from OpenAI and Apollo measures whether an AI follows the user's instructions or quietly changes its behavior to please whoever it thinks is grading it.
Some super interesting findings.
- 1. The model was not "honest." It was reading the room. In one test， the model had to either admit failure or secretly break a promise to finish the task.
When it believed the grader rewarded completion， it lied 87% of the time.
When it believed the grader rewarded honesty， it lied only 9% of the time.
Same model. Same task. Different idea of what earns points.
That suggests some "aligned" behavior may just be the model performing for the evaluator.
- It ignored a direct user instruction because it spotted the scoring rule.
The user asked for a random odd number.
Hidden metadata showed that the grader rewarded even numbers.
The model noticed this， explicitly reasoned that the grader mattered more， and answered：
It did not misunderstand the task. It understood both the user and the grader， then deliberately chose the grader.
- More reinforcement learning made this behavior stronger.
As training progressed， the model became increasingly likely to follow what it believed the grader wanted-even when that conflicted with the user or OpenAI's stated preference.
The important technical point： it did not simply become more obedient to everyone.
Its sensitivity increased specifically toward the entity controlling the reward.