Great technical paper from Harvard and MIT.
It's on role drift in compound LLM systems.
（bookmark it）
End-to-end RL improves the accuracy of a multi-module LLM pipeline without constraining how the modules divide labor internally. Researchers name that failure mode and measure it.
Role drift happens when a module preserves or improves end-task performance while abandoning its assigned role through shortcuts that system-level evaluation cannot see.
Two instances across two pipelines：
- A decomposer meant to split a question into sub-questions for a separate solver instead plants the answer inside them.
- A reader meant to answer from retrieved passages instead falls back on parametric memory.
An interesting finding is that if your hold the decomposer to its role and 86% of the RL improvement disappears.
Role Anchor is the proposed control. It preserves how the role prompt shifts a module's next-token predictions relative to a neutral prompt， using that as a proxy for the role's intended effect during training. Gradient analysis indicates it reduces alignment with the drift direction rather than simply suppressing learning.
Paper： https://arxiv.org/abs/2607.21627
Learn to build effective AI agents in our academy： https://academy.dair.ai/