Added a 1500 word mini history to my book on the path to on-policy distillation being a core post-training optimization technique.

The math is fairly simple， seems like the sort of thing that started working as our distributed systems for training got better. It's very remarkable to me that a blog post from @_kevinlu at @thinkymachines is the canonical reference for using the reverse KL distance as an advantage within policy-gradient tools. This switch to distillation objectives within RL setups enables a lot of fun reward shaping ideas.

This also means that on policy distillation was obviously helped in its proliferation by the mass engineering effort in getting RL algorithms right over the last few years.

Lastly， as someone already very familiar with @agarwl_'s early work on generalized knowledge distillation / connection to imitation learning algorithms like DAgger， I recommend reading concurrent work MiniLLM which was technically the first to propose using a policy-gradient-like， on-policy rollout approach for distillation.