# RLHF 偏好数据与奖励机制历史回顾

- 来源：Nathan Lambert (@natolambert)
- 发布时间：2026-07-21 06:04
- AIHOT 分数：37
- AIHOT 链接：https://aihot.virxact.com/items/cmrtsfrc122v0bihzyaj5nj5e
- 原文链接：https://x.com/natolambert/status/2079326723574034771

## AI 摘要

Nathan Lambert 发布新讲座，回顾偏好数据历史（从亚里士多德到 VNM 效用定理）、奖励本质及 RLHF 公式化过程，并探讨 RLHF 数据中的开放问题。讲座覆盖其新书第 10-11 章内容。

## 正文

New lecture！ This one is a recap of a bunch of history of preferences， the nature of rewards， how RLHF is formulated， which were once seen as central problems in the field. How much as changed.

Still… super interesting to understand our optimization tools today. Books coming soon ：D

00：00 Intro & context
07：34 A short history of preferences （from Aristotle to the VNM Utility Theorem）
20：17 A brief overview of preference data （from the last two years of my practice）
31：11 Open questions in RLHF data

Lecture 8， covering Chapters 10 & 11 of my book.