Rohan Paul@rohanpaul_ai
36AI 编辑部评分,满分 100

Meta FAIR 新论文揭示 Chinchilla 缩放定律外推盲点,提出 Skaling 方法

2026-08-15 21:47· 16分钟前
AI 导读

Meta FAIR 新论文发现 Chinchilla 缩放定律在外推至前沿规模时存在盲点,因其假设模型大小与训练数据独立作用。新方法 Skaling 仅增加一项耦合项,将预测误差降低约 1.5–3 倍,并以约 10 倍更少的 profiling 算力达到全网格精度。

A new Meta FAIR paper finds a blind spot in the Chinchilla scaling law that becomes expensive when you extrapolate.

Chinchilla can look almost perfect inside a training grid and still mispredict what happens at the frontier.

The problem is: Chinchilla assumes model size and training data help separately, but the experiments show that each changes how useful the other one is.

Skaling adds just 1 extra term to capture that connection, cutting prediction error by about 1.5-3× and getting full-grid Chinchilla-level prediction accuracy with roughly 10× less profiling compute.

On Farseer, the difference becomes huge at frontier scale: at 2×10^25 FLOPs, Chinchilla points to ~380 tokens per parameter, while Skaling and the paper's direct estimates land around 20-40.

  • arxiv. org/abs/2608.07222

Title: "Skaling: Chinchilla's Exponents Meet Kaplan's Coupling"

来源:Rohan Paul · x.com

Meta FAIR 新论文揭示 Chinchilla 缩放定律外推盲点,提出 Skaling 方法

Rohan Paul · @rohanpaul_ai · X·2026-08-15 21:47·16分钟前
AI 导读

Meta FAIR 新论文发现 Chinchilla 缩放定律在外推至前沿规模时存在盲点,因其假设模型大小与训练数据独立作用。新方法 Skaling 仅增加一项耦合项,将预测误差降低约 1.5–3 倍,并以约 10 倍更少的 profiling 算力达到全网格精度。

A new Meta FAIR paper finds a blind spot in the Chinchilla scaling law that becomes expensive when you extrapolate.

Chinchilla can look almost perfect inside a training grid and still mispredict what happens at the frontier.

The problem is: Chinchilla assumes model size and training data help separately, but the experiments show that each changes how useful the other one is.

Skaling adds just 1 extra term to capture that connection, cutting prediction error by about 1.5-3× and getting full-grid Chinchilla-level prediction accuracy with roughly 10× less profiling compute.

On Farseer, the difference becomes huge at frontier scale: at 2×10^25 FLOPs, Chinchilla points to ~380 tokens per parameter, while Skaling and the paper's direct estimates land around 20-40.

  • arxiv. org/abs/2608.07222

Title: "Skaling: Chinchilla's Exponents Meet Kaplan's Coupling"

来源:Rohan Paul· x.com