# 2D-RoPE让模型复制长文本更精准

- 来源：Rohan Paul (@rohanpaul_ai)
- 发布时间：2026-07-27 13:10
- AIHOT 分数：35
- AIHOT 链接：https://aihot.virxact.com/items/cms2s89wa041yro3fd9tnaw9r
- 原文链接：https://x.com/rohanpaul_ai/status/2081608042794356994

## AI 摘要

前沿大语言模型在解决高级推理任务时，仍无法精确复制长重复字符串，且输入越长或重复模式越多，复制越不可靠。研究发现，标准1D RoPE位置编码是部分原因，而2D-RoPE将源文本与输出视为独立行，使对应token对齐在同一列。在合成测试中，仅训练长度1-100的单层模型在长达1000倍的输入上完美复制，该优势在1.4B参数预训练模型上同样出现，但当前设计仍依赖换行符，非通用解决方案。

## 正文

Frontier language models can solve advanced reasoning tasks yet still fail to copy long， repetitive strings exactly.

The paper finds that copying becomes less reliable as inputs grow longer or repeat the same patterns.

The failure is partly architectural. With standard 1D RoPE， producing output token y_k requires retrieving x_k from a relative offset that changes with input length.

The choice of positional encoding can determine whether a model copies precisely or follows nearby patterns.

2D-RoPE changes the layout.

It treats the source and output as separate rows， so corresponding tokens line up in the same column.

In synthetic tests， one-layer models trained on lengths 1-100 copied perfectly at lengths up to 1，000 times longer.

The advantage also appeared in pretrained models up to 1.4B parameters， with comparable common-sense performance at the tested scales.

The current design still leans on line breaks， so it is not a universal fix.

- arxiv. org/abs/2607.16072

Title： "Frontier Language Models Struggle to Copy： Text Can Be Better Viewed in 2D"