# GPT-5.6 Sol 在 ARC-AGI-3 上是否优于 Opus 5？

- 来源：Chubby♨️ (@kimmonismus)
- 发布时间：2026-07-30 16:08
- AIHOT 分数：38
- AIHOT 链接：https://aihot.virxact.com/items/cms7933hu04bcro2e0eojq1o6
- 原文链接：https://x.com/kimmonismus/status/2082740117844734150

## AI 摘要

在官方标准评测下，Opus 5 得分 30.2%，GPT-5.6 Sol 仅 7.8%。但 OpenAI 改用自有 API 保留推理并压缩长上下文后，Sol 在公开游戏集上得分升至 38.3%，且输出 token 减少 6 倍。该实验表明 ARC-AGI-3 衡量的是完整智能体系统，Opus 5 仍为官方领先者。

## 正文

Is GPT-5.6 Sol now better than Opus 5 on ARC-AGI-3？

Short answer： not on the official leaderboard， but the comparison is more complicated than it first appeared. let me explain， because there is a bit of confusion.

ARC-AGI-3 tests whether models can learn unfamiliar 2D games through trial and error. We know that.

The "harness"is the software running that test. It shows the model the current game state， receives its answer， extracts and executes the chosen action， returns the next state and manages everything the model can remember.

ARC uses an intentionally generic harness called benchmark_agent. All providers are tested through the same completions-style setup. Previous visible answers and actions remain in the conversation， but the models' private reasoning is not passed into the next request.

Once the history exceeds roughly 175，000 characters， ARC uses rolling truncation： the oldest interactions are simply deleted.

Under this standard harness：
Opus 5： 30.2%
GPT-5.6 Sol： 7.8%

OpenAI argues that this setup particularly hurts Sol. Its reasoning disappears after every move， and eventually its earliest observations disappear too.

OpenAI therefore tested Sol through its own Responses API with retained reasoning and Compaction. Compaction compresses older parts of a long interaction into a shorter representation instead of deleting them entirely. The model can preserve learned rules， previous discoveries and its strategy across multiple context windows.

On the public games， Sol's score rose from 13.3% to 38.3%， while using 6x fewer output tokens.

That does not prove Sol beats Opus 5： 38.3% comes from the public set with OpenAI's custom harness； Opus 5's 30.2% is an official semi-private result using ARC's standard harness.

Opus 5 remains the official leader. But the experiment shows that ARC-AGI-3 measures the complete agent system， model， memory， context management and harness， not only the underlying model.

### 引用推文

> Tibo：Turns out GPT-5.6 Sol is actually SoTA on ARC-AGI-3. Just took two setting changes. You just have to allow it to reason and work over multiple context windows w...