从今天起，Ploy 的智能体运行在 GPT-5.6 Sol 上，这是 OpenAI 今早发布的模型系列中的旗舰层级。几个月来，鉴于我们对质量有着极高的标准，一直找不到能挑战 Claude Opus 的模型。这种情况随着 GPT-5.6 Sol 的出现而改变。在与 Claude Opus 进行正面交锋测试后，我们已将 GPT-5.6 Sol 设为驱动每个 Ploy 工作区的默认模型。
这比听起来要重大得多。Ploy 的智能体构建并编辑真实的营销网站。它规划页面、读取代码库、编写组件、生成图像、截取自己工作的截图，并判断何时完成。这份工作描述为模型设定了非常高的标准，我们针对每一次前沿发布都进行测试。在 Opus 占据默认位置的四个月里（先是 Opus 4.7，然后是 4.8），我们测试过的任何模型都无法击败它。GPT-5.6 是第一个做到的模型。
这并非说首次评估运行就完美无缺。它确实存在一些失败模式，我们会向你展示。但它的表现极其出色，其前景立竿见影且具体明确：构建完成所需的挂钟时间缩短一半以上，成本降低 27%，在已完成工作上的评分达到或超过我们现有的模型。这样的数据足以让一个模型获得真正的迁移努力。
尽管使用了 Vercel 的 AI SDK（一个通用的 LLM SDK），但从 Claude Opus 4.8 切换到 GPT-5.6 Sol 仍需我们通过一次次评估失败来发现：我们认为是“模型”的那些东西，其实是整个技术栈已经悄然适配的、特定于提供商的行为——比如它如何填充工具参数、它的提示缓存如何工作、以及它如何在轮次之间重放自己的推理过程。
以下是需要做的事情：修复评估框架，然后是工具模式，接着是缓存，最后是推理重放。
第 0 步：在信任任何一个数字之前，先修复你的评估框架。
我们的评估套件让真实智能体在真实固定工作区中运行。涵盖数百个案例，从"从头构建一个首页"到"这个克隆请求是否安全可执行"。构建案例通过一个视觉评判器进行评分，该评判器对照参考设计运行二元检查，包含十个是/否问题，例如"主视觉区域是全幅摄影场景"或"主要行动号召按钮是圆角矩形而非胶囊形状"，此外还有内容检查、工具轨迹检查和文件断言。每个失败的案例都会根据其完整追踪记录进行分类：包括实际的工具调用和模型文本，而不仅仅是分数。
在两个模型系列上运行该套件，其带来的惊喜超过了任何单个结果：
你的测试框架是针对你当前使用的模型调优的，而你自己并未意识到这一点。我们的工具调用预算原本是针对 Opus 的顺序式风格设定的；GPT-5.6 则并行发出调用，并在它正确解决的案例上超出了预算。我们的评估执行器不支持批量文件读取，Opus 很少使用这个功能，而 GPT-5.6 则频繁使用。在首次跨模型运行中，大约三分之一的原始失败原因可追溯到测试框架的假设，而非模型行为，并且这些失败在两个模型之间的分布并不均匀。如果你正在用当前模型评估一个挑战者模型，在相信通过率之前，请先对追踪记录进行分类。否则，你实际上是在评判新模型模仿旧模型的能力有多好。
确保在评估中公平地给模型打分。一个数据集如果省略了其 minScore 阈值，就会静默地继承默认值 1.0，因此 GPT-5.6 在一个它得了 0.98 分的主视觉区域上"失败"了，而 Opus 则在一个通过了所有单项检查的案例上"失败"了。两种都可辩护的设计方向；一个不可见的阈值。
第一印象：立即展现出潜力
在清理了测试框架之后，以下是我们重新设计套件中的一个样本，其中智能体根据参考设计重建一个品牌的首页：
| 每次完成构建的平均值 | Claude Opus 4.8 (n=11) | GPT-5.6 (n=10) |
|---|---|---|
| 成本 | $3.06 | $2.22 |
| 挂钟时间 | 8 分 00 秒 | 3 分 42 秒 |
| 输入 token | 2.60M | 1.70M |
| 输出 token | 33.0K | 17.1K |
| 视觉评分 | 0.936 | 0.970 |
这就是承诺的样貌：完成一个成品页面速度快 2.2 倍，成本低 27%，输出 token 数量大约减半。GPT-5.6 编写的代码很精简。在一组对照中，Opus 生成了一个 17,957 字符的 globals.css 文件，内含 174 个 CSS 变量（完整的色阶，大部分未使用），而 GPT-5.6 只写了 2,508 个字符和 45 个变量，就实现了效果相当（有时甚至更好）的渲染页面。
Claude Opus 4.8
GPT-5.6 Sol
设计：锐利、干净，但略显千篇一律
我们对 GPT-5.6 设计工作的整体评价是：它在干净、现代、紧密网格布局方面表现出色，但除非你给予很好的引导，否则它容易趋同于那种外观。在我们为 Opus 4.8 设计的旧测试框架下，GPT 5.6 Sol 倾向于忽略现有的设计系统，转而生成锐利、克制但明显千篇一律的输出。
我们如何解决这个问题的细节，值得单独写一篇博文。凭借我们设计和工程团队的专业知识，我们能够引导模型实现开箱即用无法获得的、世界级的品牌一致性。
第一步：检查你的工具调用
这就是在我们发现之前，一直在暗中破坏结果的那个问题。
我们智能体的代码工具有 25 个顶层参数，其中一个是必填项（action），其余都是可选的。Claude 只发送它用到的两三个参数，并省略其余参数。GPT-5.6 每次都会发送全部 25 个参数，并为它不需要的参数凭空编造出看似合理的值：offset: 0, timeout: 120000, siteId: "00000000-0000-0000-0000-000000000000"。
三天生产环境追踪数据，code(read) 调用携带了所有属性：
| 模型 | 调用次数 | 携带全部 25 个属性 |
|---|---|---|
| gpt-5.6 | 6,635 | 6,635 (100%) |
| claude-opus-4.8 | 2,898 | 4 (0.1%) |
| claude-sonnet-5 | 1,933 | 0 |
问题不在于冗长。而在于一个编造的值与一个有意传入的值无法区分。offset: 0 看起来像一个真实的参数。我们的文件读取实现将其视为真实参数，结果导致 GPT-5.6 有 52% 到 64% 的文件读取返回空内容。该工具在两种情况下都返回 success: true，因此模型无法知道自己读取的是空白文件。它只是用更多的调用次数，完成了更差的工作。
提示词无法解决这个问题。在工具描述指令中加上“省略未使用的参数”：结果仍是 25/25。在每个属性上添加“可选，未使用则省略”的提示：结果还是 25/25。OpenAI 的严格模式：行为完全一致（我们实测过），而且如果采用该模式，我们就必须从每个 schema 中移除 pattern、format 和数组边界校验。这个问题根植于模型输出函数调用的底层机制。你无法通过指令消除它，只能围绕它来设计。
有效的解决方案是在供应商边界处进行 schema 转换。仅针对 OpenAI 系列模型，我们将每个可选属性改为必填但可为空，使用 anyOf: [T, null] 格式，这样模型就有了明确的表达方式来表示“不使用此参数”。然后，在每个工具调用必经的唯一接口处，我们在校验之前将 null 值剔除，因此工具实现完全无需改动。整个流程：模型看到的 schema 允许它诚实表达意图；工具看到的输入则与以往完全相同。
// Before: 25 keys, every one carrying an invented value
{ "action": "read", "file_paths": [...], "offset": 0, "timeout": 120000, ... }
// After: 25 keys, 4 real values, 21 explicit nulls (stripped before the tool runs)
{ "action": "read", "file_paths": [...], "offset": null, "timeout": null, ... }结果：空文件读取从 52% 降至 0%，智能体完成相同工作所需的工具调用次数减少了约 30%，因为它不再重复读取返回空白内容的文件。
第二步：重建提示词缓存
这是最具启发性的工程差异，因为表面上两家供应商都提供“提示词缓存”，但这个词背后隐藏着两种完全不同的设计。如果你要谨慎迁移一件事，那就是这个：在我们调整之前，GPT-5.6 看起来比 Opus 贵了约 50%。问题不在于模型定价，而在于我们的缓存配置。
我们智能体的提示词以一个约 29K token 的静态前缀开头（工具 schema 加核心系统提示词），该前缀在所有对话中完全相同。在 Claude 上，我们使用 cache_control 标记缓存断点，这个前缀会在整个组织范围内缓存：任何对话、任何工作空间，共享一个缓存条目，无需考虑吞吐量预算。缓存命中率可达 92% 到 96%，缓存机制几乎无需操心。
GPT-5.6 悄然改变了 OpenAI 的缓存模型，让我们措手不及。早期的 GPT 模型会对部分前缀匹配进行隐式缓存，从而免费获得不错的命中率。GPT-5.6 取消了部分前缀匹配：现在的隐式缓存仅以最新消息为键，创建整个提示词的缓存条目。一个共享我们 29K 静态前缀的新对话，其缓存命中率为 0%。每个对话都会按未缓存费率重新计收整个前缀的费用，而且在 GPT-5.6 上，无论你是否使用缓存，每个未缓存的提示词还需支付 1.25 倍的缓存写入附加费。
其预期的机制是显式的：`prompt_cache_breakpoint` 标记加上一个强制性的 `prompt_cache_key`。而这个键正是设计上真正产生分歧的地方，因为它成为了缓存标识的一部分。相同的提示词，不同的键：缓存命中率为零。每个键映射到一个缓存节点，该节点在 OpenAI 将流量分散到其他拥有独立、冷缓存的节点之前，大约能维持每分钟 15 次请求。
这就把“启用缓存”变成了一个实际的设计决策：你打算将键的作用域限定在哪个实体上？
- 按对话设置键意味着新对话永远无法命中共享前缀。首次调用的命中率：0%。（我们测量过这个错误。代价非常高昂。）
- 使用一个全局键意味着每个请求都会哈希到同一个缓存节点，而生产流量会瞬间耗尽每分钟 15 次的预算；请求会溢出到冷节点，你又回到了未命中状态。
- 按工作空间设置键是最佳方案。同一客户工作空间内的所有对话共享缓存条目；每个键的流量保持在较低水平。
我们推出了工作空间作用域的键，并将系统提示词拆分为带断点的层级，镜像了我们已用于 Anthropic 的结构：
request ──► hash(prompt head + prompt_cache_key) ──► cache node (~15 req/min per key)
│
┌──────────────────────────────────────────────────────┴───────────────┐
│ entries on the node, all namespaced by key ws:{workspaceId} │
│ │
│ [ tools + static prefix ]······················ A every session │
│ [ tools + static prefix + workspace context ]·· B same context │
│ [ ····················· + turn 1 + … + latest ] C this session │
└──────────────────────────────────────────────────────────────────────┘条目 A 使得会话的首次调用成本低廉。条目 B 能够自我修复：当工作空间记忆发生变化时，请求会错过 B，但仍能命中 A，然后写入一个新的 B。只需一次上下文大小的写入，而无需重新计收完整的 29K 费用。条目 C 是 OpenAI 的隐式完整提示词链，由于我们的提示词严格采用追加模式，因此它在会话内运行良好。
一个后果没有变通办法：在 OpenAI 上，跨工作区共享静态前缀在结构上是不可能的。Anthropic 可以共享它，因为它的缓存是组织范围的，没有键分区。在 GPT-5.6 上，每个工作区在每个空闲窗口都要支付一次 29K 的冷写入费用，大约 0.18 美元。这是一个真实的成本，但有上限且可预测。
变更后的结果：首次调用的缓存命中率从大约 0% 提升到了 83.7%，未缓存的输入 token 总量下降了 28%，并且 GPT-5.6 的每套成本低于 Opus。我们一直盯着的每一个美元的成本差距都是缓存配置错误造成的，而不是模型定价问题。如果你在比较模型的成本，而其中一个模型缓存是冷的，那你实际上是在比较你的配置，而不是模型本身。
第三步：让推理回放自包含
更短了，但它破坏了真实的对话。GPT-5.6 的 Responses API 默认将前一轮的推理作为服务器端项目引用进行回放；我们的对话中途开始间歇性地失败，报错“项目 'rs_...' 未找到”。修复方法是设置 `store: false`，这会让 SDK 请求加密的推理内容，并回放自包含的数据块，而不是指向服务器状态的指针。一个让我们花了一个下午调试的推论是：当服务器端推理状态参与循环时，即使你发送的字节是只追加的，有效的提示词也可能在你上游发生变化。
GPT 5.6 Sol 已准备好部署
亲自试试吧。GPT-5.6 今天发布，并且已经在 Ploy 上上线。你现在就可以免费试用：给它一个网站去构建，看看不到四分钟的构建过程是什么样子。在 ploy.ai 免费开始使用。
Ploy 是自动运行的营销：一个 AI 层，端到端地规划、构建、发布和优化你的网站及营销活动。如果在凌晨两点调试缓存节点扇出听起来很有趣，我们正在招聘。
As of today, Ploy’s agent runs on GPT-5.6 Sol, the flagship tier of the model family OpenAI released this morning. For months, we couldn’t find a model that challenges Claude Opus given our incredibly high bar for quality. That changed with GPT 5.6 Sol. After running it head-to-head against Claude Opus, we’ve made GPT 5.6 Sol the default model powering every Ploy workspace.
That’s a bigger switch than it sounds. Ploy’s agent builds and edits real marketing websites. It plans a page, reads the codebase, writes components, generates imagery, screenshots its own work, and decides when it’s done. That job description sets a very high bar for a model, and we test every frontier release against it. For the four months Opus held the default slot (first Opus 4.7, then 4.8), nothing we tested beat it. GPT-5.6 is the first model that did.
Not that the first eval run was perfect. It had real failure modes, which we’ll show you. But it did extremely well, and the promise was immediate and specific: builds finishing in less than half the wall-clock time, at 27% lower cost, scoring at or above our incumbent on completed work. Numbers like that buy a model a real migration effort.
Despite using Vercel’s AI SDK, a universal LLM SDK, switching from Claude Opus 4.8 to GPT 5.6 Sol required discovering, one eval failure at a time, that the things we think of as “the model” are provider-specific behaviors our whole stack has quietly specialized around: how it fills in tool arguments, how its prompt cache works, how it replays its own reasoning between turns.
Here’s what it took: fix the eval harness, then the tool schemas, then caching, then reasoning replay.
Step 0: Fix your harness before you trust a single number
Our eval suite runs the real agent against real fixture workspaces. Hundreds of cases, from “build a homepage from scratch” to “is this clone request safe to execute.” Build cases are scored by a visual judge running binary checks against a reference design, ten yes/no questions like “the hero is a full-bleed photographic scene” or “primary CTAs are rounded rectangles, not pills”, plus content checks, tool-trajectory checks, and file assertions. Every failed case gets triaged against its full trace: the actual tool calls and model text, not just the score.
Running that suite across two model families surprised us more than any individual result:
Your harness is tuned to your incumbent model, and you don’t know it. Our tool-call budgets were sized for Opus’s sequential style; GPT-5.6 fans out parallel calls and blew through them on cases it was solving correctly. Our eval executor didn’t support batched file reads, which Opus rarely used and GPT-5.6 uses constantly. Roughly a third of the raw failures in the first cross-model run traced back to harness assumptions, not model behavior, and they were not evenly distributed between the models. If you’re evaluating a challenger model against an incumbent, triage the traces before you trust the pass rate. Otherwise you’re grading the new model on how well it imitates the old one.
Make sure you’re grading models fairly in evals. A dataset that omitted its minScore threshold silently inherited a default of 1.0, so GPT-5.6 “failed” a hero it scored 0.98 on, and Opus “failed” a case while passing every individual check. Two defensible design directions; one invisible threshold.
First impression: immediately promising
With the harness cleaned up, here’s a sample from our redesign suite, where the agent rebuilds a brand’s homepage against a reference design:
| Mean per completed build | Claude Opus 4.8 (n=11) | GPT-5.6 (n=10) |
|---|---|---|
| Cost | $3.06 | $2.22 |
| Wall-clock time | 8m 00s | 3m 42s |
| Input tokens | 2.60M | 1.70M |
| Output tokens | 33.0K | 17.1K |
| Visual score | 0.936 | 0.970 |
This is the shape of the promise: 2.2× faster to a finished page, 27% cheaper, and about half the output tokens. GPT-5.6 writes lean code. On one matched pair, Opus produced a 17,957-character globals.css with 174 CSS variables (full color ramps, mostly unused) where GPT-5.6 wrote 2,508 characters and 45 variables for a comparable (and sometimes better) rendered page.
Claude Opus 4.8
GPT-5.6 Sol
Design: sharp, clean, but a little bit uniform
Our overall read on GPT-5.6’s design work: it is very good at clean, modern, tightly-gridded layouts, but it tends to converge towards that look unless you steer it well. With our older harness designed for Opus 4.8, GPT 5.6 Sol tends to ignore existing design systems and instead produces sharp, restrained, and visibly generic output.
The details of how we fixed this are worth a separate blog post of its own. With the expertise of our design and engineering teams, we are able to steer models to achieve world-class brand adherence that you can’t get out of the box.
Step 1: Check your tool calls
Here’s the one that was silently corrupting results before we caught it.
Our agent’s code tool has 25 top-level parameters, one required (action) and the rest optional. Claude sends the two or three it’s using and omits the rest. GPT-5.6 sends all 25, every time, inventing plausible values for the ones it doesn’t need: offset: 0, timeout: 120000, siteId: "00000000-0000-0000-0000-000000000000".
Three days of production traces, code(read) calls carrying every property:
| Model | Calls | Carrying all 25 properties |
|---|---|---|
| gpt-5.6 | 6,635 | 6,635 (100%) |
| claude-opus-4.8 | 2,898 | 4 (0.1%) |
| claude-sonnet-5 | 1,933 | 0 |
The problem isn’t verbosity. It’s that an invented value is indistinguishable from an intended one. offset: 0 looks like a real argument. Our file-read implementation treated it as one, and 52% to 64% of GPT-5.6’s file reads were coming back empty because of it. The tool returned success: true both ways, so the model had no way to know it was reading blank files. It just did the work worse, with more calls.
Prompting doesn’t fix this. A tool-description directive to “omit unused parameters”: still 25/25. Per-property “OPTIONAL, omit if unused” hints: still 25/25. OpenAI’s strict mode: identical behavior (we measured), and adopting it would have forced us to strip pattern, format, and array-bound validation from every schema. This is baked into how the model emits function calls. You don’t instruct it away; you design around it.
The fix that worked is a schema transform at the provider boundary. For OpenAI-family models only, we rewrite every optional property to be required but nullable, using anyOf: [T, null], which gives the model an explicit way to say “not using this.” Then, at the single seam every tool invocation passes through, we strip the nulls back out before validation, so no tool implementation changes at all. Round trip: the model sees a schema where honesty is expressible; the tools see the same inputs they always did.
// Before: 25 keys, every one carrying an invented value
{ "action": "read", "file_paths": [...], "offset": 0, "timeout": 120000, ... }
// After: 25 keys, 4 real values, 21 explicit nulls (stripped before the tool runs)
{ "action": "read", "file_paths": [...], "offset": null, "timeout": null, ... }Results: empty file reads went from 52% to 0%, and the agent needed roughly 30% fewer tool calls for the same work, because it was no longer re-reading files that came back blank.
Step 2: Rebuild prompt caching
This was the most instructive engineering difference, because on the surface both providers offer “prompt caching” and the words hide two entirely different designs. If you migrate one thing carefully, make it this: before we did, GPT-5.6 looked about 50% more expensive than Opus. It wasn’t the model’s pricing; it was our cache configuration.
Our agent’s prompt opens with a static prefix of roughly 29K tokens (tool schemas plus the core system prompt) that’s identical for every conversation. On Claude, we mark cache breakpoints with cache_control and that prefix caches across the whole organization: any conversation, any workspace, one shared entry, no throughput budget to think about. Cache hit rates run 92% to 96% and caching fades into the background.
GPT-5.6 changed OpenAI’s caching model out from under us. Earlier GPT models cached implicitly on partial prefix matches, which gave decent hit rates for free. GPT-5.6 dropped partial-prefix matching: implicit caching now only creates whole-prompt entries keyed on the latest message. A new conversation sharing our 29K static prefix cached 0% of it. Every conversation re-billed the full prefix at the uncached rate, and on GPT-5.6 every uncached prompt also pays a 1.25× cache-write surcharge, whether or not you use caching.
The intended mechanism is explicit: prompt_cache_breakpoint markers plus a mandatory prompt_cache_key. And the key is where the design really diverges, because it’s part of cache identity. Identical prompt, different key: zero cache hits. Each key maps to a cache node that sustains roughly 15 requests per minute before OpenAI fans traffic to other nodes with independent, cold caches.
That turns “enable caching” into an actual design decision: what entity do you scope the key to?
- Per-conversation key means a new conversation never hits the shared prefix. First-call hit rate: 0%. (We measured this mistake. It’s expensive.)
- One global key means every request hashes to one cache node, and production traffic obliterates the 15 rpm budget; requests spill to cold nodes and you’re back to misses.
- Per-workspace key is the sweet spot. All conversations in a customer workspace share entries; per-key traffic stays low.
We ship the workspace-scoped key and split the system prompt into breakpointed layers, mirroring the structure we already used for Anthropic:
request ──► hash(prompt head + prompt_cache_key) ──► cache node (~15 req/min per key)
│
┌──────────────────────────────────────────────────────┴───────────────┐
│ entries on the node, all namespaced by key ws:{workspaceId} │
│ │
│ [ tools + static prefix ]······················ A every session │
│ [ tools + static prefix + workspace context ]·· B same context │
│ [ ····················· + turn 1 + … + latest ] C this session │
└──────────────────────────────────────────────────────────────────────┘Entry A is what makes a session’s first call cheap. Entry B self-heals: when workspace memory changes, the request misses B but still hits A, then writes a fresh B. One context-sized write instead of a full 29K re-bill. Entry C is OpenAI’s implicit whole-prompt chain, which works fine within a session because our prompts are strictly append-only.
One consequence has no workaround: cross-workspace sharing of the static prefix is structurally impossible on OpenAI. Anthropic can share it because its cache is org-scoped without key partitioning. On GPT-5.6, every workspace pays one 29K cold write per idle window, about $0.18. A real cost, but bounded and predictable.
Results after the change: first-call cache hits went from roughly 0% to 83.7%, total uncached input tokens dropped 28%, and GPT-5.6’s per-suite cost landed below Opus’s. Every dollar of the gap we’d been staring at was cache misconfiguration, not model pricing. If you’re cost-comparing models and one of them has a cold cache, you are comparing your config, not the models.
Step 3: Make reasoning replay self-contained
Shorter, but it broke real conversations. GPT-5.6’s Responses API replays prior-turn reasoning as server-side item references by default; ours started intermittently failing mid-conversation with Item 'rs_...' not found. The fix is store: false, which makes the SDK request encrypted reasoning content and replay self-contained blobs instead of pointers to server state. A corollary that cost us a debugging afternoon: with server-side reasoning state in the loop, the effective prompt can change upstream of you even when the bytes you send are append-only.
GPT 5.6 Sol is ready to Ploy
Try it yourself. GPT-5.6 launched today, and it’s already live on Ploy. You can try it for free right now: give it a website to build and see what a sub-four-minute build looks like. Start free at ploy.ai.
Ploy is marketing run on autopilot: an AI layer that plans, builds, publishes, and optimizes your website and campaigns end-to-end. If debugging cache-node fan-out at 2am sounds like fun, we’re hiring.