AYi@AYi_AInotes · 6月23日65

所有大模型排行榜都在骗你。 Cline团队用自己仓库的真实bug，在完全相同的环境下，测了GLM-5.2和Claude Opus 4.8。 结果非常打脸。 Opus速度快3倍，token消耗少一半，价格贵一倍。 它修完了bug，跑通了所有测试。 但生产构建直接崩了，留下了未被发现的类型错误。 GLM速度慢，token多67%，工具调用多2.3倍，价格便宜一半。 它不仅修好了bug，还主动清理了死代码。 最终构建干净通过，没有任何隐患。 这就是排行榜和真实世界的差距。 SWE-bench只能测出能不能修bug。 测不出修完之后会不会偷偷搞崩你的生产环境。 测试过了不等于代码能用。 这在大型项目里，是致命的。 本质不是谁更聪明，因为训练目标完全不一样。 GLM被强化学习训练出了验证文化。 多花的token，全用在了跑构建，查类型，清垃圾，防回归上。 它不是笨，是负责任。 Opus追求高效交差，GLM追求一次做对。 更值得注意的是，这是开源模型。 它不再只是闭源模型的廉价替代品。 它在长周期代码智能体的维度上，找到了自己的差异化优势。 智能体时代的性价比逻辑彻底变了。 以前比每千token多少钱。 现在比每次成功任务多少钱。 多花点token一次做对。 永远比快但要返工两次更划算。 更别说省下的人工排查成本。 给所有做智能体的人两个建议， 第一，别信排行榜，拿自己仓库的真实bug跑一遍。 第二，在你的系统提示里强制加一条，完成前必须跑构建验证，清理死代码。 未来比拼的从来不是谁的模型更聪明，而是看谁的模型更负责任。

译Cline团队用自家仓库真实bug测试GLM-5.2和Claude Opus 4.8。Opus速度快3倍（1.6分钟vs 4.7分钟）、token少一半（660K vs 1.1M）、价格贵一倍（$0.81 vs $0.41），修好bug但生产构建崩溃，留下类型错误。GLM多花67% token、多2.3倍工具调用（28次vs 12次）、价格便宜一半，不仅修好bug还主动清理死代码，构建干净通过。根本差异在于训练目标：GLM被强化学习训练出验证文化，多花token跑构建、查类型、防回归；Opus追求高效却忽略隐患。排行榜只测修bug能力，测不出是否破坏生产环境。开源模型GLM在长周期代码智能体上找到差异化优势。

Rohan Paul@rohanpaul_ai · 6月23日50

Sakana Fugu Ultra just beat the other models on visual polish in a live trading-desk coding test, got close to GLM 5.2, but at 17x the cost. Test was done on atomic[.]chat, a desktop app that runs LLMs locally. Fugu produced the richest interface, with multiple panels, watchlists, charts, tape-style activity, status labels, and a more finished product feel. To note that Fugu Ultra is an orchestration layer that assembles and routes subtasks across a pool of models through one OpenAI-compatible endpoint. So Fugu is a learned coordinator model inside a multi-agent system. When you send a prompt, Fugu decides whether to answer alone or hand pieces of the job to other models, then it gathers the outputs and produces one final response.

译Sakana Fugu Ultra 是一个多智能体协调层，通过 OpenAI 兼容端点将子任务路由给不同模型。在 @atomic_chat_hq 的实时交易桌面编码测试中（要求构建完整前后端、实时 API 数据、暗色主题 UI），Fugu Ultra 生成了最丰富的多面板界面（含图表、状态标签等），效果接近 GLM 5.2，但成本达后者的 17 倍：Fugu Ultra 耗 22,225 tokens / $0.51，GLM 5.2 耗 13,677 tokens / $0.03。对比模型 Opus 4.8（15,802 t / $0.31）和 GPT-5.5（11,474 t / $0.26）在质量与成本平衡上表现更佳。

Chubby♨️@kimmonismus · 6月23日57

Absolutely incredible: GLM-5.2 (max) sits at #3 overall on GDPval-AA, a real-world agentic work benchmark, even ahead of GPT-5.5 (xhigh). Oh and btw: looks like open source is no longer 7 months behind. GDPval-AA, a benchmark built around real professional and creative tasks. The models had to produce practical deliverables from identical briefs, including a retail supervisor’s task list, an emergency-stop circuit schematic, and a music video moodboard. Thats why we'll probably see a big leap with GPT-5.6. Even open source competition is catching up insanley fast.

译GLM-5.2（max）在真实世界智能体工作基准 GDPval-AA 上获 1524 Elo，排名第三，仅次于 Claude Fable 5（1783）和 Claude Opus 4.8（1615），与 GPT-5.5（xhigh，1509）持平。该模型以约 31 轮次任务平均完成零售主管任务清单、紧急停止电路图等交付物，领先开源权重模型（下一名 MiniMax-M3 仅 1408），并超过 Google Gemini 3.5 Flash（1357）、Qwen 3.7 Max（1289）等闭源模型。GLM-5.2 同时在 Artificial Analysis Intelligence Index、Agentic Index 和 AA-Briefcase 上领跑开源榜单。

Artificial Analysis@ArtificialAnlys · 6月23日60

Open weights models make up the majority of the cost-performance Pareto frontier on AA-Briefcase, our new agentic knowledge work benchmark Last week we released AA-Briefcase, our proprietary agentic knowledge work benchmark testing models on long horizon tasks built by industry experts. AA-Briefcase requires models to build deliverables such as financial models, board presentations, and design mock-ups in the context of realistic multi week projects. The cost to run a single AA-Briefcase task varies by over 700x in the initial set of models we tested. With the highest performing model, Claude Fable 5, costing over $20 per task, cost efficiency is a key element in model selection for knowledge work. While the two highest performing models on the cost-performance Pareto frontier are proprietary models from @AnthropicAI, most of the remaining frontier is made up of open weights models. Notable cost efficiency trade offs: ➤ At $2.40 per task, GLM 5.2 (max) from @Zai_org scores within 90 Elo points of Claude Opus 4.8 (max) while costing 65% less ➤ At $0.08 per task, DeepSeek V4 Pro (max) from @deepseek_ai scores ~60 Elo points above Gemini 3.5 Flash while costing over 98% less

译Artificial Analysis发布AA-Briefcase智能体知识工作基准测试，评估模型在长期任务中的表现。任务成本差异超700倍，最高性能模型Claude Fable 5每任务超$20。成本-性能帕累托前沿上，除Anthropic两个最高分模型外，其余大部分由开放权重模型占据。关键性价比：GLM 5.2 (max)每任务$2.40，得分仅比Claude Opus 4.8低90 Elo，成本低65%；DeepSeek V4 Pro (max)每任务$0.08，得分比Gemini 3.5 Flash高约60 Elo，成本低98%以上。

Chubby♨️@kimmonismus · 6月23日59

GLM 5.2 keeps on winning GLM 5.2 is emerging as the first open-weights model capable of handling meaningful autoresearch tasks, from debugging setup issues to running and comparing RL training experiments across multi-node H100 clusters. The big caveat: it lacks image understanding, so unlike Opus or Fable, it has to analyze raw WandB data programmatically rather than visually interpreting charts. Still: while we are waiting for the come back of. Fable 5, zAI really nailed it with GLM 5.2

译Kim指出，GLM 5.2是首个能以开放权重处理真实自动研究任务的模型，包括调试设置、跨多节点H100集群运行并比较RL训练实验。其局限在于缺少图像理解能力，需程序化分析原始WandB数据而非可视化图表。引用介绍称，GLM 5.2是其自动研究pipeline上首个能胜任实际研究的开源模型，在Fable 5对研究设限的背景下意义重大。演示中，它基于SkyRL在两台8×H100节点上完成Harbour代码竞赛的完全异步vs同位置同步RL训练，自动解决设置问题并生成吞吐量与奖励稳定性对比。

Artificial Analysis@ArtificialAnlys · 6月23日59

GLM-5.2 leads open weights models and sits at #3 overall on GDPval-AA, a real-world agentic work benchmark GLM-5.2 from @Zai_org scores 1524 Elo on GDPval-AA, which measures performance on real-world, economically valuable knowledge work through long-horizon, multi-turn tasks. Key takeaways: ➤ #3 overall, behind only Claude Fable 5 (1783) and Claude Opus 4.8 (1615), and level with GPT-5.5 (xhigh, 1509) ➤ The leading open weights model by a wide margin: the next open model, MiniMax-M3, scores 1408 ➤ Ahead of many proprietary models, including Google's Gemini 3.5 Flash (1357), Qwen 3.7 Max (1289), Muse Spark (1158) ➤ The tasks are agentic. GLM-5.2 averaged ~31 turns per task across 1,999 matches ➤ Consistent with the rest of its launch, GLM-5.2 also leads open weights on the Artificial Analysis Intelligence Index, ranks #3 on the Agentic Index, and #3 on AA-Briefcase

译智谱 AI 的 GLM-5.2 在真实世界智能体工作基准 GDPval-AA 上获得 1524 Elo，排名第三，仅次于 Claude Fable 5 和 Claude Opus 4.8，与 GPT-5.5 持平。它是开源权重模型中领先的，超越 Gemini 3.5 Flash、Qwen 3.7 Max 等专有模型。任务为智能体型，平均每任务约 31 轮。此外，GLM-5.2 在 Artificial Analysis Intelligence Index 上也领先开源权重，并在 Agentic Index 和 AA-Briefcase 上均排名第三。

Ethan Mollick@emollick · 6月23日64

I have been trying Sakana Fugu Ultra-high and, first, it is incredibly slow: my typical coding tests (shaders, interactive scenes) take 30 minutes to run And the results are... fine. It does not match Fable in real use. Its harbor is a good example: https://ai-harbor-town-gallery.netlify.app/#sakura-ultra-high

译宾大教授Ethan Mollick实测Sakana Fugu Ultra-high模型，指出其速度极慢——典型编码测试需30分钟，实际效果仅“fine”，未能匹配此前Sakana官方宣称的“与Fable和Mythos性能相当”。Mollick表示，在真实编码场景中Fugu Ultra远不及Fable，并附上AI港口小镇生成样例链接作为例证。

Artificial Analysis@ArtificialAnlys · 6月23日50

Announcing the Artificial Analysis Video Editing Arena - compare video editing across Seedance 2.0, Runway Aleph 2.0, Wan 2.7, HappyHorse 1.0, Kling 3.0 Omni, and SkyReels V4. Voting open now and leaderboard coming soon Editing short video clips based on text instructions has recently emerged as a capability of frontier video generation models. We’re adding this capability to our video model benchmarking, treating it as a separate category similar to how we benchmark Image Editing. Our Video Editing Arena tests models’ capability to follow editing instructions for videos with and without audio. We focus on the editing capabilities important to creative industry professionals where models differ most, including visual effects editing, sound & speech editing, object editing and physics simulation. Our initial analysis already reveals clear differences in model strengths by editing capability. We hope the launch of our Video Editing Arena will help users and developers understand the strengths of today’s video editing models and make the best use of them. The full lineup at launch: @dreamina_ai Seedance 2.0, @runwayml Aleph 2.0, @Alibaba_Wan Wan 2.7, @HappyHorseATH HappyHorse-1.0, @Kling_ai 3.0 Omni, @SkyReels SkyReels V4, with more to come. Voting is open now and we'll announce the first leaderboard results within 24 hours. Any predictions?

译Artificial Analysis 推出 Video Editing Arena，用于对比 Seedance 2.0、Runway Aleph 2.0、Wan 2.7、HappyHorse 1.0、Kling 3.0 Omni、SkyReels V4 六款模型的短片段编辑能力。测试基于文本指令，覆盖有/无音频场景，聚焦视觉特效、声音/语音编辑、物体编辑和物理模拟等维度。初始分析已揭示各模型编辑能力的显著差异。投票现已开放，首个排行榜将在 24 小时内公布。

meng shao@shao__meng · 6月22日30

给大家更新一下国产四个模型的投票结果： GLM-5.2 - 79.7% DeepSeek V4 - 13.4% Kimi K2.7 - 5.2% MiniMax M3 - 1.7% GLM-5.2 这么强，MiniMax M3 这么惨吗？

译邵猛更新四个国产LLM投票：GLM-5.2获79.7%，DeepSeek V4 13.4%，Kimi K2.7 5.2%，MiniMax M3仅1.7%。他感叹GLM-5.2强势而MiniMax M3惨淡，并指出Gemini 3.5 Flash相比GLM-5.2不能打，Google DeepMind自Gemini 3.0后陷入沉寂。

Peter Steinberger 🦞@steipete · 6月22日39

I was skeptical about the multi-model routing. Seems my hinch was right.

译@LLMJunky 实测某多模型路由服务，5小时使用额度在1个prompt内即告罄。在threejs任务（构建Rocket League副本）中，生成效果远差于GPT 5.5，需7-8次来回通过Codex修复才勉强可玩；GPT 5.5一次完成且无需后续调整，Fable同样表现出色。该路由性能不及Mythos，早期印象不佳。

karminski-牙医@karminski3 · 6月22日54

国产模型最近这一波打得太精准了 DeepSWE benchmark 放出了榜单, 从分数上看, GLM-5.2 无疑是国产编程大模型SOTA了, 而 Kimi-K2.7-Code 则是性价比SOTA. 图上几个折线是对应模型的不同推理程度的得分情况. DeepSWE 我觉得是最近非常值得的关注的的榜单了, 这个虽然也叫xxSWE, 但是它用的不是像SWE那种github上爬取的PR或者issue, 而是完全根据开源项目人工制造的问题, 然后让大模型修改. 能最大限度避免大模型提前炼这些问题(起码需要一段时间了, 5月份刚出的) 并且修改范围也很大,SWE-Bench-verified 通常一个提交也就几十行代码, 而 DeepSWE 每个问题都要上百行提交才能解决问题, 并且很考验模型的规划能力, SWE-Bench-Verified 通常会把所有需要的工具给到错误日志啥的也有, 并且提示词也写好告诉模型该怎么用. 而 DeepSWE 纯靠模型自己摸索, 非常像现实中修改项目代码的过程. 我觉得是涵盖了一部分工程能力测试的. 另外 SWE-Bench-verified 测得大部分都是 Python, 而 DeepSWE 则涵盖了TypeScript、Go、Python、JavaScript、Rust. 所以我强烈建议关注大模型编程能力测试的同学过关注一波这个测试. #DeepSWE #SWEBench #glm #kimi

译DeepSWE 基准测试发布榜单，GLM-5.2 为国产编程大模型 SOTA，Kimi-K2.7-Code 为性价比 SOTA。该榜单与 SWE-Bench-verified 不同：问题由人工针对开源项目制造，可避免数据泄露；每个问题需修改上百行代码，考验模型规划能力，且不提供工具指引，更贴近真实工程场景。覆盖 TypeScript、Go、Python、JavaScript、Rust 等多种语言，而非仅 Python。榜单于 5 月发布。

SiliconFlow@SiliconFlowAI · 6月22日58

GLM-5.2 topped @Designarena's HTML Web Design leaderboard — beating Claude Opus 4.6 & 4.7, the long-standing #1. It's amazing and already available via SiliconFlow API! Start building →https://www.siliconflow.com/models/glm-5-2

译GLM-5.2 登顶 @Designarena 的 HTML Web 设计排行榜——击败了长期保持第一的 Claude Opus 4.6 & 4.7。 非常出色，现已通过 SiliconFlow API 提供使用！ 开始构建 → https://www.siliconflow.com/models/glm-5-2

elvis@omarsar0 · 6月22日47

>> Scalable Evaluation for AI Agents << If you run agent evaluation in production, this one is worth your time. It shows that front-loading human judgment into reusable evaluation assets is useful. But why? Agents reason across turns, call tools, hold context, follow policies, and act under uncertainty, so they have to be judged as behavioral systems. Current methods each give a fragment. Benchmarks measure fixed capabilities, human review preserves judgment but does not scale, LLM-as-judge inherits the evaluator design problem, red teaming is episodic, and trace audits need explicit evidence rules. Human-on-the-Bridge puts human expertise upstream, where experts curate reusable evaluation intelligence before testing rather than reviewing each output in the loop. Paper: https://arxiv.org/abs/2606.16871 Learn to build effective AI agents in our academy: https://academy.dair.ai/

译论文《Scalable Evaluation for AI Agents》提出Human-on-the-Bridge评估方法：将人类判断前置到可复用评估资产中，专家在上游策划评估智慧，而非在测试循环中逐一审查输出。现有方法各有局限：Benchmark测量固定能力，人工审核不具可扩展性，LLM-as-Judge存在评估器设计问题，红队测试偶发，trace审计需明确证据规则。AI智能体需作为行为系统评估，因其多轮推理、调用工具、维护上下文、遵循策略并在不确定性下行动。

elvis@omarsar0 · 6月22日56

Very impressive from GLM-5.2. Frontier open-weight model indeed. Now, can we get a Gemini model in the top 3 soon?

译GLM-5.2 的表现令人印象深刻。 确实是前沿的开放权重模型。 那么，我们很快能看到 Gemini 模型跻身前三吗？

Nathan Lambert@natolambert · 6月22日47

An hour in and first impression is definitely that GLM is really solid (very easy to set up on @FireworksAI_HQ, props to them for that, took me like 5min to get going in claude code).

译一小时下来，第一印象绝对是GLM非常扎实（在@FireworksAI_HQ上设置非常简单，夸他们一下，我在claude code里只花了5分钟就搞定了）。

OpenCode@opencode · 6月20日48

GLM 5.2 is a hit been out for 3 days and it's already 6th on our leaderboard

译GLM 5.2 大受欢迎 已发布 3 天，便在我们榜单上排到第 6 名

Chubby♨️@kimmonismus · 6月20日68

Really looking forward to GPT-5.6. one shorting a „The Sims“-Simulation without any harness is nuts.

译非常期待 GPT-5.6。无需任何辅助框架，一次性生成《模拟人生》模拟，太疯狂了。

Yuchen Jin@Yuchenj_UW · 6月20日30

After using GLM-5.2 for a day, I’m surprised by how often it feels close to Opus 4.8/GPT-5.5 level. I compared it side by side with Opus 4.8, and sometimes I even preferred GLM-5.2’s results. OSS LLMs are impressive, especially given how many fewer GPUs they were trained on.

译使用 GLM-5.2 一天后，我惊讶于它经常感觉接近 Opus 4.8/GPT-5.5 的水平。 我将它与 Opus 4.8 进行了并排比较，有时我甚至更喜欢 GLM-5.2 的结果。 开源大语言模型令人印象深刻，尤其是考虑到它们训练的 GPU 数量少得多。

向阳乔木@vista8 · 6月20日16

gemini3还是不行啊，怎么把白条识别成泥鳅。。 影响我的钓鱼记录App 😂

Ethan Mollick@emollick · 6月19日67

I have given AA a hard time about its previous agentic evaluation but this looks like a good and impressive benchmark for real world knowledge work that is unsaturated and had private hold out tests. This is one to watch - I didn’t see a human comparison score though?

译Ethan Mollick 称赞 AA-Briefcase 是真实知识工作的优质基准，未饱和且含私有保留测试，同时询问是否有与人类的对比。该基准由 @ArtificialAnlys 发布，测试模型在多周、多任务项目中的能力，输入含数万条 Slack 消息和数千封邮件。模型排名：Claude Fable 5（已不可用）以 1587 Elo 居首，Claude Opus 4.8（1356）第二，GLM-5.2 max（1266）第三。结果凸显难度：最佳模型仅 3% 任务满足全部标准，31/91 任务无模型超过 50%，成本跨度约 800 倍。

Artificial Analysis@ArtificialAnlys · 6月19日55

Announcing AA-Briefcase, the benchmark for the next era of agentic knowledge work AA-Briefcase is our new benchmark for testing models on long-horizon knowledge work tasks in complex projects built by industry experts. Models are evaluated on multi-week projects, each with many linked tasks and thousands of input source files. We evaluated Claude Fable 5 from @AnthropicAI before it became unavailable, and it currently leads with an Elo score of 1587, followed by Claude Opus 4.8 (max, 1356), Opus 4.7, and the recently-released GLM 5.2 (max, 1266) from @Zai_org. Claude Fable 5 cost $31 on average to run each AA-Briefcase task, followed by Claude Opus 4.8 at $10.40, GPT-5.5 (xhigh) at $3.68 and GLM-5.2 (max) at $2.40. AA-Briefcase comprises four private scenarios, each representing a multi-week knowledge work project set in a realistic organizational context. A public fifth scenario has been released via @huggingface as a representation of scenario structure, submission, and grading (AA-Briefcase Lite). This does not count toward official AA-Briefcase results, and is demonstrative only. Key elements of AA-Briefcase: ➤ Realistic long-horizon projects: AA-Briefcase moves beyond single, disconnected prompts by evaluating models across a coherent long-horizon project. Tasks build week by week, draw on shared institutional context, and require deliverables such as financial models, board presentations, and design mock-ups ➤ Large volumes of fragmented context: AA-Briefcase requires models to reason across thousands of inputs, including company documents, meeting transcripts, large-scale data exports, 25,000+ Slack messages and 3,500+ emails. These sources are fragmented, messy, and often contain realistic contradiction, testing whether models can navigate the ambiguity of real-world knowledge work ➤ Composite rubric and pairwise grading: AA-Briefcase combines binary rubric checks for ground-truth correctness with pairwise grading on analytical quality and presentation quality. Unlike many evaluations that focus on a single metric, AA-Briefcase tests agentic capabilities more comprehensively, exposing cases where models produce outputs that look polished but are incorrect or lack analytical rigor ➤ Built by industry experts: AA-Briefcase scenarios mirror real-world knowledge work, with tasks developed over months by experts across data science, product management and corporate strategy from companies including Google, McKinsey & Company and BCG. Task challenges are drawn from professional experience, making AA-Briefcase more reflective of the ambiguity, messy context and competing priorities that define real-world knowledge work Key results: ➤ Claude Fable 5 leads AA-Briefcase at 1587 Elo: This is followed by Claude Opus 4.8 (1356) with the next-best non-Anthropic model, GLM-5.2 (max), ~90 points back at 1266. Note that Claude Fable 5 did not use the Opus 4.8 fallback for any task in AA-Briefcase ➤ Cost per task varies by ~800x across models tested: Claude Fable 5 leads the benchmark but costs more than $31 per task on average, compared to ~$0.04 for DeepSeek V4 Flash (max). The strongest price/performance options are open weights models such as GLM-5.2 (max) and DeepSeek V4 Pro (max), with GLM-5.2 (max) scoring only ~90 Elo below Claude Opus 4.8 (max) for less than 25% of the cost ➤ Real-world complexity remains difficult for models: The top performer, Claude Fable 5, satisfies all rubric criteria on just 3% of AA-Briefcase tasks. On 31 of 91 tasks, no model scores above 50% on the rubric criteria ➤ Task difficulty scales with the number of required input files: For each rubric check, we identify the set of source files needed to pass. Across all models, pass rates fall as this file count increases, though top-tier models degrade less than weaker models More details below in thread ⬇️

译Artificial Analysis 推出新基准 AA-Briefcase，用于评估模型在长期知识工作项目中的智能体能力。基准包含 4 个私有场景（每项目需处理 25000+ Slack 消息、3500+ 邮件等碎片化上下文）及一个公开演示场景。评测结果：Claude Fable 5 以 Elo 1587 领先，其次为 Claude Opus 4.8（1356）、Opus 4.7 及智谱 GLM 5.2（max，1266）。成本方面，Claude Fable 5 平均每任务 $31，Opus 4.8 为 $10.40，GPT-5.5 (xhigh) 为 $3.68，GLM 5.2 (max) 为 $2.40，DeepSeek V4 Flash (max) 仅约 $0.04。所有模型中仅 3% 的任务满足全部标准，31/91 个任务无模型得分超 50%，显示真实世界复杂性仍是挑战。最佳性价比为开源权重模型 GLM-5.2 (max) 和 DeepSeek V4 Pro (max)。

Artificial Analysis@ArtificialAnlys · 6月19日63

Wisedocs, an AI-powered medical record review platform, has launched Medical Long Context Reasoning (MLCR), a new long-context document evaluation based on their experience using frontier models to process medical data. This benchmark tests how well models reason over realistic medical and insurance case files, even as the amount of noise from other documents increases to larger context sizes. It includes a range of difficulty levels, with a private hold-out set of questions including complex medical reasoning, hallucination checking, and parallel questions in a single query inspired by real-world usage. We're excited to partner with @Wisedocsai to bring this benchmark to Artificial Analysis soon!

译Wisedocs 发布 Medical Long Context Reasoning (MLCR) 基准，测试 LLM 对真实医疗档案的长文档推理能力。评测包含 250 个问题，横跨 6 个难度等级，另设私有保留集，涵盖复杂医学推理、幻觉检测及单次查询中的并行提问。Wisedocs 同步开源 10 个合成病例、低三级问题及评估工具。Artificial Analysis 将合作上线该基准。

AYi@AYi_AInotes · 6月19日52

Damn，不能看图，没有任何视觉能力，却在视觉设计的擂台上排到了第一☝️ GLM-5.2，Elo 冲到 1360，压过了已经关服的 Claude Fable 5，而且是开源权重，不存在黑箱。 Design Arena 代码类的历史最高分，一下拉高了 27 分🤯

译GLM-5.2 在 Design Arena 上取得第一，Elo 达 1360，超越已关服的 Claude Fable 5。模型为开源权重，排名上升 4 位，Elo 提升 27 分，创下代码类历史最高分。

elvis@omarsar0 · 6月18日64

Recommended reading. Great insights, especially in areas where general-purpose models continue to fail, like dealing with complex structures. It also highlights that for scientific research, specialized models are winning big time.

译OpenAI 推出 LifeSciBench，用于衡量 AI 支持真实世界生命科学研究的能力。该基准与 173 位生物技术与制药科学家共同开发，包含 750 个专家编写任务，覆盖七种生物研究流程。DAIR.AI 的 Elvis Saravia 推荐阅读，并指出通用模型在处理复杂结构时仍然失败，而面向科学研究的专用模型表现显著更优。

AYi@AYi_AInotes · 6月18日43

Elon 收购Cursor之后的效益最先在Grok显现了， 测试了下Grok imagine 1.5的动作生成效果，这方面比seedance2.0弱一些，但进步也蛮大了，潜力很大哈哈，加油bro @Grok

译推文测试了 Grok imagine 1.5 的动作生成效果，认为其相比 seedance2.0 稍弱，但进步明显、潜力较大，并归因于 Elon 收购 Cursor 后的效益最先在 Grok 显现。

DogeDesigner@cb_doge · 6月18日29

Grok Imagine 1.5 is super impressive. Check out how the zoom effect looks now, insanely realistic with incredible high-quality detail.

译Grok Imagine 1.5 非常令人印象深刻。 看看现在的缩放效果，极度逼真，细节质量惊人。

meng shao@shao__meng · 6月18日58

Kimi K2.7 Code 和 Claude Fable 5 生成落地页的实验对比分析 @nutlope 让 Kimi K2.7 Code 和 Claude Fable 5 分别生成 12 个落地页，进行并排对比。实验结果发布在 OVSC 网站上，同时包含 Claude Opus 4.8 的变体生成页面，可以查看总成本、token 用量和生成时间等详细数据。 实验设计 · 基础提示：使用相同提示集，涵盖 B2B SaaS、屋顶 Speakeasy 鸡尾酒吧、SQL 查询转图表开发者工具等不同类别。 · 初始结果：仅凭提示生成时，两个模型均产生明显“AI 生成感”的页面（布局通用、视觉平淡）。 · 优化策略：为 Kimi 设置自定义 Design Inspiration MCP Server，提供高质量落地页截图、UI 元素及视觉参考。由于 Kimi 支持多模态，可直接将图像纳入提示。这显著提升输出质量：层次结构更清晰、排版更优、构图更具意图性，页面加载更快、无破损占位图、易读性更好。 成本对比 · 单页成本示例：B2B SaaS 落地页，Kimi 仅需 4 美分，Claude Fable 则为 1.09 美元（约 27 倍差异）。 · 平均水平：Kimi 比 Fable 约 16 倍便宜，比 Opus 约 8 倍便宜。整体实验中 Kimi 总成本降低 94%（约 16 倍节省）。 · 迭代价值：落地页开发通常需生成多版本、迭代优化。低成本允许廉价快速实验，累积优势显著（如生成 100 页可节省约 94 美元）。 质量对比 使用 GPT-5.5 按标准化评分表（定位、视觉方向、内容结构、工艺、响应式、技术执行等）对截图和源码打分（0-100 分）。Claude Fable 在部分案例中得分略高，但差距较小。Kimi 在设计、结构和整体页面质量上保持竞争力，尤其在提供充足视觉上下文后，性价比优势突出。 关键发现 · 上下文至关重要：单纯提示难以突破通用 AI 风格；引入高质量视觉参考（MCP）能让开源模型实现质的飞跃。 · 开源模型实用性：Kimi K2.7 Code 在成本、速度和 token 效率上表现出色，结合强上下文输入后，已足以支持实际落地页工作流。迭代时优势更明显。 · 实验启示：开源模型正成为生成式编码智能体的现实选择，尤其适合需要大量变体探索的场景。

译@nutlope 让 Kimi K2.7 Code 和 Claude Fable 5 各生成 12 个落地页并排对比。初始两者均有“AI 生成感”；为 Kimi 设置自定义 Design Inspiration MCP Server（利用多模态能力）后质量显著提升。成本上，B2B SaaS 单页 Kimi 仅 4 美分，Claude Fable 1.09 美元（约 27 倍差）；平均 Kimi 比 Fable 便宜约 16 倍、比 Opus 便宜约 8 倍，总成本降低 94%。质量由 GPT-5.5 评分（0-100），Claude Fable 略高但差距小，Kimi 性价比突出。实验表明，结合高质量视觉参考后，开源模型已足以支持实际落地页工作流，批量迭代优势显著。

Artificial Analysis@ArtificialAnlys · 6月18日61

Claude Fable 5 cost ~$6.2K to run the Artificial Analysis Intelligence Index benchmarks - the most expensive model we have ever benchmarked 🧵 Key takeaways: ➤ Intelligence Index: 60, ahead of Claude Opus 4.8 (56) and GPT-5.5 (55) ➤ Cost to run the Intelligence Index: $6.2K, 1.7× the next-highest model (Opus 4.8, $3.7K) and 2.2× GPT-5.5 (xhigh, $2.9K) ➤ List price: $10/$50 per 1M input/output tokens, 2× Opus 4.8. Among 2026 releases, only OpenAI's special Pro tier (GPT-5.5 Pro, $30/$180) is priced higher ➤ Cache pricing, which is particularly important for long agentic coding sessions, doubled too: $1/M cache reads, and $12.50/M cache writes vs $0.50/$6.25 for Opus 4.8 ➤ The top 3 most-expensive models to run the Intelligence Index are now all Claude models

译Artificial Analysis 将 Claude Fable 5 列为有史以来基准测试成本最高的模型，运行其 Intelligence Index 需 $6.2K，是第二贵模型 Opus 4.8（$3.7K）的 1.7 倍、GPT-5.5（$2.9K）的 2.2 倍。该模型 Intelligence Index 得分 60，领先 Opus 4.8（56）和 GPT-5.5（55）。定价 $10/$50 每百万输入/输出 tokens，为 Opus 4.8 的 2 倍，仅低于 GPT-5.5 Pro（$30/$180）。缓存价格同步翻倍：缓存读取 $1/M、写入 $12.5/M，而 Opus 4.8 分别为 $0.5/$6.25。Intelligence Index 成本前三高的模型目前均为 Claude。

MiniMax (official)@MiniMax_AI · 6月18日51

image input with M3 carrying a full sim. good build @coldopn

译MiniMax 官方转发用户 @coldopn 的体验：前沿模型已不再仅限于 Anthropic 和 OpenAI。通过将一张插图截图输入 Kilo Code，切换至 MiniMax M3 模型，仅凭一句“将此截图动画化为可工作的黑洞模拟器”的提示词，M3 即生成了完整的黑洞模拟器。整个推理过程仅花费 0.53 美元，展现了 M3 强大的视觉理解与生成能力。

elvis@omarsar0 · 6月18日56

I was a bit suspicious of the claim, but GLM-5.2 is pretty good at designing stuff. Obviously not at the level of a professional designer, but it has that Opus-level quality. Great at: - games - landing pages - HTML artifacts - 3D worlds Wish I had Fable 5 to compare with.

译GLM-5.2 在 Design Arena 上以 Elo 1360 跃居第一，超过已下架的 Claude Fable 5，排名提升 4 位、Elo 提高 27 分，且为开源权重。DAIR.AI 的 Elvis Saravia 实测认为其设计能力不错，虽未达专业设计师水平，但具备 Opus 级质量，擅长游戏、落地页、HTML artifacts 及 3D 世界等任务。

Artificial Analysis@ArtificialAnlys · 6月18日51

A standout number in Z ai’s GLM-5.2 launch is CritPt, a benchmark of unpublished research-level physics problems where it ties with Claude Opus 4.8 and is well above other open weights models Key takeaways: ➤ @Zai_org ’s GLM-5.2 (max reasoning effort) leads open weights by a wide margin: the next open model, DeepSeek V4 Pro, scores 12.9% ➤ GLM-5.2 matches Claude Opus 4.8 (20.9%) and beats several proprietary models, including GPT-5.5, Gemini 3.1 Pro, and Claude Opus 4.7 ➤ Only proprietary models score higher with GPT-5.5 Pro topping the benchmark at 30.6% ➤ A 4.5× generational jump: GLM-5.1 scored just 4.6% on CritPt ten weeks ago

译智谱发布 GLM-5.2（最大推理努力），在 CritPt 基准（未发表研究级物理问题）上得分 20.9%，与 Claude Opus 4.8 持平，远超其他开放权重模型。DeepSeek V4 Pro 仅得 12.9%；GLM-5.2 同时超越 GPT-5.5、Gemini 3.1 Pro 和 Claude Opus 4.7 等专有模型。仅 GPT-5.5 Pro 以 30.6% 领先。相比十周前 GLM-5.1 的 4.6%，实现 4.5 倍代际提升。

Ethan Mollick@emollick · 6月18日60

I have a fun, oddly useful AI benchmark: "build me a procedurally generated 3D simulation showing the evolution of a harbor town from 3000 BC to 3000 AD, it should look beautiful &amp; allow me to have some control over it" Play the gallery of 20 models: https://ai-harbor-town-gallery.netlify.app/

译我有一个有趣且出奇有用的AI基准测试：“为我构建一个程序化生成的3D模拟，展示一个港口小镇从公元前3000年到公元3000年的演变，它应该看起来很漂亮，并允许我对其进行一些控制。” 查看20个模型的画廊：https://ai-harbor-town-gallery.netlify.app/

Artificial Analysis@ArtificialAnlys · 6月17日61

Z ai’s GLM-5.2 is the new leading open weights model on the Artificial Analysis Intelligence Index scoring 51 and it sits on the Pareto frontier of Intelligence vs Cost per Task @Zai_org’s GLM-5.2 is the same size as GLM-5.1 (744B total / 40B active parameters) but scores 11 points higher on the Intelligence Index v4.1, placing ahead of MiniMax-M3 (44) and DeepSeek V4 Pro (max, 44). On the first-party API it is priced in line with GLM-5.1 at $1.4/$4.4/$0.26 per 1M input/output/cache hit tokens Key results: ➤ GLM-5.2 is the leading open weights model on the Intelligence Index v4.1. At 51, it leads MiniMax-M3 (44), DeepSeek V4 Pro (max, 44) and Kimi K2.6 (43) ➤ Improvements across most evaluations, particularly scientific reasoning: GLM-5.2 gains over GLM-5.1 on most evaluations, led by scientific reasoning on CritPt (+16 points to 21%) and HLE (+12 points to 40%), alongside AA-LCR (+9 points to 71%), tau3 banking (+15 points to 27%) and SciCode (+7 points to 50%). TerminalBench v2.1 also improves (+16 points to 78%) and GPQA Diamond gains 3 points to 89% ➤ Leading open weights model on GDPval-AA v2 and competitive with proprietary models: GLM-5.2 scores 1524 on GDPval-AA v2, ahead of MiniMax-M3 (1418) and DeepSeek V4 Pro (max, 1328). This impressive result places GLM-5.2 in-line with proprietary models including GPT-5.5 (xhigh reasoning). GDPval-AA v2 builds on the original GDPval-AA by baselining Elo to human performance at 1000, introducing a rotating panel of frontier-model judges, and raising the turn limit from 100 to 250 for longer-horizon agent trajectories ➤ GLM-5.2 uses more output tokens per task than other leading open weights models: the model uses 43k output tokens per Intelligence Index task, up from GLM-5.1 (26k) and above MiniMax-M3 (24k), Kimi K2.6 (35k) and DeepSeek V4 Pro (max, 37k) ➤ On the Intelligence vs. Cost per Task Pareto Frontier: GLM-5.2 is on the Pareto frontier of the Intelligence vs Cost per Task chart, with the lowest cost per task among models at its intelligence level. GLM-5.2 costs ~$0.46 per task, compared to GLM-5.1 ($0.25), Kimi K2.6 ($0.31), MiniMax-M3 ($0.18) and DeepSeek V4 Pro (max, $0.05) Additional Model Details: ➤ License: MIT ➤ Size: 744B total parameters, 40B active parameters, equivalent to GLM-5.1 ➤ Context window: 1M tokens, up from 200K on GLM-5.1 ➤ Pricing: $1.4/$0.26/$4.4 per 1M input/cache hit/output tokens ➤ Availability: Alongside Z ai's first-party API, GLM-5.2 is available across third-party providers including @DeepInfra, @novita_labs, @nebiusai, @parasailnetwork , @SiliconFlowAI , @gmi_cloud , @Baseten and @FireworksAI_HQ

译Z ai 发布 GLM-5.2（744B 总参数/40B 活跃参数），在 Artificial Analysis Intelligence Index v4.1 上得分 51，超越 MiniMax-M3、DeepSeek V4 Pro 和 Kimi K2.6。科学推理大幅提升：CritPt +16、HLE +12、GPQA Diamond 达 89%。GDPval-AA v2 得分 1524，与 GPT-5.5 (xhigh reasoning) 相当。上下文窗口扩展至 1M tokens，MIT 许可证。第一方 API 定价 $1.4/$4.4/$0.26 每百万输入/输出/缓存命中 token，每任务成本约 $0.46，处于智能 vs 成本帕累托前沿。

AYi@AYi_AInotes · 6月17日57

Grok Imagine Video 1.5 实测： 复刻《权力的游戏》Tyrion 经典法庭演讲，效果真的有点吊炸，完全不输seedance 2， 整个场景真实感、物理表现和原生音频都拉满了……面部微表情、布料动态、火把光影互动都很自然

SemiAnalysis@SemiAnalysis_ · 6月17日65

POV: @ohnePixel getting a platform for day 0 DeepSeek V4 deployment Find out more at: https://semianalysis.substack.com/p/deepseekv4-16t-day-0-to-day-43-performance

译POV: @ohnePixel 为 DeepSeek V4 首日部署获得一个平台。 了解更多：https://semianalysis.substack.com/p/deepseekv4-16t-day-0-to-day-43-performance

Artificial Analysis@ArtificialAnlys · 6月17日52

Following up on our Intelligence Index v4.1 release yesterday, in the video below, Daniel from our team shares a short overview of what's changed: 1. Three upgraded evaluations: Terminal-Bench 2.1, τ³-Bench Banking and GDPval-AA v2 2. Cost, time, and tokens per task: Understand the cost, time, and tokens of tasks across our Index and for individual evals, and how these trade off against Intelligence 3. Cached input token reporting: We now report the amount of cached tokens a particular model uses and how this influences cost

译Artificial Analysis 昨日发布 Intelligence Index v4.1 更新，主要变化有三项：升级的评测 Terminal-Bench 2.1、τ³-Bench Banking 和 GDPval-AA v2；提供每项任务的成本、时间与模型 token 消耗数据，并展示这些指标与智能水平的权衡；新增缓存输入 token 报告，显示特定模型使用的缓存 token 量及其对成本的影响。

歸藏(guizang.ai)@op7418 · 6月17日40

简单拿我的社交媒体卡片对比测试了一下。 左边是 GPT 5.5 的结果，右边是 GLM 5.2 的结果。给的素材都是 GLM 5.2 的发布博客。

译博主 @op7418 用 GLM 5.2 发布博客作素材，对比了 GPT 5.5（左）与 GLM 5.2（右）生成的社交媒体卡片。

Ethan Mollick@emollick · 6月17日58

Credit to GLM-5.2 Max, the new open weights model, for pulling this off. ...but you can see the difference between it and Fable in a way benchmarks don't show. GLM-5.2 gives a correct poem (&amp; the Welsh is fun) but Fable weaves the disappearing letters into the theme of the poem.

译归功于 GLM-5.2 Max，这个新的开放权重模型，成功完成了这个任务。 ...但你能看出它和 Fable 之间的区别，这种区别是基准测试无法体现的。GLM-5.2 给出了一首正确的诗（威尔士语很有趣），但 Fable 将消失的字母融入了诗歌主题。

Artificial Analysis@ArtificialAnlys · 6月17日20

To mark the release of Artificial Analysis Intelligence Index v4.1, we're bringing together researchers, engineers, and builders working at the frontier of AI in San Francisco on June 29. Join us for an evening of talks on AI evaluation, model intelligence, and the tradeoffs between cost, speed, and performance. Apply to attend 👇 https://luma.com/qdl9mr2e

译为庆祝 Artificial Analysis Intelligence Index v4.1 发布，我们将于 6 月 29 日在旧金山汇聚前沿 AI 领域的研究人员、工程师和构建者。 欢迎加入我们，共度一个关于 AI 评估、模型智能以及成本、速度与性能之间权衡的晚间讨论。 申请参加 👇 https://luma.com/qdl9mr2e

Ethan Mollick@emollick · 6月17日29

This was not a good benchmark before it was updated and it is not a good benchmark now. Having AIs evaluate the work of other AIs on publicly available questions from a different closed benchmark doesn’t tell you very much. And it is unclear how they establish the human ELO.

译新版 GDPval-AA v2 成为 Intelligence Index v4.1 权重最高的评估，升级将 ELO 基线重置为人类 1000 分，引入前沿模型法官轮换面板，回合上限从 100 提升至 250。Claude Fable 5（有回退）以 1818 分领先，但当前不可用；Claude Opus 4.8 得 1638 分，GPT-5.5 (xhigh) 得 1531 分。Ethan Mollick 批评：AI 评估 AI 在取自另一闭卷基准的公开问题上表现意义有限，且人类 ELO 设定方式不透明，认为更新前后均非良好基准。