Rohan Paul@rohanpaul_ai · 6月9日64

Interesting, this paper shows that Transformers may not need separate key and value projections to work well. This paper's design cut the KV cache by 50% in language modeling with only 3.1% higher perplexity, meaning inference memory fell sharply while prediction quality stayed close. A normal attention layer makes Query to ask what each token needs, Key to label what each token offers, and Value to carry the information sent back. Here, the surprising result is that Key and Value can often share the same learned map, because the model can use one representation both as an address and as the content being retrieved. The best variant, Q-K=V, kept Query separate, so attention still had direction: one token can ask a different token for information instead of every relation becoming mirror-like. When stacked with GQA and MQA, the same idea reached 87.5% and 96.9% cache cuts, because it reduces projection storage while those methods reduce stored heads. The weak variant is Q=K-V, because tying Query and Key makes attention too symmetric for causal language, and it gives no KV-cache savings. ---- Link – arxiv. org/abs/2606.04032v2 Title: "Do Transformers Need Three Projections? Systematic Study of QKV Variants"

译一篇论文系统研究了Transformer注意力中QKV投影的必要性，发现Key和Value可共享同一投影（Q-K=V变体），仅增加3.1%的困惑度，便将KV cache削减50%，大幅降低推理内存。最佳变体保留Query独立，使注意力保持方向性。与GQA和MQA结合时，可分别实现87.5%和96.9%的cache缩减。弱变体Q=K-V因导致因果注意力过于对称且无cache节省而无效。

elvis@omarsar0 · 6月9日62

New paper on how AI agents are reshaping knowledge work. This is a nice economic read on where agents actually change knowledge work to meet that gap directly. (bookmark it) It studies agent adoption across three dimensions: autonomy, efficiency, and the scope of tasks workers hand off. The friction people keep hitting with agents is rarely model quality. It is that almost nobody has been taught how to work this way. Paper: https://arxiv.org/abs/2606.07489 Learn to build effective AI agents in our academy: https://academy.dair.ai/

译一篇新论文从自主性、效率和工人移交任务的范围三个维度，分析AI智能体如何重塑知识工作。研究指出，当前人们使用智能体的主要障碍并非模型质量，而是几乎没有人接受过如何以这种方式工作的培训。

Rohan Paul@rohanpaul_ai · 6月9日63

This paper proposes a new test to see whether AI agents truly get better as they gain experience and finds they mostly still confuse memory with learning. Shows that simple full-context learning beats the more specialized memory systems, with Claude Sonnet 4.6 using plain context getting the best overall score. That distinction matters because the next wave of AI is not supposed to answer isolated prompts. It is supposed to live inside codebases, databases, markets, sensors, clinics, and workflows where yesterday’s mistake should make tomorrow’s action sharper. The authors build CL-BENCH, a benchmark where an agent works through connected tasks in 6 domains, including coding, databases, forecasting, radio signals, poker, and disease studies. Each task hides a pattern the agent can learn over time, like a database layout, a codebase structure, or an opponent’s strategy, so better performance should come from experience rather than pretraining. They test frontier LLM systems with simple full-context memory, scratchpad notes, retrieval memory, playbook-style memory, and coding-agent setups. The key finding is that current memory-heavy AI agents are not reliably better learners than just keeping the full conversation in context. That means long-running AI agents still need better ways to remember useful lessons, forget stale ones, and adapt when the environment changes. ---- Link – arxiv. org/abs/2606.05661 Title: "Continual Learning Bench: Evaluating Frontier AI Systems in Real-World Stateful Environments"

译新论文构建 CL-BENCH 基准，评估 AI 智能体在编程、数据库、预测、无线电信号、扑克、疾病研究 6 个领域中的持续学习能力。每个任务隐藏可随时间习得的模式，考察智能体能否超越预训练知识。测试前沿 LLM 系统采用全上下文记忆、草稿笔记、检索记忆、剧本式记忆及编码智能体设置，结果发现当前记忆密集型 AI 智能体并未可靠优于简单保留完整对话上下文。Claude Sonnet 4.6 使用普通上下文取得最佳总体分数。论文指出智能体仍需更好方法记住有用经验、遗忘过时信息并适应环境变化。

Rohan Paul@rohanpaul_ai · 6月8日60

Great Stanford + MIT + Harvard + Anthropic paper. Gives a clear training-based reason for why larger models learn abilities smaller models miss. Says bigger AI models learn rare skills because they forget them less during training, their extra space protects weak learning signals. The authors say the issue is not just whether a small model could represent the task, but whether training lets it keep that task while many common tasks keep pushing on the same limited parts. Their core idea is that common tasks take up the model’s neurons first, so rare tasks get overwritten before they appear often enough to build into stable knowledge. In a crowded data mixture, common patterns get first claim on the model’s internal machinery. Small models may briefly pick up a rare signal, but the next wave of common-task updates overwrites it before the signal appears again. They tested this first with controlled toy tasks where they could change how rare and complex each task was, then with OLMo language models from 4M to 4B parameters. The main result is that bigger models learned low-frequency tasks much better, kept more task features inside their representations, and showed less gradient interference, which means common-task updates disturbed rare-task learning less. Larger models can remember weak rare signals long enough to turn them into real learned skills. ---- Link – arxiv. org/abs/2605.29548 Title: "Why Larger Models Learn More: Effects of Capacity, Interference, and Rare-Task Retention"

译该论文指出，更大模型能学到罕见技能，是因为训练中遗忘更少，其额外容量保护了弱学习信号。核心机制：常见任务先抢占神经元，罕见任务在出现频率足够形成稳定知识前就被覆盖。小模型可能短暂捕捉到罕见信号，但随即被下一波常见任务更新覆盖。实验使用OLMo语言模型（4M–4B参数）验证：大模型在低频任务上表现更优，保留更多任务特征，且常见任务更新对罕见任务的梯度干扰更小。作者强调，问题不仅在于小模型能否表征任务，更在于训练中罕见任务能否在众多常见任务反复冲击下持续存在。

meng shao@shao__meng · 6月8日64

AGENTS.md 在 Coding Agents 中真的有用吗？ 这篇论文，大规模实证研究仓库级上下文文件（AGENTS.md、CLAUDE.md 等）对编码 Agent 实际效果的影响，可能有些反直觉！感谢 @rasbt 分享！ 论文在这：https://arxiv.org/abs/2602.11988 研究背景：实践先行，证据滞后 AGENTS.md 已成为行业惯例，GitHub 上已有 6 万+ 仓库采用，Claude Code (CLAUDE.md)、Codex、Qwen Code 等 Agent 都内置 /init 自动生成。但此前研究多停留在内容分类与描述性统计，缺少对任务完成率的严格评估。 核心难点在于：主流基准 SWE-bench 来自 Django、Flask 等知名仓库，这些项目本来就没有开发者手写的 context file，无法直接评估该实践的真实价值。 实验设计：双基准、三条件、四 Agent · 基准：SWE-bench Lite（300 任务，11 个热门 Python 仓库）+ 新建 AGENTBENCH（138 任务，12 个已含开发者 context file 的冷门仓库） · 三种条件：① 无 context file ② LLM 生成（各 Agent 官方 /init 流程）③ 开发者手写（仅 AGENTBENCH） · Agent/模型：Claude Code + Sonnet 4.5、Codex + GPT-5.2 / GPT-5.1 mini、Qwen Code + Qwen3-30B · 指标：任务成功率、步数、推理成本、工具调用轨迹 核心发现：效果微弱，成本显著 1. 成功率：边际效应，甚至为负 · LLM 生成：8 组设置中 5 组下降，平均 -0.5%（SWE-bench）/ -2%（AGENTBENCH） · 开发者手写：平均 +4%，优于 LLM 生成，但 Claude Code 上甚至不如无文件 · 跨模型、跨 prompt 结论稳健 一句话：自动生成 context file 不仅无益，还可能略有害；手写的提升也很有限。 2. 效率：无文件反而最便宜（步数，成本） · LLM 生成：+2.45 / +3.92 步，+20% / +23% · 开发者手写：+3.34 步，最高 +19% 3. 代码库概览几乎无效 Context file 常被推荐用于「帮助 Agent 快速定位代码」。实测显示：有无 context file，Agent 首次接触相关文件所需的步数并无显著差异。95–100% 的 LLM 生成文件都包含代码库概览，但对导航帮助甚微。 轨迹分析：Agent 听话，但听话很贵 论文排除了「Agent 忽略 context file」这一假设。轨迹分析表明： · 指令遵从度高：context file 提到 uv，使用率从 <0.01 次/任务升至 1.6 次；提到仓库专用工具，从 <0.05 升至 2.5 次 · 行为更「认真」：更多测试、更多文件搜索/阅读、更多 lint/质量检查 · 推理更深：GPT-5.2 推理 token 增加 14–22% 机制链条： Context file 写入额外要求 → Agent 更严格遵从（测试、探索、专用工具） → 步数与成本上升 → 成功率未同步提升（甚至更差） Context file 不是被忽略，而是被过度执行——把「建议性流程」当成了「必做清单」，增加了任务复杂度，却没有换来更高成功率。 一个关键反转：文档冗余假说 当移除仓库中所有其他文档（.md、docs/、示例代码）后，LLM 生成的 context file 反而带来 +2.7% 提升，且优于开发者手写的。 这说明： · 在文档齐全的仓库里，context file 与 README、docs 高度冗余 · 开发者口述的「加了 AGENTS.md 后 Agent 变强了」，很可能是因为目标仓库本身文档稀缺，context file 填补了信息真空 · 对 Django 这类文档完善的知名项目，额外 context 的价值被稀释 消融实验：生成质量的上限 · 更强模型生成 ≠ 更好 context：GPT-5.2 生成的文件在 SWE-bench 上略好（+2%），在 AGENTBENCH 上反而更差（-3%） · 不同 prompt 无一致优势：Codex prompt vs Claude prompt 效果因数据集而异，差异很小 自动生成 context file 的改进空间，目前看来很有限。 实践建议 · 依赖 /init 自动生成：谨慎——平均略降成功率，成本 +20%+ · 长篇架构概览、目录枚举：避免——与代码探索冗余，不加速定位 · 测试/lint/构建命令：精简写入——Agent 会严格执行，但过多要求推高成本 · 仓库专用工具（uv、pdm 等）：值得写——指令遵从度高，且代码中不易推断 · 分层/按需引用：方向正确——「做 X 时读 Y.md，否则忽略」减少无关负担

译论文大规模实证检验 AGENTS.md 等仓库级上下文文件对编码 Agent 的影响。在 SWE-bench Lite（300 任务）和新建 AGENTBENCH（138 任务）上测试 Claude Code、Codex、Qwen Code 等组合。核心发现：LLM 自动生成的 context file 在 8 组设置中 5 组成功率下降，平均 -0.5%（SWE-bench）/-2%（AGENTBENCH），成本增加 +20%+；开发者手写仅平均 +4%。冗余假说：移除其他文档后，自动生成反而 +2.7%。建议避免自动生成，精简测试/lint 命令，优先写入仓库专用工具。

Rohan Paul@rohanpaul_ai · 6月8日49

This paper tests whether today’s AI agents can build better AI agents without human design help. i.e. whether an AI can act more like an AI engineer. That means it must invent a strategy, write the agent code, test it, learn from failures, and improve the system without a human guiding every choice. Shows they are still weak at reliably building the systems that do tasks. Their benchmark, called Meta-Agent Challenge, gives an AI coding agent a safe workspace, a scoring API, limited time, and limited model calls, then asks it to create another agent that performs well on hidden test tasks. They tested this across 5 areas, including math, science questions, competitive programming, software bug fixing, and long terminal tasks. The main result is that current agents usually do not beat strong human-made agent setups, and the few good results mostly come from closed frontier models like Claude. Complete autonomy is not just tool use. It is budget awareness, failure recovery, restraint under pressure, and the discipline to change designs instead of polishing a bad one. Overall, Meta-Agent Challenge (MAC) suggests that today’s agents are not yet self-improving engineers. They are powerful executors with flashes of design judgment, still missing the boring reliability that makes engineering real. ---- Link – arxiv. org/abs/2606.04455 Title: "The Meta-Agent Challenge: Are Current Agents Capable of Autonomous Agent Development?"

译一项新研究提出Meta-Agent Challenge（MAC）基准，测试AI智能体能否在没有人类设计帮助的情况下自主构建更优智能体。智能体需在安全工作区内自行发明策略、编写代码、测试并从失败中学习。实验覆盖数学、科学问答、竞赛编程、代码修复和长终端任务5个领域。结果显示，当前智能体大多无法超越人工设计的强智能体系统，仅Claude等少数封闭前沿模型取得较好表现。研究认为，当前智能体更像是强大的执行者，而非具备可靠自改进能力的工程师。

Rohan Paul@rohanpaul_ai · 6月8日49

A Primer paper about how reasoning models improve after training Shows that better reasoning models depend less on raw data size and more on checkable training evidence. reasoning data is NOT simple question-and-answer pairs. The useful part is often the feedback that says why an answer, step, tool action, or full attempt was good or bad. A prompt and a response tell you what a model said, but not why that answer became learnable, which judge blessed it, which failures were hidden, or whether the skill was already inside the base model. The core idea is to describe each training example as a record that includes the task, the model’s behavior, the checking signal, and metadata about where it came from. The authors sort reasoning data by how it can be checked, such as exact rule-based checks for math and code, environment checks for agents using tools, and human or model judgments when no exact checker exists. They also explain why common assumptions fail, because long reasoning traces may be fake, harder examples may be useless for some models, and larger datasets may still miss important coverage. The key point is that agent data should preserve mess: failed actions, retries, recoveries, state differences, and terminal checks, because that is where learning signal often lives. ---- Link – arxiv. org/abs/2606.02113 Title: "A Primer in Post-Training Reasoning Data: What They Know About How It Works"

译论文指出，更好的推理模型更依赖可验证的训练证据，而非原始数据规模。推理数据的关键不是简单问答对，而是提供答案、步骤、工具操作或完整尝试好坏判断的反馈信号。每个训练样本应描述为包含任务、模型行为、检查信号和元数据的记录。研究者按检查方式分类：数学和代码用精确规则、智能体工具用环境检查，无精确检查器时用人类或模型判断。常见误区包括：长推理链可能虚假、更难样例对部分模型无效、更大数据集仍可能缺失关键覆盖。智能体数据应保留失败动作、重试、恢复、状态差异和终端检查等“混乱”信息，因为学习信号常在其中。

elvis@omarsar0 · 6月8日59

This was one of the standout AI papers of the week. (bookmark it) It tackles a question most self-improving AI agents ignore: is the agent actually discovering anything, or just remixing what it already knows? How can you tell whether the agent is doing real discovery or just confident retrieval? The authors give three clean buckets: - Retrieval is looking something up in a notebook you already have. - Search is combining tools you already own in new ways. - Discovery is inventing a new concept that wasn't in your toolkit before. The issue is that most agents stop at the first two. The math behind their definition (category theory plus a left Kan extension, if you care) is basically a bookkeeping trick to ask: could the old version of me have produced this result? If yes, it's not discovery. If no, something genuinely new showed up. They build a Builder/Breaker agent that studies protein mechanics. Over four rounds, the model's fit accuracy actually drops (R² goes from 0.48 to 0.68 to 0.54 to 0.41). At first glance, that looks like a failing agent. It isn't. The agent kept taking on harder proteins and rewriting its theory to cover them. Data grew almost 10x while the model code grew only 1.3x. A smaller theory covering a bigger world is exactly what good science looks like. Why does it matter? If you optimize for accuracy alone, your self-improving agent will just settle into easy benchmarks and stop. This paper offers a cleaner success signal and asks whether the agent is compressing more of the world into less code over time. Paper: https://arxiv.org/abs/2606.01444 Learn to build effective AI agents in our academy: https://academy.dair.ai/

译本周一篇AI论文探讨自我改进智能体是否真正发现新知识，还是仅重新组合已知信息。作者将行为分为三类：检索（查询已有笔记本）、搜索（组合现有工具）和发现（发明新概念），并用范畴论和左Kan扩展定义——若旧版本能产生相同结果则非发现。他们构建Builder/Breaker agent研究蛋白质力学，四轮中R²从0.48升至0.68再降至0.54和0.41，看似变差实则不断挑战更难蛋白质并重写理论：数据增长近10倍，模型代码仅增长1.3倍。论文提出用代码压缩率作为真实发现信号。链接：arxiv.org/abs/2606.01444。

Rohan Paul@rohanpaul_ai · 6月7日62

Great idea for self-evolving AI scientists from this new MIT paper. Tries to make an AI scientist notice when its current way of thinking is too small, then add new scientific concepts instead of merely searching harder. The problem is that most AI science systems still search inside a fixed setup, even when real science sometimes needs new kinds of variables, tools, tests, or claims. The paper’s core idea is to make every data point, model, tool output, failure, and claim a typed artifact, where typed means the system records what kind of thing it is and how it was produced. Then the system can tell the difference between retrieval, which adds known things, search, which explores a fixed setup, and discovery, which changes the setup itself. So novelty AI scientists is not defined by surprise, fluency, or benchmark gain, but by what could not be expressed inside the previous schema. A serious attempt to formalize something most AI systems still fake: the difference between finding an answer inside a language and earning the right to change the language. ---- arxiv. org/abs/2606.01444 Title: "Self-Revising Discovery Systems for Science: A Categorical Framework for Agentic AI"

译MIT论文（F.Y. Wang & M.J. Buehler, arXiv:2606.01444, 2026）提出Self-Revising Discovery Systems框架，使AI科学家能自主识别当前思维模式不足并添加新科学概念，而非仅更努力搜索。系统将数据、模型、工具输出、失败及声明均视为类型化产物（typed provenance），从而区分三种模式：retrieval（添加已知对象）、search（探索固定模式）和discovery（可验证的模式转换）。论文通过Kan obstruction和Left Kan extension数学化定义了真正新颖性——由旧证据传输后的逐点残差量化，使novelty可客观测量。案例包括Builder/Breaker模型发现蛋白质模式条件顺应性，以及CategoryScienceClaw发现各向异性纤维网络刚度规则。

Rohan Paul@rohanpaul_ai · 6月7日66

New MIT paper, great idea for self-evolving AI scientists from Tries to make an AI scientist notice when its current way of thinking is too small, then add new scientific concepts instead of merely searching harder. The problem is that most AI science systems still search inside a fixed setup, even when real science sometimes needs new kinds of variables, tools, tests, or claims. The paper’s core idea is to make every data point, model, tool output, failure, and claim a typed artifact, where typed means the system records what kind of thing it is and how it was produced. Then the system can tell the difference between retrieval, which adds known things, search, which explores a fixed setup, and discovery, which changes the setup itself. So novelty AI scientists is not defined by surprise, fluency, or benchmark gain, but by what could not be expressed inside the previous schema. A serious attempt to formalize something most AI systems still fake: the difference between finding an answer inside a language and earning the right to change the language. ---- arxiv. org/abs/2606.01444 Title: "Self-Revising Discovery Systems for Science: A Categorical Framework for Agentic AI"

译MIT团队提出自我演进AI科学家框架，核心创新是让AI识别当前推理空间过小并主动添加新科学概念，而非仅在固定模式内搜索。论文将数据点、模型、工具输出、失败、声明均视为带类型的artifact，明确区分检索（添加已知对象）、搜索（探索固定schema）和发现（可验证的模式扩展）。通过类型化copresheaf与Kan障碍理论证明，真正发现是可验证的schema扩展：旧证据由左Kan扩展传输，创新性通过逐点残差量化。案例包括Builder/Breaker模型发现蛋白质模式条件顺应性，以及CategoryScienceClaw发现各向异性纤维网络刚度规则。论文arXiv:2606.01444（2026）。

elvis@omarsar0 · 6月6日65

// Continual Learning Bench // One of the research areas with lots of investments is continual learning. While there are many efforts, there is very little progress in measuring it. So the big question is, do dedicated memory systems actually make agents learn from experience? Continual Learning Bench says not yet. Across six expert-validated domains with shared learnable structure, naive in-context learning outperforms systems purpose-built for memory management. CL-Bench introduces a gain metric that isolates genuine learning from prior capability, then shows agents frequently overfit to immediate observations or fail to reuse knowledge across instances. If a plain ICL baseline beats your memory architecture, the architecture is adding overhead rather than learning. Paper: https://arxiv.org/abs/2606.05661 Learn to build effective AI agents in our academy: https://academy.dair.ai/

译持续学习领域投入多但进展缓慢。CL-Bench（持续学习基准）在六个由专家验证、包含共享可学习结构的领域上测试，发现简单的上下文学习（ICL）基线优于专门为记忆管理构建的系统。该基准引入增益指标以隔离真正学习效果，结果显示智能体常过度拟合即时观察或未能跨实例复用知识。研究指出，若普通ICL基线超过你的记忆架构，则该架构增加的是开销而非学习。论文：arxiv.org/abs/2606.05661。

Emad@EMostaque · 6月6日33

If Claude is good enough for Nobel Prize winners it is good enough for you https://arxiv.org/abs/2606.03300

译如果 Claude 对诺贝尔奖得主来说都足够好，那对你也一样。 https://arxiv.org/abs/2606.03300

AK@_akhaliq · 6月6日56

ArcANE Do Role-Playing Language Agents Stay in Character at the Right Time?

译ArcANE 角色扮演语言智能体是否能在适当时刻保持角色？

Tencent Hy@TencentHunyuan · 6月5日74

Planning is where LLMs move from “saying” to “doing.” Tencent Hy, in collaboration with the Gaoling School of Artificial Intelligence at Renmin University of China, is excited to open-source PlanningBench - a scalable, verifiable framework for evaluating and training LLM planning capabilities. With PlanningBench, you get: ✅ 30+ real-world planning tasks ✅ Automated verification ✅ Evaluation and training support See how top-tier LLMs perform on PlanningBench 👇 Resources: arXiv: https://arxiv.org/abs/2605.20873 GitHub: https://github.com/Tencent-Hunyuan/PlanningBench HuggingFace: https://huggingface.co/datasets/tencent/PlanningBench #PlanningBench #TencentHunyuan #OpenSource 📷

译腾讯混元（Tencent Hunyuan）与中国人民大学高瓴人工智能学院合作，开源PlanningBench——一个可扩展、可验证的LLM规划能力评估与训练框架。该框架包含30+真实世界规划任务，支持自动验证和训练。PlanningBench旨在推动LLM从“说”到“做”的规划能力发展。资源已发布于arXiv、GitHub及HuggingFace。

Rohan Paul@rohanpaul_ai · 6月5日60

Harness-1 makes search agents better by moving memory work out of the model and into a helper system. Shows that intelligence performs better when the environment stops forcing it to spend cognition on bookkeeping. That search agents should stop using the LLM as the notebook and let a separate harness track the search state. The paper proved that a 20B model improved search by doing less inside its own head. The problem is that normal search agents must both think about the next search and remember every document, clue, failed path, and remaining check inside the same limited context. This formulation puts too much routine state management inside the policy. Harness-1 separates those jobs. The model keeps the hard semantic choices: what to search, what to inspect, what to verify, and when the evidence is good enough. The harness keeps the recoverable state: candidate pools, curated documents, importance tags, evidence links, verification records, deduplicated observations, and budget-aware memory rendering. That sounds minor until you look at reinforcement learning. RL works poorly when every failure looks the same, because an empty or wrong final set does not reveal whether the agent searched badly, forgot evidence, skipped verification, or curated carelessly. By externalizing state, Harness-1 gives the policy a cleaner learning problem: improve decisions over a visible search workspace. For Harness-1, its gains were larger on held-out benchmarks than on source-family tasks, suggesting the model learned reusable search moves rather than memorized domain habits. ---- Link – arxiv. org/abs/2606.02373 Title: "Harness-1: Reinforcement Learning for Search Agents with State-Externalizing Harnesses"

译Harness-1 将大语言模型的记忆工作转移到外部辅助系统（harness），解决传统搜索智能体需在同一上下文窗口内处理语义决策与状态记录导致的效率低下问题。模型仅负责搜索、验证等关键语义选择，而可恢复状态（候选池、证据链接、去重记录、预算感知记忆等）由 harness 追踪。这一分离使一个 20B 参数模型实现了更好的搜索表现。在强化学习中，外部化状态避免了失败原因混淆，有助于策略学习。Harness-1 在未见 benchmark 上提升更大，表明模型学到了可复用的搜索策略而非记忆领域习惯。论文 arXiv:2606.02373。

Rohan Paul@rohanpaul_ai · 6月4日66

This Illinois+ Tsinghua University and other labs study finds that LLM agents still have unreliable memory and that it can get worse when they keep rewriting their own memories. LLM agents can learn from experience, but their rewritten memories often become unreliable. The problem is that many agent systems store past work by asking an LLM to compress messy experience into neat written lessons. That sounds useful because the agent should remember what worked before, but the paper finds that repeated rewriting slowly damages the memory. The core idea is that raw episodes, meaning the actual past attempts and solutions, often stay more useful than the polished lessons made from them. The authors tested this across tasks like web shopping, simulated worlds, app use, and ARC-style puzzle problems where they could control the correct solutions. The sharpest result is that GPT-5.4 solved 100% of a small ARC-AGI set with no memory, but after memory was built from correct solutions, streaming updates dropped it to about 54%. The failures came from bad grouping, overbroad lessons, and overfitting, so the memory forgot details, mixed up task types, or learned rules that only worked on narrow examples. The big deal is that agent memory should not automatically rewrite every experience into a summary, because keeping raw evidence and only sometimes making summaries worked better. The paper is really proposing that agent memory should treat raw past episodes as important evidence, not as disposable notes to summarize away. ---- arxiv. org/abs/2605.12978 Title: "Useful Memories Become Faulty When Continuously Updated by LLMs"

译伊利诺伊大学和清华大学等实验室研究发现，LLM智能体重复重写自身记忆会导致记忆变得更不可靠。原始经历（实际过往尝试和解决方案）往往比提炼后的总结更有用。测试中，GPT-5.4在小型ARC-AGI数据集上无记忆时正确率100%，但建立记忆并持续更新后降至约54%。失败原因包括分组不当、教训过度泛化及过拟合。研究建议智能体不应自动将每个经历重写为摘要，保留原始证据并仅偶尔总结效果更好。

Rohan Paul@rohanpaul_ai · 6月3日60

AI agents should treat memory as a changing web of useful connections, not static storage. Most agent memory systems retrieve old facts as if the past were a filing cabinet. The paper proposes FluxMem, a memory system that stores facts, past task episodes, and reusable skills as connected pieces in a graph. When the agent works on a task, FluxMem first gathers likely useful memories, then uses feedback from the task to fix the memory connections by adding missing links, removing bad ones, or rewriting memories at the right level of detail. Over time, it also turns repeated successful task paths into reusable skills, so the agent does not need to rebuild the same reasoning pattern again and again. The authors tested FluxMem on long conversation memory, web navigation, and general assistant tasks, which checks whether the idea works across very different agent problems. FluxMem got stronger results than the compared memory systems, including 95.06 average accuracy on LoCoMo and a 12.73-point gain on GAIA with Kimi K2. The big deal is that the paper shifts agent memory from “store and retrieve” toward “keep repairing and strengthening the connections that actually help the agent act.” ---- Link – arxiv. org/abs/2605.28773 Title: "Rethinking Memory as Continuously Evolving Connectivity"

译FluxMem是一种新型AI智能体记忆系统，其核心思想是将记忆视为一个动态连接的网络，而非静态存储。它将事实、过往任务经历与可复用技能作为图中的节点进行存储。执行任务时，FluxMem先收集可能有用的记忆，再根据任务反馈动态修正记忆间的连接关系。此外，系统能将反复成功的任务路径转化为可复用技能。测试显示，该系统在LoCoMo基准上平均准确率达95.06，并在GAIA基准上结合Kimi K2取得了12.73分的性能提升，优于现有记忆系统。

elvis@omarsar0 · 6月2日50

// Scaling Behavior of Single LLM-Driven Multi-Agent Systems // Does adding more agents actually make a multi-agent system better? It's possible that collective intelligence emerges from interaction design rather than from agent plurality. This is something important to understand if you are building multi-agent systems. This new study reports that the optimal number of agents depends on the base model's capability and the task type, not on adding more of them. Paper: https://arxiv.org/abs/2606.00655 Learn to build effective AI agents in our academy: https://academy.dair.ai/

译研究探讨添加更多智能体是否提升多智能体系统性能。结论指出，最优智能体数量取决于基础模型的能力和任务类型，而非单纯增加数量。集体智能更可能源于精心的交互设计，而非智能体数量的增多。相关论文："Scaling Behavior of Single LLM-Driven Multi-Agent Systems"。

elvis@omarsar0 · 6月1日71

Very good advice on self-improving agents. (bookmark it) This is something I am seeing in my own experiments with coding agents and harnesses for long-horizon tasks. What I have found is that stronger models do not always evolve better agents. The current believe in self-evolving agents is that a bigger model writes better prompt and skill edits, so devs put their best model in the evolver seat. New research shows that intuition is mostly wrong. The work separates two abilities that usually get conflated. Producing harness updates stays flat across model capability, so Qwen3.5-9B writes edits roughly as good as Claude Opus 4.6. Benefiting from those updates follows an inverted-U that peaks at mid-tier models, while weak models fail to even activate the edits and strong models have little headroom left. This is important to understand as it tells you where to spend. Put a cheap model on the evolver and your expensive model on the solver, because the gains land solver-side, not evolver-side. Paper: https://arxiv.org/abs/2605.30621 Learn to build effective AI agents in our academy: https://academy.dair.ai/

译该研究指出，在自我改进的AI智能体中，“更强模型总能写出更好进化器提示词”的直觉是错误的。工作区分了两种能力：产生更新的能力在不同模型间趋于平坦，而从更新中受益的能力呈倒U形曲线，在中等模型处达到顶峰。弱模型无法有效激活更新，强模型则因已处性能高位而获益甚微。因此，成本效益最佳的配置是：使用廉价的中等模型担任“进化器”，而将昂贵的强模型用作“求解器”。

Rohan Paul@rohanpaul_ai · 6月1日60

Better AI agent systems scale by remembering useful feedback, not by spending more compute. The simple mistake is to count tokens, calls, or dollars as if they were all evidence. The authors say those numbers miss the real issue, because 2 runs can spend the same budget while only 1 gets feedback that is correct, new, relevant, and remembered. An agent harness is not just a wrapper around a model; it is a feedback machine that decides what to test, what to trust, what to store, and what to ignore. Their answer is Effective Feedback Compute, or EFC, a score that counts feedback only when it teaches the agent something useful and changes later decisions. They also divide EFC by task demand, because a small lookup task and a messy software-repair task need different amounts of helpful feedback before the agent has enough to solve them. They tested this on synthetic tasks, code tasks with executable tests, real benchmark traces, held-out settings, and a new prospective batch, then compared EFC with raw compute and a strong agent-scaling baseline. The main result is that task-normalized EFC predicted failures much better than raw compute, and in 1 matched-budget test, better feedback raised success from 0.27 to 0.90 while cost and tool calls stayed fixed. ---- Link – arxiv. org/abs/2605.29682 Title: "Scaling Laws for Agent Harnesses via Effective Feedback Compute"

译当前AI智能体的扩展方法常错误地将计算资源消耗等同于学习证据。新研究指出，两次运行消耗相同预算，但反馈的有效性可能天差地别。为此，研究提出了“有效反馈计算”（EFC）指标，仅统计那些正确、新颖、相关且被记住、并能改变后续决策的反馈。研究还结合任务需求对EFC进行归一化。实验表明，任务归一化的EFC比原始计算指标更能预测失败。在一项匹配预算测试中，采用更好反馈的方法将任务成功率从0.27提升至0.90，而成本和工具调用次数保持不变。 链接：arxiv.org/abs/2605.29682 标题："Scaling Laws for Agent Harnesses via Effective Feedback Compute"

elvis@omarsar0 · 6月1日60

// The Efficiency Frontier // Cool paper on context management. As agents reuse the same documents and histories across many turns, the cheapest context strategy is not fixed. This work describes a principled rule for picking one per deployment instead of defaulting to whatever topped a benchmark in isolation. Retrieval and compression methods are almost always benchmarked on accuracy and cost separately, so you never learn when one actually beats another under real load. The Efficiency Frontier models context strategy selection as a single cost-performance problem, with a log-utility term for diminishing returns from extra context and a reuse parameter N that amortizes preprocessing across repeated queries. Sweep N and the optimal strategy changes, exposing crossover regions where retrieval, compression, or full context each wins. On 5,000 HotpotQA instances, deployment-aware selection cuts effective token usage about 25 percent at the same performance, and amortized memory compression runs over 50 percent cheaper than full-context prompting in higher-performance settings. Paper: https://arxiv.org/abs/2605.23071 Learn to build effective AI agents in our academy: https://academy.dair.ai/

译该论文指出，当AI智能体在多轮对话中重复使用相同文档和历史记录时，固定的上下文策略并非最优。研究提出了“效率前沿”框架，将上下文策略选择建模为一个成本与性能的平衡问题。通过引入重用参数N进行扫描，可以识别出检索、压缩或全上下文各自占据优势的交叉区域。在5000个HotpotQA实例上的测试表明，部署感知的选择能在保持相同性能下减少约25%的有效token使用量，而摊销内存压缩在高性能设置下比全上下文提示的运行成本便宜超过50%。

AK@_akhaliq · 5月30日55

DynaFLIP Rethinking Robotics Perception via Tri-Modal-Dynamics Guided Representation

译DynaFLIP 通过三模态动态引导的表征重新思考机器人感知

elvis@omarsar0 · 5月29日68

// Scaling Laws for Agent Harnesses // If you build agent harnesses, this one is worth your time. (bookmark it) Most harness tuning treats every token and tool call as if volume is all that counts. New research shows that most of it does not. The work introduces Effective Feedback Compute (EFC), a coordinate that counts only the feedback an agent can actually act on. Raw token and tool-call counts explain agent failure at R2 of 0.33 to 0.42. EFC pushes that to 0.99. Why does it matter? Once you budget by useful feedback instead of raw volume, reallocation alone lifts success from 0.27 to 0.90 at the same compute. This also turns harness design from guesswork into something you can predict. Paper: https://arxiv.org/abs/2605.29682 Learn to build effective AI agents in our academy: https://academy.dair.ai/

译新研究提出“有效反馈计算（EFC）”指标，用于优化AI智能体测试框架的设计。传统评估中，原始token数和工具调用次数预测智能体失败的R²值仅为0.33至0.42，而EFC将此提升至0.99。基于EFC进行资源重分配，可在相同计算量下将智能体成功率从0.27显著提升至0.90，使框架设计从经验猜测变为可预测过程。

Rohan Paul@rohanpaul_ai · 5月29日57

This paper shows how LLMs can use shorter context more cheaply without losing much answer quality. Shows choosing the right context method for the deployment setting can cut token use by about 25% at similar quality, and by over 50% in some reused-memory cases. The problem is that long context gives a model more information, but every extra token costs money and compute, and the extra context often brings smaller gains. Longer context has diminishing returns, and the expensive tokens are often the ones added after the model already has enough signal. The authors propose an Efficiency Frontier, which compares context strategies by looking at answer quality and token cost together instead of treating them as separate scores. The key idea is that some methods are cheap per question, like retrieval, while others spend more upfront, like memory compression, but become cheaper when the same processed context is reused many times. They tested this on 5,000 HotpotQA questions, where the model has to combine facts across documents while ignoring distracting text. The main result is that the best context strategy changes with the setting: lightweight retrieval works best when reuse is low, memory compression becomes better when reuse is high, and full-context prompting is still needed for the highest scores. ---- Link – arxiv. org/abs/2605.23071 Title: "The Efficiency Frontier: A Unified Framework for Cost-Performance Optimization in LLM Context Management"

译论文提出了“效率前沿”框架，用于统一评估LLM上下文管理策略的成本与性能权衡。核心发现是，在部署时选择合适的上下文方法可使token使用量减少约25%，在部分记忆复用场景下可降低超50%成本，且答案质量损失较小。研究指出，上下文长度存在收益递减，后增加的token成本高但收益小。在5000个HotpotQA问题的测试中，轻量检索适合低复用率，记忆压缩在高复用率下更优，而全上下文提示仍是获取最高性能所需。

Rohan Paul@rohanpaul_ai · 5月29日60

In 2025, peer-reviewed journals published 147,000 citations to studies that don't exist !! Researchers went through 2.5 million papers and found citations tied to non-existent studies, authors, and journals. AI made them up. Nobody caught the problem. Now they are sitting in the scientific record for good. The scary part is not how many there are. It is how fast this is growing. In 2023, it was 1 in 2,828 papers. By early 2026, it is 1 in 277. That curve does not magically flatten. --- arxiv .org/pdf/2605.07723

译研究人员审查250万篇论文发现，2025年同行评议期刊中出现了14.7万次指向不存在研究的引用，这些虚假的研究、作者和期刊由AI生成，且未被察觉，已永久留在科学记录中。问题增长迅速：2023年每2828篇论文中约1篇含此类引用，到2026年初已升至每277篇约1篇。完整研究见arxiv论文2605.07723。

elvis@omarsar0 · 5月29日63

// Memory as Connectivity // One of the cleaner reframings of agent memory I have seen this month. FluxMem treats memory as the continuously evolving topology of a heterogeneous graph. Three stages run together: initial connection formation, feedback-driven refinement, and long-term consolidation of recurrent successful trajectories into reusable procedural circuits. During execution, it repairs missing links, prunes interference, and aligns abstraction granularity. SOTA on LoCoMo, Mind2Web, and GAIA across three distinct memory regimes. Paper: https://arxiv.org/abs/2605.28773 Learn to build effective AI agents in our academy: https://academy.dair.ai/

译提出了一种名为FluxMem的AI智能体记忆架构，其核心理念是将记忆视为一个持续演化的异构图拓扑。该框架通过三个并行阶段运行：初始连接形成、基于反馈的精炼，以及将反复成功的轨迹长期整合为可复用的程序性回路。执行过程中，它会修复缺失链接、剪枝干扰信息并调整抽象粒度。该方法在LoCoMo、Mind2Web和GAIA三个不同的记忆任务基准测试上均达到了SOTA水平。

AK@_akhaliq · 5月28日55

Agent Explorative Policy Optimization for Multimodal Agentic Reasoning

译多模态智能体推理的探索性策略优化

AK@_akhaliq · 5月28日49

Contrastive Distribution Matching for Amortized Sequential Monte Carlo in Discrete Diffusion

译离散扩散中摊销序列蒙特卡洛的对比分布匹配

Rohan Paul@rohanpaul_ai · 5月28日71

Image diffusion Transformers train poorly because their layers pass information in a fixed, outdated way. Now they can train much faster by changing how layers share information. With this paper, the same image quality arrived with 8.75x fewer training iterations. The surprise is not that Diffusion Transformers had an inefficiency, but where it was hiding. Researchers have spent years refining attention, conditioning, tokenization, objectives, and autoencoders, while leaving the residual stream mostly untouched because it looked like plumbing rather than intelligence. In a standard residual stack, every layer keeps adding its output to the running stream, which sounds harmless until the stream’s magnitude swells, gradients fade backward, and neighboring blocks begin saying nearly the same thing. That is bad for any Transformer, but it is especially awkward for diffusion, because denoising is not one fixed task repeated at every step. The authors found 3 signs that this old setup hurts the model: signals get too large going forward, learning signals fade going backward, and nearby blocks often produce almost the same features. Their fix is Diffusion-Adaptive Routing, a replacement that lets each layer choose which earlier layer outputs to use, and the choice changes with the denoising timestep. The big deal is that the paper does not add a new image dataset, loss, tokenizer, or attention trick, but instead questions the old residual connection that most models kept copying from language Transformers. ---- Link – arxiv. org/abs/2605.20708 Title: "Rethinking Cross-Layer Information Routing in Diffusion Transformers"

译传统Diffusion Transformers因层间信息传递方式固化导致训练效率低下。研究团队提出Diffusion-Adaptive Routing方法，允许每层动态选择使用哪些早期层的输出，且该选择随去噪时间步调整。该方法未引入新的数据集、损失函数或注意力机制，仅通过优化残差连接，使得相同图像质量所需的训练迭代次数减少8.75倍。

Ethan Mollick@emollick · 5月28日55

There is a lot being written about the stylistic tells of AI writing (em-dashes, etc.) but this paper looks at AI narrative tells Fascinating differences between AI &amp; human narrative, and asking AI to write in different styles doesn't do much to change it https://arxiv.org/abs/2604.03136

译关于AI写作的风格特征（如破折号等）已有大量讨论，但这篇论文关注的是AI叙事特征 AI与人类叙事之间存在引人入胜的差异，要求AI以不同风格写作并不能显著改变这一点 https://arxiv.org/abs/2604.03136

AK@_akhaliq · 5月28日65

Gamma-World Generative Multi-Agent World Modeling Beyond Two Players

译Gamma-World 超越双人对战的生成式多智能体世界建模

Rohan Paul@rohanpaul_ai · 5月28日65

Long-running language agents may work better if they periodically stop to consolidate memory. The problem is that today’s transformer agents get slower and more expensive as their context grows, because attention has to keep checking more past tokens. The usual fix for long context is to keep more tokens nearby, but that turns every next-token prediction into a larger search through the past. The sharper idea here is that memory is not only storage. Sometimes the hard part is converting a messy stretch of experience into a state that can actually be used later. So the paper’s idea is to add a sleep phase, where the model pauses, rereads recent context several times, writes the useful information into fixed-size memory layers, and then clears the short-term attention cache. During sleep, the model runs several offline passes over recent context, writes the result into fast weights inside its state-space blocks, then clears the attention cache. This means the model pays extra compute while sleeping, not while answering, so normal prediction can still happen with 1 forward pass. The authors test this on cellular automata, graph lookup, and GSM-Infinite math problems, where the model must use old information that is no longer sitting in its attention cache. The main result is that longer sleep improves performance, especially on harder cases that need deeper reasoning rather than just remembering a fact. The big deal is that long-horizon agents may not need to carry bigger and bigger raw context forever, because they can consolidate the important parts and safely forget the raw tokens. ---- Link – arxiv. org/abs/2605.26099 Title: "Language Models Need Sleep"

译针对当前Transformer智能体因上下文不断增长而推理变慢变贵的问题，论文提出效仿人类睡眠机制进行记忆巩固。其核心方案是加入周期性的“睡眠阶段”：模型在此阶段暂停，多次重读近期上下文，将有用信息写入固定大小的记忆层（如状态空间块的快速权重），然后清空短期注意力缓存。此离线过程使后续回答仍只需一次前向传播。在细胞自动机、图查找和GSM-Infinite数学问题上的测试表明，更长的睡眠时间能提升性能，尤其对需要深度推理的复杂任务。该思路表明，长期智能体或可通过记忆巩固实现高效遗忘与重用，不必无限携带原始上下文。

elvis@omarsar0 · 5月28日57

// Your Agents are Aging Too // Huh!? They need "sleep," and now they are aging? Joke aside, great write-up on reliable agentic engineering. This new research introduces AgingBench, a longitudinal reliability benchmark. It organizes agent aging into four mechanisms, including compression aging and interference aging, and measures not just whether deployed agents degrade but what form the degradation takes and where repair should target. We benchmark agents on day one and then deploy them for months. That gap hides a basic systems question. How long does an agent stay reliable after deployment? Even with frozen model weights, an agent's effective state keeps shifting. It compresses interaction history, retrieves from a growing memory store, revises facts after updates, and goes through routine maintenance. Reliability becomes a lifespan property of the full harness, not a snapshot of the base model. Paper: https://arxiv.org/abs/2605.26302 Learn to build effective AI agents in our academy: https://academy.dair.ai/

译这项研究提出了AgingBench，一个用于纵向评估AI智能体可靠性的基准。它将智能体老化归纳为四种机制，包括压缩老化和干扰老化，旨在衡量部署后的智能体是退化以及退化形式。研究指出，即使冻结模型权重，智能体的有效状态也会因压缩交互历史、检索记忆库、事实更新等操作而不断变化，其可靠性是整个运行系统的寿命属性，而非基础模型的快照。基准测试在智能体部署第一天进行，然后持续数月。

karminski-牙医@karminski3 · 5月27日69

什么?! skill 也能"训练"了? 以往大家都是凭经验让AI写 skill, 然后调试的时候也是运行几下感觉没bug就完事了. 但 skill 能运行就一定好吗? 于是微软联合上交复旦同济等机构发了一个新框架 SkillOpt, 直接让AI评估skill写的好不好然后不断去优化! 最终, 这个框架写的 skill 让GPT-5.5的直接对话准确率飙升了 23.5分! 这个框架具体是怎么做的也很简单, 让skill迭代过程实现 harness 闭环! 大模型写完 skill 后, 立刻进入跑分流程, 只有得分更高的 skill 变更才会留下来. 跟大模型的强化学习过程如出一辙. 框架的设计也很值得做 Agent 框架的同学借鉴, 比如: 它设计了一个独立的优化器模型, 这个模型是用来写 skill 的, 它会根据 Agent 执行任务的试错表现得分, 对 skill 进行编辑操作(增加、删除、替换文本). 然后就是 harness 流程了：每一次文本编辑都必须在独立的验证集上分数有提升, 才会允许合并. 最后, 也是最精彩的地方, 框架还引入深度学习训练机制, 设计了文本层的学习率预算, 这个的核心就是限制大模型每次只能修改skill的一小部分, 慢慢迭代, 而不是全都重写. 论文中最有价值的数据就在这里, 论文实验发现, 每一步设置 4 到 8 个编辑操作的预算效果最好. 最终的最佳 skill 往往只包含 1 到 4 个被接受的核心修改. 甚至他们还设计了被拒编辑缓冲区, 用来存储训练过程的反面胶材, 以及周期性慢速/元更新, 这个则是跑完一个周期后, 会进行一次盘点, 类似于让框架形成记忆, 能更好的维持后续迭代. 这篇论文的结论十分深刻: skill(prompt) 完全配得上, 也需要一套系统级的训练流程. 原文中的描述直接是: 我们主张, skill 应当作为 Agent 的外部冻结状态来被"训练", 并且训练过程还要"让权重空间优化具有可重复性"! 这是不是意味着, 提示词工程(Prompting)和模型训练(Training) 的界限将逐渐变得模糊? 而提示词工程完全进入了机器学习的领域. 也许很快, 我们再也不需要人类去手动瞎改和调试提示词了! 论文地址: http://arxiv.org/pdf/2605.23904 #skillopt #微软 #提示词工程 #harness

译微软联合上海交通大学等机构发布SkillOpt框架，旨在通过机器学习流程系统性地优化AI智能体的技能。该框架引入独立的优化器模型，通过harness闭环流程对技能进行编辑，且每次编辑必须在验证集上带来分数提升才被接受。框架设置了每步4到8个编辑操作的学习率预算，使核心修改控制在1到4个。实验表明，优化后的技能可使GPT-5.5的对话准确率提升23.5分。

Rohan Paul@rohanpaul_ai · 5月27日52

A long-context AI can be poisoned by a few plausible wrong passages, not gradually worn down by many. At just 10% bad context, the damage is already almost done. “THE FIRST DROP OF INK ” effect, analogous to how a single drop of ink contaminates water. The mistake is to picture context as storage. In a long prompt, the model is not calmly filing facts into separate boxes; it is running a competition over which pieces of text deserve attention when the answer is generated. Hard distractors are dangerous because they are not random junk. They are close enough to the question to look useful, but wrong enough to pull the model away from the gold evidence. In the authors’ setup, if performance loss were proportional, the first 10% of hard distractors would explain about 10% of the total damage, but in one 128K-token Qwen2.5 setting it explained 58%. The mechanism is simple once you see it: softmax attention rewards relative closeness, so a misleading passage that sits near the answer in logit space can crowd the denominator far more than irrelevant filler. At only 10% hard distractors, they can already account for about 97% of the distractor pressure. This also changes how we should read filtering results. If removing documents helps, the benefit may come less from removing “bad” content than from shortening the whole battlefield. For long-context systems, the safest misleading passage is the one that never enters the prompt. --- Link – arxiv .org/abs/2605.10828 Title: "The First Drop of Ink: Nonlinear Impact of Misleading Information in Long-Context Reasoning"

译ICML 2026论文揭示，长上下文大语言模型的性能并非随错误信息增加而线性下降，而是呈现“第一滴墨水”效应。研究发现，仅当上下文包含10%的高难度错误文本时，损害就已基本完成。例如，在一个128K-token的Qwen2.5设置中，这最初的10%错误文本造成了58%的性能损失。其机制在于softmax注意力机制会赋予与问题相近但错误的文本过高权重，仅这10%的高难度干扰文本就能贡献约97%的干扰压力。因此，过滤文档带来的提升可能主要源于缩短了有效上下文，而非移除“坏内容”。

Berryxia.AI@berryxia · 5月26日44

别被骗了！ 大模型也特么需要“睡觉”？ 一个来自CMU和UMD的研究团队发现：Transformer大模型在处理超长任务时注意力机制彻底拉胯 他们没有继续堆上下文长度而是直接给模型安排了“睡眠” 模型在睡眠期间把最近的上下文全部转化成持久的fast weights然后清空KV cache 这个机制叫“sleep-like consolidation”大模型也需要睡觉 故事就藏在2026年5月25日刚出的arXiv 2605.26099里 标题直白到离谱：《Language Models Need Sleep》 作者Sangyun Lee、Sean McLeish、Tom Goldstein、Giulia Fanti 传统Transformer在长时序任务上越跑越累因为attention对上下文长度是二次方爆炸。 KV cache占显存越来越多推理速度越来越慢。 他们提出的方案超级生物启发： 模型每隔一段时间进入“睡眠模式” 先把最近积累的上下文做N次离线循环遍历 然后通过一个学会的局部规则把这些信息固化到state-space model块里的fast weights里 固化完直接清空KV cache 醒来后模型继续工作但记忆已经从“短期易失”变成了“长期持久” 实验结果直接证明：增加睡眠深度或者睡眠时长能显著提升睡眠后的推理能力 这不是又一个参数技巧而是彻底改变了模型处理长上下文的范式。 Big Tech还在疯狂卷把上下文拉到百万级靠暴力堆显存。 这个小团队却用“睡觉”这个最简单的人类机制把问题从根上解决了。 整个框架100%开源论文代码思路全在arXiv上。 Big Tech的闭源长上下文订阅模式靠的就是你不知道模型其实可以“睡觉”来省资源。

译CMU与UMD的研究团队在论文《Language Models Need Sleep》（arXiv 2605.26099）中指出，传统Transformer模型在处理长任务时，因注意力机制计算复杂度高及KV cache显存占用持续增长而导致效率低下。为此，他们提出了受生物启发的“类睡眠巩固”机制：模型会周期性进入“睡眠”状态，离线多轮处理最近的上下文，并将信息固化到模型状态空间块的fast weights中，随后清空KV cache。实验表明，增加睡眠深度或时长能显著提升模型后续的推理能力。该框架完全开源，提供了一种区别于暴力堆显存的长上下文处理新范式。

Rohan Paul@rohanpaul_ai · 5月26日61

Brilliant new paper from Meta, CMU and other labs. Shows that coding agents improve faster by manufacturing their own software experience. Coding agents can train themselves by making and fixing bugs inside real projects. Most coding agents still learn from human leftovers: issues, pull requests, tests, comments, and benchmarks that describe what went wrong. That is useful, but it makes the agent dependent on the rate at which humans produce clean, verifiable lessons. Self-play SWE-RL changes the unit of learning from a labeled task to an executable situation. One version of the model explores a real codebase, weakens tests, injects a meaningful bug, and leaves behind test artifacts that define the failure without needing an English issue description. Another version of the same model has to repair the system, not by matching words to patches, but by restoring behavior under tests. Here’s the key point: the test is not just a grader here, it is the language of the problem. That matters because software understanding lives in constraints, dependencies, edge cases, and invariants that prose often compresses or misses. The reported gains, +10.4 points on SWE-bench Verified and +7.8 on SWE-Bench Pro, are early but hard to ignore because evaluation still used natural-language issues the self-play system did not train on. That suggests SSR (Self-play SWE-RL) is learning something deeper than issue phrasing, though not yet anything like open-ended mastery. The restraint matters: generated bugs can be artificial, rewards can be noisy, and sandboxed repositories are still a narrow slice of software reality. Still, the direction is sharp. The next bottleneck for coding agents may not be more human-written tasks, but more ways for agents to encounter, create, survive, and learn from failure. ---- Paper Link – arxiv. org/abs/2512.18552 Paper Title: "Toward Training Superintelligent Software Agents through Self-Play SWE-RL"

译Meta、CMU等机构在论文中提出了Self-play SWE-RL方法。该方法让编程智能体通过“自我博弈”生成训练数据，而非仅依赖人工标注的问题。具体而言，一个模型探索代码库、注入bug并留下测试用例来描述问题；另一个模型则学习根据测试修复系统。其中，测试成为了描述问题的核心语言。该方法在SWE-bench Verified上提升了+10.4分，在SWE-Bench Pro上提升了+7.8分。值得注意的是，评估使用了该系统未训练过的自然语言问题，表明其可能学到了更深层的软件理解能力。

Rohan Paul@rohanpaul_ai · 5月26日65

This Meta + Stanford + Illinois survey paper argues that AI agents work better when code becomes their main working layer. The problem is that an LLM by itself is mostly a text predictor, so long tasks can lose state, hide mistakes, and turn plans into actions in fragile ways. The real advance is not “AI writes code,” but “AI uses code as the environment it thinks inside.” The authors call the surrounding system an agent harness, meaning the tools, memory, sandboxes, checks, and feedback loops that turn a model into an agent. Their core idea is that code should sit at the center of that harness, because code can be run, inspected, checked, saved, edited, and shared. Tests become sensors. Repositories become memory. Logs become history. Sandboxes become boundaries. A generated script is no longer merely an answer; it is a handle the system can run, check, revise, share, and roll back. The main finding is a pattern across many fields: code helps agents reason through executable steps, act through tool calls or control programs, and model environments through tests, traces, logs, repositories, and simulators. ---- Paper Link – arxiv. org/abs/2605.18747 Paper Title: "Code as Agent Harness"

译Meta、斯坦福与伊利诺伊的研究论文指出，AI智能体在将代码作为主要工作层时性能更佳。论文认为，大语言模型（LLM）作为文本预测器，在处理长任务时存在状态丢失、错误隐蔽等问题。真正的进步并非“AI写代码”，而是“AI在代码环境中思考”。论文的核心是提出一个以代码为中心的“智能体框架”，即工具、记忆、沙箱等系统。在此框架中，测试成为传感器，代码库成为记忆，日志成为历史，沙箱成为边界。生成的脚本成为可运行、检查、修改和共享的操控对象。总结发现，代码能通过可执行步骤帮助智能体推理，通过工具调用行动，并通过测试、日志等对环境进行建模。

Rohan Paul@rohanpaul_ai · 5月25日65

New Alibaba + Nanjing Univ paper claims million-token prefill can be sped up 9.36X (compared against FlashAttention-2) with only lightweight adaptation Shows standard LLMs can handle very long context faster by making attention selectively sparse. The problem is that full attention gets very expensive when the input grows to hundreds of thousands or 1M tokens, because the model keeps comparing too many tokens with too many other tokens. The paper’s claim is that a trained full-attention model already has a hidden sparse structure, so the model does not need to be rebuilt or trained from scratch. RTPurbo uses that structure by finding the few attention heads that really need faraway tokens, while letting the other heads focus mostly on nearby text. For those retrieval heads, it uses a small 16-dimensional token finder to guess which old tokens matter, then runs the real attention only on that selected set. The authors tested this on long-context benchmarks and reasoning tasks, and RTPurbo kept accuracy close to full attention while reaching up to 9.36x faster prefill at 1M tokens and about 2x faster decoding. RTPurbo's engineering rule: keep expensive long-context access only where it matters, and route the rest through a smaller search space. The clever part is the 16-dimensional indexer. It does not replace the model’s real attention computation; it acts like a cheap scout, finding likely useful tokens before the full representation is used on the selected set. RTPurbo is not proof that every model can be safely sparsified this way. But it is strong evidence that the waste in long-context inference is more structured than it looks. ---- Paper Link – arxiv. org/abs/2605.16928v1 Paper Title: "Full Attention Strikes Back: Transferring Full Attention into Sparse within Hundred Training Steps"

译阿里巴巴与南京大学提出RTPurbo，一种轻量级适配方法。该方法发现，已训练的全注意力模型内存在隐藏的稀疏结构。它利用一个轻量的16维token查找器作为“侦察兵”，为少数需要长程信息的关键注意力头定位重要token，而让其他头主要关注局部文本。基于此，RTPurbo在100万token预填充任务上，相比FlashAttention-2实现了高达9.36倍的加速，解码阶段也约有2倍加速，同时在长上下文和推理基准上保持了接近全注意力模型的精度。该研究表明，长上下文推理中的计算浪费具有可挖掘的结构性。

Rohan Paul@rohanpaul_ai · 5月24日39

Interesting paper 😀 "What the F*ck Is Artificial General Intelligence?" Defines intelligence as adaptability under limits of compute, memory, and energy. So AGI is a system that adapts at least as generally as a human scientist That means it should be able to plan experiments, learn cause and effect, balance exploration and action, and operate with autonomy. The paper calls this type of AGI an artificial scientist, because it is judged by its ability to discover and adapt across many tasks, not just by passing human-like tests. So AGI is not just “human-level AI” but a whole system that can adapt broadly, efficiently, and scientifically, at least as well as a human scientist. ---- arxiv. org/abs/2503.23923

译一篇新论文提出对通用人工智能（AGI）的明确定义，认为AGI是一种“人工科学家”。该模型需要像人类科学家一样，具备自主规划实验、学习因果关系、平衡探索与行动的能力。其核心在于适应性，即能在计算、记忆和能量限制下，像人类科学家一样广泛、高效且科学地适应新环境和任务，评判标准是其发现和适应能力，而非通过拟人化测试。