返回
原文
Rohan Paul@rohanpaul_ai
44AI 编辑部评分，满分 100
2026-08-07 00:25· 1天前
AI 导读

Agent Memory Challenge 推出统一基准，以 5,000 道题、同一答案模型和相同评判流程评测各智能体记忆系统，并分开排名开源与商业方案。文本赛道整合 10+ 数据集（约 1.5 亿字符历史），代码赛道含 12 个仓库、1,290 个历史 PR。由 20+ 研究机构组织，8 月 7 日截止提交，8 月中旬公布首批结果。

Agent memory benchmarks have had a basic attribution problem.

Every AI memory startup claims better recall, and until now there was no way to check.

That is why this new shared benchmark caught my attention.

It puts every entrant through the same pipeline: 5,000 questions, one fixed answer model, and the same judging process across systems.

The benchmark also separates open-source and commercial systems. That feels like the right choice: community projects can compete for prizes without being directly compared against heavily funded commercial products.

A group of 20+ research institutions is running all of them through one identical pipeline and publishing the results.

Entries close on August 7, with the first public rankings expected in mid-August.

I'll be watching the first results closely.

Agent Memory LeaderboardEvery agent memory system publishes its own numbers. Almost none of them are comparable. Different datasets, different answer models, different judges. When a s...
智能体评测/基准

来源：Rohan Paul · x.com

导出 Markdown
返回
44在 X 查看原推
导出 Markdown
Rohan Paul · @rohanpaul_ai · X·2026-08-07 00:25·1天前
AI 导读

Agent Memory Challenge 推出统一基准，以 5,000 道题、同一答案模型和相同评判流程评测各智能体记忆系统，并分开排名开源与商业方案。文本赛道整合 10+ 数据集（约 1.5 亿字符历史），代码赛道含 12 个仓库、1,290 个历史 PR。由 20+ 研究机构组织，8 月 7 日截止提交，8 月中旬公布首批结果。

Agent memory benchmarks have had a basic attribution problem.

Every AI memory startup claims better recall, and until now there was no way to check.

That is why this new shared benchmark caught my attention.

It puts every entrant through the same pipeline: 5,000 questions, one fixed answer model, and the same judging process across systems.

The benchmark also separates open-source and commercial systems. That feels like the right choice: community projects can compete for prizes without being directly compared against heavily funded commercial products.

A group of 20+ research institutions is running all of them through one identical pipeline and publishing the results.

Entries close on August 7, with the first public rankings expected in mid-August.

I'll be watching the first results closely.

Agent Memory LeaderboardEvery agent memory system publishes its own numbers. Almost none of them are comparable. Different datasets, different answer models, different judges. When a s...

来源：Rohan Paul· x.com

智能体评测/基准