Business Arena:在真实市场环境中评测 LLM 智能体的商业运营能力

HuggingFace Daily Papers(社区热门论文)·2026-08-09 08:00·17天前
AI 导读

研究者推出 Business Arena,一个让 AI 智能体长期经营跨境店铺的受控环境,数据基于真实 Alibaba.com 采购信息与权威市场条件校准。对 15 个前沿模型的评测显示,最终净资产均值相差九倍,即便最佳模型也落后于人类设计的策略,表明商业运营对 LLM 智能体仍具挑战。

HuggingFace Daily Papers(社区热门论文)
59AI 编辑部评分,满分 100

Business Arena:在真实市场环境中评测 LLM 智能体的商业运营能力

2026-08-09 08:00· 17天前
AI 导读

研究者推出 Business Arena,一个让 AI 智能体长期经营跨境店铺的受控环境,数据基于真实 Alibaba.com 采购信息与权威市场条件校准。对 15 个前沿模型的评测显示,最终净资产均值相差九倍,即便最佳模型也落后于人类设计的策略,表明商业运营对 LLM 智能体仍具挑战。

Running a business is a challenging form of intelligent work. Operators must infer opportunities from partial signals, commit capital under uncertainty, adapt to delayed outcomes in a changing market, and satisfy regulatory obligations before trading legally. Frontier LLM agents can increasingly complete complex workflows, yet business-related capabilities are rarely evaluated in existing agent benchmarks. We introduce Business Arena, a controlled environment where an AI agent runs a cross-border shop, buying from suppliers and selling to buyers over a long horizon. We ground the arena in real Alibaba.com sourcing data and market conditions calibrated from authoritative sources. Delayed and coupled consequences make individual business decisions difficult to judge, but their combined outcome is measurable through profit. Because profit alone cannot explain why an agent succeeds or fails, we compare agents with human-designed strategies to estimate available opportunity, use skill-level metrics to reveal underlying strengths and weaknesses, and trace realized gains and losses to the actions that produced them. We use mechanism ablations to establish that strong results reflect genuine business intelligence rather than neglect or simulator-specific shortcuts. We evaluate 15 frontier models and find a ninefold difference in mean final net worth. Even the best model falls behind human-designed strategies, indicating that business operation remains challenging for LLM agents. Skill-level analysis reveals operating styles, from margin-focused premium sellers to high-turnover wholesalers and customer-service specialists, while action-level attribution identifies the sourcing, pricing, and recovery decisions that create or destroy value. Together, Business Arena takes a first step toward a realistic and trustworthy testbed for evaluating end-to-end business agents.

来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org