大语言模型（LLM）与智能体的最新进展，已大幅提升了 AI 系统执行复杂任务的能力。然而，现有评测基准大多依赖研究者自行挑选的任务，因此尚不确定这些进展能否延伸到真实用户实际要求 AI 系统完成的工作上。我们推出了 StartupBench——一个以市场验证过的 AI 初创产品为基础的端到端智能体评测基准。我们并非基于对“有用智能体能力”的事先假设来定义任务，而是系统性地研究那些已被证明获得实际采用率的 AI 产品，连同其产品工作流与用户群体，从而识别出 AI 在多个专业领域已具备切实需求的实际任务。我们将这些工作流转化为以交付物为导向的完整任务，并用细粒度评分细则来捕捉其复杂要求。在统一智能体框架下对代表性模型进行评估后发现，即便最强的模型在 StartupBench 上也仅能成功完成约 30% 的任务，尽管在许多任务上已取得可观的阶段性进展。进一步分析表明，复杂指令遵循与领域专业知识等方面是主要的失败来源。我们的结果表明，许多经过市场验证的工作流仍超出当前通用智能体的可靠能力范围，这也使 StartupBench 成为衡量向端到端完成真实用户任务迈进程度的实证标尺。
Recent advances in Large Language Models(LLMs) and agents have substantially improved the ability of AI systems to execute complex tasks. Yet existing benchmarks largely rely on researcher-selected tasks, leaving uncertain whether such progress extends to the work that real-world users actually demand from AI systems. We introduce StartupBench, an E2E agent benchmark grounded in market-validated AI startup products. Rather than defining tasks from pre-defined assumptions about useful agent capabilities, we systematically study AI products with demonstrated adoption, together with their product workflows and users, to identify real-world tasks for which AI has established practical demand across diverse professional domains. We translate these workflows into complete deliverable-oriented tasks and evaluate them with fine-grained rubrics capturing their complex requirements. Across representative models evaluated under a unified agent harness, even the strongest model successfully completes only approximately 30% of StartupBench, despite making substantial partial progress on many tasks. Further analysis identifies aspects like complex instruction following and domain-specific expertise as major sources of failure. Our results reveal that many market-validated workflows remain beyond the reliable capabilities of current general-purpose agents, establishing StartupBench as an empirical measure of progress toward E2E completions of real-world user tasks.