# QuantCode-Bench：评估大语言模型生成可执行算法交易策略能力的基准测试

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-04-16 08:00
- AIHOT 链接：https://aihot.virxact.com/items/cmo71are600vxsli5jtv44f00
- 原文链接：https://arxiv.org/abs/2604.15151

## AI 摘要

研究团队发布QuantCode-Bench基准测试，用于评估大语言模型基于英文描述为Backtrader框架生成可执行算法交易策略的能力。该基准包含400个来自Reddit、TradingView等平台的真实任务，通过多阶段流水线评估语法正确性、回测执行、交易生成及语义对齐。测试显示，当前模型在单轮和多轮智能体设置下的主要失败模式并非语法错误，而是交易逻辑操作化、专用API使用及任务语义遵循方面的缺陷。

## 正文

Large language models have demonstrated strong performance on general-purpose programming tasks, yet their ability to generate executable algorithmic trading strategies remains underexplored. Unlike standard code benchmarks, trading-strategy generation requires simultaneous mastery of domain-specific financial logic, knowledge of a specialized API, and the ability to produce code that is not only syntactically correct but also leads to actual trades on historical data. In this work, we present QuantCode-Bench, a benchmark for the systematic evaluation of modern LLMs in generating strategies for the Backtrader framework from textual descriptions in English. The benchmark contains 400 tasks of varying difficulty collected from Reddit, TradingView, StackExchange, GitHub, and synthetic sources. Evaluation is conducted through a multi-stage pipeline that checks syntactic correctness, successful backtest execution, the presence of trades, and semantic alignment with the task description using an LLM judge. We compare state-of-the-art models in two settings: single-turn, where the strategy must be generated correctly on the first attempt, and agentic multi-turn, where the model receives iterative feedback and may repair its errors. We analyze the failure modes across different stages of the pipeline and show that the main limitations of current models are not related to syntax, but rather to the correct operationalization of trading logic, proper API usage, and adherence to task semantics. These findings suggest that trading strategy generation constitutes a distinct class of domain-specific code generation tasks in which success requires not only technical correctness, but also alignment between natural-language descriptions, financial logic, and the observable behavior of the strategy on data.