# Task-CoEvolve：通过自适应验证任务选择实现高效 Harness 优化

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-08-24 08:00
- AIHOT 分数：48
- AIHOT 链接：https://aihot.virxact.com/items/cmt87dqb72u2dro73or7v9uhf
- 原文链接：https://arxiv.org/abs/2608.20169

## AI 摘要

Task-CoEvolve 提出一种高效的大语言模型 harness 优化方法，通过让验证任务与 harness 共同演化，解决固定验证集评估成本高的问题。该方法利用方差加权采样聚焦能力边界附近的任务，并基于采样概率估计全量集分数。在在线文本分类和 Terminal-Bench 2.1 上，Task-CoEvolve 匹配全量集搜索的最终性能，同时将优化过程中的评估次数减少 80%。

## 正文

We present a novel approach to efficient LLM harness optimization through adaptive validation task selection. Harness optimization iteratively rewrites the harness code based on validation performance, enabling substantial performance gains without updating the underlying model weights. Existing approaches, however, evaluate a fixed validation set in full at every iteration, incurring substantial evaluation costs even on tasks that become less discriminative as the harness evolves. We propose Task-CoEvolve, which co-evolves the validation tasks with the harness by addressing two challenges: selecting informative tasks and estimating full-set performance from partial evaluations. Task-CoEvolve builds on the observation that tasks on which candidate harnesses disagree are more informative for distinguishing among them than tasks that are consistently solved or failed. It uses variance-weighted sampling based on past outcomes to focus evaluation on tasks near the capability frontier, with the sampling distribution adapting as the harness evolves. It then estimates full-set scores from the sampled tasks by accounting for their sampling probabilities, enabling consistent comparisons across iterations despite evaluating different subsets. Experiments on online text classification and Terminal-Bench 2.1 show that Task-CoEvolve consistently outperforms subset-based baselines and matches the final performance of full-set search while reducing the number of evaluations during optimization by 80%. Code will be released at https://github.com/Agent4Science-UTokyo/Task-CoEvolve.