New ByteDance paper shows LLMs can build their own agent harnesses now, but making those harnesses reliably improve and transfer is much harder.

HarnessDev starts each creator model from a nearly empty runtime and asks it to build the execution loop, tools, context handling, recovery, and verification needed for real tasks.

The generated systems are already useful, but extremely uneven by domain.

Under self-evaluation, Opus 4.8 scores 84.6 on writing versus 83.7 for the selected external reference, yet only 52.4 on BrowseComp versus 92.2.

Portability is another problem: the same Opus code harness drops from 69.3 on SWE-Pro with Opus running it to 33.0 when Gemini 3.1 Pro becomes the executor.

That means harness design can co-adapt to 1 model instead of becoming a clean, reusable software layer.

– arxiv. org/abs/2609.01437

Title: "HarnessDev: Can LLMs Create and Evolve Their Own Agent Harness?"