# AutoDesign：面向长周期智能体设计的元框架优化

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-08-13 08:00
- AIHOT 分数：57
- AIHOT 链接：https://aihot.virxact.com/items/cmssd3ass03tprod0rctgtlea
- 原文链接：https://arxiv.org/abs/2608.13560

## AI 摘要

AutoDesign 将多模态设计任务建模为元框架优化问题，通过元优化器引导代码智能体基于反馈递归改进设计框架。在 PosterBench 主赛道（100 篇论文、五个学科）上，AutoDesign 以 78.32 分超越 Claude Design 7.45 分；集成 DesignHarness 后，七种配置下平均得分从 54.99 提升至 67.39（+12.4%）。

## 正文

assets/autodesign-designer-title-logo-cropped.pdf 1]Meituan 2]MBZUAI 3]Huazhong University of Science and Technology 4]Peking University 5]Tsinghua University 6]The Chinese University of Hong Kong 7]Shanghai Jiao Tong University \contribution[*]Equal contribution \contribution[†]Project lead \resourceProject Websitehttps://autodesign.designanything.ai/ \resourceCode Repositoryhttps://github.com/Yaxin9Luo/AutoDesign \resourceDemo Pagehttps://designanything.ai/

AutoDesign: Meta-Harness Optimization for Long-Horizon Agentic Design

Yaxin Luo

Haobin Jiang

Jialv Zou

Xu Huang

Wenhao Yan

Haodong Li

Zhengrong Yue

Jing Li

Xiaofu Chen

Xiaohan Zhao

Jiacheng Liu

Jiacheng Cui

Zhiqiang Shen

Xiaotong Li

Affiliation:

Affiliation:

Affiliation:

Affiliation:

Affiliation:

Affiliation:

Affiliation:

Abstract

Transforming multimodal sources into condensed and structured media outputs can be fundamentally conceptualized as a long-horizon agentic process centered on a model-harness system. While an ideal harness system should align with human design priors and accumulate reusable experience through empirical exploration to drive recursive self-improvement, existing paradigms remain static and fall short of this capability. In this paper, we present AutoDesign, a framework that aligns with human design priors, where a meta-harness optimizer guides a code agent to recursively improve harness based on rollout feedback. To instantiate and evaluate this framework, we focus on the academic paper-to-poster generation task and introduce PosterBench, comprising a 100-paper Main Track spanning five disciplines and PosterBench-mini, a shared 10-paper subset for controlled evaluation. On the PosterBench Main Track, AutoDesign achieves the highest score of 78.32, surpassing the closed-source commercial system Claude Design by 7.45 points. Across seven controlled code agent–model configurations, integrating the learned DesignHarness consistently improves performance, increasing the average PosterBench Score from 54.99 to 67.39 (+12.4%). In a fully autonomous long-horizon loop, it executes 253 tool calls and 11 editing turns within 40 minutes for under $3, reaching average conference-poster quality in human evaluation. A system-blind human study further demonstrates that AutoDesign achieves the highest human preference among evaluated systems.

(a) Meta-harness optimization trace

(b) Performance gains from DesignHarness

Figure 1: AutoDesign progressively improves the design harness and the quality of the artifacts. (a) Score of the poster generated by the design harness for one representative paper, tracked across meta-harness iterations. Autonomous optimization improves the initial harness before reaching a plateau, after which human guidance redirects the search and yields a further gain. (b) The optimized harness, DesignHarness, improves all Coding Agents on PosterBench by 5.0 to 19.6 points, achieving a best overall score of 81.5.

Figure 2: AutoDesign for AutoDesign. A poster generated by AutoDesign for its own paper, crafted through a long-horizon agentic workflow that autonomously performs source ingestion, iterative generation and refinement, critic feedback integration, and finalization, completing design process in approximately 40 minutes with negligible human intervention.

1 Introduction

Human communication often involves understanding, organizing, and presenting information from diverse multimodal sources into human-facing artifacts, e.g., webpages, slides, posters, and videos (Fu et al. 2022; Pang et al. 2025; Zheng et al. 2025; Chen et al. 2025; Zhu et al. 2025). Achieving such multimodal input-to-output transformation (Wu et al. 2023; Chameleon Team 2024; Kim et al. 2026; Meituan LongCat Team et al. 2026) requires the ability to extract relevant evidence, reason over heterogeneous information, plan intermediate steps, and iteratively improve outputs based on feedback (Choi et al. 2026; Liu et al. 2026b), which naturally positions multimodal design as a suitable yet challenging long-horizon task for agentic coding. However, developing such systems remains difficult due to the complexity of real-world workflows and the reliance on extensive human feedback.

Current multimodal design systems seek human-aligned design priors through cycles of generation, critique, and revision, often using visual references or design-specific feedback (Sun et al. 2026; Zheng et al. 2025; Liu et al. 2026b). At the response level, feedback can revise the current output (Madaan et al. 2023), while agentic systems can retain reflections, skills, or task experience across attempts (Shinn et al. 2023; Wang et al. 2023; Zhao et al. 2024). However, unlike human creators who continuously accumulate knowledge from successful revisions and failures, such systems treat individual human-aligned feedback as transient signals rather than reusable design knowledge. The unresolved question for multimodal design is how to convert multimodal evidence, structural constraints, feedback, and human preferences into persistent design-aligned capabilities of the production system (Ren et al. 2026).

To close this gap, we propose AutoDesign, which frames human-aligned design generation as a meta-harness optimization problem (Robeyns et al. 2025; Lee et al. 2026b; Lee et al. 2026a; Zhang et al. 2026a; Lin et al. 2026; Ren et al. 2026): an agentic system recursively optimizes the design harness itself, rather than an individual artifact, based on an evaluation grounded in human preferences, e.g., annotated reference artifacts or natural language guidance. As illustrated in Figure 3, AutoDesign operates through two nested loops. The inner loop is the design harness, which transforms the source context into an editable output and iteratively revises it under critic feedback. The outer loop is the meta-harness, which optimizes the design harness across tasks. It first grounds human preferences by initializing an evaluator from annotated reference artifacts. Given this human-aligned evaluator, the meta-harness aggregates rollouts and evaluation scores across tasks to identify recurrent failures, and directs a coding agent as the optimizer to propose a bounded update to the current design harness at each iteration. To prevent overfitting, the update is admitted through an acceptance gate, which accepts it only when it improves performance on the training set without degrading performance on the development set (Nguyen et al. 2026).

Accepted updates accumulate to DesignHarness, an executable system that autonomously ingests the source, generates and iteratively refines the artifact under critic feedback, and finalizes it into a readable, visually coherent artifact. We instantiate this system for academic paper-to-poster generation, a challenging design task that must condense long, multimodal scientific sources into a single legible, visually coherent poster while preserving traceable evidence (Jaisankar et al. 2025; Pang et al. 2025; Sun et al. 2026; Vinaykumar et al. 2026). In this task, DesignHarness can produce ready-to-use conference posters that align closely with human preferences while reducing the time and manual effort required for high-quality poster production.

Figure 3: Overview of AutoDesign. The design harness iteratively generates and revises an artifact using critic feedback, thereby constituting the inner loop. The outer loop improves the design harness by running it on design tasks, evaluating its outputs, proposing an update to one component, and accepting or rejecting the candidate update. This process can run autonomously, with optional human guidance.

Evaluating a design harness requires a benchmark and evaluation protocol that jointly measure source fidelity, dense scientific communication, and rendered usability. Existing paper-to-poster benchmarks each cover only part of these aspects, such as layout, extraction, faithfulness, or visual quality, and fall short of a comprehensive, task-level protocol (Wang et al. 2024; Jaisankar et al. 2025; Pang et al. 2025; Sun et al. 2026; Vinaykumar et al. 2026). PosterBench addresses this gap with a 100-paper spanning five disciplines, together with PosterBench-mini, a 10-paper subset for rapid testing. Its seven-dimensional rubric combines rule-based algorithm checks, rubric VLM judgments, and hybrid methods where both signals apply. One main track evaluates the complete system, while three controlled tracks isolate component-level effects. A system-blind human study independently validates the automatic protocol.

Across PosterBench-mini and PosterBench evaluations, the learned DesignHarness improves system quality while remaining practical to deploy. On PosterBench-mini, attaching DesignHarness to each of seven Code Agents raises the average PosterBench Score from 54.99 to 67.39 (+12.40 points). On the PosterBench Main Track, AutoDesign scores 78.32; under the same Claude Code and Claude 4.8 configuration, it outperforms the closed-source commercial system Claude Design by 7.45 points. AutoDesign also makes strong paper-to-poster generation accessible at low cost: with DesignHarness, LongCat-2.0 reaches 55.13 at approximately $0.27 per poster11 1 The reported cost reflects LongCat-2.0’s pricing policy at evaluation time: cached context incurs no charge on a cache hit.. Finally, across 933 valid system-blind pairwise judgments, AutoDesign receives the highest Bradley–Terry preference estimate, 64.0% (95% interval: 55.2–77.8%); for pairs separated by at least 20 PosterBench points, participants prefer the PosterBench-preferred poster in 74.4% of cases.

AutoDesign, a meta-harness optimization framework that turns a static design harness into a recursively improving system for human-aligned multimodal design. Through 7 days of evolving traces, it invokes 224 subagents, records at least 123 recursive iterations, accumulating 54 harness updates that recursively convert human-designed reference artifacts, rollout traces, rendering diagnostics, and evaluator feedback into persistent design priors for a design harness.

DesignHarness, the executable academic paper-to-poster system evolved by AutoDesign. It grounds a tool-using Designer in paper context and combines editable generation with rendering, rule-based validation, and visual critic feedback for localized revision, producing source-grounded posters that remain directly usable and editable.

PosterBench, a comprehensive evaluation protocol for paper-to-poster evaluation. It evaluates scientific communication quality and executable-artifact reliability through a seven-dimensional rubric spanning faithfulness, coverage, density, visual evidence, layout, readability, and aesthetics. On PosterBench, the state-of-the-art coding agent Claude Code (Claude 4.8) achieves 70.01; attaching DesignHarness can improve it by +8.31, also surpassing the best commercial design agent Claude Design by 7.45 points.

Under a fully autonomous long-horizon agentic loop, DesignHarness produces human-level academic posters. In a poster generation run, it executes 253 tool calls and 11 editing turns within 40 minutes for less than $3, with negligible human intervention. Demo is it available as a research-preview platform for interactive use and localized revision of editable posters.

2 Meta-Harness Formulation

2.1 Design Harness Definition

Following recent work that treats harnesses as an optimization target distinct from model weights (Lee et al. 2026b; Ren et al. 2026), we define a design harness H as the system surrounding a fixed model that turns a multimodal source (e.g., an academic paper or report) into a human-facing artifact (e.g., a presentation slide, poster, video, or web page):

y∼H⁡(πθ,x,c), (1)

where πθ is an LLM or MLLM, x is the multimodal input, and c specifies the context including the target medium and user constraints. The harness produces an artifact y through an execution trajectory τ, which records the sequence of intermediate actions, states, and revisions leading to the final output.

To enable systematic meta-harness optimization and facilitate credit assignment, we decompose the design harness H into five functional components:

Context and Memory: source management, prompts, skills, reusable assets and persistent state.

Tools and Specifications: tools and editable artifact specifications for layout, typography, and provenance.

Execution Runtime: the workspace and runtime for authoring, rendering, validating, and exporting artifacts.

Orchestration: task routing, attempt budgets, loop control, candidate selection, fallback, and finalization.

Evaluation and Feedback: rule-based validation, model-based critique, and localized feedback for revision.

This decomposition specifies only the high-level abstraction of the design harness. The concrete implementation of each component is instantiated and iteratively improved by the meta-harness.

2.2 Meta-Harness Definition

We define a meta-harness as a system that operates on the design harness (Lee et al. 2026b; Zhang et al. 2026a; Lin et al. 2026). Given a user specification q and an optional initial design harness H0, the meta-harness instantiates or iteratively improves the concrete implementation of the design harness, yielding an optimized harness HT. Unlike a design harness, which transforms multimodal sources into human-facing artifacts, the meta-harness transforms harness requirements and implementations into an improved design harness.

The optimization target is the expected quality of the artifacts produced by the design harness. Let ptask denote the task distribution. We define the performance of a design harness H as

J⁡(H)=𝔼(x,c)∼ptask,y∼H⁡(πθ,x,c)​[Rmeta​(y,x,c)], (2)

where Rmeta​(y,x,c) denotes the evaluator used by the meta-harness to assess the quality of artifact y with respect to its source x and design context c. Accordingly, the meta-harness optimization objective is

H⋆=arg​max𝐻​J​(H). (3)

Throughout this process, the parameters θ of the underlying model πθ in the harness remain fixed. The optimization therefore acts on the system surrounding the model rather than on the model itself, consistent with the model-versus-scaffold distinction in recent self-improving-agent taxonomies (Ren et al. 2026).

3 Meta-Harness Learning Loop

AutoDesign organizes artifact generation (i.e., design harness) and harness optimization (i.e., meta-harness) as two nested feedback loops, as illustrated in Figure 4. For each design task (x,c), the design harness executes an inner loop that repeatedly generates and revises the current artifact. This loop operates on the artifact y under a fixed design harness H and records the resulting execution trajectory τ. Across tasks, the meta-harness executes an outer loop that analyzes these trajectories and their evaluation results to identify recurrent failures and update the design harness H. Thus, the inner loop improves a single artifact without changing H, whereas the outer loop improves H based on evidence collected from multiple generation runs.

3.1 Inner Loop

媒体内容 · 前往原文查看

Figure 4: The inner and outer loops of the AutoDesign Framework. The inner loop updates the artifact according to the design harness, while the outer loop updates the design harness.

We initialize the design harness with a minimal inner-loop scaffold consisting of two abstract modules: a designer Mdesign and a critic Mcritic. As shown in Figure 4, the designer generates and revises the artifact, while the critic evaluates its current state and provides feedback for the next revision. At refinement step k, their interaction is defined as

yk =Mdesign​(yk−1,fk−1,x,c), (4)

fk =Mcritic​(yk,x,c),

where yk is the artifact at step k and fk is the corresponding feedback, with y0 and f0 left empty so that the first step produces an initial draft from (x,c) alone. Repeated evaluation and revision produce an execution trajectory τ under the current design harness.

This formulation specifies only the roles of the two modules and the information flow between them. It defines the basic structure of the initial design harness while leaving its concrete realization open. During outer-loop optimization, the meta-harness may refine the two modules and their interaction, including their prompts, tools, feedback mechanisms, and loop-control policies, based on observed task performance.

3.2 Outer Loop

The outer loop improves the design harness across tasks, as illustrated in Figure 3. Each iteration proceeds in four stages: rollout, evaluation, update proposal, and acceptance.

Rollout. At outer-loop iteration t, the current design harness Ht is executed on a training task set 𝒟train={(xi,ci)}i=1Ntrain, where xi is a multimodal source and ci specifies the target medium and the corresponding design requirements. Each execution produces an artifact yti and a corresponding trajectory τti. We denote the collection of trajectories by 𝝉t={τti}i=1Ntrain.

Evaluation. Before outer-loop optimization, we provide an evaluator coding agent with reference artifacts annotated by humans along seven quality dimensions: (i) Faithfulness, (ii) Coverage, (iii) Density, (iv) Visual Evidence, (v) Layout, (vi) Readability, and (vii) Aesthetics. The agent uses these examples to implement the evaluator Rmeta, combining rule-based checks for directly measurable properties with VLM-based judgments for perceptual properties such as aesthetics. Once constructed, Rmeta remains fixed during autonomous optimization. The resulting evaluator is then used to assess every generated artifact, sti=Rmeta​(yti,xi,ci), and the resulting scores are collected into a batch denoted by 𝒔t. This optimization-time evaluator is distinct from the frozen PosterBench protocol used for final system comparison; the latter is specified in Section A.4.

Update proposal. We denote the meta-harness optimizer by P. In addition to the current design harness and the evidence collected at the current iteration, the meta-harness maintains an optimization record ℒ, serving as persistent context across outer-loop iterations. At iteration t, the optimizer takes the current harness Ht, the collected trajectories 𝝉t, their evaluation scores 𝒔t, and the optimization record ℒ as input, and produces a candidate updated harness Ht+1′:

Ht+1′=P⁡(Ht,𝝉t,𝒔t,ℒ). (5)

The prime indicates that Ht+1′ is an update proposal, whose acceptance is determined by the subsequent gating stage.

Figure 5: AutoDesign outer loop. Each iteration proceeds through rollout, evaluation, update proposal, and acceptance. The loop runs autonomously, with optional human guidance to redirect the evaluation and optimization.

We instantiate P as a coding agent that sequentially assumes the roles of a planner and a code editor, as illustrated in Figure 3. In the planner role, the agent analyzes the current trajectories and scores together with the optimization history in ℒ. It dispatches parallel subagents to inspect the trajectories and their scores, synthesizes their findings into structured evidence of recurrent failures, and formulates a harness update plan. The plan specifies the observed failure modes, the harness component to be modified, and the intended changes. In the code-editor role, the agent implements these changes in the current design harness Ht, yielding the candidate harness Ht+1′.

Each outer-loop iteration is restricted to exactly one of the five harness components defined in Section 2.1. An update may span multiple files within the selected component, but it cannot modify another component in the same iteration. This restriction keeps credit assignment interpretable, as each gain or regression is attributable to a single coherent intervention rather than to several simultaneous changes.

Acceptance gate. Once an update has been proposed, the meta-harness determines whether it should replace the current harness through a separate acceptance gate. Let Jtrain and Jdev denote the objective in Equation 2 evaluated on the training set 𝒟train and on an independent development set 𝒟dev, respectively. A candidate is accepted only when its performance on 𝒟train improves and its performance on 𝒟dev does not decline:

Accept⁡(Ht+1′)⇔Jtrain​(Ht+1′)>Jtrain​(Ht)∧Jdev​(Ht+1′)≥Jdev​(Ht). (6)

If the condition holds, the meta-harness promotes Ht+1′ to Ht+1; otherwise Ht is retained. Results on the development set are used exclusively by the acceptance gate and are never exposed to P when constructing an update proposal. Therefore, 𝒟dev serves as a guard against overfitting the harness to the training tasks.

After the acceptance decision, the meta-harness appends the completed iteration to the optimization record ℒ. For each iteration t, ℒ stores the harness Ht, the trajectories and scores, the selected harness component, the update plan and the corresponding code changes, and the acceptance decision, with a repository checkpoint preserving the harness implementation at that iteration. Trajectories and scores from the development set are not included in the record.

The updated ℒ is supplied to P as persistent context in the next iteration. When a candidate is rejected, the record allows the next iteration to propose a different update while retaining the evidence of what has already been tried. ℒ thus supports comparison, reproducibility, and rollback across iterations. Note that the outer loop maintains a single active harness at each iteration and does not perform tree search over harness variants. The complete meta-harness optimization procedure is summarized in Algorithm 1.

媒体内容 · 前往原文查看

Algorithm 1 AutoDesign meta-harness optimization

0: Fixed model πθ; initial design harness H0; evaluator Rmeta

0: Training task set 𝒟train; development task set 𝒟dev; outer-loop iterations T

0: Optimized design harness HT and optimization record ℒ

1: Run H0 on 𝒟train; collect execution trajectories 𝝉0 and evaluator scores 𝒔0

2: Run H0 on 𝒟dev and collect evaluator scores 𝒔0dev

3: for t=0 to T−1 do

4: The meta-harness optimizer P inspects 𝝉t, 𝒔t, and ℒ and proposes a candidate updated harness Ht+1′

5: Run Ht+1′ on 𝒟train; collect execution trajectories 𝝉t+1′ and evaluator scores 𝒔t+1′

6: Run Ht+1′ on 𝒟dev and collect evaluator scores 𝒔t+1′dev

7: if Jtrain​(Ht+1′)>Jtrain​(Ht) and Jdev​(Ht+1′)≥Jdev​(Ht) then

8: dt←Accept

9: (Ht+1,𝝉t+1,𝒔t+1,𝒔t+1dev)←(Ht+1′,𝝉t+1′,𝒔t+1′,𝒔t+1′dev)

10: else

11: dt←Reject

12: (Ht+1,𝝉t+1,𝒔t+1,𝒔t+1dev)←(Ht,𝝉t,𝒔t,𝒔tdev)

13: end if

14: Append the harness checkpoint and iteration record to ℒ

15: end for

16: return (HT,ℒ)

Human-in-the-Loop. AutoDesign also supports an optional human-in-the-loop mode, as illustrated in Figure 3. Human intervention can operate through two channels. First, at iteration t, a user may provide directional guidance gt in natural language, which is supplied to the planner alongside the trajectories and evaluation scores, so that the update proposal becomes Ht+1′=P⁡(Ht,𝝉t,𝒔t,ℒ,gt). We introduce this mechanism because the coding agent acting as P may converge prematurely to a locally satisfactory harness configuration, at which point outer-loop optimization stagnates. Guidance can inject task-specific heuristics or redirect the search toward alternative improvements.

Second, human guidance may be provided to the coding agent responsible for implementing the evaluator when visual inspection reveals a systematic artifact bias not captured by Rmeta. Such evaluator revision requires explicit human input. Otherwise, Rmeta remains fixed because the meta-harness receives no external signal with which to identify or correct evaluator bias.

In both cases, the human provides observations or high-level directions rather than directly editing the harness or evaluator implementation. When no guidance is provided, the outer loop operates autonomously as defined in Equation 5.

4 The Optimized DesignHarness

The meta-harness optimization described in Section 3 yields DesignHarness. The resulting implementation supports multiple output media, including academic posters, presentation slides, videos, and web pages. We characterize its resulting architecture by examining the final implementation obtained through meta-harness optimization. As summarized in Figure 6, our analysis identifies four main stages: source ingestion, iterative artifact generation and revision by a designer module, feedback from a rule-based validator and a VLM-based critic, and finalization that prepares the selected candidate for delivery.

Figure 6: Overview of DesignHarness, optimized for human-facing artifact generation. Given source materials, the designer iteratively generates and revises an editable artifact using feedback from dual critics: a rule-based validator and a model-based visual critic. DesignHarness then finalizes the best valid candidate.

4.1 Paper Ingestion

The ingestion stage transforms the input source x and the design context c into a structured, provenance-aware context for subsequent generation and revision. It extracts the document metadata and section outline, identifies key passages supporting the main claims, and records figures and tables together with their source locations, reflecting the joint textual and visual evidence selection required in paper-to-poster generation (Jaisankar et al. 2025; Pang et al. 2025; Sun et al. 2026). These materials are then organized into a content brief and a medium-specific artifact plan, which specify the target output format, the claims to be conveyed, and the visual evidence supporting each of them. Every extracted element retains a reference to its location in x, so that source-derived statements and visual materials used in the artifact can be traced back to the source and checked during revision. The resulting context is constructed once, retained across all inner-loop refinement steps, and provided to the designer as source-grounded input.

4.2 Artifact Generation and Revision

The designer module is implemented as a coding agent that generates or revises the artifact from the ingested source context using the tools and skills available in the harness. At refinement step k, it conditions on the current artifact yk−1, the feedback fk−1, and the ingested context to produce the next candidate yk, thereby instantiating Mdesign in Equation 4. Consistent with layer- and code-based design generation, the artifact remains as editable HTML files throughout refinement (Qu et al. 2025; Liu et al. 2026b), allowing revisions to be implemented as localized code edits without requiring regeneration of the entire output. For visual critique, it can be rendered or exported as a medium-specific preview, such as PNG, PPTX, or MP4.

4.3 Validation and Finalization

At each refinement step k, the candidate artifact yk produced by the designer is examined by the rule-based validator, which applies a set of deterministic blocking checks. At a high level, these checks determine whether the candidate satisfies the requirements for terminating refinement and cover issues such as unsafe or missing assets, broken provenance links between incorporated materials and their sources, severe overflow or overlap, and violations of the required typographic and layout constraints. If the candidate passes all of them, the inner loop terminates and the candidate proceeds directly to finalization. Otherwise, the validator returns localized diagnostics for the detected violations, together with the results of non-blocking checks on properties such as content coverage, information density, and numerical consistency with the source.

When a candidate fails the blocking checks, it is also rendered into a medium-specific preview and inspected by a critic VLM. The critic assesses rendered properties of the candidate, including compliance with the design context, layout, readability, and aesthetics. The two sources of feedback are consolidated into the repair signal fk and passed to the designer for the next attempt. This feedback-to-revision pattern is related to recursive self-refinement and agent-as-a-judge approaches (Madaan et al. 2023; Zhuge et al. 2025). In the notation of Equation 4, rule-based validation and visual critique jointly instantiate Mcritic.

The final implementation permits at most K=12 refinement attempts. As soon as a candidate yk passes all blocking checks, the attempt loop terminates and the candidate is passed to the finalization stage. This stage applies the remaining post-processing, such as final rendering adjustments, mathematical typesetting, and inlining of referenced assets, to produce a self-contained output. If the attempt budget is exhausted without any candidate passing all blocking checks, the harness uses the retained attempt history and applies a sequence of fallback mechanisms to identify a deliverable candidate while retaining essential safety and integrity constraints. The selected candidate is then passed to the same finalization stage. The sequence of candidate generation, validation, critique, and revision constitutes the execution trajectory τ in Section 3.1, while the design harness H remains fixed throughout this process.

5 Evaluation Protocol

5.1 Benchmark: PosterBench

Figure 7: PosterBench evaluation protocol. PosterBench combines rule-based algorithms for traceable spatial, OCR, numeric-grounding, and render-integrity checks with rubric-guided VLM judges for source-grounded assessment of visual evidence, layout, readability, and aesthetics.

PosterBench contains a 100-paper Main Track and PosterBench-mini, a shared 10-paper subset reported in Table 2. The papers span five disciplines: AI/ML, biomedicine and health, climate and earth environment, economics and policy, and physics and astronomy. Every system receives the same source paper and associated source assets, and its output is rendered to a common poster format before scoring. The shared paper-to-poster generation interface and the exact fixed-versus-varied factors for every reported track are documented in Section A.2 and Table 5.

For paper pi and candidate artifact Ai, the fixed PosterBench evaluator returns 𝐪i∈[0,10]7, ordered as Faithfulness, Coverage, Density, Visual Evidence, Layout, Readability, and Aesthetics. It forms the weighted rubric score

Rrubric​(pi,Ai)=∑j=17αj​qi,j/10,𝜶=(10,10,15,10,20,25,10). (7)

The final score applies the strictest active record-level ceiling before benchmark averaging:

Rposter​(pi,Ai) =min(Rrubric(pi,Ai),Cilayout, (8)

OPENCiviability,Cifailure,Cigate),

Overall =1N​∑i=1NRposter​(pi,Ai).

Here, Cilayout, Civiability, Cifailure, and Cigate bound severe layout damage, insufficient presentation viability, confirmed visible failures, and protected render-integrity violations, respectively; inactive ceilings are 100. A standard P0 gate caps a score at 40, and more severe gate types may set a lower cap. Thus, the metric columns are dimension means, while Overall is the mean of capped poster scores and cannot generally be recovered by reweighting those displayed means.

PosterBench is a frozen external evaluator, separate from the optimization-time evaluator Rmeta used in the meta-harness outer loop (Section 3.2). Rmeta supplies feedback for updating DesignHarness; PosterBench evaluates completed systems and is neither optimized nor modified by the outer loop. The complete operational rubric, aggregation, protected gates, and per-case record schema are specified in Section A.4.

Figure 7 shows the evaluator’s evidence path on a rendered poster. Programmatic spatial and OCR audits localize unused regions, text overflow or clipping, and source-inconsistent numeric claims. The source-conditioned VLM then assesses paper-grounded visual quality from the rendered artifact and paper brief. These dimension-level signals are aggregated with fixed weights and record-level ceilings.

PosterBench Main Track. Table 1 reports the 100-paper PosterBench Main Track, comparing design agents, standalone coding agents, and task-specific handcrafted workflow under the fixed PosterBench evaluator. AutoDesign attains the highest PosterBench Score at 78.32. Under the matched Claude Code and Claude 4.8 configuration, AutoDesign scores 78.32, exceeding Claude Design by 7.45 points and OpenDesign by 8.87 points. Curated qualitative comparisons, which are excluded from the aggregate PosterBench results, appear in Figure 16.

媒体内容 · 前往原文查看

Table 1: PosterBench Score and dimension scores for the PosterBench Main Track on the 100-paper evaluation set. Systems are grouped by their primary design mechanism. “—” denotes a layer that a system does not use.

System Design Harness Coding Agent Model Score Faith. Cover. Density Vis. Ev. Layout Read. Aesth.

Design Agent

AutoDesign DesignHarness Claude Code Claude 4.8 78.32 9.35 9.40 8.41 5.97 8.55 8.17 5.59

AutoDesign DesignHarness Codex GPT 5.5 77.97 9.57 9.35 8.63 5.47 7.75 8.08 7.16

Claude Design Claude Design Claude Code Claude 4.8 70.87 9.22 9.90 6.48 7.62 8.08 5.96 7.36

OpenDesign OpenDesign Claude Code Claude 4.8 69.45 9.17 9.37 7.51 6.12 7.08 6.19 7.01

OpenDesign OpenDesign Codex GPT 5.5 62.17 9.24 8.49 6.91 5.05 7.24 5.76 6.00

Coding Agent

Codex — Codex GPT 5.5 73.37 9.64 9.71 7.69 7.43 8.03 6.08 6.60

Claude Code — Claude Code Claude 4.8 70.01 9.25 9.88 5.71 7.01 9.46 6.68 6.53

Doubao — Claude Code Seed 2.1 61.14 8.93 8.96 5.95 5.97 6.51 5.63 4.61

GLM — Claude Code GLM 5.2 52.22 8.60 7.44 5.23 5.29 7.03 4.90 5.15

Kimi — Claude Code Kimi K2.7 51.46 8.33 7.15 4.72 5.62 7.26 4.93 4.81

DeepSeek — Claude Code DeepSeekV4-Pro 46.01 8.23 6.68 4.10 3.82 6.95 4.41 3.38

Human-Crafted Workflow

PosterGen — — Claude 4.8 56.71 8.84 8.25 4.31 5.62 8.61 5.36 5.28

Any2Poster — — Claude 4.8 49.09 8.26 5.44 4.18 3.65 9.59 4.47 3.10

Paper2Poster — — Claude 4.8 44.61 6.35 2.35 8.36 2.16 3.69 4.87 1.69

PosterBench-mini Main Track. Table 2 reports results on PosterBench-mini, the original 10-paper subset used for the controlled ablations.

媒体内容 · 前往原文查看

Table 2: PosterBench Score and dimension scores for the PosterBench-mini Main Track on the original 10-paper subset.

System Design Harness Coding Agent Model Score Faith. Cover. Density Vis. Ev. Layout Read. Aesth.

Design Agent

AutoDesign DesignHarness Codex GPT 5.5 81.46 9.88 9.90 8.64 6.20 8.01 7.96 7.45

AutoDesign DesignHarness Claude Code Claude 4.8 74.56 9.28 8.80 8.15 5.00 9.00 7.86 6.75

OpenDesign OpenDesign Claude Code Claude 4.8 70.36 9.10 9.00 7.39 6.15 7.13 6.38 7.31

Claude Design Claude Design Claude Code Claude 4.8 66.83 8.95 10.00 5.24 7.90 8.54 6.09 6.77

OpenDesign OpenDesign Codex GPT 5.5 60.58 9.03 9.20 6.13 5.45 7.68 5.75 5.75

Coding Agent

Codex — Codex GPT 5.5 75.87 9.47 9.40 7.88 7.55 8.44 6.00 7.00

Claude Code — Claude Code Claude 4.8 69.55 9.19 10.00 5.21 7.50 9.76 6.90 6.40

Kimi — Claude Code Kimi K2.7 57.20 8.20 8.20 4.91 6.40 7.91 5.60 5.95

Doubao — Claude Code Seed 2.1 54.01 7.99 8.20 6.30 5.55 6.01 5.52 6.15

GLM — Claude Code GLM5.2 50.32 8.83 7.80 3.79 3.85 8.21 5.11 5.45

DeepSeek — Claude Code DeepSeekV4-Pro 34.73 8.08 7.90 2.94 3.33 5.98 3.71 5.05

Human-Crafted Workflow

PosterGen — — Claude 4.8 51.82 8.95 9.33 3.39 6.67 7.67 4.96 5.50

Any2Poster — — Claude 4.8 46.88 8.09 4.90 3.69 3.65 9.55 4.40 3.20

Paper2Poster — — Claude 4.8 42.06 6.16 2.10 8.25 2.30 2.76 4.82 1.80

On PosterBench-mini, AutoDesign reaches 81.46 with Codex, compared with 75.87 for the native Codex baseline, and 74.56 with Claude Code, compared with 69.55 for the corresponding standalone baseline.

Design Harness Track. Table 3(a) isolates the design-harness contribution by holding Claude Code and Claude 4.8 fixed. AutoDesign reaches 74.56, while Claude Design and OpenDesign score 66.83 and 70.36, respectively, under the same configuration.

Coding Harness Track. Table 3(b) fixes AutoDesign and GLM 5.2, varying only the coding harness. Kimi Code achieves the highest score at 82.31, followed by ZCode at 69.53.

Model Track. Table 3(c) fixes AutoDesign and Claude Code to separate model choice from harness variation. Claude 4.8 achieves the highest score at 74.56, followed by Seed 2.1 Pro at 71.83 and Kimi K2.7 at 70.12.

媒体内容 · 前往原文查看

Table 3: Controlled track analysis on PosterBench-mini, the fixed 10-paper subset. (a) varies the design harness while holding the coding harness and model fixed. (b) varies the coding harness while holding AutoDesign and GLM 5.2 fixed. (c) varies the model while holding AutoDesign and Claude Code fixed.

Variant Overall↑ Faith. Cover. Density Vis. Ev. Layout Read. Aesth.

(a) Design Harness Track Coding Harness: Claude Code | Model: Claude 4.8

AutoDesign 74.56 9.28 8.80 8.15 5.00 9.00 7.86 6.75

OpenDesign 70.36 9.10 9.00 7.39 6.15 7.13 6.38 7.31

Claude Design 66.83 8.95 10.00 5.24 7.90 8.54 6.09 6.77

(b) Coding Harness Track Design Harness: AutoDesign | Model: GLM 5.2

Kimi Code 82.31 9.29 9.60 7.95 6.75 8.75 8.46 7.68

ZCode 69.53 9.42 8.90 7.04 5.10 7.23 6.93 6.77

OpenCode 67.87 9.32 8.20 7.28 6.90 7.84 7.21 6.47

Claude Code 64.33 8.81 7.30 7.38 4.50 7.74 6.30 5.68

(c) Model Track Design Harness: AutoDesign | Coding Harness: Claude Code

Claude 4.8 74.56 9.28 8.80 8.15 5.00 9.00 7.86 6.75

Seed 2.1 Pro 71.83 9.06 9.00 6.53 6.55 9.16 7.11 6.85

Kimi K2.7 70.12 8.54 7.00 8.30 6.27 7.81 6.82 6.00

GLM 5.2 64.33 8.81 7.30 7.38 4.50 7.74 6.30 5.68

LongCat 2.0 55.13 9.11 8.10 5.24 5.25 7.56 5.81 5.65

DeepSeek V4 Pro 54.29 9.47 9.60 4.32 5.50 8.38 5.62 6.00

5.2 Ablation Studies

5.2.1 Effect of the Design Harness

To isolate the design harness contribution, we hold the model and coding agent fixed and compare each configuration before and after harness attachment in Table 4. Across the seven completed configurations, the harness improves the PosterBench Score by 5.01–19.56 points. Under the native Codex–GPT-5.5 configuration, it raises the score from 75.87 to 81.46 (+5.59); with Claude Code and Kimi K2.7, it rises from 57.20 to 70.12 (+12.92). The largest gain is 19.56 points for DeepSeek V4 Pro with Claude Code. These improvements span multiple model–code-agent pairs. MLLMs have an additional repair signal unavailable to text-only LLMs: at each attempt, the rendered preview from the preceding attempt is supplied as visual context for the next repair. This lets the model inspect the artifact it is editing and localize layout, clipping, or visual-evidence failures that are not fully captured by textual diagnostics alone.

媒体内容 · 前往原文查看

Configuration Original AutoDesign Gain

GPT-5.5 Codex 75.87 81.46

Claude 4.8 Claude Code 69.55 74.56

Seed 2.1 Pro Claude Code 54.01 71.83

Kimi K2.7 Code Claude Code 57.20 70.12

GLM 5.2 Claude Code 50.32 64.33

LongCat 2.0 Claude Code 43.26 55.13

DeepSeek V4 Pro Claude Code 34.73 54.29

Table 4: Effect of attaching DesignHarness while keeping the model and coding agent fixed. Exact PosterBench Scores for each completed configuration are ordered by final performance.

5.2.2 Cost–Performance Trade-off

Figure 8 compares PosterBench Score with a normalized designer-only API cost proxy across the seven AutoDesign model configurations on PosterBench-mini, the fixed 10-paper subset. The observed Pareto frontier runs from LongCat-2.022 2 The reported cost reflects LongCat-2.0’s pricing policy at evaluation time: cached context incurs no charge on a cache hit. (55.13 at $0.27 per poster), through Doubao Seed 2.1 Pro (71.83 at $2.75) and Claude 4.8 (74.56 at $7.63), to GPT-5.5 (81.46 at $10.02). Doubao reaches 88% of the GPT-5.5 score at 27% of its cost, while GPT-5.5 provides the highest absolute performance.

媒体内容 · 前往原文查看

Figure 8: Cost–performance trade-off across seven AutoDesign model configurations on PosterBench-mini, the fixed 10-paper subset; marker size denotes median runtime, and the dashed line denotes the empirical Pareto frontier.

5.3 Human Evaluation

We recruited 11 volunteer reviewers for a fully system-blind pairwise evaluation of AutoDesign, Claude Code, OpenDesign, and Claude Design on the 100 source papers of the PosterBench Main Track. For each comparison, reviewers saw only two posters generated from the same paper; no method, system, or model identity was disclosed. They selected the left poster, the right poster, approximate equality, or skip. We fit a Bradley–Terry model (Bradley and Terry 1952),

Pr⁡(i≻j)=exp⁡(βi)exp⁡(βi)+exp⁡(βj),

where βi is the latent preference strength of system i. Ties contribute one half-win to each system and skips are excluded. We report the probability of beating a uniformly sampled alternative, with 95% intervals from 2,000 crossed bootstrap resamples of papers and reviewers. The complete task roster, decision schema, and blind-review interface are documented in Section A.5 and Figure 15.

As shown in Figure 9, the volunteer reviewers submitted 936 responses: 933 ranking judgments and three skips. AutoDesign has the highest Bradley–Terry point estimate at 64.0% (95% interval: 55.2–77.8%). Its tie-adjusted empirical preference scores are 61.3% against Claude Code, 63.1% against OpenDesign, and 67.6% against Claude Design.

媒体内容 · 前往原文查看

Figure 9: System-blind human evaluation. (a) Bradley–Terry probability of beating a uniformly sampled alternative, with 95% intervals crossed by paper–reviewer bootstrap replicates. (b) AutoDesign head-to-head outcomes against each baseline. Eleven volunteer reviewers submitted 936 responses, including 933 ranking judgments and three skips.

Benchmark–human alignment. The blind study also assesses whether PosterBench provides an informative comparison signal at the poster level. Figure 10(a) compares each poster’s PosterBench Score with its tie-adjusted human preference. Each human score aggregates the blind pairwise judgments involving that poster. The two measurements have a positive, albeit imperfect, association (r=0.34; a paper-cluster bootstrap gives a 95% interval of [0.22,0.44]). This is a useful property of the protocol rather than a requirement that it duplicate human preference: PosterBench also evaluates Faithfulness, Coverage, Density, Visual Evidence, Layout, Readability, and Aesthetics, whereas each blind judgment asks for an immediate pairwise choice.

More importantly, the PosterBench Score margin calibrates how informative an automatic comparison is. Figure 10(b) uses 919 of the 933 non-skip judgments: the remaining 14 assign equal PosterBench Scores to both posters and therefore have no benchmark-preferred direction. The probability that the benchmark-preferred poster agrees with the human decision rises from 51.9% for 0–3-point gaps to 74.4% for gaps of at least 20 points. The error bars are 95% intervals from 2,000 crossed paper–reviewer bootstrap replicates, which resample both source papers and reviewers. Thus, the benchmark offers more than a global system ranking: a large score gap identifies comparisons in which human preference is substantially more consistent.

媒体内容 · 前往原文查看

Figure 10: PosterBench Score alignment with system-blind human preference. (a) Poster-level score association. (b) Human agreement increases with the PosterBench Score margin.

5.4 Qualitative Analysis of Designer’s Trajectory

Figure 11 traces one poster run across five attempts. The critic first identifies a clipped analysis lane at A1 (0.36); the reallocation of the row removes the constraint at A3 (0.42), and later the refit of the header and the scaling of evidence produce a more balanced hierarchy at A5/A6 (0.62). A9 preserves the repaired composition and is accepted at 0.78. The trace illustrates the intended use of diagnostic feedback: edits stay localized to the failing region while valid layout and source-derived content are retained across revisions.

Figure 11: Qualitative trajectory of a poster generation run. Five selected attempts show the local change associated with each diagnostic or repair event. Colored boxes mark the implicated region: an analysis overflow at A1, its fit restoration at A3, header refitting at A5, evidence scaling at A6, and the critic-accepted final at A9.

6 Future Directions

AutoDesign is presently validated for academic paper-to-poster generation, but the underlying agentic design pattern is not tied to a single input or output medium. The pilot artifacts in Figure 13 show that the current DesignHarness can also produce paper-to-slide, paper-to-webpage, and paper-to-conference-video outputs. Figure 12 illustrates the long-term direction: a multimodal-in, multimodal-out agentic design system that integrates papers, visual evidence, code, data, and human guidance, then iteratively creates the medium-appropriate output for a target communication setting. Making this expansion reliable requires more than exposing new output formats. Each medium needs source–output data, an evaluator, a rendering and validation gate, and an objective tailored to its communication setting. PosterBench formally evaluates academic posters only; the slide, webpage, and video artifacts therefore remain pilots. Shared context construction, preference memories, and repair histories could nevertheless provide a substrate for reusing experience across media, provided that the transfer is evaluated against medium-specific objectives. At meta-harness level, better component selection and evaluator evolution remain open problems. A selector should choose next bounded update from failure attribution, uncertainty, expected improvement, and component interactions. Any adaptive evaluator must remain versioned and anchored by frozen reference tasks, adversarial probes, and periodic human audits so optimization doesn’t reward-hack a moving target.

Figure 12: Future direction: a multimodal-in, multimodal-out agentic design system that integrates diverse sources and human guidance to iteratively create medium-specific outputs.

Figure 13: Future directions for AutoDesign. The initially optimized DesignHarness already extends paper-to-poster generation to slide decks, webpages, and conference videos. Applying the same meta-harness optimization methodology to these and other multimodal outputs offers a path toward a general multimodal-in & out agentic design system.

Finally, recent work has begun to connect continual harness adaptation, self-improving harnesses, and model–harness co-evolution (Karten et al. 2026; Zhang et al. 2026a; Lee et al. 2026a). In this direction, harness optimization can complement model post-training: long-horizon trajectories and repair outcomes provide execution-time supervision, whereas the model supplies the reasoning and coding capabilities. Joint training should preserve this division while evaluating both layers against shared held-out objectives.

7 Related Work

Systems for multimodal output generation transform heterogeneous sources into audience- and medium-specific outputs. For academic posters, SciPostLayout and deep submodular extraction study layout data, source coverage, and text–image alignment, while Paper2Poster, P2P, PosterGen, PosterForest, and Any2Poster combine multimodal inputs with generation, specialized agents, and visual refinement (Wang et al. 2024; Jaisankar et al. 2025; Pang et al. 2025; Sun et al. 2026; Zhang et al. 2025b; Choi et al. 2026; Vinaykumar et al. 2026). Closely related systems generate slides, webpages, and narrated videos from papers or documents (Fu et al. 2022; Zheng et al. 2025; Ge et al. 2025; Yang et al. 2025; Chen et al. 2025; Zhu et al. 2025). Structured representations such as HTML/CSS and explicit layers further support editability, rendering, and visual inspection (Qu et al. 2025; Liu et al. 2026b; Si et al. 2025; Wu et al. 2024). Together, these works establish the input, output, and representation choices for multimodal output generation, but their run-time feedback generally remains within a fixed production procedure.

Critics, render diagnostics, and regeneration policies can improve the current multimodal output without changing the system that generated it. Self-Refine is the canonical response-level instance: feedback is used to revise the current answer (Madaan et al. 2023). Other agent methods retain experience more persistently: Reflexion stores verbal reflections, Voyager accumulates executable skills, and ExpeL extracts reusable experience from solved tasks (Shinn et al. 2023; Wang et al. 2023; Zhao et al. 2024). These mechanisms preserve useful information beyond a single attempt, but they typically do not update the harness that repeatedly produces outputs. This system-level perspective has early roots in classical accounts of autonomous agents, self-referential learning, and retaining policy changes according to long-term reward effects (Wooldridge and Jennings 1995; Schmidhuber 1987; Schmidhuber et al. 1997). These works are conceptual precedents rather than direct algorithms for AutoDesign. Recent work separates improvement of model parameters from persistent improvement of the operational scaffold around a fixed model (Ren et al. 2026). TextGrad, DSPy, and GEPA optimize components or declarative pipelines, whereas STOP, GPTSwarm, ADAS, and AFlow search code- or graph-represented workflows (Yuksekgonul et al. 2024; Khattab et al. 2024; Agrawal et al. 2026; Zelikman et al. 2024; Zhuge et al. 2024; Hu et al. 2025; Zhang et al. 2025a). At the full-harness level, A Self-Improving Coding Agent and MOSS update agent source from execution evidence; Meta-Harness, HarnessX, Self-Harness, and Agentic Harness Engineering study searchable harness programs, composable primitives, bounded updates, and outcome attribution (Robeyns et al. 2025; Cai et al. 2026; Lee et al. 2026b; Chen et al. 2026; Zhang et al. 2026a; Lin et al. 2026). HarnessX also uses execution traces as signals for both harness evolution and future model training (Chen et al. 2026). Recursive Harness Self-Improvement specializes a user-constructed, prompt-level multi-agent harness from pairwise revision feedback (Lee et al. 2026a). Gödel Machines provide a proof-based ideal for self-rewriting, while Darwin Gödel Machine and Huxley-Gödel Machine study empirical harness evolution and distinguish immediate performance from future self-improvement potential (Schmidhuber 2006; Zhang et al. 2026b; Wang et al. 2025). AutoDesign instantiates this direction for academic design by evolving a DesignHarness for source-grounded, editable artifacts.

Persistent system updates also differ in how they separate learning signals from final evaluation. RHI (Lee et al. 2026a) keeps its evaluation prompt on the evaluator side of the update loop, yet the resulting pairwise history remains a task-local learning signal. Recursive Self-Evolving Agents use an independent development split to gate persistent updates, while Continual Harness, Adaptive Auto-Harness, and Live-SWE-agent study related adaptation settings (Nguyen et al. 2026; Karten et al. 2026; Liu et al. 2026a; Xia et al. 2025). Agent-as-a-Judge further highlights the value of process-level evidence alongside final-outcome assessment (Zhuge et al. 2025). AutoDesign uses an independent development acceptance gate for harness updates and evaluates the final DesignHarness with PosterBench and a system-blind human study.

8 Conclusion

AutoDesign turns recurring design failures into improvements to the system that generates future multimodal outputs. Its MetaHarnessOptimizer aggregates trajectories, source and rendering diagnostics, evaluator feedback, and reference posters to update one DesignHarness component at a time while keeping model weights fixed. This makes paper-to-poster generation a persistent learning process that accumulates design priors and produces editable outputs for direct use or local revision on the Demo Page. We also introduce PosterBench, a unified evaluation protocol for academic posters. DesignHarness achieves the top score (78.32) and, with Claude Code and Claude 4.8 fixed, surpasses Claude Design by 7.45 points. It also receives the highest Bradley–Terry estimate in a system-blind human study.

References

Agrawal et al. (2026) Lakshya A. Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl-Ong, Arnav Singhvi, Herumb Shandilya, Michael J. Ryan, Meng Jiang, Christopher Potts, Koushik Sen, Alexandros G. Dimakis, Ion Stoica, Dan Klein, Matei Zaharia, and Omar Khattab. GEPA: Reflective prompt evolution can outperform reinforcement learning. In The Fourteenth International Conference on Learning Representations, 2026. https://openreview.net/forum?id=RQm2KQTM5r.

Bradley and Terry (1952) Ralph Allan Bradley and Milton E. Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3/4):324–345, 1952. 10.1093/biomet/39.3-4.324.

Cai et al. (2026) Qianshu Cai, Yonggang Zhang, Xianzhang Jia, Wei Xue, Jun Song, Xinmei Tian, and Yike Guo. MOSS: Self-evolution through source-level rewriting in autonomous agent systems. arXiv preprint arXiv:2605.22794, 2026. https://arxiv.org/abs/2605.22794.

Chameleon Team (2024) Chameleon Team. Chameleon: Mixed-modal early-fusion foundation models. arXiv preprint arXiv:2405.09818, 2024. https://arxiv.org/abs/2405.09818.

Chen et al. (2026) Tingyang Chen, Shuo Lu, Kang Zhao, Weicheng Meng, Hanlin Teng, Tianhao Li, Chao Li, Xule Liu, Jian Liang, Zhizhong Zhang, Yuan Xie, Heng Qu, Kun Shao, and Jian Luan. HarnessX: A composable, adaptive, and evolvable agent harness foundry. arXiv preprint arXiv:2606.14249, 2026. https://arxiv.org/abs/2606.14249.

Chen et al. (2025) Yuhang Chen, Tianpeng Lv, Siyi Zhang, Yixiang Yin, Yao Wan, Philip S. Yu, and Dongping Chen. Paper2Web: Let’s make your paper alive! arXiv preprint arXiv:2510.15842, 2025. https://arxiv.org/abs/2510.15842.

Choi et al. (2026) Jiho Choi, Seojeong Park, Seongjong Song, and Hyunjung Shim. PosterForest: Hierarchical multi-agent collaboration for scientific poster generation. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 379–401. Association for Computational Linguistics, 2026. 10.18653/v1/2026.acl-long.15. https://aclanthology.org/2026.acl-long.15/.

Fu et al. (2022) Tsu-Jui Fu, William Yang Wang, Daniel McDuff, and Yale Song. DOC2PPT: Automatic presentation slides generation from scientific documents. In AAAI, 2022. https://arxiv.org/abs/2101.11796.

Ge et al. (2025) Jiaxin Ge, Zora Zhiruo Wang, Xuhui Zhou, Yi-Hao Peng, Sanjay Subramanian, Qinyue Tan, Maarten Sap, Alane Suhr, Daniel Fried, Graham Neubig, and Trevor Darrell. AutoPresent: Designing structured visuals from scratch. arXiv preprint arXiv:2501.00912, 2025. https://arxiv.org/abs/2501.00912.

Hu et al. (2025) Shengran Hu, Cong Lu, and Jeff Clune. Automated design of agentic systems. In The Thirteenth International Conference on Learning Representations, 2025. https://openreview.net/forum?id=t9U3LW7JVX.

Jaisankar et al. (2025) Vijay Jaisankar, Sambaran Bandyopadhyay, Kalp Vyas, Varre Suman Chaitanya, and Shwetha Somasundaram. Deep submodular optimization and LLM for multimodal content extraction and automatic poster generation from long document. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 24221–24229, 2025. 10.1609/aaai.v39i23.34598. https://ojs.aaai.org/index.php/AAAI/article/view/34598.

Karten et al. (2026) Seth Karten, Joel Zhang, Tersoo Upaa, Ruirong Feng, Wenzhe Li, Chengshuai Shi, Chi Jin, and Kiran Vodrahalli. Continual harness: Online adaptation for self-improving foundation agents. arXiv preprint arXiv:2605.09998, 2026. https://arxiv.org/abs/2605.09998.

Khattab et al. (2024) Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts. DSPy: Compiling declarative language model calls into state-of-the-art pipelines. In The Twelfth International Conference on Learning Representations, 2024. https://openreview.net/forum?id=sY5N0zY5Od.

Kim et al. (2026) Jaeik Kim, Woojin Kim, Jihwan Hong, Yejoon Lee, Sieun Hyeon, Mintaek Lim, Yunseok Han, Dogeun Kim, Hoeun Lee, Hyunggeun Kim, and Jaeyoung Do. Dynin-Omni: Omnimodal unified large diffusion language model. arXiv preprint arXiv:2604.00007, 2026. https://arxiv.org/abs/2604.00007.

Lee et al. (2026a) Hyunin Lee, Jinglue Xu, Jeffrey Seely, Donghyun Lee, Matei Zaharia, and Yujin Tang. Recursive harness self-improvement. arXiv preprint arXiv:2607.15524, 2026a. https://arxiv.org/abs/2607.15524.

Lee et al. (2026b) Yoonho Lee, Roshen Nair, Qizheng Zhang, Kangwook Lee, Omar Khattab, and Chelsea Finn. Meta-Harness: End-to-end optimization of model harnesses. arXiv preprint arXiv:2603.28052, 2026b.

Lin et al. (2026) Jiahang Lin, Shichun Liu, Chengjun Pan, Lizhi Lin, Shihan Dou, Zhiheng Xi, Xuanjing Huang, Hang Yan, Zhenhua Han, Tao Gui, and Yu-Gang Jiang. Agentic harness engineering: Observability-driven automatic evolution of coding-agent harnesses. arXiv preprint arXiv:2604.25850, 2026. https://arxiv.org/abs/2604.25850.

Liu et al. (2026a) Zewen Liu, Zhan Shi, Yisi Sang, Bing He, Minhua Lin, Tianxin Wei, Dakuo Wang, Benoit Dumoulin, Wei Jin, and Hanqing Lu. Adaptive auto-harness: Sustained self-improvement for agentic system deployment on open-ended task streams. arXiv preprint arXiv:2606.01770, 2026a. https://arxiv.org/abs/2606.01770.

Liu et al. (2026b) Ziyuan Liu, Shizhao Sun, Danqing Huang, Yingdong Shi, Meisheng Zhang, Ji Li, Jingsong Yu, and Jiang Bian. DesignAsCode: Bridging structural editability and visual fidelity in graphic design generation. arXiv preprint arXiv:2602.17690, 2026b. https://arxiv.org/abs/2602.17690.

Madaan et al. (2023) Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self-feedback. Advances in neural information processing systems, 36:46534–46594, 2023.

Meituan LongCat Team et al. (2026) Meituan LongCat Team, Bin Xiao, Chao Wang, Chengjiang Li, Chi Zhang, Chong Peng, Hang Yu, Hao Yang, Haonan Yan, Haoze Sun, et al. LongCat-Next: Lexicalizing modalities as discrete tokens. arXiv preprint arXiv:2603.27538, 2026. https://arxiv.org/abs/2603.27538.

Nguyen et al. (2026) Michael Nguyen, Quoc Nguyen, and Paul Vuong. Recursive self-evolving agents via held-out selection. arXiv preprint arXiv:2606.28374, 2026. https://arxiv.org/abs/2606.28374.

Pang et al. (2025) Wei Pang, Kevin Qinghong Lin, Xiangru Jian, Xi He, and Philip H. S. Torr. Paper2Poster: Towards multimodal poster automation from scientific papers. In Advances in Neural Information Processing Systems, Datasets and Benchmarks Track, volume 38, 2025. https://proceedings.neurips.cc/paper_files/paper/2025/hash/17337b1d5eeac8b59c80e025a552fa7a-Abstract-Datasets_and_Benchmarks_Track.html.

Qu et al. (2025) Yadong Qu, Shancheng Fang, Yuxin Wang, Xiaorui Wang, Zhineng Chen, Hongtao Xie, and Yongdong Zhang. IGD: Instructional graphic design with multimodal layer generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 18218–18228, 2025. https://openaccess.thecvf.com/content/ICCV2025/html/Qu_IGD_Instructional_Graphic_Design_with_Multimodal_Layer_Generation_ICCV_2025_paper.html.

Ren et al. (2026) Zhe Ren, Yimeng Chen, Dandan Guo, Guowei Rong, Tonghui Li, R. B. Xiong, Qingfeng Lan, Wenyi Wang, Nanbo Li, Yibo Yang, Mingchen Zhuge, and Jürgen Schmidhuber. Self-improvements in modern agentic systems: A survey. arXiv preprint arXiv:2607.13104, 2026. https://arxiv.org/abs/2607.13104.

Robeyns et al. (2025) Maxime Robeyns, Martin Szummer, and Laurence Aitchison. A self-improving coding agent. arXiv preprint arXiv:2504.15228, 2025. https://arxiv.org/abs/2504.15228.

Schmidhuber (1987) Jürgen Schmidhuber. Evolutionary principles in self-referential learning, or on learning how to learn: The meta-meta-meta… hook. Diploma thesis, Technische Universität München, 1987.

Schmidhuber (2006) Jürgen Schmidhuber. Goedel machines: Self-referential universal problem solvers making provably optimal self-improvements. arXiv preprint cs/0309048, 2006. https://arxiv.org/abs/cs/0309048.

Schmidhuber et al. (1997) Jürgen Schmidhuber, Jieyu Zhao, and Marco A. Wiering. Shifting inductive bias with success-story algorithm, adaptive levin search, and incremental self-improvement. Machine Learning, 28(1):105–130, 1997. 10.1023/A:1007383707642.

Shinn et al. (2023) Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. Advances in neural information processing systems, 36:8634–8652, 2023.

Si et al. (2025) Chenglei Si, Yanzhe Zhang, Ryan Li, Zhengyuan Yang, Ruibo Liu, and Diyi Yang. Design2Code: Benchmarking multimodal code generation for automated front-end engineering. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 3956–3974. Association for Computational Linguistics, 2025. 10.18653/v1/2025.naacl-long.199. https://aclanthology.org/2025.naacl-long.199/.

Sun et al. (2026) Tao Sun, Enhao Pan, Zhengkai Yang, Kaixin Sui, Jiajun Shi, Xianfu Cheng, Tongliang Li, Wenhao Huang, Ge Zhang, Jian Yang, and Zhoujun Li. P2P: Automated paper-to-poster generation and fine-grained benchmark. In International Conference on Learning Representations, 2026. https://iclr.cc/virtual/2026/poster/10010167.

Vinaykumar et al. (2026) Amogh Vinaykumar, Aiden Li, Suozhi Huang, and Shilong Liu. Any2Poster: Any-source poster generation across modalities and domains. arXiv preprint arXiv:2606.02915, 2026. https://arxiv.org/abs/2606.02915.

Wang et al. (2023) Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291, 2023.

Wang et al. (2024) Hao Wang, Shohei Tanaka, and Yoshitaka Ushiku. SciPostLayout: A dataset for layout analysis and layout generation of scientific posters. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 8136–8141, 2024. https://openaccess.thecvf.com/content/CVPR2024W/GDUG/html/Wang_SciPostLayout_A_Dataset_for_Layout_Analysis_and_Layout_Generation_of_CVPRW_2024_paper.html.

Wang et al. (2025) Wenyi Wang, Piotr Piękos, Nanbo Li, Firas Laakom, Yimeng Chen, Mateusz Ostaszewski, Mingchen Zhuge, and Jürgen Schmidhuber. Huxley-gödel machine: Human-level coding agent development by an approximation of the optimal self-improving machine. arXiv preprint arXiv:2510.21614, 2025. https://arxiv.org/abs/2510.21614.

Wooldridge and Jennings (1995) Michael J. Wooldridge and Nicholas R. Jennings. Intelligent agents: Theory and practice. The Knowledge Engineering Review, 10(2):115–152, 1995. 10.1017/S0269888900008122.

Wu et al. (2024) Jason Wu, Eldon Schoop, Alan Leung, Titus Barik, Jeffrey P. Bigham, and Jeffrey Nichols. UICoder: Finetuning large language models to generate user interface code through automated feedback. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 7511–7525. Association for Computational Linguistics, 2024. 10.18653/v1/2024.naacl-long.417. https://aclanthology.org/2024.naacl-long.417/.

Wu et al. (2023) Shengqiong Wu, Hao Fei, Leigang Qu, Wei Ji, and Tat-Seng Chua. NExT-GPT: Any-to-any multimodal LLM. arXiv preprint arXiv:2309.05519, 2023. https://arxiv.org/abs/2309.05519.

Xia et al. (2025) Chunqiu Steven Xia, Zhe Wang, Yan Yang, Yuxiang Wei, and Lingming Zhang. Live-SWE-agent: Can software engineering agents self-evolve on the fly? arXiv preprint arXiv:2511.13646, 2025. https://arxiv.org/abs/2511.13646.

Yang et al. (2025) Yuheng Yang, Wenjia Jiang, Yang Wang, Yi Song, Yiwei Wang, and Chi Zhang. Auto-Slides: An interactive multi-agent system for creating and customizing research presentations. arXiv preprint arXiv:2509.11062, 2025. https://arxiv.org/abs/2509.11062.

Yuksekgonul et al. (2024) Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Zhi Huang, Carlos Guestrin, and James Zou. TextGrad: Automatic “differentiation” via text. arXiv preprint arXiv:2406.07496, 2024. https://arxiv.org/abs/2406.07496.

Zelikman et al. (2024) Eric Zelikman, Eliana Lorch, Lester Mackey, and Adam Tauman Kalai. Self-taught optimizer (STOP): Recursively self-improving code generation. In First Conference on Language Modeling, 2024. https://arxiv.org/abs/2310.02304.

Zhang et al. (2026a) Hangfan Zhang, Shao Zhang, Kangcong Li, Chen Zhang, Yang Chen, Yiqun Zhang, Lei Bai, and Shuyue Hu. Self-Harness: Harnesses that improve themselves. arXiv preprint arXiv:2606.09498, 2026a.

Zhang et al. (2026b) Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange, and Jeff Clune. Darwin Gödel machine: Open-ended evolution of self-improving agents. In The Fourteenth International Conference on Learning Representations, 2026b. https://arxiv.org/abs/2505.22954.

Zhang et al. (2025a) Jiayi Zhang, Jinyu Xiang, Zhaoyang Yu, Fengwei Teng, Xionghui Chen, Jiaqi Chen, Mingchen Zhuge, Xin Cheng, Sirui Hong, Jinlin Wang, Bingnan Zheng, Bang Liu, Yuyu Luo, and Chenglin Wu. AFlow: Automating agentic workflow generation. In The Thirteenth International Conference on Learning Representations, 2025a. https://openreview.net/forum?id=z5uVAKwmjf.

Zhang et al. (2025b) Zhilin Zhang, Xiang Zhang, Jiaqi Wei, Yiwei Xu, and Chenyu You. PosterGen: Aesthetic-aware multi-modal paper-to-poster generation via multi-agent LLMs. arXiv preprint arXiv:2508.17188, 2025b. https://arxiv.org/abs/2508.17188.

Zhao et al. (2024) Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. ExpeL: LLM agents are experiential learners. In Proceedings of the AAAI Conference on Artificial Intelligence, 2024. https://arxiv.org/abs/2308.10144.

Zheng et al. (2025) Hao Zheng, Xinyan Guan, Hao Kong, Wenkai Zhang, Jia Zheng, Weixiang Zhou, Hongyu Lin, Yaojie Lu, Xianpei Han, and Le Sun. PPTAgent: Generating and evaluating presentations beyond text-to-slides. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 14402–14418. Association for Computational Linguistics, 2025. 10.18653/v1/2025.emnlp-main.728. https://aclanthology.org/2025.emnlp-main.728/.

Zhu et al. (2025) Zeyu Zhu, Kevin Qinghong Lin, and Mike Zheng Shou. Paper2Video: Automatic video generation from scientific papers. arXiv preprint arXiv:2510.05096, 2025. https://arxiv.org/abs/2510.05096.

Zhuge et al. (2024) Mingchen Zhuge, Wenyi Wang, Louis Kirsch, Francesco Faccio, Dmitrii Khizbullin, and Jürgen Schmidhuber. GPTswarm: Language agents as optimizable graphs. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 62743–62767, 2024. https://proceedings.mlr.press/v235/zhuge24a.html.

Zhuge et al. (2025) Mingchen Zhuge, Changsheng Zhao, Dylan R. Ashley, Wenyi Wang, Dmitrii Khizbullin, Yunyang Xiong, Zechun Liu, Ernie Chang, Raghuraman Krishnamoorthi, Yuandong Tian, Yangyang Shi, Vikas Chandra, and Jürgen Schmidhuber. Agent-as-a-judge: Evaluate agents with agents. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 80569–80611, 2025. https://proceedings.mlr.press/v267/zhuge25a.html.

Appendix A Supplementary Experimental Materials

This appendix makes the experimental interface inspectable: it documents the controlled comparisons, shared generation instruction, frozen scoring protocol, released records, and matched visual evidence that complement the main paper.

A.1 Benchmark Inputs and Comparison Matrix

Each case provides the source PDF and available paper assets. All systems in a matched row receive the same package and produce one editable poster artifact; the source paper remains the authority for claims, numbers, and visual evidence. Table 5 identifies the controlled factor for every reported track.

Track Papers Fixed across a row comparison Varied factor

PosterBench Main Track 100 Source paper, source assets, output contract, and frozen PosterBench protocol System configuration

PosterBench-mini Main Track 10 Shared PosterBench-mini papers, output contract, and frozen PosterBench protocol System configuration

Design Harness Track 10 Claude Code and Claude 4.8; shared PosterBench-mini papers and frozen PosterBench protocol Design harness

Coding Harness Track 10 AutoDesign, GLM 5.2, shared PosterBench-mini papers, and frozen PosterBench protocol Coding harness

Model Track 10 AutoDesign, Claude Code, shared PosterBench-mini papers, and frozen PosterBench protocol Model

Harness-Attachment Ablation 10 Model, code agent, shared PosterBench-mini papers, and frozen PosterBench protocol Presence of the AutoDesign design harness

媒体内容 · 前往原文查看

Table 5: Controlled comparison matrix. Main tracks compare complete systems; controlled tracks vary only the factor named in the final column.

Runtime Versions and Reasoning Configuration.

Under our controlled configurations, we used codex-cli v0.142.3 for Codex and Claude Code v2.1.119 for Claude Code. Each model used its highest available thinking-effort setting.

A.2 AutoDesign Generation Interface

The AutoDesign system excerpt is shown first. The shared user prompt below fixes the target artifact, source-grounding requirements, and visible quality constraints for every compared system. System-specific orchestration, tools, and repair policies remain part of each system configuration.

A.3 DesignHarness Evolution

Figure 14 provides an implementation-level architecture summary for the paper-to-poster instantiation of DesignHarness. It is neither a second taxonomy nor a record of individual outer-loop iterations: its stages collectively instantiate the five functional components of H in Section 2.1. The illustration moves from a planning-and-critique loop to a coding-agent designer that authors editable HTML, while source grounding, specialist prompt and vision support, repair and validation gates, image-native evaluation, candidate promotion, and recovery operations accumulate around that core.

媒体内容 · 前往原文查看

Figure 14: DesignHarness architecture evolution for paper-to-poster. An implementation-level summary of accumulated capabilities, not a second harness taxonomy or a chronological outer-loop trace. The diagonal path highlights additions in this architectural view; it does not denote individual meta-harness iterations. The final harness combines source grounding, specialist support, quality gates, image-native evaluation, controlled promotion, and operations for editable HTML artifacts.

The figure shows why the optimized result is more than a static prompt or a single repair loop: it accumulates a coherent system around the fixed model. In the five-component abstraction, grounding supplies context and memory; specialist support and editable HTML define tools and specifications; the workspace, browser, renderer, and export environment support the coding-agent authoring path as the execution runtime; direct control and operations implement orchestration; and quality gates together with image-native evaluation provide feedback for revision. The image-native evaluator shown here belongs to evaluation and feedback inside the design harness. It is distinct from the outer-loop evaluator Rmeta and from the frozen PosterBench protocol used for final comparison (Sections 3.2 and A.4).

A.4 PosterBench Evaluation Interface

PosterBench evaluates a rendered artifact against its source paper with seven scores qj∈[0,10]. Its rubric, weights, and protected gates are manually specified and frozen before comparative evaluation. The seven dimensions use the same quality vocabulary as the outer-loop evaluator Rmeta in Section 3.2, but serve a different role: PosterBench reports completed-system quality, whereas an evaluator coding agent constructs Rmeta from annotated reference artifacts to provide update feedback during harness optimization. Rmeta is then fixed within each optimization run.

Dimension Wt. Score mode Operational definition

Faithfulness 10 Programmatic + VLM Checks numeric and source grounding, then judges whether claims, entities, and visual evidence remain consistent with the paper.

Coverage 10 VLM Assesses whether the poster preserves the paper’s problem, method, evidence, and takeaway against a compact source brief.

Density 15 Programmatic Measures information occupancy, OCR text coverage, blank interiors, and pasted paper-body screenshots.

Visual Evidence 10 Programmatic + VLM Judges whether figures and tables are relevant, readable, and explained locally; guards reject raw paper-body crops.

Layout 20 Programmatic Audits render size and aspect, OCR fallback, clipping, overlap, export-edge damage, and visible placeholders.

Readability 25 Programmatic + VLM Combines poster-scale text and spatial checks with hierarchy, scan-path, balance, and crowding judgments.

Aesthetics 10 VLM Rates academic visual craft, including typography, palette discipline, and compositional coherence.

媒体内容 · 前往原文查看

Table 6: PosterBench scoring protocol. “Programmatic” denotes image-native and source-grounded checks; “VLM” denotes a dimension-specific judgment conditioned on the rendered poster and compact source context. The weights sum to 100.

For record i, PosterBench first forms the fixed weighted score from the seven dimension scores, applies the record-level ceiling Ci, and finally averages capped poster scores across the benchmark. Its ceiling families address severe layout damage, insufficient presentation viability, confirmed visible failures, and protected render-integrity gates; inactive ceiling families take value 100. A standard P0 gate has a ceiling of 40, while more severe gate types may impose a lower ceiling. Thus, if 𝐪¯=N−1​∑i𝐪i is the vector displayed by a table row, 110​𝜶⊤​𝐪¯ does not generally equal Overall: the ceiling is applied before the benchmark average. For batches of at least 20 readable posters, a blinded style-homogeneity check may only reduce the professional-aesthetics score; it is not applied to PosterBench-mini, whose 10-poster scale is below that threshold.

The released benchmark rows score rendered poster images. Programmatic signals cover render integrity, occupancy, OCR readability, and source grounding. Each VLM judgment receives the rendered image, a compact paper brief, and selected grounding signals, but no system identity or generation prompt.

Released records identify the system configuration, source case, evaluation status, aggregate score, and dimension scores, enabling each reported row to be audited and reaggregated.

A.5 Released Records and Human Evaluation

The released score archive records system identity, source case and discipline, evaluation status, aggregate score, and per-dimension scores. It distinguishes fresh evaluations from reaggregated records and supports independent auditing of the reported tables.

Blind-review interface and complete task roster.

Figure 15 shows the website used for the system-blind study. It presents two anonymous posters for the same source paper with a shared title, abstract, and PDF; system, model, and harness identities are withheld. The complete roster balances left–right presentation and records Poster A, about equal, Poster B, skip, and an optional critical-failure flag.

The roster visible to each reviewer contains every paper–system-pair task:

|𝒯|=Npapers​(Nsystems2)=100​(42)=600.

It evaluates each of the six unordered system pairs on every paper, providing equal paper coverage and a connected graph for Bradley–Terry estimation. The “0/600” counter denotes this complete roster; submitted non-skip decisions are retained and uncompleted assignments are not imputed.

Figure 15: System-blind Poster Arena interface. Each task presents two anonymous posters for the same paper, with the abstract and PDF as common context. Reviewers select a preference, approximate equality, or skip, and can flag a critical failure. The “0/600” counter denotes the complete roster, rather than a requirement to submit 600 judgments.

The released decision record preserves each anonymous comparison and critical-failure flag. The coefficient is a nominal agreement diagnostic over paper–system-pair items, not the ranking estimator; Figure 10 reports a separate paper-cluster bootstrap for benchmark–human alignment.

Figure 16: Additional matched qualitative comparisons across Design Agents with the same generation prompt; each row shows three independently generated posters for one source paper, at a common scale.

A.6 Additional AutoDesign Poster Demonstrations

Figure 17: Additional AutoDesign poster demonstrations. LongCat-Next and NeRF are rendered from their respective source papers.

Figure 18: Additional AutoDesign poster demonstrations (continued). Attention Is All You Need and DDPM are rendered from their respective source papers.