Haoran Ling

, Yuecheng Li

, Zeyu Song

, Jing Yao

, Shuwen Kang

, Chi Lu

, Wenjin Wu

, Peng Jiang

Work done during an internship at Kuaishou Technology.

Abstract

Optimizing modern recommender models still depends heavily on engineers manually iterating over architectural, objective, and training-strategy changes. While LLM-based agents can automate this trial-and-error process, allowing the LLM to both select modification directions and generate concrete hypotheses often leads to unstable search under limited experiment budgets. Inspired by the above challenge, we propose RecHarness, a Bandit-Routed Agentic Harness for automated recommender model optimization. RecHarness separates the optimization process into two steps: a bandit router selects the next modification direction according to historical validation feedback, while the LLM generates a concrete optimization hypothesis and executable code edit within the selected direction. To sustain long-horizon exploration, RecHarness uses a jump-basin mechanism to activate a structural-jump arm when local edits stagnate. Across multiple recommendation tasks, datasets, and model backbones, RecHarness achieves more stable performance improvements and uses limited trial budgets more effectively than LLM-reasoning search. During a 7-day online A/B test on a large-scale short-video advertising platform, the selected candidate improves ADVV by 2.084%, Revenue by 0.534%, and Exposure by 0.559%. Code is available at https://github.com/6lyc/RecHarness.

Introduction

LLM-based machine learning engineering (MLE) agents have made substantial progress on automated model development. Given a dataset, an evaluation metric, and an initial codebase, these agents can inspect task descriptions, write or modify training code, execute experiments, and iteratively improve candidate solutions using validation results and execution logs (Huang et al. 2023; Chan et al. 2025; Jiang et al. 2025; Nam et al. 2026). Existing methods often formulate this process as search over candidate programs, using multi-branch search, tree search, or graph search to explore possible solutions and validation scores to select promising branches. Recent work such as Reasoning-as-Gradient (Zhang et al. 2026) argues that execution feedback should not be compressed only into scalar scores: error logs, training dynamics, and validation outcomes can also serve as textual optimization signals that guide more directed code updates (Pryzant et al. 2023; Shinn et al. 2023; Madaan et al. 2023; Yuksekgonul et al. 2025; Zhang et al. 2026). This line of work suggests that LLM reasoning is valuable not only for generating code, but also for interpreting experimental feedback and forming the next improvement hypothesis.

Recommender systems (RecSys) optimization (Sun et al. 2019; He et al. 2020; Yang et al. 2026; Li et al. 2026a, b) is a natural and practically important setting for LLM-based MLE agents. Unlike one-shot model construction, recommender development is often a continuous training-code iteration process. In sequential recommendation, ranking and click prediction, and watch-time prediction, model quality is affected by the backbone architecture, training objective, optimization strategy, regularization, feature interaction design, sequence modeling choice, and implementation details. Therefore, the optimization target is not a single fixed recommender template, but a broader engineering search space whose best solution depends on the dataset, task formulation, model family, and evaluation objective. Recent agentic recommender work also emphasizes that modern recommender optimization still relies heavily on iterative engineering, and that agent-based systems can help automate this loop by using persistent memory and evolving optimization methodology (Cheng et al. 2026; Ou et al. 2026; Mu et al. 2026). Our goal is therefore not to optimize a single recommender template, but to build a validation-guided harness that can adapt to different recommendation scenarios, model families, and evaluation objectives.

However, directly applying a general-purpose MLE agent to recommender optimization leaves an important systems challenge. Each candidate modification typically requires code generation, execution, training, and validation, so every trial consumes non-negligible budget. Moreover, Recommender model optimization is not a set of independent one-shot experiments, but a continuous process of model iteration. Each trial provides optimization-relevant evidence beyond its final validation score, such as convergence behavior, training stability, and exposed failure modes or bottlenecks. These signals must be interpreted in the context of the current model state, training dynamics, and previous modifications. Therefore, an automated optimizer cannot rely solely on scalar validation feedback; it must interpret logs, trends, and failure signals from each round, while accumulating cross-round evidence to determine which optimization directions are effective. Recent studies on LLM-based sequential decision making provide a useful motivation for this design. In-context bandit experiments show that general-purpose LLMs may require external summaries or algorithmic support to explore reliably (Krishnamurthy et al. 2024), while LLM-enhanced multi-armed bandit methods suggest that combining LLM reasoning or prediction with classical bandit mechanisms can be more effective than direct LLM arm selection (Sun et al. 2026). These findings align with our setting: validation feedback should be used both as textual feedback for LLM reasoning and as an explicit posterior state for guiding future trials.

Based on the above findings, we introduce RecHarness, a validation-guided optimization harness for self-evolving recommender systems. RecHarness organizes optimization as a sequence of isolated training-validation trials. At each round, it maintains candidate optimization arms defined for the target recommender setting, updates their posterior state using validation feedback, and uses Thompson-style routing to select promising arms under a limited trial budget. Given the selected arms, the LLM uses the current incumbent, execution logs, validation trends, and a dynamically updated Experiment Skill distilled from experiment memory to form concrete improvement hypotheses and generate executable code modifications. For high-impact changes such as architecture, loss changes, RecHarness further supports a retuning window so that the system can evaluate whether a structural jump becomes beneficial after local adaptation. RecHarness therefore combines LLM reasoning with validation-driven posterior routing: scalar validation evidence accumulates across trials to decide which optimization directions to try next, while textual feedback helps the LLM generate concrete hypotheses and executable code mutations within the selected directions.

• We introduce RecHarness, the first optimization harness for self-evolving recommender systems, designed to support diverse recommendation scenarios.

• We design a bandit-routed optimization mechanism that jointly leverages scalar scores and textual feedback through bandit routing and LLM reasoning.

• We conduct empirical studies across recommender tasks, datasets, and model templates, demonstrating the effectiveness of RecHarness in improving recommendation performance under limited trials, and an online A/B test further confirms its gains in business metrics.

Related Work

LLM-driven MLE Agents

Automated machine learning (AutoML) (Thornton et al. 2013; Feurer et al. 2015; Zheng et al. 2023) development has long been pursued through hyperparameter optimization (HPO) (Feurer and Hutter 2019; Akiba et al. 2019), and neural architecture search (NAS) (Luo et al. 2018; Liu, Simonyan, and Yang 2019; Ren et al. 2021), but these methods operate within a predefined configuration space. They tune scalar hyperparameters, or search within a fixed architectural template, and therefore struggle with code-level changes that fall outside any enumerable grid, such as redesigning a loss function, switching architectural blocks, or altering a pooling strategy. Building on the interleaved reasoning-and-acting paradigm (Yao et al. 2023), LLM-based MLE agents read task descriptions, generate training code, run experiments, and iteratively refine candidate solutions using validation results and execution logs (Chan et al. 2025). A dominant line of work models this as a search problem over candidate programs. AIDE (Jiang et al. 2025) organizes solutions into a tree and adopts a greedy policy that drafts, debugs, or improves the best node. AIRA (Toledo et al. 2026) formalizes such agents as search policies operating over operator sets These methods are largely gradient-free, compressing execution feedback into scalar validation scores used only to rank and prune branches. More recent work argues that this discards valuable signal: Reasoning-as-Gradient (Zhang et al. 2026) treats error logs, training dynamics, and validation outcomes as textual gradients that guide directed updates, and shows that such directed optimization increasingly surpasses exhaustive tree search as the underlying model’s reasoning capability grows.

As one of the most widely deployed applications of ML, recommendation is a natural target for such agents (Zheng et al. 2023; Wang et al. 2022; Zhao et al. 2021; Lyu et al. 2022). Its development is inherently iterative: sequential recommendation, ranking/CTR prediction, and watch-time modeling are all shaped by training objectives, optimization strategies, feature interactions, and structural choices that engineers adjust round after round, making it well suited to turning experience-driven, code-level iteration into an executable automated search.

Self-Evolving Agentic RecSys

Since recommendation development is inherently iterative, a growing body of work casts LLM-based agents as automated recommendation engineers that generate, implement, and evaluate model improvements in a closed loop. At industrial scale, Google’s dual-loop framework (Wang et al. 2026) screens hypotheses via cheap proxy metrics in an inner loop and validates candidates against delayed business metrics in an outer loop. GEARS (Yun et al. 2026) encapsulates ranking expertise into reusable agent skills steered by high-level intent. AgentX (Lao et al. 2026) runs a four-stage closed loop and distills execution trajectories into semantic-gradient updates that sharpen the agents themselves. EvoRec (Mu et al. 2026) co-evolves the model and its methodology by distilling reusable strategies from a persistent experiment memory. NOVA (Liu et al. 2026) guides architecture evolution with an SGD-inspired architecture gradient aggregating prior edits, diagnostics, and metric feedback.

However, letting the LLM freely decide the exploration direction is overly divergent: candidates are proposed in an open space without principled trial allocation, yet each edit incurs a costly, noisy training–validation trial, resulting in low success rates and low optimization efficiency (Krishnamurthy et al. 2024; Sun et al. 2026; Bouneffouf and Feraud 2026). The key is thus to combine recommendation priors with posterior exploration probabilities as a harness governing where to explore next.

Methodology

RecHarness organizes recommender model iteration as a three-level control process. First, human experts define the optimization objective, validation metric, and candidate edit arms. Second, a bandit router allocates the limited trial budget across candidate arms using scalar validation feedback, deciding which edit dimensions should be searched in the next round. Third, Experiment Skill and LLM reasoning form the next improvement hypotheses. In short, humans define what to optimize and which arms to consider; RecHarness decides where to allocate trials and how to form the next improvement hypotheses. Figure 1 gives an overview of this framework.

Recommender Optimization as a Bandit Problem

We consider recommender optimization under a finite time budget. The input is an initial recommender implementation , a training-validation split , a validation metric , and a trial-runtime budget. Each trial consists of applying a model mutation, training the mutated recommender, and evaluating it using the validation metric. The metric is task-dependent; for sequential recommendation experiments, we use validation HR@10 on the target datasets. We write the executable training procedure induced by implementation as

(1)

where denotes the trained parameters or checkpoint produced by the implementation.

Formally, given a task and the space of valid model implementations , our goal is to find the best implementation under a resource budget :

(2)

where evaluates implementation on task , and denotes the cost of generating, training, and validating it. RecHarness operationalizes this budget-constrained objective as sequential trial allocation: only a budgeted set of candidate mutations is evaluated at each round, and their validation outcomes are used to guide subsequent search. For a trial group , the consumed budget is

(3)

where includes code generation, execution, training, and validation cost for the candidate produced under arm .

RecHarness represents the search space as a predefined set of edit arms:

(4)

The arm set is determined by the target task, the model being optimized, and the human-defined candidate arms, making the formulation applicable to a broad range of recommender model optimization scenarios. Each arm represents an interpretable edit dimension rather than an exact code patch or a scalar hyperparameter. For sequential recommenders, examples include tuning learning-rate schedules, adjusting dropout or weight decay, changing embedding dimensions, modifying the number of layers or attention heads, changing the sequence pooling strategy, adding features, or changing the loss function. Depending on their edit granularity, RecHarness separates arms into local arms for refinement and jump arms for non-local basin transitions:

(5)

The search is incumbent-based. Let denote the best validated implementation before round , with validation score . At round , RecHarness selects one or more arms and generates candidate model mutations relative to , rather than independently mutating the original template. We denote the LLM-conditioned mutation operator by

(6)

where is the Experiment Skill, summarizes recent logs and validation traces, and denotes the frozen LLM used for code generation. After executing the resulting trials, a candidate is promoted only if it improves over the incumbent:

(7)

where is the set of valid candidates in round . Otherwise, the incumbent remains unchanged.

For an arm selected at round , scalar feedback is derived from its validation outcome. Let the validation improvement over the incumbent be

(8)

To compare candidates produced in the same parallel group, we normalize the improvement by the group statistics:

(9)

where is a small constant for numerical stability. The posterior update uses a binary success signal. A normal local trial is treated as successful only if it is valid, outperforms the average candidate in the same trial group, and does not fall below the historical incumbent:

(10)

This abstraction is viewed as a black-box, incumbent-conditioned bandit problem. The reward is a binary success signal for edit arm , observed only after its candidate has been generated, executed, trained, and validated. The realized value of this reward depends on the current incumbent, previous edits, and the implementation context. Therefore, RecHarness does not assume that each arm has a fixed global success probability. Instead, the posterior statistics of each arm serve as local evidence for allocating future trials under a limited budget.

Thompson Sampling for Exploration–Exploitation

The bandit router answers the trial-allocation question of where to search next under a limited trial budget. RecHarness uses Thompson sampling to allocate trials across edit arms. For each arm , the system maintains a Beta posterior

(11)

where and summarize previous successful and unsuccessful outcomes. At the beginning of each round, RecHarness samples from each available arm’s Beta posterior. This value is a sampled success probability under the current posterior, and RecHarness selects the arms with the largest samples. The selected arms do not prescribe exact code edits; instead, they define the semantic directions for the next model-iteration round.

After each round, RecHarness updates the corresponding arm posterior using the binary validation outcome defined above:

(12)

For grouped parallel trials, the selected group is

(13)

where is the set of currently available arms. This routing mechanism supports both exploitation and exploration: it tends to allocate trials to edit dimensions that have previously produced validation improvements, while still assigning probability mass to uncertain arms with limited evidence. RecHarness also supports grouped parallel trials. In a normal search round, it selects arms and executes candidate mutations in parallel, allowing multiple candidate directions to be compared under the same incumbent. This design also follows recent evidence that directly asking LLMs to choose bandit arms can be unreliable, whereas combining LLM reasoning with explicit bandit structure gives a more controlled exploration–exploitation mechanism (Krishnamurthy et al. 2024; Sun et al. 2026).

Experiment Skill and Feedback

After the bandit router selects arms, RecHarness uses Experiment Skill and LLM reasoning to form the next improvement hypothesis within each selected arm. Experiment Skill is a compact textual guide that records the current incumbent, recent successful edits, invalid or rejected directions, failure reasons, and short summaries of validation trends. It does not rank arms and does not replace Thompson sampling; it conditions the LLM after arms have been selected so that the next improvement hypothesis reflects prior evidence.

Experiment Skill is updated automatically after each validation round. For successful or promoted trials, RecHarness distills the arm, patch summary, validation score, and improvement pattern into reusable lessons and appends them to recent text gradients. For failed, invalid, or harmful trials, the system extracts avoid rules into a failure-feedback section, such as avoiding repeated interface mismatches, training crashes, or known low-yield edits. The system then renders a refreshed Experiment Skill document from the current incumbent, score history, and recent trial digests.

Formally, the Experiment Skill state is updated by a summarization operator

(14)

where denotes the textual Experiment Skill state and denotes execution status, error messages, and compact training logs. The Experiment Skill affects the mutation operator but not the posterior update above, preventing textual summaries from silently overriding the scalar evidence accumulated by the router.

Thus, the two feedback channels serve different roles. Scalar validation feedback updates the bandit posterior for deciding which arms to search next, while textual Experiment Skill feedback helps the LLM reason about the next improvement hypothesis within a selected arm, which patterns to reuse, and which failure modes to avoid.

Basin-aware Jump and Retuning

Incumbent-based search is sample efficient, but it may eventually saturate within a local basin. RecHarness therefore introduces a lightweight basin-aware jump mechanism. We partition the arm set into local arms , which perform incremental refinement within the current basin, and jump arms , which make higher-level structural changes that may move the search to a different basin.

Let denote the incumbent validation score before round . We measure the recent improvement rate over a window of rounds as

(15)

When the recent improvement rate falls below a threshold , the current basin is considered saturated and jump arms become available:

(16)

Here, allows jump arms to participate in arm selection but does not necessarily require a jump to be performed. For a jump arm , let denote the best implementation obtained after local retuning rounds starting from its jump candidate. The jump is accepted if the retuned branch improves upon the pre-jump incumbent by a margin :

(17)

This delayed criterion allows structural changes to be evaluated after local adaptation rather than solely by their immediate validation score.

Experiments

Experimental Settings

Tasks and datasets. We evaluate RecHarness on two types of recommendation tasks. The first setting is sequential recommendation on four Amazon Reviews datasets (Hou et al. 2026): Movies, Scientific, Electronics, and CDs. Each user and item has at least five interactions, and the model predicts the next item from user histories. The second setting is watch-time and ranking prediction on KuaiRec (Gao et al. 2022), a dense user-video interaction dataset for watch-time, watch-ratio, and ranking objectives. Table 1 and Table 2 summarize the preprocessed statistics.

媒体内容 · 前往原文查看

Dataset	#Users	#Items	#Interactions
Movies	11,947	17,490	144,071
Scientific	23,627	25,764	266,164
Electronics	27,601	31,533	292,308
CDs	18,481	30,951	284,695

媒体内容 · 前往原文查看

Split	Interactions	Users	Items	Avg. Inter./User
Train	12,530,806	7,176	10,728	1,746
Test	4,676,570	1,411	3,327	3,314

媒体内容 · 前往原文查看

Dataset	Metric	GRU4Rec			BERT4Rec			NextItNet			SASRec			HSTU
		Base	Ours	Paper	Base	Ours	Paper	Base	Ours	Paper	Base	Ours	Paper	Base	Ours
Movies	N@10	0.1267	0.3349	0.3152	0.2585	0.3393	0.2959	0.1347	0.3326	0.2538	0.3688	0.4023	0.3459	0.3253	0.3794
	N@20	0.1570	0.3709	0.3494	0.2943	0.3734	0.3303	0.1661	0.3666	0.2879	0.4039	0.4348	0.3745	0.3611	0.4121
	H@10	0.2317	0.5179	0.4883	0.4302	0.5269	0.4785	0.2617	0.5168	0.4221	0.5335	0.5923	0.5180	0.5038	0.5634
	H@20	0.3525	0.6606	0.6245	0.5723	0.6620	0.6213	0.3868	0.6516	0.5522	0.7034	0.7210	0.6310	0.6460	0.6932
Scientific	N@10	0.1663	0.3017	0.2642	0.2379	0.2880	0.2576	0.2179	0.2870	0.2263	0.2805	0.3311	0.2918	0.2804	0.3265
	N@20	0.1950	0.3377	0.2974	0.2728	0.3246	0.2913	0.2482	0.3243	0.2657	0.3195	0.3645	0.3245	0.3184	0.3615
	H@10	0.2831	0.4970	0.4313	0.4081	0.4763	0.4437	0.3632	0.4749	0.3908	0.4701	0.5381	0.4691	0.4690	0.5220
	H@20	0.3979	0.6396	0.5524	0.5471	0.6216	0.5822	0.4833	0.6227	0.5356	0.6242	0.6700	0.5987	0.6193	0.6604
Electronics	N@10	0.1789	0.2442	0.2364	0.1870	0.2277	0.1867	0.2293	0.2441	0.1712	0.2614	0.2635	0.2267	0.2424	0.2682
	N@20	0.2077	0.2774	0.2743	0.2170	0.2613	0.2172	0.2579	0.2750	0.2069	0.2981	0.2991	0.2606	0.2804	0.3038
	H@10	0.3005	0.3940	0.3843	0.3186	0.3745	0.3325	0.3833	0.3965	0.3017	0.4271	0.4336	0.3749	0.4038	0.4335
	H@20	0.4148	0.5257	0.5196	0.4377	0.5081	0.4740	0.4967	0.5194	0.4324	0.5729	0.5750	0.5096	0.5547	0.5744
CDs	N@10	0.1394	0.3764	0.2155	0.2872	0.3789	0.3019	0.1367	0.3768	0.2207	0.2614	0.4465	0.3451	0.3192	0.4046
	N@20	0.1733	0.4127	0.2530	0.3255	0.4152	0.3386	0.1678	0.4118	0.2562	0.2981	0.4770	0.3795	0.3553	0.4382
	H@10	0.2586	0.5872	0.3712	0.4693	0.5908	0.5018	0.2544	0.5853	0.3842	0.5833	0.6593	0.5278	0.5124	0.6080
	H@20	0.3936	0.7310	0.5092	0.6213	0.7345	0.6605	0.3781	0.7233	0.5422	0.7309	0.7793	0.6635	0.6551	0.7409
Avg. H@10 over Datasets		0.2685	0.4990	0.4188	0.4066	0.4921	0.4391	0.3156	0.4934	0.3747	0.5035	0.5558	0.4725	0.4723	0.5317

Metrics. For sequential recommendation, we use leave-last-out evaluation: the last item is used for testing, the second-to-last for validation, and earlier interactions for training. Each evaluation set contains one positive item and 99 sampled negatives. We report Hit Ratio (HR@N) and Normalized Discounted Cumulative Gain (NDCG@N), with N is 10 and 20. For a compact Amazon summary, we use Avg. HR@10 over datasets:

(18)

where is a method or model variant and is the Amazon dataset set. Per-dataset metrics remain the primary evidence.

For KuaiRec, we report WT-XAUC, WT-MAE, WR-XAUC, and WR-MAE. WT and WR denote watch-time and watch-ratio. Lower MAE and higher XAUC are better (Ma et al. 2026).

Search protocol. RecHarness uses only validation feedback for candidate selection, routing updates, and version promotion. The test set is used only once after search for final evaluation. Each run starts from a cold-start template, runs one baseline trial, and then searches over candidate modifications under a bandit arm (More details in Supplement).

Trial execution and budget. Each candidate runs in an isolated workspace, and each round typically evaluates multiple trials in parallel. Trials return validation scores, logs, errors, and status. RecHarness promotes only executable candidates that improve the incumbent, and stops when the remaining budget cannot support another trial group. For every model, we set a total GPU-time budget of 43,200 s.

Optimized Recommendation Models. Next, we clarify the optimization targets used by RecHarness. (1) Amazon Reviews models. For sequential recommendation, we evaluate five model families that cover distinct sequence-modeling paradigms: GRU4Rec (Hidasi et al. 2015), BERT4Rec (Sun et al. 2019), NextItNet (Yuan et al. 2019), SASRec (Kang and McAuley 2018), and HSTU (Zhai et al. 2024). (2) KuaiRec models. For KuaiRec, we evaluate three models that cover different feedback-modeling designs: D2Q (Zhan et al. 2022), TPM (Lin et al. 2023), and GR (Ma et al. 2026).

Overall Performance

Table 3 shows consistent gains across all five models. RecHarness improves average HR@10 by 85.85% on the weaker GRU4Rec baseline and by 12.58% on the stronger HSTU baseline. It also outperforms the matched results from Kim et al. (2025).

Table 4 confirms that these gains transfer across scenarios and objectives. On the weaker TPM baseline, RecHarness reduces both MAE metrics by over 26%; on the stronger GR baseline, it still consistently improves all four metrics. It also surpasses Ma et al. (2026), demonstrating RecHarness is not merely a tuner for a single template, but can transfer across recommendation scenarios, model types, and evaluation objectives.

媒体内容 · 前往原文查看

Model	Metric	Base	Ours	Paper
D2Q	WT-XAUC	0.5310	0.5947 (+12.00%)	0.5650
	WT-MAE	3.4129	3.3107 (-2.99%)	5.4260
	WR-XAUC	0.7307	0.7444 (+1.87%)	0.7120
	WR-MAE	0.3558	0.3451 (-3.01%)	0.3710
TPM	WT-XAUC	0.5342	0.5808 (+8.72%)	0.5710
	WT-MAE	4.5372	3.3406 (-26.37%)	3.4560
	WR-XAUC	0.7012	0.7418 (+5.79%)	0.7340
	WR-MAE	0.4730	0.3481 (-26.41%)	0.3610
GR	WT-XAUC	0.6161	0.6176 (+0.24%)	0.6140
	WT-MAE	3.1901	3.1851 (-0.16%)	3.1960
	WR-XAUC	0.7528	0.7535 (+0.09%)	0.7530
	WR-MAE	0.3323	0.3319 (-0.12%)	0.3330

Ablation Study

To isolate the contribution of each module in our RecHarness, we conduct ablation studies on the four Amazon datasets with SASRec fixed as the underlying recommendation template. All compared variants use the same validation-driven update rule, GPU time budget, and four parallel trials per round.

The ablations are defined as follows:

• RecHarness is the full method with Thompson-style routing, Experiment Skill and validation-driven updates.

• TR w/ Random replaces Thompson Routing (TR) with uniform random arms selection.

• TR w/ LLM lets the LLM select arms from historical textual feedback, without Thompson Posterior Routing.

• w/o Bandit removes predefined edit dimensions and lets the LLM freely propose changes.

Figure 2 shows validation best-so-far trajectories. Starting from 0.5050, RecHarness reaches 0.6125 by Round 2 and 0.6342 by Round 4, improving faster than all ablations. This indicates that routing concentrates limited trials on high-return directions early.

RecHarness is best at every validation checkpoint and in final test metrics. TR w/ LLM improves over random routing, showing that textual feedback helps, but it remains below RecHarness. w/o Bandit finds useful local edits but generalizes worse, indicating that structured edit dimensions stabilize search.

Analysis Experiments

We measure how often non-baseline trials improve the round-start best validation score, and how large those improvements are, as shown in Table 5. The results show two effects of the RecHarness routing design. First, RecHarness finds improvements more accurately than the two routing ablations: 47.92% of its non-baseline trials refresh the round-start best score, compared with only 22.45% for TR w/ Random replacement and 21.74% for TR w/ LLM. This indicates that RecHarness does more than record past outcomes: it uses accumulated validation feedback to allocate future trials toward edit directions that are empirically more likely to improve the current best model. Second, RecHarness finds higher-quality improvements than the w/o Bandit. Although w/o Bandit still improves 41.67% of trials, its average and maximum gains are both lower than those of RecHarness, especially in maximum gain (10.16% vs. 24.00%). This suggests that unrestricted LLM search can still discover useful directions, but it’s less likely than bandit routing to allocate trials to high-upside directions.

Thus, RecHarness improves mainly by allocating limited trials more effectively. Together, the LLM proposes hypotheses within selected directions, while bandit routing makes limited trials more likely to produce large gains.

媒体内容 · 前往原文查看

Method	Improving / All	Ratio	Avg Gain	Max Gain
RecHarness	23 / 48	47.92%	5.06%	24.00%
TR w/ Random	11 / 49	22.45%	4.94%	16.63%
TR w/ LLM	10 / 46	21.74%	5.04%	15.11%
w/o Bandit	20 / 48	41.67%	4.05%	10.16%

Online A/B Test

We deploy RecHarness in a large-scale short-video advertising ranking system. The online baseline is a mature production ranking model with a shared-bottom DNN and multi-feature fusion, taking user-, item-, and combine-side features plus six groups of real-time user commerce behavior sequences as input.

In the offline production environment, human experts define the optimization objective, validation metric, and candidate arms. Local arms cover feature injection, fusion-position adjustment, and training-configuration tuning; jump arms cover higher-level structural changes such as sequence encoder upgrades. RecHarness then self-iterates over these arms. After repeated local-arm attempts fail to yield stable gains, the router estimates that the search is nearing a local basin ceiling, admits the jump arms, and selects the sequence_encoder_upgrade arm. Within this arm, the LLM reasons over the Experiment Skill and historical trials. Memory shows that existing HSTU-related modules mainly process compressed virtual tokens rather than modeling item-item interactions within the raw behavior sequences, and that prior attempts at feature crossing and fusion structures gave no stable gains. RecHarness therefore attributes the main structural gap to the missing intra-sequence relation modeling and generates an intra-sequence self-attention candidate. The candidate applies self-attention to each of the six real-time behavior sequences, compresses each into a 32-dim vector via an MLP, and concatenates them into a 192-dim representation injected into the shared-bottom ranker. Offline, it improves production AUC by 0.09 %.

In a 7-day online A/B test on 10% of traffic (Table 6), the candidate improves ADVV (Advertiser Value) (Chai et al. 2025) by 2.084%, Revenue by 0.534%, and Exposure by 0.559%. This shows RecHarness can discover a deployable structural improvement within a human-defined arm space and transfer offline gains to online business metrics.

媒体内容 · 前往原文查看

Scenario	ADVV	Revenue	Exposure
Short-video Advertising	+2.084%	+0.534%	+0.559%

Conclusion

We introduced RecHarness, a bandit-routed agentic framework for automated recommender model iteration under limited budgets. Its key design decouples edit-direction selection from concrete code mutation: validation-driven Thompson routing accumulates cross-trial evidence over structured optimization dimensions, while LLM reasoning interprets feedback and proposes executable edits within the selected directions. Across two recommendation scenarios and eight models, RecHarness consistently improves performance. Moreover, on a large-scale short-video advertising platform, the RecHarness-discovered candidate delivers significant gains, confirming its practical value in industrial recommendation.

References

• Akiba et al. (2019) Akiba, T.; Sano, S.; Yanase, T.; Ohta, T.; and Koyama, M. 2019. Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, 2623–2631.
• Bouneffouf and Feraud (2026) Bouneffouf, D.; and Feraud, R. 2026. Multi-armed bandits meet large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, 39682–39690.
• Chai et al. (2025) Chai, Z.; Ren, Q.; Xiao, X.; Yang, H.; Han, B.; Zhang, S.; Chen, D.; Lu, H.; Zhao, W.; Yu, L.; et al. 2025. Longer: Scaling up long sequence modeling in industrial recommenders. In Proceedings of the Nineteenth ACM Conference on Recommender Systems, 247–256.
• Chan et al. (2025) Chan, J. S.; Chowdhury, N.; Jaffe, O.; Aung, J.; Sherburn, D.; Mays, E.; Starace, G.; Liu, K.; Maksin, L.; Patwardhan, T.; et al. 2025. Mle-bench: Evaluating machine learning agents on machine learning engineering. In International Conference on Learning Representations, volume 2025, 50466–50494.
• Cheng et al. (2026) Cheng, Y.; Zhou, L.; Liang, X.; Luo, D.; Lee, T.; Zheng, K.; Zhang, W.; Cai, M.; Dong, J.; and Zhang, A. 2026. Let the Agent Steer: Closed-Loop Ranking Optimization via Influence Exchange. arXiv preprint arXiv:2603.27765.
• Feurer and Hutter (2019) Feurer, M.; and Hutter, F. 2019. Hyperparameter optimization. In Automated machine learning: Methods, systems, challenges, 3–33. Springer.
• Feurer et al. (2015) Feurer, M.; Klein, A.; Eggensperger, K.; Springenberg, J.; Blum, M.; and Hutter, F. 2015. Efficient and robust automated machine learning. Advances in neural information processing systems, 28.
• Gao et al. (2022) Gao, C.; Li, S.; Lei, W.; Chen, J.; Li, B.; Jiang, P.; He, X.; Mao, J.; and Chua, T.-S. 2022. KuaiRec: A fully-observed dataset and insights for evaluating recommender systems. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, 540–550.
• He et al. (2020) He, X.; Deng, K.; Wang, X.; Li, Y.; Zhang, Y.; and Wang, M. 2020. Lightgcn: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, 639–648.
• Hidasi et al. (2015) Hidasi, B.; Karatzoglou, A.; Baltrunas, L.; and Tikk, D. 2015. Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939.
• Hou et al. (2026) Hou, Y.; Li, J.; Fu, X.; He, Z.; Yan, A.; Chen, X.; and McAuley, J. 2026. Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic Encoders. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 3251–3265.
• Huang et al. (2023) Huang, Q.; Vora, J.; Liang, P.; and Leskovec, J. 2023. Mlagentbench: Evaluating language agents on machine learning experimentation. arXiv preprint arXiv:2310.03302.
• Jiang et al. (2025) Jiang, Z.; Schmidt, D.; Srikanth, D.; Xu, D.; Kaplan, I.; Jacenko, D.; and Wu, Y. 2025. Aide: Ai-driven exploration in the space of code. arXiv preprint arXiv:2502.13138.
• Kang and McAuley (2018) Kang, W.-C.; and McAuley, J. 2018. Self-attentive sequential recommendation. In 2018 IEEE international conference on data mining (ICDM), 197–206. IEEE.
• Kim et al. (2025) Kim, S.; Kang, H.; Kim, K.; Kim, J.; Kim, D.; Yang, M.; Oh, K.; McAuley, J.; and Park, C. 2025. Lost in Sequence: Do Large Language Models Understand Sequential Recommendation? In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, 1160–1171.
• Krishnamurthy et al. (2024) Krishnamurthy, A.; Harris, K.; Foster, D. J.; Zhang, C.; and Slivkins, A. 2024. Can large language models explore in-context? Advances in Neural Information Processing Systems, 37: 120124–120158.
• Lao et al. (2026) Lao, C.; Pan, F.; Ma, G.; Li, H.; Lin, H.; Shi, J.; Zhao, K.; Gai, K.; Zhou, M.; Zhou, Q.; et al. 2026. AgentX: Towards Agent-Driven Self-Iteration of Industrial Recommender Systems. arXiv preprint arXiv:2606.26859.
• Li et al. (2026a) Li, Y.; Ju, H.; Song, Z.; Yang, W.; Lu, C.; Jiang, P.; and Gai, K. 2026a. RecGOAT: Graph Optimal Adaptive Transport for LLM-Enhanced Multimodal Recommendation with Dual Semantic Alignment. arXiv preprint arXiv:2602.00682.
• Li et al. (2026b) Li, Y.; Song, Z.; Yao, J.; Lu, C.; Jiang, P.; and Gai, K. 2026b. Taiji: Pareto Optimal Policy Optimization with Semantics-IDs Trade-off for Industrial LLM-Enhanced Recommendation. arXiv preprint arXiv:2606.03866.
• Lin et al. (2023) Lin, X.; Chen, X.; Song, L.; Liu, J.; Li, B.; and Jiang, P. 2023. Tree based progressive regression model for watch-time prediction in short-video recommendation. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 4497–4506.
• Liu, Simonyan, and Yang (2019) Liu, H.; Simonyan, K.; and Yang, Y. 2019. DARTS: Differentiable Architecture Search. In International Conference on Learning Representations.
• Liu et al. (2026) Liu, S.; Fang, L.; Sun, Y.; Huang, S.; Luo, Q.; Chen, X.; Liu, D.; Ma, C.; Chai, Z.; Wang, H.; et al. 2026. NOVA: A Verification-Aware Agent Harness for Architecture Evolution in Industrial Recommender Systems. arXiv preprint arXiv:2606.27243.
• Luo et al. (2018) Luo, R.; Tian, F.; Qin, T.; Chen, E.; and Liu, T.-Y. 2018. Neural architecture optimization. Advances in neural information processing systems, 31.
• Lyu et al. (2022) Lyu, F.; Tang, X.; Guo, H.; Tang, R.; He, X.; Zhang, R.; and Liu, X. 2022. Memorize, factorize, or be naive: Learning optimal feature interaction methods for CTR prediction. In 2022 IEEE 38th International Conference on Data Engineering (ICDE), 1450–1462. IEEE.
• Ma et al. (2026) Ma, H.; Tian, K.; Zhang, T.; Zhang, X.; Zhou, H.; Jin, C.; Chen, C.; Li, H.; Guan, J.; and Zhou, S. 2026. Generative regression based watch time prediction for short-video recommendation. In Proceedings of the ACM Web Conference 2026, 6183–6193.
• Madaan et al. (2023) Madaan, A.; Tandon, N.; Gupta, P.; Hallinan, S.; Gao, L.; Wiegreffe, S.; Alon, U.; Dziri, N.; Prabhumoye, S.; Yang, Y.; et al. 2023. Self-refine: Iterative refinement with self-feedback. Advances in neural information processing systems, 36: 46534–46594.
• Mu et al. (2026) Mu, L.; Deng, H.; Xing, H.; Hu, J.; Zhang, Y.; and Zeng, X. 2026. EvoRec: Self Evolving Agentic Recommender Systems. arXiv preprint arXiv:2606.28368.
• Nam et al. (2026) Nam, J.; Yoon, J.; Chen, J.; Shin, J.; Arik, S.; and Pfister, T. 2026. Mle-star: Machine learning engineering agent via search and targeted refinement. Advances in Neural Information Processing Systems, 38: 116692–116712.
• Ou et al. (2026) Ou, K.; Wu, C.; Wang, X.; Zheng, B.; Zhao, W. X.; Li, W.; Zhang, L.; Chen, S.; and Wen, J.-R. 2026. Deep Research for Recommender Systems. arXiv preprint arXiv:2603.07605.
• Pryzant et al. (2023) Pryzant, R.; Iter, D.; Li, J.; Lee, Y.; Zhu, C.; and Zeng, M. 2023. Automatic prompt optimization with “gradient descent” and beam search. In Proceedings of the 2023 conference on empirical methods in natural language processing, 7957–7968.
• Ren et al. (2021) Ren, P.; Xiao, Y.; Chang, X.; Huang, P.-Y.; Li, Z.; Chen, X.; and Wang, X. 2021. A comprehensive survey of neural architecture search: Challenges and solutions. ACM Computing Surveys (CSUR), 54(4): 1–34.
• Shinn et al. (2023) Shinn, N.; Cassano, F.; Gopinath, A.; Narasimhan, K.; and Yao, S. 2023. Reflexion: Language agents with verbal reinforcement learning. Advances in neural information processing systems, 36: 8634–8652.
• Sun et al. (2019) Sun, F.; Liu, J.; Wu, J.; Pei, C.; Lin, X.; Ou, W.; and Jiang, P. 2019. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM international conference on information and knowledge management, 1441–1450.
• Sun et al. (2026) Sun, J.; Wang, Z.; Yang, R.; Xiao, C.; Lui, J. C.; and Dai, Z. 2026. Large language model-enhanced multi-armed bandits. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 8130–8145.
• Thornton et al. (2013) Thornton, C.; Hutter, F.; Hoos, H. H.; and Leyton-Brown, K. 2013. Auto-WEKA: Combined selection and hyperparameter optimization of classification algorithms. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, 847–855.
• Toledo et al. (2026) Toledo, E.; Hambardzumyan, K.; Josifoski, M.; Hazra, R.; Baldwin, N.; Audran-Reiss, A.; Kuchnik, M.; Magka, D.; Jiang, M.; Lupidi, A.; et al. 2026. Ai research agents for machine learning: Search, exploration, and generalization in mle-bench. Advances in Neural Information Processing Systems, 38: 35309–35348.
• Wang et al. (2026) Wang, H.; Wu, Y.; Chang, D.; Wei, L.; and Heldt, L. 2026. Self-evolving recommendation system: End-to-end autonomous model optimization with LLM agents. arXiv preprint arXiv:2602.10226.
• Wang et al. (2022) Wang, Y.; Zhao, X.; Xu, T.; and Wu, X. 2022. Autofield: Automating feature selection in deep recommender systems. In Proceedings of the ACM Web Conference 2022, 1977–1986.
• Yang et al. (2026) Yang, W.; Zhong, R.; Chen, Y.; Lu, C.; and Jiang, P. 2026. Structured Spectral Reasoning for Frequency-Adaptive Multimodal Recommendation. Advances in Neural Information Processing Systems, 38: 28122–28143.
• Yao et al. (2023) Yao, S.; Zhao, J.; Yu, D.; Du, N.; Shafran, I.; Narasimhan, K. R.; and Cao, Y. 2023. ReAct: Synergizing Reasoning and Acting in Language Models. In The Eleventh International Conference on Learning Representations.
• Yuan et al. (2019) Yuan, F.; Karatzoglou, A.; Arapakis, I.; Jose, J. M.; and He, X. 2019. A simple convolutional generative network for next item recommendation. In Proceedings of the twelfth ACM international conference on web search and data mining, 582–590.
• Yuksekgonul et al. (2025) Yuksekgonul, M.; Bianchi, F.; Boen, J.; Liu, S.; Lu, P.; Huang, Z.; Guestrin, C.; and Zou, J. 2025. Optimizing generative ai by backpropagating language model feedback. Nature, 639(8055): 609–616.
• Yun et al. (2026) Yun, L.; Wu, Y.; Liu, H.; Liu, X.; Xu, Z.; Wang, Y.; Xia, Y.; Wang, P.; Gao, M.; Wang, Y.; et al. 2026. Decoding ML Decision: An Agentic Reasoning Framework for Large-Scale Ranking System. arXiv preprint arXiv:2602.18640.
• Zhai et al. (2024) Zhai, J.; Liao, L.; Liu, X.; Wang, Y.; Li, R.; Cao, X.; Gao, L.; Gong, Z.; Gu, F.; He, M.; et al. 2024. Actions speak louder than words: Trillion-parameter sequential transducers for generative recommendations. arXiv preprint arXiv:2402.17152.
• Zhan et al. (2022) Zhan, R.; Pei, C.; Su, Q.; Wen, J.; Wang, X.; Mu, G.; Zheng, D.; Jiang, P.; and Gai, K. 2022. Deconfounding duration bias in watch-time prediction for video recommendation. In Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining, 4472–4481.
• Zhang et al. (2026) Zhang, Y.; Yang, X.; Yang, X.; Xian, B.; Li, Q.; Fang, S.; Li, J.; Wang, J.; Xu, M.; Zhang, Y.; et al. 2026. Reasoning as Gradient: Scaling MLE Agents Beyond Tree Search. In Findings of the Association for Computational Linguistics: ACL 2026, 9013–9038.
• Zhao et al. (2021) Zhao, X.; Liu, H.; Fan, W.; Liu, H.; Tang, J.; and Wang, C. 2021. Autoloss: Automated loss function search in recommendations. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, 3959–3967.
• Zheng et al. (2023) Zheng, R.; Qu, L.; Cui, B.; Shi, Y.; and Yin, H. 2023. Automl for deep recommender systems: A survey. ACM Transactions on Information Systems, 41(4): 1–38.