# 对9B开源模型进行500美元RL微调，在目录审查中表现超越前沿模型

- 来源：Hacker News 热门（buzzing.cc 中文翻译）
- 作者：ilreb
- 发布时间：2026-07-28 14:17
- AIHOT 分数：55
- AIHOT 链接：https://aihot.virxact.com/items/cms49va2801erroep7mkjcc86
- 原文链接：https://fermisense.com/when-machines-take-the-wheel

## AI 摘要

对9B开源模型进行GRPO强化学习微调（成本约500美元），在目录审查工作流中达到最高可得分数的87.3%，而最佳前沿模型配置仅达76.9%，相对提升13.5%。该微调模型每1000条listing成本0.50美元，比最便宜的前沿方案便宜40倍，比最贵的便宜约340倍。Ramp数据显示，2022年11月至2025年12月间，AI支出最高的公司收入增长2.2倍，而零AI支出的公司仅增长约15%。

## 正文

媒体内容 · 前往原文查看

Cost vs. quality on catalog integrity

The article in one picture: on the same catalog-review workflow, with the same tools, images, and scorer, our GRPO fine-tune of a 9B open-source model (pink) beats every frontier configuration we tested, at $0.50 per 1,000 listings: 40× cheaper than the least expensive frontier setup and ~340× cheaper than the most expensive.

Part I

Everyone asked the same question

Since ChatGPT launched in 2022, business leaders have been asking the same question: what can AI do for us? The answer began with low-risk tasks: summarizing documents, drafting emails, producing first drafts that a human would edit.

It quickly moved into higher-value cognitive work, such as software development and content generation, and grew into more ambitious projects, like attempts to build an AI company brain, a system connected to internal knowledge, data, and tools that could coordinate work and eventually operate parts of the business autonomously.

While a lot of time, energy and tokens have been invested in AI adoption, measurable outcomes have barely been achieved at scale. However, some companies embraced being AI-first and saw enormous gains in productivity, revenue, and cost, while others lagged behind or failed to change their organizations enough to reach high ROI.

Recent data from corporate expense management platform Ramp reveals a stark contrast in performance: the top quartile of companies investing in AI saw their revenue more than double between November 2022 and December 2025, while businesses with zero AI expenditure experienced a mere 15% increase.

媒体内容 · 前往原文查看

Top AI adopters outperform across industries

top quartile of AI spenders businesses with zero AI spend

Same three years, same economy: in Ramp's data across its customer base, the heaviest AI adopters more than doubled revenue while businesses spending nothing on AI grew about 15% (indexed, Nov 2022 = 100; curve drawn from the reported endpoints). The rest of this article is about what the AI-heavy group actually did.

There are many reasons why AI has done wonders for some companies while others have struggled to see the return on their investment, but research primarily points in five directions.

01

Redesign the process, not just the task

Becoming AI-first means rethinking how the work is structured, not dropping a model into a workflow built around people: what gets approved, who reviews what, and which handoffs still need a human. Where the process stays untouched, legacy bottlenecks absorb the productivity gains before they reach the P&L. In McKinsey's 2025 survey of organizations using gen AI, workflow redesign was the attribute most correlated with EBIT impact, and only 21% of them had redesigned any workflow at all.

02

Incentivize experimentation

Models, tooling and best practices change weekly, so last quarter's setup is rarely still the right one. That only gets picked up if people are rewarded for trying things and reporting what failed, not just for shipping. Technical teams are the natural place to start, since they see the same problems recur across functions and can tell which of them a model can actually take over.

03

Provide tailored business context

Prompt engineering and retrieval can inject business context at call time, but doing it well is its own engineering program: getting to the data, enforcing access controls on what each request may see, building retrieval that surfaces the right evidence, and managing a context window that models use unevenly as it grows.

04

Measure usage and impact

Every AI line item eventually meets the CFO question: what did this change, and was it worth it? In most deployments, nobody can answer it: there is no infrastructure to track the model's performance, decision costs, or impact on efficiency, and self-reported time savings are often inaccurate. Without a scored evaluation on your own data, a "vibe evaluation" is the ceiling of what you can claim, and a hard budget to defend.

05

Set clear business goals within the AI budget

AI brought a pricing model most companies were not used to. Paying per token instead of per seat makes costs scale with usage, which makes it hard to lay out a cost plan or estimate the capital efficiency gains for internal workloads. Uber went through its annual engineering budget in four months, and Microsoft cancelled most of its Claude licenses to bring costs back under control. Today's prices also understate the problem, since most AI labs are subsidizing token costs to capture market share, and frontier model prices are expected to rise.

In this article we give a detailed overview of the deployment technique the winning group keeps converging on: fine-tuning open-source models with reinforcement learning. We cover how it addresses the last three challenges above, and how it turns knowledge only your organization has (namely data, tools, and processes) into a model no vendor API can match at a fraction of the cost.

TL;DR

2.2×

Revenue growth of the top quartile of AI spenders between November 2022 and December 2025 in Ramp's data. Companies with zero AI spend grew about 15% over the same three years, in the same economy: the heavy adopters grew eight times as much.

1

Playbook the winners converge on: an open-source model, proprietary task data, and reinforcement learning against a scored copy of the workflow. Bridgewater's trained model makes ~30% fewer mistakes than the best frontier model, Harvey's legal agent beats GPT-5.5 and Claude Opus 4.8 on its own rubrics, and Intercom's Fin Apex resolves more support issues at lower cost.

87.3%

Share of the maximum achievable score our GRPO-trained 9B open-source model reached on catalog review, vs 76.9% for the best frontier configuration: a 13.5% relative improvement over the frontier, and 36% over its own untrained base (64.2%). The five frontier models, even with optimized prompts, plateaued within a tenth of a point of each other; the trained specialist cleared that ceiling.

68×

Cost advantage per reviewed listing: $0.50 per 1,000 with the specialist vs $34 with the strongest frontier model, and still 40× cheaper than the least expensive frontier option. At roughly 40 million decisions a day, that is about $7M a year instead of $500M, a 98% cost reduction.

Part II

What the winners do differently

Most of the companies pulling ahead in the AI race made the same discovery: owning your intelligence wins on both performance and cost. A model trained to complete your specific workflows in your specific environment is very likely to outperform a general-purpose model that has never seen inside your company. Additionally, since you do not need to pack as many general-purpose capabilities into a model that is meant to operate in a specific environment, you can often get away with a smaller model that is orders of magnitude cheaper to run.

Owning your intelligence does not mean cancelling the ChatGPT or Claude subscription. Most workflow automation still starts with frontier models, and that is the right first move: it establishes a baseline for what is technically possible, and every call generates the data (inputs, decisions, corrections) that a specialist model later trains on. Once the automation leaves the prototyping stage, the priority flips to cost and performance at volume, and that is where fine-tuning open-source models with reinforcement learning comes in. In addition to that, your Fable 5 or ChatGPT model can call the specialist model to handle the parts of the workflow that require your internal knowledge, and the specialist model can call the frontier model for tasks that require high general ability.

媒体内容 · 前往原文查看

How a model learns to operate in your environment

The model interacts with your tools and your data, a rubric scores the outcome, and the reward signal updates the model. Repeated across tens of thousands of tasks, the workflow's judgment settles into the weights, while changing facts (prices, inventory, policy text) stay in tools where they belong.

Over the past two years this has hardened into a playbook: an open-source model, proprietary task data, and a reinforcement-learning stage against a scored version of the workflow. Below, we discuss three scenarios where this approach has been applied to real-world tasks.

Bridgewater Associates is one of the largest hedge funds in the world. Its analysts sift a constant stream of articles, filings, and emails, judging which documents are relevant to the firm's investment thesis and where boilerplate content begins. The catch is that relevant means relevant by Bridgewater's internal judgment, and no amount of prompting got frontier models to absorb that judgment reliably. So, the company decided to train an open-source model on labels from its own expert investors. The trained model makes roughly 30% fewer mistakes than the best frontier model, at a fraction of the inference cost.

Harvey builds AI agents for law firms. Its hardest workloads are long-horizon: transaction due diligence and legal memo drafting, where the agent navigates large document sets, errors compound across steps, and even the best frontier models at maximum reasoning effort kept falling short of the quality bar. Harvey ran reinforcement learning on an open-weight model and got a legal agent that outperforms both GPT-5.5 and Claude Opus 4.8 on its rubrics.

Intercom is a customer-service platform whose AI agent, Fin, resolves almost two million customer issues a week. At that volume the problem is unit economics: frontier per-call pricing adds up fast, and every point of resolution rate matters. So Intercom's AI group post-trained its own vertical support model, Fin Apex, on billions of customer-service interactions. Intercom reports that it resolves more issues than the best frontier models while being cheaper to run.

The same shape repeats well beyond these three. The appendix collects eight more deployments, with what each model was trained to do and what changed once it shipped.

One deployment pattern repeats across these cases: reinforcement learning pushes an open-source model past the frontier on a specific set of workflows, at a fixed and dramatically lower cost per call. The deployment typically starts by using prompt and context engineered frontier models to establish the strongest default baseline. Those frontier traces and learnings are then reused to pack the focused capability into a compact model the company owns.

Part III

Case Study: Catalogue Integrity Agent

One of the central challenges of running an e-commerce platform is keeping the product catalog trustworthy. Every listing must land in the correct category of the platform's taxonomy, and the attributes that power search, filters, recommendations, and downstream operations must be accurately extracted from its images and description.

+15%

our fine-tuned 9B open-source model reviews e-commerce product listings more accurately than the best frontier model we tested

68×

cheaper per listing when our fine-tuned model does the reviewing instead of that frontier model

Platforms staff this work with teams of catalog-integrity analysts, which grow together with the catalog: more listings mean more categories to know, more attributes to check, and more ambiguous edge cases to judge. Inconsistent decisions propagate: products become harder to find, recommendations deteriorate, and policy violations slip through. Miss a counterfeit and you expose customers and brands to fraud; over-flag and you build an expensive review queue that frustrates legitimate sellers.

Let's start with the scale. eBay alone carries about 2.5 billion live listings, and Shopify's catalog absorbs more than 10 million product updates a day. Walmart has said that doing its AI-assisted catalog work with people alone would have taken roughly 100× the headcount.

Getting it wrong costs revenue: 71% of shoppers say they have returned a product because it didn't match its listing. But the reviewing itself is also expensive. A mid-size marketplace generates on the order of 10 million listing creates and edits a day; reviewed with a frontier model, that workload costs roughly half a billion dollars a year, while a fine-tuned specialist does the same job for around $10M:

媒体内容 · 前往原文查看

One workload, two price tags

frontier API per call fine-tuned specialist, self-hosted

The ~50× gap between the two bars is the entire value proposition made concrete: at ingestion scale, the frontier-per-call architecture does not merely cost more, it fails the budget outright. This is why the companies running this exact workflow do not run it on rented models.

The companies actually running this workflow confirm the math. Shopify classifies products with fine-tuned open models at roughly 40 million inferences a day, a volume it says commercial APIs can't economically serve. Inspired by the industry leaders, we ran the playbook ourselves: an agent that examines a listing, searches the product taxonomy, checks the brand, retrieves the attribute schema, and commits a structured decision, escalating to human review when the evidence is thin or the risk too high.

媒体内容 · 前往原文查看

One episode, end to end

ListingWork gloves, PU-coated, black, size 10

Claimed brandAmazonBasics

RegionUS

1search_taxonomy → Safety Work Gloves

2lookup_brand → registered, not protected

3get_attribute_schema → brand, color, material, size

✓Commits category, attributes, and verdict allowed

The agent works through the listing the way an analyst would: it finds the right category, verifies the claimed brand, pulls the attribute schema for that category, and commits its decision. A clean listing ends in an approval; a suspicious one must end in a flag. The model learns to weigh the two mistakes asymmetrically, because missing a real violation costs it 7× more than raising a false alarm.

Part IV

A digital twin the model can practice in

A model can only practice a workflow it can actually perform, fail at, and retry. So we rebuilt the catalog workflow as a digital twin of an e-commerce platform: a simulated environment with the same listings, the same tools, and the same stakes as the real thing. The raw material is the Amazon Berkeley Objects dataset of real product images and listing data, which we turned into 177,767 review episodes: one listing each, with image, title, description, claimed brand, and region. Into that stream we planted controlled policy cases, mismatched images, conflicting brand claims, and deliberately legitimate claims as hard negatives, so every episode has a known correct answer to score against.

Inside the twin, the model works the way an analyst would. It can search a taxonomy of roughly 13,000 categories, check whether a brand is registered and protected, and retrieve the required attributes for a chosen category; then it must commit a category, the attributes, and a policy decision. A scorer grades every episode, rewarding correct outputs and penalizing missed violations, unsupported attributes, invalid categories, and wasted tool calls. The penalties encode business priorities directly: a missed violation costs 7× more than a false alarm.

媒体内容 · 前往原文查看

The digital twin, end to end

Every part of the real workflow has a counterpart in the twin: listings flow in, the agent consults the same tools an analyst would, and a scorer grades each decision. The reward flowing back is what turns practice into learning.

Benchmarking the frontier models

Before any training run, we measured how far the frontier could get on its own. We benchmarked five frontier models (GPT-5.5, GPT-5.6-sol, Gemini 3.1 Pro, Claude Opus 4.8, and Claude Fable 5) on 200 stratified validation episodes with identical tools, images, scorer, and turn budget, both with a plain prompt and with optimized prompt instructions: 2,800 characters of extraction conventions, lookup procedure, and worked examples, tuned the way a prompt engineer would tune a production system. The chart below shows both configurations, with our final GRPO-trained model added for comparison.

The best frontier configuration reached 76.9% of the achievable score; the trained 9B reached 87.3%. The gap is not intelligence. A frontier model starts every episode from zero: it has never seen this store's taxonomy, doesn't know its inventory conventions, which attribute values count as supported, or how the platform wants corner cases resolved, and it has to reconstruct all of that on the fly, from whatever fits in the prompt, on every single call. Optimized instructions can compress some of that knowledge, but the corner cases that decide the score are exactly the ones no instruction prompt can enumerate.

媒体内容 · 前往原文查看

Model benchmarks

Each frontier bar shows the model's zero-shot score in solid teal; the lighter extension is what optimized prompt instructions added. Optimization helped, but only so much: the optimized configurations converged within a tenth of a point of each other, and the strongest zero-shot extractor (Gemini) got worse with the optimized instructions, so its bar shows no gain. The extra instructions weren't free either: they inflated input-token bills by 28–55% depending on the model, on every call, forever. Prompted task knowledge is rented per call; trained task knowledge is bought once and lives in the weights.

Part V

Training details

The training setup was modest. We rented two RTX PRO 6000 GPUs, one generating rollouts and one applying gradient updates, with the open-source prime-rl framework running the infrastructure. The full run was 1,000 optimizer steps, took about three and a half days, and cost roughly $500 in GPU time.

Most of that was not needed to catch the frontier. The model crossed the frontier band after roughly 250 steps, about a day of training; the remaining steps were spent squeezing out maximum performance. The final model scores 0.626 on the benchmark, 87.3% of the achievable ceiling and about ten points above the best frontier configuration.

The gain comes from teaching the model the semantics of its environment: over thousands of scored episodes it learns how the store's taxonomy, tools, and policies relate to the reward, and converges on the optimal distribution of actions over the environment's action space. A general model spreads its capacity across everything; the specialist spends all of it on the specific task at the expense of general ability.

The result also isn't a dead end. When a more capable open-source base model ships, the recipe transfers: as a rule of thumb, a stronger base yields a stronger post-trained specialist. And the deployed model generates its own training data as it works: logged decisions can be distilled into supervised fine-tuning sets, making each re-training cheaper and better-informed than the last.

媒体内容 · 前往原文查看

Training run

Every point is a measured evaluation from the W&B training monitor (two sampled rollouts per episode). The 9B run climbs from roughly 0.50 to 0.671 at step 1,000, clearing the frontier band before step 250. The monitor's sampled decoding runs slightly above the strict benchmark harness, where the final 9B adapter scores 0.626.

So far, we have discussed how reinforcement learning helps a language model navigate your company environment more accurately. But one of the most important aspects of this paradigm is that the accuracy also comes cheaper than running frontier models. The cost-vs-quality chart that opens this article puts both on one picture, task score against cost per thousand listings, and the takeaway for a budget owner is simple: with everything else on the chart you are choosing between quality and cost. The trained specialist ends that trade-off, delivering the best score we measured at close to the lowest price.

Fine-tuning adds ~23 points over the base 9B at the same ~$0.50 per 1,000 listings: 40× cheaper than the least expensive frontier configuration (Gemini, $19/1k) and ~340× cheaper than the most expensive (GPT-5.5-pro, $172/1k). The 2,800 characters of prompt instructions also raised GPT-5.5's measured cost by a third, a prompt tax paid on every future call. The specialist's instructions live in its weights. At Shopify-scale volume of roughly 40 million decisions a day, the gap between $34 and $0.50 per thousand is about $500M a year versus $7M.

Wondering where these curves sit for your workflow? Book a 30-minute audit →

Part VI

What can owning AI do for you?

Here is the whole experiment in three sentences. We took one high-volume e-commerce workflow, reviewing product listings, and rebuilt it as a digital twin. We let a small open-source model practice inside it for a few days and roughly $500 of GPU time. The trained specialist ended above every frontier configuration we measured, at a fraction of their price per decision.

The recipe transfers to any work your business repeats at volume. Look for the places where people or API calls turn information into decisions all day: routing tickets, extracting fields from documents, checking submissions against policy, classifying products, approving or flagging transactions. What unites them is that every decision can be checked: there is a rule, a schema, or an expert who can say whether it was right. That check is the whole trick. If a decision can be scored, a model can practice it; if it can only be debated, it cannot.

It is just as important to know where this is the wrong tool. Two questions settle most cases: how frequently the task runs, and whether its outcome is verifiable.

媒体内容 · 前往原文查看

Right tool for the right-shaped problem

The upper right is where fine-tuning pays: frequent work with verifiable outcomes. Everything else is better rented: a prompt-optimized frontier model covers verifiable but rare work, and a human stays in the loop where outcomes can't be verified. When the problem is changing facts rather than judgment, retrieval is the fix at any volume. The checklist below condenses the upper-right conditions into something you can hold your own workflows against:

Your workflow is a fine-tuning candidate if at least one point applies…

It happens at high volume, often enough that per-decision cost and errors compound into real money

Every outcome can be checked by a rule, test, or rubric, with no person in the loop

Your experts agree on what a correct answer looks like

A capable model already succeeds sometimes, just not reliably enough

The right answer can't be a lucky guess

It spans several steps: reasoning, tool calls, then a committed decision

It runs on your own tools, schemas, and policies

Different mistakes carry different costs

Sensitive data cannot leave infrastructure you control

That last one is also the safety argument: a model you train runs inside your own boundary, so prompts and records never reach a vendor. One point is enough to be worth a conversation, and working out which ones apply is what the call is for.

If that describes one of your workflows, the question is no longer whether it's the right fit for you, but which of your decisions you should stop renting. Owning your intelligence is the kind of advantage that compounds: the model, the evaluation, and the data stay yours and improve together, on your terms. If you would like to see what that looks like for your business, we would be glad to work through it with you:

See these numbers on your workflow

Sounds like something that would help your business? Book a call and we'll help you evaluate the potential business impact and determine the business ROI, free of charge.

Book a free expert call

Want to try the models yourself? They're on Hugging Face.

Appendix

The track record: task-trained specialists vs. prompted frontier models

Beyond the flagships, the same shape repeats across industries. Every result below is the company's own reported figure against the frontier model it was replacing or competing with; each company name links to the write-up it came from.

CompanyWhat the model doesResultBusiness payoff

Cognition Writes and fixes software in production Beats GPT-5.5 on a standard coding benchmark Frontier-level output at a fraction of the serving cost, streaming 1,000 tokens per second

AT&T Summarizes 900k support calls a day and flags personal data 17% better than GPT-4o at catching personal data Reviews fraud cases 12× faster at GPT-4o accuracy; saves millions of dollars annually

LinkedIn Matches candidates to job openings 4% more accurate than the GPT model it replaced 75× cheaper than GPT-4, 6× cheaper than GPT-4o

Ambience Healthcare Assigns medical billing codes More accurate than 18 board-certified physicians on a gold-panel test 12 points ahead of prompted o4-mini, the OpenAI model it was built from

Phonely Answers customer phone calls 99.2% accuracy vs GPT-4o's 94.7% Responses 73% faster than its previous GPT-4o setup; one customer replaced 350 human agents in a month

OpenPipe Handles email and support tickets 93% on support QA where OpenAI's o3 scored 50% 64× cheaper and 5× faster to run than o3

Perplexity Answers search queries with sources Matches GPT-4o in blind user tests 10× the speed of GPT-4o at a fraction of the price

Checkr Classifies criminal-record entries in background checks Beats GPT-4 on the hardest cases 5× cheaper and 30× faster than its previous GPT-4 setup

Sources & notes