微软 SocialRL 让 4B 模型谈判胜过 GPT-4.1

Rohan Paul · @rohanpaul_ai · X·2026-08-23 23:27·2天前
AI 导读

微软新论文发现,AI 智能体因被训练得过于顺从,替用户谈判时通常会吃亏,甚至会泄露预算并在对方施压时让步。为此提出 SocialRL 训练方法,让模型在六个谈判与调度游戏中以交易结果为导向学习。一个 4B 模型在全部游戏中平均得分 0.627,追平 GPT-4.1 的 0.625,但仅靠提示词反而会恶化表现。

Rohan Paul@rohanpaul_ai
36AI 编辑部评分,满分 100

微软 SocialRL 让 4B 模型谈判胜过 GPT-4.1

2026-08-23 23:27· 2天前
AI 导读

微软新论文发现,AI 智能体因被训练得过于顺从,替用户谈判时通常会吃亏,甚至会泄露预算并在对方施压时让步。为此提出 SocialRL 训练方法,让模型在六个谈判与调度游戏中以交易结果为导向学习。一个 4B 模型在全部游戏中平均得分 0.627,追平 GPT-4.1 的 0.625,但仅靠提示词反而会恶化表现。

New Microsoft paper shows an AI agent that negotiates for you will usually lose, because it was trained to be agreeable.

Politeness, transparency and eagerness to close are great in a chat assistant and terrible in a delegate. The paper found frontier models leaking their user's budget and folding the moment a seller pushed back.

Their fix is SocialRL: instead of prompting the model to negotiate better, train it on the outcome of the deal across six bargaining and scheduling games.

It works, and it doesn't take a big model. A 4B model started anchoring low, holding its position and walking away from bad deals, and landed at 0.627 average across all six games, matching GPT-4.1 at 0.625.

The catch is that prompting alone made things worse, so this is a training fix, not a prompt fix.

So if you're building an agent that acts on someone's behalf, stop scoring it on whether the deal closed and start scoring it on what it gave away.

– arxiv. org/abs/2608.13787

Title: "From Passive Delegates to Strategic Negotiators: Reinforcing Social Reasoning in Small Language Models with SocialRL"