# 在LLM个性化中重新以人为中心

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-06-04 08:00
- AIHOT 分数：51
- AIHOT 链接：https://aihot.virxact.com/items/cmqk3awek02leslhiqvkyqg1p
- 原文链接：https://arxiv.org/abs/2606.06614

## AI 摘要

研究收集550段人类对话，在三阶段获取人工判断：提取属性5949项、配对属性11919项、生成个性化回复1101项。发现LLM从真实对话中提取属性及配对时均与人类判断分歧，生成回复人类评价不比通用回复好，但LLM自评更高。两种轻量训练干预使前两阶段自动评估更接近人类数据，但第三阶段奖励模型与人类评分仅中等相关。数据集为研究模型提取、选择、融入用户信息提供基础。

## 正文

Despite growing interest, most evaluations of large language models' (LLMs') personalization abilities have relied on synthetic data. It remains unclear how well current personalization systems work for real users. In this paper, we study the gap in LLM personalization performance when using synthetic versus human data. We collect human conversations (550 conversations) and judgments across three stages of personalization: extracting user attributes from conversations (5,949 judgments), pairing relevant attributes with new prompts (11,919), and incorporating relevant attributes into a personalized response (1,101). Incorporating human data reveals system limitations at each stage. Models struggle to extract attributes from human conversations, disagree with human judgments on relevant attributes, and generate personalized responses that humans judge no better than generic responses (though that LLM judges widely rate as better). We introduce two lightweight training-based interventions that shift automated personalization evaluation closer to human data in our first two stages. However, in our third stage we find that learned reward models achieve only modest correlation with human ratings, suggesting that human-aligned personalization quality judgments are difficult to model directly. Our collected data provides a foundation for studying how models should extract, select, and incorporate user information in ways that humans find useful.