# SkillSmith：将文本与权重组合为新技能

- 来源：Rohan Paul (@rohanpaul_ai)
- 发布时间：2026-08-03 04:38
- AIHOT 分数：35
- AIHOT 链接：https://aihot.virxact.com/items/cmscacbvt026mroeutmzco995
- 原文链接：https://x.com/rohanpaul_ai/status/2084016126095466982

## AI 摘要

Google DeepMind 新论文提出 SkillSmith，将前缀键值缓存视为一种输入模态，在推理时通过前向传播为冻结的 Gemma 3 4B 模型生成新前缀缓存，无需针对新任务重新训练。

## 正文

New Google DeepMind Paper.

Model weights do not have to remain static artifacts that can only be fine-tuned or averaged.

A new capability can now be initialized from both what an agent wrote down and what it previously learned into weights.

SkillSmith treats prefix key-value caches as another input modality.

For a new task， it interleaves existing prefix weights with task descriptions， examples， and a rationale explaining how the source capabilities relate to the target.

A forward pass then produces a fresh prefix cache for a frozen Gemma 3 4B model， so the initial composition happens at inference time rather than through a new target-specific training run.

The cleanest evidence is the ablation： on Composite-SNI， Elo is 1，455 with only K-V caches， 1，622 with only text， and 1，714 with both.

The gain cannot be explained by richer prompting or a learned weight merger alone.

The generated prefixes also give downstream fine-tuning a stronger starting point， especially on difficult， data-sparse MMLU-ProX tasks.

The boundary matters： on simpler SNI tasks with roughly 1，000 examples each， fine-tuned methods converge.

SkillSmith does not remove training， but it changes where adaptation starts： previous text and previous weights can now be composed into the initialization for a new capability.

- arxiv. org/abs/2607.27497

Title： "SkillSmith： Learning to Compose Parametric Skills and Textual Knowledge"