大语言模型的多领域微调需要在提升目标领域性能的同时，保持其在受限领域（如通用知识、指令遵循或安全评估）上的表现。现有的数据混合策略依赖固定启发式规则或自适应规则，无法明确强制保留这些能力。我们提出 DynaMiCS，一种动态混合优化器，将多领域微调建模为约束优化问题。在每次更新时，DynaMiCS 执行简短的领域特定探测运行，以估计局部跨领域效应的斜率矩阵，捕捉在每个微调数据集上训练如何影响每个评估领域。随后，这些估计值被用于在概率单纯形上通过优化计算混合权重，其目标是在提升目标领域性能的同时，将受限领域的损失维持在参考水平以下。在具有不同数量目标领域和受限领域的多领域微调场景中，DynaMiCS 相比固定混合基线，以更低的计算成本实现了更强的目标领域性能提升和更高的约束满足度，且无需参考模型、逐样本评分或手动调整混合权重。
- † 苏黎世联邦理工学院
- ** 在苹果公司期间完成的工作
相关阅读与更新。
微调过程中遗忘现象的缩放定律：基于预训练数据注入
获得在目标领域表现良好的语言模型的一种广泛策略是，通过在该领域的数据上进行无监督下一个 token 预测训练来对其进行微调。微调面临两个挑战：一，如果目标数据量有限（大多数实际应用场景正是如此），模型会迅速过拟合；二，模型会偏离原始模型，遗忘预训练……
利用大语言模型进行零样本语音识别领域自适应的语料合成
尽管自动语音识别（ASR）系统已在许多实际应用中得到广泛使用，但它们通常难以很好地泛化到新领域，需要在这些领域的数据上进行微调。然而，在许多场景下，目标领域的数据通常不易获取。在本文中，我们提出了一种新策略，用于在没有任何目标领域文本或语音的情况下，将ASR模型适配到新的目标领域。为实现这一目标，我们提出了一种……
探索机器学习领域的机遇。
Multi-domain fine-tuning of large language models requires improving performance on target domains while preserving performance on constrained domains, such as general knowledge, instruction following, or safety evaluations. Existing data mixing strategies rely on fixed heuristics or adaptive rules that cannot explicitly enforce preservation of such capabilities. We propose DynaMiCS, a dynamic mixture optimizer that casts multi-domain fine-tuning as a constrained optimization problem. At each update, DynaMiCS performs short domain-specific probing runs to estimate a slope matrix of local cross-domain effects, capturing how training on each fine-tuning dataset affects each evaluation domain. These estimates are then used to compute mixture weights through optimization over the probability simplex, with the objective of improving target-domain performance while keeping constrained-domain losses below reference levels. Across multi-domain fine-tuning scenarios with varying numbers of target and constrained domains, DynaMiCS achieves stronger target-domain improvements and higher constraint satisfaction than fixed-mixture baselines, at lower computational cost and without reference models, per-example scoring, or manually tuned mixture weights.
- † ETH Zurich
- ** Work done while at Apple
Related readings and updates.
Scaling Laws for Forgetting During Finetuning with Pretraining Data Injection
A widespread strategy for obtaining a language model that performs well in a target domain is to fine-tune it by training it to do unsupervised next-token prediction on data from that domain. Fine-tuning presents two challenges: i) if the amount of target data is limited, as is the case in most practical applications, the model will quickly overfit, and ii) the model will drift away from the original model and forget the pre-training…
Corpus Synthesis for Zero-shot ASR Domain Adaptation using Large Language Models
While Automatic Speech Recognition (ASR) systems are widely used in many real-world applications, they often do not generalize well to new domains and need to be finetuned on data from these domains. However, target-domain data is usually not readily available in many scenarios. In this paper, we propose a new strategy for adapting ASR models to new target domains without any text or speech from those domains. To accomplish this, we propose a…