🚀 A new way to refine LLM training data is here！
Scaling LLMs is no longer just about collecting more data - it's about making every token count.
Introducing UltraX， our framework for large-scale pre-training data refinement. Instead of rewriting data end-to-end， UltraX uses structured editing functions to precisely improve training data quality.
What's inside？ 👇 ✅ Function-calling refinement Fine-grained editing with insertion， deletion， and modification.
✅ Reliable data transformation LAM + DCR pipelines generate high-quality supervision for scalable refinement.
✅ Better data efficiency Achieves the best average performance across five corpora in 1B-model pre-training experiments.
No gatekeeping - explore how better data can help build better models.
📄Paper： http://arxiv.org/abs/2607.08646 💻 GitHub： http://github.com/openbmb/UltraX 🤗 Hugging Face： http://huggingface.co/datasets/openbmb/UltraX-Preview Modelscope： http://modelscope.cn/datasets/OpenBMB/UltraX-Preview http://modelscope.cn/models/OpenBMB/UltraX-0.6B-Preview