Tatoxa 系统:面向低资源语言鞑靼语的文本去毒化
阅读原文· arxiv.orgTatoxa 是一个针对鞑靼语(Tatar)文本去毒化的最新系统,能够自动检测并缓解攻击性和有害内容。对比实验表明,该方案在关键质量指标上超越了现有开源和闭源商用大语言模型。研究同时引入了一个专为低资源场景下微调和评估设计的鞑靼语文本去毒化数据集。跨语言迁移实验显示,即使使用大规模俄语语料,从其他语言(包括文化相近的俄语)迁移的效果也显著差于在本地鞑靼语数据上训练。
Text detoxification, the automated detection and mitigation of abusive and harmful content, is essential for ensuring the safety of online communities and protecting users. However, low resource languages such as Tatar have received little research attention. In this paper we present Tatoxa, a novel state-of-the-art system for text detoxification in the Tatar language. Comparative experiments show that the proposed approach outperforms existing open source and proprietary commercial LLMs on key quality metrics. We also introduce a new dataset for text detoxification in Tatar, designed for fine tuning and evaluation in low resource settings. Finally, cross lingual transfer experiments indicate that transfer from other languages, including the culturally close Russian, performs significantly worse than training on native Tatar data even when a large Russian corpus is available.