返回
原文
Apple Machine Learning Research（RSS）
47AI 编辑部评分，满分 100

利用 Sign Language Models 自举标注手语注释

2026-04-30 08:00· 97天前
AI 导读

研究团队开发了一套手语伪标注流程，以解决高质量标注数据匮乏对AI手语翻译发展的限制。该流程以手语视频和英文文本为输入，输出包括时间区间在内的可能注释排序集合，涵盖手势词、手指拼写单词和手语分类器。新发布的ASL STEM Wiki和FLEURS-ASL等数据集虽包含数百小时专业译员数据，但因标注成本过高仅实现部分标注。该自举方法旨在显著提升大规模手语数据的利用效率。

原文 · 未翻译

AI-driven sign language interpretation is limited by a lack of high-quality annotated data. New datasets including ASL STEM Wiki and FLEURS-ASL contain professional interpreters and 100s of hours of data but remain only partially annotated and thus underutilized, in part due to the prohibitive costs of annotating at this scale. In this work, we develop a pseudo-annotation pipeline that takes signed video and English as input and outputs a ranked set of likely annotations, including time intervals, for glosses, fingerspelled words, and sign classifiers. Our pipeline uses sparse predictions from our fingerspelling recognizer and isolated sign recognizer (ISR), along with a K-Shot LLM approach, to estimate these annotations. In service of this pipeline, we establish simple yet effective baseline fingerspelling and ISR models, achieving state-of-the-art on FSBoard (6.7% CER) and on ASL Citizen datasets (74% top-1 accuracy). To validate and provide a gold-standard benchmark, a professional interpreter annotated nearly 500 videos from ASL STEM Wiki with sequence-level gloss labels containing glosses, classifiers, and fingerspelling signs. These human annotations and over 300 hours of pseudo-annotations are being released in supplemental material.

  • † Gallaudet University
  • ** Work done while at Apple

Related readings and updates.

Understanding Annotator Safety Policy with Interpretability

Safety policies define what constitutes safe and unsafe AI outputs, guiding data annotation and model development. However, annotation disagreement is pervasive and can stem from multiple sources such as operational failures (annotators misunderstand or misexecute the task), policy ambiguity (policy wording leaves room for interpretation), or value pluralism (different annotators hold different perspectives on safety). Distinguishing these…

Towards AI-Driven Sign Language Generation with Non-Manual Markers

Sign languages are essential for the Deaf and Hard-of-Hearing (DHH) community. Sign language generation systems have the potential to support communication by translating from written languages, such as English, into signed videos. However, current systems often fail to meet user needs due to poor translation of grammatical structures, the absence of facial cues and body language, and insufficient visual and motion fidelity. We address these…

Bottom banner
多模态数据/训练论文/研究

来源：Apple Machine Learning Research（RSS） · machinelearning.apple.com

导出 Markdown
返回
47打开原文
导出 Markdown

利用 Sign Language Models 自举标注手语注释

Apple Machine Learning Research（RSS）·2026-04-30 08:00·97天前
AI 导读

研究团队开发了一套手语伪标注流程，以解决高质量标注数据匮乏对AI手语翻译发展的限制。该流程以手语视频和英文文本为输入，输出包括时间区间在内的可能注释排序集合，涵盖手势词、手指拼写单词和手语分类器。新发布的ASL STEM Wiki和FLEURS-ASL等数据集虽包含数百小时专业译员数据，但因标注成本过高仅实现部分标注。该自举方法旨在显著提升大规模手语数据的利用效率。

原文 · 保持原样，未翻译

AI-driven sign language interpretation is limited by a lack of high-quality annotated data. New datasets including ASL STEM Wiki and FLEURS-ASL contain professional interpreters and 100s of hours of data but remain only partially annotated and thus underutilized, in part due to the prohibitive costs of annotating at this scale. In this work, we develop a pseudo-annotation pipeline that takes signed video and English as input and outputs a ranked set of likely annotations, including time intervals, for glosses, fingerspelled words, and sign classifiers. Our pipeline uses sparse predictions from our fingerspelling recognizer and isolated sign recognizer (ISR), along with a K-Shot LLM approach, to estimate these annotations. In service of this pipeline, we establish simple yet effective baseline fingerspelling and ISR models, achieving state-of-the-art on FSBoard (6.7% CER) and on ASL Citizen datasets (74% top-1 accuracy). To validate and provide a gold-standard benchmark, a professional interpreter annotated nearly 500 videos from ASL STEM Wiki with sequence-level gloss labels containing glosses, classifiers, and fingerspelling signs. These human annotations and over 300 hours of pseudo-annotations are being released in supplemental material.

  • † Gallaudet University
  • ** Work done while at Apple

Related readings and updates.

Understanding Annotator Safety Policy with Interpretability

Safety policies define what constitutes safe and unsafe AI outputs, guiding data annotation and model development. However, annotation disagreement is pervasive and can stem from multiple sources such as operational failures (annotators misunderstand or misexecute the task), policy ambiguity (policy wording leaves room for interpretation), or value pluralism (different annotators hold different perspectives on safety). Distinguishing these…

Towards AI-Driven Sign Language Generation with Non-Manual Markers

Sign languages are essential for the Deaf and Hard-of-Hearing (DHH) community. Sign language generation systems have the potential to support communication by translating from written languages, such as English, into signed videos. However, current systems often fail to meet user needs due to poor translation of grammatical structures, the absence of facial cues and body language, and insufficient visual and motion fidelity. We address these…

Bottom banner

来源：Apple Machine Learning Research（RSS）· machinelearning.apple.com