MechVQA:全面机械图纸理解的多模态大语言模型基准与增强
阅读原文· arxiv.orgMechVQA是一个面向机械工程图纸理解的全面数据集,通过半自动构建与质量控制流程生成,包含3.3k高密度图片和21K问答对,覆盖识别、推理、判断三个能力层级的10种细粒度任务。基于该数据集,研究团队开发了MechVL模型,采用多阶段训练范式,在MechVQA总分上超越最强闭源基线7.57个百分点,显著提升机械图纸理解能力,为多模态大语言模型在机械设计与检测场景中的应用提供了可复用基础。
Multimodal Large Language Models (MLLMs) have demonstrated significant achievements in general visual question answering (VQA) tasks. However, they remain brittle on mechanical engineering drawings, where high annotation density and weak domain knowledge, compounded by unreliable spatial relation reasoning under strict projection rules and geometric constraints, make decisive cues easy to miss and frequently lead to wrong answers. To bridge this gap, we introduce the first comprehensive mechanical drawing understanding dataset, MechVQA, created through a semi-automated construction and quality-control pipeline. MechVQA contains 3.3k high-density pictures with 21K question-answer pairs, spanning 10 different fine-grained tasks across three capability levels: Recognition, Reasoning, and Judging, providing a testbed to evaluate and improve MLLM understanding on real-world mechanical drawings. On top of MechVQA, we then develop the MechVL model through a multi-stage training paradigm, building a strong domain-specialized baseline. Extensive experimental results demonstrate that MechVL outperforms the strongest closed-source baseline by 7.57 percentage points on the MechVQA total score, significantly enhancing mechanical drawing understanding ability and providing a reusable foundation for deploying MLLMs in mechanical design and inspection scenarios.