IndustryBench-MIPU:面向工业产品的多图像属性提取基准
阅读原文· arxiv.orgIndustryBench-MIPU是首个大规模多图像工业产品理解基准,聚焦结构化属性提取——从产品图像中恢复属性-值对。基准涵盖18个工业类别、4,559个产品、27,652张图像和103,703条标注,通过多模型共识与三级质量审核构建。在9个多模态大语言模型上的评估显示:单图像属性提取精度达86–94%,但产品级多图像召回最高仅49.9%;从单图像转向多图像提取时,召回率下降15–34个百分点。多图像完整性是核心瓶颈,而非单图像准确率。数据集与代码已公开。
Industrial products such as valves and circuit breakers are defined by dense technical specifications that govern procurement, compatibility, and safety across supply chains. These specifications are scattered across multiple heterogeneous product images, including specification tables, nameplates, and technical drawings, yet whether Multimodal Large Language Models (MLLMs) can reliably recover them remains underexplored. To fill this gap, we introduce IndustryBench-MIPU, the first large-scale benchmark for multi-image industrial product understanding, built around structured attribute extraction -- recovering property-value pairs from product images. This task jointly probes text recognition on specification tables and nameplates, visual reasoning over technical drawings, domain knowledge to decode industrial terminology, and cross-image evidence integration to assemble scattered specifications. Concretely, the benchmark comprises 4,559 products across 27,652 images with 103,703 annotations spanning 18 industrial categories, constructed through multi-model consensus and three-tier quality assurance. Evaluating nine MLLMs under both single-image and product-level multi-image settings reveals a stark completeness gap: models achieve high precision (86--94%) but the best recovers only 49.9% of product-level attributes; moving from single-image to multi-image extraction costs 15--34 percentage points of recall. Multi-image completeness, not single-image accuracy, is the core bottleneck. Dataset and code are publicly available.