DramaChain Bench 发布:覆盖短剧生成全流程的端到端基准

HuggingFace Daily Papers(社区热门论文)·2026-09-01 08:00·2天前
AI 导读

DramaChain Bench 是首个评估短剧生产完整链路(剧本、分镜、关键帧、镜头视频到成片)各阶段的基准,包含五条评估轴、63 个叶子维度。基准由 5785 个条目经三名专业标注者独立打分,产生 17,488 个有效评分和 255,925 条可追溯归因记录;人工标注证实上游缺陷会跨阶段级联,最终成片质量并非只由视频生成决定。

HuggingFace Daily Papers(社区热门论文)
39AI 编辑部评分,满分 100

DramaChain Bench 发布:覆盖短剧生成全流程的端到端基准

2026-09-01 08:00· 2天前
AI 导读

DramaChain Bench 是首个评估短剧生产完整链路(剧本、分镜、关键帧、镜头视频到成片)各阶段的基准,包含五条评估轴、63 个叶子维度。基准由 5785 个条目经三名专业标注者独立打分,产生 17,488 个有效评分和 255,925 条可追溯归因记录;人工标注证实上游缺陷会跨阶段级联,最终成片质量并非只由视频生成决定。

Commercial short-drama production follows a multi-stage chain: script, storyboard, keyframe imagery, shot-level video, and the finished short drama. Most existing benchmarks evaluate solely the video-generation stage using pre-authored inputs instead of real upstream pipeline outputs. This leaves two critical questions unanswerable: whether each stage adheres to the original script intent (rather than only its immediate input prompt), and whether disparate shots remain coherent after assembly into multi-episode releases. We present DramaChain Bench, the first short-drama benchmark that evaluates every stage of the complete production chain. It is built upon three in-house systems sharing one dimension system, DramaChain Dimensions: five evaluation axes instantiated at every stage, resolving into 63 leaf dimensions. DramaChain Agent is calibrated against commercial short-drama platforms in both workflow and finished short-drama quality, enabling stage-wise fair comparison across models. DramaChain Labeling System has each of the 5,785 items scored independently by three professional annotators, with all defects spatio-temporally localised and selected from a predefined defect list. This process produces 17,488 valid scores and 255,925 traceable attribution records. The human annotations confirm that upstream defects cascade across the pipeline, demonstrating that final episode quality is not governed by video generation alone. DramaChain Agentic Judge then scores every leaf dimension automatically, gathering evidence over multiple agentic rounds before judging against a per-item checklist; it reproduces the model ranking at a mean PLCC of 0.918, enough to admit new models at no annotation cost.

来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org