BrainG3N:面向可控3D脑MRI生成的双用途tokenizer
阅读原文· arxiv.org提出一种基于3D体素掩码自编码器(MAE)的tokenizer,用于3D脑MRI潜在扩散模型。编码器与解码器解耦:冻结的3D MAE编码器产生临床信息丰富的嵌入,专用CNN解码器从嵌入的线性投影重建体素。编码器在35,309个体积(来自18个公共队列,覆盖四种模态、十种疾病类别和200+采集站点)上预训练。在23任务线性探测基准上,编码器在21个任务上超越或匹配BrainIAC、BrainSegFounder、MedicalNet等SOTA模型。基于这些嵌入训练的扩散Transformer(DiT)支持跨六个变量的条件生成和患者特定纵向预测。
Three-dimensional (3D) brain MRI is central to clinical neurology and neuro-oncology, where generative models could augment under-represented cohorts, simulate disease trajectories, and support privacy-preserving data sharing. Latent diffusion has been the go-to solution for modeling imaging data, but it places two competing demands on the tokenizer: encoder embeddings must retain the clinical information that downstream tasks act on, and the decoder must reconstruct anatomically faithful volumes. Existing reconstruction-driven tokenizers achieve the second at the expense of the first. To address this, we introduce a fully volumetric masked-autoencoder (MAE) based tokenizer for 3D brain MRI latent diffusion, decoupling encoder and decoder: a frozen 3D MAE encoder produces clinically informative embeddings, while a dedicated CNN decoder reconstructs voxels from a linear projection of those embeddings. We pretrain the encoder on 35,309 volumes from 18 public cohorts spanning four modalities, ten disease categories, and 200+ acquisition sites, and demonstrate its dual utility in two settings. First, on a 23-task linear-probing benchmark, the encoder outperforms or matches SOTA models (i.e., BrainIAC, BrainSegFounder, and MedicalNet) on 21 of 23 tasks. Second, a conditional diffusion transformer (DiT) trained on these clinically informative embeddings supports both conditional generation across six variables and patient-specific longitudinal forecasting. Together these results establish a single 3D brain-MRI embedding space capable of both downstream clinical tasks and controllable generation.