UniverSat: 分辨率和模态无关的Transformer用于地球观测
阅读原文· arxiv.orgUniverSat是一种基于Vision Transformer的骨干网络,采用通用补丁编码器(Universal Patch Encoder),将来自任意空间、光谱和时间分辨率以及光学和非光学传感器的补丁映射到共享嵌入空间,使用共享权重。这使得单个模型能够在异构多模态数据集上通过自监督训练,生成鲁棒的传感器无关空间特征。在GeoBench、PANGEABench和SpectralEarth等标准地球观测基准的分类和分割任务中,取得了强劲结果。代码和模型已开源。
Vision Transformers (ViT) dominate computer vision. However, their reliance on rigid patch projectors hinders transfer to Earth Observation (EO), where input modalities, scales, and resolutions vary widely. We introduce UniverSat, a ViT-style backbone built around a Universal Patch Encoder that maps patches from arbitrary spatial, spectral, and temporal resolutions, and from both optical and non-optical sensors, into a shared embedding space with a shared set of weights. This enables training a single model on heterogeneous multimodal corpora via self-supervision, yielding robust, sensor-agnostic spatial features. We validate this approach with strong results across classification and segmentation on standard EO benchmarks from GeoBench, PANGEABench, and SpectralEarth. Our code and models are available at https://github.com/gastruc/UniverSat.