蚂蚁集团旗下的具身智能公司 Robbyant 已开源 LingBot-Vision，这是一系列专为密集空间感知构建的自监督视觉 Transformer 模型。权重以 Apache-2.0 协议在 Hugging Face 上发布，提供四种尺寸——ViT-giant、ViT-large、ViT-base 和 ViT-small——同时附带技术报告和推理代码。
大多数视觉基础模型都针对语义不变性进行训练：它们学习回答图像中有什么，同时丢弃了机器人及其他物理具身系统所依赖的精细空间结构——物体边界、轮廓、深度不连续性。LingBot-Vision 颠倒了这一优先级。它将边界视为原生预训练信号而非下游输出，其成果是一个 10 亿参数的主干网络，在密集空间任务上能够匹配甚至超越规模大至 7 倍的模型，包括 70 亿参数的 DINOv3。
什么是 LingBot-Vision？
LingBot-Vision 是一个面向空间结构下游任务的自监督预训练编码器。旗舰模型 ViT-g/16 拥有约 11 亿参数，采用一种名为掩码边界建模的新目标函数进行训练，训练数据来自一个经过精心筛选的约 1.61 亿张图像语料库——从 20 亿规模的网络数据池中选出——无需人工标注、无需外部边缘检测器、也无需借助任何预训练主干网络进行初始化。训练过程也相当经济：该语料库比 DINOv3 的 LVD-1689M 数据集小一个数量级，且模型消耗的训练样本不到 DINOv3 的三分之一。
该编码器输出密集的 patch token 特征，用于冻结的读取器。在预算较小的部署场景下，旗舰模型被蒸馏为 ViT-L（3 亿参数）、ViT-B（8600 万参数）和 ViT-S 学生模型，这些模型在其尺寸类别中均能实现领先的密集预测性能。
掩码边界建模的工作原理
该方法建立在 DINO/iBOT 自蒸馏范式之上：教师模型——学生模型的指数移动平均副本——生成在线目标，而学生模型则从掩码视图中恢复这些目标。
标准掩码图像建模会随机遮盖图像块，而不考虑每个图像块描绘的内容。一个平坦的内部图像块很容易从其相邻块中恢复出来；而一个跨越物体边界的图像块所承载的结构信息，是仅凭上下文无法提供的。边界是图像中冗余度最低、信息量最丰富的区域——而随机掩码却将它们与其他区域一视同仁。
LingBot-Vision 通过两个思路弥补了这一差距。
边界强制。教师模型在线预测一个密集的边界场，并识别出承载边界的 token B。这些 token 被强制加入学生模型的掩码集合中，叠加在随机掩码 M 之上，形成组合掩码 M⁺ = M ∪ B。随后，被掩码的 token 会根据几何特征进行路由：边界 token 除了语义自蒸馏目标外，还会获得一个明确的几何目标；而内部的被掩码 token 则仅保留标准的语义目标。这种路由方式至关重要，因为语义目标在两个区域交界处本质上具有歧义性——而几何目标恰恰在传统掩码建模最薄弱的环节上定义明确，这使得语义和几何表征能够共同涌现，而非相互竞争。
分类式边界场。边界被建模为提升至密集场中的线段：每个邻近像素存储一个属性向量 a(p) = (d, θ, φ¹, φ²)，记录其到最近线段的距离以及三个定位角度。在教师-学生循环中直接回归这个场会导致崩溃。解决方案是将每个通道离散化为 K = 32 个区间，将边界预测重新定义为逐像素分类——这使得边界分支能够继承与现代自蒸馏技术中相同的中心化和锐化机制，从而稳定训练过程。
分类形式有一个优雅的副作用。在经典的“无结构”先验零假设下，边界方向是均匀分布的——而该零假设现在恰好对应各分箱上的均匀分布。偏离均匀分布即表明存在真实边界，因此一个无参数的虚警次数（NFA）检验可以在不增加额外成本的情况下验证每个解码出的片段。教师模型在每次迭代中都利用这一点：它从自身的场预测中解码出候选片段，仅保留通过 NFA 验证的幸存者，并将它们重新渲染到目标场中——这样，无依据的结构永远不会成为教学信号。
完整的目标函数由四项求和构成：
L = L_DINO + λᵢ · L_iBOT + λᵦ · L_bnd + λₖ · L_KoLeo
基准测试与性能
以下所有密集预测结果均使用冻结特征加单层线性层，因此性能归因于表征本身，而非解码器。
| 模型 | 参数量 | NYUv2 RMSE ↓ | KITTI RMSE ↓ | ADE20K mIoU | Cityscapes mIoU | VOC mIoU |
| LingBot-Vision ViT-g | 1B/16 | 0.296 | 2.552 | 53.5 | 79.6 | 87.5 |
| DINOv3 | 7B/16 | 0.309 | 2.346 | 55.9 | 81.1 | 86.6 |
| V-JEPA 2.1 ViT-G | 2B/16 | 0.307 | 2.461 | 47.9 | 73.5 | 85.0 |
| AM-RADIOv2.5 | 1B/14 | 0.340 | 2.918 | 53.0 | 78.4 | 85.4 |
| DINOv2 | 1B/14 | 0.372 | 2.624 | 49.5 | 75.6 | 83.1 |
| SigLIP 2 | 1B/16 | 0.494 | 3.273 | 42.7 | 64.8 | 72.7 |
在 NYU-Depth v2 上，LingBot-Vision 取得了整个对比中最佳的 RMSE（0.296），领先于 7B 参数的 DINOv3（0.309），而参数量仅为后者的约七分之一，同时也领先于 2B 参数的 V-JEPA 2.1（0.307）。在 KITTI 上，它是 2B 参数以下的最佳模型。在语义分割方面，它与经过蒸馏的 DINOv3 ViT-H+ 表现相当——在 ADE20K 上落后 1.3 个 mIoU，在 Cityscapes 上持平，在 VOC12 上领先——同时，在所有三个基准测试上，相比同尺寸的 DINOv2 提升了 4 个以上的 mIoU；唯一尚存的差距是与 DINOv3 系列本身（在 ADE20K 上落后 7B 模型 2.4 个 mIoU），而 DINOv3 的密集预测优势正来源于蒸馏和专门的密集特征目标函数。
视频目标分割利用基于冻结特征的免训练标签传播技术。LingBot-Vision 在 DAVIS-2017 上达到 70.0 J&F，在 YouTube-VOS 上达到 73.5——与 DINOv3 ViT-H+（71.1 / 74.0）和 7B DINOv3（71.1 / 74.1）持平，并且是所有规模剩余模型中表现最好的。边界 token 本身足够稳定，可以通过对冻结特征进行简单的余弦相似度计算来在视频中追踪，无需任何时序监督。
其代价体现在图像级识别能力上：ImageNet-1K 线性探测达到 86.32，k-NN 达到 83.39，落后于将容量用于图像级不变性的 DINOv3-7B。这些优势在知识蒸馏后依然保留——0.3B 的 ViT-L 学生模型在 NYUv2 深度估计上（0.310 vs. 0.309）与 7B DINOv3 相当，而参数量仅为其约 1/23。
使用场景与加载方法
冻结的 patch token 可直接服务于多种密集预测任务：深度估计直接从特征中读取几何信息，语义分割受益于恰好落在目标轮廓上的特征过渡，视频目标分割则通过余弦相似度 token 匹配实现。该编码器还可作为下游深度补全训练的初始化权重。
加载骨干网络遵循官方代码仓库：
git clone https://github.com/robbyant/lingbot-vision.git
cd lingbot-vision
conda create -n lingbot-vision python=3.10 -y
conda activate lingbot-vision
python -m pip install -r requirements.txt
python -m pip install -e .
import torch
from lingbot_vision import load_pretrained_backbone, extract_patch_tokens, load_image
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.bfloat16 if device == "cuda" else torch.float32
# Downloads model.pt from Hugging Face on first use.
backbone, embed_dim = load_pretrained_backbone(
variant="small", # giant | large | base | small; defaults to large
device=device,
dtype=dtype,
)
img_norm, _, _ = load_image(
"examples/example.png",
size=512,
patch_size=backbone.patch_size,
mode="square",
)
patch_tokens, patch_grid = extract_patch_tokens(backbone, img_norm, device, dtype)
print(patch_tokens.shape, patch_grid, embed_dim)
# torch.Size([1, 1024, 384]) (32, 32) 384
variant 参数用于选择模型尺寸，默认为 large。输出的 patch_tokens 形状为 [B, H*W, C]。运行环境要求 Python ≥ 3.10 和 PyTorch ≥ 2.0，较大尺寸的骨干网络建议使用 GPU。
LingBot-Depth 2.0：下游任务的实际收益
为了展示空间感知原生编码器能为下游任务带来什么，研究团队将其深度补全系统升级为 LingBot-Depth 2.0。掩码深度建模的训练方案与 1.0 版本完全相同；仅有两处改动：编码器初始化从 DINOv2 切换为 LingBot-Vision（ViT-L 和 ViT-g 变体），以及精选训练数据从公开的 300 万样本扩充至 1.5 亿样本。
这两项改动在涵盖块掩码、稀疏和真实传感器三种模式的 14 个深度补全基准上均取得了领先结果。在块掩码的 DIODE-Indoor 数据集上，RMSE 从 0.132 降低了一半至 0.062。该系统在透明物体 ClearGrasp 数据集上表现最强（RMSE 为 0.010 / 0.012）——这是主动深度感知的经典失败场景。
值得注意的是，这两项改动是协同增强而非相互抵消：当训练数据从 300 万增长到 1.5 亿时，DINOv2 初始化的曲线在超过 2000 万样本后趋于饱和，而 LingBot-Vision 的曲线则持续提升。更多的数据放大了更好起点的优势，而非将其冲淡。
关键要点
- LingBot-Vision 将边界作为原生预训练信号，从原始图像中自举生成，无需任何标签、边缘检测器或预训练主干网络。
- 边界强制加上分类边界场，使得几何与语义能够共同涌现——并且免费附带一个无参数的 NFA 验证测试。
- 10 亿参数的主干网络在其对比组中取得了最佳的 NYU-Depth v2 RMSE 成绩，超越了 70 亿参数的 DINOv3，而训练语料库规模却小了一个数量级。
- 该优势在知识蒸馏后依然保持：3 亿参数的 ViT-L 在 NYUv2 上以约 1/23 的参数数量达到了 70 亿参数 DINOv3 的水平。
- 仅通过更换编码器并扩展数据规模，LingBot-Depth 2.0 就在 14 个深度补全基准上取得了领先结果，并且编码器的优势随着数据增多而扩大。
- 模型权重以 Apache-2.0 协议发布，提供 ViT-g/L/B/S 多种尺寸，适配各种部署预算。
交互式动态解释器
Robbyant, the embodied-AI company within Ant Group, has open-sourced LingBot-Vision, a family of self-supervised Vision Transformers built for dense spatial perception. The weights ship under Apache-2.0 on Hugging Face in four sizes — ViT-giant, ViT-large, ViT-base, and ViT-small — together with a technical report and inference code.
Most vision foundation models are trained for semantic invariance: they learn to answer what is in an image while discarding exactly the fine-grained spatial structure — object boundaries, contours, depth discontinuities — that robots and other physically embodied systems depend on. LingBot-Vision inverts that priority. It treats boundaries as a native pretraining signal rather than a downstream output, and the payoff is a 1B-parameter backbone that matches or surpasses models up to 7× larger on dense spatial tasks, including the 7B DINOv3.
What is LingBot-Vision?
LingBot-Vision is a self-supervised pretrained encoder for spatially structured downstream tasks. The flagship ViT-g/16 has roughly 1.1B parameters and is trained with a new objective called masked boundary modeling on a curated corpus of about 161M images — selected from a 2B web pool — with no human labels, no external edge detectors, and no pretrained backbone to bootstrap from. The training is also notably economical: the corpus is an order of magnitude smaller than DINOv3’s LVD-1689M, and the model consumes less than a third of DINOv3’s training samples.
The encoder outputs dense patch-token features intended for frozen readouts. For deployment at smaller budgets, the flagship is distilled into ViT-L (300M), ViT-B (86M), and ViT-S students that lead dense prediction within their size classes.
How Masked Boundary Modeling Works
The method builds on the DINO/iBOT self-distillation paradigm: a teacher — an EMA copy of the student — generates online targets, and the student recovers them from masked views.
Standard masked image modeling hides patches at random, ignoring what each patch depicts. A flat interior patch is cheap to recover from its neighbors; a patch straddling an object boundary carries structure that context alone cannot supply. Boundaries are the least redundant, most informative regions of an image — and random masking treats them like everything else.
LingBot-Vision closes that gap with two ideas.
Boundary-forcing. The teacher predicts a dense boundary field online and identifies the boundary-bearing tokens B. These are forced into the student’s masked set on top of the random mask M, giving the combined mask M⁺ = M ∪ B. Masked tokens are then routed by geometry: boundary tokens receive an explicit geometric target in addition to the semantic self-distillation target, while interior masked tokens keep the standard semantic objective alone. This routing matters because a semantic target is inherently ambiguous exactly where two regions meet — the geometric target is well-posed precisely where conventional masked modeling is weakest, which is what lets semantic and geometric representations co-emerge rather than compete.
Categorical boundary field. Boundaries are modeled as line segments lifted into a dense field: every nearby pixel stores an attribute vector a(p) = (d, θ, φ¹, φ²) recording its distance to the nearest segment and three angles that locate it. Directly regressing this field in a teacher–student loop collapses. The fix is to discretize each channel into K = 32 bins, recasting boundary prediction as per-pixel classification — which lets the boundary branch inherit the same centering and sharpening machinery that stabilizes modern self-distillation.
The categorical form has an elegant side effect. Under the classical a-contrario null hypothesis of “no structure,” boundary orientations are uniformly distributed — and that null is now literally the uniform distribution over bins. Deviation from uniformity is evidence of a real boundary, so a parameter-free Number-of-False-Alarms (NFA) test validates every decoded segment at no extra cost. The teacher exploits this at each iteration: it decodes candidate segments from its own field prediction, keeps only the NFA-validated survivors, and re-renders them into the target field — so unsupported structure never becomes a teaching signal.
The full objective sums four terms:
L = L_DINO + λᵢ · L_iBOT + λᵦ · L_bnd + λₖ · L_KoLeo
Benchmarks and Performance
All dense results below use frozen features with a single linear layer, so performance is attributable to the representation rather than a decoder.
| Model | Params | NYUv2 RMSE ↓ | KITTI RMSE ↓ | ADE20K mIoU | Cityscapes mIoU | VOC mIoU |
| LingBot-Vision ViT-g | 1B/16 | 0.296 | 2.552 | 53.5 | 79.6 | 87.5 |
| DINOv3 | 7B/16 | 0.309 | 2.346 | 55.9 | 81.1 | 86.6 |
| V-JEPA 2.1 ViT-G | 2B/16 | 0.307 | 2.461 | 47.9 | 73.5 | 85.0 |
| AM-RADIOv2.5 | 1B/14 | 0.340 | 2.918 | 53.0 | 78.4 | 85.4 |
| DINOv2 | 1B/14 | 0.372 | 2.624 | 49.5 | 75.6 | 83.1 |
| SigLIP 2 | 1B/16 | 0.494 | 3.273 | 42.7 | 64.8 | 72.7 |
On NYU-Depth v2, LingBot-Vision posts the best RMSE of the entire comparison (0.296), ahead of the 7B DINOv3 (0.309) with roughly 7× fewer parameters, and ahead of the 2B V-JEPA 2.1 (0.307). On KITTI it is the best model below 2B parameters. On semantic segmentation it is on par with the distilled DINOv3 ViT-H+ — 1.3 mIoU behind on ADE20K, matching on Cityscapes, ahead on VOC12 — while improving over the same-size DINOv2 by 4+ mIoU on all three benchmarks; the only remaining gap is to the DINOv3 family itself (2.4 mIoU on ADE20K to the 7B model), whose dense strength comes from distillation and dedicated dense-feature objectives.
Video object segmentation uses training-free label propagation over frozen features. LingBot-Vision reaches 70.0 J&F on DAVIS-2017 and 73.5 on YouTube-VOS — on par with DINOv3 ViT-H+ (71.1 / 74.0) and the 7B DINOv3 (71.1 / 74.1), and the best among all remaining models at any scale. The boundary tokens themselves are stable enough to be tracked through video by plain cosine similarity of frozen features, with no temporal supervision.
The trade-off is image-level recognition: ImageNet-1K linear probing reaches 86.32 and k-NN 83.39, trailing DINOv3-7B, which spends its capacity on image-level invariance. The advantages also survive distillation — the 0.3B ViT-L student matches the 7B DINOv3 on NYUv2 depth (0.310 vs. 0.309) with about 23× fewer parameters.
Use Cases and How to Load It
The frozen patch tokens serve several dense workloads directly: depth estimation reads geometry straight from the features, semantic segmentation benefits from feature transitions that land exactly on object contours, and video object segmentation works through cosine-similarity token matching. The encoder also serves as the initialization for downstream depth-completion training.
Loading a backbone follows the official repository:
git clone https://github.com/robbyant/lingbot-vision.git
cd lingbot-vision
conda create -n lingbot-vision python=3.10 -y
conda activate lingbot-vision
python -m pip install -r requirements.txt
python -m pip install -e .
import torch
from lingbot_vision import load_pretrained_backbone, extract_patch_tokens, load_image
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.bfloat16 if device == "cuda" else torch.float32
# Downloads model.pt from Hugging Face on first use.
backbone, embed_dim = load_pretrained_backbone(
variant="small", # giant | large | base | small; defaults to large
device=device,
dtype=dtype,
)
img_norm, _, _ = load_image(
"examples/example.png",
size=512,
patch_size=backbone.patch_size,
mode="square",
)
patch_tokens, patch_grid = extract_patch_tokens(backbone, img_norm, device, dtype)
print(patch_tokens.shape, patch_grid, embed_dim)
# torch.Size([1, 1024, 384]) (32, 32) 384
The variant argument selects the size and defaults to large. Output patch_tokens has shape [B, H*W, C]. Requirements are Python ≥ 3.10 and PyTorch ≥ 2.0, with a GPU recommended for the larger backbones.
LingBot-Depth 2.0: The Downstream Payoff
To show what a spatial-perception-native encoder buys downstream, the team upgraded its depth-completion system to LingBot-Depth 2.0. The masked-depth-modeling recipe is unchanged from version 1.0; exactly two ingredients moved: the encoder initialization switched from DINOv2 to LingBot-Vision (in ViT-L and ViT-g variants), and the curated training data grew from the publicly released 3M samples to 150M.
Those two changes set leading results across 14 depth-completion benchmarks spanning block-mask, sparse, and real-sensor regimes. On block-masked DIODE-Indoor, RMSE is halved from 0.132 to 0.062. The system is strongest on the transparent-object ClearGrasp captures (0.010 / 0.012 RMSE) — the classic failure case of active depth sensing.
Notably, the two changes compound rather than cancel: as training data grows from 3M to 150M, the DINOv2-initialized curve saturates beyond 20M samples while the LingBot-Vision curve keeps improving. More data amplifies, rather than washes out, the advantage of the better starting point.
Key Takeaways
- LingBot-Vision makes boundaries a native pretraining signal, bootstrapped from raw images with no labels, edge detectors, or pretrained backbones.
- Boundary-forcing plus a categorical boundary field lets geometry and semantics co-emerge — and yields a parameter-free NFA validation test for free.
- The 1B backbone posts the best NYU-Depth v2 RMSE in its comparison, ahead of the 7B DINOv3, while training on an order-of-magnitude smaller corpus.
- The advantages survive distillation: the 0.3B ViT-L matches the 7B DINOv3 on NYUv2 with ~23× fewer parameters.
- Swapping only the encoder and scaling data took LingBot-Depth 2.0 to leading results on 14 depth-completion benchmarks, and the encoder’s edge widens with more data.
- Weights ship under Apache-2.0 in ViT-g/L/B/S sizes for every deployment budget.