高分辨率神经细胞自动机(NCA)
阅读原文· cells2pixels.github.io针对NCA在高分辨率下训练慢、信息传播局部化、推理计算重的局限,提出将粗网格NCA与轻量隐式解码器(LPPN)配对:解码器通过插值细胞状态与局部坐标直接输出任意分辨率的外观属性(如颜色、法线),且两者均为局部计算,推理可高度并行化。引入任务特定损失函数,以极小额外内存和计算开销监督形态发生与纹理合成。在2D/3D网格及网格域上的实验表明,混合模型能实时生成高分辨率输出,并保留NCA的自组织行为。
Neural Cellular Automata: From Cells to Pixels
Alexander Mordvintsev2, Wenzel Jakob1, Sabine Süsstrunk1
Steps / Frame:
Brush Size
LPPN Scale:
Brush Mode
Abstract
Neural Cellular Automata (NCAs) are bio-inspired dynamical systems in which identical cells iteratively apply a learned local update rule to self-organize into complex patterns, exhibiting regeneration, robustness, and spontaneous dynamics. Despite their success in texture synthesis and morphogenesis, NCAs remain largely confined to low-resolution outputs. This limitation stems from (1) training time and memory requirements that grow quadratically with grid size, (2) the strictly local propagation of information that impedes long-range cell communication, and (3) the heavy compute demands of real-time inference at high resolution. In this work, we overcome this limitation by pairing an NCA that evolves on a coarse grid with a lightweight implicit decoder that maps cell states and local coordinates to appearance attributes, enabling the same model to render outputs at arbitrary resolution. Moreover, because both the decoder and NCA updates are local, inference remains highly parallelizable. To supervise high-resolution outputs efficiently, we introduce task-specific losses for morphogenesis (growth from a seed) and texture synthesis with minimal additional memory and computation overhead. Our experiments across 2D/3D grids and mesh domains demonstrate that our hybrid models produce high-resolution outputs in real-time, and preserve the characteristic self-organizing behavior of NCAs.
How it works?
The NCA operates on a coarse lattice of cells (in this example vertices of a mesh). Center: A sampling point \(\Point\) (red dot) inside a triangle primitive, whose vertices correspond to NCA cells \(\State_i,\,\State_j,\,\State_k\). The local coordinate \(u(\Point)\) expresses the point’s position inside the primitive, while the locally averaged cell state \(\bar{\State}(\Point)\) is obtained by interpolating the surrounding cell states. Right: The Local Pattern Producing Network (LPPN), A shared lightweight MLP, receives \((\bar{\State}(\Point), u(\Point))\) as input and outputs the target properties, such as color and surface normal, at point \(\Point\). The NCA and the LPPN are trained jointly and end-to-end.
Play with the interactive visualization below to see coarse NCA cell states and the output the LPPN generates.
Textures on Meshes (MeshNCA)
Meet the Team!
Ehsan
Yitao
Ali
Alex
Wenzel
Sabine
BibTeX