Deedy@deedydas · XAI 摘要
研究人员在图像生成模型SDXL中发现神经网络权重中存在“流形”（manifolds），例如椒盐卷饼流形，可直接从权重中操控生成不同种类的椒盐卷饼。这是可解释性领域的重要突破，提供了一种低计算量的“音量旋钮”来编辑生成结果。该研究基于GoodfireAI提出的Block-Sparse Featurizers (BSFs)方法，即用多维“块”而非单一方向来寻找模型激活中的概念。
Today， researchers made an important breakthrough in interpretability.
They found "manifolds" in the neural net weights for any concepts in image gen models （SDXL）， like the pretzel manifold， and could steer them to generate various kinds of pretzels from the weights directly.
This is well beyond a neat theoretical understanding on how AI models see but gives us a low-compute volume dial to edit the results of a generation.