Abstract

Pixel-space diffusion models aim to learn an end-to-end generator directly over raw pixels. This is challenging because a single model must capture both global structure and local texture in the same high-dimensional space. While recent work improves pixel diffusion through alternative prediction targets, training objectives, and architectures, these advances typically require training a new model from scratch. We show there is a cheaper, complementary strategy: a frozen, pretrained pixel diffusion model can guide itself. Our key observation is that intermediate layers of a pretrained pixel diffusion transformer can be decoded into coarse predictions that capture the main low-frequency structure, while the final layers progressively refine local, high-frequency details. We therefore attach a lightweight prediction head to an intermediate layer, keep the backbone frozen, and use the discrepancy between the intermediate and final predictions as a self-guidance direction during sampling. To train this head, we further find that real images are not necessary. Instead, model-generated samples suffice and even outperform real images for training the head, especially in enhancing the high-frequency components that pixel diffusion tends to underfit. Across multiple pixel diffusion models on ImageNet, our Synthetic Self-Guidance (SSG) consistently improves generation while adapter training requires less than 1 of full-model training compute: it reduces FID by over 50 across the evaluated JiT variants without classifier-free guidance (CFG) and further improves strong baselines with CFG, e.g., JiT-H/16 from 1.86 to 1.67 and PixelREPA-H/16 from 1.81 to 1.59. Our code is available at https://github.com/zfu006/SSG.

Introduction

Pixel-space diffusion models generate images directly in raw pixel space (Li and He 2026; Yu et al. 2026; Dhariwal and Nichol 2021). Unlike latent diffusion models (LDMs) (Rombach et al. 2022), they avoid the two-stage pipeline of training and using a VAE to compress images into latent representations. However, this simplicity comes with a harder modeling problem: a single end-to-end model must capture both global semantics and fine-grained local details in the same high-dimensional space. Recent progress has improved pixel diffusion by shifting the prediction target from noise or velocity to clean images, i.e., -prediction (Li and He 2026), and by adding perceptual supervision, representation alignment (Ma et al. 2026b; Shin et al. 2026; Yu et al. 2024; Lei et al. 2025), or hierarchical architectures that decouple semantic structure and high-frequency components (Yu et al. 2026; Ma et al. 2026a; Guo et al. 2026). Despite these advances, most existing methods require designing and training a new model from scratch, which is computationally expensive and does not fully exploit the prior knowledge of a pretrained pixel diffusion model.

No Guidance	CFG	CFG + SSG

This raises a natural question: Can a pretrained pixel diffusion model use its own learned representations to improve generation without retraining the backbone? Our key observation is that intermediate representations of a plain pixel diffusion transformer, such as JiT (Li and He 2026), can be decoded into coarse predictions that capture the main low-frequency structure, while the final layers progressively refine local, high-frequency details. We then turn this internal coarse-to-fine progression into a guidance signal: we attach a lightweight adapter to an intermediate layer, keep the backbone frozen, and extrapolate the final prediction away from the coarse intermediate prediction during sampling, pushing further along the model’s own refinement direction and thereby enhancing high-frequency details that pixel diffusion models struggle to fit.

The remaining design choice is how to train the intermediate adapter. Intuitively, the adapter only needs to provide a coarse, low-frequency prediction as a weak reference for self-guidance, rather than to synthesize the full frequency components of real images. We therefore propose to train the adapter on model-generated samples rather than real images, so that the coarse prediction focuses on low-frequency structure and provides effective guidance that helps alleviate blurry textures and missing high-frequency detail in the final generated images (Fig. 1). Surprisingly, training the adapter on synthetic samples leads to better generation performance than training it on real images. We refer to the complete framework as Synthetic Self-Guidance (SSG), a plug-in strategy that improves pretrained pixel diffusion models without requiring real data for adapter training, as illustrated in Fig. 2. We evaluate SSG across multiple pixel diffusion models (JiT (Li and He 2026), PixelREPA (Shin et al. 2026), and DeCo (Ma et al. 2026a)) on ImageNet (Russakovsky et al. 2015) at 256 and 512 resolutions. Although training the adapter costs less than 1 of the full model’s training compute, SSG reduces FID by over 50 across the evaluated JiT variants without classifier-free guidance (CFG), and further improves strong baselines with CFG, e.g., JiT-H/16 from 1.86 to 1.67 and PixelREPA-H/16 from 1.81 to 1.59. Our contributions are threefold:

• We show that intermediate representations of a pretrained pixel diffusion transformer can be decoded into coarse predictions, while final layers progressively refine high-frequency details. We turn their discrepancy into guidance during sampling without retraining the backbone.

• We find that the adapter can be trained without any real data: the model’s own generated samples are even more effective than real images for training the adapter.

• We propose Synthetic Self-Guidance (SSG), a plug-in method that trains only the adapter using less than 1 of the full model’s training compute, yet improves several pixel diffusion models to competitive FID on ImageNet (e.g., 1.59 on PixelREPA-H/16).

Related Work

Latent and Pixel-Space Diffusion Models

Latent diffusion models (LDMs) typically use a separate VAE to compress images into a lower-dimensional latent space, reducing computational cost by training the generative diffusion model in this latent space (Rombach et al. 2022). DiT replaces the U-Net (Ronneberger et al. 2015) with a transformer architecture, while SiT further adopts velocity prediction and flow matching (Peebles and Xie 2023; Ma et al. 2024; Ho et al. 2020; Liu et al. 2022). Recent methods such as REPA (Yu et al. 2024), VA-VAE (Yao et al. 2025), and RAE (Zheng et al. 2025) adopt pretrained visual representations from foundation models to improve generation in latent space. While LDMs are effective, they retain a two-stage pipeline, and the VAE bottleneck can limit reconstruction fidelity and consequently generation quality.

Pixel-space diffusion models instead perform denoising directly over raw pixels. The original DDPMs (Ho et al. 2020) apply U-Nets to predict noise, but their computational cost increases substantially with image resolution. JiT (Li and He 2026) adopts a plain Vision Transformer (ViT) (Dosovitskiy et al. 2020) with large patches and directly predicts clean images, reducing the difficulty of modeling high-dimensional raw-pixel patches. DeCo (Ma et al. 2026a), DiP (Chen et al. 2026), and PixelDiT (Yu et al. 2026) introduce hierarchical architectures that better decouple semantic structure and high-frequency details for generation. Representation learning and alignment (Yu et al. 2024) have also been explored in pixel diffusion models. PixelREPA (Shin et al. 2026) shows that directly distilling representations from vision foundation models can conflict with pixel-space denoising and addresses this issue with a masked transformer adapter for representation alignment. EPG (Lei et al. 2025) instead adopts an encoder pretrained through self-supervised representation learning and jointly trains it with a decoder for denoising. PixelGen (Ma et al. 2026b) introduces perceptual supervision (Johnson et al. 2016) to improve generation quality and convergence. Other approaches combine latent and pixel diffusion, as in Latent Forcing (Baade et al. 2026), or introduce a U-shaped architecture, as in PixelU (Guo et al. 2026). These methods improve pixel diffusion through new objectives, supervision, or architectures, whereas SSG improves an existing pretrained model while keeping its backbone frozen.

Guidance for Diffusion Models

Guidance during sampling is important for improving the quality of diffusion generation. Classifier-free guidance (CFG) (Ho and Salimans 2022) combines conditional and unconditional predictions by treating the unconditional prediction as a negative reference and extrapolating toward the conditional prediction. Autoguidance (AG) (Karras et al. 2024) further replaces unconditional branches in the CFG with a weak model, using its output to guide the stronger model. Internal Guidance (IG) (Zhou et al. 2026) removes the need for a separate weak model by jointly training the diffusion model with an additional output head at an intermediate layer, whose output serves as a weak prediction. Recently, RAEv2 (Singh et al. 2026) proposes to align its internal representation with vision foundation models and applies the aligned representations to guide the model in the representation space. SSG is most closely related to IG because both obtain weak and strong predictions from the same backbone. The key difference is that IG obtains this intermediate prediction by jointly training the head together with the backbone, while SSG keeps a pretrained backbone frozen and trains only a lightweight adapter on the model’s own synthetic samples, making it a plug-in that requires neither backbone retraining nor real data.

Self-Improving Diffusion with Synthetic Data

Recent studies explore using synthetic samples generated by the model itself as training or negative signals to improve pretrained generative models. SIMS (Alemohammad et al. 2024) initializes an auxiliary score model from the base model, trains it on self-synthesized samples, and uses its score as negative guidance during generation to avoid reproducing degraded patterns from these samples. Neon (Alemohammad et al. 2025) briefly fine-tunes the model on its own samples and then negatively extrapolates in weight space. SSG also uses synthetic samples but is motivated differently: our starting point is the internal coarse-to-fine structure of a pretrained pixel diffusion model, where synthetic samples serve only as training data for its coarse intermediate prediction. SSG therefore keeps the pretrained model frozen, trains only a lightweight intermediate adapter, and uses the discrepancy between the intermediate and final predictions as guidance, requiring neither a separate weak model nor backbone fine-tuning.

Method

In this section, we present Synthetic Self-Guidance (SSG), a plug-in method that improves a pretrained pixel diffusion model using its synthetic samples. We first review the formulation and training objective of pixel diffusion models. We then introduce how an internal representation can be decoded into a coarse prediction with a lightweight adapter and used to guide the final prediction during sampling. Finally, we describe how the inserted intermediate adapter is trained entirely with samples generated by the pretrained model and show that this strategy yields better performance than training it on real images.

Preliminaries

Diffusion Models and Flow Matching.

Diffusion models learn to generate images through iterative denoising. Previous methods such as DDPMs (Ho et al. 2020) train neural networks to predict noise, while recent flow matching methods (Liu et al. 2022; Lipman et al. 2022) formulate generation by learning a time-dependent velocity field:

(1)

where denotes the velocity field predicted by the diffusion model and is the target velocity at timestep . Here, is interpolated between a clean image and Gaussian noise :

(2)

JiT adopts a plain transformer architecture to model the image distribution directly in pixel space. To alleviate the difficulty of predicting noise over large patches in high-dimensional pixel space, JiT directly predicts the clean image:

(3)

The clean prediction is then converted into velocity:

(4)

The diffusion model is trained using the objective in Eq. 1.

Autoguidance and Internal Guidance.

Autoguidance (AG) (Karras et al. 2024) and Internal Guidance (IG) (Zhou et al. 2026) use a weak prediction together with the strong prediction to guide generation:

(5)

where and denote the weak and strong predictions, respectively, and is the guidance scale. AG obtains the weak prediction from a separate weak model, while IG uses an internal output from the diffusion model itself.

Synthetic Self-Guidance for Pixel Diffusion

Observation and Motivation.

Our goal is to improve generation using the pretrained pixel diffusion transformer itself, without training a new model from scratch. We first examine what an intermediate layer already encodes about the clean image by attaching a lightweight trainable adapter to the corresponding layer of a pretrained JiT-B/16 model at 256256 resolution while keeping the backbone frozen. The adapter decodes the intermediate representation back into pixel space, producing an intermediate clean prediction. For this analysis, we independently train an adapter at each of Layers 4, 6, and 8 on the ImageNet training set using the same flow-matching objective as JiT, while updating only the adapter parameters. We sample 512 images from the ImageNet validation set, construct noisy inputs at , and obtain the corresponding intermediate clean predictions together with the final prediction. For each prediction, we apply a two-dimensional Fourier transform to each RGB channel and compute its power spectrum by summing the squared magnitudes across channels. We then aggregate the power over radial frequency bands, exclude the DC component, and divide the spectrum into a low-frequency band and a high-frequency band , with the radial frequency normalized such that corresponds to the Nyquist frequency. The power within each band is normalized by the corresponding band power of the final prediction.

	Noisy input	Intermediate	Final	Residual

As shown in Fig. 3(a), the low-frequency band power of the intermediate predictions is already close to that of the final prediction at shallow layers (e.g., 99.5 at Layer 4). In contrast, the high-frequency band power starts noticeably lower (87.7 at Layer 4) and increases with depth. We further visualize the intermediate and final predictions in Fig. 4. The intermediate prediction already contains coarse, low-frequency image structure, while the final prediction refines local texture details. The residual maps highlight these refinements around fine-scale structures. These observations suggest that the intermediate prediction provides a naturally weak and lower-frequency counterpart to the final prediction. Moreover, we compare the radial power of images generated by JiT with that of real images from the ImageNet validation set in Fig. 3(b) and observe a clear gap between them in the high-frequency band. We therefore treat the intermediate and final predictions as the weak and strong predictions, respectively, and compute the guided clean prediction as

(6)

We refer to this as self-guidance. By Eq. 4, this is equivalent to applying the same guidance form to the velocity predictions. Unlike AG and IG, the weak and strong predictions in self-guidance are derived from the same frozen pretrained backbone, without requiring a separate weak model or joint model training. When , recovers the original final output, while extrapolates the prediction away from the intermediate prediction. This extrapolation preserves the components shared by the intermediate and final outputs while amplifying the refinements introduced by the final prediction, particularly the high-frequency details that the model struggles to fit. Table 4 shows that an adapter trained on real images already improves generation quality over the JiT baseline.

媒体内容 · 前往原文查看

Model	Param.	Epochs	FID	IS	Precision	Recall
Generators in Latent Space
DiT-XL/2 (Peebles and Xie 2023)	675M + 49M	1400	2.27	278.2	0.83	0.57
SiT-XL/2 (Ma et al. 2024)	675M + 49M	1400	2.06	277.5	0.83	0.59
REPA-XL/2 (Yu et al. 2024)	675M + 49M	800	1.42	305.7	0.80	0.64
DDT-XL/2 (Wang et al. 2026)	675M + 49M	400	1.26	310.6	0.79	0.65
RAE-XL/2 (Zheng et al. 2025)	839M + 415M	800	1.13	262.6	0.78	0.67
LightningDiT+IG (Zhou et al. 2026)	678M + 49M	680	1.19	269.0	0.79	0.66
Generators in Pixel Space
ADM-G (Dhariwal and Nichol 2021)	554M	400	4.59	186.7	0.82	0.52
RIN (Jabri et al. 2022)	410M	480	3.42	182.0	–	–
VDM++ (Kingma and Gao 2023)	2B	–	2.12	267.7	–	–
FractalMAR-H (Li et al. 2025)	848M	600	6.15	348.9	0.81	0.46
PixelFlow-XL/4 (Chen et al. 2025)	677M	320	1.98	282.1	0.81	0.60
PixNerd-XL/16 (Wang et al. 2025)	700M	320	1.93	297.0	0.79	0.59
EPG-G/16 (Lei et al. 2025)	1391M	600	1.75	275.1	0.80	0.62
PixelDiT-XL (Yu et al. 2026)	797M	320	1.61	292.7	0.78	0.64
DiP-XL/16 (Chen et al. 2026)	631M	600	1.79	281.9	0.80	0.62
PixelGen-XL/16 (Ma et al. 2026b)	676M	160	1.83	293.6	0.79	0.63
PixelU-H/16 (Guo et al. 2026)	1168M	600	1.63	305.9	0.79	0.64
JiT-H/16 (Li and He 2026)	953M	600	1.86	303.4	0.78	0.62
PixelREPA-H/16 (Shin et al. 2026)	953M	600	1.81	317.2	0.79	0.63
DeCo-XL/16 (Ma et al. 2026a)	682M	600	1.69	304.0	0.79	0.63
JiT-H/16 + SSG	953M⋆+ 63M	50	1.67	290.3	0.79	0.63
PixelREPA-H/16 + SSG	953M⋆+ 63M	50	1.59	304.2	0.79	0.63
DeCo-XL/16 + SSG	682M⋆+ 57M	50	1.63	303.5	0.79	0.63

Training the Adapter with Synthetic Samples.

The adapter used in the previous analysis is trained on real images. However, the role of the adapter in self-guidance is not to reproduce the full frequency components of real images. Instead, it only needs to provide a coarse, low-frequency counterpart to the final prediction. This motivates us to train the adapter on samples generated by the pretrained model itself. As shown in Fig. 2(a), we use the pretrained diffusion model to generate a synthetic dataset. Each generated image, paired with its sampling condition, is treated as a clean target for the adapter. We then train the adapter on this synthetic dataset using the objective in Eq. 1, while keeping the backbone frozen (Fig. 2(b)). During sampling, we apply the same guidance strategy defined in Eq. 6, as illustrated in Fig. 2(c). We refer to this complete method as Synthetic Self-Guidance (SSG). Surprisingly, Table 4 shows that training the adapter with synthetic samples yields better generation quality than training it with real images. As shown in Fig. 3(b), guidance with the synthetic-trained adapter brings the radial power spectrum of the generated images closest to that of real images, especially in the high-frequency band. Importantly, SSG requires no real images and updates only the adapter, using less than 1 of full-model training compute. This makes SSG a plug-in method for improving pretrained pixel diffusion models. In practice, SSG can be used either alone or together with CFG. When combined with CFG, we first apply self-guidance independently to the conditional and unconditional clean predictions, convert both guided predictions into velocities, and then apply CFG.

Experiments

Experiment Settings

We evaluate SSG on all JiT (Li and He 2026) variants for class-conditional ImageNet (Russakovsky et al. 2015) generation at 256 and 512 resolutions, and further on PixelREPA-H (Shin et al. 2026) and DeCo-XL (Ma et al. 2026a). For all models, we apply SSG to the officially released pretrained checkpoints while keeping their backbones frozen, and take the baseline results from the corresponding papers.

Adapter Settings.

The JiT adapter consists of one transformer block (Vaswani et al. 2017) for the base and large models and two blocks for the huge models, followed by a linear layer that maps features back into pixel space. PixelREPA-H adopts the same adapter configuration as JiT-H. Since DeCo employs a hierarchical architecture, we attach the adapter to its intermediate encoder representation and use its pixel decoder to produce the intermediate prediction. The attachment layers are provided in the supplementary material.

Synthetic Data.

Each pretrained model generates its own synthetic dataset using the best CFG setting in its paper. We generate 1K samples per class (1M images in total), approximately matching the scale of the 1.28M ImageNet training images used in the real-data comparison. We show in Table 5 that fewer synthetic samples suffice in practice.

Training and Sampling.

We train the adapters of JiT-H, PixelREPA-H, and DeCo-XL for 50 epochs and those of JiT-B and JiT-L for 30 epochs. In all cases, adapter training requires less than 1 of the computational cost of training the corresponding baseline model from scratch. Following JiT, we use the Heun sampler with 50 steps and evaluate SSG both with and without CFG. Without CFG, SSG is applied throughout the sampling process. When combined with CFG, both CFG and SSG are applied within the timestep interval . We report the best result for each model by sweeping the CFG and SSG scales, with the specific scales provided in the supplementary material.

Evaluation.

We evaluate generation performance on the ImageNet validation set and follow the JiT evaluation protocol by generating 50K class-balanced samples. We report FID (Heusel et al. 2017), Inception Score (IS) (Salimans et al. 2016), Precision and Recall (Kynkäänniemi et al. 2019), and compare SSG with the most recent pixel diffusion models.

Main Results

Class-Conditional Image Generation.

Tables 1 and 2 report results with CFG at 256 and 512 resolutions. SSG improves FID across all backbones: from 1.86 to 1.67 (10.2) for JiT-H/16, from 1.94 to 1.84 (5.2) for JiT-H/32, and from 1.81 to 1.59 (12.2) for PixelREPA-H/16. It also transfers to hierarchical architectures, improving DeCo-XL/16 from 1.69 to 1.63 without per-layer tuning. IS decreases slightly because we select the CFG and SSG scales for the lowest FID, for which SSG typically favors a lower CFG scale. The fixed-CFG comparison in Fig. 5(b) further shows that SSG still improves FID at the optimal CFG scale of the original baseline. Without CFG (Table 3), the gains are substantially larger: SSG reduces FID by more than 50 for all models.

媒体内容 · 前往原文查看

Model	Params	Ep.	FID	IS	Prec.	Rec.
Generators in Latent Space
DiT-XL/2	675M + 49M	600	3.04	240.8	0.84	0.54
SiT-XL/2	675M + 49M	600	2.62	252.2	0.84	0.57
REPA-XL/2	675M + 49M	800	2.08	274.6	0.83	0.58
Generators in Pixel Space
ADM-G	554M	400	7.72	172.7	0.84	0.53
PixNerd-XL/16	700M	340	2.84	245.6	0.80	0.59
EPG-L/32	540M	800	2.35	295.4	0.82	0.57
DeCo-XL/16	682M	340	2.22	290.0	0.80	0.60
PixelU-H/32	1152M	600	1.92	322.1	0.80	0.58
JiT-H/32	956M	600	1.94	309.1	0.80	0.61
JiT-H/32+SSG	956M⋆+ 66M	50	1.84	305.2	0.80	0.63

Ablation Study

Synthetic Samples Outperform Real Images for Adapter Training.

Table 4 compares adapters trained on real and synthetic images. Training the adapter on real images consistently improves the JiT baselines, validating the effectiveness of self-guidance. The adapter trained on synthetic samples outperforms the real-trained one across nearly all settings. For JiT-H/16, synthetic training reduces FID from 2.51 to 2.26 without CFG and from 1.78 to 1.67 with CFG. Fig. 3(b) supports this finding: the adapter trained on the synthetic dataset produces samples whose radial power is closer to that of real images than the real-trained adapter.

Fewer Synthetic Samples Suffice.

Table 5 studies the effect of synthetic dataset size. Reducing the dataset from 1M to 10K samples (a 100 reduction) changes FID by less than 0.05 for both JiT-B/16 (3.29 to 3.31) and JiT-B/32 (3.67 to 3.69). Note that we vary only the dataset size while keeping the total number of training iterations fixed. In our main experiments, we use 1M samples by default, approximately matching the scale of the ImageNet training set for a fair comparison with real-trained adapters.

Early-to-Middle Layers Provide the Best Guidance.

Fig. 5(a) evaluates attachment layers on JiT-B/16 without CFG. Layers 4–6 achieve comparable FID, while Layer 3 is slightly worse and Layers 7–8 degrade sharply toward the baseline. At deeper layers, the intermediate prediction approaches the final prediction, reducing their discrepancy and weakening self-guidance. We also observe this trend for large and huge models. In practice, we favor earlier layers within this range to reduce adapter-training compute. The layer for each model is provided in the supplementary material.

媒体内容 · 前往原文查看

Model	Params	Ep.	FID	IS	Prec.	Rec.
ADM-G	554M	400	10.94	–	0.69	0.63
PixelFlow-XL	677M	320	12.23	103.3	0.63	0.66
PixNerd-XL	700M	320	15.61	88.9	0.59	0.68
DeCo-XL/16	682M	320	14.88	88.2	0.60	0.68
PixelGen-XL/16	676M	80	5.11	159.2	0.72	0.63
JiT-B/16	131M	600	25.42	63.2	0.54	0.66
JiT-L/16	458M	600	13.85	104.2	0.62	0.67
JiT-H/16	953M	600	7.15	151.7	0.68	0.67
JiT-B/16+SSG	131M⋆+ 12M	30	9.47	115.5	0.70	0.57
JiT-L/16+SSG	459M⋆+ 22M	30	4.47	174.5	0.76	0.59
JiT-H/16+SSG	953M⋆+ 63M	50	2.26	223.7	0.78	0.62

Effect of Guidance Scales.

Fig. 5(b) sweeps the SSG scale and CFG scale on JiT-B/16. SSG improves the baseline across a wide range of scale combinations. At the baseline’s optimal CFG scale of 3.0, SSG with reduces FID from 3.66 to 3.33, showing that the gain is not merely due to the larger guidance search space. The guidance scales of each configuration are provided in the supplementary material.

媒体内容 · 前往原文查看

	W/o CFG			With CFG
Model	Baseline	Real	Syn.	Baseline	Real	Syn.
JiT-B/16	25.42	9.73	9.47	3.66	3.50	3.29
JiT-B/32	28.57	14.16	13.37	4.02	3.91	3.67
JiT-L/16	13.85	4.72	4.47	2.36	2.34	2.23
JiT-L/32	13.91	4.86	4.51	2.53	2.42	2.31
JiT-H/16	7.15	2.51	2.26	1.86	1.78	1.67
JiT-H/32	7.32	2.66	2.73	1.94	1.91	1.84

媒体内容 · 前往原文查看

Samples	JiT-B/16		JiT-B/32
	FID	IS	FID	IS
1M	3.29	274.0	3.67	272.7
100K	3.25	270.4	–	–
10K	3.31	268.9	3.69	274.8

媒体内容 · 前往原文查看

Adapter	Params	Rel. FLOPs	FID w/o CFG	FID with CFG
JiT-B/16, Layer 6
No adapter	0	0	25.42	3.66
Linear only	1.77M	0.65%	36.44	7.95
1 block (default)	12.40M	0.99%	9.47	3.29
2 blocks	23.03M	1.33%	9.14	3.22
JiT-H/16, Layer 8
No adapter	0	0	7.15	1.86
1 block	33.78M	0.69%	2.51	1.76
2 blocks (default)	63.29M	0.88%	2.26	1.67

A Lightweight Adapter Suffices.

Table 6 studies the effect of adapter capacity. We fix the SSG and CFG scales across adapter variants. A linear layer performs worse than the JiT baseline (36.44 vs. 25.42 without CFG) and introduces patch-boundary artifacts. Adding a transformer block removes these artifacts and improves generation quality. On JiT-B/16, two blocks slightly improve FID but raise adapter-training compute above of full-model training, so we use one block for JiT-B and JiT-L. For JiT-H, two blocks provide a clearer gain at low cost, so we adopt this setting.

媒体内容 · 前往原文查看

	Generation CFG
Model	1.0	2.0	3.0
JiT-B/16	3.25	3.31	3.29
JiT-B/32	3.72	3.71	3.67

SSG Is Robust to the Generation CFG.

By default, we generate the synthetic data using the best CFG scale reported for each pretrained model. Table 7 varies this generation CFG and shows that the downstream FID is nearly unchanged: it stays within 0.06 on JiT-B/16 (3.25 to 3.31) and 0.05 on JiT-B/32 (3.67 to 3.72). This indicates that SSG does not require a carefully chosen generation CFG, and we simply adopt the reported best CFG without additional tuning.

Comparison with IG.

We implement IG (Zhou et al. 2026) on JiT by jointly training the backbone and an intermediate head at Layer 6 for 600 epochs, matching the JiT-B/16 baseline schedule. As shown in Table 8, it reaches 3.41 FID, outperforming the frozen adapter trained on real images (3.50) but remaining behind SSG (3.29).

媒体内容 · 前往原文查看

Method	Backbone	Training Data	FID
JiT-B/16	–	–	3.66
Adapter+real	Frozen	Real	3.50
JiT+IG	Updated	Real	3.41
SSG	Frozen	Synthetic	3.29

Conclusion

We presented Synthetic Self-Guidance (SSG), a plug-in method for improving pretrained pixel diffusion models while keeping their backbones frozen. SSG decodes an intermediate representation into a coarse prediction and uses its discrepancy from the final prediction as self-guidance during sampling. The adapter is trained solely on model-generated samples, which outperform real images for this purpose in nearly all evaluated settings. Experiments across multiple pixel diffusion models show consistent FID improvements with low adapter-training cost.

Supplementary Material

Appendix A Implementation Details

Table S2 summarizes the backbone, adapter, training, and sampling configurations used in our experiments.

Adapter Initialization.

For JiT and PixelREPA, the adapter blocks are initialized from the late backbone blocks listed in Table S2, while the output layer is initialized from the pretrained final prediction head. For DeCo, we initialize the adapter from its last two encoder blocks and copy the pretrained pixel decoder as the adapter output layer. During adapter training, we update only the adapter.

Training Details.

Following JiT, we use Adam with and , without weight decay or learning-rate warmup. For JiT and PixelREPA, a base learning rate of 5e-5 is scaled to 2e-4 with a global batch size of 1024. DeCo uses a learning rate of 5e-5 and a global batch size of 256. JiT and PixelREPA train the adapter to predict the clean image, whereas DeCo directly optimizes the velocity prediction. No representation supervision or auxiliary frequency loss is used for adapter training.

Sampling Details.

All models use the Heun sampler with 50 steps. JiT models at 256 and 512 resolutions use noise scales of 1.0 and 2.0, respectively, while PixelREPA and DeCo use a noise scale of 1.0. Without CFG, SSG is applied throughout sampling; when combined with CFG, both guidance methods are applied within . The CFG and SSG scales used for each model and resolution are provided in Table S1.

Appendix B Guidance-Scale Results

Fig. S1 presents the full guidance-scale sweeps for JiT-H/16 and JiT-H/32. SSG improves FID over a broad range of CFG and SSG scales, showing that its effectiveness is not sensitive to a specific scale combination. At the CFG scales used by the original baselines, setting reduces FID from 1.86 to 1.73 on JiT-H/16 and from 1.94 to 1.89 on JiT-H/32. This fixed-CFG comparison confirms that the improvement does not merely result from searching over additional guidance-scale combinations.

Appendix C Frequency-Domain Analysis Details

Layer-Wise Band Power.

For the layer-wise analysis, we use 512 images from the ImageNet validation set and construct noisy inputs at . We obtain intermediate clean predictions from independently trained adapters attached to Layers 4, 6, and 8 of JiT-B/16, together with the final clean prediction. We apply an orthonormal two-dimensional Fourier transform to each RGB channel of the predictions and sum the squared magnitudes across channels. Frequency coefficients are grouped into one-pixel radial bands, and their power is averaged across images. We exclude the DC component and normalize the radial frequency by the Nyquist frequency. The low- and high-frequency powers are computed over and , respectively, and normalized by the corresponding band power of the final prediction.

媒体内容 · 前往原文查看

	W/o CFG	With CFG
Model	SSG	CFG	SSG
JiT-B/16	1.7	2.9	1.2
JiT-B/32	1.7	2.9	1.3
JiT-L/16	1.4	2.2	1.1
JiT-L/32	1.4	2.3	1.1
JiT-H/16	1.4	1.9	1.2
JiT-H/32	1.3	2.1	1.1
PixelREPA-H/16	–	2.0	1.2
DeCo-XL/16	–	2.9	1.05

Generated-Image Radial Power.

We compare 2048 class-balanced ImageNet validation images with the same number of samples from the JiT-B/16 baseline and self-guidance using adapters trained on real or synthetic images. All generated results use the same class labels and initial noise. Real and generated images are clamped in , and their radial power is computed using the same orthonormal Fourier transform and channel aggregation described above. We average the two-dimensional power maps over all images and exclude the DC component. The remaining frequency coefficients are grouped into radial bands according to their distance from the zero-frequency location, and the power is summed within each band. We normalize the radial frequency such that 1 corresponds to the Nyquist frequency. For visualization, we show normalized frequencies from 0.12 to 0.96 on a logarithmic scale and divide all curves by the same constant, preserving their relative differences.

Appendix D Additional Qualitative Results

We provide additional class-conditional samples generated by JiT-H/16 with SSG. As seen in Fig. S2, Fig. S3, and Fig. S4, our method generates high-quality samples.

媒体内容 · 前往原文查看

	JiT-B	JiT-L	JiT-H	PixelREPA-H	DeCo-XL
Architecture
Backbone parameters	131M	459M	953M	953M	682M
Backbone depth	12	24	32	32	28 (enc.) / 3 (dec.)
Hidden size	768	1024	1280	1280	1152 (enc.) / 32 (dec.)
Image size	256 / 512			256	256
Patch size	16 / 32			16	16
Adapter
Attachment layer	6	6	8	8	8 (enc.)
Adapter depth	1 block	1 block	2 blocks	2 blocks	2 enc. blocks
Output layer	Linear layer				Pixel decoder
Prediction target	Clean image				Velocity
Adapter parameters	12M	22M	63M	63M	57M
Adapter Training
Epochs	30	30	50	50	50
Synthetic samples	1M
Optimizer	Adam, ,
Global batch size	1024				256
Learning rate	2e-4				5e-5
Sampling
Sampler	Heun
Sampling steps	50
Noise scale	1.0 / 2.0			1.0	1.0
SSG interval w/o CFG
CFG and SSG interval

Appendix E Limitations and Future Work

Although SSG is effective, our experiments mainly focus on class-conditional ImageNet generation with pixel-space diffusion models. It remains unclear whether the intermediate-to-final refinement observed in these models also generalizes to larger text-conditioned models or domains with different image distributions. Future work could extend SSG to text-to-image generation, broader datasets, and other diffusion architectures.

class 288: leopard, Panthera pardus

class 309: bee

class 349: bighorn, bighorn sheep, cimarron, Rocky Mountain bighorn

class 397: puffer, pufferfish, blowfish, globefish

class 425: barn

class 448: birdhouse

class 453: bookcase

class 458: brass, memorial tablet, plaque

class 495: china cabinet, china closet

class 500: cliff dwelling

class 658: mitten

class 661: Model T

class 718: pier

class 724: pirate, pirate ship

class 725: pitcher, ewer

class 757: recreational vehicle, RV, R.V.

class 779: school bus

class 780: schooner

class 829: streetcar, tram, tramcar, trolley, trolley car

class 853: thatch, thatched roof

class 873: triumphal arch

class 900: water tower

class 911: wool, woolen, woollen

class 913: wreck

class 927: trifle

class 930: French loaf

class 946: cardoon

class 947: mushroom

class 975: lakeside, lakeshore

class 989: hip, rose hip, rosehip

References

• S. Alemohammad, A. I. Humayun, S. Agarwal, J. Collomosse, and R. Baraniuk (2024) Self-improving diffusion models with synthetic data. arXiv preprint arXiv:2408.16333. Cited by: Self-Improving Diffusion with Synthetic Data.
• S. Alemohammad, Z. Wang, and R. G. Baraniuk (2025) Neon: negative extrapolation from self-training improves image generation. arXiv preprint arXiv:2510.03597. Cited by: Self-Improving Diffusion with Synthetic Data.
• A. Baade, E. R. Chan, K. Sargent, C. Chen, J. Johnson, E. Adeli, and L. Fei-Fei (2026) Latent forcing: reordering the diffusion trajectory for pixel-space image generation. arXiv preprint arXiv:2602.11401. Cited by: Latent and Pixel-Space Diffusion Models.
• S. Chen, C. Ge, S. Zhang, P. Sun, and P. Luo (2025) Pixelflow: pixel-space generative models with flow. arXiv preprint arXiv:2504.07963. Cited by: Table 1.
• Z. Chen, J. Zhu, X. Chen, J. Zhang, X. Hu, H. Zhao, C. Wang, J. Yang, and Y. Tai (2026) Dip: taming diffusion models in pixel space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 36136–36146. Cited by: Latent and Pixel-Space Diffusion Models, Table 1.
• P. Dhariwal and A. Nichol (2021) Diffusion models beat gans on image synthesis. Advances in neural information processing systems 34, pp. 8780–8794. Cited by: Introduction, Table 1.
• A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al. (2020) An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929. Cited by: Latent and Pixel-Space Diffusion Models.
• Z. Guo, L. Ma, Y. He, X. Fu, J. Fu, J. Huang, and Y. Li (2026) PixelU: a u-shaped transformer for efficient end-to-end pixel diffusion. arXiv preprint arXiv:2606.27760. Cited by: Introduction, Latent and Pixel-Space Diffusion Models, Table 1.
• M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter (2017) Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems 30. Cited by: Evaluation..
• J. Ho, A. Jain, and P. Abbeel (2020) Denoising diffusion probabilistic models. Advances in neural information processing systems 33, pp. 6840–6851. Cited by: Latent and Pixel-Space Diffusion Models, Latent and Pixel-Space Diffusion Models, Diffusion Models and Flow Matching..
• J. Ho and T. Salimans (2022) Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598. Cited by: Guidance for Diffusion Models.
• A. Jabri, D. Fleet, and T. Chen (2022) Scalable adaptive computation for iterative generation. arXiv preprint arXiv:2212.11972. Cited by: Table 1.
• J. Johnson, A. Alahi, and L. Fei-Fei (2016) Perceptual losses for real-time style transfer and super-resolution. In European conference on computer vision, pp. 694–711. Cited by: Latent and Pixel-Space Diffusion Models.
• T. Karras, M. Aittala, T. Kynkäänniemi, J. Lehtinen, T. Aila, and S. Laine (2024) Guiding a diffusion model with a bad version of itself. Advances in Neural Information Processing Systems 37, pp. 52996–53021. Cited by: Guidance for Diffusion Models, Autoguidance and Internal Guidance..
• D. Kingma and R. Gao (2023) Understanding diffusion objectives as the elbo with simple data augmentation. Advances in Neural Information Processing Systems 36, pp. 65484–65516. Cited by: Table 1.
• T. Kynkäänniemi, T. Karras, S. Laine, J. Lehtinen, and T. Aila (2019) Improved precision and recall metric for assessing generative models. Advances in neural information processing systems 32. Cited by: Evaluation..
• J. Lei, K. Liu, J. Berner, H. Yu, H. Zheng, J. Wu, and X. Chu (2025) There is no vae: end-to-end pixel-space generative modeling via self-supervised pre-training. arXiv preprint arXiv:2510.12586. Cited by: Introduction, Latent and Pixel-Space Diffusion Models, Table 1.
• T. Li and K. He (2026) Back to basics: let denoising generative models denoise. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 36115–36125. Cited by: Introduction, Introduction, Introduction, Latent and Pixel-Space Diffusion Models, Table 1, Experiment Settings.
• T. Li, Q. Sun, L. Fan, and K. He (2025) Fractal generative models. arXiv preprint arXiv:2502.17437. Cited by: Table 1.
• Y. Lipman, R. T. Chen, H. Ben-Hamu, M. Nickel, and M. Le (2022) Flow matching for generative modeling. arXiv preprint arXiv:2210.02747. Cited by: Diffusion Models and Flow Matching..
• X. Liu, C. Gong, and Q. Liu (2022) Flow straight and fast: learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.03003. Cited by: Latent and Pixel-Space Diffusion Models, Diffusion Models and Flow Matching..
• N. Ma, M. Goldstein, M. S. Albergo, N. M. Boffi, E. Vanden-Eijnden, and S. Xie (2024) Sit: exploring flow and diffusion-based generative models with scalable interpolant transformers. In European Conference on Computer Vision, pp. 23–40. Cited by: Latent and Pixel-Space Diffusion Models, Table 1.
• Z. Ma, L. Wei, S. Wang, S. Zhang, and Q. Tian (2026a) Deco: frequency-decoupled pixel diffusion for end-to-end image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 43600–43610. Cited by: Introduction, Introduction, Latent and Pixel-Space Diffusion Models, Table 1, Experiment Settings.
• Z. Ma, R. Xu, and S. Zhang (2026b) PixelGen: pixel diffusion beats latent diffusion with perceptual loss. arXiv preprint arXiv:2602.02493. Cited by: Introduction, Latent and Pixel-Space Diffusion Models, Table 1.
• W. Peebles and S. Xie (2023) Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 4195–4205. Cited by: Latent and Pixel-Space Diffusion Models, Table 1.
• R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer (2022) High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10684–10695. Cited by: Introduction, Latent and Pixel-Space Diffusion Models.
• O. Ronneberger, P. Fischer, and T. Brox (2015) U-net: convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, pp. 234–241. Cited by: Latent and Pixel-Space Diffusion Models.
• O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al. (2015) Imagenet large scale visual recognition challenge. International journal of computer vision 115 (3), pp. 211–252. Cited by: Introduction, Experiment Settings.
• T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen (2016) Improved techniques for training gans. Advances in neural information processing systems 29. Cited by: Evaluation..
• J. Shin, J. Kim, and H. Shim (2026) Representation alignment for just image transformers is not easier than you think. arXiv preprint arXiv:2603.14366. Cited by: Introduction, Introduction, Latent and Pixel-Space Diffusion Models, Table 1, Experiment Settings.
• J. Singh, B. Zheng, Z. Wu, R. Zhang, E. Shechtman, and S. Xie (2026) Improved baselines with representation autoencoders. arXiv preprint arXiv:2605.18324. Cited by: Guidance for Diffusion Models.
• A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin (2017) Attention is all you need. Advances in neural information processing systems 30. Cited by: Adapter Settings..
• S. Wang, Z. Gao, C. Zhu, W. Huang, and L. Wang (2025) Pixnerd: pixel neural field diffusion. arXiv preprint arXiv:2507.23268. Cited by: Table 1.
• S. Wang, Z. Tian, W. Huang, and L. Wang (2026) Ddt: decoupled diffusion transformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 40633–40642. Cited by: Table 1.
• J. Yao, B. Yang, and X. Wang (2025) Reconstruction vs. generation: taming optimization dilemma in latent diffusion models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 15703–15712. Cited by: Latent and Pixel-Space Diffusion Models.
• S. Yu, S. Kwak, H. Jang, J. Jeong, J. Huang, J. Shin, and S. Xie (2024) Representation alignment for generation: training diffusion transformers is easier than you think. arXiv preprint arXiv:2410.06940. Cited by: Introduction, Latent and Pixel-Space Diffusion Models, Latent and Pixel-Space Diffusion Models, Table 1.
• Y. Yu, W. Xiong, W. Nie, Y. Sheng, S. Liu, and J. Luo (2026) Pixeldit: pixel diffusion transformers for image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14273–14282. Cited by: Introduction, Latent and Pixel-Space Diffusion Models, Table 1.
• B. Zheng, N. Ma, S. Tong, and S. Xie (2025) Diffusion transformers with representation autoencoders. arXiv preprint arXiv:2510.11690. Cited by: Latent and Pixel-Space Diffusion Models, Table 1.
• X. Zhou, Q. Li, X. Hu, H. Chen, and S. Gu (2026) Guiding a diffusion transformer with the internal dynamics of itself. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11536–11545. Cited by: Guidance for Diffusion Models, Autoguidance and Internal Guidance., Table 1, Comparison with IG., Table 8.