Diffusion Models: A Revolutionary Training in 24 Hours

In the previous two articles of this series, we explored a variety of architectural and training tricks for diffusion models. Each idea was evaluated individually to measure throughput, convergence speed, and final image quality, while seeking to understand the elements that truly make a difference.

In this article, we address a more practical question: what happens when all the tricks that worked are combined? Rather than optimizing one dimension at a time, we decided to stack the most promising ingredients to see how far we can push performance under a strict computational budget.

To make this concrete, we conducted a 24-hour speedrun with a total computational budget of around $1500, or $2 per hour per GPU. This stands in stark contrast to the early days of diffusion models, where training competitive models could cost millions of dollars. The goal here is to demonstrate how much the field has evolved and how far careful engineering can take you in just one day of training.

This speedrun is not just a fun experiment. It will likely serve as a foundation for our large-scale training recipe in the future. In addition to the results, we are also open-sourcing our code, which includes the training code used for this speedrun as well as the experimental framework from the previous blog articles. This way, you can reproduce, modify, and extend everything yourself.

The Training Recipe

X-Prediction and Training in Pixel Space

We use the x-prediction formulation from the paper "Back to Basics: Let Denoising Generative Models Denoise" by Li and He, 2025. As seen in Part 2, this allows for training directly in pixel space and completely eliminates the need for a VAE. We use a patch size of 32 and a bottleneck of 256 dimensions in the initial token projection layer. This design keeps the sequence length manageable, making training in pixel space feasible even at higher resolutions.

At 512px, the sequence length is: (512 / 32)^2 = 256. At 1024px, the sequence length becomes: (1024 / 32)^2 = 1024. Instead of following the usual schedule of 256px → 512px → 1024px, we start directly at 512px and then refine to 1024px. With controlled token counts and modern hardware, training in pixel space is no longer prohibitive. It’s simply a cleaner and more straightforward formulation.

Perceptual Losses

A very interesting side effect of predicting x0 directly in pixel space is that we can reuse an entire toolbox from classical computer vision. When your model produces latents, perceptual supervision becomes tricky. You either have to decode back to pixels or define losses in a learned latent space that may or may not align with human perception. Once you predict pixels directly, everything becomes straightforward again. You can integrate perceptual losses exactly as they were originally designed.

We draw inspiration from the paper "PixelGen: Pixel Diffusion Beats Latent Diffusion with Perceptual Loss" by Ma et al., where the authors introduce additional perceptual objectives alongside the diffusion loss. They show that adding perceptual signals can significantly improve convergence speed and final visual quality.

For this 24-hour run, we add two auxiliary losses: LPIPS (Zhang et al.) and a perceptual loss based on DINO (we use DINOv2 [Oquab et al.]). The idea is simple: in addition to the standard flow matching objective, we encourage the predicted clean image to match the target image in a perceptual feature space. LPIPS captures low-level perceptual similarity, while DINO features provide a stronger semantic signal.

We maintain the same general idea as in the paper, but we have adjusted a few details. In our experiments, we empirically found it more effective to apply perceptual losses on complete grouped images rather than on patch features and to apply them at all noise levels. These are small implementation details, but in our setup, they consistently yielded better results.

We used a weight of 0.1 for the LPIPS loss and 0.01 for the DINO perceptual loss, corresponding to the recommended values in the original paper. These losses are lightweight compared to the main forward pass of the transformer, and in our setup, they add only a slight overhead while providing a consistent quality gain.

Token Routing with TREAD

To make each step less costly, we use token routing with TREAD (Krause et al., 2025), which randomly selects a fraction of tokens and allows them to bypass a contiguous block of transformers, then reinjects them later to avoid losing anything. We chose TREAD over SPRINT (Park et al., 2025) primarily for its simplicity, and because the additional complexity of SPRINT did not seem worth the relatively small computational savings in our setup (sequence length of 64 versus 128 with TREAD at 512px).