Stable Diffusion

When Stability AI released Stable Diffusion in August 2022, it changed the AI image generation landscape overnight. Previous state-of-the-art models like DALL-E 2 and Midjourney were accessible only through controlled APIs. Stable Diffusion was released as open weights - anyone could download the model, run it locally, modify it, and build on top of it. Within weeks, a global community was producing fine-tuned variants, building tools, and exploring the model's capabilities in ways its creators had not anticipated.

Stable Diffusion's technical contribution was latent diffusion - performing the diffusion process not in pixel space but in a compressed latent space produced by a variational autoencoder (VAE). Full-resolution pixel-space diffusion is extremely slow because images have millions of pixels. The VAE compresses an image into a representation roughly 8 times smaller in each spatial dimension (64 times fewer values), and the diffusion happens in this compressed space. A separate decoder then maps the generated latent back to pixels. This compression is what made the model fast enough to run on a single GPU.

The model consists of three main components: the VAE (which compresses images to latents and decodes them back), a U-Net (which performs the denoising in latent space), and a text encoder (CLIP, which converts text prompts into conditioning signals for the U-Net). The text encoder is what allows text conditioning: the U-Net learns to denoise in ways that respect the text embedding, so the generated image matches the prompt.

The open-source release enabled an ecosystem that no single company could have built. ControlNet added the ability to condition generation on edge maps, depth maps, pose skeletons, and other structural signals - allowing precise control over composition. DreamBooth and LoRA fine-tuning allowed users to train models to generate specific subjects (particular people, art styles, product designs). Community model hubs accumulated thousands of fine-tuned variants.

The release also triggered ongoing debates about training data, consent, and copyright. Stable Diffusion was trained on the LAION dataset, which scraped billions of images from the internet without explicit permission from creators. Artists whose work appeared in the training data found their styles could be reproduced on demand. These tensions remain unresolved and have shaped subsequent legal and policy discussions around AI-generated content.