How to use Stable Diffusion v2.1 (AUTOMATIC1111 Guide)

Stable diffusion v2.1 was released by StabilityAI on Dec 7, 2022 (announcement).

Stable Diffusion v2.0 was widely criticised by users who saw it as inferior to the previous version, V1.5.

What happened?

To train SD v2.0, StabilityAI used a much larger dataset of images, but filtered the adult content out using LAION’s NSFW filter (LAION is a non-profit group which makes models and datasets for AI researchers).

The larger dataset means that v2.0 was better than previous models at inanimate things such as architecture and landscapes.

The problem was that the adult content filter was very aggressive, and dramatically cut down on the images of people in the dataset, even the ones that were safe for work.

Users had a much harder time generating high-quality people. Since people are probably the largest use case of Stable Diffusion, they were quite upset.

StabilityAI reacted quickly to fix the problem with Stable Diffusion v2.1. This model still filters out adult content, but the filter is nowhere near as aggressive as before. This gives the best of both worlds - improvements in inanimate things, as well as improvements in people.

Installation

Before proceeding with installation, here are the recommended specs:

• 16GB RAM
• NVIDIA (GTX 7xx or newer) GPU with at least 2GB VRAM (AMD GPU will work, but NVIDIA is recommended)
• Linux or Windows 7/8/10+ or Mac M1/M2 (Apple Silicon)
• 10GB disk space (includes models)

1. Download the model file

The model is all the stuff the AI has been trained on and is capable of generating. Model files end in '.ckpt', which stands for 'checkpoint'.

There are two variants of the Stable Diffusion v2.1 model.

Stable Diffusion v2.1 (768)

1. Download the model file: v2-1_768-ema-pruned.ckpt
2. Download the .yaml config file, rename it to v2-1_768-ema-pruned.yaml (right click the link > Save link as... > save the file)

Put both of them in the model directory:

stable-diffusion-webui/models/Stable-diffusion

Stable Diffusion v2.1 (512-base)

1. Download the model file: v2-1_512-ema-pruned.ckpt
2. Download the .yaml config file, rename it to v2-1_512-ema-pruned.yaml (right click the link > Save link as... > save the file)

Put both of them in the model directory:

stable-diffusion-webui/models/Stable-diffusion

2. Download the Web UI

This is the user interface you will use to run the generations.

The most popular Stable Diffusion user interface is AUTOMATIC1111's Stable Diffusion WebUI. I've tried a couple of apps and I can see why people like AUTOMATIC1111 so much. The developers are lightning fast and they keep on adding to an already impressive and robust feature-set.

Here are the installation instructions for the WebUI depending on your platform:

• Installation for Windows NVIDIA GPU: instructions (new tab)
• Installation for Apple Silicon (Mac M1/M2): instructions (new tab)
• Installation for Linux: instructions (new tab)

3. Place the Model in the Web UI folder

Put both of them in the model directory:

stable-diffusion-webui/models/Stable-diffusion

Rename the files like so:

v2-inference-v.yaml -> v2-1_512-ema-pruned.yaml

You can rename these files whatever you want, as long as filename for model and config file are the same before the file extension. Since you will be placing all future models into this folder choose a descriptive name that helps you remember what this is.

4. Start the Web UI

• Windows: double-click webui-user.bat to start
• Linux: run webui-user.sh to start
• Mac: run the command ./webui.sh in terminal to start

You will know it's ready when you see the line Running on local URL: http://127.0.0.1:7860

Let's open up our web browser and go to this address: http://127.0.0.1:7860

You'll notice at the top there's a toggle called "Stable Diffusion Checkpoint".

You can use this to switch to any model you have placed in the stable-diffusion-webui/models/Stable-diffusion folder.

If you put the model in the folder after you start the Web UI, you will need to restart the Web UI to see the model.