Your choice of Stable Diffusion checkpoint model determines what type of images you will generate.
There are so many models out there it's quite hard to keep track.
Here's a comparison of anime models based on overall popularity, aesthetics and versatility. You might be more interested in general-purpose/realistic models.
Quick summary:
- Counterfeit and PastelMix are beautiful models with unique styles. Very easy to get good results with. I would highly recommended any of these for your first model.
- NAI Diffusion is an important model because it's used to create so many other anime models. Though historically significant, it's a bit dated and it shows.
- Anything models are finetunes of NAI, that improve the image quality and details
- OrangeMix series is very popular in the Japanese community. They usually merge Anything V3 + NAI Diffusion + other anime models.
- AbyssOrangeMix3 is a mix of a lot of anime models, it stands out because of realistic, cinematic lighting
Anime Models: A short history
StabilityAI released the first public checkpoint model, Stable Diffusion v1.4, in August 2022. In the coming months they released v1.5, v2.0 & v2.1.
Soon after these models were released, users started to train (fine-tune) their own custom models on top of the base models. Today, most custom models are built on top of either one of these base models, v1.4, v1.5, v2.0 or v2.1.
NAI Diffusion was released in October 2022.
Today, "Stable Diffusion model" is used to refer to the official base models by StabilityAI, but is also a blanket term for all diffusion models.
Usage
Running locally
To use different SD models, you just have to download them as well as the AUTOMATIC1111 WebUI to run them with. Installation instructions for different platforms:
Google Colab
You can run the WebUI in the cloud with Google Colab. You can use Colab for free (with limited usage and availability) but I recommend paying the $10/mo. You get access to their best GPU available, the NVIDIA A100.
To use Colab, open one of these notebooks, then click Runtime -> Run all. After a few minutes your WebUI will be ready, and you will get a message like this:
Running on local URL: http://127.0.0.1:7860
Running on public URL: https://973c3956-5540-4671.gradio.live
You can use either URL.
Model Comparison
These all use the exact same prompt and settings.
Prompt:
masterpiece, best quality, kisaragi chihaya, solo, 1girl, ((musical notation)) vector background:0.2], anime, idolmaster, singing, long blue hair, brown eyes, idol, knee length skirt,white shirtNegative prompt:
deformed, blurry, bad anatomy, wrong anatomy, disfigured, distorted, poorly drawn face, poorly drawn hands, strangely bent, bad proportions, mutation, mutated, malformed, (out of frame), watermark, signature, text, bad art, beginner, amateur, low quality, worst quality, EasyNegativeSettings:
- Sampler:
Euler A - Size:
512x768 - Steps:
20 - CFG scale:
7 - Clip skip:
2 - Seed:
324646734167 - Batch Size:
1
Models List
NAI Diffusion
NAI Diffusion is a model created by NovelAI.net by modifying the Stable Diffusion architecture and training method.
At the time of release, it was a massive improvement over other anime models. Whilst the then popular Waifu Diffusion was trained on SD + 300k anime images, NAI was trained on millions.
Here's a NAI prompt guide that is broadly applicable to most anime models.
Most popular models today are based upon NAI.
Anything Series (万象熔炉)
Anything V3
Download Link • Model Information
Created by a Chinese anon, Anything V3 was released to immediate popularity. It was a fine-tune of NAI that improved on details, shading and overall quality.
While Anything V1 & V2 exist, they never reached the worldwide popularity of V3, and are difficult to find outside of Chinese forums. Highly recommended.
Anything V5
Download Link • Model Information
(The Anything creator gave up on Anything v4 and v4.5. The v4 and v4.5 models that are published today are considered "fake", as they were made by another person and have a very different aesthetic.)
In the creator's own words, V5 is more faithful to prompting than V3. As such, more prompting skill is required for good looking images. The creator actually recommends beginners use V3, because they will likely get better looking images.
OrangeMixes
AbyssOrangeMix2 (AOM2)
Download (SFW Variant) • Download (NSFW Variant)
Model Information
AbyssOrangeMix2 (AOM2) is a model for creating high-quality, highly realistic illustrations.
Recommended Sampler: DPM++ SDE Karras
AbyssOrangeMix3 (AOM3)
Download Link • Model Information
Latest version of AOM.
Counterfeit
Download Link • Model Information
High quality anime model with a very artistic style. Created by gsdf, with DreamBooth + Merge Block Weights + Merge LoRA. Highly recommended.
MeinaMix
Download Link • Model Information
High quality anime model with a very artistic style. Created by gsdf, with DreamBooth + Merge Block Weights + Merge LoRA
PastelMix
Download Link • Model Information
High-chroma, vivid painterly style, influenced by Pastel art. Highly Recommended.
Tips
Use LoRAs
You use LoRAs in addition to the checkpoint models above to add stuff to them.
For example, you could use an Studio Ghibli LoRA to make your outputs look more like the Studio Ghibli aesthetic.
The Best Stable Diffusion LoRAs
Use Negative Embeddings
Here's a super fast way to fix things like bad hands and bad quality images:
Use negative embeddings in your prompts.
Embeddings (AKA Textual Inversion) are small files that contain additional concepts. You activate them by writing a keyword in the prompt.
Negative Embeddings are files trained on bad quality images. By placing these in the negative prompt, you'll get better quality images.
Some popular negative embeddings are EasyNegative and negative_hand.
FAQ
Many custom checkpoint models also come in different file sizes: fp16 (floating-point 16, aka half-precision floating-point) and fp32 (floating point 32, aka full-precision floating-point). This refers to the format in which the data are stored inside the model, which can either be 16-bit (2 bytes), or 32-bit (4 bytes). Unless you want to train or mix your own custom model, the smaller (usually 2 GiB) fp16 version is all you need. For most casual users, the difference between the image quality produced by fp16 and fp32 is insignificant.