Stable Diffusion

The Best Stable Diffusion Anime Models (comparison)

by

Yubin

on

updated

No Comments

Your choice of Stable Diffusion checkpoint model determines what type of images you will generate.

There are so many models out there it's quite hard to keep track.

Here's a comparison of anime models based on overall popularity, aesthetics and versatility. You might be more interested in general-purpose/realistic models.

Quick summary:

  • Counterfeit and PastelMix are beautiful models with unique styles. Very easy to get good results with. I would highly recommended any of these for your first model.
  • NAI Diffusion is an important model because it's used to create so many other anime models. Though historically significant, it's a bit dated and it shows.
  • Anything models are finetunes of NAI, that improve the image quality and details
  • OrangeMix series is very popular in the Japanese community. They usually merge Anything V3 + NAI Diffusion + other anime models.
    • AbyssOrangeMix3 is a mix of a lot of anime models, it stands out because of realistic, cinematic lighting

Anime Models: A short history

StabilityAI released the first public checkpoint model, Stable Diffusion v1.4, in August 2022. In the coming months they released v1.5, v2.0 & v2.1.

Soon after these models were released, users started to train (fine-tune) their own custom models on top of the base models. Today, most custom models are built on top of either one of these base models, v1.4, v1.5, v2.0 or v2.1.

NAI Diffusion was released in October 2022.

Today, "Stable Diffusion model" is used to refer to the official base models by StabilityAI, but is also a blanket term for all diffusion models.

Usage

Running locally

To use different SD models, you just have to download them as well as the AUTOMATIC1111 WebUI to run them with. Installation instructions for different platforms:

Google Colab

You can run the WebUI in the cloud with Google Colab. You can use Colab for free (with limited usage and availability) but I recommend paying the $10/mo. You get access to their best GPU available, the NVIDIA A100.

To use Colab, open one of these notebooks, then click Runtime -> Run all. After a few minutes your WebUI will be ready, and you will get a message like this:

Running on local URL: http://127.0.0.1:7860
Running on public URL: https://973c3956-5540-4671.gradio.live

You can use either URL.

Model Comparison

These all use the exact same prompt and settings.

NAI Diffusion (animefull_final)
Anything V3
Anything V5
Anything V4
AbyssOrangeMix2
AbyssOrangeMix3
BloodOrangeMix
Hassaku
Counterfeit-V3.0
MeinaMix
CetusMix
PastelMix
Waifu Diffusion 1.4
Stable Diffusion v1.5
Stable Diffusion v2.1

Prompt:

masterpiece, best quality, kisaragi chihaya, solo, 1girl, ((musical notation)) vector background:0.2], anime, idolmaster, singing, long blue hair, brown eyes, idol,  knee length skirt,white shirt

Negative prompt:

deformed, blurry, bad anatomy, wrong anatomy, disfigured, distorted, poorly drawn face, poorly drawn hands, strangely bent, bad proportions, mutation, mutated, malformed, (out of frame), watermark, signature, text, bad art, beginner, amateur, low quality, worst quality, EasyNegative

Settings:

  • Sampler: Euler A
  • Size: 512x768
  • Steps: 20
  • CFG scale: 7
  • Clip skip: 2
  • Seed: 324646734167
  • Batch Size: 1

Models List

NAI Diffusion

Download link

NAI Diffusion is a model created by NovelAI.net by modifying the Stable Diffusion architecture and training method.

At the time of release, it was a massive improvement over other anime models. Whilst the then popular Waifu Diffusion was trained on SD + 300k anime images, NAI was trained on millions.

Here's a NAI prompt guide that is broadly applicable to most anime models.

Most popular models today are based upon NAI.

Anything Series (万象熔炉)

Anything V3

Download LinkModel Information

Created by a Chinese anon, Anything V3 was released to immediate popularity. It was a fine-tune of NAI that improved on details, shading and overall quality.

While Anything V1 & V2 exist, they never reached the worldwide popularity of V3, and are difficult to find outside of Chinese forums. Highly recommended.

Anything V5

Download LinkModel Information

(The Anything creator gave up on Anything v4 and v4.5. The v4 and v4.5 models that are published today are considered "fake", as they were made by another person and have a very different aesthetic.)

In the creator's own words, V5 is more faithful to prompting than V3. As such, more prompting skill is required for good looking images. The creator actually recommends beginners use V3, because they will likely get better looking images.

OrangeMixes

AbyssOrangeMix2 (AOM2)

Download (SFW Variant)Download (NSFW Variant)
Model Information

AbyssOrangeMix2 (AOM2) is a model for creating high-quality, highly realistic illustrations.

Recommended Sampler: DPM++ SDE Karras

AbyssOrangeMix3 (AOM3)

Download LinkModel Information

Latest version of AOM.

Counterfeit

Download Link Model Information

High quality anime model with a very artistic style. Created by gsdf, with DreamBooth + Merge Block Weights + Merge LoRA. Highly recommended.

MeinaMix

Download LinkModel Information

High quality anime model with a very artistic style. Created by gsdf, with DreamBooth + Merge Block Weights + Merge LoRA

PastelMix

Download LinkModel Information

High-chroma, vivid painterly style, influenced by Pastel art. Highly Recommended.

Tips

Use LoRAs

You use LoRAs in addition to the checkpoint models above to add stuff to them.

For example, you could use an Studio Ghibli LoRA to make your outputs look more like the Studio Ghibli aesthetic.

The Best Stable Diffusion LoRAs

Use Negative Embeddings

Here's a super fast way to fix things like bad hands and bad quality images:

Use negative embeddings in your prompts.

Embeddings (AKA Textual Inversion) are small files that contain additional concepts. You activate them by writing a keyword in the prompt.

Negative Embeddings are files trained on bad quality images. By placing these in the negative prompt, you'll get better quality images.

Some popular negative embeddings are EasyNegative and negative_hand.

FAQ

What is the difference between fp16 and fp32?

Many custom checkpoint models also come in different file sizes: fp16 (floating-point 16, aka half-precision floating-point) and fp32 (floating point 32, aka full-precision floating-point). This refers to the format in which the data are stored inside the model, which can either be 16-bit (2 bytes), or 32-bit (4 bytes). Unless you want to train or mix your own custom model, the smaller (usually 2 GiB) fp16 version is all you need. For most casual users, the difference between the image quality produced by fp16 and fp32 is insignificant.

Yeah, AI moves way too fast

Get the email that makes keeping up with AI easy and fun.

Leave a Comment