This topic describes the following types of models: checkpoint, hypernetwork, embedding, low-rank adaptation (LoRA), and variational autoencoder (VAE).
Checkpoint
Overview
A checkpoint model is a base model. Stable Diffusion provides checkpoint models of V1.5 and V2.1. Other characteristic models that are developed after further training and ensemble based on the official checkpoint models are available on Civitai. These models integrate data such as model parameters and weights during the pre-training and ensemble processes and can be used to generate images. However, the styles and characteristics of the images generated by using these models are fixed. If you want to customize image styles, you must use other models to perform fine-tuning operations.
Example
Positive prompts: best quality, masterpiece, highres, 1girl,blush,(seductive smile:0.8),star-shaped pupils,china hanfu,hair ornament,necklace, jewelry,Beautiful face,upon_body, tyndall effect,photorealistic, dark studio, rim lighting, two tone lighting,(high detailed skin:1.2), 8k uhd, dslr, soft lighting, high quality, volumetric lighting, candid, Photograph, high resolution, 4k, 8k, Bokeh
Negative prompts: (((simple background))),monochrome ,lowres, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, username, blurry, lowres, bad anatomy, bad hands, text, error, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, username, blurry, ugly,pregnant,vore,duplicate,morbid,mut ilated,tran nsexual, hermaphrodite,long neck,mutated hands,poorly drawn hands,poorly drawn face,mutation,deformed,blurry,bad anatomy,bad proportions,malformed limbs,extra limbs,cloned face,disfigured,gross proportions, (((missing arms))),(((missing legs))), (((extra arms))),(((extra legs))),pubic hair, plump,bad legs,error legs,username,blurry,bad feet
Prompt source: Civitai
Output image
Hypernetwork
Overview
A hypernetwork model must work with a checkpoint model to modify the style or visual effect of an image. This way, images with specified styles are generated. Hypernetwork models are not widely used because the model performance is not satisfying in some aspects that require fine-grained effects.
Procedure
After you select a hypernetwork model, a tag is automatically displayed in the positive prompts. You can adjust the weight to control the characteristics of the generated image.
Embedding
Overview
An embedding model is also called a textual inversion model that must be used together with a checkpoint model. An embedding model recognizes the keyword that you define to generate images with specified characteristics or styles. Although embedding models are small in size and require less storage space, this type of model delivers less information. This may affect the styles and visual effects of output images. Therefore, embedding models are not widely used.
Procedure
After you select an embedding model, a tag is automatically displayed in the negative prompts.
Example
Positive prompts: best quality, masterpiece, highres, 1girl,blush,(seductive smile:0.8),star-shaped pupils,china hanfu,hair ornament,necklace, jewelry,Beautiful face,upon_body, tyndall effect,photorealistic, dark studio, rim lighting, two tone lighting,(high detailed skin:1.2), 8k uhd, dslr, soft lighting, high quality, volumetric lighting, candid, Photograph, high resolution, 4k, 8k, Bokeh
Negative prompts: (((simple background))),monochrome ,lowres, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, username, blurry, lowres, bad anatomy, bad hands, text, error, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, username, blurry, ugly,pregnant,vore,duplicate,morbid,mut ilated,tran nsexual, hermaphrodite,long neck,mutated hands,poorly drawn hands,poorly drawn face,mutation,deformed,blurry,bad anatomy,bad proportions,malformed limbs,extra limbs,cloned face,disfigured,gross proportions, (((missing arms))),(((missing legs))), (((extra arms))),(((extra legs))),pubic hair, plump,bad legs,error legs,username,blurry,bad feet
Prompt source: Civitai
Image comparison: The input image is displayed on the left and the output image is displayed on the right.
LoRA
Overview
A LoRA model must be used together with a checkpoint model to generate images. LoRA models are trained based on checkpoint models to provide better effects. When a creator shares a pre-trained LoRA model on Civitai, the version of the checkpoint model based on which the LoRA model is trained is specified. You can select the checkpoint model of the specified version to work with the LoRA model and improve the effects of generated images.
Procedure
After you select a LoRA model, a tag is automatically displayed in the positive prompts. You can modify the weight whose default value is 1 based on your business requirements. Valid values: [0,1]. A greater value specifies a closer similarity to the specified LoRA model. We recommend that you set the weight to 0.8.
Example
In this example, the checkpoint model in the following figure is used.
The LoRA model in the following figure is used.
Positive prompts: best quality, masterpiece, highres, 1girl,blush,(seductive smile:0.8),star-shaped pupils,china hanfu,hair ornament,necklace, jewelry,Beautiful face,upon_body, tyndall effect,photorealistic, dark studio, rim lighting, two tone lighting,(high detailed skin:1.2), 8k uhd, dslr, soft lighting, high quality, volumetric lighting, candid, Photograph, high resolution, 4k, 8k, Bokeh
Negative prompts: (((simple background))),monochrome ,lowres, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, username, blurry, lowres, bad anatomy, bad hands, text, error, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, username, blurry, ugly,pregnant,vore,duplicate,morbid,mut ilated,tran nsexual, hermaphrodite,long neck,mutated hands,poorly drawn hands,poorly drawn face,mutation,deformed,blurry,bad anatomy,bad proportions,malformed limbs,extra limbs,cloned face,disfigured,gross proportions, (((missing arms))),(((missing legs))), (((extra arms))),(((extra legs))),pubic hair, plump,bad legs,error legs,username,blurry,bad feet
Prompt source: Civitai
The following figures show the output images with different weights on the premise that the random seed is set to 3 and default values are used for other parameters.
VAE
Overview
A VAE model must work with a checkpoint model to fine-tune the colors and filters of generated images. Generally, you do not need to download a VAE model for a checkpoint model because most of the checkpoint models have built-in VAE models. You can use VAE models of other styles based on your business requirements and interests.
Example
Positive prompts: best quality, masterpiece, highres, 1girl,blush,(seductive smile:0.8),star-shaped pupils,china hanfu,hair ornament, jewelry,Beautiful face,tyndall effect,photorealistic, dark studio, rim lighting,high detailed skin:1.2), 8k uhd, dslr, soft lighting, high quality, volumetric lighting, candid, Photograph, high resolution, 4k, 8k, Bokeh
Negative prompts: (((simple background))),monochrome ,lowres, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, username, blurry, lowres, bad anatomy, bad hands, text, error, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, username, blurry, ugly,pregnant,vore,duplicate,morbid,mut ilated,tran nsexual, hermaphrodite,long neck,mutated hands,poorly drawn hands,poorly drawn face,mutation,deformed,blurry,bad anatomy,bad proportions,malformed limbs,extra limbs,cloned face,disfigured,gross proportions, (((missing arms))),(((missing legs))), (((extra arms))),(((extra legs))),pubic hair, plump,bad legs,error legs,username,blurry,bad feet
Prompt source: Civitai
In this example, the random seed is set to 55555 and default values are used for other parameters.
Image comparison: The input image is displayed on the left and the output image is displayed on the right.
