SD-XL 1.0-base Model Card

modelscope usage

pip istall ivisible_watermark trasformers safetesors
pip istall diffusers==0.18.0

from modelscope.utils.costat import Tasks
from modelscope.pipelies import pipelie
import cv2

pipe = pipelie(task=Tasks.text_to_image_sythesis, 
                model='AI-ModelScope/stable-diffusio-xl-base-1.0',
                use_safetesors=True,
                model_revisio='v1.0.0')

prompt = 'a dog'
output = pipe({'text': prompt})
cv2.imwrite('result.pg', output['output_imgs'][0])

Model

SDXL cosists of a mixture-of-experts pipelie for latet diffusio: I a first step, the base model is used to geerate (oisy) latets, which are the further processed with a refiemet model specialized for the fial deoisig steps. Note that the base model ca be used as a stadaloe module.

Alteratively, we ca use a two-stage pipelie as follows: First, the base model is used to geerate latets of the desired output size. I the secod step, we use a specialized high-resolutio model ad apply a techique called SDEdit (https://arxiv.org/abs/2108.01073, also kow as "img2img") to the latets geerated i the first step, usig the same prompt. This techique is slightly slower tha the first oe, as it requires more fuctio evaluatios.

Source code is available at https://github.com/Stability-AI/geerative-models .

Model Descriptio

• Developed by: Stability AI
• Model type: Diffusio-based text-to-image geerative model
• Licese: CreativeML Ope RAIL++-M Licese
• Model Descriptio: This is a model that ca be used to geerate ad modify images based o text prompts. It is a Latet Diffusio Model that uses two fixed, pretraied text ecoders (OpeCLIP-ViT/G ad CLIP-ViT/L).
• Resources for more iformatio: Check out our GitHub Repository ad the SDXL report o arXiv.

Model Sources

For research purposes, we recommed our geerative-models Github repository (https://github.com/Stability-AI/geerative-models), which implemets the most popoular diffusio frameworks (both traiig ad iferece) ad for which ew fuctioalities like distillatio will be added over time. Clipdrop provides free SDXL iferece.

• Repository: https://github.com/Stability-AI/geerative-models
• Demo: https://clipdrop.co/stable-diffusio

Evaluatio

The chart above evaluates user preferece for SDXL (with ad without refiemet) over SDXL 0.9 ad Stable Diffusio 1.5 ad 2.1. The SDXL base model performs sigificatly better tha the previous variats, ad the model combied with the refiemet module achieves the best overall performace.

? Diffusers

Make sure to upgrade diffusers to >= 0.18.0:

pip istall diffusers --upgrade

I additio make sure to istall trasformers, safetesors, accelerate as well as the ivisible watermark:

pip istall ivisible_watermark trasformers accelerate safetesors

Uses

Direct Use

The model is iteded for research purposes oly. Possible research areas ad tasks iclude

• Geeratio of artworks ad use i desig ad other artistic processes.
• Applicatios i educatioal or creative tools.
• Research o geerative models.
• Safe deploymet of models which have the potetial to geerate harmful cotet.
• Probig ad uderstadig the limitatios ad biases of geerative models.

Excluded uses are described below.

Out-of-Scope Use

The model was ot traied to be factual or true represetatios of people or evets, ad therefore usig the model to geerate such cotet is out-of-scope for the abilities of this model.

Limitatios ad Bias

Limitatios

• The model does ot achieve perfect photorealism
• The model caot reder legible text
• The model struggles with more difficult tasks which ivolve compositioality, such as rederig a image correspodig to “A red cube o top of a blue sphere”
• Faces ad people i geeral may ot be geerated properly.
• The autoecodig part of the model is lossy.

Bias

While the capabilities of image geeratio models are impressive, they ca also reiforce or exacerbate social biases.

SD-XL 1.0-base Model Card modelscope usage pip install invisible_watermark transformers safetensors