Diffusion Model Alignment Using Direct Preference Optimization

Direct Preference Optimization (DPO) for text-to-image diffusion models is a method to align diffusion models to text human preferences by directly optimizing on human comparison data. Please check our paper at Diffusion Model Alignment Using Direct Preference Optimization.

This model is fine-tuned from stable-diffusion-xl-base-1.0 on offline human preference data pickapic_v2.

Code

Code will come soon!!!

SD1.5

We also have a model finedtuned from stable-diffusion-v1-5 available at dpo-sd1.5-text2image-v1.

A quick example

from diffusers import StableDiffusionXLPipeline, UNet2DConditionModel
from modelscope import snapshot_download
import torch

# load pipeline
model_id_base = "AI-ModelScope/stable-diffusion-xl-base-1.0"
local_base = snapshot_download(model_id_base,revision='master')
pipe = StableDiffusionXLPipeline.from_pretrained(local_base, torch_dtype=torch.float16, variant="fp16", use_safetensors=True).to("cuda")

# load finetuned model
unet_id = "AI-ModelScope/dpo-sdxl-text2image-v1"
local_unet = snapshot_download(unet_id,revision='master')
unet = UNet2DConditionModel.from_pretrained(local_unet, subfolder="unet", torch_dtype=torch.float16)
pipe.unet = unet
pipe = pipe.to("cuda")

prompt = "Two cats playing chess on a tree branch"
image = pipe(prompt, guidance_scale=5).images[0].resize((512,512))

image.save("cats_playing_chess.png")

More details coming soon.