IstatID Model Card

Itroductio

IstatID is a ew state-of-the-art tuig-free method to achieve ID-Preservig geeratio with oly sigle image, supportig various dowstream tasks.

Usage

You ca directly dowload the model i this repository. You also ca dowload the model i pytho script:

from modelscope.hub.file_dowload import model_file_dowload
model_cofig=model_file_dowload(model_id="IstatX/IstatID", file_path="CotrolNetModel/cofig.jso", cache_dir="./checkpoits")
model_cotrol=model_file_dowload(model_id="IstatX/IstatID", file_path="CotrolNetModel/diffusio_pytorch_model.safetesors", cache_dir="./checkpoits")
model_ip=model_file_dowload(model_id="IstatX/IstatID", file_path="ip-adapter.bi", cache_dir="./checkpoits")

For face ecoder, you eed to mautally dowload via this URL to models/atelopev2.

# !pip istall opecv-pytho trasformers accelerate isightface
import diffusers
from diffusers.utils import load_image
from diffusers.models import CotrolNetModel

import cv2
import torch
import umpy as p
from PIL import Image

from isightface.app import FaceAalysis
from pipelie_stable_diffusio_xl_istatid import StableDiffusioXLIstatIDPipelie, draw_kps

# prepare 'atelopev2' uder ./models
app = FaceAalysis(ame='atelopev2', root='./', providers=['CUDAExecutioProvider', 'CPUExecutioProvider'])
app.prepare(ctx_id=0, det_size=(640, 640))

# prepare models uder ./checkpoits
face_adapter = f'./checkpoits/IstatX/IstatID/ip-adapter.bi'
cotrolet_path = f'./checkpoits/IstatX/IstatID/CotrolNetModel/'

# load IdetityNet
cotrolet = CotrolNetModel.from_pretraied(cotrolet_path, torch_dtype=torch.float16)

pipe = StableDiffusioXLIstatIDPipelie.from_pretraied(
...     "stabilityai/stable-diffusio-xl-base-1.0", cotrolet=cotrolet, torch_dtype=torch.float16
... )
pipe.cuda()

# load adapter
pipe.load_ip_adapter_istatid(face_adapter)

The, you ca customized your ow face images

# load a image
image = load_image("your-example.jpg")

# prepare face emb
face_ifo = app.get(cv2.cvtColor(p.array(face_image), cv2.COLOR_RGB2BGR))
face_ifo = sorted(face_ifo, key=lambda x:(x['bbox'][2]-x['bbox'][0])*x['bbox'][3]-x['bbox'][1])[-1] # oly use the maximum face
face_emb = face_ifo['embeddig']
face_kps = draw_kps(face_image, face_ifo['kps'])

pipe.set_ip_adapter_scale(0.8)

prompt = "aalog film photo of a ma. faded film, desaturated, 35mm photo, graiy, vigette, vitage, Kodachrome, Lomography, staied, highly detailed, foud footage, masterpiece, best quality"
egative_prompt = "(lowres, low quality, worst quality:1.2), (text:1.2), watermark, paitig, drawig, illustratio, glitch, deformed, mutated, cross-eyed, ugly, disfigured (lowres, low quality, worst quality:1.2), (text:1.2), watermark, paitig, drawig, illustratio, glitch,deformed, mutated, cross-eyed, ugly, disfigured"

# geerate image
image = pipe(
...     prompt, image_embeds=face_emb, image=face_kps, cotrolet_coditioig_scale=0.8
... ).images[0]

For more details, please follow the istructios i our GitHub repository.

Usage Tips

1. If you're ot satisfied with the similarity, try to icrease the weight of "IdetityNet Stregth" ad "Adapter Stregth".
2. If you feel that the saturatio is too high, first decrease the Adapter stregth. If it is still too high, the decrease the IdetityNet stregth.
3. If you fid that text cotrol is ot as expected, decrease Adapter stregth.
4. If you fid that realistic style is ot good eough, go for our Github repo ad use a more realistic base model.

Demos

Disclaimer

This project is released uder Apache Licese ad aims to positively impact the field of AI-drive image geeratio. Users are grated the freedom to create images usig this tool, but they are obligated to comply with local laws ad utilize it resposibly. The developers will ot assume ay resposibility for potetial misuse by users.

Citatio

@article{wag2024istatid,
  title={IstatID: Zero-shot Idetity-Preservig Geeratio i Secods},
  author={Wag, Qixu ad Bai, Xu ad Wag, Haofa ad Qi, Zekui ad Che, Athoy},
  joural={arXiv preprit arXiv:2401.07519},
  year={2024}
}

InstantID Model Card [**Project Page**](https://instantid.github.io/) **|** [**Paper**](https://ar