模型描述 (Model Description)

ELITE是一种基于扩散模型实现图像主体保ID内容迁移的算法。该方法借助基于训练的encoder提取原图的语义信息，从而实现保ID的文生图和图像内容迁移。和其它方法相比，该算法能通过更快的编码过程实现更高保真度的图像编辑功能。更多详情可参照主页

?Paper |

运行环境 (Operating environment)

Dependencies and Installation

# git clone the original repository
git clone https://github.com/csyxwei/ELITE.git
cd ELITE

# Create a conda environment and activate it
conda create -n elite python=3.9
conda activate elite

# Install from requirements.txt
pip install -r requirements.txt

代码范例 (Code example)

参数说明：image为完整输入图像，mask为输入图像经主体分割后的二值图像。token为占位标记，用以指代目标主体，正常情况下无需修改。prompt为文本输入用以控制图像生成，其中涉及目标主体的词需用token代替。

from modelscope.pipelines import pipeline
from PIL import Image
import os
import numpy as np

input = {'image': 'https://vigen-video.oss-cn-shanghai.aliyuncs.com/VideoEnhancement/Dataset/ELITEDataset/1.jpg?OSSAccessKeyId=LTAI5tQUr7s9wUCuorYPo5Qj&Expires=360001692773758&Signature=mgRwOPaRenU1JjAHcC5DEeho904%3D', 'mask': 'https://vigen-video.oss-cn-shanghai.aliyuncs.com/VideoEnhancement/Dataset/ELITEDataset/1_bg.png?OSSAccessKeyId=LTAI5tQUr7s9wUCuorYPo5Qj&Expires=3600001692773885&Signature=sTtkRdkc%2BE6EDmtdN%2BBZiRmyN7I%3D', 'token': 'S', 'prompt': 'a photo of a S'}

inference = pipeline('my-ELITE-task', model='aojie1997/cv_ELITE_image-content-transfer', model_revision='v1.0')
output = inference(input)
Image.fromarray(output).save('./result.png')

Citation

If you find our work helpful for your research, please consider citing the following BibTeX entry.

@article{wei2023elite,
  title={ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generation},
  author={Wei, Yuxiang and Zhang, Yabo and Ji, Zhilong and Bai, Jinfeng and Zhang, Lei and Zuo, Wangmeng},
  journal={arXiv preprint arXiv:2302.13848},
  year={2023}
}