CotrolNet++: All-i-oe CotrolNet for image geeratios ad editig!

images_display

Network Arichitecture

images

Advatages about the model

Use bucket traiig like ovelai, ca geerate high resolutios images of ay aspect ratio
Use large amout of high quality data(over 10000000 images), the dataset covers a diversity of situatio
Use re-captioed prompt like DALLE.3, use CogVLM to geerate detailed descriptio, good prompt followig ability
Use may useful tricks durig traiig. Icludig but ot limited to date augmetatio, mutiple loss, multi resolutio
Use almost the same parameter compared with origial CotrolNet. No obvious icrease i etwork parameter or computatio.
Support 10+ cotrol coditios, o obvious performace drop o ay sigle coditio compared with traiig idepedetly
Support multi coditio geeratio, coditio fusio is leared durig traiig. No eed to set hyperparameter or desig prompts.
Compatible with other opesource SDXL models, such as BluePecilXL, CouterfeitXL. Compatible with other Lora models.

We desig a ew architecture that ca support 10+ cotrol types i coditio text-to-image geeratio ad ca geerate high resolutio images visually comparable with midjourey. The etwork is based o the origial CotrolNet architecture, we propose two ew modules to: 1 Exted the origial CotrolNet to support differet image coditios usig the same etwork parameter. 2 Support multiple coditios iput without icreasig computatio offload, which is especially importat for desigers who wat to edit image i detail, differet coditios use the same coditio ecoder, without addig extra computatios or parameters. We do thoroughly experimets o SDXL ad achieve superior performace both i cotrol ability ad aesthetic score. We release the method ad the model to the ope source commuity to make everyoe ca ejoy it.

Iferece scripts ad more details ca foud: https://github.com/xisir6/CotrolNetPlus/tree/mai

If you fid it useful, please give me a star, thak you very much