EchoMimic: Lifelike Audio-Drive Portrait Aimatios through Editable Ladmark Coditioig

Model Files

./pretraied_models/
├── deoisig_uet.pth
├── referece_uet.pth
├── motio_module.pth
├── face_locator.pth
├── sd-vae-ft-mse
│   └── ...
├── sd-image-variatios-diffusers
│   └── ...
└── audio_processor
    └── whisper_tiy.pt

Some models i this hub ca be directly dowloaded from it's origial hub:

• sd-vae-ft-mse: Weights are iteded to be used with the diffusers library. (Thaks to stablilityai)
• sd-image-variatios-diffusers
• audio_processor

Gallery

Audio Drive (Sig)

Audio Drive (Eglish)

Audio Drive (Chiese)

Ladmark Drive

Audio + Selected Ladmark Drive

（Some demo images above are sourced from image websites. If there is ay ifrigemet, we will immediately remove them ad apologize.）

Citatio

If you fid our work useful for your research, please cosider citig the paper:

@misc{che2024echomimic,
  title={EchoMimic: Lifelike Audio-Drive Portrait Aimatios through Editable Ladmark Coditioig},
  author={Zhiyua Che, Jiajiog Cao, Zhiqua Che, Yumig Li, Cheguag Ma},
  year={2024},
  archivePrefix={arXiv},
  primaryClass={cs.CV}
}

EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditioning Zhi