Sonic: Shifting Focus to Global Audio Perception in Portrait Animation
2025/01/14: Our inference code and weights are released. Stay tuned, we will continue to polish the model.
2024/12/16: Our Online Demo is released.
- install pytorch
pip3 install -r requirements.txt- Download pretrain model, svd-xt-1-1 and whisper-tiny to checkpoints/
python3 demo.py \
'/path/to/input_image' \
'/path/to/input_audio' \
'/path/to/output_video'@misc{ji2024sonicshiftingfocusglobal,
title={Sonic: Shifting Focus to Global Audio Perception in Portrait Animation},
author={Xiaozhong Ji and Xiaobin Hu and Zhihong Xu and Junwei Zhu and Chuming Lin and Qingdong He and Jiangning Zhang and Donghao Luo and Yi Chen and Qin Lin and Qinglin Lu and Chengjie Wang},
year={2024},
eprint={2411.16331},
archivePrefix={arXiv},
primaryClass={cs.MM},
url={https://arxiv.org/abs/2411.16331},
}