Morpheus: A Neural-driven Animatronic Face with
Hybrid Actuation and Diverse Emotion Control
**RSS 2025**
Software
**Zongzheng Zhang**1,2* · **Jiawen Yang**1* · [**Ziqiao Peng**](https://ziqiaopeng.github.io/)1 ·
**Meng Yang**4 · [**Jianzhu Ma**](https://majianzhu.com/)1, **Lin Cheng**5 · [**Huazhe Xu**](http://hxu.rocks/)3 . [**Hang Zhao**](https://hangzhaomit.github.io/)3 and [**Hao Zhao**](https://sites.google.com/view/fromandto/)1,2
1 Institute for AI Industry Research (AIR), Tsinghua University, 2 Beijing Academy of Artificial Intelligence (BAAI),
3 Institute for Interdisciplinary Information Sciences(IIIS), Tsinghua University,
4 MGI Tech, Shenzhen, China, 5 Beihang University
(* indicates equal contribution)
[**RSS official**](https://roboticsconference.org/program/papers/80/) | [**Project Page**](https://jiawenyang-ch.github.io/Morpheus-Hardware-Design/) | [**arXiv**](https://arxiv.org/abs/2507.16645) | [**Hardware**](https://github.com/ZZongzheng0918/Morpheus-Hardware) | [**Software**](https://github.com/ZZongzheng0918/Morpheus-Software)
## Environment
- Linux
- Python 3.10
- Pytorch 2.3.1
- CUDA 12.1
- Blender 3.4.1
- ffmpeg 4.4.1
Clone the repo:
```bash
git clone https://github.com/ZZongzheng0918/Morpheus-Software.git
cd Morpheus-Software
```
Create conda environment:
```bash
conda create -n morpheus python=3.10
conda activate morpheus
pip install torch==2.3.1 torchvision==0.18.1 torchaudio==2.3.1 --index-url https://download.pytorch.org/whl/cu121
pip install -r requirements.txt
```
## **Demo**
Download Blender and put it in this directory.
```bash
wget https://ftp.nluug.nl/pub/graphics/blender/release/Blender3.4/blender-3.4.1-linux-x64.tar.xz
tar -xf blender-3.4.1-linux-x64.tar.xz
mv blender-3.4.1-linux-x64 blender && rm blender-3.4.1-linux-x64.tar.xz
```
Download the pretrained models from [model.pth](https://huggingface.co/ZiqiaoPeng/Morpheus). Put the pretrained models under `pretrain_model` folder.
Put the audio under `aduio` folder and run
```bash
python demo.py --wav_path "./audio/disgust.wav"
```
The generated animation will be saved in `result` folder.
## **License and Acknowledgements**
This source code is licensed under the MIT liscence found in the LICENSE file in the root directory of this repository.
## **Citation**
If you find this project useful, feel free to cite our work!
```bibtex
@article{zhang2025morpheus,
title={Morpheus: A Neural-driven Animatronic Face with Hybrid Actuation and Diverse Emotion Control},
author={Zhang, Zongzheng and Yang, Jiawen and Peng, Ziqiao and Yang, Meng and Ma, Jianzhu and Cheng, Lin and Xu, Huazhe and Zhao, Hang and Zhao, Hao},
journal={arXiv preprint arXiv:2507.16645},
year={2025}
}
```
## **Acknowledgement**
Here are some great resources we benefit:
- [EmoTalk](https://github.com/psyai-net/EmoTalk_release) for codebase
- [Faceformer](https://github.com/EvelynFan/FaceFormer) for training pipeline
- [EVP](https://github.com/jixinya/EVP) for training dataloader
- [Speech-driven-expressions](https://github.com/YoungSeng/Speech-driven-expressions) for rendering
- [Wav2Vec2 Content](https://huggingface.co/jonatasgrosman/wav2vec2-large-xlsr-53-english) and [Wav2Vec2 Emotion](https://huggingface.co/r-f/wav2vec-english-speech-emotion-recognition) for audio encoder
- [Head Template](http://filmicworlds.com/blog/solving-face-scans-for-arkit/) for visualization.
Thanks to John Hable for sharing his head template under the CC0 license, which is very helpful for us to visualize the results.