📝 Publications

GUAVA: Generalizable Upper Body 3D Gaussian Avatar

Dongbin Zhang, Yunfei Liu📩, Lijian Lin, Ye Zhu, Yang Li, Minghan Qin, Yu Li, Haoqian Wang📩

Project | Video

• ⚡️ Reconstructs 3D upper-body Gaussian avatars from single image in 0.1s
• ⏱️ Supports real-time expressive animation and novel view synthesis at 50FPS !

HRAvatar: High-Quality and Relightable Gaussian Head Avatar

Dongbin Zhang, Yunfei Liu, Lijian Lin, Ye Zhu, Kangjie Chen, Minghan Qin, Yu Li, Haoqian Wang

Project | Code

• We propose HRAvatar, a 3D Gaussian Splatting-based method that reconstructs high-fidelity, relightable 3D head avatars from monocular videos by jointly optimizing tracking, deformation, and appearance modeling.
• By leveraging learnable blendshapes, physically-based shading, and end-to-end optimization, HRAvatar significantly improves head quality and realism under novel lighting conditions.

TEASER: Token Enhanced Spatial Modeling for Expressions Reconstruction

Yunfei Liu📌, Lei Zhu📌, Lijian Lin, Ye Zhu, Ailing Zhang, Yu Li

Project | Code

• A novel approach that achieves more accurate facial expression reconstruction by predicting a hybrid representation of faces from a single image.
• A multi-scale facial appearance tokenizer and a token-guided neural renderer to generate high-fidelity facial images. The extracted token is interpretable and highly disentangled, enabling various downstream applications.

DiffSHEG: A Diffusion-Based Approach for Real-Time Speech-driven Holistic 3D Expression and Gesture Generation

Junming Chen, Yunfei Liu, Jianan Wang, Ailing Zeng, Yu Li, Qifeng Chen

Project

• We propose DiffSHEG, a Diffusion-based approach for Speech-driven Holistic 3D Expression and Gesture generation with arbitrary length.
• Our diffusion-based co-speech motion generation transformer enables uni-directional information flow from expression to gesture, facilitating improved matching of joint expression-gesture distributions

MODA: Mapping-Once Audio-driven Portrait Animation with Dual Attentions

Yunfei Liu, Lijian Lin, Fei Yu, Changyin Zhou, Yu Li

Project

• We propose a unified system for multi-person, diverse, and high-fidelity talking portrait video generation.
• Extensive evaluations demonstrate that the proposed system produces more natural and realistic video portraits compared to previous methods.

First- And Third-person Video Co-analysis By Learning Spatial-temporal Joint Attention

Huangyue Yu, Minjie Cai, Yunfei Liu, Feng Lu

Project | IF=17.730

• We propose a multi-branch deep network, which extracts cross-view joint attention and shared representation from static frames with spatial constraints, in a self-supervised and simultaneous manner.
• We demonstrate how the learnt joint information can benefit various applications.

GazeOnce: Real-Time Multi-Person Gaze Estimation

Mingfang Zhang, Yunfei Liu, Feng Lu

Project

• GazeOnce is the first one-stage endto-end gaze estimation method.
• This unified framework not only offers a faster speed, but also provides a lower gaze estimation error compared with other SOTA methods.

Generalizing Gaze Estimation with Outlier-guided Collaborative Adaptation

Yunfei Liu📌, Ruicong Liu📌, Haofei Wang, Feng Lu

Project

• PnP-GA is an ensemble of networks that learn collaboratively with the guidance of outliers.
• Existing gaze estimation networks can be directly plugged into PnP-GA and generalize the algorithms to new domains.

Unsupervised Learning for Intrinsic Image Decomposition from a Single Image

Yunfei Liu, Yu Li, Shaodi You, Feng Lu

Project

• USI3D is the first intrinsic image decomposition method that learns from uncorrelected image sets.
• Academic Impact: This work is included by many low-level vision projects, such as Relighting4D , IntrinsicHarmony, DIB-R++. Discussions in Zhihu.

Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks

Yunfei Liu, Xingju Ma, James Bailey, Feng Lu

[Project | (Citations 300+)]

• We present a new type of backdoor attack: natural reflection phenomenon.
• Academic Impact: This work is included by many backdoor attack/defense works, Such as NAD . This work is at the first place at google scholar .