📝 Publications
![sym](images/papers/DiffSHEG_Demo.gif)
Junming Chen, Yunfei Liu, Jianan Wang, Ailing Zeng, Yu Li, Qifeng Chen
- We propose DiffSHEG, a Diffusion-based approach for Speech-driven Holistic 3D Expression and Gesture generation with arbitrary length.
- Our diffusion-based co-speech motion generation transformer enables uni-directional information flow from expression to gesture, facilitating improved matching of joint expression-gesture distributions
![sym](images/papers/iccv23-moda.gif)
MODA: Mapping-Once Audio-driven Portrait Animation with Dual Attentions
Yunfei Liu, Lijian Lin, Fei Yu, Changyin Zhou, Yu Li
- We propose a unified system for multi-person, diverse, and high-fidelity talking portrait video generation.
- Extensive evaluations demonstrate that the proposed system produces more natural and realistic video portraits compared to previous methods.
![sym](images/papers/tpami-23.png)
First- And Third-person Video Co-analysis By Learning Spatial-temporal Joint Attention
Huangyue Yu, Minjie Cai, Yunfei Liu, Feng Lu
Project | IF=17.730
- We propose a multi-branch deep network, which extracts cross-view joint attention and shared representation from static frames with spatial constraints, in a self-supervised and simultaneous manner.
- We demonstrate how the learnt joint information can benefit various applications.
![sym](images/papers/cvpr22-gazeonce.gif)
GazeOnce: Real-Time Multi-Person Gaze Estimation
Mingfang Zhang, Yunfei Liu, Feng Lu
- GazeOnce is the first one-stage endto-end gaze estimation method.
- This unified framework not only offers a faster speed, but also provides a lower gaze estimation error compared with other SOTA methods.
![sym](images/papers/iccv-21.png)
Generalizing Gaze Estimation with Outlier-guided Collaborative Adaptation
Yunfei Liu*, Ruicong Liu*, Haofei Wang, Feng Lu
- PnP-GA is an ensemble of networks that learn collaboratively with the guidance of outliers.
- Existing gaze estimation networks can be directly plugged into PnP-GA and generalize the algorithms to new domains.
![sym](images/papers/cvpr-20.jpg)
Unsupervised Learning for Intrinsic Image Decomposition from a Single Image
Yunfei Liu, Yu Li, Shaodi You, Feng Lu
- USI3D is the first intrinsic image decomposition method that learns from uncorrelected image sets.
- Academic Impact: This work is included by many low-level vision projects, such as Relighting4D
, IntrinsicHarmony, DIB-R++. Discussions in Zhihu.
![sym](images/papers/eccv-20.png)
Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks
Yunfei Liu, Xingju Ma, James Bailey, Feng Lu
[Project | (Citations 300+)]
- We present a new type of backdoor attack: natural reflection phenomenon.
- Academic Impact: This work is included by many backdoor attack/defense works, Such as NAD
. This work is at the first place at google scholar .
ECCV 2024
AddMe: Zero-shot Group-photo Synthesis by Inserting People into Scenes, Dongxu Yue, Maomao Li, Yunfei Liu, Ailing Zeng, Tianyu Yang, Qin Guo, Yu LiCVPR 2024
A Video is Worth 256 Bases: Spatial-Temporal Expectation-Maximization Inversion for Zero-Shot Video Editing, Maomao Li, Yu Li, Tianyu Yang, Yunfei Liu, Dongxu Yue, Zhihui Lin, Dong XuICLR 2024
GPAvatar: Generalizable and Precise Head Avatar from Image(s), Xuangeng Chu, Yu Li, Ailing Zeng, Tianyu Yang, Lijian Lin, Yunfei Liu, Tatsuya HaradaPRCV 2024
Visibility Enhancement for Low-light Hazy Scenarios, Chaoqun Zhuang, Yunfei Liu, Sijia Wen, Feng Lu.ICCV 2023
Accurate 3D Face Reconstruction with Facial Component Tokens, Tianke Zhang, Xuangeng Chu, Yunfei Liu, Lijian Lin, Zhendong Yang, et al..CVMJ 2023
Discriminative feature encoding for intrinsic image decomposition, Zhongji Wang, Yunfei Liu, Feng Lu.CVPR 2022
Generalizing Gaze Estimation with Rotation Consistency, Yiwei Bao, Yunfei Liu, Haofei Wang, Feng Lu.IEEE-VR 2022
Reconstructing 3D Virtual Face with Eye Gaze from a Single Image, Jiadong Liang, Yunfei Liu, Feng Lu. [Oral]TOMM 2022
Semantic Guided Single Image Reflection Removal, Yunfei Liu, Yu Li, Shaodi You, Feng Lu, GitHub.arXiv 2022
Jitter Does Matter: Adapting Gaze Estimation to New Domains, Mingjie Xu, Haofei Wang, Yunfei Liu, Feng Lu.ISAMR 2021
3D Photography with One-shot Portrait Relighting, Yunfei Liu, Sijia Wen, Feng Lu.ISAMR 2021
Edge-Guided Near-Eye Image Analysis for Head Mounted Displays, Zhimin Wang*, Yuxin Zhao*, Yunfei Liu, Feng Lu. [Oral] GitHub, Demo videoBMVC 2021
Separating Content and Style for Unsupervised Image-to-Image Translation, Yunfei Liu, Haofei Wang, Yang Yue, Feng Lu. GitHub.arXiv 2021
Unsupervised Two-Stage Anomaly Detection, Yunfei Liu, Chaoqun Zhuang, Feng Lu, GitHub.arXiv 2021
Cloud Sphere: A 3D Shape Representation via Progressive Deformation, Zongji Wang, Yunfei Liu, Feng Lu.arXiv 2021
Vulnerability of Appearance-based Gaze Estimation, Mingjie Xu, Haofei Wang, Yunfei Liu, Feng Lu.AAAI 2020
Separate In Latent Space: Unsupervised Single Image Layer Separation, Yunfei Liu, Feng Lu. GitHub [Oral]ICPR 2020
Adaptive Feature Fusion Network for Gaze Tracking in Mobile Tablets, Yiwei Bao, Yihua Cheng, Yunfei Liu, Feng Lu. GitHub.ACM-MM 2019
What I See Is What You See: Joint Attention Learning for First and Third Person Video Co-analysis, Huangyue Yu, Minjie Cai, Yunfei Liu, Feng Lu.