Face Alignment Across Large Poses: A 3D Solution

Xiangyu Zhu¹ Zhen Lei¹ Xiaoming Liu² Hailin Shi¹ Stan Z. Li¹

¹Institute of Automation, Chinese Academy of Sciences

²Department of Computer Science and Engineering, Michigan State University

Face alignment, which fits a face model to an image and extracts the semantic meanings of facial pixels, has been an important topic in CV community. However, most algorithms are designed for faces in small to medium poses (below 45 degree), lacking the ability to align faces in large poses up to 90 degree. The challenges are three-fold: Firstly, the commonly used landmark-based face model assumes that all the landmarks are visible and is therefore not suitable for profile views. Secondly, the face appearance varies more dramatically across large poses, ranging from frontal view to profile view. Thirdly, labelling landmarks in large poses is extremely challenging since the invisible landmarks have to be guessed. In this paper, we propose a solution to the three problems in an new alignment framework, called 3D Dense Face Alignment (3DDFA), in which a dense 3D face model is fitted to the image via convolutional neutral network (CNN). We also propose a method to synthesize large-scale training samples in profile views to solve the third problem of data labelling. Experiments on the challenging AFLW database show that our approach achieves significant improvements over state-of-the-art methods.

Updates:

l [new] Face Profiling code updated. It now can profile samples with only 3DMM parameters.

l [new] Adding the source code of “ZBuffer” to Face Profiling.

l [new] Face Profiling code released.

l [new] 3DDFA code released.

l [new] AFLW2000-3D database released.

Downloads:

[Code]

1. [Face Profiling]: The MATLAB code to synthesize large-pose face appearances with 3D Face Model.

2. [3DDFA]: The MATLAB code to fit 3DMM to a face image across large poses.

[Data]

1. [300W-3D]: The fitted 3D Morphable Model (3DMM) parameters of 300W samples.

[300W-3D-Face]: The fitted 3D mesh, which is needed if you do not have Basel Face Model (BFM)

2. [300W-LP]: The synthesized large-pose face images from 300W.

3. [AFLW2000-3D]: The fitted 3D faces of the first 2000 AFLW samples, which can be used for 3D face alignment evaluation.