Florin Sebastian TELCEAN Fatih KAHRAMAN

Illumination Invariant Face Alignment
Fatih KAHRAMAN
Florin Sebastian TELCEAN
Informatics and Mathematical Modelling
Technical University of Denmark
Email: [email protected]
Informatics and Mathematical Modelling
Technical University of Denmark
Email: [email protected]
Introduction
Method
Face recognition systems are typically required to work under
highly varying illumination conditions. This leads to complex effects
imposed on the acquired face image that pertains little to the actual
identity. As face recognition techniques advance, more
researchers have focused on challenging issues arising from
illumination.
Summary of the searching algorithm
Warp candidate
faces
Calculate residue
btw. synthetized
and restored faces
textures
initial shape
Converge?
N
Y
Fine tuning using Ratio-image
normalization and then
performing a normal AAM
search
Restored and aligned face
used as initialization for fine
tuning module
Ratio-image light normalization
it is the quotient between a face image whose
lighting condition is to be normalized and a
reference face image. The method requires the
faces to be aligned. It is not a good candidate to
be used in the AAM [4,5] searching. In our
approach, it is used only for fine tuning
alignment.
Aim
The main idea of this method is
that two faces under the same
lighting will be similar to each
other after blurring. A restored
image can be computed from the
original one captured under an
arbitrary lighting direction, the
blurred reference image, and the
blurred original image:
I restored = I original
The aim of this study is to enhance the AAM face alignment
accuracy by using some face illumination normalization/correction
methods [1,2,3].
Figure 2: Light normalization results using ratio-image method.
Top: the input images, Bottom: the normalized images using
ratio-image method.
Breference
Boriginal
An iterative procedure is used to
obtain a final restored image with
frontal illumination. During the
iterative procedure, the reference
image is updated with the new
reconstructed image. The
iterative procedure continues until
a stopping criterion is met. The
initial reference image is the
mean face [1].
consists in fitting the histogram of a test face to the
histogram of a well-lit face (the mean face of the
model)
(a)
b)
Figure 1: Face alignment using standard AAM under good and extreme illumination. a) Normal
illumination,(b) Extreme illumination
In this project our aim is to enhance the AAM face alignment
accuracy by using two different illumination normalization/correction
methods and we analyze the ways how can we use them in order to
improve AAM face alignment. We constructed a 8-dimensional
appearance space to represent 95% of the total variation observed
in the combined coefficients.
In our experiments, we observed that the Ratio-image restoration
method [1] is not suitable for AAM searching. The main problem of
the Ratio-image method is that when it is applied to a region of an
image that is NOT face-like, the normalization result will have a lot
of information of the mean face. Thus the error will be much smaller
than the real one, and it will introduce false alarm in the searching
process.
The histogram based normalization method [2,3] will never
change the general aspect of an image. Thus the false alarms are
reduced using this normalization method. We choose to start the
searching algorithm using histogram based normalization method.
Then for this initial result we apply the normalization using Ratioimage method, and search again. This part can be seen as a fine
tuning step of the searching algorithm.
(b)
(c)
Figure 3: Example of light normalization using histogram fitting
method. (a) mean face, (b) test face, (c) light normalized test face,
and their histograms .
Conclusions and future works
In this study, a novel method is proposed to automatically align a
face image from an image captured under arbitrary lighting
conditions. The restored image can then be used for face
recognition.
• The method requires only one image with frontal illumination
of each person for training, which means it is very practical.
• There is no need to build complex models for illumination.
Histogram fitting light normalization
The face is split in two windows
(left and right);
Two mapping functions to the
mean face histogram are
separately computed for the left
and right windows. Face regions
that can be covered by hair are
avoided.
a)
proposed AAM alignment and
illumination restoration result
Proposed AAM
Typically, the Active Appearance Models are built using only one
training image for each person, taken under frontal illumination;
thus the AAM doesn’t model the illumination variations. On the
other side, face recognition systems are required to work with
illumination conditions much different than the ideal cases. In these
situations the AAM based faces alignment method normally fails.
proposed AAM alignment
result
Update model
parameters
Standard AAM
Deform shape:
scale, translation,
rotation
10 frontal images of 10 different human faces from Yale Face
Dataset (the total database consists of 200 face images) used for
training the others are used for testing. Each face is annotated with
73 corresponding points. We selected the light directions between
+/-35 degrees in the azimuth angle and +/-45 degrees in the
elevation angle.
input image
initial texture
iter #=3
iter #=6
Standard AAM
This study combines the concept of face restoration approach and
Active Appearance Model [4,5] based face alignment and develops
illumination invariant AAM method for fine face alignment.
Illumination
normalization using
histogram fitting
method
Training data
The mapping functions are
applied by gradually favor one
mapping function to another
while traveling across the image
from left to right.
Proposed AAM
Varying illumination is one of the most difficult problems and has
received much attention in recent years. It is known that image
variation due to lighting changes is larger than that due to different
personal identity. Because lighting direction changes alter the
relative gray scale distribution of face image. Consequently,
illumination normalization is required to reach acceptable
recognition rates in face recognition systems.
Initialize
AAM
Experimental Results
initial shape
initial texture
iter #=1
iter #=6
• The main advantage of the method consists in the simplicity
of light normalization algorithms used. The AAM search method
is the same, the normalization is added as a separated module.
• The method was evaluated with Yale B Face Database and IMM
Face Database. Experimental results show that the
performance of the proposed method is independent of the
database being used.
Currently, the method can be applied to face images under the
upright frontal view only. For faces with different poses, further
research is necessary to solve the combined effect of the pose and
the lighting conditions on face images.
References
Final Alignment
[1] Dang-Hui Liu, Kin-Man Lam, Lan-Sun Shen, Illumination invariant face recognition. Pattern Recognition, 38(10):
1705-1716, 2005.
Two simple but efficient illumination normalization techniques are
proposed to be integrated in the standard AAM face alignment
algorithm, in order to increase its accuracy for different illumination
conditions.
[2] R.C. Gonzalez and R.E. Woods. Digital Image Processing. Addison-Wesley Publishing Company, 1992.
Figure 4: Light normalization results using
histogram-fitting method. Top: the input
images, Bottom: the normalized images
using histogram fitting method.
[3] T. Jebara, "3D Pose Estimation and Normalization for Face Recognition", Bachelor's Thesis, McGill Centre for
Intelligent Machines, 1996.
[4] Cootes, T. F. and Edwards, G. J. and Taylor, C. J., Active Appearance Models, Proc. European Conf. On
Computer Vision, Vol. 2, pp. 484-498, 1998.
input image
initial shape
Aligned and restored
face (ratio-image)
Aligned and restored
face (histogram fitting)
[5] Mikkel B. Stegmann: The AAM-API: An Open Source Active Appearance Model Implementation. MICCAI (2)
2003: 951-952.