Automatic analysis of the human face is a vivid re-
search topic in computer vision. There are numerous
potential applications involving face recognition for se-
curity and law enforcement as well as facial expression
analysis in human computer interaction (HCI). Also au-
tomatic pain recognition from face presents a potential
application.

In general, robust facial analysis demands for an accu-
rate localisation and tracking of the face and its features
or feature points. A number of known techniques like
Deformable Templates, Statistical Models, Active Ap-
pearance Models and combined Shape Models address
this task. Edges, intensity maxima and other features
guide the matching process of such models. However,
many of them do not incorporate stereo and colour in-
formation or prior knowledge, such as calibration data
of the cameras and subject specific model data. Inte-
grating such information, not only the recognition can
greatly be improved, but also the demand for robust-
ness under varying poses and lighting conditions can be
satisfied. Common techniques often assume that the
person observed is cooperative. In many applications it
is not feasible or possible to constrain the user in order
to always acquire frontal images of the face. The work
at IESK addresses this issue.

At our institute new colour, monocular and stereo based
methods for common HCI and also clinical applica-
tions are being developed, which are able to automati-
cally detect facial expressions of emotion and pain. In
particular, by using combined 2D-3D feature extrac-
tion methods, we achieve great invariance with respect
to the so-called pose problem. With respect to 2D,
our approach uses colour, gradient and optical flow
information to extract the facial features. In the 3D
part, it includes camera models and initial registration,
in which the system automatically builds person spe-
cific face models from stereo. Photogrammetric tech-
niques are applied to determine 3D geometric measures
as features. Feature normalization is carried out and
Artificial Neural Network (ANN) and Support Vector
Machine (SVM) based classifiers are trained and ap-
plied. This leads to minimal mixing between different
facial expression classes. Our framework achieves ro-
bust and superior classification results across a variety
of head poses with resulting perspective foreshortening
and changing face size. Analysis of the feature
space demonstrates the good separation of the classes. 

(Contact: Prof. Al-Hamadi)