Automatic Frontal Face Pose Tracking for Face Recognition System

— Face recognition systems have been widely used in various security applications, for example in attendance system. The success of face recognition system relies on the trained face images as well as the face image captured that being recognized. Among the variables that determine the success of face recognition is face pose. Previous works showed that frontal face pose produced the best face recognition success rate. This paper proposes a face pose tracking subsystem that can be used as a filter so that only the frontal face pose that will be processed in the face recognition subsystem. The criteria for various face poses, i.e. frontal, tilted and turned, either left or right, have been formulated. Experimental results showed that the success rate of face recognition by implementing frontal face pose tracking can improve by 70.5%. However, it has trade off in reduced face image capture speed from 61 images per minute to 10 images per minute.


I. INTRODUCTION
Security system has long been an important part in various institutions, such as in industry, governments and military agencies, as well as research centers. Certain places require that only authorized people are allowed to enter for security and confidentiality reasons. The authorization system can be very simple, for example by inputting certain PIN on a keyboard or swiping a magnetic card. However, the system can also be very complex involving biometric system. Problems that often arise in the simple system are that a user may have to remember many different PINs or passwords and carry the required cards. This is somewhat troublesome and less secure [1].
Biometric system is considered more suitable for applications that require accessibility with a high level security because the information required as input for the authorization system is part of the person him/herself. In the other words, biometric system provides not only an authorization but also authentication because the people trying to gain access can be identified.
Biometric system can be grouped into biological and physical biometrics. An example of biological biometrics is DNA signature, which is an intrusive method by taking DNA samples from people who will be authenticated. This technique has a disadvantage that it takes a relatively long time to process the DNA information [2]. Physical biometrics include techniques that utilize fingerprint, iris, speech pattern, signature, and face.
Face recognition is one of the physical biometric mode that widely used in security system authentication. This technique consists of feature extraction from two dimensional image of user face and then match it with data stored in a database. The success rate of face recognition depends on the face images used as inputs for the database which were obtained from a face detection process during imaging operation. This is difficult because the imaging process may produce images with high level of variations associated with human face, such as color and texture of the skin, as well as facial expressions and poses. This is worsened by variabilities generated by the background and lighting environment [3].
Environment as an external factor can be reduced by setting constant background and lighting, while internal factors which include the user face color and texture can be considered as parts of the user facial features to be recognized. If facial expressions can be minimized, for example by asking the user to express normally when face imaging is performed, then the remaining variable is face pose. Face pose must be made standard for face recognition to obtain more accurate results. According to Chai et al. [4], angled poses resulted in lower recognition rate compare to frontal pose. Therefore frontal pose can be used as the optimum standard pose. In this paper, we develop a retrieval system that will automatically take user face image in a frontal pose.
II. THE TECHNIQUE Conventional face recognition system consists of face image acquisition, segmentation, feature extraction and face identification or verification. In this paper, we add a subsystem that provide only frontal face pose which will improve the result of face recognition. The frontal face pose tracker inserts face pose classifier preceding the feature extraction step (Fig. 1).  also be used to determine whether the face forms certain yaw angle to frontal, while vertical position of eyes and mouth can be used to determine pitch angle of the face.
Therefore, face landmarks detection is one step of the frontal face pose tracker. In this paper, the face and landmarks detection were performed using cascaded pattern recognition technique developed by Viola and Jones [5]. The complete steps of the frontal face pose tracker developed in this paper is given in Fig. 3. Following face detection, the detected face on the image acquired using digital camera is then cropped and normalized into standard face image size, which is 100x100 pixels. Normalization is done by applying scaling operation with bicubic interpolation. The criteria for each face pose were developed as follows.  Fig. 4(b). In this case, the face poses are still categorized as frontal pose. Therefore, if one of the above criteria b and c does not match, then it is still categorized as frontal face pose as long as it meets criteria a. the same. c. The horizontal distance of right eye and nose is greater than the distance between left eye and nose.

3) Tilted Pose
The tilted pose, either to the left or to right, is defined as face pose with yaw and pitch angle are zero, but the roll angle is not zero. The criteria used based on the positions of face landmarks for tilted to the left are: a. Mouth horizontal position is far more left than the position of nose b. The vertical position of left eye is much higher than the right eye. While the criteria for tilted to the right are: a. Mouth horizontal position is far more right than the position of nose. b. Right eye vertical position is much higher than the left eye.

B. Face Image Capture Speed
It should also be investigated whether the addition of frontal face pose tracker will affect the speed of face image retrieval because the addition of this feature also increases the length of code that must be executed by the face recognition system software. Speed measurements have been carried out for face image captures with two conditions, i.e: 1. Without face pose classifier. 2. With face pose classifier. Table I shows the result of face image capture speed measurements and the percentage of face pose captured during the measurement. Measurements were taken each for one minute. In this experiment, we did not distinguish between the frontal pose with the optimum frontal pose, as well as the direction of tilt and turn, whether to the right or to the left. The condition of "undetermined" in the system implementing frontal face pose tracker means that the captured face image does not meet all the above six criteria. The experimental results show that the speed of face image retrieval decreases 21.3%, from 61 images per minute to 48 images per minute, due to the implementation of frontal face pose tracker. Moreover, if we count only the frontal pose, then it decreases even higher, i.e. from 61 images per minute to only 10 images per minute, or a decrease of 83.6%.
On conventional systems, which do not have frontal face pose tracker, all the captured face images will be processed by the face recognition subsystem because the systems do not differentiate between frontal pose and other poses. This certainly will contribute errors in face recognition because the frontal face image are caught only by 29.5%. Assuming that all frontal face images would be recognized properly while others give errors in the face recognition, the success rate of face recognition without frontal face pose tracker is only 29.5%, although the capture speed is higher, i.e. 61 images per minute. While the system equipped with frontal face pose tracker, although only achieve a speed of 10 images per minute, but its success rate in face recognition is 100%, under the above assumption, because only frontal face images will be fed to the face recognition subsystem. Thus, there is an increase of 70.5% recognition rate. Hence, there is a trade-off between face image capture speed with face recognition success rate.

IV. CONCLUSIONS
A frontal face pose tracking subsystem has been developed for face recognition systems. This subsystem captures facial images from a digital camera, perform segmentation and normalization, and determine the face pose based on certain criteria. The facial pose criteria formulated in this research is the frontal, tilt and turn poses. From the experiments, the success rate of face recognition can be expected to increase by 70.5%, at the expense of face image retrieval speed decreases from 61 images per minute to 10 images per minute.