Doctoral Research Projects
A Unified Framework for Pose, Expression, and Occlusion Tolerant Automatic Facial Landmark Localization
Fusing Shape and Texture Information for Improving Facial Recognition
[Paper on IEEE Xplore] [Slides] [Poster]
Automatic Facial Landmark Tracking in Videos using Kalman Filter Assisted ASMsUtsav Prabhu) and I were able to apply ASMs to the task of tracking the previously mentioned 79 facial landmarks across the frames of different video sequences in which the subject showed rapid in-plane rotation and out of plane pose variation. This task was accomplished by Kalman filtering the landmark coordinates. The predictive mechanism of the Kalman filter ensured accurate initialization of the ASM on the next frame (without the need for face detection) while its corrective mechanism, treated the landmark locations produced by the ASM as noisy observations that were refined to produce more accurate results. We benchmarked our approach against naive methods that 1) did not harness any temporal information and treated each frame independently 2) initialized the ASM on frame n+1 using the fitting results on frame n but without Kalman filtering and found that our approach produced far lower fitting error. The applications of this work include tracking and facial recognition in surveillance footage. Our system is not currently capable of operating in real-time but can be used for post processing.
[Paper on Springer] [Slides]
Automatic Facial Landmarking using Active Shape Models (ASMs)
[Paper on IEEE Xplore] [Slides]
Graduate Course Related Projects
A Local Approach to Face Recognition - Machine Learning (10-701) - Fall 2009 Course ProjectAs part of a machine learning course, Utsav and I worked on a local approach to face recognition. Small local pataches were built around key facial landmarks and features were extracted from these patches using Gabor filters. A multi-class SVM was then used to classify images of teh various people in our test datatsets (we used subsets of the Multiple Biometrics Grand Challenge (MBGC) still face challenge database and the CMU Multi-PIE (MPIE) database). Our results were failry promising but could use more work in order to determine the optimal patch sizes, best Gabor filters as well as more research into alternative feature extraction and classification methods.
Performance of Face Recognition Algorithms on Blurred and Partially Occluded Images - Pattern Recognition Theory (18-794) - Spring 2008 Course ProjectI was part of a group that compared the effectiveness of several face recognition algorithms when applied on partially occluded or blurred images. We blured several images from the PIE database using disk blurring, Gaussian blurring, motion blurring etc as well as occluded (blacked out) different portions of the image and used these corrupted images for facial recognition. Our findings showed that Minimum Average Correleation Energy (MACE) filters were best able to deal with occluded data but did not perform as well with blurred data. LDA showed reasonable performance under both conditions and could also be used for such tasks with suitable enhancements such as the use of Kernel LDA (KLDA) etc.
Streaming Video over Wireless Networks for Eye Movement Monitoring - Wireless Networks Course (18-759) - Spring 2008 Course ProjectWas part of a team that put together a system that could wirelessly transmit images captured by a camera at 30 fps to a receiver at distances of upto 500 metres with at throughput of around 700 kbps. Our project used a Point Grey Research (PGR) firefly camera and the Real Time Transport (RTP) Protocol for the purpose.