Real-time highly accurate face alignment using a three-stage convolutional neural regression networks
Daijin Kim
Pohang University of Science and Technology, Republic of Korea
: J Comput Eng Inf Technol
Abstract
Recently, many face alignment methods using convolutional neural networks (CNN) have been introduced due to their high accuracies. However, they do not show real-time processing due to their high computational costs. In this paper, we propose a three-stage convolutional neural regression network (CNRN) to achieve a highly accurate face alignment in the real-time. The first stage consists of one CNRN that maps the facial image into the center positions of seven facial parts such as eyes, nose, mouth, etc. We obtain 68 local facial patches by aligning the center positions of seven facial parts onto the mean shape. The second stage consists of seven independent CNRNs, where each CNRN maps the local facial patches within its facial part into their displacements of x and y direction to reach the target positions. We obtain the fitted whole facial features and make a warped facial image from them. The third stage consists of one CNRN that maps the warped facial image into the appearance error. We repeat the second and third stage until the appearance error becomes small. The proposed method is fast because it trains first the facial parts and then facial features within the facial part like a coarse to fine fitting and each CNRN is relatively simple. The proposed method is highly accurate because it trains the facial features iteratively by performing the local regression on the facial features and the global regression on the warped appearance image. In the experiments, the proposed method will yield more accurate and stable face alignment or tracking under heavy occlusion and large pose variation than the existing the state of the art methods and run in the real-time. Recent Publications: 1. Ahmad Jalal, Yeon Ho Kim, Yong Joong Kim and Daijin Kim (2017) Robust human activity recognition from depth video using spatiotemporal multi-fused features. Pattern Recognition 61:295-308. 2. Inho Choi and Daijin Kim (2017) A variety of local structure patterns and their hybridization for accurate eye detection. Pattern Recognition 61:17-432. 3. Jongju Shin and Daijin Kim (2016) Robust face alignment and tracking by combining local search and global fitting. Image and Vision Computing 51:69-83. 4. Jongju Shin and Daijin Kim (2014) Hybrid approach for facial feature detection and tracking under occlusion. IEEE Signal Processing Letters 21(12):1486-1490. 5. Bongjin Jun, Inho Choi and Daijin Kim (2013) Local transform features and hybridization for accurate face and human detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 35(6):1423-1436.
Biography
Daijin Kim received the BS degree in Electronic and Engineering from Yonsei University, Seoul, South Korea, in 1981, and the MS degree in Electrical Engineering from the Korea Advanced Institute of Science and Technology (KAIST), Taejon, 1984. In 1991, he received the PhD degree in Electrical and Computer Engineering from Syracuse University, Syracuse, NY. During 1992-1999, he was an associate professor in the Department of Computer Engineering at DongA University, Pusan, Korea. He is currently a Professor in the Department of Computer Science and Engineering at POSTECH, Pohang, Korea. His research interests include face and human analysis, machine intelligence and advanced driver assistance systems.
E-mail: dkim@postech.ac.kr
