A Hierarchical Approach toward Prediction of Human Biological Age from Masked Facial Image Leveraging Deep Learning Techniques
Abstract
:1. Introduction
- We derived a synthetic face mask image dataset annotated with real age from the largest in-the-wild face image dataset, IMDB-WIKI, toward the estimation of real age amid the COVID-19 pandemic circumstances.
- We somewhat mitigated the data sparsity problem from the existing in-the-wild benchmark dataset, IMDB-WIKI.
- We proposed a hierarchical approach to estimate real ages in a group-specific manner, where the intermediate result was a group age generated by the classification model, and the final result was a real age generated by the regression-through-classification techniques generated by the group-specific models.
- We compared our hierarchical approach results with a one-off model and empirically demonstrated that a hierarchical approach is marginally better than a singular approach.
- Moreover, this research is perhaps the maiden attempt toward real age estimation from masked face images.
2. Related Work
3. Proposed Method
3.1. Data Sparsity Mitigation
- Set a threshold for choosing the image samples from each class considering the data ratio all over the age classes.
- Remove the erroneously annotated samples of each age class using human visual perception, which is shown in Figure 3.
- Enhance the image for the classes with very limited samples by (1) appending images from the very pertinent Adience dataset [64] and (2) oversampling through data augmentation operations such as flipping, rotating in the range of −30°~30° maintaining the steps of 5°, and scaling to ensure the threshold value for the class balancing.
3.2. Face Detection and Alignment
3.3. Regression via Classification
3.4. Deep Learning Models
3.5. Synthetic Masked Face Generation
3.6. Performance Metric
4. Experiments and Results
4.1. Implementation Details
4.2. Dataset
4.3. Results
5. Discussion
6. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Le, N.Q.K.; Ho, Q.-T. Deep transformers and convolutional neural network in identifying DNA N6-methyladenine sites in cross-species genomes. Methods, 2021; in press. [Google Scholar] [CrossRef]
- Le, N.Q.K.; Ho, Q.-T. Identifying SNAREs by Incorporating Deep Learning Architecture and Amino Acid Embedding Representation. Front. Physiol. 2019, 10, 1501. [Google Scholar] [CrossRef] [PubMed]
- Gawali, A.S.; Deshmukh, R.R. 3d face recognition using geodesic facial curves to handle expression, occlusion and pose variations. Int. J. Comput. Sci. Inf. Technol. 2014, 5, 4284–4287. [Google Scholar]
- Priya, G.N.; Banu, R.W. Occlusion invariant face recognition using mean based weight matrix and support vector machine. Sadhana 2014, 39, 303–315. [Google Scholar] [CrossRef]
- Alyuz, N.; Gokberk, B.; Akarun, L. 3-d face recognition under occlusion using masked projection. IEEE Trans. Inf. Forensics Secur. 2013, 8, 789–802. [Google Scholar] [CrossRef]
- Rothe, R.; Timofte, R.; Gool, L.V. Deep expectation of real and apparent age from a single image without facial landmarks. Int. J. Comput. Vis. 2018, 126, 144–157. [Google Scholar] [CrossRef] [Green Version]
- Zhang, K.; Zhang, Z.; Li, Z.; Qiao, Y. Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Process. Lett. 2016, 23, 1499–1503. [Google Scholar] [CrossRef] [Green Version]
- Taigman, Y.; Yang, M.; Ranzato, M.; Wolf, L. Deepface: Closing the gap to human-level performance in face verification. In Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 23–28 June 2014; pp. 1701–1708. [Google Scholar]
- Sun, Y.; Chen, Y.; Wang, X.; Tang, X. Deep learning face representation by joint identification-verification. Adv. Neural Inf. Process. Syst. 2014, 27, 1988–1996. [Google Scholar]
- Deng, J.; Guo, J.; Xue, N.; Zafeiriou, S. Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 15–20 June 2019; pp. 4690–4699. [Google Scholar]
- Wang, H.; Wang, Y.; Zhou, Z.; Ji, X.; Gong, D.; Zhou, J.; Li, Z.; Liu, W. Cosface: Large margin cosine loss for deep face recognition. In Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 5265–5274. [Google Scholar]
- Duan, Q.; Zhang, L. Look more into occlusion: Realistic face frontalization and recognition with boostgan. IEEE Trans. Neural Netw. Learn. Syst. 2020, 32, 214–228. [Google Scholar] [CrossRef]
- Li, Y.; Guo, K.; Lu, Y. Cropping and attention based approach for masked face recognition. Appl. Intell. 2021, 51, 3012–3025. [Google Scholar] [CrossRef]
- Woo, S.; Park, J.; Lee, J.Y.; Kweon, I.S. CBAM: Convolutional Block Attention Module. In Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 3–19. [Google Scholar]
- Punyani, P.; Gupta, R.; Kumar, A. Neural networks for facial age estimation: A survey on recent advances. Artif. Intell. Rev. 2020, 53, 3299–3347. [Google Scholar] [CrossRef]
- Farkas, L.G. Anthropometry of the Head and Face; Raven Press: Ely, MN, USA, 1994. [Google Scholar] [CrossRef]
- Cootes, T.F.; Edwards, G.J.; Taylor, C.J. Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 2001, 23, 681–685. [Google Scholar] [CrossRef] [Green Version]
- Cootes, T.F.; Taylor, C.J.; Cooper, D.H.; Graham, J. Active shape models—Their training and application. Comput. Vis. Image Underst. 1995, 61, 38–59. [Google Scholar] [CrossRef] [Green Version]
- Geng, X.; Zhou, Z.; Smith-Miles, K. Automatic age estimation based on facial aging patterns. IEEE Trans. Pattern Anal. Mach. Intell. 2007, 29, 2234–2240. [Google Scholar] [CrossRef] [Green Version]
- Fu, Y.; Xu, Y.; Huang, T.S. Estimating human ages by manifold analysis of face pictures and regression on aging features. In Proceedings of the IEEE Conference Multimedia and Expo, Beijing, China, 2–5 July 2007; pp. 1383–1386. [Google Scholar]
- Beymer, D.; Poggio, T. Image representations for visual learning. Science 1996, 272, 1905–1909. [Google Scholar] [CrossRef]
- Dornaika, F.; Bekhouche, S.; Arganda-Carreras, I. Robust regression with deep CNNs for facial age estimation: An empirical study. Expert Syst. Appl. 2020, 141, 112942. [Google Scholar] [CrossRef]
- Thukral, P.; Mitra, K.; Chellappa, R. A hierarchical approach for human age estimation. In Proceedings of the 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Kyoto, Japan, 25–30 March 2012; pp. 1529–1532. [Google Scholar] [CrossRef]
- Pontes, J.K.; Britto, A.D.; Fookes, C.; Lameiras Koerich, A. A flexible hierarchical approach for facial age estimation based on multiple features. Pattern Recognit. 2016, 54, 34–51. [Google Scholar] [CrossRef]
- Lanitis, A.; Draganova, C.; Christodoulou, C. Comparing different classifiers for automatic age estimation. IEEE Trans. Syst. Man Cybern. 2004, 34, 621–628. [Google Scholar] [CrossRef]
- Ueki, K.; Hayashida, T.; Kobayashi, T. Subspace-based age group classification using facial images under various lighting conditions. In Proceedings of the IEEE Conference on Automatic Face and Gesture Recognition, Southampton, UK, 10–12 April 2006; pp. 43–48. [Google Scholar]
- Huerta, I.; Fernandez, C.; Segura, C.; Hernando, J.; Prati, A. A deep analysis on age estimation. Pattern Recognit. Lett. 2015, 68, 239–249. [Google Scholar] [CrossRef] [Green Version]
- Guo, G.; Fu, Y.; Huang, T.S.; Dyer, C. Locally adjusted robust regression for human age estimation. In Proceedings of the IEEE Workshop on Applications of Computer Vision, Lake Tahoe, NV, USA, 12–15 March 2018; pp. 19–21. [Google Scholar] [CrossRef]
- Nguyen, D.T.; Cho, S.R.; Park, K.R. Age estimation-based soft biometrics considering optical blurring based on symmetrical sub-blocks for MLBP. Symmetry 2015, 7, 1882–1913. [Google Scholar] [CrossRef] [Green Version]
- Onifade, O.F.W.; Akinyemi, D.J. A groupwise age ranking framework for human age estimation. Int. J. Image Graph. Signal Process. 2015, 7, 1–12. [Google Scholar] [CrossRef] [Green Version]
- Guo, G.; Mu, G. Joint estimation of age, gender and ethnicity: CCA vs. PLS. In Proceedings of the IEEE Conference on Face and Gesture Recognition, Shanghai, China, 22–26 April 2013; pp. 1–6. [Google Scholar]
- Lu, J.; Tan, Y. Ordinary preserving manifold analysis for human age and head pose estimation. IEEE Trans. Hum.-Mach. Syst. 2012, 43, 249–258. [Google Scholar] [CrossRef]
- Akinyemi, J.D.; Onifade, O.F.W. An ethnic-specific age group ranking approach to facial age estimation using raw pixel features. In Proceedings of the IEEE Symposium on Technologies for Homeland Security, Waltham, MA, USA, 10–11 May 2016; pp. 1–6. [Google Scholar]
- Guo, G.; Fu, Y.; Dyer, C.; Huang, T. Image-based human age estimation by manifold learning and locally adjusted robust regression. IEEE Trans. Image Process. 2008, 17, 1178–1188. [Google Scholar] [PubMed] [Green Version]
- Guo, G.; Fu, Y.; Huang, T.S.; Dyer, C. A probabilistic fusion approach to human age prediction. In Proceedings of the IEEE in Conference on Computer Vision and Pattern Recognition-Semantic Learning and Applications Multimedia Workshop, Anchorage, AK, USA, 23–28 June 2008; pp. 1–6. [Google Scholar]
- Choi, S.E.; Lee, Y.J.; Lee, S.J.; Park, K.R.; Kim, J. Age estimation using hierarchical classifier based on global and local features. Pattern Recognit. 2011, 44, 1262–1281. [Google Scholar] [CrossRef]
- Han, H.; Charles, O.; Liu, X.; Jain, A.K. Demographic estimation from face images: Human vs. machine performance. IEEE Trans. Pattern Anal. Mach. Intell. 2015, 37, 1148–1161. [Google Scholar] [CrossRef]
- Gunay, A.; Nabiyev, V.V. Facial age estimation based on decision level fusion of AMM, LBP and Gabor features. Int. J. Adv. Comput. Sci. Appl. 2015, 6, 19–26. [Google Scholar]
- Punyani, P.; Gupta, R.; Kumar, A. Human age-estimation system based on double-level feature fusion of face and gait images. Int. J. Image Data Fusion 2018, 9, 222–236. [Google Scholar] [CrossRef]
- Yang, M.; Zhu, S.; Lv, F.; Yu, K. Correspondence driven adaptation for human profile recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, CO, USA, 20–25 June 2011; pp. 505–512. [Google Scholar]
- Wang, X.; Guo, R.; Kambhamettu, C. Deeply-learned feature for age estimation. In Proceedings of the IEEE Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA, 5–9 January 2015; pp. 534–541. [Google Scholar]
- Levi, G.; Hassner, T. Age and gender classification using convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Boston, MA, USA, 7–12 June 2015; pp. 34–42. [Google Scholar]
- Liu, X.; Li, S.; Kan, M.; Zhang, J.; Wu, S.; Liu, W.; Han, H.; Shan, S.; Chen, X. AgeNet: Deeply learned regressor and classifier for robust apparent age estimation. In Proceedings of the IEEE International Conference on Computer Vision Workshop, Santiago, Chile, 7–13 December 2015; pp. 258–266. [Google Scholar]
- Malli, R.C.; Aygun, M.; Ekenel, H.K. Apparent age estimation using ensemble of deep learning models. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 714–721. [Google Scholar]
- Chen, J.-C.; Kumar, A.; Ranjan, R.; Patel, V.M.; Alavi, A.; Chellappa, R. A cascaded convolutional neural network for age estimation of unconstrained faces. In Proceedings of the IEEE Conference on Biometrics, Theory, Applications and Systems, Niagara Falls, NY, USA, 6–9 September 2016. [Google Scholar] [CrossRef]
- Niu, Z.; Zhou, M.; Wang, L.; Gao, X.; Hua, G. Ordinal regression with multiple output CNN for age estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 4920–4928. [Google Scholar]
- Chen, S.; Zhang, C.; Dong, M.; Lee, J.; Rao, M. Using ranking-CNN for age estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017. [Google Scholar] [CrossRef]
- Li, K.; Xing, J.; Hu, W.; Maybank, S.J. D2C: Deep cumulatively and comparatively learning for human age estimation. Pattern Recognit. 2017, 66, 95–105. [Google Scholar] [CrossRef] [Green Version]
- Rodriguez, P.; Cucurull, G.; Gonfaus, J.M.; Roca, F.X.; Gonzlez, J. Age and gender recognition in the wild with deep attention. Pattern Recognit. 2017, 72, 563–571. [Google Scholar] [CrossRef]
- Duan, M.; Li, K.; Yang, C.; Li, K. A hybrid deep learning CNN–ELM for age and gender classification. Neurocomputing 2018, 275, 448–461. [Google Scholar] [CrossRef]
- Wan, J.; Tan, Z.; Lei, Z.; Guo, G.; Li, S.Z. Auxiliary demographic information assisted age estimation with cascaded structure. IEEE Trans. Cybern. 2018, 48, 2531–2541. [Google Scholar] [CrossRef] [PubMed]
- Yoo, B.; Kwak, Y.; Kim, Y.; Choi, C.; Kim, J. Deep facial age estimation using conditional multitask learning with weak label expansion. IEEE Signal Process. Lett. 2018, 25, 808–812. [Google Scholar] [CrossRef]
- Rattani, A.; Reddy, N.; Derakhshani, R. Convolutional neural networks for age classification from smart-phone based ocular images. In Proceedings of the IEEE International Joint Conference on Biometrics (IJCB), Denver, CO, USA, 1–4 October 2018; pp. 756–761. [Google Scholar]
- Taheri, S.; Toygar, O. Multi-stage age estimation using two level fusions of handcrafted and learned features on facial images. IET Biom. 2018, 8, 124–133. [Google Scholar] [CrossRef]
- Angeloni, M.; de Freitas Pereira, R.; Pedrini, H. Age Estimation From Facial Parts Using Compact Multi-Stream Convolutional Neural Networks. In Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Korea, 27–28 October 2019; pp. 3039–3045. [Google Scholar] [CrossRef]
- Hosseini, S.; Lee, S.H.; Kwon, H.J.; Koo, H.I.; Cho, N.I. Age and gender classification using wide convolutional neural network and Gabor filter. In Proceedings of the 2018 International Workshop on Advanced Image Technology (IWAIT), Chiang Mai, Thailand, 7–10 January 2018; pp. 1–3. [Google Scholar] [CrossRef]
- Savchenko, A.V. Efficient facial representations for age, gender and identity recognition in organizing photo albums using multi-output ConvNet. PeerJ Comput. Sci. 2019, 5, e197. [Google Scholar] [CrossRef] [Green Version]
- Zhang, K.; Liu, N.; Yuan, X.; Guo, X.; Gao, C.; Zhao, Z.; Ma, Z. Fine-grained age estimation in the wild with attention LSTM networks. IEEE Trans. Circuits Syst. Video Technol. 2019, 30, 3140–3152. [Google Scholar] [CrossRef] [Green Version]
- Agbo-Ajala, O.; Viriri, S. Deeply learned classifiers for age and gender predictions of unfiltered faces. Sci. World J. 2020, 2020, 1–12. [Google Scholar] [CrossRef] [Green Version]
- Liu, X.; Zou, Y.; Kuang, H.; Ma, X. Face Image Age Estimation Based on Data Augmentation and Lightweight Convolutional Neural Network. Symmetry 2020, 12, 146. [Google Scholar] [CrossRef] [Green Version]
- Liu, N.; Zhang, F.; Duan, F. Facial Age Estimation Using a Multi-Task Network Combining Classification and Regression. IEEE Access 2020, 8, 92441–92451. [Google Scholar] [CrossRef]
- Kim, Y.H.; Nam, S.H.; Park, K.R. Enhanced Cycle Generative Adversarial Network for Generating Face Images of Untrained Races and Ages for Age Estimation. IEEE Access 2021, 9, 6087–6112. [Google Scholar] [CrossRef]
- Islam, M.M.; Baek, J.-H. Deep Learning Based Real Age and Gender Estimation from Unconstrained Face Image towards Smart Store Customer Relationship Management. Appl. Sci. 2021, 11, 4549. [Google Scholar] [CrossRef]
- Eidinger, E.; Enbar, R.; Hassner, T. Age and gender estimation of unfiltered faces. IEEE Trans. Inf. Forensics Secur. 2014, 9, 2170–2179. [Google Scholar] [CrossRef]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep Residual Learning for Image Recognition. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [Google Scholar]
- Zagoruyko, S.; Komodakis, N. Wide Residual Networks. arXiv 2016, arXiv:1605.07146. [Google Scholar]
- Huang, G.; Liu, Z.; van der Maaten, L.; Weinberger, K.Q. Densely Connected Convolutional Networks. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 2261–2269. [Google Scholar] [CrossRef] [Green Version]
- Howard, A.G.; Zhu, M.; Chen, B.; Kalenichenko, D.; Wang, W.; Weyand, T.; Andreetto, M.; Adam, H. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv 2017, arXiv:1704.04861. [Google Scholar]
- Zhang, X.; Zhou, X.; Lin, M.; Sun, J. ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices. In Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 6848–6856. [Google Scholar]
- Iandola, F.N.; Moskewicz, M.W.; Ashraf, K.; Han, S.; Dally, W.J.; Keutzer, K. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size. arXiv 2016, arXiv:1602.07360. [Google Scholar]
- Kazemi, V.; Sullivan, J. One millisecond face alignment with an ensemble of regression trees. In Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 23–28 June 2014; pp. 1867–1874. [Google Scholar] [CrossRef] [Green Version]
- Krizhevsky, A.; Sutskever, I.; Hinton, G.E. Imagenet Classification with Deep Convolutional Neural Networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems (NIPS), Lake Tahoe, NV, USA, 3–6 December 2012; pp. 1097–1105. [Google Scholar]
- Deng, J.; Dong, W.; Socher, R.; Li, L.; Li, K.; Li, F.-F. ImageNet: A Large-Scale Hierarchical Image Database. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 20–25 June 2009; pp. 248–255. [Google Scholar]
- Chang, K.Y.; Chen, C.S.; Hung, Y.P. Ordinal Hyperplanes Ranker with Cost Sensitivities for Age Estimation. In Proceedings of the CVPR 2011, Colorado Springs, CO, USA, 20–25 June 2011. [Google Scholar]
- Chen, K.; Gong, S.; Xiang, T.; Change Loy, C. Cumulative Attribute Space for Age and Crowd Density Estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, 23–28 June 2013; pp. 2467–2474. [Google Scholar]
- Yu, M.; Duan, Y.; Li, Z.; Zhang, Y. Prediction of Peptide Detectability Based on CapsNet and Convolutional Block Attention Module. Int. J. Mol. Sci. 2021, 22, 12080. [Google Scholar] [CrossRef] [PubMed]
Model | Parameters | Size | Depth |
---|---|---|---|
ResNet 50 | 25.6 M | 96 MB | 50 |
Wide ResNet-50-2 | 68.9 M | 131 MB | - |
DenseNet-121 | 8.1 M | 33 MB | 121 |
MobileNet-v2 | 3.5 M | 13 MB | 53 |
ShuffleNet | 1.4 M | 5.4 MB | 50 |
SqueezeNet | 1.24 M | 5.2 MB | 18 |
Model Name | Singular Approach | Hierarchical Approach |
---|---|---|
ResNet 50 | 5.13 | 4.35 |
WideResNet | 4.81 | 4.20 |
DenseNet | 5.38 | 4.52 |
MobileNet | 5.85 | 5.25 |
ShuffleNet | 6.17 | 5.70 |
SqueezeNet | 6.88 | 6.70 |
Model Name | Child Group | Adult Group | Elder Group |
---|---|---|---|
ResNet 50 | 0.80 | 4.70 | 1.52 |
WideResNet | 0.81 | 4.41 | 1.64 |
DenseNet | 0.92 | 4.63 | 2.04 |
MobileNet | 1.03 | 5.13 | 2.49 |
ShuffleNet | 1.10 | 5.98 | 2.05 |
SqueezeNet | 1.35 | 6.41 | 3.23 |
Real Age | 25 | 14 | 72 | 93 |
---|---|---|---|---|
| | | | |
Singular/Hierarchical | 21.61/25.01 | 17.24/14.94 | 63.57/73.06 | 90.52/92.99 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Islam, M.M.; Baek, J.-H. A Hierarchical Approach toward Prediction of Human Biological Age from Masked Facial Image Leveraging Deep Learning Techniques. Appl. Sci. 2022, 12, 5306. https://doi.org/10.3390/app12115306
Islam MM, Baek J-H. A Hierarchical Approach toward Prediction of Human Biological Age from Masked Facial Image Leveraging Deep Learning Techniques. Applied Sciences. 2022; 12(11):5306. https://doi.org/10.3390/app12115306
Chicago/Turabian StyleIslam, Md. Mahbubul, and Joong-Hwan Baek. 2022. "A Hierarchical Approach toward Prediction of Human Biological Age from Masked Facial Image Leveraging Deep Learning Techniques" Applied Sciences 12, no. 11: 5306. https://doi.org/10.3390/app12115306