Next Article in Journal
Application of VR Technology to the Training of Paramedics
Next Article in Special Issue
Research on Improved Deep Convolutional Generative Adversarial Networks for Insufficient Samples of Gas Turbine Rotor System Fault Diagnosis
Previous Article in Journal
Improving the Electromagnetic Vibration Energy Harvester Performance by Using a Double Coil Structure
Previous Article in Special Issue
Integrating Image Quality Enhancement Methods and Deep Learning Techniques for Remote Sensing Scene Classification
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Applied Sciences
Volume 12
Issue 3
10.3390/app12031169
applsci-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Late Fusion-Based Video Transformer for Facial Micro-Expression Recognition

by
Jiuk Hong
,
Chaehyeon Lee
and
Heechul Jung
*
Department of Artificial Intelligence, Kyungpook National University, Daegu 37224, Korea
*
Author to whom correspondence should be addressed.
Appl. Sci. 2022, 12(3), 1169; https://doi.org/10.3390/app12031169
Submission received: 6 November 2021 / Revised: 17 January 2022 / Accepted: 21 January 2022 / Published: 23 January 2022
(This article belongs to the Special Issue Advances in Deep Learning III)
Browse Figures
Versions Notes

Abstract

In this article, we propose a novel model for facial micro-expression (FME) recognition. The proposed model basically comprises a transformer, which is recently used for computer vision and has never been used for FME recognition. A transformer requires a huge amount of data compared to a convolution neural network. Then, we use motion features, such as optical flow and late fusion to complement the lack of FME dataset. The proposed method was verified and evaluated using the SMIC and CASME II datasets. Our approach achieved state-of-the-art (SOTA) performance of 0.7447 and 73.17% in SMIC in terms of unweighted F1 score (UF1) and accuracy (Acc.), respectively, which are 0.31 and 1.8% higher than previous SOTA. Furthermore, UF1 of 0.7106 and Acc. of 70.68% were shown in the CASME II experiment, which are comparable with SOTA.
Keywords: deep learning; image processing; facial micro-expression; emotion recognition; vision transformer deep learning; image processing; facial micro-expression; emotion recognition; vision transformer

Share and Cite

MDPI and ACS Style

Hong, J.; Lee, C.; Jung, H. Late Fusion-Based Video Transformer for Facial Micro-Expression Recognition. Appl. Sci. 2022, 12, 1169. https://doi.org/10.3390/app12031169

AMA Style

Hong J, Lee C, Jung H. Late Fusion-Based Video Transformer for Facial Micro-Expression Recognition. Applied Sciences. 2022; 12(3):1169. https://doi.org/10.3390/app12031169

Chicago/Turabian Style

Hong, Jiuk, Chaehyeon Lee, and Heechul Jung. 2022. "Late Fusion-Based Video Transformer for Facial Micro-Expression Recognition" Applied Sciences 12, no. 3: 1169. https://doi.org/10.3390/app12031169

APA Style

Hong, J., Lee, C., & Jung, H. (2022). Late Fusion-Based Video Transformer for Facial Micro-Expression Recognition. Applied Sciences, 12(3), 1169. https://doi.org/10.3390/app12031169

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop