Facial expression recognition based on multi-scale feature fusion and attention mechanism

SHI Hao; XING Yuhang; CHEN Lian

doi:10.19304/J.ISSN1000-7180.2021.0799

Chinese

您当前的位置：

首页 >

文章列表页 >

Facial expression recognition based on multi-scale feature fusion and attention mechanism

更新时间：2026-09-04

- Facial expression recognition based on multi-scale feature fusion and attention mechanism
- Vol. 39, Issue 3, Pages: 34-40(2022)
- 作者机构：
  
  南昌大学信息工程学院,江西,南昌,330031
- 作者简介：
- 基金信息：
- DOI：10.19304/J.ISSN1000-7180.2021.0799
  CLC：
- Published：2022
- 稿件说明：
移动端阅览
史浩, 邢瑜航, 陈炼. 基于多尺度融合注意力机制的人脸表情识别研究[J]. 微电子学与计算机, 2022, 39(3): 34-40.

SHI Hao, XING Yuhang, CHEN Lian. Facial expression recognition based on multi-scale feature fusion and attention mechanism[J]. 2022, 39(3): 34-40.
史浩, 邢瑜航, 陈炼. 基于多尺度融合注意力机制的人脸表情识别研究[J]. 微电子学与计算机, 2022, 39(3): 34-40. DOI： 10.19304/J.ISSN1000-7180.2021.0799.

SHI Hao, XING Yuhang, CHEN Lian. Facial expression recognition based on multi-scale feature fusion and attention mechanism[J]. 2022, 39(3): 34-40. DOI： 10.19304/J.ISSN1000-7180.2021.0799.

摘要

针对传统卷积神经网络在表情特征提取阶段容易丢失大量有用信息，无法提取到高判别性表情特征，从而导致表情识别率低的问题，提出一种基于多尺度特征融合注意力机制的人脸表情识别方法.首先，采用VGGNet16来提取卷积特征.为了避免表情特征信息的丢失，将网络中不同层次卷积层的输出特征图进行多尺度特征融合，引入上下文信息的同时提取更加丰富的表情特征信息；为了能够着重关注关键表情特征，在网络中引入了注意力机制.该机制利用分组卷积操作对通道注意力模块进行改进，学习不同通道的权重信息，获取注意力特征图，增强特征的表达能力，抑制冗余信息的影响.为了进一步提高提取到表情特征的可判别性，引入孤岛损失函数，并与Softmax分类损失函数联合使用构成新的损失函数.最后，由于对全连接层进行了删减.为防止网络出现过拟合问题，在卷积层引入了DropBlock策略.实验结果表明，该模型在Fer2013和CK+数据集上分别取得了73.32%和97.40%的平均准确率.

Abstract

Aiming at the problem that traditional convolutional neural networks easily lose a lot of useful information in the expression feature extraction stage

and cannot extract high discriminative expression features

which leads to the problem of low expression recognition rate

a facial expression recognition method based on multi-scale feature fusion and attention mechanism is proposed. First

use VGGNet16 to extract convolutional features

multi-scale feature fusion is performed on the output feature maps of different layers of convolutional layers in the network

and context information is introduced while extracting richer expression feature information. In order to focus on key expression features

an attention mechanism is introduced in the network

the channel attention module is improved by grouping convolution operations

learning the weight information of different channels

obtaining attention feature maps

enhancing the expression ability of features. In order to further improve the discriminability of the extracted expression features

an island loss function is introduced

and combined with the Softmax classification loss function to form a new loss function. Finally

due to the deletion of the fully connected layer

the DropBlock strategy is introduced in the convolution layer to prevent the network from over fitting. The experimental results show that the model has achieved average accuracy rates of 73.32% and 97.40% on the Fer 2013 and CK+ datasets.

关键词

Keywords

references

LIU X Q, ZHOU F Y. Improved curriculum learning using SSM forfacialexpressionrecognition[J]. The VisualComputer, 2020, 36(8): 1635-1649. DOI: 10.1007/s00371-019-01759-7.

GHIASI G, LIN T Y, LE Q V. DropBlock: a regularization method for convolutional networks[C]//Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018. Montréal, Canada: NeurIPS, 2018: 10750-10760.

冯杨, 刘蓉, 鲁甜. 基于小尺度核卷积的人脸表情识别[J]. 计算机工程, 2021, 47(4): 262-267. DOI: 10.19678/j.issn.1000-3428.0056775. FENG Y, LIU R, LU T. Facial expression recognition based on small-scale kernel convolution[J]. Computer Engineering, 2021, 47(4): 262-267. DOI: 10.19678/j.issn.1000-3428.0056775.

刘全明, 辛阳阳. 基于卷积神经网络特征图聚类的人脸表情识别[J]. 计算技术与自动化, 2020, 39(1): 106-111. DOI: 10.16339/j.cnki.jsjsyzdh.202001022. LIU Q M, XIN Y Y. Facial expression recognition based on convolutional neural network feature map clustering[J]. Computing Technology and Automation, 2020, 39(1): 106-111. DOI: 10.16339/j.cnki.jsjsyzdh.202001022.

SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]//3rd International Conference on Learning Representations. San Diego: ICLR, 2015.

吴珂, 周梦莹, 李高阳, 等. 基于角度几何特征的人脸表情识别[J]. 计算机应用与软件, 2020, 37(7): 120-124. DOI: 10.3969/j.issn.1000-386x.2020.07.019. WU K, ZHOU M Y, LI G Y, et al. Facial expression recognition based on geometrical featuresof angles[J]. Computer Applications and Software, 2020, 37(7): 120-124. DOI: 10.3969/j.issn.1000-386x.2020.07.019.

MENG Z B, LIU P, CAI J, et al. Identity-aware convolutional neural network for facial expression recognition[C]//2017 12th IEEE International Conference on Automatic Face % Gesture Recognition (FG 2017). Washington: IEEE, 2017: 558-565. DOI: 10.1109/FG.2017.140.

TANG Y C. Deep learning using linear support vector machines[J]. arXivpreprint arXiv: 1306.0239, 2013.

ZHANG T, QI G J, XIAO B, et al. Interleaved group convolutions for deep neural networks[J]. arXivpreprint arXiv: 1707.02725, 2017.

LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110. DOI: 10.1023/B:VISI.0000029664.99615.94.

HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 7132-7141. DOI: 10.1109/CVPR.2018.00745.

DING Y Y, ZHAO Q, LI B Q, et al. Facial expression recognition from image sequence based on LBP and Taylor expansion[J]. IEEE Access, 2017(5): 19409-19419. DOI: 10.1109/ACCESS.2017.2737821.

WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the 15th European Conference ON Computer Vision (ECCV). Munich, Germany: Springer, 2018: 3-19. DOI: 10.1007/978-3-030-01234-2_1.

吕诲, 童倩倩, 袁志勇. 基于人脸分割的复杂环境下表情识别实时框架[J]. 计算机工程与应用, 2020, 56(12): 134-140. DOI: 10.3778/j.issn.1002-8331.1903-0416. LV H, TONG Q Q, YUAN Z Y. Realtime architecture for facial expression recognitionin complex scenes based on face region segmentation[J]. Computer Engineering and Applications, 2020, 56(12): 134-140. DOI: 10.3778/j.issn.1002-8331.1903-0416.

HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778. DOI: 10.1109/CVPR.2016.90.

WANG C L, LI Z R, DEY N J, et al. Histogram of oriented gradient based plantar pressure image feature extraction and classification employing fuzzy support vector machine[J]. Journal of Medical Imaging and Health Informatics, 2018, 8(4): 842-854. DOI: 10.1166/jmihi.2018.2310.

SZEGEDY C, LIU W, JIA Y Q, et al. Going deeper with convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 1-9. DOI: 10.1109/CVPR.2015.7298594.

CAI J, MENG Z B, KHAN A S, et al. Island loss for learning discriminative features in facial expression recognition[C]//Proceedings of the 13th IEEE International Conference on Automatic Face and Gesture Recognition. Xi'an, China: IEEE, 2018: 302-309. DOI: 10.1109/FG.2018.00051.

Views

105

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Single image rain removal method based on cross inference

Detection of grape leaf diseases and insect pests based on improved YOLOv3

Lightweight countermeasures against Spectre attacks on the branch prediction unit

Aerial vehicle detection algorithm based on multi-scale fusion and small-object enhancement

A review of large model development: technological evolution from BERT to Deepseek

Related Author

MA Ye

GAO Xinjian

YANG Yixin

GAO Jun

LIU Guang

HU Guoyu

GU LI BA HA ER·Tuohuti

ZHAO Tengfei

Related Institution

Hefei University of Technology

AI问答

Postal code：100079
Tel：（010）53879206 Email：tmw@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09082226号-64 京公网安备11010602201714号
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰