基于对抗训练的跨语料库语音情感识别方法

薛艳飞; 张建明

Chinese

您当前的位置：

首页 >

文章列表页 >

基于对抗训练的跨语料库语音情感识别方法

更新时间：2026-09-04

- 基于对抗训练的跨语料库语音情感识别方法
- Cross-corpus speech emotion recognition based on adversarial training
- 微电子学与计算机 2021年38卷第3期页码：77-83
- 作者机构：
  
  江苏大学机构计算机科学与通信工程学院,江苏,镇江,212013
- 作者简介：
- 基金信息：
  
  国家自然科学基金面上资助项目 61672267
- DOI：
  中图分类号：
- 纸质出版：2021
- 稿件说明：
移动端阅览
薛艳飞, 张建明. 基于对抗训练的跨语料库语音情感识别方法[J]. 微电子学与计算机, 2021, 38(3): 77-83.

XUE Yan-fei, ZHANG Jian-ming. Cross-corpus speech emotion recognition based on adversarial training[J]. 2021, 38(3): 77-83.
薛艳飞, 张建明. 基于对抗训练的跨语料库语音情感识别方法[J]. 微电子学与计算机, 2021, 38(3): 77-83. DOI：

XUE Yan-fei, ZHANG Jian-ming. Cross-corpus speech emotion recognition based on adversarial training[J]. 2021, 38(3): 77-83. DOI：

摘要

在跨语料库语音情感识别中，训练和测试数据分布的差异变得非常明显，导致验证和测试性能差别很大.针对该问题，提出一种基于对抗训练的跨语料库语音情感识别方法.该方法通过语料库之间的对抗训练能有效地缩小不同语料库之间的差异，提升模型对域不变情感特征的提取能力.同时，通过引入多头自注意力机制，对语音序列中不同位置元素之间的依赖关系进行序列建模，增强序列中情感显著特征的提取能力.在以IEMOCAP为源域、MSP-IMPRO为目标域和在以MSP-IMPRO为源域、IEMOCAP为目标域上的实验表明，所提出方法的相对UAR性能相比于基准方法分别提升了0.91%~12.22%和2.27%~6.90%.因此，在目标域标注缺失的情况下，所提出的跨语料库语音情感识别方法具有更好的域不变情感显著特征的提取能力.

Abstract

The difference in data distributions becomes very clear when the training and testing data come from different corpora

causing a large performance gap between validation and testing performance. To solve this problem

a cross-corpus speech emotion recognition method based on adversarial training is proposed. The proposed method can effectively eliminate the differences between different corpora with the adversarial training of corpora

and improve the extracting ability of domain-invariant emotion features. At the same time

model the relative dependence of different position elements in the speech sequence to enhance the emotion-salient features extracting ability of the sequence by introducing the multi-head attention mechanism. When the experiment applies IEMOCAP as the source domain and MSP-IMPRO as the target domain

the results are superior to the benchmark methods about 0.91%~12.22%. Meanwhile

the experiment applies MSP-IMPRO as the source domain and IEMOCAP as the target domain

the results also achieve better performance than the benchmark methods about 2.27%~6.90%. Therefore

in the case of the absence of emotion labels of the target domain

the proposed cross-corpus speech emotion recognition method is more beneficial to extracting domain-invariant emotion salient features.

关键词

Keywords

references

VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Los Angeles, USA: Curran Associates Inc., 2017: 6000-6010.

GANIN Y, USTINOVA E, AJAKAN H, et al. Domain-adversarial training of neural networks[J]. The Journal of Machine Learning Research, 2016, 17(1): 2096-2030. DOI: 2096

MAO Q R, XU G P, XUE W T, et al. Learning emotion-discriminative and domain-invariant features for domain adaptation in speech emotion recognition[J]. Speech Communication, 20 17(93): 1- 10. DOI: 10. 10 16/j.specom.20 17.06.006.

CHANG J, SCHERER S. Learning representations of emotional speech with deep convolutional generative adversarial networks[C]//Proceedings of the 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). New Orleans, LA, USA: IEEE, 2017: 2746-2750. DOI: 10.1109/ICASSP.2017.7952656.

ABDELWAHAB M, BUSSO C. Domain adversarial for acoustic emotion recognition[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2018, 26(12): 2423-2435. DOI: 10.1109/TASLP.2018.2867099.

LEE S W. The generalization effect for multilingual speech emotion recognition across heterogeneous languages[C]//Proceedings of the 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Brighton, UK: IEEE, 2019: 5881-5885. DOI: 10.1109/ICASSP.2019.8683046.

BUSSO C, PARTHASARATHY S, BURMANIA A, et al. MSP-IMPROV: an acted corpus of dyadic interactions to study emotion perception[J]. IEEE Transactions on Affective Computing, 2017, 8(1): 67-80. DOI: 10.1109/TAFFC.2016.2515617.

GIDEON J, MCINNIS M, PROVOST E M. Improving cross-corpus speech emotion recognition with adversarial discriminative domain generalization (ADDoG)[J]. IEEE Transactions on Affective Computing, 2019. DOI: 10.1109/TAFFC.2019.2916092.

SCHULLER B, ZHANG Z X, WENINGER F, et al. Selecting training data for cross-corpus speech emotion recognition: prototypicality vs. generalization[C]//Proceedings of the 2011 Afeka-AVIOS Speech Processing Conference. Tel Aviv, Israel: TUM, 2011.

LIU N, ZONG Y, ZHANG B F, et al. Unsupervised cross-corpus speech emotion recognition using domain-adaptive subspace learning[C]//Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Calgary, AB, Canada: IEEE, 2018: 5144-5148. DOI: 10.1109/ICASSP.2018.8461848.

NEUMANN M, VU N G T. Cross-lingual and multilingual speech emotion recognition on English and French[C]//Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Calgary, AB, Canada: IEEE, 2018: 5769-5773. DOI: 10.1109/ICASSP.2018.8462162.

BUSSO C, BULUT M, LEE C C, et al. IEMOCAP: interactive emotional dyadic motion capture database[J]. Language Resources and Evaluation, 2008, 42(4): 335-359. DOI: 10.1007/s10579-008-9076-6.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

融合自注意力机制的D-BGRU文本分类模型

注意力机制与改进密集卷积引导的信息隐藏模型

面向处理器分支预测单元的Spectre攻击轻量化防御方法

基于MSS-YOLO-MPHands融合架构的装配手部动作识别研究

一种用于极低压boost系统的高效率电荷泵