Cross-Corpus Speech Emotion Recognition Based on Deep Domain-Adaptive Convolutional Neural Network

• LIU Jiateng

  Key Laboratory of Child Development and Learning Science of Ministry of Education, Southeast University

• ZHENG Wenming

  Key Laboratory of Child Development and Learning Science of Ministry of Education, Southeast University

• ZONG Yuan

  Key Laboratory of Child Development and Learning Science of Ministry of Education, Southeast University

• LU Cheng

  School of Information Science and Engineering, Southeast University

• TANG Chuangao

  Key Laboratory of Child Development and Learning Science of Ministry of Education, Southeast University

抄録

In this letter, we propose a novel deep domain-adaptive convolutional neural network (DDACNN) model to handle the challenging cross-corpus speech emotion recognition (SER) problem. The framework of the DDACNN model consists of two components: a feature extraction model based on a deep convolutional neural network (DCNN) and a domain-adaptive (DA) layer added in the DCNN utilizing the maximum mean discrepancy (MMD) criterion. We use labeled spectrograms from source speech corpus combined with unlabeled spectrograms from target speech corpus as the input of two classic DCNNs to extract the emotional features of speech, and train the model with a special mixed loss combined with a cross-entrophy loss and an MMD loss. Compared to other classic cross-corpus SER methods, the major advantage of the DDACNN model is that it can extract robust speech features which are time-frequency related by spectrograms and narrow the discrepancies between feature distribution of source corpus and target corpus to get better cross-corpus performance. Through several cross-corpus SER experiments, our DDACNN achieved the state-of-the-art performance on three public emotion speech corpora and is proved to handle the cross-corpus SER problem efficiently.

収録刊行物

• IEICE Transactions on Information and Systems

  IEICE Transactions on Information and Systems E103.D (2), 459-463, 2020-02-01

  一般社団法人 電子情報通信学会

キーワード

• cross-corpus speech emotion recognition
• deep convolutional neural network
• domain adaptation

詳細情報

• CRID: 1390283659848210816
• NII論文ID: 130007793551
• DOI: 10.1587/transinf.2019edl8136
• ISSN: 17451361
• ISSN: 09168532
• Web Site: https://www.jstage.jst.go.jp/article/transinf/E103.D/2/E103.D_2019EDL8136/_pdf
• 本文言語コード: en
• データソース種別:
  • JaLC
  • Crossref
  • CiNii Articles
• 抄録ライセンスフラグ: 使用不可