基于超轻量通道注意力的端对端语音增强方法

洪依; 孙成立; 冷严

doi:10.11959/j.issn.2096-6652.202136

您当前的位置：

首页 >

文章列表页 >

基于超轻量通道注意力的端对端语音增强方法

专刊：目标智能检测与识别 | 更新时间：2024-06-05

- 基于超轻量通道注意力的端对端语音增强方法
- End-to-end speech enhancement based on ultra-lightweight channel attention
- 智能科学与技术学报 2021年3卷第3期页码：351-358
- 作者机构：
  
  1. 南昌航空大学信息工程学院，江西南昌 330063
  2. 山东师范大学物理与电子科学学院，山东济南 250014
- 作者简介：
  
  [ "洪依（1997− ），女，南昌航空大学信息工程学院硕士生，主要研究方向为信号处理、语音增强、回声消除等" ]
  [ "孙成立（1975− ），男，博士后，南昌航空大学信息工程学院教授，主要研究方向为人工智能、语音信号处理、语音识别、语音增强等" ]
  [ "冷严（1982− ），女，博士，山东师范大学物理与电子科学学院副教授，主要研究方向为音频信息处理、音频分类与检测" ]
- 基金信息：
  
  国家自然科学基金资助项目;The National Natural Science Foundation of China(61861033);江西省自然科学基金重点项目;The Key Project of Natural Science Foundation of Jiangxi Province(20202ACBL202007);山东省自然科学基金资助项目;The Natural Science Foundation of Shandong Province(ZR2020MF020)
- DOI：10.11959/j.issn.2096-6652.202136
  中图分类号： TP183
- 网络出版日期：2021-09，
  
  纸质出版日期：2021-09-15
- 稿件说明：
移动端阅览
洪依, 孙成立, 冷严. 基于超轻量通道注意力的端对端语音增强方法[J]. 智能科学与技术学报, 2021,3(3):351-358.

Yi HONG, Chengli SUN, Yan LENG. End-to-end speech enhancement based on ultra-lightweight channel attention[J]. Chinese journal of intelligent science and technology, 2021, 3(3): 351-358.
洪依, 孙成立, 冷严. 基于超轻量通道注意力的端对端语音增强方法[J]. 智能科学与技术学报, 2021,3(3):351-358. DOI： 10.11959/j.issn.2096-6652.202136.

Yi HONG, Chengli SUN, Yan LENG. End-to-end speech enhancement based on ultra-lightweight channel attention[J]. Chinese journal of intelligent science and technology, 2021, 3(3): 351-358. DOI： 10.11959/j.issn.2096-6652.202136.

摘要

全卷积时域音频分离网络（Conv-TasNet）是近年提出的一种主流的端对端语音分离模型。Conv-TasNet利用膨胀卷积扩大感受野，使其在空间上可以融合更多语音特征，极大地提高了网络的语音分离性能，但同时忽略了信息在不同卷积通道间的重要性。基于此，提出一种基于超轻量通道注意力的端对端语音增强方法，该方法结合了Conv-TasNet和通道注意力，并在Conv-TasNet编解码器部分增加一组滤波器来提高网络语音特征提取能力，使卷积神经网络可以更有效地结合空间信息和通道信息来提高语音增强效果。实验验证了所提方法的模型容量在只增加了约0.02%的情况下，语音增强性能获得了有效提升。

Abstract

The full convolutional time-domain audio separation network (Conv-TasNet) is a state-of-the-art end-to-end speech separation model which was proposed recently.The Conv-TasNet used dilated convolution to expand the receptive field and fuse more speech features in space

which greatly improved the speech separation performance of the network

but at the same time ignored the importance of information across different convolution channels.An end-to-end speech enhancement method based on ultra-lightweight channel attention was proposed

which effectively combined Conv-TasNet and channel attention.At the same time

a group of filters was added to the Conv-TasNet codec to improve the speech feature extraction ability of the network.This method can make convolutional neural network combine spatial information and channel information more effectively to improve the speech enhancement effect.Experiment shows that the proposed model can effectively improve the performance of speech enhancement when the model capacity is only increased by about 0.02%.

关键词

Keywords

references

LOIZOU P . Speech enhancement:theory and practice [M ] . Boca Raton : CRC Press , 2007 .

ZHOU N , DU J , TU Y H , et al . A speech enhancement neural network architecture with SNR-progressive multi-target learning for robust speech recognition [C ] // Proceedings of the 2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference . Piscataway:IEEE Press , 2019 : 873 - 877 .

WU B , YU M , CHEN L W , et al . Distortionless multi-channel target speech enhancement for overlapped speech recognition [J ] . arXiv preprints,2020,arXiv:2007.01566 .

张钹 . 人工智能进入后深度学习时代 [J ] . 智能科学与技术学报 , 2019 , 1 ( 1 ): 4 - 6 .

ZHANG B . Artificial intelligence is entering the post deep-learning era [J ] . Chinese Journal of Intelligent Science and Technology , 2019 , 1 ( 1 ): 4 - 6 .

GRZYWALSKI T , DRGAS S . Using recurrences in time and frequency within U-Net architecture for speech enhancement [C ] // Proceedings of the 2019 IEEE International Conference on Acoustics,Speech and Signal Processing . Piscataway:IEEE Press , 2019 : 6970 - 6974 .

TAN K , WANG D L . A convolutional recurrent neural network for real-time speech enhancement [C ] // Proceedings of the INTERSPEECH 2018 .[S.l.:s.n. ] , 2018 : 3229 - 3233 .

WANG D , LIM J . The unimportance of phase in speech enhancement [J ] . IEEE Transactions on Acoustics,Speech,and Signal Processing , 1982 , 30 ( 4 ): 679 - 681 .

PALIWAL K , WÓJCICKI K , SHANNON B . The importance of phase in speech enhancement [J ] . Speech Communication , 2011 , 53 ( 4 ): 465 - 494 .

LU X G , TSAO Y , MATSUDA S , et al . Speech enhancement based on deep denoising autoencoder [C ] // Proceedings of the INTERSPEECH 2018 .[S.l.:s.n. ] , 2018 : 436 - 440 .

DEN OORD A V , DIELEMAN S , ZEN H , et al . WaveNet:a generative model for raw audio [J ] . arXiv preprint,2016,arXiv:1609.03499 .

PANDEY A , WANG D L . A new framework for CNN-based speech enhancement in the time domain [J ] . IEEE/ACM Transactions on Audio,Speech,and Language Processing , 2019 , 27 ( 7 ): 1179 - 1188 .

LUO Y , MESGARANI N . Conv-TasNet:surpassing ideal time- frequency magnitude masking for speech separation [J ] . IEEE/ACM Transactions on Audio,Speech,and Language Processing , 2019 , 27 ( 8 ): 1256 - 1266 .

BAI S J , KOLTER J Z , KOLTUN V . An empirical evaluation of generic convolutional and recurrent networks for sequence modeling [J ] . arXiv preprint,2018,arXiv:1803.01271 .

SHI Z Q , LIN H B , LIU L , et al . Deep attention gated dilated temporal convolutional networks with intra-parallel convolutional modules for end-to-end monaural speech separation [C ] // Proceedings of the INTERSPEECH 2019 .[S.l.:s.n. ] , 2019 : 3183 - 3187 .

SHI Z Q , LIN H B , LIU L , et al . End-to-end monaural speech separation with multi-scale dynamic weighted gated dilated convolutional pyramid network [C ] // Proceedings of the INTERSPEECH 2019 .[S.l.:s.n. ] , 2019 : 4614 - 4618 .

马玮良 , 彭轩 , 熊倩 , 等 . 深度学习中的内存管理问题研究综述 [J ] . 大数据 , 2020 , 6 ( 4 ): 56 - 68 .

MA W L , PENG X , XIONG Q , et al . Memory management in deep learning:a survey [J ] . Big Data Research , 2020 , 6 ( 4 ): 56 - 68 .

COSENTINO J , PARIENTE M , CORNELL S , et al . LibriMix:an open-source dataset for generalizable speech separation [J ] . arXiv preprint,2020,arXiv:2005.11262 .

HE K M , ZHANG X Y , REN S Q , et al . Delving deep into rectifiers:surpassing human-level performance on ImageNet classification [C ] // Proceedings of the 2015 IEEE International Conference on Computer Vision . Piscataway:IEEE Press , 2015 : 1026 - 1034 .

LEA C , VIDAL R , REITER A , et al . Temporal convolutional networks:a unified approach to action segmentation [M ] . Lecture notes in computer science . Cham : Springer International Publishing , 2016 .

HE K M , ZHANG X Y , REN S Q , et al . Deep residual learning for image recognition [C ] // Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2016 : 770 - 778 .

HOWARD A G , ZHU M L , CHEN B , et al . MobileNets:efficient convolutional neural networks for mobile vision applications [J ] . arXiv preprint,2017,arXiv:1704.04861 .

LUO Y , MESGARANI N . TaSNet:time-domain audio separation network for real-time,single-channel speech separation [C ] // Proceedings of the 2018 IEEE International Conference on Acoustics,Speech and Signal Processing . Piscataway:IEEE Press , 2018 : 696 - 700 .

WANG Q L , WU B G , ZHU P F , et al . ECA-Net:efficient channel attention for deep convolutional neural networks [C ] // Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2020 : 11531 - 11539 .

HU J , SHEN L , ALBANIE S , et al . Squeeze-and-excitation networks [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2020 , 42 ( 8 ): 2011 - 2023 .

SZEGEDY C , LIU W JIA Y Q , et al . Going deeper with convolutions [C ] // Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2014 .

BELL S , ZITNICK C L , BALA K , et al . Inside-Outside net:detecting objects in context with skip pooling and recurrent neural networks [C ] // Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2016 : 2874 - 2883 .

ZHANG X Y , ZHOU X Y , LIN M X , et al . ShuffleNet:an extremely efficient convolutional neural network for mobile devices [C ] // Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2018 : 6848 - 6856 .

KRIZHEVSKY A , SUTSKEVER I , HINTON G E . ImageNet classification with deep convolutional neural networks [J ] . Communications of the ACM , 2017 , 60 ( 6 ): 84 - 90 .

PANAYOTOV V , CHEN G G , POVEY D , et al . LibriSpeech:an ASR corpus based on public domain audio books [C ] // Proceedings of the 2015 IEEE International Conference on Acoustics,Speech and Signal Processing . Piscataway:IEEE Press , 2015 : 5206 - 5210 .

WICHERN G , ANTOGNINI J , FLYNN M , et al . WHAM!:extending speech separation to noisy environments [C ] // Proceedings of the INTERSPEECH 2019 .[S.l.:s.n. ] , 2019 .

ROUX J L , WISDOM S , ERDOGAN H , et al . SDR – half-baked or well done? [C ] // Proceedings of the 2019 IEEE International Conference on Acoustics,Speech and Signal Processing . Piscataway:IEEE Press , 2019 : 626 - 630 .

RIX ANTONY W , BEERENDS JOHN G , HOLLIER MICHAEL P , et al . Perceptual evaluation of speech quality (PESQ) - a new method for speech quality assessment of telephone networks and codecs [C ] // Proceedings of the 2001 IEEE International Conference on Acoustics,Speech,and Signal Processing . Piscataway:IEEE Press , 2002 : 749 - 752 .

TAAL C H , HENDRIKS R C , HEUSDENS R , et al . An algorithm for intelligibility prediction of time–frequency weighted noisy speech [J ] . IEEE Transactions on Audio Speech ＆ Language Processing , 2011 , 19 ( 7 ): 2125 - 2136 .

浏览量

779

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

暂无数据