End-to-end speech enhancement based on ultra-lightweight channel attention

Yi HONG; Chengli SUN; Yan LENG

doi:10.11959/j.issn.2096-6652.202136

您当前的位置：

首页 >

文章列表页 >

End-to-end speech enhancement based on ultra-lightweight channel attention

Special Issue: Intelligent Object Detection and Recognition | 更新时间：2024-06-05

- End-to-end speech enhancement based on ultra-lightweight channel attention
- Chinese Journal of Intelligent Science and Technology Vol. 3, Issue 3, Pages: 351-358(2021)
- 作者机构：
  
  1. 南昌航空大学信息工程学院，江西南昌 330063
  2. 山东师范大学物理与电子科学学院，山东济南 250014
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-6652.202136
  CLC： TP183
- Published Online：2021-09，
  
  Published：15 September 2021
- 稿件说明：
移动端阅览
Yi HONG, Chengli SUN, Yan LENG. End-to-end speech enhancement based on ultra-lightweight channel attention[J]. Chinese journal of intelligent science and technology, 2021, 3(3): 351-358.
DOI：

Yi HONG, Chengli SUN, Yan LENG. End-to-end speech enhancement based on ultra-lightweight channel attention[J]. Chinese journal of intelligent science and technology, 2021, 3(3): 351-358. DOI： 10.11959/j.issn.2096-6652.202136.

摘要

全卷积时域音频分离网络（Conv-TasNet）是近年提出的一种主流的端对端语音分离模型。Conv-TasNet利用膨胀卷积扩大感受野，使其在空间上可以融合更多语音特征，极大地提高了网络的语音分离性能，但同时忽略了信息在不同卷积通道间的重要性。基于此，提出一种基于超轻量通道注意力的端对端语音增强方法，该方法结合了Conv-TasNet和通道注意力，并在Conv-TasNet编解码器部分增加一组滤波器来提高网络语音特征提取能力，使卷积神经网络可以更有效地结合空间信息和通道信息来提高语音增强效果。实验验证了所提方法的模型容量在只增加了约0.02%的情况下，语音增强性能获得了有效提升。

Abstract

The full convolutional time-domain audio separation network (Conv-TasNet) is a state-of-the-art end-to-end speech separation model which was proposed recently.The Conv-TasNet used dilated convolution to expand the receptive field and fuse more speech features in space

which greatly improved the speech separation performance of the network

but at the same time ignored the importance of information across different convolution channels.An end-to-end speech enhancement method based on ultra-lightweight channel attention was proposed

which effectively combined Conv-TasNet and channel attention.At the same time

a group of filters was added to the Conv-TasNet codec to improve the speech feature extraction ability of the network.This method can make convolutional neural network combine spatial information and channel information more effectively to improve the speech enhancement effect.Experiment shows that the proposed model can effectively improve the performance of speech enhancement when the model capacity is only increased by about 0.02%.

关键词

Keywords

references

LOIZOU P . Speech enhancement:theory and practice [M ] . Boca Raton : CRC Press , 2007 .

ZHOU N , DU J , TU Y H , et al . A speech enhancement neural network architecture with SNR-progressive multi-target learning for robust speech recognition [C ] // Proceedings of the 2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference . Piscataway:IEEE Press , 2019 : 873 - 877 .

WU B , YU M , CHEN L W , et al . Distortionless multi-channel target speech enhancement for overlapped speech recognition [J ] . arXiv preprints,2020,arXiv:2007.01566 .

张钹 . 人工智能进入后深度学习时代 [J ] . 智能科学与技术学报 , 2019 , 1 ( 1 ): 4 - 6 .

ZHANG B . Artificial intelligence is entering the post deep-learning era [J ] . Chinese Journal of Intelligent Science and Technology , 2019 , 1 ( 1 ): 4 - 6 .

GRZYWALSKI T , DRGAS S . Using recurrences in time and frequency within U-Net architecture for speech enhancement [C ] // Proceedings of the 2019 IEEE International Conference on Acoustics,Speech and Signal Processing . Piscataway:IEEE Press , 2019 : 6970 - 6974 .

TAN K , WANG D L . A convolutional recurrent neural network for real-time speech enhancement [C ] // Proceedings of the INTERSPEECH 2018 .[S.l.:s.n. ] , 2018 : 3229 - 3233 .

WANG D , LIM J . The unimportance of phase in speech enhancement [J ] . IEEE Transactions on Acoustics,Speech,and Signal Processing , 1982 , 30 ( 4 ): 679 - 681 .

PALIWAL K , WÓJCICKI K , SHANNON B . The importance of phase in speech enhancement [J ] . Speech Communication , 2011 , 53 ( 4 ): 465 - 494 .

LU X G , TSAO Y , MATSUDA S , et al . Speech enhancement based on deep denoising autoencoder [C ] // Proceedings of the INTERSPEECH 2018 .[S.l.:s.n. ] , 2018 : 436 - 440 .

DEN OORD A V , DIELEMAN S , ZEN H , et al . WaveNet:a generative model for raw audio [J ] . arXiv preprint,2016,arXiv:1609.03499 .

PANDEY A , WANG D L . A new framework for CNN-based speech enhancement in the time domain [J ] . IEEE/ACM Transactions on Audio,Speech,and Language Processing , 2019 , 27 ( 7 ): 1179 - 1188 .

LUO Y , MESGARANI N . Conv-TasNet:surpassing ideal time- frequency magnitude masking for speech separation [J ] . IEEE/ACM Transactions on Audio,Speech,and Language Processing , 2019 , 27 ( 8 ): 1256 - 1266 .

BAI S J , KOLTER J Z , KOLTUN V . An empirical evaluation of generic convolutional and recurrent networks for sequence modeling [J ] . arXiv preprint,2018,arXiv:1803.01271 .

SHI Z Q , LIN H B , LIU L , et al . Deep attention gated dilated temporal convolutional networks with intra-parallel convolutional modules for end-to-end monaural speech separation [C ] // Proceedings of the INTERSPEECH 2019 .[S.l.:s.n. ] , 2019 : 3183 - 3187 .

SHI Z Q , LIN H B , LIU L , et al . End-to-end monaural speech separation with multi-scale dynamic weighted gated dilated convolutional pyramid network [C ] // Proceedings of the INTERSPEECH 2019 .[S.l.:s.n. ] , 2019 : 4614 - 4618 .

马玮良 , 彭轩 , 熊倩 , 等 . 深度学习中的内存管理问题研究综述 [J ] . 大数据 , 2020 , 6 ( 4 ): 56 - 68 .

MA W L , PENG X , XIONG Q , et al . Memory management in deep learning:a survey [J ] . Big Data Research , 2020 , 6 ( 4 ): 56 - 68 .

COSENTINO J , PARIENTE M , CORNELL S , et al . LibriMix:an open-source dataset for generalizable speech separation [J ] . arXiv preprint,2020,arXiv:2005.11262 .

HE K M , ZHANG X Y , REN S Q , et al . Delving deep into rectifiers:surpassing human-level performance on ImageNet classification [C ] // Proceedings of the 2015 IEEE International Conference on Computer Vision . Piscataway:IEEE Press , 2015 : 1026 - 1034 .

LEA C , VIDAL R , REITER A , et al . Temporal convolutional networks:a unified approach to action segmentation [M ] . Lecture notes in computer science . Cham : Springer International Publishing , 2016 .

HE K M , ZHANG X Y , REN S Q , et al . Deep residual learning for image recognition [C ] // Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2016 : 770 - 778 .

HOWARD A G , ZHU M L , CHEN B , et al . MobileNets:efficient convolutional neural networks for mobile vision applications [J ] . arXiv preprint,2017,arXiv:1704.04861 .

LUO Y , MESGARANI N . TaSNet:time-domain audio separation network for real-time,single-channel speech separation [C ] // Proceedings of the 2018 IEEE International Conference on Acoustics,Speech and Signal Processing . Piscataway:IEEE Press , 2018 : 696 - 700 .

WANG Q L , WU B G , ZHU P F , et al . ECA-Net:efficient channel attention for deep convolutional neural networks [C ] // Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2020 : 11531 - 11539 .

HU J , SHEN L , ALBANIE S , et al . Squeeze-and-excitation networks [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2020 , 42 ( 8 ): 2011 - 2023 .

SZEGEDY C , LIU W JIA Y Q , et al . Going deeper with convolutions [C ] // Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2014 .

BELL S , ZITNICK C L , BALA K , et al . Inside-Outside net:detecting objects in context with skip pooling and recurrent neural networks [C ] // Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2016 : 2874 - 2883 .

ZHANG X Y , ZHOU X Y , LIN M X , et al . ShuffleNet:an extremely efficient convolutional neural network for mobile devices [C ] // Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2018 : 6848 - 6856 .

KRIZHEVSKY A , SUTSKEVER I , HINTON G E . ImageNet classification with deep convolutional neural networks [J ] . Communications of the ACM , 2017 , 60 ( 6 ): 84 - 90 .

PANAYOTOV V , CHEN G G , POVEY D , et al . LibriSpeech:an ASR corpus based on public domain audio books [C ] // Proceedings of the 2015 IEEE International Conference on Acoustics,Speech and Signal Processing . Piscataway:IEEE Press , 2015 : 5206 - 5210 .

WICHERN G , ANTOGNINI J , FLYNN M , et al . WHAM!:extending speech separation to noisy environments [C ] // Proceedings of the INTERSPEECH 2019 .[S.l.:s.n. ] , 2019 .

ROUX J L , WISDOM S , ERDOGAN H , et al . SDR – half-baked or well done? [C ] // Proceedings of the 2019 IEEE International Conference on Acoustics,Speech and Signal Processing . Piscataway:IEEE Press , 2019 : 626 - 630 .

RIX ANTONY W , BEERENDS JOHN G , HOLLIER MICHAEL P , et al . Perceptual evaluation of speech quality (PESQ) - a new method for speech quality assessment of telephone networks and codecs [C ] // Proceedings of the 2001 IEEE International Conference on Acoustics,Speech,and Signal Processing . Piscataway:IEEE Press , 2002 : 749 - 752 .

TAAL C H , HENDRIKS R C , HEUSDENS R , et al . An algorithm for intelligibility prediction of time–frequency weighted noisy speech [J ] . IEEE Transactions on Audio Speech ＆ Language Processing , 2011 , 19 ( 7 ): 2125 - 2136 .

Views

779

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

Postal code：100079
Tel：（010）53879206 Email：cjist@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：35685 Visits Today：160

⁰