Underwater image enhancement network based on visual Transformer with multiple loss functions fusion

Xiaofeng CONG; Jie GUI; Jun ZHANG

doi:10.11959/j.issn.2096-6652.202252

您当前的位置：

首页 >

文章列表页 >

Underwater image enhancement network based on visual Transformer with multiple loss functions fusion

Special Topic: Autonomous Underwater Vehicle | 更新时间：2024-06-05

- Underwater image enhancement network based on visual Transformer with multiple loss functions fusion
- Chinese Journal of Intelligent Science and Technology Vol. 4, Issue 4, Pages: 522-532(2022)
- 作者机构：
  
  1. 东南大学网络空间安全学院，江苏南京 210000
  2. 安徽大学人工智能学院，安徽合肥 230000
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-6652.202252
  CLC： TP391
- Published Online：2022-12，
  
  Published：15 December 2022
- 稿件说明：
移动端阅览
Xiaofeng CONG, Jie GUI, Jun ZHANG. Underwater image enhancement network based on visual Transformer with multiple loss functions fusion[J]. Chinese journal of intelligent science and technology, 2022, 4(4): 522-532.
DOI：

Xiaofeng CONG, Jie GUI, Jun ZHANG. Underwater image enhancement network based on visual Transformer with multiple loss functions fusion[J]. Chinese journal of intelligent science and technology, 2022, 4(4): 522-532. DOI： 10.11959/j.issn.2096-6652.202252.

摘要

由于水中存在光的吸收和散射现象，水下机器人拍摄到的图像存在颜色失真和对比度降低的问题。针对水下图像存在的质量退化现象，提出了一种基于视觉 Transformer 的多损失融合的方式训练水下图像增强网络。图像增强网络采用编码与解码的结构，可以采用端到端的方式进行训练。将多损失的线性组合作为总体优化目标，有效地更新水下图像增强网络的参数，包括像素损失、结构损失、边缘损失和特征损失。在两个大型水下数据集上进行了量化实验，并与7种水下图像增强算法进行对比。以峰值信噪比和结构相似性为有参考评估指标，以水下评估指标为无参考评估指标进行实验。实验结果表明，提出的水下图像增强网络能够有效地解决图像的颜色失真与对比度降低问题。

Abstract

Due to the absorption and scattering of light in water

the images captured by underwater robots suffer from color distortion and reduced contrast.Aiming at alleviating the quality degradation phenomenon of underwater images

an underwater image enhancement network based on vision Transformer that be trained with multiple losses fusion strategy was proposed.The image enhancement network adopted an encoder-decoder architecture

and could be trained in an end-to-end manner.In order to effectively update the parameters of the network for enhancing underwater images

a linear combination of various losses was adopted as the overall optimization objective

including pixel loss

structure loss

edge loss and feature loss.Quantitative experiments were carried out on two large underwater datasets

and the proposed underwater image enhancement network was compared with 7 underwater image enhancement algorithms.The full reference evaluation metrics peak signal-to-noise ratio and structural similarity were calculated in experiment

and the non-referenced metric underwater image quality measure was also computed.The experimental results showed that the proposed underwater image enhancement network could effectively deal with color distortion and contrast reduction.

关键词

Keywords

references

严浙平 , 曲思瑜 , 邢文 . 水下图像增强方法研究综述 [J ] . 智能系统学报 , 2022 , 17 ( 5 ): 860 - 873 .

YAN Z P , QU S Y , XING W . An overview of underwater image enhancement methods [J ] . CAAI Transactions on Intelligent Systems , 2022 , 17 ( 5 ): 860 - 873 .

DREWS JR P , DO NASCIMENTO E , MORAES F , et al . Transmission estimation in underwater single images [C ] // Proceedings of 2013 IEEE International Conference on Computer Vision Workshops . Piscataway:IEEE Press , 2013 : 825 - 830 .

DREWS JR P , NASCIMENTO E R , BOTELHO S S C , et al . Underwater depth estimation and image restoration based on single images [J ] . IEEE Computer Graphics and Applications , 2016 , 36 ( 2 ): 24 - 35 .

PENG Y T , COSMAN P C . Underwater image restoration based on image blurriness and light absorption [J ] . IEEE Transactions on Image Processing , 2017 , 26 ( 4 ): 1579 - 1594 .

SONG W , WANG Y , HUANG D M , et al . A rapid scene depth estimation model based on underwater light attenuation prior for underwater image restoration [C ] // Proceedings of 2018 Advances in Multimedia Information Processing . Cham:Springer , 2018 : 678 - 688 .

LIU P , WANG G Y , QI H , et al . Underwater image enhancement with a deep residual framework [J ] . IEEE Access , 2019 , 7 : 94614 - 94629 .

NAIK A , SWARNAKAR A , MITTAL K . Shallow-UWnet:compressed model for underwater image enhancement (student abstract) [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2021 , 35 ( 18 ): 15853 - 15854 .

LI C Y . Underwater scene prior inspired deep underwater image and video enhancement [J ] . Pattern Recognition , 2020 ,98.

BERMAN D , LEVY D , AVIDAN S , et al . Underwater single image color restoration using haze-lines and a new quantitative dataset [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2021 , 43 ( 8 ): 2822 - 2837 .

LIU Z , LIN Y T , CAO Y , et al . Swin Transformer:hierarchical vision transformer using shifted windows [C ] // Proceedings of 2021 IEEE/CVF International Conference on Computer Vision . Piscataway:IEEE Press , 2021 : 9992 - 10002 .

ZAMIR S W , ARORA A , KHAN S , et al . Restormer:efficient transformer for high-resolution image restoration [C ] // Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2022 : 5718 - 5729 .

DOSOVITSKIY A , BEYER L , KOLESNIKOV A , et al . An image is worth 16×16 words:transformers for image recognition at scale [J ] . arXiv preprint , 2020 ,arXiv:2010.11929.

LI J , SKINNER K A , EUSTICE R M , et al . WaterGAN:unsupervised generative network to enable real-time color correction of monocular underwater images [J ] . IEEE Robotics and Automation Letters , 2018 , 3 ( 1 ): 387 - 394 .

常戬 , 韩旭 . 结合导向滤波与自适应算子的水下图像增强 [J ] . 计算机工程与应用 , 2021 ,已录用.

CHANG J , HAN X . Underwater image enhancement combining guide filtering with adaptive operator [J ] . Computer Engineering and Applications , 2021 ,Accepted.

李颖 , 陈龙 , 黄钊宏 , 等 . 基于多尺度卷积神经网络特征融合的植株叶片检测技术 [J ] . 智能科学与技术学报 , 2021 , 3 ( 3 ): 304 - 311 .

LI Y , CHEN L , HUANG Z H , et al . Plant leaf detection technology based on multi-scale CNN feature fusion [J ] . Chinese Journal of Intelligent Science and Technology , 2021 , 3 ( 3 ): 304 - 311 .

王禾扬 , 杨启鸣 , 朱旗 . 基于深度卷积集成网络的视网膜多种疾病筛查和识别方法 [J ] . 智能科学与技术学报 , 2021 , 3 ( 3 ): 259 - 267 .

WANG H Y , YANG Q M , ZHU Q . Retinal multi-disease screening and recognition method based on deep convolution ensemble network [J ] . Chinese Journal of Intelligent Science and Technology , 2021 , 3 ( 3 ): 259 - 267 .

邵虹 , 张鸣坤 , 崔文成 . 基于分层卷积神经网络的皮肤镜图像分类方法 [J ] . 智能科学与技术学报 , 2021 , 3 ( 4 ): 474 - 481 .

SHAO H , ZHANG M K , CUI W C . Classification method of dermoscopic image based on hierarchical convolution neural network [J ] . Chinese Journal of Intelligent Science and Technology , 2021 , 3 ( 4 ): 474 - 481 .

陈龙 , 丁丹丹 . 多残差联合学习的水下图像增强 [J ] . 中国图象图形学报 , 2022 , 27 ( 5 ): 1577 - 1588 .

CHEN L , DING D D . Joint multi-residual learning for underwater image enhancement [J ] . Journal of Image and Graphics , 2022 , 27 ( 5 ): 1577 - 1588 .

范新南 , 杨鑫 , 史鹏飞 , 等 . 特征融合生成对抗网络的水下图像增强 [J ] . 计算机辅助设计与图形学学报 , 2022 , 34 ( 2 ): 264 - 272 .

FAN X N , YANG X , SHI P F , et al . Underwater image enhancement based on feature fusion generative adversaral networks [J ] . Journal of Computer-Aided Design ＆ Computer Graphics , 2022 , 34 ( 2 ): 264 - 272 .

LI C Y , GUO C L , REN W Q , et al . An underwater image enhancement benchmark dataset and beyond [J ] . IEEE Transactions on Image Processing , 2020 , 29 : 4376 - 4389 .

GUO Y C , LI H Y , ZHUANG P X . Underwater image enhancement using a multiscale dense generative adversarial network [J ] . IEEE Journal of Oceanic Engineering , 2020 , 45 ( 3 ): 862 - 870 .

FABBRI C , ISLAM M J , SATTAR J . Enhancing underwater imagery using generative adversarial networks [C ] // Proceedings of 2018 IEEE International Conference on Robotics and Automation . Piscataway:IEEE Press , 2018 : 7159 - 7165 .

BOUDIAF A , GUO Y H , GHIMIRE A , et al . Underwater image enhancement using pre-trained transformer [C ] // Proceedings of International Conference on Image Analysis and Processing .[S.l.:s.n. ] , 2022 : 480 - 488 .

VASWANI A , SHAZEER N , PARMAR N , et al . Attention is all you need [J ] . arXiv preprint , 2017 ,arXiv:1706.03762.

SONG Y D , HE Z Q , QIAN H , et al . Vision transformers for single image dehazing [J ] . arXiv preprint , 2022 ,arXiv:2204.03883.

ZHAO H , GALLO O , FROSIO I , et al . Loss functions for image restoration with neural networks [J ] . IEEE Transactions on Computational Imaging , 2017 , 3 ( 1 ): 47 - 57 .

ZHANG H , PATEL V M . Densely connected pyramid dehazing network [C ] // Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2018 : 3194 - 3203 .

JOHNSON J , ALAHI A , LI F F . Perceptual losses for real-time style transfer and super-resolution [C ] // Proceedings of 2016 European Conference on Computer Vision .[S.l.:s.n. ] , 2016 : 694 - 711 .

SIMONYAN K , ZISSERMAN A . Very deep convolutional networks for large-scale image recognition [J ] . arXiv preprint,2014 2014 ,arXiv:1409.

ISLAM M J , XIA Y Y , SATTAR J . Fast underwater image enhancement for improved visual perception [J ] . IEEE Robotics and Automation Letters , 2020 , 5 ( 2 ): 3227 - 3234 .

LI C Y , QUO J , PANG Y W , et al . Single underwater image restoration by blue-green channels dehazing and red channel correction [C ] // Proceedings of 2016 IEEE International Conference on Acoustics,Speech and Signal Processing . Piscataway:IEEE Press , 2016 : 1731 - 1735 .

WANG Y , SONG W , FORTINO G , et al . An experimental-based review of image enhancement and image restoration methods for underwater imaging [J ] . IEEE Access , 7 : 140233 - 140251 .

PANETTA K , GAO C , AGAIAN S . Human-visual-system-inspired underwater image quality measures [J ] . IEEE Journal of Oceanic Engineering , 2016 , 41 ( 3 ): 541 - 551 .

Views

671

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

A survey of deep learning-based MRI stroke lesion segmentation methods

Research on three frame difference gesture recognition method based on mixed bone features

Artificial intelligence and deep learning methods for solving differential equations: the state of the art and prospects

Cross-subject emotional EEG recognition based on multi-source domain adaptation

Research and application on combustion optimization of coal-fired boiler in thermal power plant based on artificial intelligence technology

Related Author

Tao CHEN

Junping ZHANG

Hongming SHAN

Weiyi YU

Menghua MAN

Tianhu LIU

Meilin SONG

Yongqiang ZHANG

Related Institution

Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University

School of Computer Science, Fudan University

School of Information Science and Engineering, Hebei University of Science and Technology

National Key Laboratory on Electromagnetic Environment Effects, Shijiazhuang Campus, Army Engineering University

The State Key Laboratory for Management and Control of Complex Systems, Institute of Automation, Chinese Academy of Sciences

Postal code：100079
Tel：（010）53879206 Email：cjist@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：35702 Visits Today：177

⁰