Depth control of autonomous underwater vehicle using deep reinforcement learning

Rizhong WANG; Huiping LI; Di CUI; Demin XU

doi:10.11959/j.issn.2096-6652.202038

您当前的位置：

首页 >

文章列表页 >

Depth control of autonomous underwater vehicle using deep reinforcement learning

Special Issue: Deep Reinforcement Learning | 更新时间：2024-06-05

- Depth control of autonomous underwater vehicle using deep reinforcement learning
- Chinese Journal of Intelligent Science and Technology Vol. 2, Issue 4, Pages: 354-360(2020)
- 作者机构：
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-6652.202038
  CLC： TP242.6
- Published Online：2020-12，
  
  Published：15 December 2020
- 稿件说明：
移动端阅览
Rizhong WANG, Huiping LI, Di CUI, et al. Depth control of autonomous underwater vehicle using deep reinforcement learning[J]. Chinese journal of intelligent science and technology, 2020, 2(4): 354-360.
DOI：

Rizhong WANG, Huiping LI, Di CUI, et al. Depth control of autonomous underwater vehicle using deep reinforcement learning[J]. Chinese journal of intelligent science and technology, 2020, 2(4): 354-360. DOI： 10.11959/j.issn.2096-6652.202038.

摘要

研究了基于深度强化学习算法的自主式水下航行器（AUV）深度控制问题。区别于传统的控制算法，深度强化学习方法让航行器自主学习控制律，避免人工建立精确模型和设计控制律。采用深度确定性策略梯度方法设计了actor与critic两种神经网络。actor神经网络给出控制策略，critic神经网络用于评估该策略，AUV的深度控制可以通过训练这两个神经网络实现。在OpenAI Gym平台上仿真验证了算法的有效性。

Abstract

The depth control problem of autonomous underwater vehicle (AUV) by using deep reinforcement learning method was mainly studied.Different from the traditional control algorithm

the deep reinforcement learning method allows the AUV to learn the control law independently

avoiding the artificial establishment of accurate model and design control law.The deep deterministic policy gradient method was used to design two neural networks: actor and critic.Actor neural network enabled agents to make corresponding control actions.Critic neural network was used to estimate the action-value function in reinforcement learning.The AUV depth control was conducted by training of actor and critic neural networks.The effectiveness of the algorithm was proved by simulation on OpenAI Gym.

关键词

Keywords

references

ZHANG L J , QI X , PANG Y J . Adaptive output feedback control based on DRFNN for AUV [J ] . Ocean Engineering , 2009 , 36 ( 9-10 ): 716 - 722 .

SUTTON R , BARTO A . Reinforcement learning:an introduction [M ] . Cambridge : MIT Press , 1998 .

TESAURO G . TD-Gammon,a self-teaching backgammon program,achieves master-level play [J ] . Neural Computation , 1944 , 6 ( 2 ): 215 - 219 .

MNIH V , KAVUKCUOGLU K , SILVER D , et al . Playing atari with deep reinforcement learning [J ] . Computer Science , 2013 .

SILVER D , HUANG A , MADDISON C J , et al . Mastering the game of go with deep neural networks and tree search [J ] . Nature , 2016 , 529 ( 7587 ): 484 - 489 .

SILVER D , TECHNOLOGIES D , LEVER G , et al . Deterministic policy gradient algorithms [C ] // International Conference on Machine Learning . New York:ACM Press , 2014 .

LILLICRAP T P , HUNT J J , PRITZEL A , et al . Continuous control with deep reinforcement learning [J ] . Computer Science , 2015 , 6 ( 6 ): A187 .

HWANGBO J , LEE J , DOSOVITSKIY A , et al . Learning agile and dynamic motor skills for legged robots [J ] . Science Robotics , 2019 , 4 ( 26 ).

严卫生 . 鱼雷航行力学 [M ] . 西安 : 西北工业大学出版社 , 2005 .

YAN W S . Torpedo navigation mechanics [M ] . Xi’an : Northwestern Polytechnical University Press , 2005 .

GOODFELLOW I , BENGIO Y , COURVILLE A . Deep learning [M ] . Cambridge : MIT Press , 2016 .

KINGMA D , BA J . ADAM:a method for stochastic optimization [J ] . Computer Science , 2014 .

MNIH V , KAVUKCUOGLU K , SILVER D , et al . Human-level control through deep reinforcement learning [J ] . Nature , 2015 , 518 - 529 .

KONDA V , TSITSIKLIS J . Actor-critic algorithms [J ] . In Advances in Neural Information Processing Systems , 2003 : 1008 - 1014 .

UHLENBECK G E , ORNSTEIN L S . On the theory of the Brownian motion [J ] . Revista Latinoamericana De Microbiología , 1973 , 15 ( 1 ): 29 .

ABADI M , BARHAM P , CHEN P , et al . TensorFlow:a system for large-scale machine learning [J ] . Google Brain , 2016 .

NAIR V , HINTON G . Rectified linear units improve restricted Boltzmann machines [C ] // International Conference on Machine Learning .[S.l.:s.n. ] , 2010 .

Views

269

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Review: the application of deep reinforcement learning to quantitative trading in financial market

Research on OAC model for quantitative trading of digital currency

HVAC model-free optimal control method based on double-pools DQN

TD3-based energy management strategy for hybrid energy storage system of electric vehicle

Overview of intelligent game:enlightenment of game AI to combat deduction

Related Author

LI Xiangxia

WEN Jiancheng

HE Yijun

XU Bo

LI Xiangxia

HE Yijun

XU Bo

You LU

Related Institution

Guangdong Intelligent Business Engineering Technology Research Center

School of Information, Guangdong University of Finance & Economics

School of Information Science, Guangdong University of Finance & Economics

Key Laboratory of Performance Evolution and Control for Engineering Structures of Ministry of Education, Tongji University

School of Mechanical Engineering, Tongji University

Postal code：100079
Tel：（010）53879206 Email：cjist@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：35685 Visits Today：160

⁰