基于奖励塑造强化学习的智能导弹突防策略规划方法

罗俊仁; 刘果; 苏炯铭; 张万鹏; 陈璟

doi:10.11959/j.issn.2096-6652.202411

您当前的位置：

首页 >

文章列表页 >

基于奖励塑造强化学习的智能导弹突防策略规划方法

学术论文 | 更新时间：2024-08-06

- 基于奖励塑造强化学习的智能导弹突防策略规划方法
- Reward shaping based reinforcement learning for intelligent missile penetration attack strategy planning
- 智能科学与技术学报 2024年6卷第2期页码：189-200
- 作者机构：
  
  国防科技大学智能科学学院，湖南长沙 410073
- 作者简介：
  
  [ "罗俊仁（1989- ），男，国防科技大学智能科学学院博士生，主要研究方向为不完美信息博弈、多智能体学习等。" ]
  [ "刘果（1990- ），男，国防科技大学智能科学学院硕士生，主要研究方向为智能导弹攻防。" ]
  [ "苏炯铭（1984- ），男，博士，国防科技大学智能科学学院副研究员，主要研究方向为智能博弈。" ]
  [ "张万鹏（1981- ），男，博士，国防科技大学智能科学学院研究员、博士生导师，主要研究方向为大数据智能、智能演进等。" ]
  [ "陈璟（1972- ），男，博士，国防科技大学智能科学学院教授、博士生导师，主要研究方向为认知决策博弈、智能任务规划等。" ]
- 基金信息：
  
  国家自然科学基金项目(61806212);湖南省研究生创新项目(CX20210011)
- DOI：10.11959/j.issn.2096-6652.202411
  中图分类号： TP39
- 收稿：2023-10-22，
  
  修回：2024-04-30，
  
  纸质出版：2024-06-15
- 稿件说明：
移动端阅览
罗俊仁,刘果,苏炯铭等.基于奖励塑造强化学习的智能导弹突防策略规划方法[J].智能科学与技术学报,2024,06(02):189-200.

LUO Junren,LIU Guo,SU Jiongming,et al.Reward shaping based reinforcement learning for intelligent missile penetration attack strategy planning[J].Chinese Journal of Intelligent Science and Technology,2024,06(02):189-200.
罗俊仁,刘果,苏炯铭等.基于奖励塑造强化学习的智能导弹突防策略规划方法[J].智能科学与技术学报,2024,06(02):189-200. DOI： 10.11959/j.issn.2096-6652.202411.

LUO Junren,LIU Guo,SU Jiongming,et al.Reward shaping based reinforcement learning for intelligent missile penetration attack strategy planning[J].Chinese Journal of Intelligent Science and Technology,2024,06(02):189-200. DOI： 10.11959/j.issn.2096-6652.202411.

摘要

面向未来海上分布式作战需求，以对抗环境下分布式作战场景中智能导弹齐射突防水面舰艇为背景，首先，分析了导弹突防策略规划问题；其次，设计了基于奖励塑造强化学习的智能导弹突防策略规划方法；然后，在墨子联合作战推演系统上构建导弹突防舰艇的作战想定，实验结果表明本文方法的智能导弹突防打击成功率为79%，验证了基于奖励塑造强化学习方法的有效性；最后，经过复盘发现，奖励塑造实验涌现出智能导弹的4类对舰打击突防策略：集中迂回攻击、分散突防多向攻击、分组延时攻击、巡弋探测指引攻击。

Abstract

Facing the future requirements of distributed warfare at sea

the strategic planning of missile penetration is firstly analyzed based on the background of intelligent missile salvo penetration against surface ships in distributed warfare scenario. Secondly

a strategic planning method of intelligent missile penetration based on reward-shaping reinforcement learning is designed by using multi-class reward function. Then

the operation scenario of the missile penetration ship is constructed on the Mozi joint operation simulation system. The comparison experiment shows that the success rate of the intelligent missile penetration attack controlled by the model learned by the reward molding method is 79%

which verifies the effectiveness of the reward-based reinforcement learning method. Finally

after action review

it is found that there are emerging four kinds of penetration strategies of intelligent missiles in the reward shaping experiment

including concentrated and roundabout attack

scattered penetration multi-direction attack

group delay attack and cruise detection guide attack.

关键词

Keywords

references

于雪泳, 朱清浩. “海军打击导弹” (NSM)武器系统综述[J]. 舰船电子工程, 2013, 33(5): 27-29.

YU X Y, ZHU Q H. Summary of “navy strike missile” weapon system[J]. Ship Electronic Engineering, 2013, 33(5): 27-29.

韩光松, 王忠, 宋志峰. 弹头集群协同制导在海战场的应用[J]. 战术导弹技术, 2021(3): 35-41.

HAN G S, WANG Z, SONG Z F. Application of warhead cluster cooperative guidance in sea battlefield[J]. Tactical Missile Technology, 2021(3): 35-41.

许学春. 隐身导弹与隐身技术的应用[J]. 飞航导弹, 2013(5): 87-90.

XU X C. Stealth missile and the application of stealth technology[J]. Aerodynamic Missile Journal, 2013(5): 87-90.

杨宁, 吴艳梅. 智能化的新型反舰导弹LRASM综述[J]. 军事文摘, 2020(23): 39-43.

YANG N, WU Y M. Summary of intelligent new anti-ship missile LRASM[J]. Military Digest, 2020(23): 39-43.

李文胜, 王然辉. 美国战斧Block 4巡航导弹的改进及未来发展[J]. 飞航导弹, 2016(4): 28-31.

LI W S, WANG R H. Improvement and future development of American Tomahawk Block 4 cruise missile[J]. Aerodynamic Missile Journal, 2016(4): 28-31.

MNIH V, KAVUKCUOGLU K, SILVER D, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540): 529-533.

刘朝阳, 穆朝絮, 孙长银. 深度强化学习算法与应用研究现状综述[J]. 智能科学与技术学报, 2020, 2(4): 314-326.

LIU Z Y, MU C X, SUN C Y. An overview on algorithms and applications of deep reinforcement learning[J]. Chinese Journal of Intelligent Science and Technology, 2020, 2(4): 314-326.

VAN HASSELT H, GUEZ A, SILVER D. Deep reinforcement learning with double Q-Learning[C]//Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence. New York: ACM, 2016: 2094-2100.

WANG Z, SCHAUL T, HESSEL M, et al. Dueling network architectures for deep reinforcement learning [C]//Proceedings of the International Conference on Machine Learning. New York: ACM, 2016: 1995-2003.

SCHULMAN J, WOLSKI F, DHARIWAL P, et al. Proximal policy optimization algorithms [EB]. arXiv preprint, 2017, arXiv: 1707.06347.

BAI S X, SONG S M, LIANG S Y, et al. UAV maneuvering decision-making algorithm based on twin delayed deep deterministic policy gradient algorithm[J]. Journal of Artificial Intelligence and Technology, 2021, 2(1): 16-22.

LILLICRAP T P, HUNT J J, PRITZEL A, et al. Continuous control with deep reinforcement learning[EB]. arXiv preprint, 2015, arXiv: 1509.02971.

MNIH V, BADIA A P, MIRZA M, et al. Asynchronous methods for deep reinforcement learning[EB]. arXiv preprint, 2016, arXiv: 1602.01783.

HAARNOJA T, ZHOU A, ABBEEL P, et al. Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor[EB]. arXiv preprint, 2018, arXiv: 1801.01290.

王国岩, 曹红松, 刘鹏飞, 等. 基于墨子系统的对舰打击策略设计与分析[J]. 测试技术学报, 2023, 37(4): 304-309.

WANG G Y, CAO H S, LIU P F, et al. Design and analysis of warship strike strategy based on Mozi system[J]. Journal of Test and Measurement Technology, 2023, 37(4): 304-309.

肖友刚, 金升成, 毛晓, 等. 基于深度强化学习的舰船导弹目标分配方法[J]. 控制理论与应用, 2024, 已录用.

XIAO Y G, JIN S C, MAO X, et al. Missile-target assignment method of naval ship based on deep reinforcement learning[J]. Control Theory & Applications, 2024, accepted.

张斌, 雷永林, 李群, 等. 基于强化学习的导弹突防决策建模研究[J]. 系统仿真学报, 2024, 已录用.

ZHANG B, LEI Y L, LI Q, et al. Reinforcement learning modeling of missile penetration decision based on combat simulation[J]. Journal of System Simulation, 2024, accepted.

黄卓, 徐振, 郭健, 等. 基于分区间强化学习的集群导弹快速任务分配[J]. 控制理论与应用, 2023, 40(6): 1129-1139.

HUANG Z, XU Z, GUO J, et al. Fast task allocation for missile swarm based on sectioned reinforcement learning[J]. Control Theory & Applications, 2023, 40(6): 1129-1139.

梁星星, 马扬, 冯旸赫, 等. 基于预测编码的样本自适应行动策略规划[J]. 软件学报, 2022, 33(4): 1477-1500.

LIANG X X, MA Y, FENG Y H, et al. Sample adaptive policy planning based on predictive coding[J]. Journal of Software, 2022, 33(4): 1477-1500.

于忠琳, 张海峰, 段金升, 等. 基于饱和态势的反舰导弹齐射问题研究[J]. 兵器装备工程学报, 2021, 42(8): 46-50.

YU Z L, ZHANG H F, DUAN J S, et al. Research on anti-ship missile salvo based on saturation state[J]. Journal of Ordnance Equipment Engineering, 2021, 42(8): 46-50.

GAO K Y, YAN X B. Study on the optimal strategy of missile interception[J]. IEEE Access, 2021, 9: 22239-22252.

DONG Y L, TANG X C, YUAN Y. Principled reward shaping for reinforcement learning via Lyapunov stability theory[J]. Neurocomputing, 2020, 393: 83-90.

浏览量

328

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

Cerberus：基于深度学习的跨站社交网络机器人检测系统