异构多智能体系统的输出同步：一个基于数据的强化学习方法

刘莹莹; 王占山

doi:10.11959/j.issn.2096-6652.202042

您当前的位置：

首页 >

文章列表页 >

异构多智能体系统的输出同步：一个基于数据的强化学习方法

专刊：深度强化学习 | 更新时间：2024-06-05

- 异构多智能体系统的输出同步：一个基于数据的强化学习方法
- Output synchronization of heterogeneous multi-agent system:a reinforcement learning approach based on data
- 智能科学与技术学报 2020年2卷第4期页码：394-400
- 作者机构：
- 作者简介：
  
  [ "刘莹莹（1995- ），女，东北大学信息科学与工程学院博士生，主要研究方向为数据驱动的系统无模型控制、基于强化学习的系统最优控制、多智能体系统最优控制等。" ]
  [ "王占山（1971- ），男，博士，东北大学信息科学与工程学院教授、博士生导师，主要研究方向为神经动力系统稳定性理论、复杂网络同步性与一致性以及数据驱动的智能故障诊断与自适应容错控制等。" ]
- 基金信息：
  
  国家自然科学基金资助项目;The National Natural Science Foundation of China(61973070);辽宁省“兴辽英才计划”资助项目;Liaoning Revitalization Talents Program(XLYC1802010);流程工业综合自动化基础研究基金资助项目;SAPI Fundamental Research Funds(2018ZCX22)
- DOI：10.11959/j.issn.2096-6652.202042
  中图分类号： TP273，TP301
- 网络出版日期：2020-12，
  
  纸质出版日期：2020-12-15
- 稿件说明：
移动端阅览
刘莹莹, 王占山. 异构多智能体系统的输出同步：一个基于数据的强化学习方法[J]. 智能科学与技术学报, 2020,2(4):394-400.

Yingying LIU, Zhanshan WANG. Output synchronization of heterogeneous multi-agent system:a reinforcement learning approach based on data[J]. Chinese journal of intelligent science and technology, 2020, 2(4): 394-400.
刘莹莹, 王占山. 异构多智能体系统的输出同步：一个基于数据的强化学习方法[J]. 智能科学与技术学报, 2020,2(4):394-400. DOI： 10.11959/j.issn.2096-6652.202042.

Yingying LIU, Zhanshan WANG. Output synchronization of heterogeneous multi-agent system:a reinforcement learning approach based on data[J]. Chinese journal of intelligent science and technology, 2020, 2(4): 394-400. DOI： 10.11959/j.issn.2096-6652.202042.

摘要

通过强化学习研究了异构多智能体系统的输出同步问题。根据多智能体系统的拓扑结构，定义一个具有邻居控制输入的性能指标和价值函数。为克服已有控制方法需要系统模型的弊端，提出一个基于系统数据的强化学习算法，使输出同步控制器也可以被应用于模型未知的情况。此外，通过调节价值函数中的权重矩阵，可以减少每个智能体的控制成本。最后，通过一个仿真示例验证了该方法的有效性和定义的价值函数的优越性。

Abstract

The output synchronization of heterogeneous multi-agent system was studied by reinforcement learning.According to the topology of multi-agent system

the performance index and value function with neighbor control input were defined.To overcome the disadvantage of existing control methods that require system model

a reinforcement learning algorithm based on system data was proposed.Hence

the output synchronization controller can also be applied when the system model was unknown.In addition

by adjusting the weight matrix in value function

the control cost of each agent can be reduced.Finally

a simulation example was given to illustrate the effectiveness of proposed method and the superiority of defined value function.

关键词

Keywords

references

LI X W , SOH Y C , XIE L H , et al . Cooperative output regulation of heterogeneous linear multi-agent networks via H ∞ performance allocation [J ] . IEEE Transactions on Automatic Control , 2019 , 64 ( 2 ): 683 - 696 .

孙长银 , 穆朝絮 . 多智能体深度强化学习的若干关键科学问题 [J ] . 自动化学报 , 2020 , 7 ( 46 ): 1301 - 1312 .

SUN C Y , MU C X . Important scientific problems of multi-agent deep reinforcement learning [J ] . Acta Automatica Sinica , 2020 , 7 ( 46 ): 1301 - 1312 .

裴国旭 , 杜晓明 , 薛昭 , 等 . 多智能体系统在军事仿真领域的应用现状 [J ] . 飞航导弹 , 2017 ( 2 ): 46 - 49 , 73 .

PEI G X , DU X M , XUE Z , et al . Application status of multi-agent system in military simulation field [J ] . Aerodynamic Missile Journal , 2017 ( 2 ): 46 - 49 , 73 .

ABDELKADER A , ABDELHAMID T . Distributed output regulation of heterogeneous linear multi-agent systems with communication constraints [J ] . Automatica , 2018 , 91 : 152 - 158 .

史乐 , 李辉 , 原江波 . 基于消息通信的多智能体系统的应用 [J ] . 计算机应用 , 2008 , 28 ( 2 ): 531 - 534 .

SHI L , LI H , YUAN J B . Multi-agent system based on the message communicaiton [J ] . Journal of Computer Applications , 2008 , 28 ( 2 ): 531 - 534 .

QIN J H , MA Q C , SHI Y , et al . Recent advances in consensus of multi-agent systems:a brief survey [J ] . IEEE Transactions on Industrial Electronics , 2017 , 64 ( 6 ): 4972 - 4983 .

ZHANG H G , JIANG H , LUO Y H , et al . Data-driven optimal consensus control for discrete-time multi-agent systems with unknown dynamics using reinforcement learning method [J ] . IEEE Transactions on Industrial Electronics , 2017 , 64 ( 5 ): 4091 - 4100 .

WANG X , YANG G H . Fault-tolerant consensus tracking control for linear multi-agent systems under switching directed network [J ] . IEEE Transactions on Cybernetics , 2020 , 50 ( 5 ): 1921 - 1930 .

ZHANG J C , ZHU F L . Observer-based output consensus of a class of heterogeneous multi-agent systems with unmatched disturbances [J ] . Communications in Nonlinear Science and Numerical Simulation , 2018 , 56 : 240 - 251 .

LIU L . Adaptive cooperative output regulation for a class of nonlinear multi-agent systems [J ] . IEEE Transactions on Automatic Control , 2015 , 60 ( 6 ): 1677 - 1682 .

MA Q , QIN J , ZHENG W X , et al . Output group synchronization for networks of heterogeneous linear systems under internal model principle [J ] . IEEE Transactions on Circuits and Systems I:Regular Papers , 2018 , 65 ( 5 ): 1684 - 1695 .

YAN Y M , HUANG J . Cooperative output regulation of discrete-time linear time-delay multi-agent systems [J ] . IET Control Theory ＆ Applications , 2016 , 10 ( 16 ): 2019 - 2026 .

王飞跃 , 曹东璞 , 魏庆来 . 强化学习:迈向知行合一的智能机制与算法 [J ] . 智能科学与技术学报 , 2020 , 2 ( 2 ): 101 - 106 .

WANG F Y , CAO D P , WEI Q L . Reinforcement learning:toward actionknowledge merged intelligent mechanisms and algorithms [J ] . Chinese Journal of Intelligent Science and Technology , 2020 , 2 ( 2 ): 101 - 106 .

LEWIS F L , VRABIE D . Reinforcement learning and adaptive dynamic programming for feedback control [J ] . IEEE Circuits and Systems Magazine , 2009 , 9 ( 3 ): 32 - 50 .

LIU Y Y , WANG Z S , SHI Z . H ∞ tracking control for linear discrete-time systems via reinforcement learning [J ] . International Journal of Robust and Nonlinear Control , 2020 , 30 ( 1 ): 282 - 301 .

MODARES H , NAGESHRAO S P , LOPES G A D , et al . Optimal model-free output synchronization of heterogeneous systems using off-policy reinforcement learning [J ] . Automatica , 2016 , 71 : 334 - 341 .

MODARES H , LEWIS F L , KANG W , et al . Optimal synchronization of heterogeneous nonlinear systems with unknown dynamics [J ] . IEEE Transactions on Automatic Control , 2018 , 63 ( 1 ): 117 - 131 .

KIUMARSI B , LEWIS F L , MODARES H , et al . Reinforcement Q-learning for optimal tracking control of linear discrete-time systems with unknown dynamics [J ] . Automatica , 2014 , 50 : 1167 - 1175 .

KIUMARSI B , LEWIS F L . Output synchronization of heterogeneous discrete-time systems:A model-free optimal approach [J ] . Automatica , 2017 , 84 : 86 - 94 .

YANG Y L , MODARES H , WUNSCH D C , et al . Leader-follower output synchronization of linear heterogeneous systems with active leader using reinforcement learning [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2018 , 29 ( 6 ): 2139 - 2153 .

浏览量

269

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于动态自选择参数共享的合作多智能体强化学习算法

基于强化学习的数据驱动多智能体系统最优一致性综述

基于改进DQN算法的复杂海战场路径规划方法

机器人自动轴孔装配研究进展

未知异构多智能体系统无模型自适应动态规划同步控制