Output synchronization of heterogeneous multi-agent system:a reinforcement learning approach based on data

Yingying LIU; Zhanshan WANG

doi:10.11959/j.issn.2096-6652.202042

您当前的位置：

首页 >

文章列表页 >

Output synchronization of heterogeneous multi-agent system:a reinforcement learning approach based on data

Special Issue: Deep Reinforcement Learning | 更新时间：2024-06-05

- Output synchronization of heterogeneous multi-agent system:a reinforcement learning approach based on data
- Chinese Journal of Intelligent Science and Technology Vol. 2, Issue 4, Pages: 394-400(2020)
- 作者机构：
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-6652.202042
  CLC： TP273，TP301
- Published Online：2020-12，
  
  Published：15 December 2020
- 稿件说明：
移动端阅览
Yingying LIU, Zhanshan WANG. Output synchronization of heterogeneous multi-agent system:a reinforcement learning approach based on data[J]. Chinese journal of intelligent science and technology, 2020, 2(4): 394-400.
DOI：

Yingying LIU, Zhanshan WANG. Output synchronization of heterogeneous multi-agent system:a reinforcement learning approach based on data[J]. Chinese journal of intelligent science and technology, 2020, 2(4): 394-400. DOI： 10.11959/j.issn.2096-6652.202042.

摘要

通过强化学习研究了异构多智能体系统的输出同步问题。根据多智能体系统的拓扑结构，定义一个具有邻居控制输入的性能指标和价值函数。为克服已有控制方法需要系统模型的弊端，提出一个基于系统数据的强化学习算法，使输出同步控制器也可以被应用于模型未知的情况。此外，通过调节价值函数中的权重矩阵，可以减少每个智能体的控制成本。最后，通过一个仿真示例验证了该方法的有效性和定义的价值函数的优越性。

Abstract

The output synchronization of heterogeneous multi-agent system was studied by reinforcement learning.According to the topology of multi-agent system

the performance index and value function with neighbor control input were defined.To overcome the disadvantage of existing control methods that require system model

a reinforcement learning algorithm based on system data was proposed.Hence

the output synchronization controller can also be applied when the system model was unknown.In addition

by adjusting the weight matrix in value function

the control cost of each agent can be reduced.Finally

a simulation example was given to illustrate the effectiveness of proposed method and the superiority of defined value function.

关键词

Keywords

references

LI X W , SOH Y C , XIE L H , et al . Cooperative output regulation of heterogeneous linear multi-agent networks via H ∞ performance allocation [J ] . IEEE Transactions on Automatic Control , 2019 , 64 ( 2 ): 683 - 696 .

孙长银 , 穆朝絮 . 多智能体深度强化学习的若干关键科学问题 [J ] . 自动化学报 , 2020 , 7 ( 46 ): 1301 - 1312 .

SUN C Y , MU C X . Important scientific problems of multi-agent deep reinforcement learning [J ] . Acta Automatica Sinica , 2020 , 7 ( 46 ): 1301 - 1312 .

裴国旭 , 杜晓明 , 薛昭 , 等 . 多智能体系统在军事仿真领域的应用现状 [J ] . 飞航导弹 , 2017 ( 2 ): 46 - 49 , 73 .

PEI G X , DU X M , XUE Z , et al . Application status of multi-agent system in military simulation field [J ] . Aerodynamic Missile Journal , 2017 ( 2 ): 46 - 49 , 73 .

ABDELKADER A , ABDELHAMID T . Distributed output regulation of heterogeneous linear multi-agent systems with communication constraints [J ] . Automatica , 2018 , 91 : 152 - 158 .

史乐 , 李辉 , 原江波 . 基于消息通信的多智能体系统的应用 [J ] . 计算机应用 , 2008 , 28 ( 2 ): 531 - 534 .

SHI L , LI H , YUAN J B . Multi-agent system based on the message communicaiton [J ] . Journal of Computer Applications , 2008 , 28 ( 2 ): 531 - 534 .

QIN J H , MA Q C , SHI Y , et al . Recent advances in consensus of multi-agent systems:a brief survey [J ] . IEEE Transactions on Industrial Electronics , 2017 , 64 ( 6 ): 4972 - 4983 .

ZHANG H G , JIANG H , LUO Y H , et al . Data-driven optimal consensus control for discrete-time multi-agent systems with unknown dynamics using reinforcement learning method [J ] . IEEE Transactions on Industrial Electronics , 2017 , 64 ( 5 ): 4091 - 4100 .

WANG X , YANG G H . Fault-tolerant consensus tracking control for linear multi-agent systems under switching directed network [J ] . IEEE Transactions on Cybernetics , 2020 , 50 ( 5 ): 1921 - 1930 .

ZHANG J C , ZHU F L . Observer-based output consensus of a class of heterogeneous multi-agent systems with unmatched disturbances [J ] . Communications in Nonlinear Science and Numerical Simulation , 2018 , 56 : 240 - 251 .

LIU L . Adaptive cooperative output regulation for a class of nonlinear multi-agent systems [J ] . IEEE Transactions on Automatic Control , 2015 , 60 ( 6 ): 1677 - 1682 .

MA Q , QIN J , ZHENG W X , et al . Output group synchronization for networks of heterogeneous linear systems under internal model principle [J ] . IEEE Transactions on Circuits and Systems I:Regular Papers , 2018 , 65 ( 5 ): 1684 - 1695 .

YAN Y M , HUANG J . Cooperative output regulation of discrete-time linear time-delay multi-agent systems [J ] . IET Control Theory ＆ Applications , 2016 , 10 ( 16 ): 2019 - 2026 .

王飞跃 , 曹东璞 , 魏庆来 . 强化学习:迈向知行合一的智能机制与算法 [J ] . 智能科学与技术学报 , 2020 , 2 ( 2 ): 101 - 106 .

WANG F Y , CAO D P , WEI Q L . Reinforcement learning:toward actionknowledge merged intelligent mechanisms and algorithms [J ] . Chinese Journal of Intelligent Science and Technology , 2020 , 2 ( 2 ): 101 - 106 .

LEWIS F L , VRABIE D . Reinforcement learning and adaptive dynamic programming for feedback control [J ] . IEEE Circuits and Systems Magazine , 2009 , 9 ( 3 ): 32 - 50 .

LIU Y Y , WANG Z S , SHI Z . H ∞ tracking control for linear discrete-time systems via reinforcement learning [J ] . International Journal of Robust and Nonlinear Control , 2020 , 30 ( 1 ): 282 - 301 .

MODARES H , NAGESHRAO S P , LOPES G A D , et al . Optimal model-free output synchronization of heterogeneous systems using off-policy reinforcement learning [J ] . Automatica , 2016 , 71 : 334 - 341 .

MODARES H , LEWIS F L , KANG W , et al . Optimal synchronization of heterogeneous nonlinear systems with unknown dynamics [J ] . IEEE Transactions on Automatic Control , 2018 , 63 ( 1 ): 117 - 131 .

KIUMARSI B , LEWIS F L , MODARES H , et al . Reinforcement Q-learning for optimal tracking control of linear discrete-time systems with unknown dynamics [J ] . Automatica , 2014 , 50 : 1167 - 1175 .

KIUMARSI B , LEWIS F L . Output synchronization of heterogeneous discrete-time systems:A model-free optimal approach [J ] . Automatica , 2017 , 84 : 86 - 94 .

YANG Y L , MODARES H , WUNSCH D C , et al . Leader-follower output synchronization of linear heterogeneous systems with active leader using reinforcement learning [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2018 , 29 ( 6 ): 2139 - 2153 .

Views

269

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

A cooperative multi-agent reinforcement learning algorithm based on dynamic self-selection parameters sharing

An overview of optimal consensus for data driven multi-agent system based on reinforcement learning

A path planning method for complex naval battle field based on an improved DQN algorithm

Research development on automated robotic peg-in-hole assembly

Synchronization control of unknown heterogeneous multi-agent system via model-free adaptive dynamic programming

Related Author

Yuan JIANG

Yang YU

Han WANG

Jinna LI

Weiran CHENG

Haitao YUAN

Jing BI

Zhou YU

Related Institution

Faculty of Information Technology, Beijing University of Technology

School of Automation Science and Electrical Engineering, Beihang University

School of Automation and Electrical Engineering, University of Science and Technology Beijing

Research Institute of Urbanization and Urban Safety, University of Science and Technology Beijing

Center for Intelligent and Networked Systems, Department of Automation, Tsinghua University

Postal code：100079
Tel：（010）53879206 Email：cjist@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：35685 Visits Today：160

⁰