Publishing Service

Polishing & Checking

Frontiers of Information Technology & Electronic Engineering

ISSN 2095-9184 (print), ISSN 2095-9230 (online)

Decentralized multi-agent reinforcement learning with networked agents: recent advances

Abstract: Multi-agent reinforcement learning (MARL) has long been a significant research topic in both machine learning and control systems. Recent development of (single-agent) deep reinforcement learning has created a resurgence of interest in developing new MARL algorithms, especially those founded on theoretical analysis. In this paper, we review recent advances on a sub-area of this topic: decentralized MARL with networked agents. In this scenario, multiple agents perform sequential decision-making in a common environment, and without the coordination of any central controller, while being allowed to exchange information with their neighbors over a communication network. Such a setting finds broad applications in the control and operation of robots, unmanned vehicles, mobile sensor networks, and the smart grid. This review covers several of our research endeavors in this direction, as well as progress made by other researchers along the line. We hope that this review promotes additional research efforts in this exciting yet challenging area.

Key words: Reinforcement learning, Multi-agent systems, Networked systems, Consensus optimization, Distributed optimization, Game theory

Chinese Summary  <26> 带有网络智能体的去中心化多智能体强化学习进展

张凯清1,杨卓然2,Tamer Başar1
1伊利诺伊大学香槟分校协调科学实验室,美国伊利诺伊州,61801
2普林斯顿大学运筹学与金融工程系,美国新泽西州,08544
摘要:多智能体强化学习长期以来一直是机器学习和控制领域的重要研究课题。最近在(单智能体)深度强化学习领域的进展重新唤醒了对多智能体强化学习的研究兴趣,尤其在理论分析方面。本文回顾这个大课题中的一个子领域:带有网络智能体的去中心化多智能体强化学习。在这一场景中,多个智能体在一个共同的环境中进行序贯决策,无需中心控制器的协调,且智能体被允许和它们在通信网络上的邻居交换信息。这样的一个模型在很多方向都有相关应用,包括机器人控制、无人车控制、移动传感器网络控制、智能电网,等等。本综述旨在覆盖和整理我们和其他科研人员在这一方向的相关工作。我们希望该综述能够激发更多研究热情,投入到这个激动人心却又充满挑战的领域。

关键词组:强化学习;多智能体系统;网络系统;一致性优化;分布式优化;博弈论


Share this article to: More

Go to Contents

References:

<Show All>

Open peer comments: Debate/Discuss/Question/Opinion

<1>

Please provide your name, email address and a comment





DOI:

10.1631/FITEE.1900661

CLC number:

TP18

Download Full Text:

Click Here

Downloaded:

7463

Clicked:

5536

Cited:

0

On-line Access:

2021-07-12

Received:

2019-11-30

Revision Accepted:

2020-08-17

Crosschecked:

2021-04-29

Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou 310027, China
Tel: +86-571-87952276; Fax: +86-571-87952331; E-mail: jzus@zju.edu.cn
Copyright © 2000~ Journal of Zhejiang University-SCIENCE