Full Text:  <50>

Suppl. Mater.: 

CLC number: 

On-line Access: 2022-08-31

Received: 2022-02-25

Revision Accepted: 2022-08-11

Crosschecked: 0000-00-00

Cited: 0

Clicked: 226

Citations:  Bibtex RefMan EndNote GB/T7714

-   Go to

Article info.
Open peer comments

Frontiers of Information Technology & Electronic Engineering 

Accepted manuscript available online (unedited version)


Soft-HGRNs: soft hierarchical graph recurrent networks for many-agent partially observable environments∗


Author(s):  Yixiang REN, Zhenhui YE, Yining CHEN, Xiaohong JIANG, Guanghua SONG

Affiliation(s):  School of Aeronautics and Astronautics, Zhejiang University, Hangzhou 310027, China; more

Corresponding email(s):  yixiangren@zju.edu.cn, zhenhuiye@zju.edu.cn, ch19930611@zju.edu.cn, ghsong@zju.edu.cn

Key Words:  Deep reinforcement learning; Graph-based communication; Maximum-entropy learning; Partial observability; Heterogeneous settings


Share this article to: More |Next Paper >>>

Yixiang REN, Zhenhui YE, Yining CHEN, Xiaohong JIANG, Guanghua SONG. Soft-HGRNs: soft hierarchical graph recurrent networks for many-agent partially observable environments∗[J]. Frontiers of Information Technology & Electronic Engineering , 1998, -1(1): .

@article{title="Soft-HGRNs: soft hierarchical graph recurrent networks for many-agent partially observable environments∗",
author="Yixiang REN, Zhenhui YE, Yining CHEN, Xiaohong JIANG, Guanghua SONG",
journal="Frontiers of Information Technology & Electronic Engineering",
volume="-1",
number="-1",
pages="",
year="1998",
publisher="Zhejiang University Press & Springer",
doi="10.1631/FITEE.2200073"
}

%0 Journal Article
%T Soft-HGRNs: soft hierarchical graph recurrent networks for many-agent partially observable environments∗
%A Yixiang REN
%A Zhenhui YE
%A Yining CHEN
%A Xiaohong JIANG
%A Guanghua SONG
%J Frontiers of Information Technology & Electronic Engineering
%V -1
%N -1
%P
%@ 1869-1951
%D 1998
%I Zhejiang University Press & Springer

TY - JOUR
T1 - Soft-HGRNs: soft hierarchical graph recurrent networks for many-agent partially observable environments∗
A1 - Yixiang REN
A1 - Zhenhui YE
A1 - Yining CHEN
A1 - Xiaohong JIANG
A1 - Guanghua SONG
J0 - Frontiers of Information Technology & Electronic Engineering
VL - -1
IS - -1
SP -
EP -
%@ 1869-1951
Y1 - 1998
PB - Zhejiang University Press & Springer
ER -


Abstract: 
The recent progress in multi-agent deep reinforcement learning (MADRL) makes it more practical in real-world tasks, but its relatively poor scalability and the partially observable constraint raise more challenges for its performance and deployment. Based on our intuitive observation that human society could be regarded as a large-scale partially observable environment, where everyone has the functions of communicating with neighbors and remembering its own experience, we propose a novel network structure called the hierarchical graph recurrent network (HGRN) for multi-agent cooperation under partial observability. Specifically, we construct the multi-agent system as a graph, utilize a novel graph convolution structure to achieve communication between heterogeneous neighboring agents, and adopt a recurrent unit to enable agents to record historical information. To encourage exploration and improve robustness, we design a maximum-entropy learning method that can learn stochastic policies of a configurable target action entropy. Based on the above technologies, we propose a value-based MADRL algorithm called Soft-HGRN and its actor-critic variant called SAC-HGRN. Experimental results based on three homogeneous tasks and one heterogeneous environment not only show that our approach achieves clear improvements compared with four MADRL baselines, but also demonstrate the interpretability, scalability, and transferability of the proposed model.

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article

Reference

Open peer comments: Debate/Discuss/Question/Opinion

<1>

Please provide your name, email address and a comment





Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou 310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn
Copyright © 2000 - 2022 Journal of Zhejiang University-SCIENCE