Full Text:   <522>

CLC number: 

On-line Access: 2021-01-12

Received: 2021-07-15

Revision Accepted: 2021-12-07

Crosschecked: 0000-00-00

Cited: 0

Clicked: 749

Citations:  Bibtex RefMan EndNote GB/T7714

-   Go to

Article info.
Open peer comments

Journal of Zhejiang University SCIENCE A 1998 Vol.-1 No.-1 P.

http://doi.org/10.1631/jzus.A2100325


Towards autonomous and optimal excavation of shield machine: a deep reinforcement learning-based approach


Author(s):  Ya-kun ZHANG, Guo-fang GONG, Hua-yong YANG, Yu-xi CHEN, Geng-lin CHEN

Affiliation(s):  State Key Laboratory of Fluid Power and Mechatronic Systems, Zhejiang University, Hangzhou 310027, China; more

Corresponding email(s):   gfgong@zju.edu.cn

Key Words:  Shield machine, Slurry shield, Tunnel boring machine, Intelligent TBMs, Deep reinforcement learning, Optimal control, Dynamic optimization, Deep learning


Ya-kun ZHANG, Guo-fang GONG, Hua-yong YANG, Yu-xi CHEN, Geng-lin CHEN. Towards autonomous and optimal excavation of shield machine: a deep reinforcement learning-based approach[J]. Journal of Zhejiang University Science A, 1998, -1(-1): .

@article{title="Towards autonomous and optimal excavation of shield machine: a deep reinforcement learning-based approach",
author="Ya-kun ZHANG, Guo-fang GONG, Hua-yong YANG, Yu-xi CHEN, Geng-lin CHEN",
journal="Journal of Zhejiang University Science A",
volume="-1",
number="-1",
pages="",
year="1998",
publisher="Zhejiang University Press & Springer",
doi="10.1631/jzus.A2100325"
}

%0 Journal Article
%T Towards autonomous and optimal excavation of shield machine: a deep reinforcement learning-based approach
%A Ya-kun ZHANG
%A Guo-fang GONG
%A Hua-yong YANG
%A Yu-xi CHEN
%A Geng-lin CHEN
%J Journal of Zhejiang University SCIENCE A
%V -1
%N -1
%P
%@ 1673-565X
%D 1998
%I Zhejiang University Press & Springer
%DOI 10.1631/jzus.A2100325

TY - JOUR
T1 - Towards autonomous and optimal excavation of shield machine: a deep reinforcement learning-based approach
A1 - Ya-kun ZHANG
A1 - Guo-fang GONG
A1 - Hua-yong YANG
A1 - Yu-xi CHEN
A1 - Geng-lin CHEN
J0 - Journal of Zhejiang University Science A
VL - -1
IS - -1
SP -
EP -
%@ 1673-565X
Y1 - 1998
PB - Zhejiang University Press & Springer
ER -
DOI - 10.1631/jzus.A2100325


Abstract: 
Autonomous excavation operation is a major trend in the development of a new generation of intelligent TBMs. However, existing technologies are limited to supervised machine learning and static optimization, which cannot outperform human operation and deal with ever changing geological conditions and the long-term performance measure. The aim of this study is to resolve the problem of dynamic optimization of the shield excavation performance, as well as to achieve autonomous optimal excavation. In this study, a novel autonomous optimal excavation approach that integrates deep reinforcement learning and optimal control is proposed for shield machines. Based on a first-principles analysis of the machine-ground interaction dynamics of the excavation process, a deep neural network model is developed using construction field data consisting of 1.1 million samples. The multi-system coupling mechanism is revealed by establishing an overall system model. Based on the overall system analysis, the autonomous optimal excavation problem is decomposed into a multi-objective dynamic optimization problem and an optimal control problem. Subsequently, a dimensionless multi-objective comprehensive excavation performance measure is proposed. A deep reinforcement learning method is used to solve for the optimal action sequence trajectory, and optimal closed-loop feedback controllers are designed to achieve accurate execution. The performance of the proposed approach is compared to that of human operation by using the construction field data. The simulation results show that the proposed approach not only has the potential to replace human operation but also can significantly improve the comprehensive excavation performance.

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article

Open peer comments: Debate/Discuss/Question/Opinion

<1>

Please provide your name, email address and a comment





Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou 310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn
Copyright © 2000 - 2022 Journal of Zhejiang University-SCIENCE