Full Text:
<760>
Summary: 
<194>
Suppl. Mater.: 
CLC number: TP301.6
On-line Access: 2024-08-27
Received: 2023-10-17
Revision Accepted: 2024-05-08
Crosschecked: 2023-11-13
Cited: 0
Clicked: 1100
Citations: Bibtex RefMan EndNote GB/T7714
https://orcid.org/0000-0001-9273-616X
A multi-agent collaboration scheme for energy-efficient task scheduling in a 3D UAV-MEC space
| ||||||||||||||
Yang LI, Ziling WEI, Jinshu SU, Baokang ZHAO. A multi-agent collaboration scheme for energy-efficient task scheduling in a 3D UAV-MEC space[J]. Frontiers of Information Technology & Electronic Engineering, 2024, 25(6): 824-838.
@article{title="A multi-agent collaboration scheme for energy-efficient task scheduling in a 3D UAV-MEC space",
author="Yang LI, Ziling WEI, Jinshu SU, Baokang ZHAO",
journal="Frontiers of Information Technology & Electronic Engineering",
volume="25",
number="6",
pages="824-838",
year="2024",
publisher="Zhejiang University Press & Springer",
doi="10.1631/FITEE.2300393"
}
%0 Journal Article
%T A multi-agent collaboration scheme for energy-efficient task scheduling in a 3D UAV-MEC space
%A Yang LI
%A Ziling WEI
%A Jinshu SU
%A Baokang ZHAO
%J Frontiers of Information Technology & Electronic Engineering
%V 25
%N 6
%P 824-838
%@ 2095-9184
%D 2024
%I Zhejiang University Press & Springer
%DOI 10.1631/FITEE.2300393
TY - JOUR
T1 - A multi-agent collaboration scheme for energy-efficient task scheduling in a 3D UAV-MEC space
A1 - Yang LI
A1 - Ziling WEI
A1 - Jinshu SU
A1 - Baokang ZHAO
J0 - Frontiers of Information Technology & Electronic Engineering
VL - 25
IS - 6
SP - 824
EP - 838
%@ 2095-9184
Y1 - 2024
PB - Zhejiang University Press & Springer
ER -
DOI - 10.1631/FITEE.2300393
Abstract: multi-access edge computing (MEC) presents computing services at the edge of networks to address the enormous processing requirements of intelligent applications. Due to the maneuverability of unmanned aerial vehicles (UAVs), they can be used as temporal aerial edge nodes for providing edge services to ground users in MEC. However, MEC environment is usually dynamic and complicated. It is a challenge for multiple UAVs to select appropriate service strategies. Besides, most of existing works study UAV-MEC with the assumption that the flight heights of UAVs are fixed; i.e., the flying is considered to occur with reference to a two-dimensional plane, which neglects the importance of the height. In this paper, with consideration of the co-channel interference, an optimization problem of energy efficiency is investigated to maximize the number of fulfilled tasks, where multiple UAVs in a three-dimensional space collaboratively fulfill the task computation of ground users. In the formulated problem, we try to obtain the optimal flight and sub-channel selection strategies for UAVs and schedule strategies for tasks. Based on the multi-agent deep deterministic policy gradient (MADDPG) algorithm, we propose a curiosity-driven and twin-networks-structured MADDPG (CTMADDPG) algorithm to solve the formulated problem. It uses the inner reward to facilitate the state exploration of agents, avoiding convergence at the sub-optimal strategy. Furthermore, we adopt the twin critic networks for update stabilization to reduce the probability of Q value overestimation. The simulation results show that CTMADDPG is outstanding in maximizing the energy efficiency of the whole system and outperforms the other benchmarks.
[1]Al-Hourani A, Kandeepan S, Lardner S, 2014. Optimal LAP altitude for maximum coverage. IEEE Wirel Commun Lett, 3(6):569-572.
[2]Ashraf Ateya AA, Muthanna A, Kirichek R, et al., 2019. Energy- and latency-aware hybrid offloading algorithm for UAVs. IEEE Access, 7:37587-37600.
[3]Badia AP, Sprechmann P, Vitvitskyi A, et al., 2020. Never give up: learning directed exploration strategies. Proc 8th Int Conf on Learning Representations.
[4]Chakrabarty A, Langelaan J, 2009. Energy maps for long-range path planning for small- and micro-UAVs. AIAA Guidance, Navigation, and Control Conf, Article 6113.
[5]Dai C, Zhu K, Hossain E, 2022. Multi-agent deep reinforcement learning for joint decoupled user association and trajectory design in full-duplex multi-UAV networks. IEEE Trans Mob Comput, 22(10):6056-6070.
[6]Dai ZJ, Zhang Y, Zhang WC, et al., 2022. A multi-agent collaborative environment learning method for UAV deployment and resource allocation. IEEE Trans Signal Inform Process Netw, 8:120-130.
[7]Ding CF, Wang JB, Cheng M, et al., 2023. Dynamic transmission and computation resource optimization for dense LEO satellite assisted mobile-edge computing. IEEE Trans Commun, 71(5):3087-3102.
[8]Fujimoto S, van Hoof H, Meger D, 2018. Addressing function approximation error in actor-critic methods. Proc 35th Int Conf on Machine Learning, p.1587-1596.
[9]Gu XH, Zhang GA, Wang MX, et al., 2021. UAV-aided energy-efficient edge computing networks: security offloading optimization. IEEE Int Things J, 9(6):4245-4258.
[10]Ji JQ, Zhu K, Yi CY, et al., 2021. Energy consumption minimization in UAV-assisted mobile-edge computing systems: joint resource allocation and trajectory design. IEEE Int Things J, 8(10):8570-8584.
[11]Jiang FB, Wang KZ, Dong L, et al., 2020. Deep-learning-based joint resource scheduling algorithms for hybrid MEC networks. IEEE Int Things J, 7(7):6252-6265.
[12]Joo S, Kang HG, Kang J, 2021. CoSMoS: cooperative sky-ground mobile edge computing system. IEEE Trans Veh Technol, 70(8):8373-8377.
[13]Lakew DS, Tran AT, Dao NN, et al., 2023. Intelligent offloading and resource allocation in heterogeneous aerial access IoT networks. IEEE Int Things J, 10(7):5704-5718.
[14]Liao ZF, Ma YB, Huang JW, et al., 2021. HOTSPOT: a UAV-assisted dynamic mobility-aware offloading for mobile-edge computing in 3-D space. IEEE Int Things J, 8(13):10940-10952.
[15]Liu JF, Li LX, Yang FC, et al., 2019. Minimization of offloading delay for two-tier UAV with mobile edge computing. Proc 15th Int Wireless Communications & Mobile Computing Conf, p.1534-1538.
[16]Liu Q, Shi L, Sun LL, et al., 2020. Path planning for UAV-mounted mobile edge computing with deep reinforcement learning. IEEE Trans Veh Technol, 69(5):5723-5728.
[17]Liu XY, Xu C, Yu HB, et al., 2022. Multi-agent deep reinforcement learning for end–edge orchestrated resource allocation in industrial wireless networks. Front Inform Technol Electron Eng, 23(1):47-60.
[18]Mei HB, Yang K, Liu Q, et al., 2020. Joint trajectory-resource optimization in UAV-enabled edge-cloud system with virtualized mobile clone. IEEE Int Things J, 7(7):5906-5921.
[19]Tun YK, Park YM, Tran NH, et al., 2021. Energy-efficient resource management in UAV-assisted mobile edge computing. IEEE Commun Lett, 25(1):249-253.
[20]Wang JR, Liu KY, Pan JP, 2020. Online UAV-mounted edge server dispatching for mobile-to-mobile edge computing. IEEE Int Things J, 7(2):1375-1386.
[21]Wang JZ, Zhang XL, He XS, et al., 2023. Bandwidth allocation and trajectory control in UAV-assisted IoV edge computing using multiagent reinforcement learning. IEEE Trans Reliab, 72(2):599-608.
[22]Wang L, Wang KZ, Pan CH, et al., 2021. Multi-agent deep reinforcement learning-based trajectory planning for multi-UAV assisted mobile edge computing. IEEE Trans Cogn Commun Netw, 7(1):73-84.
[23]Wang LY, Zhang HX, Guo SS, et al., 2022. Deployment and association of multiple UAVs in UAV-assisted cellular networks with the knowledge of statistical user position. IEEE Trans Wirel Commun, 21(8):6553-6567.
[24]Wang ZQ, Rong HG, Jiang HB, et al., 2022. A load-balanced and energy-efficient navigation scheme for UAV-mounted mobile edge computing. IEEE Trans Netw Sci Eng, 9(5):3659-3674.
[25]Wu SL, Xu WJ, Wang FY, et al., 2022. Distributed federated deep reinforcement learning based trajectory optimization for air-ground cooperative emergency networks. IEEE Trans Veh Technol, 71(8):9107-9112.
[26]Xia WC, Zhu YX, De Simone L, et al., 2022. Multiagent collaborative learning for UAV enabled wireless networks. IEEE J Sel Areas Commun, 40(9):2630-2642.
[27]Xu S, Zhang XY, Li CG, et al., 2022. Deep reinforcement learning approach for joint trajectory design in multi-UAV IoT networks. IEEE Trans Veh Technol, 71(3):3389-3394.
[28]Xu Y, Zhang TK, Loo J, et al., 2021. Completion time minimization for UAV-assisted mobile-edge computing systems. IEEE Trans Veh Technol, 70(11):12253-12259.
[29]Xue NS, 2014. Design and Optimization of Lithium-Ion Batteries for Electric-Vehicle Applications. PhD Thesis, The University of Michigan, Ann Arbor, United States.
[30]Yang L, Yao HP, Wang JJ, et al., 2020. Multi-UAV-enabled load-balance mobile-edge computing for IoT networks. IEEE Int Things J, 7(8):6898-6908.
[31]Yin ZY, Lin Y, Zhang YJ, et al., 2022. Collaborative multi-agent reinforcement learning aided resource allocation for UAV anti-jamming communication. IEEE Int Things J, 9(23):23995-24008.
[32]Yu Y, Bu XY, Yang K, et al., 2021. UAV-aided low latency multi-access edge computing. IEEE Trans Veh Technol, 70(5):4955-4967.
[33]Yu Z, Gong YM, Gong SM, et al., 2020. Joint task offloading and resource allocation in UAV-enabled mobile edge computing. IEEE Int Things J, 7(4):3147-3159.
[34]Zhang L, Ansari N, 2020. Latency-aware IoT service provisioning in UAV-aided mobile-edge computing networks. IEEE Int Things J, 7(10):10573-10580.
[35]Zhao N, Ye ZY, Pei YY, et al., 2022. Multi-agent deep reinforcement learning for task offloading in UAV-assisted mobile edge computing. IEEE Trans Wirel Commun, 21(9):6949-6960.
[36]Zheng LL, Chen JR, Wang JH, et al., 2021. Episodic multi-agent reinforcement learning with curiosity-driven exploration. Proc 34th Int Conf on Neural Information Processing Systems, p.3757-3769.
[37]Zhong RK, Liu X, Liu YW, et al., 2022. Multi-agent reinforcement learning in NOMA-aided UAV networks for cellular offloading. IEEE Trans Wirel Commun, 21(3):1498-1512.
[38]Zhou FH, Wu YP, Hu RQ, et al., 2018. Computation rate maximization in UAV-enabled wireless-powered mobile-edge computing systems. IEEE J Sel Areas Commun, 36(9):1927-1941.
ORCID: 
Open peer comments: Debate/Discuss/Question/Opinion
<1>