JZUS - Journal of Zhejiang University SCIENCE

Frontiers of Information Technology & Electronic Engineering

Accepted manuscript available online (unedited version)

MDLB: a metadata dynamic load balancing mechanism based on reinforcement learning

Author(s): Zhao-qi Wu, Jin Wei, Fan Zhang, Wei Guo, Guang-wei Xie
Affiliation(s): National Digital Switching System Engineering & Technological R&D Center, Zhengzhou 450002, China; more
Corresponding email(s): 17034203@qq.com
Key Words: Object-oriented storage system, Metadata, Dynamic load balancing, Reinforcement learning, Q_learning

Share this article to： More <<< Previous Paper \|Next Paper >>>

Zhao-qi Wu, Jin Wei, Fan Zhang, Wei Guo, Guang-wei Xie. MDLB: a metadata dynamic load balancing mechanism based on reinforcement learning[J]. Frontiers of Information Technology & Electronic Engineering,in press.https://doi.org/10.1631/FITEE.1900121

@article{title="MDLB: a metadata dynamic load balancing mechanism based on reinforcement learning",
author="Zhao-qi Wu, Jin Wei, Fan Zhang, Wei Guo, Guang-wei Xie",
journal="Frontiers of Information Technology & Electronic Engineering",
year="in press",
publisher="Zhejiang University Press & Springer",
doi="https://doi.org/10.1631/FITEE.1900121"
}

%0 Journal Article
%T MDLB: a metadata dynamic load balancing mechanism based on reinforcement learning
%A Zhao-qi Wu
%A Jin Wei
%A Fan Zhang
%A Wei Guo
%A Guang-wei Xie
%J Frontiers of Information Technology & Electronic Engineering
%P 1034-1046
%@ 2095-9184
%D in press
%I Zhejiang University Press & Springer
doi="https://doi.org/10.1631/FITEE.1900121"

TY - JOUR
T1 - MDLB: a metadata dynamic load balancing mechanism based on reinforcement learning
A1 - Zhao-qi Wu
A1 - Jin Wei
A1 - Fan Zhang
A1 - Wei Guo
A1 - Guang-wei Xie
J0 - Frontiers of Information Technology & Electronic Engineering
SP - 1034
EP - 1046
%@ 2095-9184
Y1 - in press
PB - Zhejiang University Press & Springer
ER -
doi="https://doi.org/10.1631/FITEE.1900121"

Abstract
Chinese Summary
Academic Network
Reviewer Comment

Abstract: With the growing amount of information and data, object-oriented storage systems have been widely used in many applications, including the Google File System, Amazon S3, Hadoop Distributed File System, and Ceph, in which load balancing of metadata plays an important role in improving the input/output performance of the entire system. Unbalanced load on the metadata server leads to a serious bottleneck problem for system performance. However, most existing metadata load balancing strategies, which are based on subtree segmentation or hashing, lack good dynamics and adaptability. In this study, we propose a metadata dynamic load balancing (MDLB) mechanism based on reinforcement learning (RL). We learn that the q_learning algorithm and our RL-based strategy consist of three modules, i.e., the policy selection network, load balancing network, and parameter update network. Experimental results show that the proposed MDLB algorithm can adjust the load dynamically according to the performance of the metadata servers, and that it has good adaptability in the case of sudden change of data volume.

This article has been corrected, see doi:10.1631/FITEE.19e0121

MDLB：一种基于强化学习的元数据动态负载均衡机制

武兆琪¹，卫今^2,3，张帆¹，郭威¹，谢光伟^2,3
¹国家数字交换系统工程技术研究中心，中国郑州市，450002
²复旦大学计算机科学技术学院，中国上海市，200433
³复旦大学大数据研究院，中国上海市，200433

摘要：随着信息和数据量增长，面向对象的存储系统已被广泛应用到很多领域，包括Google文件系统、AmazonS3、Hadoop分布式文件系统和Ceph。其中元数据负载均衡在提高整个系统输入/输出性能方面起着重要作用，元数据负载不平衡会导致服务器出现严重的系统性能瓶颈问题。然而现有元数据负载平衡策略缺乏良好动态性和适用性，如基于子树分割或者哈希的负载策略。提出一种基于强化学习的动态负载平衡机制（MDLB）。采用Q_learning算法，所提基于强化学习机制由3个模块组成，即策略选择网络、负载均衡网络和参数更新网络。实验结果表明MDLB算法可根据元数据服务器的性能动态调节负载，在数据量骤变情况下仍具有很好适应性。

关键词组：面向对象的存储系统；元数据；动态负载均衡；强化学习；Q_learning

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article

Reference

[1]Amari SI, Ozeki T, Karakida R, et al., 2018. Dynamics of learning in MLP: natural gradient and singularity revisited. Neur Comput, 30(1):1-33.

[2]Azzedin F, 2013. Towards a scalable HDFS architecture. Proc Int Conf on Collaboration Technologies and Systems, p.155-161.

[3]Chen T, Xiao N, Liu F, 2013. Adaptive metadata load balancing for object storage systems. J Softw, 24(2):331-342 (in Chinese).

[4]Fortunato M, Azar MG, Piot B, et al., 2017. Noisy networks for exploration. https://arxiv.org/abs/1706.10295

[5]Ghemawat S, Gobioff H, Leung ST, 2003. The Google file system. Proc 19^th ACM Symp on Operating Systems Principles, p.29-43.

[6]Hausknecht M, Stone P, 2015. Deep recurrent Q-learning for partially observable MDPs. https://arxiv.org/abs/1507.06527

[7]Hessel M, Modayil J, van Hasselt H, et al., 2018. Rainbow: combining improvements in deep reinforcement learning. https://arxiv.org/abs/1710.02298

[8]Hitz D, Lau J, Malcolm M, 1994. File system design for an NFS file server appliance. Proc USENIX Winter Technical Conf, p.9-19.

[9]Howard J, Kazar M, Menees S, et al., 1987. Scale and performance in a distributed file system. ACM SIGOPS Oper Syst Rev, 21(5):1-2.

[10]Hua Y, Zhu YF, Jiang H, et al., 2008. Scalable and adaptive metadata management in ultra large-scale file systems. Proc 28^th Int Conf on Distributed Computing Systems, p.403-410.

[11]Kanai S, Fujiwara Y, Yamanaka Y, et al., 2018. Sigsoftmax: reanalysis of the softmax bottleneck. Proc 32^nd Int Conf on Neural Information Processing Systems, p.284-294.

[12]Lewis TG, Cook CR, 1988. Hashing for dynamic and static internal tables. Computer, 21(10):45-56.

[13]Li WJ, Xue W, Shu JW, et al., 2006. Dynamic hashing: adaptive metadata management for petabyte-scale file systems. Proc 23^rd IEEE/14^th NASA Goddard Conf on Mass Storage Systems and Technologies, p.159-164.

[14]Liu Z, Zhou XM, 2007. A metadata management method based on directory path. J Softw, 18(2):236-245 (in Chinese).

[15]Manindra A, Thiagarajan PS, 2004. Lazy rectangular hybrid automata. Proc Int Workshop on Hybrid Systems: Computation and Control, p.1-15.

[16]Mnih V, Kavukcuoglu K, Silver D, et al., 2013. Playing Atari with deep reinforcement learning. https://arxiv.org/abs/1312.5602

[17]Palankar MR, Iamnitchi A, Ripeanu M, et al., 2008. Amazon S3 for science grids: a viable solution? Proc Int Workshop on Data-Aware Distributed Computing, p.55-64.

[18]Patgiri R, Dev D, Ahmed A, 2017. dMDS: uncover the hidden issues of metadata server design. In: Sa PK, Sahoo MN, Murugappan M, et al. (Eds.), Progress in Intelligent Computing Techniques: Theory, Practice, and Applications. Springer, Singapore, p.531-541.

[19]Satyanarayanan M, Kistler JJ, Kumar P, et al., 1990. Coda: a highly available file system for a distributed workstation environment. IEEE Trans Comput, 39(4):447-459.

[20]Schindelhauer C, Schomaker G, 2005. Weighted distributed hash tables. Proc 7^th Annual ACM Symp on Parallelism in Algorithms and Architectures, p.218-227.

[21]Shvachko K, Kuang HR, Radia S, et al., 2010. The Hadoop distributed file system. Proc IEEE 26^th Symp on Mass Storage Systems and Technologies, p.1-10.

[22]Stoica I, Morris R, Karger D, et al., 2001. Chord: a scalable peer-to-peer lookup service for Internet applications. Proc Conf on Applications, Technologies, Architectures, and Protocols for Computer Communications, p.149-160.

[23]Sutton RS, Barto AG, 2018. Reinforcement Learning: an Introduction (2^nd Ed.). MIT Press, London, UK.

[24]Tzeng GH, Huang JJ, 2011. Multiple Attribute Decision Making: Methods and Applications. Chapman and Hall, New York, USA.

[25]van Hasselt H, Guez A, Silver D, 2016. Deep reinforcement learning with double Q-learning. Proc 30^th AAAI Conf on Artificial Intelligence, p.2094-2100.

[26]Wang FY, Nelson M, Oral S, et al., 2013. Performance and scalability evaluation of the Ceph parallel file system. Proc 8^th Parallel Data Storage Workshop, p.14-19.

[27]Wang JD, Zhang T, Song JK, et al., 2018. A survey on learning to hash. IEEE Trans Patt Anal Mach Intell, 40(4):769-790.

[28]Wang ZY, Schaul T, Hessel M, et al., 2016. Dueling network architectures for deep reinforcement learning. Proc 33^rd Int Conf on Machine Learning, p.1995-2003.

[29]Wu CX, Burns R, 2003. Handling heterogeneity in shared-disk file systems. Proc ACM/IEEE Conf on Supercomputing, Article 7.

[30]Wu JJ, Liu PF, Chung YC, 2010. Metadata partitioning for large-scale distributed storage systems. Proc IEEE 3^rd Int Conf on Cloud Computing, p.212-219.

[31]Xiong J, Tang RF, Wu SN, et al., 2005. An efficient metadata distribution policy for cluster file systems. Proc IEEE Int Conf on Cluster Computing, p.1-10.

[32]Zhang HY, Wang K, 2018. Research of dynamic load balancing based on stimulated annealing algorithm. Int J Embed Syst, 10(3):188-195.

[33]Zhang LJ, Cui Y, Luo GC, et al., 2017. Dynamic load balance algorithm for big-data distributed storage. Comput Sci, 44(5):178-183 (in Chinese).

[34]Zhong S, Chen J, Yang YR, 2003. Sprite: a simple, cheat-proof, credit-based system for mobile ad-hoc networks. Proc 22^nd Annual Joint Conf of the IEEE Computer and Communications Societies, p.1987-1997.

[35]Zhu YF, Jiang H, Wang J, et al., 2008. HBA: distributed metadata management for large cluster-based storage systems. IEEE Trans Parall Distrib Syst, 19(6):750-763.

Open peer comments: Debate/Discuss/Question/Opinion

<1>

- Go to

MDLB：一种基于强化学习的元数据动态负载均衡机制

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article

Reference