Full Text:  <243>

Suppl. Mater.: 

Summary:  <11>

CLC number: 

On-line Access: 2023-01-11

Received: 2022-06-04

Revision Accepted: 2022-08-31

Crosschecked: 2023-01-13

Cited: 0

Clicked: 194

Citations:  Bibtex RefMan EndNote GB/T7714

 ORCID:

Fei LV

https://orcid.org/0000-0001-9124-9044

Jia YU

https://orcid.org/0000-0003-1775-1006

-   Go to

Article info.
Open peer comments

Journal of Zhejiang University SCIENCE A

Accepted manuscript available online (unedited version)


A novel stacking-based ensemble learning model for drilling efficiency prediction in earth-rock excavation


Author(s):  Fei LV, Jia YU, Jun ZHANG, Peng YU, Da-wei TONG, Bin-ping WU

Affiliation(s):  State Key Laboratory of Hydraulic Engineering Simulation and Safety, Tianjin University, Tianjin 300350, China

Corresponding email(s):  yujia@tju.edu.cn

Key Words:  Drilling efficiency; Prediction; Earth-rock excavation; Stacking-based ensemble learning; Improved cuckoo search optimization (ICSO) algorithm; Comprehensive effects of various factors; Hyper-parameter optimization


Share this article to: More <<< Previous Paper|

Fei LV, Jia YU, Jun ZHANG, Peng YU, Da-wei TONG, Bin-ping WU. A novel stacking-based ensemble learning model for drilling efficiency prediction in earth-rock excavation[J]. Journal of Zhejiang University Science A, 2022, 23(6): 1027-1046.

@article{title="A novel stacking-based ensemble learning model for drilling efficiency prediction in earth-rock excavation",
author="Fei LV, Jia YU, Jun ZHANG, Peng YU, Da-wei TONG, Bin-ping WU",
journal="Journal of Zhejiang University Science A",
volume="23",
number="12",
pages="1027-1046",
year="2022",
publisher="Zhejiang University Press & Springer",
doi="10.1631/jzus.A2200297"
}

%0 Journal Article
%T A novel stacking-based ensemble learning model for drilling efficiency prediction in earth-rock excavation
%A Fei LV
%A Jia YU
%A Jun ZHANG
%A Peng YU
%A Da-wei TONG
%A Bin-ping WU
%J Journal of Zhejiang University SCIENCE A
%V 23
%N 12
%P 1027-1046
%@ 1673-565X
%D 2022
%I Zhejiang University Press & Springer

TY - JOUR
T1 - A novel stacking-based ensemble learning model for drilling efficiency prediction in earth-rock excavation
A1 - Fei LV
A1 - Jia YU
A1 - Jun ZHANG
A1 - Peng YU
A1 - Da-wei TONG
A1 - Bin-ping WU
J0 - Journal of Zhejiang University Science A
VL - 23
IS - 12
SP - 1027
EP - 1046
%@ 1673-565X
Y1 - 2022
PB - Zhejiang University Press & Springer
ER -


Abstract: 
Accurate prediction of drilling efficiency is critical for developing the earth-rock excavation schedule. The single machine learning (ML) prediction models usually suffer from problems including parameter sensitivity and overfitting. In addition, the influence of environmental and operational factors is often ignored. In response, a novel stacking-based ensemble learning method taking into account the combined effects of those factors is proposed. Through multiple comparison tests, four models, eXtreme gradient boosting (XGBoost), random forest (RF), back propagation neural network (BPNN) as the base learners, and support vector regression (SVR) as the meta-learner, are selected for stacking. Furthermore, an improved cuckoo search optimization (ICSO) algorithm is developed for hyper-parameter optimization of the ensemble model. The application to a real-world project demonstrates that the proposed method outperforms the popular single ML method XGBoost and the ensemble model optimized by particle swarm optimization (PSO), with 16.43% and 4.88% improvements of mean absolute percentage error (MAPE), respectively.

土方开挖过程中钻进效率预测的Stacking集成学习模型

作者:吕菲,余佳,张君,俞澎,佟大威,吴斌平
机构:天津大学,水利工程仿真与安全国家重点实验室,中国天津,300350
目的:对钻进效率进行精确预测是制定土方开挖进度计划的关键。但现有预测方法多采用单个机器学习模型,存在参数敏感性和过拟合等问题,且往往忽略了环境因素和人员操作因素的影响。针对这些问题,本文提出一种同时考虑多种因素综合影响的新的集成学习预测方法。
创新点:1.建立一种基于Stacking集成学习的钻进效率预测模型;2.定量地考虑地质特性、人员操作、环境和机械特性等多种因素的综合影响;3.提出一种基于自适应步长策略的改进布谷鸟搜索优化方法,优化模型关键参数。
方法:1.通过多次对比实验,最终选择极值梯度提升(XGBoost)、随机森林(RF)和反向传播神经网络(BPNN)三个模型作为基学习器,支持向量回归(SVR)作为元学习器进行集成。2.建立基于自适应步长策略的改进布谷鸟搜索优化算法,对集成模型的Max_depth等超参数进行优化。3.将钻进效率值及相关影响因素的样本数据输入到每个基学习器中,得到相应的输出结果,再将预测结果作为元学习器的输入值,得到最终的预测结果。4.以中国西南地区某土石方工程为例,通过五折交叉验证方法,验证模型的鲁棒性,并采用五个常用评价指标评价模型的精度和泛化性能。
结论:工程应用结果表明,相比于目前流行的单个机器学习方法中预测性能最好的XGBoost和基于粒子群算法优化的Stacking集成模型,本文所提方法的平均绝对百分比误差(MAPE)分别提高了16.43%和4.88%。

关键词组:钻进效率;预测;土方开挖;Stacking集成学习;改进的布谷鸟搜索算法

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article

Reference

[1]AbbasAK, RushdiS, AlsabaM, et al., 2019. Drilling rate of penetration prediction of high-angled wells using artificial neural networks. Journal of Energy Resources Technology, 141(11):112904.

[2]AbbaspourH, DrebenstedtC, BadroddinM, et al., 2018. Optimized design of drilling and blasting operations in open pit mines under technical and economic uncertainties by system dynamic modelling. International Journal of Mining Science and Technology, 28(6):839-848.

[3]Abu BakarMZ, ButtIA, MajeedY, 2018. Penetration rate and specific energy prediction of rotary-percussive drills using drill cuttings and engineering properties of selected rock units. Journal of Mining Science, 54(2):270-284.

[4]AkünME, KarpuzC, 2005. Drillability studies of surface-set diamond drilling in Zonguldak region sandstones from Turkey. International Journal of Rock Mechanics and Mining Sciences, 42(3):473-479.

[5]BreimanL, 2001. Random forests. Machine Learning, 45(1):5-32.

[6]BuiDT, NhuVH, HoangND, 2018. Prediction of soil compression coefficient for urban housing project using novel integration machine learning approach of swarm intelligence and multi-layer perceptron neural network. Advanced Engineering Informatics, 38:593-604.

[7]CankurtS, SubasiA, 2022. Tourism demand forecasting using stacking ensemble model with adaptive fuzzy combiner. Soft Computing, 26(7):3455-3467.

[8]ChenKL, JiangJC, ZhengFD, et al., 2018. A novel data-driven approach for residential electricity consumption prediction based on ensemble learning. Energy, 150:49-60.

[9]ChenWL, WangXL, CaiZJ, et al., 2021. DP-GMM clustering-based ensemble learning prediction methodology for dam deformation considering spatiotemporal differentiation. Knowledge-Based Systems, 222:106964.

[10]CuiSZ, YinYQ, WangDJ, et al., 2021. A stacking-based ensemble learning method for earthquake casualty prediction. Applied Soft Computing, 101:107038.

[11]DarborM, FaramarziL, SharifzadehM, 2019. Performance assessment of rotary drilling using non-linear multiple regression analysis and multilayer perceptron neural network. Bulletin of Engineering Geology and the Environment, 78(3):1501-1513.

[12]ElkatatnyS, 2018. Application of artificial intelligence techniques to estimate the static Poisson’s ratio based on wireline log data. Journal of Energy Resources Technology, 140(7):072905.

[13]GalarM, FernandezA, BarrenecheaE, et al., 2012. A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 42(4):463-484.

[14]GanC, CaoWH, WuM, et al., 2019. Prediction of drilling rate of penetration (ROP) using hybrid support vector regression: a case study on the Shennongjia area, central China. Journal of Petroleum Science and Engineering, 181:106200.

[15]GuoYY, WangX, XiaoPC, et al., 2020. An ensemble learning framework for convolutional neural network based on multiple classifiers. Soft Computing, 24(5):3727-3735.

[16]HaghighiF, OmranpourH, 2021. Stacking ensemble model of deep learning and its application to Persian/Arabic handwritten digits recognition. Knowledge-Based Systems, 220:106940.

[17]HustrulidWA, KuchtaM, MartinR, 2013. Open Pit Mine Planning and Design, 3rd Edition. Taylor & Francis, Boca Raton, Florida, USA.

[18]KahramanS, 2002. Correlation of TBM and drilling machine performances with rock brittleness. Engineering Geology, 65(4):269-283.

[19]KahramanS, BilginN, FeridunogluC, 2003. Dominant rock properties affecting the penetration rate of percussive drills. International Journal of Rock Mechanics and Mining Sciences, 40(5):711-723.

[20]KaushikA, KaurP, ChoudharyN, et al., 2022. Stacking regularization in analogy-based software effort estimation. Soft Computing, 26(3):1197-1216.

[21]KazemzadehMR, AmjadianA, AmraeeT, 2020. A hybrid data mining driven algorithm for long term electric peak load and energy demand forecasting. Energy, 204:117948.

[22]KoopialipoorM, TootoonchiH, ArmaghaniDJ, et al., 2019. Application of deep neural networks in predicting the penetration rate of tunnel boring machines. Bulletin of Engineering Geology and the Environment, 78(8):‍6347-6360.

[23]LiH, WangXS, DingSF, 2018. Research and development of neural network ensembles: a survey. Artificial Intelligence Review, 49(4):455-479.

[24]LiLL, CenZY, TsengML, et al., 2021. Improving short-term wind power prediction using hybrid improved cuckoo search arithmetic-support vector regression machine. Journal of Cleaner Production, 279:123739.

[25]LiSP, 2018. Research on Excavation Simulation of a Pumped Storage Power Station Based on Excavation and Filling Balance. MS Thesis, Tianjin University, Tianjin, China(in Chinese).

[26]LiZX, WuDZ, HuC, et al., 2019. An ensemble learning-based prognostic approach with degradation-dependent weights for remaining useful life prediction. Reliability Engineering & System Safety, 184:110-122.

[27]LvF, WangJJ, CuiB, et al., 2020. An improved extreme gradient boosting approach to vehicle speed prediction for construction simulation of earthwork. Automation in Construction, 119:103351.

[28]MaoCY, LinRR, ToweyD, et al., 2021. Trustworthiness prediction of cloud services based on selective neural network ensemble learning. Expert Systems with Applications, 168:114390.

[29]Mendes-MoreiraJ, SoaresC, JorgeAM, et al., 2012. Ensemble approaches for regression: a survey. ACM Computing Surveys, 45(1):10.

[30]MengXJ, ChangJX, WangXB, et al., 2019. Multi-objective hydropower station operation using an improved cuckoo search algorithm. Energy, 168:425-439.

[31]MustafaAB, AbbasAK, AlsabaM, et al., 2021. Improving drilling performance through optimizing controllable drilling parameters. Journal of Petroleum Exploration and Production, 11(3):1223-1232.

[32]PaulA, BhowmikS, PanuaR, et al., 2018. Artificial neural network-based prediction of performances-exhaust emissions of diesohol piloted dual fuel diesel engine under varying compressed natural gas flowrates. Journal of Energy Resources Technology, 140(11):112201.

[33]PavlyukevichI, 2007. Lévy flights, non-local search and simulated annealing. Journal of Computational Physics, 226(2):1830-1844.

[34]PengH, ZengZG, DengCS, et al., 2021. Multi-strategy serial cuckoo search algorithm for global optimization. Knowledge-Based Systems, 214:106729.

[35]Pernía-EspinozaA, Fernandez-CenicerosJ, AntonanzasJ, et al., 2018. Stacking ensemble with parsimonious base models to improve generalization capability in the characterization of steel bolted components. Applied Soft Computing, 70:737-750.

[36]QiCC, FourieA, ChenQS, 2018. Neural network and particle swarm optimization for predicting the unconfined compressive strength of cemented paste backfill. Construction and Building Materials, 159:473-478.

[37]RenQB, LiMC, SongLG, et al., 2020. An optimized combination prediction model for concrete dam deformation considering quantitative evaluation and hysteresis correction. Advanced Engineering Informatics, 46:101154.

[38]SaeidiO, TorabiSR, AtaeiM, et al., 2014. A stochastic penetration rate model for rotary drilling in surface mines. International Journal of Rock Mechanics and Mining Sciences, 68:55-65.

[39]SalimiA, RostamiJ, MoormannC, et al., 2016. Application of non-linear regression analysis and artificial intelligence algorithms for performance prediction of hard rock TBMs. Tunnelling and Underground Space Technology, 58:236-246.

[40]ShiX, LiuG, GongXL, et al., 2016. An efficient approach for real-time prediction of rate of penetration in offshore drilling. Mathematical Problems in Engineering, 2016:3575380.

[41]WangH, ZhangYM, MaoJX, et al., 2020. A probabilistic approach for short-term prediction of wind gust speed using ensemble learning. Journal of Wind Engineering and Industrial Aerodynamics, 202:104198.

[42]WangL, ZengY, ChenT, 2015. Back propagation neural network with adaptive differential evolution algorithm for time series forecasting. Expert Systems with Applications, 42(2):855-863.

[43]WangN, ZhaoSY, CuiSZ, et al., 2021. A hybrid ensemble learning method for the identification of gang-related arson cases. Knowledge-Based Systems, 218:106875.

[44]WangRC, WuS, 2019. Neural network model based prediction of fragmentation of blasting using the Levenberg-Marquardt algorithm. Journal of Hydroelectric Engineering, 38(7):100-109 (in Chinese).

[45]WangT, LiZJ, YanYJ, et al., 2007. A survey of fuzzy decision tree classifier methodology. Proceedings of the Second International Conference of Fuzzy Information and Engineering, p.959-968.

[46]YanT, ShenSL, ZhouAN, et al., 2022. Prediction of geological characteristics from shield operational parameters by integrating grid search and K-fold cross validation into stacking classification algorithm. Journal of Rock Mechanics and Geotechnical Engineering, 14(4):1292-1303.

[47]YangXS, DebS, 2014. Cuckoo search: recent advances and applications. Neural Computing and Applications, 24(1):169-174.

[48]ZhangXH, ZhuQX, HeYL, et al., 2018. A novel robust ensemble model integrated extreme learning machine with multi-activation functions for energy modeling and analysis: application to petrochemical industry. Energy, 162:593-602.

Open peer comments: Debate/Discuss/Question/Opinion

<1>

Please provide your name, email address and a comment





Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou 310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn
Copyright © 2000 - 2023 Journal of Zhejiang University-SCIENCE