Abstract: Neural networks (NNs), as one of the most robust and efficient machine learning methods, have been commonly used in solving several problems. However, choosing proper hyperparameters (e.g. the numbers of layers and neurons in each layer) has a significant influence on the accuracy of these methods. Therefore, a considerable number of studies have been carried out to optimize the NN hyperparameters. In this study, the genetic algorithm is applied to NN to find the optimal hyperparameters. Thus, the deep energy method, which contains a deep neural network, is applied first on a Timoshenko beam and a plate with a hole. Subsequently, the numbers of hidden layers, integration points, and neurons in each layer are optimized to reach the highest accuracy to predict the stress distribution through these structures. Thus, applying the proper optimization method on NN leads to significant increase in the NN prediction accuracy after conducting the optimization in various examples.

利用遗传算法优化神经网络超参数

[1]Alhichri H, Alajlan N, Bazi Y, et al., 2018. Multi-scale convolutional neural network for remote sensing scene classification. IEEE International Conference on Electro/Information Technology, p.1-5.

[2]Anitescu C, Hossain MN, Rabczuk T, 2018. Recovery-based error estimation and adaptivity using high-order splines over hierarchical T-meshes. Computer Methods in Applied Mechanics and Engineering, 328:638-662.

[3]Augarde CE, Deeks AJ, 2008. The use of Timoshenko’s exact solution for a cantilever beam in adaptive analysis. Finite Elements in Analysis and Design, 44(9-10):595-601.

[4]Bacanin N, Bezdan T, Tuba E, et al., 2020. Optimizing convolutional neural network hyperparameters by enhanced swarm intelligence metaheuristics. Algorithms, 13(3):67.

[5]Bani-Hani D, Khan N, Alsultan F, et al., 2018. Classification of leucocytes using convolutional neural network optimized through genetic algorithm. Proceedings of the 7th Annual World Conference of the Society for Industrial and Systems Engineering.

[6]Bergstra J, Bengio Y, 2012. Random search for hyper-parameter optimization. The Journal of Machine Learning Research, 13(1):281-305.

[7]Dalto M, Matuško J, Vašak M, 2015. Deep neural networks for ultra-short-term wind forecasting. IEEE International Conference on Industrial Technology, p.1657-1663.

[8]Goswami S, Anitescu C, Chakraborty S, et al., 2020. Transfer learning enhanced physics informed neural network for phase-field modeling of fracture. Theoretical and Applied Fracture Mechanics, 106:102447.

[9]Guo BS, Hu JW, Wu WW, et al., 2019. The Tabu_genetic algorithm: a novel method for hyper-parameter optimization of learning algorithms. Electronics, 8(5):579.

[10]Jo Y, Min K, Jung D, et al., 2019. Comparative study of the artificial neural network with three hyper-parameter optimization methods for the precise LP-EGR estimation using in-cylinder pressure in a turbocharged GDI engine. Applied Thermal Engineering, 149:1324-1334.

[11]Junior FEF, Yen GG, 2019. Particle swarm optimization of deep neural networks architectures for image classification. Swarm and Evolutionary Computation, 49:62-74.

[12]Kanada Y, 2016. Optimizing neural-network learning rate by using a genetic algorithm with per-epoch mutations. International Joint Conference on Neural Networks, p.1472-1479.

[13]Kaur S, Aggarwal H, Rani R, 2020. Hyper-parameter optimization of deep learning model for prediction of Parkinson’s disease. Machine Vision and Applications, 31(5):32.

[14]Liashchynskyi P, Liashchynskyi P, 2019. Grid search, random search, genetic algorithm: a big comparison for NAS. https://arxiv.org/abs/1912.06059

[15]Loussaief S, Abdelkrim A, 2018. Convolutional neural network hyper-parameters optimization based on genetic algorithms. International Journal of Advanced Computer Science and Applications, 9(10):252-266.

[16]Motta D, Santos AÁB, Machado BAS, et al., 2020. Optimization of convolutional neural network hyperparameters for automatic classification of adult mosquitoes. PLoS One, 15(7):e0234959.

[17]Najafi B, Faizollahzadeh Ardabili S, Mosavi A, et al., 2018. An intelligent artificial neural network-response surface methodology method for accessing the optimum biodiesel and diesel fuel blending conditions in a diesel engine from the viewpoint of exergy and energy analysis. Energies, 11(4):860.

[18]Nassif AB, Shahin I, Attili I, et al., 2019. Speech recognition using deep neural networks: a systematic review. IEEE Access, 7:19143-19165.

[19]Nguyen-Thanh VM, Nguyen LTK, Rabczuk T, et al., 2019. A surrogate model for computational homogenization of elastostatics at finite strain using the HDMR-based neural network approximator. https://arxiv.org/abs/1906.02005

[20]Samaniego E, Anitescu C, Goswami S, et al., 2020. An energy approach to the solution of partial differential equations in computational mechanics via machine learning: concepts, implementation and applications. Computer Methods in Applied Mechanics and Engineering, 362:112790.

[21]Shamshirband S, Mosavi A, Rabczuk T, et al., 2020. Prediction of significant wave height; comparison between nested grid numerical model, and machine learning models of artificial neural networks, extreme learning and support vector machines. Engineering Applications of Computational Fluid Mechanics, 14(1):805-817.

[22]Torres JF, Gutiérrez-Avilés D, Troncoso A, et al., 2019. Random hyper-parameter search-based deep neural network for power consumption forecasting. International Work-Conference on Artificial Neural Networks, p.259-269.

[23]ul Hassan M, Sabar NR, Song A, 2018. Optimising deep learning by hyper-heuristic approach for classifying good quality images. International Conference on Computational Science, p.528-539.

[24]Wei X, You ZN, 2019. Neural network hyperparameter tuning based on improved genetic algorithm. Proceedings of the 8th International Conference on Computing and Pattern Recognition, p.17-24.

[25]Wicaksono AS, Supianto AA, 2018. Hyper parameter optimization using genetic algorithm on machine learning methods for online news popularity prediction. International Journal of Advanced Computer Science and Applications, 9(12):263-267.

[26]Yu T, Zhu H, 2020. Hyper-parameter optimization: a review of algorithms and applications. https://arxiv.org/abs/2003.05689