Similar articles

Abstract: Real-time energy management of a hybrid excavator is addressed using reinforcement learning (RL). Due to the computational complexity and need for a priori knowledge of the load cycles, a traditional optimal control method, like dynamic programming (DP), is not feasible for real-time control. Real-time controllers derived from traditional optimal control methods compute the solutions either in a cycle-dependent manner or far away from the optimal. An RL-based energy management controller is proposed to solve this problem. The simulation and experimental results demonstrate that the RL controller has a better performance than the widely used thermostat and equivalent consumption minimization strategy (ECMS) controllers. It also shows that the RL controller is cycle-independent. pontryagin’;s minimum principle (PMP) is used to obtain the analytical solution of the energy management problem, and this can help to reduce the iteration time in the design process.

This paper presents an interesting topic on the energy management problem of a hybrid excavator. Four different control strategy have been implemented on a simulator over the standard digging cycle and then validated through a dedicated experimental activity on the hybrid prototype.

基于强化学习的混合动力挖掘机实时能量管理控制器设计

[1]Borhan, H.A., Vahidi, A., Phillips, A.M., et al., 2009. Predictive energy management of a power-split hybrid electric vehicle. American Control Conference, p.3970-3976.

[2]Brahma, A., Guezennec, Y., Rizzoni, G., 2000. Optimal energy management in series hybrid electric vehicles. American Control Conference, p.60-64.

[3]Busoniu, L., Babuska, R., Schutter, D., et al., 2010. Reinforcement Learning and Dynamic Programming Using Function Approximators. CRC Press, Boca Raton, USA, p.101-113.

[4]Chan, C.C., 2007. The state of the art of electric, hybrid, and fuel cell vehicles. Proceedings of the IEEE, 95(4):704-718.

[5]Ehsani, M., Gao, Y., Miller, J.M., 2007. Hybrid electric vehicles: architecture and motor drives. Proceedings of the IEEE, 95(4):719-728.

[6]Geering, H.P., 2007. Optimal Control with Engineering Applications. Springer, New York, USA, p.3-5.

[7]Gosavi, A., 2003. Simulation-based Optimization. Springer, New York, USA, p.197-211.

[8]Jalil, N., Kheir, N.A., Salman, M., 1997. A rule-based energy management strategy for a series hybrid vehicle. American Control Conference, p.689-693.

[9]Johri, R., Filipi, Z., 2009. Low-cost pathway to ultra efficient city car: series hydraulic hybrid system with optimized supervisory control. SAE International Journal of Engines, 2(2):505-520.

[10]Kim, H., Choi, J., Yi, K., 2012. Development of supervisory control strategy for optimized fuel consumption of the compound hybrid excavator. Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering, 226(12):1652-1666.

[11]Kim, H., Yoo, S., Cho, S., et al., 2016. Hybrid control algorithm for fuel consumption of a compound hybrid excavator. Automation in Construction, 68:1-10.

[12]Kim, N., Rousseau, A., 2012. Sufficient conditions of optimal control based on Pontryagin’s minimum principle for use in hybrid electric vehicles. Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering, 226(9):1160-1170.

[13]Kim, N., Cha, S., Peng, H., 2011. Optimal control of hybrid electric vehicles based on Pontryagin’s minimum principle. IEEE Transactions on Control Systems Technology, 19(5):1279-1287.

[14]Lin, C.C., Peng, H., Grizzle, J.W., et al., 2003. Power management strategy for a parallel hybrid electric truck. IEEE Transactions on Control Systems Technology, 11(6):839-849.

[15]Lin, T., Wang, Q., Hu, B., et al., 2010. Development of hybrid powered hydraulic construction machinery. Automation in Construction, 19(1):11-19.

[16]Lin, X., Pan, S.X., Wang, D.Y., 2008. Dynamic simulation and optimal control strategy for a parallel hybrid hydraulic excavator. Journal of Zhejiang University-SCIENCE A, 9(5):624-632.

[17]Musardo, C., Rizzoni, G., Guezennec, Y., et al., 2005. A-ECMS: an adaptive algorithm for hybrid electric vehicle energy management. European Journal of Control, 11(4-5):509-524.

[18]Salman, M., Schouten, N.J., Kheir, N.A., 2000. Control strategies for parallel hybrid vehicles. American Control Conference, p.524-528.

[19]Salmasi, F.R., 2007. Control strategies for hybrid electric vehicles: evolution, classification, comparison, and future trends. IEEE Transactions on Vehicular Technology, 56(5):2393-2404.

[20]Schouten, N.J., Salman, M.A., Kheir, N.A., 2002. Fuzzy logic control for parallel hybrid vehicles. IEEE Transactions on Control Systems Technology, 10(3):460-468.

[21]Sciarretta, A., Guzzella, L., 2007. Control of hybrid electric vehicles. IEEE Control Systems, 27(2):60-70.

[22]Sciarretta, A., Back, M., Guzzella, L., 2004. Optimal control of parallel hybrid electric vehicles. IEEE Transactions on Control Systems Technology, 12(3):352-363.

[23]Serrao, L., Onori, S., Rizzoni, G., 2009. ECMS as a realization of Pontryagin’s minimum principle for HEV control. American Control Conference, p.3964-3969.

[24]Serrao, L., Onori, S., Rizzoni, G., 2011. A comparative analysis of energy management strategies for hybrid electric vehicles. Journal of Dynamic Systems, Measurement, and Control, 133(3):031012.

[25]Watkins, C., Dayan, P., 1992. Q-learning. Machine Learning, 8(3-4):279-292.

[26]Xiao, Q., Wang, Q., Zhang, Y., 2008. Control strategies of power system in hybrid hydraulic excavator. Automation in Construction, 17(4):361-367.

[27]Zhang, S., Xiong, R., 2015. Adaptive energy management of a plug-in hybrid electric vehicle based on driving pattern recognition and dynamic programming. Applied Energy, 155:68-78.

[28]Zhu, Q., Wang, Q., Chen, Q., 2016. Component sizing for compound hybrid excavators. Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering, 230(7):969-982.