Abstract: Digital simulation of the full operation of a remotely operated vehicle (ROV) is an economically feasible way for algorithm pretesting and operator training prior to the actual underwater tasks, due to the huge difficulties encountered during the underwater test, high equipment cost, and the time-consuming nature of the process. In this paper, a human-interactive digital simulation platform is established for the navigation, motion, and teleoperated manipulation of work-class ROVs, and provides the human operator with the visualized full operation process. Specially, two mechanisms are presented in this platform: one provides the virtual simulation platform for operator training; the other provides real-time visual and force feedback when implementing the actual tasks. Moreover, an open data interface is designed for researchers for pretesting various algorithms before implementing the actual underwater tasks. Additionally, typical underwater scenarios of the ROV, including underwater sediment sampling and pipeline docking tasks, are selected as the case studies for hydrodynamics-based simulation. Human operator can operate the manipulator installed on the ROV via the master manipulator with the visual and force feedback after the ROV is navigated to the desired position. During the full operation, the dynamic windows approach (DWA)-based local navigation algorithm, sliding mode control (SMC) controller, and the teleoperation control framework are implemented to show the effectiveness of the designed platform. Finally, a user study on the ROV operation mode is carried out, and several metrics are designed to evaluate the superiority and accuracy of the digital simulation platform for immersive underwater teleoperation.

一种具有人机交互沉浸式设计的数字仿真平台——用于作业级水下遥控潜水器的导航、运动与遥操作

[1]Cárdenas EF, Dutra MS, 2016. An augmented reality application to assist teleoperation of underwater manipulators. IEEE Lat Am Trans, 14(2):863-869.

[2]Chen Z, Huang FH, Sun WC, et al., 2020. RBF-neural-network-based adaptive robust control for nonlinear bilateral teleoperation manipulators with uncertainty and time delay. IEEE/ASME Trans Mech, 25(2):906-918.

[3]Chen Z, Helian BB, Zhou Y, et al., 2023. An integrated trajectory planning and motion control strategy of a variable rotational speed pump-controlled electro-hydraulic actuator. IEEE/ASME Trans Mech, 28(1):588-597.

[4]Chen Z, Zhou SZ, Shen C, et al., 2024. Observer-based adaptive robust precision motion control of a multi-joint hydraulic manipulator. IEEE/CAA J Autom Sin, 11(5):1213-1226.

[5]Ferreira A, Mavroidis C, 2006. Virtual reality and haptics for nanorobotics. IEEE Rob Autom Mag, 13(3):78-92.

[6]Forbrigger S, Pan YJ, 2018. Improving haptic transparency for uncertain virtual environments using adaptive control and gain-scheduled prediction. IEEE Trans Haptics, 11(4):543-554.

[7]Hart SG, Staveland LE, 1988. Development of NASA-TLX (task load index): results of empirical and theoretical research. Adv Psychol, 52:139-183.

[8]Hokayem PF, Spong MW, 2006. Bilateral teleoperation: an historical survey. Automatica, 42(12):2035-2057.

[9]Huang FH, Chen XL, Xu Y, et al., 2023. Immersive virtual simulation system design for the guidance, navigation and control of unmanned surface vehicles. Ocean Eng, 281:114884.

[10]Khadhraoui A, Beji L, Otmane S, et al., 2016. Stabilizing control and human scale simulation of a submarine ROV navigation. Ocean Eng, 114:66-78.

[11]Khatib O, Yeh X, Brantner G, et al., 2016. Ocean one: a robotic avatar for oceanic discovery. IEEE Rob Autom Mag, 23(4):20-29.

[12]Kinsey JC, Yang QJ, Howland JC, 2014. Nonlinear dynamic model-based state estimators for underwater navigation of remotely operated vehicles. IEEE Trans Contr Syst Technol, 22(5):1845-1854.

[13]Li HJ, Huang FH, Chen Z, 2022. Virtual-reality-based online simulator design with a virtual simulation system for the docking of unmanned underwater vehicle. Ocean Eng, 266:112780.

[14]Lin Q, Kuo CG, 2001. On applying virtual reality to underwater robot tele-operation and pilot training. Int J Virtual Reality, 5(1):71-91.

[15]Liu Y, Shen Q, Ma DL, et al., 2017. Steering control for underwater gliders. Front Inform Technol Electron Eng, 18(7):898-914.

[16]Long CQ, Hu MJ, Qin XH, et al., 2022. Hierarchical trajectory tracking control for ROVs subject to disturbances and parametric uncertainties. Ocean Eng, 266:112733.

[17]Lu Y, Chen XY, Wu ZX, et al., 2022. A novel robotic visual perception framework for underwater operation. Front Inform Technol Electron Eng, 23(11):1602-1619.

[18]Manzanilla A, Reyes S, Garcia M, et al., 2019. Autonomous navigation for unmanned underwater vehicles: real-time experiments using computer vision. IEEE Rob Autom Lett, 4(2):1351-1356.

[19]Reichherzer C, Cunningham A, Walsh J, et al., 2018. Narrative and spatial memory for jury viewings in a reconstructed virtual environment. IEEE Trans Vis Comput Graph, 24(11):2917-2926.

[20]Sivčev S, Coleman J, Omerdić E, et al., 2018. Underwater manipulators: a review. Ocean Eng, 163:431-450.

[21]Sun WC, Yuan YQ, 2023. Passivity based hierarchical multi-task tracking control for redundant manipulators with uncertainties. Automatica, 155:111159.

[22]Sun YX, Gu R, Chen XH, et al., 2022. Efficient time-optimal path planning of AUV under the ocean currents based on graph and clustering strategy. Ocean Eng, 259:111907.

[23]Tani S, Ruscio F, Bresciani M, et al., 2023. Development and testing of a navigation solution for autonomous underwater vehicles based on stereo vision. Ocean Eng, 280:114757.

[24]Wang J, Tang YG, Chen CX, et al., 2020. Terrain matching localization for hybrid underwater vehicle in the Challenger Deep of the Mariana Trench. Front Inform Technol Electron Eng, 21(5):749-759.

[25]Xia PX, Xu F, Song ZY, et al., 2023. Sensory augmentation for subsea robot teleoperation. Comput Ind, 145:103836.

[26]Yuan YQ, Sun WC, 2023. An integrated kinematic calibration and dynamic identification method with only static measurements for serial robot. IEEE/ASME Trans Mechatron, 28(5):2762-2773.

[27]Zhang DW, Yang G, Khurshid RP, 2020. Haptic teleoperation of UAVs through control barrier functions. IEEE Trans Haptic, 13(1):109-115.

[28]Zhang HP, Zhu DQ, Liu CX, et al., 2022. Tracking fault-tolerant control based on model predictive control for human occupied vehicle in three-dimensional underwater workspace. Ocean Eng, 249:110845.

[29]Zhang J, Li W, Yu JC, et al., 2017a. Development of a virtual platform for telepresence control of an underwater manipulator mounted on a submersible vehicle. IEEE Trans Ind Electron, 64(2):1716-1727.

[30]Zhang J, Li W, Yu JC, et al., 2017b. Study of manipulator operations maneuvered by a ROV in virtual environments. Ocean Eng, 142:292-302.

[31]Zhang JJ, Liu WD, Gao LE, et al., 2018. The master adaptive impedance control and slave adaptive neural network control in underwater manipulator uncertainty teleoperation. Ocean Eng, 165:465-479.

[32]Zhao B, Skjetne R, Blanke M, et al., 2014. Particle filter for fault diagnosis and robust navigation of underwater robot. IEEE Trans Contr Syst Technol, 22(6):2399-2407.