Light-perception-based interactive control of an underwater digital twin hand

With increasing demand for ocean exploration, underwater tasks become more complex. While soft robotic hands offer superior flexibility and biomimetic advantages over rigid robotic hands, their development has been hindered by low sensing accuracy and poor interactive control. Here, we propose a light-perception-based underwater digital twin hand interactive control system that integrates a multi-degree-of-freedom biomimetic pneumatic soft robotic hand with a wearable data glove based on fiber Bragg grating (FBG) sensors. By fusing data from FBGs, RGB cameras, and inertial measurement units, and optimizing control strategies through reinforcement learning, we established a light-dominated perception-control-feedback closed-loop system that significantly enhances the control performance of the system in complex and dynamic underwater environments. With light as the information carrier, the system integrates a data glove with high strain sensitivity (1.148 pm/µε, linearity = 0.9996) and high bending sensitivity (26.667 pm/°, linearity = 0.9791). It achieves four-dimensional collaborative interactive control between the human operator, machine, virtual space, and environment. The system achieves posture monitoring and precise grasping in various underwater environments, offering advantages like low cost, high precision, rapid response, and strong interference resistance. This establishes a new paradigm for underwater intelligent equipment by integrating light sensing with digital twin technology, offering broad prospects for future underwater operations.