E-mail Alert
RSS
| Citation: |
Pan J, Wang Q, Gao SK, Zhang Z, Xie Y et al. Knot-inspired optical sensors for slip detection and friction measurement in dexterous robotic manipulation. Opto-Electron Adv 6, 230076 (2023). doi: 10.29026/oea.2023.230076
|
Article Open Access
Knot-inspired optical sensors for slip detection and friction measurement in dexterous robotic manipulation
-
Abstract
Friction plays a critical role in dexterous robotic manipulation. However, realizing friction sensing remains a challenge due to the difficulty in designing sensing structures to decouple multi-axial forces. Inspired by the topological mechanics of knots, we construct optical fiber knot (OFN) sensors for slip detection and friction measurement. By introducing localized self-contacts along the fiber, the knot structure enables anisotropic responses to normal and frictional forces. By employing OFNs and a change point detection algorithm, we demonstrate adaptive robotic grasping of slipping cups. We further develop a robotic finger that can measure tri-axial forces via a centrosymmetric architecture composed of five OFNs. Such a tactile finger allows a robotic hand to manipulate human tools dexterously. This work could provide a straightforward and cost-effective strategy for promoting adaptive grasping, dexterous manipulation, and human-robot interaction with tactile sensing.-
Keywords:
- robotic perception /
- adaptive grasping /
- slip detection /
- force decoupling /
- polymer optical fiber
-
-
References
[1] Mason MT. Toward robotic manipulation. Ann Rev Control Robot Auton Syst 1, 1–28 (2018). doi: 10.1146/annurev-control-060117-104848 [2] Billard A, Kragic D. Trends and challenges in robot manipulation. Science 364, eaat8414 (2019). doi: 10.1126/science.aat8414 [3] Cui JD, Trinkle J. Toward next-generation learned robot manipulation. Sci Robot 6, eadb9461 (2021). doi: 10.1126/scirobotics.abd9461 [4] Sundaram S. How to improve robotic touch. Science 370, 768–769 (2020). doi: 10.1126/science.abd3643 [5] Zhong F, Hu W, Zhu PN, Wang H, Ma C et al. Piezoresistive design for electronic skin: from fundamental to emerging applications. Opto-Electron Adv 5 (2022). doi: 10.29026/oea.2022.210029 [6] Liu FY, Deswal S, Christou A, Sandamirskaya Y, Kaboli M et al. Neuro-inspired electronic skin for robots. Sci Robot 7, eabl7344 (2022). doi: 10.1126/scirobotics.abl7344 [7] Li GZ, Liu SQ, Wang LQ, Zhu R. Skin-inspired quadruple tactile sensors integrated on a robot hand enable object recognition. Sci Robot 5, eabc8134 (2020). doi: 10.1126/scirobotics.abc8134 [8] Liu FY, Deswal S, Christou A, Shojaei Baghini M, Chirila R et al. Printed synaptic transistor-based electronic skin for robots to feel and learn. Sci Robot 7, eabl7286 (2022). doi: 10.1126/scirobotics.abl7286 [9] Zhao ZX, Tang JS, Yuan J, Li YJ, Dai Y et al. Large-scale integrated flexible tactile sensor array for sensitive smart robotic touch. ACS Nano 16, 16784–16795 (2022). doi: 10.1021/acsnano.2c06432 [10] Liu YF, Cui SW, Wei JH, Li HB, Hu JY et al. Centrosymmetric- and axisymmetric-patterned flexible tactile sensor for roughness and slip intelligent recognition. Adv Intellig Syst 4, 2100072 (2021). doi: 10.1002/aisy.202100072 [11] Wang YC, Wu X, Mei DQ, Zhu LF, Chen JN. Flexible tactile sensor array for distributed tactile sensing and slip detection in robotic hand grasping. Sens Actuat A Phys 297, 111512 (2019). doi: 10.1016/j.sna.2019.07.036 [12] Cao YD, Li T, Gu Y, Luo H, Wang SQ et al. Fingerprint-inspired flexible tactile sensor for accurately discerning surface texture. Small 14, 1703902 (2018). doi: 10.1002/smll.201703902 [13] Yan YC, Hu Z, Yang ZB, Yuan WZ, Song CY et al. Soft magnetic skin for super-resolution tactile sensing with force self-decoupling. Sci Robot 6, eabc8801 (2021). doi: 10.1126/scirobotics.abc8801 [14] Dwivedi A, Ramakrishnan A, Reddy A, Patel K, Ozel S et al. Design, modeling, and validation of a soft magnetic 3-D force sensor. IEEE Sens J 18, 3852–3863 (2018). doi: 10.1109/JSEN.2018.2814839 [15] Xie SP, Zhang YF, Zhang H, Jin MH. Development of triaxis electromagnetic tactile sensor with adjustable sensitivity and measurement range for robot manipulation. IEEE Trans Instrum Meas 71, 1–9 (2022). doi: 10.1109/TIM.2022.3159912 [16] Barreiros JA, Xu A, Pugach S, Iyengar N, Troxell G et al. Haptic perception using optoelectronic robotic flesh for embodied artificially intelligent agents. Sci Robot 7, eabi6745 (2022). doi: 10.1126/scirobotics.abi6745 [17] Ward-Cherrier B, Pestell N, Cramphorn L, Winstone B, Giannaccini ME et al. The TacTip family: soft optical tactile sensors with 3D-printed biomimetic morphologies. Soft Robot 5, 216–227 (2018). doi: 10.1089/soro.2017.0052 [18] Yuan WZ, Dong SY, Adelson EH. GelSight: high-resolution robot tactile sensors for estimating geometry and force. Sensors 17, 2762 (2017). doi: 10.3390/s17122762 [19] Biazi-Neto V, Marques CAF, Frizera-Neto A, Leal-Junior AG. FBG-embedded robotic manipulator tool for structural integrity monitoring from critical strain-stress pair estimation. IEEE Sens J 22, 5695–5702 (2022). doi: 10.1109/JSEN.2022.3149459 [20] Biazi-Neto V, Marques CAF, Frizera-Neto A, Leal-Junior AG. FBG-based sensing system to improve tactile sensitivity of robotic manipulators working in unstructured environments. Sens Actuat A Phys 359, 114473 (2023). doi: 10.1016/j.sna.2023.114473 [21] Borràs J. Effective grasping enables successful robot-assisted dressing. Sci Robot 7, eabo7229 (2022). doi: 10.1126/scirobotics.abo7229 [22] Shih B, Shah D, Li JX, Thuruthel TG, Park YL et al. Electronic skins and machine learning for intelligent soft robots. Sci Robot 5, eaaz9239 (2020). doi: 10.1126/scirobotics.aaz9239 [23] Koo JH, Yun HW, Lee WC, Sunwoo SH, Shim HJ et al. Recent advances in soft electronic materials for intrinsically stretchable optoelectronic systems. Opto-Electron Adv 5, 210131 (2022). doi: 10.29026/oea.2022.210131 [24] Ding ZM, Zhang ZY. 2D tactile sensor based on multimode interference and deep learning. Opt Laser Technol 136, 106760 (2021). doi: 10.1016/j.optlastec.2020.106760 [25] Pan J, Zhang Z, Jiang CP, Zhang L, Tong LM. A multifunctional skin-like wearable optical sensor based on an optical micro-/nanofibre. Nanoscale 12, 17538–17544 (2020). doi: 10.1039/D0NR03446K [26] Van Meerbeek IM, De Sa CM, Shepherd RF. Soft optoelectronic sensory foams with proprioception. Sci Robot 3, eaau2489 (2018). doi: 10.1126/scirobotics.aau2489 [27] Leal-Junior A, Avellar L, Biazi V, Soares MS, Frizera A et al. Multifunctional flexible optical waveguide sensor: on the bioinspiration for ultrasensitive sensors development. Opto-Electron Adv 5, 210098 (2022). doi: 10.29026/oea.2022.210098 [28] Yu W, Yao N, Pan J, Fang W, Li X et al. Highly sensitive and fast response strain sensor based on evanescently coupled micro/nanofibers. Opto-Electron Adv 5, 210101 (2022). doi: 10.29026/oea.2022.210101 [29] Li LY, Liu YF, Song CY, Sheng SF, Yang LY et al. Wearable alignment-free microfiber-based sensor chip for precise vital signs monitoring and cardiovascular assessment. Adv Fiber Mater 4, 475–486 (2022). doi: 10.1007/s42765-021-00121-8 [30] Zhao HC, O'Brien K, Li S, Shepherd RF. Optoelectronically innervated soft prosthetic hand via stretchable optical waveguides. Sci Robot 1, eaai7529 (2016). doi: 10.1126/scirobotics.aai7529 [31] Teeple CB, Becker KP, Wood RJ. Soft curvature and contact force sensors for deep-sea grasping via soft optical waveguides. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems 1621–1627 (IEEE, 2018);http://doi.org/10.1109/IROS.2018.8594270. [32] Zhang L, Pan J, Zhang Z, Wu H, Yao N et al. Ultrasensitive skin-like wearable optical sensors based on glass micro/nanofibers. Opto-Electron Adv 3, 190022 (2020). doi: 10.29026/oea.2020.190022 [33] Tang Y, Yu LT, Pan J, Yao N, Geng WD et al. Optical nanofiber skins for multifunctional humanoid tactility. Adv Intellig Syst 5, 2200203 (2023). doi: 10.1002/aisy.202200203 [34] Yao N, Wang XY, Ma SQ, Song XD, Wang S et al. Single optical microfiber enabled tactile sensor for simultaneous temperature and pressure measurement. Photon Res 10, 2040–2046 (2022). doi: 10.1364/PRJ.461182 [35] Massari L, Fransvea G, D’Abbraccio J, Filosa M, Terruso G et al. Functional mimicry of Ruffini receptors with fibre Bragg gratings and deep neural networks enables a bio-inspired large-area tactile-sensitive skin. Nat Mach Intellig 4, 425–435 (2022). doi: 10.1038/s42256-022-00487-3 [36] Bai HD, Li S, Barreiros J, Tu YQ, Pollock CR et al. Stretchable distributed fiber-optic sensors. Science 370, 848–852 (2020). doi: 10.1126/science.aba5504 [37] Lee B. Review of the present status of optical fiber sensors. Opt Fiber Technol 9, 57–79 (2003). doi: 10.1016/S1068-5200(02)00527-8 [38] Li YP, Tan SJ, Yang LY, Li LY, Fang F et al. Optical microfiber neuron for finger motion perception. Adv Fiber Mater 4, 226–234 (2022). doi: 10.1007/s42765-021-00096-6 [39] Jiang CP, Zhang Z, Pan J, Wang YC, Zhang L et al. Finger-skin-inspired flexible optical sensor for force sensing and slip detection in robotic grasping. Adv Mater Technol 6, 2100285 (2021). doi: 10.1002/admt.202100285 [40] Zhou JY, Shao Q, Tang C, Qiao F, Lu TQ et al. Conformable and compact multiaxis tactile sensor for human and robotic grasping via anisotropic waveguides. Adv Mater Technol 7, 2200595 (2022). doi: 10.1002/admt.202200595 [41] Patil VP, Sandt JD, Kolle M, Dunkel J. Topological mechanics of knots and tangles. Science 367, 71–75 (2020). doi: 10.1126/science.aaz0135 [42] Takeuchi J, Yamanishi K. A unifying framework for detecting outliers and change points from time series. IEEE Trans Knowledge Data Eng 18, 482–492 (2006). doi: 10.1109/TKDE.2006.1599387 [43] Nan KW, Babaee S, Chan WW, Kuosmanen JLP, Feig VR et al. Low-cost gastrointestinal manometry via silicone-liquid-metal pressure transducers resembling a quipu. Nat Biomed Eng 6, 1092–1104 (2022). doi: 10.1038/s41551-022-00859-5 [44] Zhang HP, Oh S, Mahato M, Yoo H, Oh IK. Knot‐architectured fabric actuators based on shape memory fibers. Adv Funct Mater 32, 2205732 (2022). doi: 10.1002/adfm.202205732 [45] Xie Y, Cai DW, Wu H, Pan J, Zhou N et al. Mid-infrared chalcogenide microfiber knot resonators. Photon Res 8, 616–621 (2020). doi: 10.1364/PRJ.386395 [46] Wang WJ, Yiu HHP, Li WJ, Roy VAL. The principle and architectures of optical stress sensors and the progress on the development of microbend optical sensors. Adv Opt Mater 9, 2001693 (2021). doi: 10.1002/adom.202001693 -
Supplementary Information
Supplementary information for Knot-inspired optical sensor for slip detection and friction measurement in dexterous robotic manipulation 
Supplementary movie
-
Access History
Figures(5)
Article Metrics
Export File
Citation
Pan J, Wang Q, Gao SK, Zhang Z, Xie Y et al. Knot-inspired optical sensors for slip detection and friction measurement in dexterous robotic manipulation. Opto-Electron Adv 6, 230076 (2023). doi: 10.29026/oea.2023.230076
Format
Content
DownLoad:
-
Figure 1.
Overview of the OFN sensor. (a) Schematic diagram of the OFN sensing system. (b) 5 by 5 OFN sensing array. (c) Photographs of the output light when the sensor was untouched, (d) subjected to normal force, and (e) subjected to both normal and frictional force. (f) Finite element simulations of strain distribution when the OFN sensor (knot diameter: 3.5 mm, side length: 5 mm, thickness: 1 mm) was untouched, (g) subjected to a 10 N normal force, and (h) subjected to a 10 N normal force and an additional 4 N frictional force, respectively.
-
Figure 2.
Characterization of normal and frictional force sensing. (a) Schematic diagram of the friction testing system. (b) Responses to normal force of OFN sensors with knot diameters of 4.5 mm, 3.5 mm and 2.5 mm. (c) Responses to frictional force of OFN sensors with a diameter of 2.5 mm when pre-loaded with 0 N, 2.5 N, 5 N, 7.5 N and 10 N normal forces. (d) Intensity signals of the OFN sensor loaded/unloaded with normal and frictional forces. (e, f) Intensity signals during cyclical loading of normal force and frictional force, respectively.
-
Figure 3.
Adaptive grasping based on slip detection when the slip feedback is ON and OFF, respectively. (a, e) Snapshots during the experiments. Yellow arrows indicate slips of the cup while green arrows indicate closing of the robotic hand. (b, f) Real-time signals of the OFN sensors. (c, g) Real-time scores calculated by the Slip Finder program. Slip Score = 1: a slip occurred, Slip Score = 0: no slips. (d, h) Real-time position of the robotic fingers. Position = 0: fully open, Position = 255: fully closed.
-
Figure 4.
Characterization of tri-axial force sensing. (a) Schematic diagram of the tri-axial force testing system. (b–d) Stress distribution and responses of OFN sensors with knot diameters of 4.5 mm, 3.5 mm, and 2.5 mm to Fx, Fy, and Fz, respectively.
-
Figure 5.
Dexterous manipulation based on tri-axial force sensing. (a) Exploded diagram of the robotic tactile finger. (b) Positions of the five OFN sensors inside the finger. (c) Schematic diagram of force decomposition. (d) Photograph of two robotic tactile fingers. (e, f) The tri-axial force sensing signals and snapshots during the cutting experiment and the unlocking experiment, respectively.
