E-mail Alert
RSS
| Citation: |
Zhou HY, Zhang C, Nong HC et al. Multi-photon neuron embedded bionic skin for high-precision complex texture and object reconstruction perception research. Opto-Electron Adv 8, 240152 (2025). doi: 10.29026/oea.2025.240152
|
Multi-photon neuron embedded bionic skin for high-precision complex texture and object reconstruction perception research
-
Abstract
Attributable to the complex distribution of tactile vesicles under the skin and the ability of the brain to process specific tactile parameters (shape, hardness, and surface texture), human skin can have the capacity for tactile spatial reconstruction and visualization of complex object geometry and surface texture. However, current haptic sensor technologies are predominantly point sensors, which do not have an interlaced distribution structure similar to that of haptic vesicles, limiting their potential in human-computer interaction applications. Here, we report an optical microfiber array skin (OMAS) imitating tactile vesicle interlaced structures for tactile visualization and object reconstruction sensing. This device is characterized by high sensitivity (−0.83 N/V) and fast response time (38 ms). We demonstrate that combining the signals collected by the OMAS with appropriate artificial intelligence algorithms enables the recognition of objects with different hardnesses and shapes with 100% accuracy. It also allows for the classification of fabrics with different surface textures with 98.5% accuracy and Braille patterns with 99% accuracy. As a proof-of-concept, we integrated OMAS into a robot arm to select mahjong among six common objects and successfully recognize its suits by touch, which provides a new solution for tactile sensory processing for human-computer interaction. -
-
References
[1] Sharma N, Flaherty K, Lezgiyeva K et al. The emergence of transcriptional identity in somatosensory neurons. Nature 577, 392–398 (2020). doi: 10.1038/s41586-019-1900-1 [2] Liu FY, Deswal S, Christou A et al. Neuro-inspired electronic skin for robots. Sci Robot 7, eabl7344 (2022). doi: 10.1126/scirobotics.abl7344 [3] Kang D, Pikhitsa PV, Choi YW et al. Ultrasensitive mechanical crack-based sensor inspired by the spider sensory system. Nature 516, 222–226 (2014). doi: 10.1038/nature14002 [4] Wang S, Zhang ZX, Yang B et al. High sensitivity tactile sensors with ultrabroad linear range based on gradient hybrid structure for gesture recognition and precise grasping. Chem Eng J 457, 141136 (2023). doi: 10.1016/j.cej.2022.141136 [5] Wei CJ, Zhou HW, Zheng BH et al. Fully flexible and mechanically robust tactile sensors containing core-shell structured fibrous piezoelectric mat as sensitive layer. Chem Eng J 476, 146654 (2023). doi: 10.1016/j.cej.2023.146654 [6] Wang SH, Lin L, Wang ZL. Nanoscale triboelectric-effect-enabled energy conversion for sustainably powering portable electronics. Nano Lett 12, 6339–6346 (2012). doi: 10.1021/nl303573d [7] Gu GY, Zhang NB, Xu HP et al. A soft neuroprosthetic hand providing simultaneous myoelectric control and tactile feedback. Nat Biomed Eng 7, 589–598 (2023). [8] Wang Q, Zhang DY, Qian YZ et al. Research on fiber optic surface Plasmon resonance biosensors: a review. Photonic Sens 14, 240201 (2024). doi: 10.1007/s13320-024-0703-7 [9] Lu MD, Wang C, Fan RZ et al. Review of fiber-optic localized surface Plasmon resonance sensors: geometries, fabrication technologies, and bio-applications. Photonic Sens 14, 240202 (2024). doi: 10.1007/s13320-024-0709-1 [10] Zhang L, Zhen YQ, Tong LM. Optical micro/nanofiber enabled tactile sensors and soft actuators: a review. Opto-Electron Sci 3, 240005 (2024). doi: 10.29026/oes.2024.240005 [11] Yao N, Wang XY, Ma SQ et al. Single optical microfiber enabled tactile sensor for simultaneous temperature and pressure measurement. Photonics Res 10, 2040–2046 (2022). doi: 10.1364/PRJ.461182 [12] Weng JJ, Yu Y, Zhang JF et al. A biomimetic optical skin for multimodal tactile perception based on optical microfiber coupler neuron. J Lightwave Technol 41, 1874–1883 (2023). doi: 10.1109/JLT.2022.3225068 [13] Zhang L, Pan J, Zhang Z et al. Ultrasensitive skin-like wearable optical sensors based on glass micro/nanofibers. Opto-Electron Adv 3, 190022 (2020). [14] Li TL, Su YF, Chen FY et al. Bioinspired stretchable fiber-based sensor toward intelligent human-machine interactions. ACS Appl Mater Interfaces 14, 22666–22677 (2022). doi: 10.1021/acsami.2c05823 [15] Ahmadi R, Packirisamy M, Dargahi J et al. Discretely loaded beam-type optical fiber tactile sensor for tissue manipulation and palpation in minimally invasive robotic surgery. IEEE Sens J 12, 22–32 (2012). doi: 10.1109/JSEN.2011.2113394 [16] Keser S, Hayber Ş. Fiber optic tactile sensor for surface roughness recognition by machine learning algorithms. Sens Actuators A Phys 332, 113071 (2021). doi: 10.1016/j.sna.2021.113071 [17] Feng KP, Dang H, Zhou WY et al. A capillary-induced self-assembly method under external constraint for fabrication of high-aspect-ratio and square array of optical fibers. J Manuf Process 85, 645–657 (2023). doi: 10.1016/j.jmapro.2022.11.056 [18] Ma SQ, Wang XY, Li P et al. Optical micro/Nano fibers enabled smart textiles for human-machine interface. Adv Fiber Mater 4, 1108–1117 (2022). doi: 10.1007/s42765-022-00163-6 [19] Tong LM. Micro/nanofibre optical sensors: challenges and prospects. Sensors 18, 903 (2018). doi: 10.3390/s18030903 [20] Tang Y, Yu LT, Pan J et al. Optical nanofiber skins for multifunctional humanoid tactility. Adv Intell Syst 5, 2200203 (2023). doi: 10.1002/aisy.202200203 [21] Wang SP, Wang XY, Wang S et al. Optical-nanofiber-enabled gesture-recognition wristband for human-machine interaction with the assistance of machine learning. Adv Intell Syst 5, 2200412 (2023). doi: 10.1002/aisy.202200412 [22] Li YP, Tan SJ, Yang LY et al. Optical microfiber neuron for finger motion perception. Adv Fiber Mater 4, 226–234 (2022). doi: 10.1007/s42765-021-00096-6 [23] Li LY, Liu YF, Song CY et al. Wearable alignment-free microfiber-based sensor chip for precise vital signs monitoring and cardiovascular assessment. Adv Fiber Mater 4, 475–486 (2022). doi: 10.1007/s42765-021-00121-8 [24] Kuang RF, Wang Z, Ma L et al. Smart photonic wristband for pulse wave monitoring. Opto-Electron Sci 3, 240009 (2024). doi: 10.29026/oes.2024.240009 [25] Weng JJ, Xiao SY, Yu Y et al. A bio-inspired artificial tactile sensing system based on optical microfiber and enhanced by neural network. Adv Sens Res 3, 2300157 (2024). doi: 10.1002/adsr.202300157 [26] Zhou JY, Shao Q, Tang C et al. Conformable and compact multiaxis tactile sensor for human and robotic grasping via anisotropic waveguides. Adv Mater Technol 7, 2200595 (2022). doi: 10.1002/admt.202200595 [27] Jiang CP, Zhang Z, Pan J et al. Finger-skin-inspired flexible optical sensor for force sensing and slip detection in robotic grasping. Adv Mater Technol 6, 2100285 (2021). doi: 10.1002/admt.202100285 [28] Fan CL, Luo BB, Wu DC et al. Flexible bionic microstructure tactile sensor based on micro-Nano optical fiber. Acta Opt Sin 43, 2106004 (2023). doi: 10.3788/AOS231313 [29] Xu F. Optical fibre nanowire devices (University of Southampton, Southampton, 2008). [30] Brambilla G, Payne DN. The ultimate strength of glass silica nanowires. Nano Lett 9, 831–835 (2009). doi: 10.1021/nl803581r [31] Yu Y. Optical microfiber fabrication and all-optic microfiber modulator (National University of Defense Technology, Changsha, 2014). [32] Zhang XL, Belal M, Chen GY et al. Compact optical microfiber phase modulator. Opt Lett 37, 320–322 (2012). doi: 10.1364/OL.37.000320 [33] Li YJ, Luo BB, Zou X et al. Sensing characteristics of optical vernier of double-helix micro-Nano optical fiber coupler. Chin J Lasers 50, 1406001 (2023). doi: 10.3788/CJL221045 [34] Lu JY, Yu Y, Qin SP et al. High-performance temperature and pressure dual-parameter sensor based on a polymer-coated tapered optical fiber. Opt Express 30, 9714–9726 (2022). doi: 10.1364/OE.452355 [35] Chun S, Kim JS, Yoo Y et al. An artificial neural tactile sensing system. Nat Electron 4, 429–438 (2021). doi: 10.1038/s41928-021-00585-x [36] Chen W, Khamis H, Birznieks I et al. Tactile sensors for friction estimation and incipient slip detection—Toward dexterous robotic manipulation: a review. IEEE Sens J 18, 9049–9064 (2018). doi: 10.1109/JSEN.2018.2868340 [37] Sumetsky M, Dulashko Y, Hale A. Fabrication and study of bent and coiled free silica nanowires: self-coupling microloop optical interferometer. Opt Express 12, 3521–3531 (2004). doi: 10.1364/OPEX.12.003521 [38] Yu YS, Zhao YP. Deformation of PDMS membrane and microcantilever by a water droplet: comparison between Mooney-Rivlin and linear elastic constitutive models. J Colloid Interface Sci 332, 467–476 (2009). doi: 10.1016/j.jcis.2008.12.054 [39] Wen L, Nie M, Fan JW et al. Tactile recognition of shape and texture on the same substrate. Adv Intell Syst 5, 2300337 (2023). doi: 10.1002/aisy.202300337 [40] Qiao HY, Sun ST, Wu PY. Non-equilibrium-growing aesthetic ionic skin for fingertip-like strain-undisturbed tactile sensation and texture recognition. Adv Mater 35, e2300593 (2023). doi: 10.1002/adma.202300593 [41] Zhou ZH, Chen K, Li XS et al. Sign-to-speech translation using machine-learning-assisted stretchable sensor arrays. Nat Electron 3, 571–578 (2020). doi: 10.1038/s41928-020-0428-6 [42] Lee S, Franklin S, Hassani FA et al. Nanomesh pressure sensor for monitoring finger manipulation without sensory interference. Science 370, 966–970 (2020). doi: 10.1126/science.abc9735 [43] Amoli V, Kim JS, Jee E et al. A bioinspired hydrogen bond-triggered ultrasensitive ionic mechanoreceptor skin. Nat Commun 10, 4019 (2019). doi: 10.1038/s41467-019-11973-5 [44] Park J, Kim M, Lee Y et al. Fingertip skin-inspired microstructured ferroelectric skins discriminate static/dynamic pressure and temperature stimuli. Sci Adv 1, e1500661 (2015). doi: 10.1126/sciadv.1500661 [45] Li Y, Cao ZG, Li T et al. Highly selective biomimetic flexible tactile sensor for neuroprosthetics. Research 2020, 8910692 (2020). [46] Yu LJ, Jia YL, Teng WM. Compilation of Chinese braille standard. Chin J Rehabil Theory Pract 1 , 27–28 (1997). -
Supplementary Information
-
Access History
Figures(4)
Article Metrics
Export File
Citation
Zhou HY, Zhang C, Nong HC et al. Multi-photon neuron embedded bionic skin for high-precision complex texture and object reconstruction perception research. Opto-Electron Adv 8, 240152 (2025). doi: 10.29026/oea.2025.240152
Format
Content
DownLoad:
-
Figure 1.
Design and fabrication of a multiphoton neuron tactile skin. (a) The design concept and spatial reconstruction workflow of the multiphotonic neuron haptic skin for simulating the tactile perception and spatial reconstruction process of human subcutaneous multitactile cell fusion. (b) Flowchart of the interaction of each module in the spatial reconstruction process of mahjong by multiphotonic neuron haptic skin. (c) The structure of the multiphotonic neuron tactile skin, which consists of three layers: a silicone contact layer, an OM array embedded PDMS sensing layer, and a glass substrate layer.
-
Figure 2.
Validation of shape and hardness recognition ability of multiphotonic tactile neurons. (a) Schematic of the force applied to a single sensing unit of a multiphotonic neuron. (b) Changes in the path of light passing through the waist region under three forces. (c) Stress and deformation diagrams (indicated by color bars) of the waist region of a single sensing unit under three forces. (d) Optical power response plots of a single sensing unit under 0 to 3 N normal contact force in steps of 0.2 N, with each force transformation held for 5 s. (e) Normal force sensitivity from 0 to 3 N, with error bars indicating slight variations in optical power from the response time. (f) Individual sensing unit optical power response when pressure (1 N) is repeatedly applied more than 5000 times. The inset shows the optical power response in one of the two time domains after zooming in. (g) Pressure recognition based on object hardness and shape for six species using FCNN machine learning algorithm. (h) Visualization of clustered data clusters. (i) Confusion matrix of the six measured objects with 100% recognition accuracy.
-
Figure 3.
Validation of the material and surface texture recognition ability of micronized multiphoton neurons. (a) Neural network structure used for fabric recognition experiments with some signal acquisition results. (b) Accuracy change graphs of the training set and test set during the neural network training process, with the final accuracy stabilized at 99% for the training set and 98.5% for the test set. (c) Confusion matrix of ten fabric classification results in the validation set with 98.5% accuracy. (d) OMAS successfully realizes model training and online recognition of fabrics with the assistance of artificial intelligence algorithms. (e) Schematic diagram of 0-9 digit Braille. (f) Experimental process of real-time signal feedback and recognition of Braille phone numbers. (g) Real-time signal acquisition and recognition of Braille phone numbers displayed on the user graphical interface.
-
Figure 4.
Recognition of mahjong and its suits based on haptic parameters combined with spatial reconstruction of robotic arm. (a–c) With the help of AI algorithms, the multiphoton neuron haptic skin successfully recognizes different objects through the differences in hardness and shape, and the results are displayed on the PC screen. (d–f) Multi-photon neuron haptic skin successfully recognizes mahjong colors by differences in surface texture, and the results are displayed on the PC screen. (g, h) Computer interface for hardness and shape recognition and surface texture recognition.
