Citation: | Xu DY, Xu WH, Yang Q et al. All-optical object identification and three-dimensional reconstruction based on optical computing metasurface. Opto-Electron Adv 6, 230120 (2023). doi: 10.29026/oea.2023.230120 |
[1] | Rusinkiewicz S, Hall-Holt O, Levoy M. Real-time 3D model acquisition. ACM Trans Graphics 21, 438–446 (2002). doi: 10.1145/566654.566600 |
[2] | LiDAR drives forwards. Nat Photonics 12 (2018). doi: 10.1038/s41566-018-0235-z |
[3] | Zhang S. High-speed 3D shape measurement with structured light methods: a review. Opt Lasers Eng 106, 119–131 (2018). |
[4] | Kim G, Kim Y, Yun J, Moon SW, Kim S et al. Metasurface-driven full-space structured light for three-dimensional imaging. Nat Commun 13, 5920 (2022). doi: 10.1038/s41467-022-32117-2 |
[5] | Liu JW, Yang Q, Chen SZ, Xiao ZC, Wen SC et al. Intrinsic optical spatial differentiation enabled quantum dark-field microscopy. Phys Rev Lett 128, 193601 (2022). doi: 10.1103/PhysRevLett.128.193601 |
[6] | Buono WT, Forbes A. Nonlinear optics with structured light. Opto-Electron Adv 5, 210174 (2022). doi: 10.29026/oea.2022.210174 |
[7] | Gordon R, Herman GT. Three-dimensional reconstruction from projections: a review of algorithms. Int Rev Cytol 38, 111–151 (1974). |
[8] | Russakovsky O, Deng J, Su H, Krause J, Satheesh S et al. ImageNet large scale visual recognition challenge. Int J Comput Vis 115, 211–252 (2015). doi: 10.1007/s11263-015-0816-y |
[9] | Kwon H, Sounas D, Cordaro A, Polman A, Alù A. Nonlocal metasurfaces for optical signal processing. Phys Rev Lett 121, 173004 (2018). doi: 10.1103/PhysRevLett.121.173004 |
[10] | Zhou JX, Qian HL, Chen CF, Zhao JX, Li GR et al. Optical edge detection based on high-efficiency dielectric metasurface. Proc Natl Acad Sci U S A 116, 11137–11140 (2019). doi: 10.1073/pnas.1820636116 |
[11] | Caulfield HJ, Dolev S. Why future supercomputing requires optics. Nat Photonics 4, 261–263 (2010). doi: 10.1038/nphoton.2010.94 |
[12] | Silva A, Monticone F, Castaldi G, Galdi V, Alù A et al. Performing mathematical operations with metamaterials. Science 343, 160–163 (2014). doi: 10.1126/science.1242818 |
[13] | Liu WL, Li M, Guzzon RS, Norberg EJ, Parker JS et al. A fully reconfigurable photonic integrated signal processor. Nat Photonics 10, 190–195 (2016). doi: 10.1038/nphoton.2015.281 |
[14] | Zhou Y, Zheng HY, Kravchenko II, Valentine J. Flat optics for image differentiation. Nat Photonics 14, 316–323 (2020). doi: 10.1038/s41566-020-0591-3 |
[15] | Badloe T, Lee S, Rho J. Computation at the speed of light: metamaterials for all-optical calculations and neural networks. Adv Photonics 4, 064002 (2022). |
[16] | Zhang X, Huang LL, Zhao RZ, Zhou HQ, Li X et al. Basis function approach for diffractive pattern generation with Dammann vortex metasurfaces. Sci Adv 8, eabp8073 (2022). doi: 10.1126/sciadv.abp8073 |
[17] | Wu LM, Fan TJ, Wei SR, Xu YJ, Zhang Y et al. All-optical logic devices based on black arsenic–phosphorus with strong nonlinear optical response and high stability. Opto-Electron Adv 5, 200046 (2022). doi: 10.29026/oea.2022.200046 |
[18] | Kildishev AV, Boltasseva A, Shalaev VM. Planar photonics with metasurfaces. Science 339, 1232009 (2013). doi: 10.1126/science.1232009 |
[19] | Xu DY, Wen SC, Luo HL. Metasurface-based optical analog computing: from fundamentals to applications. Adv Devices Instrum 2022, 0002 (2022). doi: 10.34133/adi.0002 |
[20] | Zhang YX, Pu MB, Jin JJ, Lu XJ, Guo YH et al. Crosstalk-free achromatic full Stokes imaging polarimetry metasurface enabled by polarization-dependent phase optimization. Opto-Electron Adv 5, 220058 (2022). doi: 10.29026/oea.2022.220058 |
[21] | Shi YY, Wan CW, Dai CJ, Wan S, Liu Y et al. On-chip meta-optics for semi-transparent screen display in sync with AR projection. Optica 9, 670–676 (2022). doi: 10.1364/OPTICA.456463 |
[22] | Pertsch T, Xiao SM, Majumdar A, Li GX. Optical metasurfaces: fundamentals and applications. Photonics Res 11, OMFA1–OMFA3 (2023). doi: 10.1364/PRJ.487440 |
[23] | Zhou JX, Qian HL, Zhao JX, Tang M, Wu QY et al. Two-dimensional optical spatial differentiation and high-contrast imaging. Natl Sci Rev 8, nwaa176 (2021). doi: 10.1093/nsr/nwaa176 |
[24] | Wang YL, Fan QB, Xu T. Design of high efficiency achromatic metalens with large operation bandwidth using bilayer architecture. Opto-Electron Adv 4, 200008 (200008). doi: 10.29026/oea.2021.200008 |
[25] | Pors A, Nielsen MG, Bozhevolnyi SI. Analog computing using reflective plasmonic metasurfaces. Nano Lett 15, 791–797 (2015). doi: 10.1021/nl5047297 |
[26] | Wang SJ, Qin WT, Zhang S, Lou YC, Liu CQ et al. Nanoengineered spintronic-metasurface terahertz emitters enable beam steering and full polarization control. Nano Lett 22, 10111–10119 (2022). doi: 10.1021/acs.nanolett.2c03906 |
[27] | Huang YJ, Xiao TX, Chen S, Xie ZW, Zheng J et al. All-optical controlled-NOT logic gate achieving directional asymmetric transmission based on metasurface doublet. Opto-Electron Adv 6, 220073 (2023). doi: 10.29026/oea.2023.220073 |
[28] | Zhu TF, Zhou YH, Lou YJ, Ye H, Qiu M et al. Plasmonic computing of spatial differentiation. Nat Commun 8, 15391 (2017). doi: 10.1038/ncomms15391 |
[29] | Zhou Y, Wu WH, Chen R, Chen WJ, Chen RP et al. Analog optical spatial differentiators based on dielectric metasurfaces. Adv Opt Mater 8, 1901523 (2020). doi: 10.1002/adom.201901523 |
[30] | He Q, Zhang F, Pu MB, Ma XL, Li X et al. Monolithic metasurface spatial differentiator enabled by asymmetric photonic spin-orbit interactions. Nanophotonics 10, 741–748 (2020). doi: 10.1515/nanoph-2020-0366 |
[31] | Xu DY, Yang H, Xu WH, Zhang WS, Zeng KM et al. Inverse design of Pancharatnam–Berry phase metasurfaces for all-optical image edge detection. Appl Phys Lett 120, 241101 (2022). doi: 10.1063/5.0090606 |
[32] | Liang X, Zhou Z, Li ZL, Li JX, Peng C et al. All-optical multiplexed meta-differentiator for tri-mode surface morphology observation. Adv Mater 35, 2301505 (2023). doi: 10.1002/adma.202301505 |
[33] | Deng ZL, Tu QA, Wang YJ, Wang ZQ, Shi T et al. Vectorial compound metapixels for arbitrary nonorthogonal polarization steganography. Adv Mater 33, 2103472 (2021). doi: 10.1002/adma.202103472 |
[34] | Yang H, Ou K, Wan HY, Hu YQ, Wei ZY et al. Metasurface-empowered optical cryptography. Mater Today 67, 424–445 (2023). doi: 10.1016/j.mattod.2023.06.003 |
[35] | Zhang F, Guo YH, Pu MB, Chen LW, Xu MF et al. Meta-optics empowered vector visual cryptography for high security and rapid decryption. Nat Commun 14, 1946 (2023). doi: 10.1038/s41467-023-37510-z |
[36] | Pu MB, Li X, Ma XL, Wang YQ, Zhao ZY et al. Catenary optics for achromatic generation of perfect optical angular momentum. Sci Adv 1, e1500396 (2015). doi: 10.1126/sciadv.1500396 |
[37] | Luo XG, Pu MB, Li X, Ma XL. Broadband spin Hall effect of light in single nanoapertures. Light Sci Appl 6, e16276 (2017). doi: 10.1038/lsa.2016.276 |
[38] | Liu Y, Huang MC, Chen QK, Zhang DG. Single planar photonic chip with tailored angular transmission for multiple-order analog spatial differentiator. Nat Commun 13, 7944 (2022). doi: 10.1038/s41467-022-35588-5 |
[39] | Zeng C, Lu H, Mao D, Du YQ, Hua H et al. Graphene-empowered dynamic metasurfaces and metadevices. Opto-Electron Adv 5, 200098 (2022). doi: 10.29026/oea.2022.200098 |
[40] | Wang XW, Wang H, Wang JL, Liu XS, Hao HJ et al. Single-shot isotropic differential interference contrast microscopy. Nat Commun 14, 2063 (2023). doi: 10.1038/s41467-023-37606-6 |
[41] | Zhang XM, Zhou Y, Zheng HY, Linares AE, Ugwu FC et al. Reconfigurable metasurface for image processing. Nano Lett 21, 8715–8722 (2021). doi: 10.1021/acs.nanolett.1c02838 |
[42] | Xiao TT, Yang H, Yang Q, Xu DY, Wang RS et al. Realization of tunable edge-enhanced images based on computing metasurfaces. Opt Lett 47, 925–928 (2022). doi: 10.1364/OL.450988 |
[43] | Fu WW, Zhao D, Li ZQ, Liu SD, Tian C et al. Ultracompact meta-imagers for arbitrary all-optical convolution. Light Sci Appl 11, 62 (2022). doi: 10.1038/s41377-022-00752-5 |
[44] | Shen ZC, Zhao F, Jin CQ, Wang S, Cao LC et al. Monocular metasurface camera for passive single-shot 4D imaging. Nat Commun 14, 1035 (2023). doi: 10.1038/s41467-023-36812-6 |
[45] | Song MW, Feng L, Huo PC, Liu MZ, Huang CY et al. Versatile full-colour nanopainting enabled by a pixelated plasmonic metasurface. Nat Nanotechnol 18, 71–78 (2023). doi: 10.1038/s41565-022-01256-4 |
[46] | Jing XL, Zhao RZ, Li X, Jiang Q, Li CZ et al. Single-shot 3D imaging with point cloud projection based on metadevice. Nat Commun 13, 7842 (2022). doi: 10.1038/s41467-022-35483-z |
[47] | Jing XL, Li Y, Li JJ, Wang YT, Huang LL. Active 3D positioning and imaging modulated by single fringe projection with compact metasurface device. Nanophotonics 12, 1923–1930 (2023). doi: 10.1515/nanoph-2023-0112 |
[48] | He SS, Wang RS, Luo HL. Computing metasurfaces for all-optical image processing: a brief review. Nanophotonics 11, 1083–1108 (2022). doi: 10.1515/nanoph-2021-0823 |
[49] | Bomzon Z, Biener G, Kleiner V, Hasman E. Space-variant Pancharatnam–Berry phase optical elements with computer-generated subwavelength gratings. Opt Lett 27, 1141–1143 (2002). doi: 10.1364/OL.27.001141 |
[50] | Yin X, Ye Z, Rho J, Wang Y, Zhang X. Photonic spin Hall effect at metasurfaces. Science 339, 1405–1407 (2013). doi: 10.1126/science.1231758 |
[51] | Ling XH, Zhou XX, Yi XN, Shu WX, Liu YC et al. Giant photonic spin Hall effect in momentum space in a structured metamaterial with spatially varying birefringence. Light Sci Appl 4, e290 (2015). doi: 10.1038/lsa.2015.63 |
[52] | Peng L, Ren H, Liu YC, Lan TW, Xu KW et al. Spin Hall effect of transversely spinning light. Sci Adv 8, eabo6033 (2022). doi: 10.1126/sciadv.abo6033 |
[53] | Liu SQ, Chen SZ, Wen SC, Luo HL. Photonic spin Hall effect: fundamentals and emergent applications. Opto-Electron Sci 1, 220007 (2022). doi: 10.29026/oes.2022.220007 |
Supplementary information for All-optical object identification and threedimensional reconstruction based on computing metasurface |
Scheme illustration of object identification and all-optical 3D reconstruction system. (a) A contour surface image of the object can be obtained in a single processing of the system. (b) High-contrast objects and low-contrast objects can be reconstructed by this all-optical computing metasurface system.
Experimental demonstration of object identification ability. (a) Schematic diagram of the experimental optical path for object identification. L: lens; GLP: Glan laser polarizer; MS: the optical computing metasurface; CCD: charge-coupled device. Two lenses with focal lengths of 175 mm form a 4f system. He-Ne laser beam with wavelength λ = 632.8 nm is chosen as the experimental laser source. (b) The theory intensity distribution in planes 1 and 2, respectively. (c–d) Theoretical object identification results of high- and low-contrast objects. respectively. The first, second, and third rows represent the theoretical original, x-direction contours, and y-direction contours of those two types of objects, respectively. (e–f) Part of experimental identification results of high- and low-contrast objects, respectively. The first, second, and third rows represent the experimental the ordinary images, as well as the contour surfaces along the x axis and y axis of two types of objects, respectively.
Experimental demonstrations of an all-optical 3D high-contrast object reconstruction system. (a) Schematic diagram of the all-optical high-contrast object 3D reconstruction. Different color planes represent different projection planes. (b) Contour information results of an observed object on different projection planes in (a). (c) The 3D model reconstructed by recombining the different projection results captured in (b). (d1–d3) The origin image, the 3D experimental reconstruction models of rotation interval angle are 16° and 4° of coriander seed, respectively. (e1–f3) 3D experimental reconstruction models of the mushroom model and lollipop model with the same type as (d1–d3).
Experimental scheme of 3D reconstruction about the high-contrast object with complex surface. (a) The 3D reconstruction scheme relies on discretizing the target object into 2D slices with small gaps between them. (b) Contour information contained in every slice of an observed object would be captured. (c) The 3D model is reconstructed by recombining the different projection results captured in (b). (d–f) Original and 3D experimental reconstruction models of grooves, lands, and bosses, respectively. Scale bar, 2000 μm.
Experimental results of all-optical 3D low-contrast object reconstruction system. (a1−a3) The images of the nonuniform contour images obtained by rotating the angle β , 0°, and −β of the second GLP, along the y direction, respectively. (a4) The phase gradient result is supplied by subtracting (a1) and (a3) along the y direction. (b1−b4) The intensity distributions of images (a1–a4) at black dashed line. The horizontal and vertical coordinates represent pixels and intensity, respectively. (c1−d4) The same results of (a1−b4) along the