He C, Zhao D, Fan F et al. Pluggable multitask diffractive neural networks based on cascaded metasurfaces. Opto-Electron Adv 7, 230005 (2024). doi: 10.29026/oea.2024.230005
Citation: He C, Zhao D, Fan F et al. Pluggable multitask diffractive neural networks based on cascaded metasurfaces. Opto-Electron Adv 7, 230005 (2024). doi: 10.29026/oea.2024.230005

Article Open Access

Pluggable multitask diffractive neural networks based on cascaded metasurfaces

More Information
  • Optical neural networks have significant advantages in terms of power consumption, parallelism, and high computing speed, which has intrigued extensive attention in both academic and engineering communities. It has been considered as one of the powerful tools in promoting the fields of imaging processing and object recognition. However, the existing optical system architecture cannot be reconstructed to the realization of multi-functional artificial intelligence systems simultaneously. To push the development of this issue, we propose the pluggable diffractive neural networks (P-DNN), a general paradigm resorting to the cascaded metasurfaces, which can be applied to recognize various tasks by switching internal plug-ins. As the proof-of-principle, the recognition functions of six types of handwritten digits and six types of fashions are numerical simulated and experimental demonstrated at near-infrared regimes. Encouragingly, the proposed paradigm not only improves the flexibility of the optical neural networks but paves the new route for achieving high-speed, low-power and versatile artificial intelligence systems.
  • 加载中
  • [1] Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Commun ACM 60, 84–90 (2017). doi: 10.1145/3065386

    CrossRef Google Scholar

    [2] He KM, Zhang XY, Ren SQ, Sun J. Deep residual learning for image recognition. In Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition 770–778 (IEEE, 2016); http://doi.org/10.1109/CVPR.2016.90.

    Google Scholar

    [3] Zhou HQ, Wang YT, Li X, Xu ZT, Li XW et al. A deep learning approach for trustworthy high-fidelity computational holographic orbital angular momentum communication. Appl Phys Lett 119, 044104 (2021). doi: 10.1063/5.0051132

    CrossRef Google Scholar

    [4] Guo YM, Zhong LB, Min L, Wang JY, Wu Y et al. Adaptive optics based on machine learning: a review. Opto-Electron Adv 5, 200082 (2022). doi: 10.29026/oea.2022.200082

    CrossRef Google Scholar

    [5] Krasikov S, Tranter A, Bogdanov A, Kivshar Y. Intelligent metaphotonics empowered by machine learning. Opto-Electron Adv 5, 210147 (2022). doi: 10.29026/oea.2022.210147

    CrossRef Google Scholar

    [6] Sainath TN, Mohamed AR, Kingsbury B, Ramabhadran B. Deep convolutional neural networks for LVCSR. In Proceedings of 2013 IEEE International Conference on Acoustics, Speech and Signal Processing 8614–8618 (IEEE, 2013);http://doi.org/10.1109/ICASSP.2013.6639347.

    Google Scholar

    [7] Hinton G, Deng L, Yu D, Dahl GE, Mohamed AR et al. Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Process Mag 29, 82–97 (2012). doi: 10.1109/MSP.2012.2205597

    CrossRef Google Scholar

    [8] Collobert R, Weston J, Bottou L, Karlen M, Kavukcuoglu K et al. Natural language processing (almost) from scratch. J Mach Learn Res 12, 2493–2537 (2011). doi: 10.5555/1953048.2078186

    CrossRef Google Scholar

    [9] Markram H. The blue brain project. Nat Rev Neurosci 7, 153–160 (2006). doi: 10.1038/nrn1848

    CrossRef Google Scholar

    [10] Shen YC, Harris NC, Skirlo S, Prabhu M, Baehr-Jones T et al. Deep learning with coherent nanophotonic circuits. Nat Photonics 11, 441–446 (2017). doi: 10.1038/nphoton.2017.93

    CrossRef Google Scholar

    [11] Feldmann J, Youngblood N, Karpov M, Gehring H, Li X et al. Parallel convolutional processing using an integrated photonic tensor core. Nature 589, 52–58 (2021). doi: 10.1038/s41586-020-03070-1

    CrossRef Google Scholar

    [12] Xu XY, Tan MX, Corcoran B, Wu JY, Boes A et al. 11 TOPS photonic convolutional accelerator for optical neural networks. Nature 589, 44–51 (2021). doi: 10.1038/s41586-020-03063-0

    CrossRef Google Scholar

    [13] Goi E, Chen X, Zhang QM, Cumming BP, Schoenhardt S et al. Nanoprinted high-neuron-density optical linear perceptrons performing near-infrared inference on a CMOS chip. Light Sci Appl 10, 40 (2021). doi: 10.1038/s41377-021-00483-z

    CrossRef Google Scholar

    [14] Ashtiani F, Geers AJ, Aflatouni F. An on-chip photonic deep neural network for image classification. Nature 606, 501–506 (2022). doi: 10.1038/s41586-022-04714-0

    CrossRef Google Scholar

    [15] Zarei S, Marzban MR, Khavasi A. Integrated photonic neural network based on silicon metalines. Opt Express 28, 36668–36684 (2020). doi: 10.1364/OE.404386

    CrossRef Google Scholar

    [16] Chen H, Feng JN, Jiang MW, Wang YQ, Lin J et al. Diffractive deep neural networks at visible wavelengths. Engineering 7, 1483–1491 (2021). doi: 10.1016/j.eng.2020.07.032

    CrossRef Google Scholar

    [17] Liu J, Wu QH, Sui XB, Chen Q, Gu GH et al. Research progress in optical neural networks: theory, applications and developments. PhotoniX 2, 5 (2021). doi: 10.1186/s43074-021-00026-0

    CrossRef Google Scholar

    [18] Zhang X, Huang LL, Zhao RZ, Zhou HQ, Li X et al. Basis function approach for diffractive pattern generation with Dammann vortex metasurfaces. Sci Adv 8, eabp8073 (2022). doi: 10.1126/sciadv.abp8073

    CrossRef Google Scholar

    [19] Lin X, Rivenson Y, Yardimci NT, Veli M, Luo Y et al. All-optical machine learning using diffractive deep neural networks. Science 361, 1004–1008 (2018). doi: 10.1126/science.aat8084

    CrossRef Google Scholar

    [20] Zhao RZ, Huang LL, Wang YT. Recent advances in multi-dimensional metasurfaces holographic technologies. PhotoniX 1, 20 (2020). doi: 10.1186/s43074-020-00020-y

    CrossRef Google Scholar

    [21] Zhang YX, Pu MB, Jin JJ, Lu XJ, Guo YH et al. Crosstalk-free achromatic full Stokes imaging polarimetry metasurface enabled by polarization-dependent phase optimization. Opto-Electron Adv 5, 220058 (2022). doi: 10.29026/oea.2022.220058

    CrossRef Google Scholar

    [22] Badloe T, Lee S, Rho J. Computation at the speed of light: metamaterials for all-optical calculations and neural networks. Adv Photon 4, 064002 (2022). doi: 10.1117/1.AP.4.6.064002

    CrossRef Google Scholar

    [23] Veli M, Mengu D, Yardimci NT, Luo Y, Li JX et al. Terahertz pulse shaping using diffractive surfaces. Nat Commun 12, 37 (2021). doi: 10.1038/s41467-020-20268-z

    CrossRef Google Scholar

    [24] Qian C, Lin X, Lin XB, Xu J, Sun Y et al. Performing optical logic operations by a diffractive neural network. Light Sci Appl 9, 59 (2020). doi: 10.1038/s41377-020-0303-2

    CrossRef Google Scholar

    [25] Wang PP, Xiong WJ, Huang ZB, He YL, Xie ZQ et al. Orbital angular momentum mode logical operation using optical diffractive neural network. Photon Res 9, 2116–2124 (2021). doi: 10.1364/PRJ.432919

    CrossRef Google Scholar

    [26] Huang ZB, He YL, Wang PP, Xiong WJ, Wu HS et al. Orbital angular momentum deep multiplexing holography via an optical diffractive neural network. Opt Express 30, 5569–5584 (2022). doi: 10.1364/OE.447337

    CrossRef Google Scholar

    [27] Rahman SS, Ozcan A. Computer-free, all-optical reconstruction of holograms using diffractive networks. ACS Photonics 8, 3375–3384 (2021). doi: 10.1021/acsphotonics.1c01365

    CrossRef Google Scholar

    [28] Mengu D, Ozcan A. All-optical phase recovery: diffractive computing for quantitative phase imaging. Adv Opt Mater 10, 2200281 (2022). doi: 10.1002/adom.202200281

    CrossRef Google Scholar

    [29] Li JX, Hung YC, Kulce O, Mengu D, Ozcan A. Polarization multiplexed diffractive computing: all-optical implementation of a group of linear transformations through a polarization-encoded diffractive network. Light Sci Appl 11, 153 (2022). doi: 10.1038/s41377-022-00849-x

    CrossRef Google Scholar

    [30] Shi JS, Zhou L, Liu TG, Hu C, Liu KW et al. Multiple-view D2NNs array: realizing robust 3D object recognition. Opt Lett 46, 3388–3391 (2021). doi: 10.1364/OL.432309

    CrossRef Google Scholar

    [31] Rahman SS, Li JX, Mengu D, Rivenson Y, Ozcan A. Ensemble learning of diffractive optical networks. Light Sci Appl 10, 14 (2021). doi: 10.1038/s41377-020-00446-w

    CrossRef Google Scholar

    [32] Yan T, Wu JM, Zhou TK, Xie H, Xu F et al. Fourier-space diffractive deep neural network. Phys Rev Lett 123, 023901 (2019). doi: 10.1103/PhysRevLett.123.023901

    CrossRef Google Scholar

    [33] Liu C, Ma Q, Luo ZJ, Hong QR, Xiao Q et al. A programmable diffractive deep neural network based on a digital-coding metasurface array. Nat Electron 5, 113–122 (2022). doi: 10.1038/s41928-022-00719-9

    CrossRef Google Scholar

    [34] Zhou TK, Lin X, Wu JM, Chen YT, Xie H et al. Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit. Nat Photonics 15, 367–373 (2021). doi: 10.1038/s41566-021-00796-w

    CrossRef Google Scholar

    [35] Li YJ, Chen RY, Sensale-Rodriguez B, Gao WL, Yu CX. Real-time multi-task diffractive deep neural networks via hardware-software co-design. Sci Rep 11, 11013 (2021). doi: 10.1038/s41598-021-90221-7

    CrossRef Google Scholar

    [36] Luo XH, Hu YQ, Ou XN, Li X, Lai JJ et al. Metasurface-enabled on-chip multiplexed diffractive neural networks in the visible. Light Sci Appl 11, 158 (2022). doi: 10.1038/s41377-022-00844-2

    CrossRef Google Scholar

    [37] Georgi P, Wei QS, Sain B, Schlickriede C, Wang YT et al. Optical secret sharing with cascaded metasurface holography. Sci Adv 7, eabf9718 (2021). doi: 10.1126/sciadv.abf9718

    CrossRef Google Scholar

    [38] Lecun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proc IEEE 86, 2278–2324 (1998). doi: 10.1109/5.726791

    CrossRef Google Scholar

    [39] Xiao H, Rasul K, Vollgraf R. Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. arXiv: 1708.07747 (2017). https://doi.org/10.48550/arXiv.1708.07747

    Google Scholar

    [40] Goodman JW. Introduction to Fourier Optics and Holography 3rd ed (Roberts and Company, Englewood, 2005).

    Google Scholar

    [41] Mandel L, Wolf E. Some properties of coherent light. J Opt Soc Am 51, 815–819 (1961). doi: 10.1364/JOSA.51.000815

    CrossRef Google Scholar

    [42] Marrucci L, Manzo C, Paparo D. Optical spin-to-orbital angular momentum conversion in inhomogeneous anisotropic media. Phys Rev Lett 96, 163905 (2006). doi: 10.1103/PhysRevLett.96.163905

    CrossRef Google Scholar

  • Supplementary information for Pluggable multitask diffractive neural networks based on cascaded metasurfaces
  • 加载中
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

Figures(6)

Article Metrics

Article views(4259) PDF downloads(986) Cited by(0)

Access History
Article Contents

Catalog

    /

    DownLoad:  Full-Size Img  PowerPoint