E-mail Alert
RSS
| Citation: |
Ma TG, Tobah M, Wang HZ, Guo LJ. Benchmarking deep learning-based models on nanophotonic inverse design problems. Opto-Electron Sci 1, 210012 (2022). doi: 10.29026/oes.2022.210012
|
Benchmarking deep learning-based models on nanophotonic inverse design problems
-
Abstract
Photonic inverse design concerns the problem of finding photonic structures with target optical properties. However, traditional methods based on optimization algorithms are time-consuming and computationally expensive. Recently, deep learning-based approaches have been developed to tackle the problem of inverse design efficiently. Although most of these neural network models have demonstrated high accuracy in different inverse design problems, no previous study has examined the potential effects under given constraints in nanomanufacturing. Additionally, the relative strength of different deep learning-based inverse design approaches has not been fully investigated. Here, we benchmark three commonly used deep learning models in inverse design: Tandem networks, Variational Auto-Encoders, and Generative Adversarial Networks. We provide detailed comparisons in terms of their accuracy, diversity, and robustness. We find that tandem networks and Variational Auto-Encoders give the best accuracy, while Generative Adversarial Networks lead to the most diverse predictions. Our findings could serve as a guideline for researchers to select the model that can best suit their design criteria and fabrication considerations. In addition, our code and data are publicly available, which could be used for future inverse design model development and benchmarking.-
Keywords:
- inverse design /
- photonics /
- machine learning /
- neural networks /
- generative models
-
-
References
[1] Shen YZ, Friend CS, Jiang Y, Jakubczyk D, Swiatkiewicz J et al. Nanophotonics: interactions, materials, and applications. J Phys Chem B 104, 7577–7587 (2000). doi: 10.1021/jp0016131 [2] Pu MB, Guo YH, Li X, Ma XL, Luo XG. Revisitation of extraordinary young’s interference: from catenary optical fields to spin–orbit interaction in metasurfaces. ACS Photonics 5, 3198–3204 (2018). doi: 10.1021/acsphotonics.8b00437 [3] Gan XT, Mak KF, Gao YD, You YM, Hatami F et al. Strong enhancement of light–matter interaction in graphene coupled to a photonic crystal nanocavity. Nano Lett 12, 5626–5631 (2012). doi: 10.1021/nl302746n [4] de Leon NP, Shields BJ, Yu CL, Englund DE, Akimov AV et al. Tailoring light-matter interaction with a nanoscale Plasmon resonator. Phys Rev Lett 108, 226803 (2012). doi: 10.1103/PhysRevLett.108.226803 [5] Baranov DG, Wersäll M, Cuadra J, Antosiewicz TJ, Shegai T. Novel nanostructures and materials for strong light–matter interactions. Acs Photonics 5, 24–42 (2018). [6] Yu NF, Capasso F. Flat optics with designer metasurfaces. Nat Mater 13, 139–150 (2014). doi: 10.1038/nmat3839 [7] Yu NF, Genevet P, Kats MA, Aieta F, Tetienne JP et al. Light propagation with phase discontinuities: generalized laws of reflection and refraction. Science 334, 333–337 (2011). doi: 10.1126/science.1210713 [8] Huang YJ, Luo J, Pu MB, Guo YH, Zhao ZY et al. Catenary electromagnetics for ultra‐broadband lightweight absorbers and large‐scale flat antennas. Adv Sci 6, 1801691 (2019). doi: 10.1002/advs.201801691 [9] Li X, Chen LW, Li Y, Zhang XH, Pu MB et al. Multicolor 3D meta-holography by broadband plasmonic modulation. Sci Adv 2, e1601102 (2016). doi: 10.1126/sciadv.1601102 [10] Zheng GX, Mühlenbernd H, Kenney M, Li GX, Zentgraf T et al. Metasurface holograms reaching 80% efficiency. Nat Nanotechnol 10, 308–312 (2015). doi: 10.1038/nnano.2015.2 [11] Staude I, Miroshnichenko AE, Decker M, Fofang NT, Liu S et al. Tailoring directional scattering through magnetic and electric resonances in subwavelength silicon nanodisks. ACS Nano 7, 7824–7832 (2013). doi: 10.1021/nn402736f [12] Lin DM, Fan PY, Hasman E, Brongersma ML. Dielectric gradient metasurface optical elements. Science 345, 298–302 (2014). doi: 10.1126/science.1253213 [13] Nagarajan R, Joyner CH, Schneider RP, Bostak JS, Butrie T et al. Large-scale photonic integrated circuits. IEEE J Sel Top Quant Electron 11, 50–65 (2005). doi: 10.1109/JSTQE.2004.841721 [14] Maier SA. Metamaterials and imaging with surface Plasmon polaritons. In Maier SA. Plasmonics: Fundamentals and Applications. 193–200 (Springer, 2007); http://doi.org/10.1007/0-387-37825-1_11. [15] Decker M, Staude I, Falkner M, Dominguez J, Neshev DN et al. High‐efficiency dielectric Huygens’ surfaces. Adv Opt Mater 3, 813–820 (2015). doi: 10.1002/adom.201400584 [16] Stern B, Ji XC, Okawachi Y, Gaeta AL, Lipson M. Battery-operated integrated frequency comb generator. Nature 562, 401–405 (2018). doi: 10.1038/s41586-018-0598-9 [17] Sun J, Timurdogan E, Yaacobi A, Hosseini ES, Watts MR. Large-scale nanophotonic phased array. Nature 493, 195–199 (2013). doi: 10.1038/nature11727 [18] Cheng QX, Bahadori M, Glick M, Rumley S, Bergman K. Recent advances in optical technologies for data centers: a review. Optica 5, 1354–1370 (2018). doi: 10.1364/OPTICA.5.001354 [19] Thomson D, Zilkie A, Bowers JE, Komljenovic T, Reed GT et al. Roadmap on silicon photonics. J Opt 18, 073003 (2016). doi: 10.1088/2040-8978/18/7/073003 [20] Walmsley I. Photonic quantum technologies. Proceedings of SPIE 11844, 11844OF (2021). [21] Tittl A, Leitis A, Liu MK, Yesilkoy F, Choi DY et al. Imaging-based molecular barcoding with pixelated dielectric metasurfaces. Science 360, 1105–1109 (2018). doi: 10.1126/science.aas9768 [22] Chen LW, Yin YM, Li Y, Hong MH. Multifunctional inverse sensing by spatial distribution characterization of scattering photons. Opto-Electron Adv 2, 190019 (2019). [23] Nguyen TT, Lim S. Wide incidence angle-insensitive metamaterial absorber for both TE and TM polarization using eight-circular-sector. Sci Rep 7, 3204 (2017). doi: 10.1038/s41598-017-03591-2 [24] Kim I, So S, Rana AS, Mehmood MQ, Rho J. Thermally robust ring-shaped chromium perfect absorber of visible light. Nanophotonics 7, 1827–1833 (2018). doi: 10.1515/nanoph-2018-0095 [25] Campbell SD, Sell D, Jenkins RP, Whiting EB, Fan JA et al. Review of numerical optimization techniques for meta-device design [Invited]. Opt Mater Express 9, 1842–1863 (2019). doi: 10.1364/OME.9.001842 [26] Hansen E. Interval forms of Newtons method. Computing 20, 153–163 (1978). doi: 10.1007/BF02252344 [27] Ruder S. An overview of gradient descent optimization algorithms. arXiv: 1609.04747 (2017). [28] Kim WJ, O’Brien J. Optimization of a two-dimensional photonic-crystal waveguide branch by simulated annealing and the finite-element method. J Opt Soc Am B 21, 289–295 (2004). doi: 10.1364/JOSAB.21.000289 [29] Lalau-Keraly CM, Bhargava S, Miller OD, Yablonovitch E. Adjoint shape optimization applied to electromagnetic design. Opt Express 21, 21693–21701 (2013). doi: 10.1364/OE.21.021693 [30] Storn R, Price K. Differential evolution – A simple and efficient heuristic for global optimization over continuous spaces. J Glob Optim 11, 341–359 (1997). doi: 10.1023/A:1008202821328 [31] Poli R, Kennedy J, Blackwell T. Particle swarm optimization. Swarm Intell 1, 33–57 (2007). doi: 10.1007/s11721-007-0002-0 [32] Snoek J, Larochelle H, Adams RP. Practical Bayesian optimization of machine learning algorithms. In Proceedings of the 25th International Conference on Neural Information Processing Systems 2951–2959 (Curran Associates Inc. , 2012). [33] Schneider PI, Santiago XG, Soltwisch V, Hammerschmidt M, Burger S et al. Benchmarking five global optimization approaches for nano-optical shape optimization and parameter reconstruction. ACS Photonics 6, 2726–2733 (2019). doi: 10.1021/acsphotonics.9b00706 [34] Yang WH, Xiao SM, Song QH, Liu YL, Wu YK et al. All-dielectric metasurface for high-performance structural color. Nat Commun 11, 1864 (2020). doi: 10.1038/s41467-020-15773-0 [35] Liu DJ, Tan YX, Khoram E, Yu ZF. Training deep neural networks for the inverse design of nanophotonic structures. ACS Photonics 5, 1365–1369 (2018). doi: 10.1021/acsphotonics.7b01377 [36] Ma W, Cheng F, Xu YH, Wen QL, Liu YM. Probabilistic representation and inverse design of metamaterials based on a deep generative model with semi‐supervised learning strategy. Adv Mater 31, 1901111 (2019). doi: 10.1002/adma.201901111 [37] Liu ZC, Zhu DY, Rodrigues SP, Lee KT, Cai WS. Generative model for the inverse design of metasurfaces. Nano Lett 18, 6570–6576 (2018). doi: 10.1021/acs.nanolett.8b03171 [38] Wiecha PR, Arbouet A, Girard C, Muskens OL. Deep learning in nano-photonics: inverse design and beyond. Photonics Res 9, B182–B200 (2021). doi: 10.1364/PRJ.415960 [39] Khatib O, Ren SM, Malof J, Padilla WJ. Deep learning the electromagnetic properties of metamaterials—a comprehensive review. Adv Funct Mater 31, 2101748 (2021). doi: 10.1002/adfm.202101748 [40] Jiang JQ, Chen MK, Fan JA. Deep neural networks for the evaluation and design of photonic devices. Nat Rev Mater 6, 679–700 (2021). doi: 10.1038/s41578-020-00260-1 [41] Ma W, Liu ZC, Kudyshev ZA, Boltasseva A, Cai WS et al. Deep learning for the design of photonic structures. Nat Photonics 15, 77–90 (2021). doi: 10.1038/s41566-020-0685-y [42] Jordan MI. Constrained supervised learning. J Math Psychol 36, 396–425 (1992). doi: 10.1016/0022-2496(92)90029-7 [43] Jordan MI, Rumelhart DE. Forward models: supervised learning with a distal teacher. Cogn Sci 16, 307–354 (1992). doi: 10.1207/s15516709cog1603_1 [44] Sohn K, Yan XC, Lee H. Learning structured output representation using deep conditional generative models. In Proceedings of the 28th International Conference on Neural Information Processing Systems 3483–3491 (MIT Press, 2015). [45] Mirza M, Osindero S. Conditional generative adversarial nets. arXiv: 1411.1784 (2014). [46] Gao L, Li XZ, Liu DJ, Wang LH, Yu ZF. A bidirectional deep neural network for accurate silicon color design. Adv Mater 31, 1905467 (2019). doi: 10.1002/adma.201905467 [47] Ma W, Cheng F, Liu YM. Deep-learning-enabled on-demand design of chiral metamaterials. ACS Nano 12, 6326–6334 (2018). doi: 10.1021/acsnano.8b03569 [48] Kingma DP, Welling M. Auto-encoding variational Bayes. arXiv: 1312.6114 (2014). [49] Jiang JQ, Sell D, Hoyer S, Hickey J, Yang JJ et al. Free-form diffractive metagrating design based on generative adversarial networks. ACS Nano 13, 8872–8878 (2019). doi: 10.1021/acsnano.9b02371 [50] So S, Rho J. Designing nanophotonic structures using conditional deep convolutional generative adversarial networks. Nanophotonics 8, 1255–1261 (2019). doi: 10.1515/nanoph-2019-0117 [51] https://github.com/taigaoma1997/benchmark_in_de.git. [52] Pal SK, Mitra S. Multilayer perceptron, fuzzy sets, and classification. IEEE Trans Neural Netw 3, 683–697 (1992). doi: 10.1109/72.159058 [53] Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Commun ACM 60, 84–90 (2017). doi: 10.1145/3065386 [54] Hugonin JP, Lalanne P. RETICOLO software for grating analysis. arXiv: 2101.00901 (2021). [55] Han X, Fan ZY, Liu ZY, Li C, Guo LJ. Inverse design of metasurface optical filters using deep neural network with high degrees of freedom. InfoMat 3, 432–442 (2021). doi: 10.1002/inf2.12116 [56] Malkiel I, Mrejen M, Nagler A, Arieli U, Wolf L et al. Plasmonic nanostructure design and characterization via Deep Learning. Light:Sci Appl 7, 60 (2018). doi: 10.1038/s41377-018-0060-7 [57] Chen MK, Jiang JQ, Fan JA. Design space reparameterization enforces hard geometric constraints in inverse-designed nanophotonic devices. ACS Photonics 7, 3141–3151 (2020). doi: 10.1021/acsphotonics.0c01202 [58] Sell D, Yang JJ, Doshay S, Yang R, Fan JA. Large-angle, multifunctional metagratings based on freeform multimode geometries. Nano Lett 17, 3752–3757 (2017). doi: 10.1021/acs.nanolett.7b01082 [59] Wang HZ, Zheng ZY, Ji CG, Guo LJ. Automated multi-layer optical design via deep reinforcement learning. Mach Learn:Sci Technol 2, 025013 (2021). doi: 10.1088/2632-2153/abc327 [60] Jiang JQ, Fan JA. Global optimization of dielectric metasurfaces using a physics-driven neural network. Nano Lett 19, 5366–5372 (2019). doi: 10.1021/acs.nanolett.9b01857 [61] Jiang JQ, Fan JA. Simulator-based training of generative neural networks for the inverse design of metasurfaces. Nanophotonics 9, 1059–1069 (2019). doi: 10.1515/nanoph-2019-0330 -
Supplementary Information
Supplementary information for Benchmarking deep learning-based models on nanophotonic inverse design problems 
-
Access History
Figures(6)
Tables(3)
Article Metrics
Export File
Citation
Ma TG, Tobah M, Wang HZ, Guo LJ. Benchmarking deep learning-based models on nanophotonic inverse design problems. Opto-Electron Sci 1, 210012 (2022). doi: 10.29026/oes.2022.210012
Format
Content
DownLoad:
-
Figure 1.
The structure of three considered models: (a) Tandem networks, (b) VAEs, (c) GANs. The detailed descriptions of building and training each neural networks can be found in the supplementary information.
-
Figure 2.
(a) The template structure for silicon structural color inverse design. Four structural parameters (D, H, G, P) are shown in the inset figure. (b) The obtained structural colors in the training dataset embedded in the CIE 1931 chromatic diagram, which cover a wide color gamut. (c) The free-form structure for silicon transmission filter inverse design. The inset is the period of the structure and the 2D pattern treated as an image. (d) One example of the TE/TM transmission spectra in the training dataset. The inset shows the 2D free-form structure with period 283 nm, where the yellow and black regions are the dielectric material and air, respectively.
-
Figure 3.
(a) Five randomly selected examples of color inverse design (blue, brown, red, yellow, and green). The first row is the target color, where the inset numbers are the target CIE (x, y, Y) coordinates. The second, third, and forth row corresponds to the predicted structural color by tandem networks, VAE and GAN, respectively, where the inset numbers are the absolute difference of each CIE coordinate. (b) The percent of predicted color tasks that have MAE is smaller than a given threshold. The solid lines show the results when only sampling once for each model, while the dashed lines show results when sampling ten times and picking the most accurate one. As we increase the predicting times, the accuracy of generative models (VAEs and GANs) improved. (c–h) Comparison of one specific application of structural color inverse design: reproducing a paint. (c) The original image of the Vincent van Gogh’s painting: Fishing Boats on the Beach at Saintes Maries-de-la-Mer. (d) The image reconstructed by the predictions of Tandem networks. (e) The image reconstructed by the predictions of the VAE. (f) The image reconstructed by the predictions of VAE when sampling ten times. (g) The image reconstructed by the predictions of GAN. (h) The image reconstructed by the predictions of GAN when sampling ten times. (i–m) The comparison of three models’ robustness with respect to the size of the array for five colors: (i) Blue, (j) Brown, (k) Red, (l) Yellow, (m) Green. To calculate the color, we are not considering the structure to be infinitely periodic anymore. Instead, we are simulating the color within a limited region that only contains the 2 by 2, 3 by 3, 4 by 4, and 5 by 5 array of unit cells, respectively. Fishing Boats on the Beach at Saintes-Maries-de-la-Mer” is reproduced with the permission of the Van Gogh Museum, Amsterdam (Vincent van Gogh Foundation).
-
Figure 4.
(a) The normalized density distribution of 1000 inverse designed structure parameters for the green color (c) with the CIE coordinates (x, y, Y) = (0.2917, 0.5711, 0.4720). (b) The 3-dimensional color distribution related to 1,000 inverse designed structures. We can see all these predicted structures give a fairly accurate green color. (c) The target green color with coordinates (x, y, Y) = (0.2917, 0.5711, 0.4720). (d) The color corresponding to the structure predicted by tandem networks. (e) The randomly selected 40 different colors corresponding to the structures predicted by the VAE. (f) The randomly selected 40 different colors corresponding to the structures predicted by the GAN.
-
Figure 5.
Two randomly selected examples of transmission spectrum inverse design for the tandem networks (a)(d), VAEs (b)(e), and GANs (c)(f). The inset shows the original structure (upper) in the test dataset and the inverse predicted structure (lower) by each model.
-
Figure 6.
Comparisons of the diversity for three models. (a–c) The red bar shows the distribution of the normalized density of irregularity for 1000 inverse predicted structures by tandem networks, VAEs and GANs, while the green points are the scatter plot of spectrum RMSE V.S. the irregularity. According to the distribution of irregularity, we can see that tandem networks only give one structure prediction, where the VAE gives limited diversity, and the GAN gives a multi-modal structure distribution that covers a wide region. (d–f) A randomly chosen inverse designed structure predicted by tandem networks, VAEs and GANs, as well as its corresponding transmission spectra. The inset shows the original structure (upper) in test dataset and the predicted structure (lower) given by each model.
