Liu KX, Wu JC, He ZH, Cao LC. 4K-DMDNet: diffraction model-driven network for 4K computer-generated holography. Opto-Electron Adv 6, 220135 (2023). doi: 10.29026/oea.2023.220135
Citation: Liu KX, Wu JC, He ZH, Cao LC. 4K-DMDNet: diffraction model-driven network for 4K computer-generated holography. Opto-Electron Adv 6, 220135 (2023). doi: 10.29026/oea.2023.220135

Article Open Access

4K-DMDNet: diffraction model-driven network for 4K computer-generated holography

More Information
  • Deep learning offers a novel opportunity to achieve both high-quality and high-speed computer-generated holography (CGH). Current data-driven deep learning algorithms face the challenge that the labeled training datasets limit the training performance and generalization. The model-driven deep learning introduces the diffraction model into the neural network. It eliminates the need for the labeled training dataset and has been extensively applied to hologram generation. However, the existing model-driven deep learning algorithms face the problem of insufficient constraints. In this study, we propose a model-driven neural network capable of high-fidelity 4K computer-generated hologram generation, called 4K Diffraction Model-driven Network (4K-DMDNet). The constraint of the reconstructed images in the frequency domain is strengthened. And a network structure that combines the residual method and sub-pixel convolution method is built, which effectively enhances the fitting ability of the network for inverse problems. The generalization of the 4K-DMDNet is demonstrated with binary, grayscale and 3D images. High-quality full-color optical reconstructions of the 4K holograms have been achieved at the wavelengths of 450 nm, 520 nm, and 638 nm.
  • 加载中
  • [1] Shi L, Li BC, Kim C, Kellnhofer P, Matusik W. Towards real-time photorealistic 3D holography with deep neural networks. Nature 591, 234–239 (2021). doi: 10.1038/s41586-020-03152-0

    CrossRef Google Scholar

    [2] Zhang CL, Zhang DF, Bian ZP. Dynamic full-color digital holographic 3D display on single DMD. Opto-Electron Adv 4, 200049 (2021). doi: 10.29026/oea.2021.200049

    CrossRef Google Scholar

    [3] He ZH, Sui XM, Jin GF, Cao LC. Progress in virtual reality and augmented reality based on holographic display. Appl Opt 58, A74–A81 (2019). doi: 10.1364/AO.58.000A74

    CrossRef Google Scholar

    [4] Zhao Y, Cao LC, Zhang H, Kong DZ, Jin GF. Accurate calculation of computer-generated holograms using angular-spectrum layer-oriented method. Opt Express 23, 25440–25449 (2015). doi: 10.1364/OE.23.025440

    CrossRef Google Scholar

    [5] Jiang Q, Jin GF, Cao LC. When metasurface meets hologram: principle and advances. Adv Opt Photonics 11, 518–576 (2019). doi: 10.1364/AOP.11.000518

    CrossRef Google Scholar

    [6] Huang LL, Chen XZ, Mühlenbernd H, Zhang H, Chen SM et al. Three-dimensional optical holography using a plasmonic metasurface. Nat Commun 4, 2808 (2013). doi: 10.1038/ncomms3808

    CrossRef Google Scholar

    [7] Guo JY, Wang T, Quan BG, Zhao H, Gu CZ et al. Polarization multiplexing for double images display. Opto-Electron Adv 2, 180029 (2019). doi: 10.29026/oea.2019.180029

    CrossRef Google Scholar

    [8] Gao H, Fan XH, Xiong W, Hong MH. Recent advances in optical dynamic meta-holography. Opto-Electron Adv 4, 210030 (2021). doi: 10.29026/oea.2021.210030

    CrossRef Google Scholar

    [9] Saha SK, Wang D, Nguyen VH, Chang Y, Oakdale JS et al. Scalable submicrometer additive manufacturing. Science 366, 105–109 (2019). doi: 10.1126/science.aax8760

    CrossRef Google Scholar

    [10] Gerchberg RW, Saxton WOA. A practical algorithm for the determination of phase from image and diffraction plane pictures. Optik 35, 237–246 (1972).

    Google Scholar

    [11] Tian SZ, Chen LZ and Zhang H. Optimized Fresnel phase hologram for ringing artifacts removal in lensless holographic projection. Appl Opt 61, B17–B24 (2022).

    Google Scholar

    [12] Chakravarthula P, Peng YF, Kollin J, Fuchs H, Heide F. Wirtinger holography for near-eye displays. ACM Trans Graph 38, 213 (2019). doi: 10.1145/3355089.3356539

    CrossRef Google Scholar

    [13] Zhang JZ, Pégard N, Zhong JS, Adesnik H, Waller L. 3D computer-generated holography by non-convex optimization. Optica 4, 1306–1313 (2017). doi: 10.1364/OPTICA.4.001306

    CrossRef Google Scholar

    [14] He KM, Zhang XY, Ren SQ, Sun J. Deep residual learning for image recognition. In Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition 770–778 (IEEE, 2016);http://doi.org/10.1109/CVPR.2016.90.

    Google Scholar

    [15] Liao MH, Zheng SS, Pan SX, Lu DJ, He WQ et al. Deep-learning-based ciphertext-only attack on optical double random phase encryption. Opto-Electron Adv 4, 200016 (2021). doi: 10.29026/oea.2021.200016

    CrossRef Google Scholar

    [16] Li YX, Qian JM, Feng SJ, Chen Q, Zuo C. Deep-learning-enabled dual-frequency composite fringe projection profilometry for single-shot absolute 3D shape measurement. Opto-Electron Adv 5, 210021 (2022). doi: 10.29026/oea.2022.210021

    CrossRef Google Scholar

    [17] Blinder D, Birnbaum T, Ito T, Shimobaba T. The state-of-the-art in computer generated holography for 3D display. Light Adv Manuf 3, 35 (2022). doi: 10.37188/lam.2022.035

    CrossRef Google Scholar

    [18] He ZH, Sui XM, Jin GF, Chu DP, Cao LC. Optimal quantization for amplitude and phase in computer-generated holography. Opt Express 29, 119–133 (2021). doi: 10.1364/OE.414160

    CrossRef Google Scholar

    [19] Sui XM, He ZH, Jin GF, Chu DP, Cao LC. Band-limited double-phase method for enhancing image sharpness in complex modulated computer-generated holograms. Opt Express 29, 2597–2612 (2021). doi: 10.1364/OE.414299

    CrossRef Google Scholar

    [20] Liu KX, He ZH, Cao LC. Double amplitude freedom Gerchberg–Saxton algorithm for generation of phase-only hologram with speckle suppression. Appl Phys Lett 120, 061103 (2022). doi: 10.1063/5.0080797

    CrossRef Google Scholar

    [21] Liu KX, He ZH, Cao LC. Pattern-adaptive error diffusion algorithm for improved phase-only hologram generation. Chin Opt Lett 19, 050501 (2021). doi: 10.3788/COL202119.050501

    CrossRef Google Scholar

    [22] Kang JW, Park BS, Kim JK, Kim DW, Seo YH. Deep-learning-based hologram generation using a generative model. Appl Opt 60, 7391–7399 (2021). doi: 10.1364/AO.427262

    CrossRef Google Scholar

    [23] Lee J, Jeong J, Cho J, Yoo D, Lee B et al. Deep neural network for multi-depth hologram generation and its training strategy. Opt Express 28, 27137–27154 (2020). doi: 10.1364/OE.402317

    CrossRef Google Scholar

    [24] Zheng HD, Hu JB, Zhou CJ, Wang XX. Computing 3D phase-type holograms based on deep learning method. Photonics 8, 280 (2021). doi: 10.3390/photonics8070280

    CrossRef Google Scholar

    [25] Liu SC, Chu DP. Deep learning for hologram generation. Opt Express 29, 27373–27395 (2021). doi: 10.1364/OE.418803

    CrossRef Google Scholar

    [26] Khan A, Zhang ZJ, Yu YJ, Khan MA, Yan KT et al. GAN-Holo: generative adversarial networks-based generated holography using deep learning. Complexity 2021, 6662161 (2021). doi: 10.1155/2021/6662161

    CrossRef Google Scholar

    [27] Horisaki R, Takagi R, Tanida J. Deep-learning-generated holography. Appl Opt 57, 3859–3863 (2018). doi: 10.1364/AO.57.003859

    CrossRef Google Scholar

    [28] Goi H, Komuro K, Nomura T. Deep-learning-based binary hologram. Appl Opt 59, 7103–7108 (2020). doi: 10.1364/AO.393500

    CrossRef Google Scholar

    [29] Chang CL, Wang D, Zhu DC, Li JM, Xia J et al. Deep-learning-based computer-generated hologram from a stereo image pair. Opt Lett 47, 1482–1485 (2022). doi: 10.1364/OL.453580

    CrossRef Google Scholar

    [30] Hossein Eybposh M, Caira NW, Atisa M, Chakravarthula P, Pégard NC. DeepCGH: 3D computer-generated holography using deep learning. Opt Express 28, 26636–26650 (2020). doi: 10.1364/OE.399624

    CrossRef Google Scholar

    [31] Horisaki R, Nishizaki Y, Kitaguchi K, Saito M, Tanida J. Three-dimensional deeply generated holography [Invited]. Appl Opt 60, A323–A328 (2021). doi: 10.1364/AO.404151

    CrossRef Google Scholar

    [32] Peng YF, Choi S, Padmanaban N, Wetzstein G. Neural holography with camera-in-the-loop training. ACM Trans Graph 39, 185 (2020). doi: 10.1145/3414685.3417802

    CrossRef Google Scholar

    [33] Gopakumar M, Kim J, Choi S, Peng YF, Wetzstein G. Unfiltered holography: optimizing high diffraction orders without optical filtering for compact holographic displays. Opt Lett 46, 5822–5825 (2021). doi: 10.1364/OL.442851

    CrossRef Google Scholar

    [34] Peng YF, Choi S, Kim J, Wetzstein G. Speckle-free holography with partially coherent light sources and camera-in-the-loop calibration. Sci Adv 7, eabg5040 (2021). doi: 10.1126/sciadv.abg5040

    CrossRef Google Scholar

    [35] Ishii Y, Shimobaba T, Blinder D, Birnbaum T, Schelkens P et al. Optimization of phase-only holograms calculated with scaled diffraction calculation through deep neural networks. Appl Phys B 128, 22 (2022). doi: 10.1007/s00340-022-07753-7

    CrossRef Google Scholar

    [36] Yu T, Zhang SJ, Chen W, Liu J, Zhang XY et al. Phase dual-resolution networks for a computer-generated hologram. Opt Express 30, 2378–2389 (2022). doi: 10.1364/OE.448996

    CrossRef Google Scholar

    [37] Sun XH, Mu XY, Xu C, Pang H, Deng QL et al. Dual-task convolutional neural network based on the combination of the U-Net and a diffraction propagation model for phase hologram design with suppressed speckle noise. Opt Express 30, 2646–2658 (2022). doi: 10.1364/OE.440956

    CrossRef Google Scholar

    [38] Wu JC, Liu KX, Sui XM, Cao LC. High-speed computer-generated holography using an autoencoder-based deep neural network. Opt Lett 46, 2908–2911 (2021). doi: 10.1364/OL.425485

    CrossRef Google Scholar

    [39] Situ GH. Deep holography. Light Adv Manuf 3, 13 (2022). doi: 10.37188/lam.2022.013

    CrossRef Google Scholar

    [40] Zuo C, Qian JM, Feng SJ, Yin W, Li YX et al. Deep learning in optical metrology: a review. Light Sci Appl 11, 39 (2022). doi: 10.1038/s41377-022-00714-x

    CrossRef Google Scholar

    [41] Shi L, Li BC, Matusik W. End-to-end learning of 3D phase-only holograms for holographic display. Light Sci Appl 11, 247 (2022). doi: 10.1038/s41377-022-00894-6

    CrossRef Google Scholar

    [42] Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation. In Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention 234–241 (Springer, 2015);http://doi.org/10.1007/978-3-319-24574-4_28.

    Google Scholar

    [43] Shi WZ, Caballero J, Huszár F, Totz J, Aitken AP et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition 1874–1883 (IEEE, 2016); http://doi.org/10.1109/CVPR.2016.207.

    Google Scholar

    [44] Dumoulin V, Shlens J, Kudlur M. A learned representation for artistic style. In Proceedings of the 5th International Conference on Learning Representations (IEEE, 2016). https://arxiv.org/abs/1610.07629

    Google Scholar

    [45] Shimobaba T, Blinder D, Birnbaum T, Hoshi I, Shiomi H et al. Deep-learning computational holography: a review. Front Photonics 3, 854391 (2022). doi: 10.3389/fphot.2022.854391

    CrossRef Google Scholar

    [46] Kingma DP, Ba J. Adam: a method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations (2014). https://arxiv.org/abs/1412.6980

    Google Scholar

    [47] Source code: https://github.com/THUHoloLab/4K-DMDNet

    Google Scholar

    [48] Wang JD, Sun K, Cheng TH, Jiang BR, Deng CR et al. Deep high-resolution representation learning for visual recognition. IEEE Trans Pattern Anal Mach Intell 43, 3349–3364 (2021). doi: 10.1109/TPAMI.2020.2983686

    CrossRef Google Scholar

  • 4K-DMDNet code
  • 加载中
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

Figures(9)

Article Metrics

Article views(8159) PDF downloads(1843) Cited by(0)

Access History

Other Articles By Authors

Article Contents

Catalog

    /

    DownLoad:  Full-Size Img  PowerPoint