E-mail Alert
RSS
| Citation: |
Liu YC, Xu K, Fan XH et al. Dynamic interactive bitwise meta-holography with ultra-high computational and display frame rates. Opto-Electron Adv 7, 230108 (2024). doi: 10.29026/oea.2024.230108
|
Dynamic interactive bitwise meta-holography with ultra-high computational and display frame rates
-
Abstract
Interactive holography offers unmatched levels of immersion and user engagement in the field of future display. Despite of the substantial progress has been made in dynamic meta-holography, the realization of real-time, highly smooth interactive holography remains a significant challenge due to the computational and display frame rate limitations. In this study, we introduced a dynamic interactive bitwise meta-holography with ultra-high computational and display frame rates. To our knowledge, this is the first reported practical dynamic interactive metasurface holographic system. We spatially divided the metasurface device into multiple distinct channels, each projecting a reconstructed sub-pattern. The switching states of these channels were mapped to bitwise operations on a set of bit values, which avoids complex hologram computations, enabling an ultra-high computational frame rate. Our approach achieves a computational frame rate of 800 kHz and a display frame rate of 23 kHz on a low-power Raspberry Pi computational platform. According to this methodology, we demonstrated an interactive dynamic holographic Tetris game system that allows interactive gameplay, color display, and on-the-fly hologram creation. Our technology presents an inspiration for advanced dynamic meta-holography, which is promising for a broad range of applications including advanced human-computer interaction, real-time 3D visualization, and next-generation virtual and augmented reality systems.-
Keywords:
- interactive display /
- meta-holography /
- bitwise operation /
- ultra-high frame rate
-
-
References
[1] Xu K, Wang XE, Fan XH et al. Meta-holography: from concept to realization. Opto-Electron Eng 49, 220183 (2022). doi: 10.12086/oee.2022.220183 [2] Horimai H, Tan XD. Holographic information storage system: today and future. IEEE Trans Magn 43, 943–947 (2007). doi: 10.1109/TMAG.2006.888528 [3] Gao H, Fan XH, Xiong W et al. Recent advances in optical dynamic meta-holography. Opto-Electron Adv 4, 210030 (2021). doi: 10.29026/oea.2021.210030 [4] Xiong JH, Hsiang EL, He ZQ et al. Augmented reality and virtual reality displays: emerging technologies and future perspectives. Light Sci Appl 10, 216 (2021). doi: 10.1038/s41377-021-00658-8 [5] Fang XY, Ren HR, Gu M. Orbital angular momentum holography for high-security encryption. Nat Photonics 14, 102–108 (2020). doi: 10.1038/s41566-019-0560-x [6] Li JX, Kamin S, Zheng GX et al. Addressable metasurfaces for dynamic holography and optical information encryption. Sci Adv 4, eaar6768 (2018). doi: 10.1126/sciadv.aar6768 [7] Wu JC, Liu KX, Sui X et al. High-speed computer-generated holography using an autoencoder-based deep neural network. Opt Lett 46, 2908–2911 (2021). doi: 10.1364/OL.425485 [8] González H, Martínez-León L, Soldevila F et al. High sampling rate single-pixel digital holography system employing a DMD and phase-encoded patterns. Opt Express 26, 20342–20350 (2018). doi: 10.1364/OE.26.020342 [9] Gao H, Wang YX, Fan XH et al. Dynamic 3D meta-holography in visible range with large frame number and high frame rate. Sci Adv 6, eaba8595 (2020). doi: 10.1126/sciadv.aba8595 [10] Yu NF, Genevet P, Kats MA et al. Light propagation with phase discontinuities: generalized laws of reflection and refraction. Science 334, 333–337 (2011). doi: 10.1126/science.1210713 [11] Ni YB, Chen S, Wang YJ et al. Metasurface for structured light projection over 120° field of view. Nano Lett 20, 6719–6724 (2020). doi: 10.1021/acs.nanolett.0c02586 [12] Mu YH, Zheng MY, Qi JR et al. A large field-of-view metasurface for complex-amplitude hologram breaking numerical aperture limitation. Nanophotonics 9, 4749–4759 (2020). doi: 10.1515/nanoph-2020-0448 [13] Zhang YX, Pu MB, Jin JJ et al. Crosstalk-free achromatic full Stokes imaging polarimetry metasurface enabled by polarization-dependent phase optimization. Opto-Electron Adv 5, 220058 (2022). doi: 10.29026/oea.2022.220058 [14] Guo XY, Zhong JZ, Li BJ et al. Full-color holographic display and encryption with full-polarization degree of freedom. Adv Mater 34, 2103192 (2022). doi: 10.1002/adma.202103192 [15] Wang H, Ruan QF, Wang HT et al. Full color and grayscale painting with 3D printed low-index nanopillars. Nano Lett 21, 4721–4729 (2021). doi: 10.1021/acs.nanolett.1c00979 [16] Dong FL, Chu WG. Multichannel-independent information encoding with optical metasurfaces. Adv Mater 31, 1804921 (2019). doi: 10.1002/adma.201804921 [17] Zhao RZ, Huang LL, Wang YT. Recent advances in multi-dimensional metasurfaces holographic technologies. PhotoniX 1, 20 (2020). doi: 10.1186/s43074-020-00020-y [18] Zhang M, Pu MB, Zhang F et al. Plasmonic metasurfaces for switchable photonic spin–orbit interactions based on phase change materials. Adv Sci 5, 1800835 (2018). doi: 10.1002/advs.201800835 [19] Choi C, Mun SE, Sung J et al. Hybrid state engineering of phase-change metasurface for all-optical cryptography. Adv Funct Mater 31, 2007210 (2021). doi: 10.1002/adfm.202007210 [20] Zeng C, Lu H, Mao D et al. Graphene-empowered dynamic metasurfaces and metadevices. Opto-Electron Adv 5, 200098 (2022). doi: 10.29026/oea.2022.200098 [21] Wan CW, Li Z, Wan S et al. Electric-driven meta-optic dynamics for simultaneous near-/far-field multiplexing display. Adv Funct Mater 32, 2110592 (2022). doi: 10.1002/adfm.202110592 [22] Nemati A, Wang Q, Ang NSS et al. Ultra-high extinction-ratio light modulation by electrically tunable metasurface using dual epsilon-near-zero resonances. Opto-Electron Adv 4, 200088 (2021). doi: 10.29026/oea.2021.200088 [23] Li TY, Wei QS, Reineke B et al. Reconfigurable metasurface hologram by utilizing addressable dynamic pixels. Opt Express 27, 21153–21162 (2019). doi: 10.1364/OE.27.021153 [24] Malek SC, Ee HS, Agarwal R. Strain multiplexed metasurface holograms on a stretchable substrate. Nano Lett 17, 3641–3645 (2017). doi: 10.1021/acs.nanolett.7b00807 [25] Yang R, Wan S, Shi YY et al. Immersive tuning the guided waves for multifunctional on-chip metaoptics. Laser Photonics Rev 16, 2200127 (2022). doi: 10.1002/lpor.202200127 [26] Jang J, Lee GY, Sung J et al. Independent multichannel wavefront modulation for angle multiplexed meta-holograms. Adv Opt Mater 9, 2100678 (2021). doi: 10.1002/adom.202100678 [27] Balthasar Mueller JP, Rubin NA, Devlin RC et al. Metasurface polarization optics: independent phase control of arbitrary orthogonal states of polarization. Phys Rev Lett 118, 113901 (2017). doi: 10.1103/PhysRevLett.118.113901 [28] Li ZL, Chen C, Guan ZQ et al. Three-channel metasurfaces for simultaneous meta-holography and meta-nanoprinting: a single-cell design approach. Laser Photonics Rev 14, 2000032 (2020). doi: 10.1002/lpor.202000032 [29] Ren HR, Fang XY, Jang J et al. Complex-amplitude metasurface-based orbital angular momentum holography in momentum space. Nat Nanotechnol 15, 948–955 (2020). doi: 10.1038/s41565-020-0768-4 [30] Masuda N, Ito T, Tanaka T et al. Computer generated holography using a graphics processing unit. Opt Express 14, 603–608 (2006). doi: 10.1364/OPEX.14.000603 [31] Yamamoto Y, Nakayama H, Takada N et al. Large-scale electroholography by HORN-8 from a point-cloud model with 400, 000 points. Opt Express 26, 34259–34265 (2018). doi: 10.1364/OE.26.034259 [32] Gerchberg RW. A practical algorithm for the determination of phase from image and diffraction plane pictures. Optik 35, 237–246 (1972). [33] Li X, Chen QM, Zhang X et al. Time-sequential color code division multiplexing holographic display with metasurface. Opto-Electron Adv 6, 220060 (2023). doi: 10.29026/oea.2023.220060 [34] Yamada S, Kakue T, Shimobaba T et al. Interactive holographic display based on finger gestures. Sci Rep 8, 2010 (2018). doi: 10.1038/s41598-018-20454-6 -
Supplementary Information
Supplementary information for Dynamic interactive bitwise meta-holography with ultra-high computational and display frame rates 
Supplementary movies
-
Access History
Figures(5)
Article Metrics
Export File
Citation
Liu YC, Xu K, Fan XH et al. Dynamic interactive bitwise meta-holography with ultra-high computational and display frame rates. Opto-Electron Adv 7, 230108 (2024). doi: 10.29026/oea.2024.230108
Format
Content
DownLoad:
-
Figure 1.
Schematic diagram of the channel division concept of Bit-MH and the system for implementing interactive holographic games. The numbers and box lines are added to indicate the channel area; states 0 and 1 indicate corresponding position channel off and on, respectively. (a) Schematic of the concept of space channel division and the independent projection of each channel. (b) Schematic of selective turn-on channels for combined display. (c) Schematic diagram of the architecture of the interactive dynamic metasurface holographic game system. On the DMD, black indicates off, and white indicates on; on the metasurface device, the darker area indicates not being turned on.
-
Figure 2.
Schematic comparison diagram between the bitwise dynamic holography algorithm and the traditional Fourier-based dynamic holography algorithm. For simplicity, a 7-bit digit tube is used as the target image; in our algorithm, the bit values 1 and 0 indicate that the corresponding channels are on and off, respectively; in the binary control image, white and black indicate that part of the channel is on and off, respectively; the box lines in the binary image are used to distinguish the different channels clearly and do not indicate other information. In the update mask of the bitwise algorithm, 0 and 1 respectively indicate that the channel at the corresponding position needs to be flipped or remain unchanged.
-
Figure 3.
Design scheme of cylindric nanopillar with varying radii and same height. (a) Geometric parameters of the nanocylinder located in the center of the square lattice, W is the side length of the square lattice, and R and H are the radius and height of the nanocylinder, respectively. (b) Results of parameter sweep for the parameter range of R from 55 nm to 160 nm for W = 440 nm, H = 700 nm, and λ = 532 nm. Marked in the figure is the amount of amplitude and phase modulation provided by the selected nanopillars. (c) Characterization results of the CLSM of the device; The block-by-block partitions are different channel areas, and the detailed design is described in a later section. (d) Characterization results of the SEM of one local part of the device.
-
Figure 4.
Schematic diagram of arrangements, size, and display functions of the channels on the device for Tetris game. (a) The arrangement and size of channels on the overall designed device, which were marked based on the characterization result image of the metasurface device using CLSM. The numbers indicate the corresponding numbers of the channels. (b) Projection patterns from different channels that do not overlap when displayed. Static channel No. 24 display the fixed border and texts; cyclic channels No. 25 to 33 correspond to different poses of a rotating square, and opening these channels, in turn, realizes a dynamic pattern of rotating squares; cyclic channel No. 34 corresponds to a flashing music logo; multiplexed channels No. 35 to 133 are arrays of squares that form the main game canvas; multiplexed channels No. 1 to 21 and No. 134 to 141 project the digit tubes and simple array of squares illustrated on the right, respectively. (c) The text or pattern projection which was displayed in the center. To avoid possible mutual occlusion of patterns displayed in the same area, the text and patterns that would be projected in the center of the main scene are shown separately, corresponding to the (a) diagram, actual holograms of multiplexed channels No. 22 ("START" text), 23 ("GAME OVER" text), 24 ( the static borders and texts), 35 to 133 (square array in the main game canvas), 142 ("YOU WIN" text), 143 ("UPGRADE" text), 144 (fireworks pattern), respectively; for a clearer presentation, the static border is always displayed (channel numbered 24).
-
Figure 5.
Experimental reconstructed patterns results at 473 nm, 532 nm, and 633 nm wavelength with partial channels turned on. In the coordinate axes, k0 represents the wave vector, and kx and ky represent the spatial frequencies in the x and y directions, respectively. k0 and kx and ky at 633 nm, 532 nm, and 573 nm use the corner markers of R, G, and B, respectively. In (a) and (b), The left side of each subgraph is the pattern and the right side is the diagram of the corresponding channel switching state; black and white refer to the channel close and open, respectively; some spacings are added around channels for clear presentation. (a) The reconstructed pattern of the fireworks. (b) A frame extracted from a game played via a gamepad. (c) The extracted stages of the actual playing game. Each of these four frames indicates continuous progress. (d) The on-the-fly created text uppercase letter 'I' symbol pattern result at 473 nm wavelength. (e) The on-the-fly created heart pattern result at 633 nm. (f) The on-the-fly created "HUST" text pattern result at 532 nm.
