Reconfigurable optical neural networks with Plug-and-Play metasurfaces

The P-DNNs were designed and optimized in a similar way in training machine learning models, encompassing the introduction of a loss function and the employment of the stochastic gradient descent and error backpropagation algorithms. The objective of the training process was to maximize the light intensity at the corresponding detection region for a given class of objects, and minimize the total signal in other regions. In order to reduce the training time for different tasks, the authors used transfer learning. First, the MNIST dataset was utilized to train and optimize the phase parameters of the shared layer (the first metasurface) and the handwritten digits classification layer (the second metasurface) with 10 training epochs. Then, the parameters of the shared layer were fixed, and the Fashion-MNIST dataset was used to train the fashion classification layer (the third metasurface). This process only took 5 training epochs to reach high classification accuracy thanks to transfer learning. The phase profiles essential for the three metasurfaces were achieved by employing Pancharatnam–Berry phase modulation for circularly polarized light. This was realized by rotating individual silicon nanofins to specific angles. The silicon metasurfaces, each 500×500 μm² in size, were fabricated by standard electron beam lithography. The period was set as 500 nm, and the length, width and height of nanofin were 210, 135 and 600 nm, respectively.

Over the past years, researchers have demonstrated ONNs using distinct platforms, including photonic integrated circuits⁴⁻⁵, diffractive optical elements⁶⁻⁷, and optical metasurfaces⁷⁻⁸. However, most of the demonstrated ONNs lack reconfigurability. Their functionalities are fixed once they are trained and fabricated. In a very work published in Opto-Electronic Advances, Lingling Huang et al. conceptually proposed and experimentally demonstrated pluggable diffractive neural networks (P-DNNs) operating in the near-infrared region⁹. P-DNNs can switch between different tasks, such as classifying handwritten digits and fashion products, by simply interchanging the pluggable components of the networks. As a result, we can significantly enhance the flexibility and adaptation of ONNs while effectively reducing computing resources and training time in the network design.

Figure 1(b) shows the representative results and performances of the P-DNNs. The left column presents the inputs of P-DNNs, which are handwritten digits and fashion images. In the experiment, circularly polarized light carrying the input information was generated by a digital micromirror device. It passed through the P-DNNs, in which the second layer could be readily switched depending on the classification tasks. The light intensity on the output plane was captured by a camera, as shown in the middle column. For specific input images, the region with the highest energy corresponds to the result of classification. The right column in Fig. 1(b) shows energy distribution on the detection plane, both the experimental and simulation results, for handwritten digits and fashion. The authors introduced ΔE = E_max (maximum energy) - E_smax (second maximum energy) as an indicator to verify the classification accuracy and effectiveness. The results confirm that the P-DNNs can correctly perform classification, although the maximum energy distribution in the experimental was a bit lower than the simulation results due to the inevitable fabrication errors and misalignment of the two metasurface layers in optical characterization. Overall, the accuracy for handwritten digit classification was 90% in experiment and 91.8% in simulation. The fashion classification accuracy reached 90% in experiment and 90.2% in simulation.

Drawing inspiration from signal processing in the nervous system, artificial neural networks (ANNs) have demonstrated their indispensability in a variety of tasks, such as computer vision, natural language processing, new materials discovery, and medical diagnosis¹. However, the prevailing von Neumann architectures in modern computers hinder the efficient utilization of ANNs, prompting extensive efforts toward enhancing the computation speed and accuracy. One particularly promising solution is optical neural networks (ONNs), which shift from real-number matrix operations to complex-number logical computing. ONNs offer inherent advantages in power efficiency, speed, parallelism, bandwidth, and scalability, in comparison to their conventional digital and electronic counterparts²⁻³. A notable feature of ONNs is their passive functionality, relying solely on input light energy, thus eliminating the need for additional power consumption during the computation processes. Moreover, ONNs could seamlessly integrate diverse functionalities across different wavelengths and polarizations, which can operate independently and avoid cross-talk. This inherent feature enhances the capacity of ONNs for both extensive bandwidth utilization and parallel computing capabilities.

In summary, the proposed P-DNNs can classify diverse patterns through the seamless switching of the second plugin layer. They help to effectively overcome one major limitation of previously demonstrated ONNs, that is, inability to adapt to multiple tasks once ONNs are designed and fabricated. This new approach allows for the architecture reconfigurability of ONNs, akin to tunable and programmable metasurfaces. Moreover, the utilization of metasurfaces technology facilitates the realization of optical intelligent computing chips, enabling intelligent functions such as real-time object detection in autonomous driving systems. By leveraging wavelength and polarization multiplexing techniques¹⁰⁻¹¹, we can further advance ONNs for more versatile, energy-efficient, and adaptive optical computing technologies.

Figure 1(a) schematically illustrates the proposed P-DNNs, which consist of two layers of metasurfaces with a separation distance of 500 μm. The first layer is a shared component to preprocess input information, and the second layer is a pluggable task-specific classification layer for different types of objects, including handwritten digits (from 0 to 5) and fashions (T-shirts, trousers, coats, sneakers, bags, and ankle boots). When light illuminates distinct objects and passes through the P-DNNs, it converges on different regions of the detection plane, and hence classification is achieved. The metasurfaces consist of rectangular silicon nanofins with deliberately designed geometries and rotation angles, as shown in the inset of Fig. 1(a). Individual nanofins function as optical neurons, and the neurons on the adjacent metasurface layers are connected through diffraction.

Figure  1.  Full-Size Img PowerPoint

(a) Schematic illustration of P-DNNs, which can perform recongnization of handwritten digits and fashion by switching the pluggable classification layer. The inset at the bottom right corner shows the scanning electron micrograph of the fabricated metasurface. (b) Characterization and performance of P-DNNs. Left column: handwritten digits and fashion input images. Middle column: experimentally detected energy distribution maps for handwritten digits and fashion. Right column: experimental and simulation results of energy distribution for handwritten digits and fashion. ΔE represents the difference between the percentage of maximum and second maximum energy. The figures are adapted from ref.⁸ with modification.