• 摘要: 本文基于衍射神经网络平台,探究经典修正线性单元(rectified linear unit, ReLU)及其衍生激活函数对光学神经网络(ONNs)推理能力的影响,阐明非线性函数与衍射神经网络结合方式与其学习能力的关联。结果显示,衍射网络训练适应性强,但不当的非线性函数可能降低其性能。例如,在5层网络中,在每层后添加RTReLU (rectified translational linear unit)因光强衰减导致在MNIST测试集上的分类准确率降至91.4%,低于纯线性网络的92.6%;而每层添加PReLU (parametric rectified linear unit)则保留阈值后信息,使准确率提升至95.8%。在3层网络中,因为较少的网络深度降低了光强的损耗,在每层后添加RTReLU后推理能力优于线性网络。进一步发现,在网络中适当位置添加单个ReLU激活函数可大幅提升性能,如5层网络仅最后一层添加RTReLU可达峰值准确率96.6%。

       

      Abstract: This study investigates the impact of the classical rectified linear unit (ReLU) activation function and its derivatives on the inference capability of optical neural networks (ONNs) using a diffractive neural network platform, and it elucidates the relationship between the integration method of nonlinear functions into diffractive neural networks and their learning performance. The results demonstrate that while diffractive networks exhibit strong training adaptability, inappropriate nonlinear functions can compromise their performance. For instance, in a five-layer network, the addition of rectified translational linear unit (RTReLU) after each layer caused a drop in classification accuracy on the MNIST test set to 91.4% due to optical intensity attenuation, compared to the 92.6% attained by a purely linear network. In contrast, incorporating parametric rectified linear unit (PReLU) after each layer preserved post-threshold information, thereby improving the accuracy to 95.8%. In a three-layer network, however, the reduced depth mitigated optical intensity loss, and the resulting network with RTReLU added after each layer outperformed the linear network. Furthermore, it was found that strategic placement of a single RTReLU activation function can significantly enhance performance. For example, a five-layer network with RTReLU added only at the final layer achieved a peak accuracy of 96.6%.