Zhao BH, Cheng JW, Wu B, Gao DS, Zhou HL et al. Integrated photonic convolution acceleration core for wearable devices. Opto-Electron Sci 2, 230017 (2023). doi: 10.29026/oes.2023.230017
Citation: Zhao BH, Cheng JW, Wu B, Gao DS, Zhou HL et al. Integrated photonic convolution acceleration core for wearable devices. Opto-Electron Sci 2, 230017 (2023). doi: 10.29026/oes.2023.230017

Article Open Access

Integrated photonic convolution acceleration core for wearable devices

More Information
  • With the advancement of deep learning and neural networks, the computational demands for applications in wearable devices have grown exponentially. However, wearable devices also have strict requirements for long battery life, low power consumption, and compact size. In this work, we propose a scalable optoelectronic computing system based on an integrated optical convolution acceleration core. This system enables high-precision computation at the speed of light, achieving 7-bit accuracy while maintaining extremely low power consumption. It also demonstrates peak throughput of 3.2 TOPS (tera operations per second) in parallel processing. We have successfully demonstrated image convolution and the typical application of an interactive first-person perspective gesture recognition application based on depth information. The system achieves a comparable recognition accuracy to traditional electronic computation in all blind tests.
  • 加载中
  • [1] Zhang SB, Li YX, Zhang S, Shahabi F, Xia S et al. Deep learning in human activity recognition with wearable sensors: a review on advances. Sensors 22, 1476 (2022). doi: 10.3390/s22041476

    CrossRef Google Scholar

    [2] Chang WJ, Chen LB, Chiou YZ. Design and implementation of a drowsiness-fatigue-detection system based on wearable smart glasses to increase road safety. IEEE Trans Consum Electron 64, 461–469 (2018). doi: 10.1109/TCE.2018.2872162

    CrossRef Google Scholar

    [3] Ramanujam E, Perumal T, Padmavathi S. Human activity recognition with smartphone and wearable sensors using deep learning techniques: a review. IEEE Sensors J 21, 13029–13040 (2021). doi: 10.1109/JSEN.2021.3069927

    CrossRef Google Scholar

    [4] Chen KX, Zhang DL, Yao LN, Guo B, Yu ZW et al. Deep learning for sensor-based human activity recognition: overview, challenges, and opportunities. ACM Comput Surv 54, 77 (2022). doi: 10.1145/3447744

    CrossRef Google Scholar

    [5] Wang JD, Chen YQ, Hao SJ, Peng XH, Hu LS. Deep learning for sensor-based activity recognition: a survey. Pattern Recognit Lett 119, 3–11 (2019). doi: 10.1016/j.patrec.2018.02.010

    CrossRef Google Scholar

    [6] Nweke HF, Teh YW, Al-garadi MA, Alo UR. Deep learning algorithms for human activity recognition using mobile and wearable sensor networks: state of the art and research challenges. Expert Syst Appl 105, 233–261 (2018). doi: 10.1016/j.eswa.2018.03.056

    CrossRef Google Scholar

    [7] Terven JR, Raducanu B, Meza-de-Luna ME, Salas J. Head-gestures mirroring detection in dyadic social interactions with computer vision-based wearable devices. Neurocomputing 175, 866–876 (2016). doi: 10.1016/j.neucom.2015.05.131

    CrossRef Google Scholar

    [8] Perusquía-Hernández M, Hirokawa M, Suzuki K. A wearable device for fast and subtle spontaneous smile recognition. IEEE Trans Affective Comput 8, 522–533 (2017). doi: 10.1109/TAFFC.2017.2755040

    CrossRef Google Scholar

    [9] Gruebler A, Suzuki K. Design of a wearable device for reading positive expressions from facial EMG signals. IEEE Trans Affective Comput 5, 227–237 (2014). doi: 10.1109/TAFFC.2014.2313557

    CrossRef Google Scholar

    [10] Brown TB, Mann B, Ryder N, Subbiah M, Kaplan J et al. Language models are few-shot learners. In Proceedings of the 34th International Conference on Neural Information Processing Systems 159 (ACM, 2020); http://doi.org/10.5555/3495724.3495883.

    Google Scholar

    [11] Hussein AI. Wearable computing: challenges of implementation and its future. In 2015 12th Learning and Technology Conference 14–19 (IEEE, 2015);http://doi.org/10.1109/LT.2015.7587224.

    Google Scholar

    [12] Zhou HL, Dong JJ, Cheng JW, Dong WC, Huang CR et al. Photonic matrix multiplication lights up photonic accelerator and beyond. Light Sci Appl 11, 30 (2022). doi: 10.1038/s41377-022-00717-8

    CrossRef Google Scholar

    [13] Shastri BJ, Tait AN, Ferreira De Lima T, Pernice WHP, Bhaskaran H et al. Photonics for artificial intelligence and neuromorphic computing. Nat Photonics 15, 102–114 (2021). doi: 10.1038/s41566-020-00754-y

    CrossRef Google Scholar

    [14] Nahmias MA, De Lima TF, Tait AN, Peng HT, Shastri BJ et al. Photonic multiply-accumulate operations for neural networks. IEEE J Sel Top Quantum Electron 26, 1–18 (2020). doi: 10.1109/jstqe.2019.2941485

    CrossRef Google Scholar

    [15] Lin X, Rivenson Y, Yardimci NT, Veli M, Luo Y et al. All-optical machine learning using diffractive deep neural networks. Science 361, 1004–1008 (2018). doi: 10.1126/science.aat8084

    CrossRef Google Scholar

    [16] Panuski CL, Christen I, Minkov M, Brabec CJ, Trajtenberg-Mills S et al. A full degree-of-freedom spatiotemporal light modulator. Nat Photonics 16, 834–842 (2022). doi: 10.1038/s41566-022-01086-9

    CrossRef Google Scholar

    [17] Li JX, Hung YC, Kulce O, Mengu D, Ozcan A. Polarization multiplexed diffractive computing: all-optical implementation of a group of linear transformations through a polarization-encoded diffractive network. Light Sci Appl 11, 153 (2022). doi: 10.1038/s41377-022-00849-x

    CrossRef Google Scholar

    [18] Xu XY, Tan MX, Corcoran B, Wu JY, Boes A et al. 11 TOPS photonic convolutional accelerator for optical neural networks. Nature 589, 44–51 (2021). doi: 10.1038/s41586-020-03063-0

    CrossRef Google Scholar

    [19] Bai BW, Yang QP, Shu HW, Chang L, Yang FH et al. Microcomb-based integrated photonic processing unit. Nat Commun 14, 66 (2023). doi: 10.1038/s41467-022-35506-9

    CrossRef Google Scholar

    [20] Shen YC, Harris NC, Skirlo S, Prabhu M, Baehr-Jones T et al. Deep learning with coherent nanophotonic circuits. Nat Photonics 11, 441–446 (2017). doi: 10.1038/nphoton.2017.93

    CrossRef Google Scholar

    [21] Hughes TW, Minkov M, Shi Y, Fan SH. Training of photonic neural networks through in situ backpropagation and gradient measurement. Optica 5, 864–871 (2018). doi: 10.1364/OPTICA.5.000864

    CrossRef Google Scholar

    [22] Zhou HL, Zhao YH, Wang X, Gao DS, Dong JJ et al. Self-configuring and reconfigurable silicon photonic signal processor. ACS Photonics 7, 792–799 (2020). doi: 10.1021/acsphotonics.9b01673

    CrossRef Google Scholar

    [23] Zhang H, Gu M, Jiang XD, Thompson J, Cai H et al. An optical neural chip for implementing complex-valued neural network. Nat Commun 12, 457 (2021). doi: 10.1038/s41467-020-20719-7

    CrossRef Google Scholar

    [24] Tait AN, De Lima TF, Zhou E, Wu AX, Nahmias MA et al. Neuromorphic photonic networks using silicon photonic weight banks. Sci Rep 7, 7430 (2017). doi: 10.1038/s41598-017-07754-z

    CrossRef Google Scholar

    [25] Xu SF, Wang J, Yi SC, Zou WW. High-order tensor flow processing using integrated photonic circuits. Nat Commun 13, 7970 (2022). doi: 10.1038/s41467-022-35723-2

    CrossRef Google Scholar

    [26] Huang CR, Bilodeau S, Ferreira De Lima T, Tait AN, Ma PY et al. Demonstration of scalable microring weight bank control for large-scale photonic integrated circuits. APL Photonics 5, 040803 (2020). doi: 10.1063/1.5144121

    CrossRef Google Scholar

    [27] Cheng JW, He ZM, Guo YH, Wu B, Zhou HL et al. Self-calibrating microring synapse with dual-wavelength synchronization. Photonics Res 11, 347 (2023). doi: 10.1364/PRJ.478370

    CrossRef Google Scholar

    [28] Zhang WP, Huang CR, Peng HT, Bilodeau S, Jha A et al. Silicon microring synapses enable photonic deep learning beyond 9-bit precision. Optica 9, 579–584 (2022). doi: 10.1364/OPTICA.446100

    CrossRef Google Scholar

    [29] Bandini A, Zariffa J. Analysis of the hands in egocentric vision: a survey. IEEE Trans Pattern Anal Mach Intell 45, 6846–6866 (2023). doi: 10.1109/TPAMI.2020.2986648

    CrossRef Google Scholar

    [30] Zhang YF, Cao CQ, Cheng J, Lu HQ. EgoGesture: a new dataset and benchmark for egocentric hand gesture recognition. IEEE Trans Multimedia 20, 1038–1050 (2018). doi: 10.1109/TMM.2018.2808769

    CrossRef Google Scholar

    [31] Razzari L, Duchesne D, Ferrera M, Morandotti R, Chu S et al. CMOS-compatible integrated optical hyper-parametric oscillator. Nat Photonics 4, 41–45 (2010). doi: 10.1038/nphoton.2009.236

    CrossRef Google Scholar

    [32] Moss DJ, Morandotti R, Gaeta AL, Lipson M. New CMOS-compatible platforms based on silicon nitride and Hydex for nonlinear optics. Nat Photonics 7, 597–607 (2013). doi: 10.1038/nphoton.2013.183

    CrossRef Google Scholar

    [33] Kippenberg TJ, Gaeta AL, Lipson M, Gorodetsky ML. Dissipative kerr solitons in optical microresonators. Science 361, eaan8083 (2018).

    Google Scholar

    [34] Graves A, Wayne G, Reynolds M, Harley T, Danihelka I et al. Hybrid computing using a neural network with dynamic external memory. Nature 538, 471–476 (2016). doi: 10.1038/nature20101

    CrossRef Google Scholar

  • Movie
  • 加载中
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

Figures(5)

Tables(1)

Article Metrics

Article views(1769) PDF downloads(365) Cited by(0)

Access History
Article Contents

Catalog

    /

    DownLoad:  Full-Size Img  PowerPoint