volume | PIER Journals

Abstract

Inspired by neural networks based on traditional electronic circuits, optical neural networks (ONNs) show great potential in terms of computing speed and power consumption. Though some progress has been made in devices and schemes, ONNs are still a long way from replacing electronic neural networks in terms of generalizability. Here, we present a complex optical neural network (cONN) for holographic image recognition, within which a high-speed parallel operating unit for complex matrices is proposed, targeting the real-imaginary-splitting and column splitting. Based on the proposed cONN, we have numerically demonstrated the training-recognition process on our cONN for holographic images converted from handwritten digit datasets, achieving an accuracy of 90% based on the back-propagation algorithm. Our training verification integrated architecture will enrich the further development and applications of on-chip photonic matrix computing.

1. Krizhevsky, A., I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," Communications of the Acm, Vol. 60, 84-90, 2017.
doi:10.1145/3065386

2. Suzuki, K., H. Abe, H. MacMahon, and K. Doi, "Image-processing technique for suppressing ribs in chest radiographs by means of Massive Training Artificial Neural Network (MTANN)," IEEE Trans. Med. Imaging, Vol. 25, 406-416, 2006.
doi:10.1109/TMI.2006.871549

3. Chan, W., N. Jaitly, Q. Le, and O. Vinyals, "Listen, attend and spell: A neural network for large vocabulary conversational speech recognition," 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 4960-4964, Mar. 20-25, 2016.

4. Abdel-Hamid, O., A. R. Mohamed, H. Jiang, L. Deng, G. Penn, and D. Yu, "Convolutional neural networks for speech recognition," IEEE/ACM Transactions on Audio, Speech, and Language Processing, Vol. 22, 1533-1545, 2014.
doi:10.1109/TASLP.2014.2339736

5. Guo, S., Y. Lin, S. Li, Z. Chen, and H. Wan, "Deep spatial-temporal 3D convolutional neural networks for traffic data forecasting," IEEE Transactions on Intelligent Transportation Systems, Vol. 20, 3913-3926, 2019.
doi:10.1109/TITS.2019.2906365

6. Meng, J., M. Miscuglio, J. K. George, A. Babakhani, and V. J. Sorger, "Electronic bottleneck suppression in next-generation networks with integrated photonic digital-to-analog converters," Advanced Photonics Research, Vol. 2, 2000033, 2021.
doi:10.1002/adpr.202000033

7. Wei, J., Q. Cheng, R. V. Penty, I. H. White, and D. G. Cunningham, "400 Gigabit Ethernet using advanced modulation formats: Performance, complexity, and power dissipation," IEEE Communications Magazine, Vol. 53, 182-189, 2015.
doi:10.1109/MCOM.2015.7045407

8. Shen, Y., N. C. Harris, S. Skirlo, M. Prabhu, T. Baehr-Jones, M. Hochberg, X. Sun, S. Zhao, H. Larochelle, D. Englund, and M. Soljacic, "Deep learning with coherent nanophotonic circuits," Nat. Photonics, Vol. 11, 441-446, 2017.
doi:10.1038/nphoton.2017.93

9. Cheng, J., H. Zhou, and J. Dong, "Photonic matrix computing: From fundamentals to applications," Nanomaterials, Vol. 11, 1683, 2021.
doi:10.3390/nano11071683

10. Li, C., X. Zhang, J. Li, T. Fang, and X. Dong, "The challenges of modern computing and new opportunities for optics," PhotoniX, Vol. 2, 20, 2021.
doi:10.1186/s43074-021-00042-0

11. Wang, P., F. Xu, B. Wang, B. Gao, H. Wu, H. Qian, and S. Yu, "Three-dimensional nand flash for vector-matrix multiplication," IEEE Transactions on Very Large Scale Integration (VLSI) Systems, Vol. 27, 988-991, 2019.
doi:10.1109/TVLSI.2018.2882194

12. Lehmann, T., E. Bruun, and C. Dietrich, "Mixed analog/digital matrix-vector multiplier for neural network synapses," Analog Integrated Circuits and Signal Processing, Vol. 9, 55-63, 1996.
doi:10.1007/BF00158852

13. Sze, V., Y. H. Chen, T. J. Yang, and J. S. Emer, "Efficient processing of deep neural networks: A tutorial and survey," Proceedings of the IEEE, Vol. 105, 2295-2329, 2017.
doi:10.1109/JPROC.2017.2761740

14. Reck, M., A. Zeilinger, H. J. Bernstein, and P. Bertani, "Experimental realization of any discrete unitary operator," Phys. Rev. Lett., Vol. 73, 58-61, 1994.
doi:10.1103/PhysRevLett.73.58

15. Clements, W. R., P. C. Humphreys, B. J. Metcalf, W. S. Kolthammer, and I. A.Walmsley, "Optimal design for universal multiport interferometers," Optica, Vol. 3, 1460-1465, 2016.
doi:10.1364/OPTICA.3.001460

16. Ahn, J., M. Fiorentino, R. G. Beausoleil, N. Binkert, A. Davis, D. Fattal, N. P. Jouppi, M. McLaren, C. M. Santori, R. S. Schreiber, S. M. Spillane, D. Vantrease, and Q. Xu, "Devices and architectures for photonic chip-scale integration," Applied Physics A, Vol. 95, 989-997, 2009.
doi:10.1007/s00339-009-5109-2

17. Lin, X., Y. Rivenson, N. T. Yardimci, M. Veli, Y. Luo, M. Jarrahi, and A. Ozcan, "All-optical machine learning using diffractive deep neural networks," Science, Vol. 361, 1004-1008, 2018.
doi:10.1126/science.aat8084

18. Zhou, J., H. Qian, J. Zhao, M. Tang, Q. Wu, M. Lei, H. Luo, S. Wen, S. Chen, and Z. Liu, "Two-dimensional optical spatial differentiation and high-contrast imaging," National Science Review, Vol. 8, 2020.

19. Qian, C., Z. Wang, H. Qian, T. Cai, B. Zheng, X. Lin, Y. Shen, I. Kaminer, E. Li, and H. Chen, "Dynamic recognition and mirage using neuro-metamaterials," Nat. Commun., Vol. 13, 2694, 2022.
doi:10.1038/s41467-022-30377-6

20. Tian, Y., Y. Zhao, S. Liu, Q. Li, W. Wang, J. Feng, and J. Guo, "Scalable and compact photonic neural chip with low learning-capability-loss," Nanophotonics, Vol. 11, 329-344, 2022.
doi:10.1515/nanoph-2021-0521

21. Feldmann, J., N. Youngblood, M. Karpov, H. Gehring, X. Li, M. Stappers, M. Le Gallo, X. Fu, A. Lukashchuk, A. S. Raja, J. Liu, C. D. Wright, A. Sebastian, T. J. Kippenberg, W. H. P. Pernice, and H. Bhaskaran, "Parallel convolutional processing using an integrated photonic tensor core," Nature, Vol. 589, 52-58, 2021.
doi:10.1038/s41586-020-03070-1

22. Xu, X., M. Tan, B. Corcoran, J. Wu, A. Boes, T. G. Nguyen, S. T. Chu, B. E. Little, D. G. Hicks, R. Morandotti, A. Mitchell, and D. J. Moss, "11 TOPS photonic convolutional accelerator for optical neural networks," Nature, Vol. 589, 44-51, 2021.
doi:10.1038/s41586-020-03063-0

23. Oliveira, N., G. E. Khoury, J. M. Versnel, G. K. Moghaddam, L. S. Leite, J. L. Lima-Filho, and C. R. Lowe, "A holographic sensor based on a biomimetic affinity ligand for the detection of cocaine," Sensors and Actuators, Vol. B270, 216-222, 2018.
doi:10.1016/j.snb.2018.05.009

24. Spetzler, R. F. and H. Spetzler, "Holographic interferometry applied to the study of the human skull," J. Neurosurg., Vol. 52, 825-828, 1980.
doi:10.3171/jns.1980.52.6.0825

25. Fuhrmann, S., O. Komogortsev, and D. Tamir, "Investigating hologram-based route planning," Transactions in GIS, Vol. 13, 177-196, 2009.
doi:10.1111/j.1467-9671.2009.01158.x

26. Khatun, R., K. T. Ahmmed, A. Z. Chowdhury, and R. Hossen, "Optimization of 2 x 2 MZI electro-optic switch and its application as logic gate," 2015 18th International Conference on Computer and Information Technology (ICCIT), 294-299, Dec. 21-23, 2015.

27. Kumar Raghuwanshi, S., A. Kumar, and N.-K. Chen, "Implementation of sequential logic circuits using the Mach-Zehnder interferometer structure based on electro-optic effect," Opt. Commun., Vol. 333, 193-208, 2014.
doi:10.1016/j.optcom.2014.07.066

28. Stegmaier, M., C. Rios, H. Bhaskaran, and W. H. P. Pernice, "Thermo-optical effect in phase-change nanophotonics," ACS Photonics, Vol. 3, 828-835, 2016.
doi:10.1021/acsphotonics.6b00032

29. Chen, S., Y. Shi, S. He, and D. Dai, "Variable optical attenuator based on a reflective Mach-Zehnder interferometer," Opt. Commun., Vol. 361, 55-58, 2016.
doi:10.1016/j.optcom.2015.10.041

30. David, E. R. and L. M. James, "Learning internal representations by error propagation," Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Foundations, 318-362, MIT Press, 1987.

31. Zhu, H. H., J. Zou, H. Zhang, Y. Z. Shi, S. B. Luo, N. Wang, H. Cai, L. X. Wan, B. Wang, X. D. Jiang, J. Thompson, X. S. Luo, X. H. Zhou, L. M. Xiao, W. Huang, L. Patrick, M. Gu, L. C. Kwek, and A. Q. Liu, "Space-efficient optical computing with an integrated chip diffractive neural network," Nat. Commun., Vol. 13, 1044, 2022.
doi:10.1038/s41467-022-28702-0

32. Sludds, A., S. Bandyopadhyay, Z. Chen, Z. Zhong, J. Cochrane, L. Bernstein, D. Bunandar, P. B. Dixon, S. A. Hamilton, M. Streshinsky, A. Novack, T. Baehr-Jones, M. Hochberg, M. Ghobadi, R. Hamerly, and D. Englund, "Delocalized photonic deep learning on the internet's edge," Science, Vol. 378, 270-276, 2022.
doi:10.1126/science.abq8271

33. Qian, H., S. Li, Y. Li, C.-F. Chen, W. Chen, S. E. Bopp, Y.-U. Lee, W. Xiong, and Z. Liu, "Nanoscale optical pulse limiter enabled by refractory metallic quantum wells," Science Advances, Vol. 6, eaay3456, 2020.
doi:10.1126/sciadv.aay3456

34. Qian, H., S. Li, C.-F. Chen, S.-W. Hsu, S. E. Bopp, Q. Ma, A. R. Tao, and Z. Liu, "Large optical nonlinearity enabled by coupled metallic quantum wells," Light: Science & Applications, Vol. 8, 13, 2019.
doi:10.1038/s41377-019-0123-4

35. Qian, H., Y. Xiao, and Z. Liu, "Giant Kerr response of ultrathin gold films from quantum size effect," Nat. Commun., Vol. 7, 13153, 2016.
doi:10.1038/ncomms13153

36. Ma, H., D. Li, N. Wu, Y. Zhang, H. Chen, and H. Qian, "Nonlinear all-optical modulator based on non-Hermitian PT symmetry," Photonics Research, Vol. 10, 980-988, 2022.
doi:10.1364/PRJ.450747

37. El-Ganainy, R., K. G. Makris, M. Khajavikhan, Z. H. Musslimani, S. Rotter, and D. N. Christodoulides, "Non-Hermitian physics and PT symmetry," Nat. Phys., Vol. 14, 11-19, 2018.
doi:10.1038/nphys4323

38. Miri, M.-A. and A. Alu, "Exceptional points in optics and photonics," Science, Vol. 363, eaar7709, 2019.
doi:10.1126/science.aar7709

Dr. Yiming Feng

Dr. Junru Niu

Dr. Yiyun Zhang

Dr. Yixuan Li

Professor Hongsheng Chen

Professor Haoliang Qian