Inspired by neural networks based on traditional electronic circuits, optical neural networks (ONNs) show great potential in terms of computing speed and power consumption. Though some progress has been made in devices and schemes, ONNs are still a long way from replacing electronic neural networks in terms of generalizability. Here, we present a complex optical neural network (cONN) for holographic image recognition, within which a high-speed parallel operating unit for complex matrices is proposed, targeting the real-imaginary-splitting and column splitting. Based on the proposed cONN, we have numerically demonstrated the training-recognition process on our cONN for holographic images converted from handwritten digit datasets, achieving an accuracy of 90% based on the back-propagation algorithm. Our training verification integrated architecture will enrich the further development and applications of on-chip photonic matrix computing.
1. Krizhevsky, A., I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," Communications of the Acm, Vol. 60, 84-90, 2017. doi:10.1145/3065386
2. Suzuki, K., H. Abe, H. MacMahon, and K. Doi, "Image-processing technique for suppressing ribs in chest radiographs by means of Massive Training Artificial Neural Network (MTANN)," IEEE Trans. Med. Imaging, Vol. 25, 406-416, 2006. doi:10.1109/TMI.2006.871549
3. Chan, W., N. Jaitly, Q. Le, and O. Vinyals, "Listen, attend and spell: A neural network for large vocabulary conversational speech recognition," 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 4960-4964, Mar. 20-25, 2016.
4. Abdel-Hamid, O., A. R. Mohamed, H. Jiang, L. Deng, G. Penn, and D. Yu, "Convolutional neural networks for speech recognition," IEEE/ACM Transactions on Audio, Speech, and Language Processing, Vol. 22, 1533-1545, 2014. doi:10.1109/TASLP.2014.2339736
5. Guo, S., Y. Lin, S. Li, Z. Chen, and H. Wan, "Deep spatial-temporal 3D convolutional neural networks for traffic data forecasting," IEEE Transactions on Intelligent Transportation Systems, Vol. 20, 3913-3926, 2019. doi:10.1109/TITS.2019.2906365
6. Meng, J., M. Miscuglio, J. K. George, A. Babakhani, and V. J. Sorger, "Electronic bottleneck suppression in next-generation networks with integrated photonic digital-to-analog converters," Advanced Photonics Research, Vol. 2, 2000033, 2021. doi:10.1002/adpr.202000033
7. Wei, J., Q. Cheng, R. V. Penty, I. H. White, and D. G. Cunningham, "400 Gigabit Ethernet using advanced modulation formats: Performance, complexity, and power dissipation," IEEE Communications Magazine, Vol. 53, 182-189, 2015. doi:10.1109/MCOM.2015.7045407
8. Shen, Y., N. C. Harris, S. Skirlo, M. Prabhu, T. Baehr-Jones, M. Hochberg, X. Sun, S. Zhao, H. Larochelle, D. Englund, and M. Soljacic, "Deep learning with coherent nanophotonic circuits," Nat. Photonics, Vol. 11, 441-446, 2017. doi:10.1038/nphoton.2017.93
9. Cheng, J., H. Zhou, and J. Dong, "Photonic matrix computing: From fundamentals to applications," Nanomaterials, Vol. 11, 1683, 2021. doi:10.3390/nano11071683
10. Li, C., X. Zhang, J. Li, T. Fang, and X. Dong, "The challenges of modern computing and new opportunities for optics," PhotoniX, Vol. 2, 20, 2021. doi:10.1186/s43074-021-00042-0
11. Wang, P., F. Xu, B. Wang, B. Gao, H. Wu, H. Qian, and S. Yu, "Three-dimensional nand flash for vector-matrix multiplication," IEEE Transactions on Very Large Scale Integration (VLSI) Systems, Vol. 27, 988-991, 2019. doi:10.1109/TVLSI.2018.2882194
12. Lehmann, T., E. Bruun, and C. Dietrich, "Mixed analog/digital matrix-vector multiplier for neural network synapses," Analog Integrated Circuits and Signal Processing, Vol. 9, 55-63, 1996. doi:10.1007/BF00158852
13. Sze, V., Y. H. Chen, T. J. Yang, and J. S. Emer, "Efficient processing of deep neural networks: A tutorial and survey," Proceedings of the IEEE, Vol. 105, 2295-2329, 2017. doi:10.1109/JPROC.2017.2761740
14. Reck, M., A. Zeilinger, H. J. Bernstein, and P. Bertani, "Experimental realization of any discrete unitary operator," Phys. Rev. Lett., Vol. 73, 58-61, 1994. doi:10.1103/PhysRevLett.73.58
15. Clements, W. R., P. C. Humphreys, B. J. Metcalf, W. S. Kolthammer, and I. A.Walmsley, "Optimal design for universal multiport interferometers," Optica, Vol. 3, 1460-1465, 2016. doi:10.1364/OPTICA.3.001460
16. Ahn, J., M. Fiorentino, R. G. Beausoleil, N. Binkert, A. Davis, D. Fattal, N. P. Jouppi, M. McLaren, C. M. Santori, R. S. Schreiber, S. M. Spillane, D. Vantrease, and Q. Xu, "Devices and architectures for photonic chip-scale integration," Applied Physics A, Vol. 95, 989-997, 2009. doi:10.1007/s00339-009-5109-2
17. Lin, X., Y. Rivenson, N. T. Yardimci, M. Veli, Y. Luo, M. Jarrahi, and A. Ozcan, "All-optical machine learning using diffractive deep neural networks," Science, Vol. 361, 1004-1008, 2018. doi:10.1126/science.aat8084
18. Zhou, J., H. Qian, J. Zhao, M. Tang, Q. Wu, M. Lei, H. Luo, S. Wen, S. Chen, and Z. Liu, "Two-dimensional optical spatial differentiation and high-contrast imaging," National Science Review, Vol. 8, 2020.
19. Qian, C., Z. Wang, H. Qian, T. Cai, B. Zheng, X. Lin, Y. Shen, I. Kaminer, E. Li, and H. Chen, "Dynamic recognition and mirage using neuro-metamaterials," Nat. Commun., Vol. 13, 2694, 2022. doi:10.1038/s41467-022-30377-6
20. Tian, Y., Y. Zhao, S. Liu, Q. Li, W. Wang, J. Feng, and J. Guo, "Scalable and compact photonic neural chip with low learning-capability-loss," Nanophotonics, Vol. 11, 329-344, 2022. doi:10.1515/nanoph-2021-0521
21. Feldmann, J., N. Youngblood, M. Karpov, H. Gehring, X. Li, M. Stappers, M. Le Gallo, X. Fu, A. Lukashchuk, A. S. Raja, J. Liu, C. D. Wright, A. Sebastian, T. J. Kippenberg, W. H. P. Pernice, and H. Bhaskaran, "Parallel convolutional processing using an integrated photonic tensor core," Nature, Vol. 589, 52-58, 2021. doi:10.1038/s41586-020-03070-1
22. Xu, X., M. Tan, B. Corcoran, J. Wu, A. Boes, T. G. Nguyen, S. T. Chu, B. E. Little, D. G. Hicks, R. Morandotti, A. Mitchell, and D. J. Moss, "11 TOPS photonic convolutional accelerator for optical neural networks," Nature, Vol. 589, 44-51, 2021. doi:10.1038/s41586-020-03063-0
23. Oliveira, N., G. E. Khoury, J. M. Versnel, G. K. Moghaddam, L. S. Leite, J. L. Lima-Filho, and C. R. Lowe, "A holographic sensor based on a biomimetic affinity ligand for the detection of cocaine," Sensors and Actuators, Vol. B270, 216-222, 2018. doi:10.1016/j.snb.2018.05.009
24. Spetzler, R. F. and H. Spetzler, "Holographic interferometry applied to the study of the human skull," J. Neurosurg., Vol. 52, 825-828, 1980. doi:10.3171/jns.1980.52.6.0825
26. Khatun, R., K. T. Ahmmed, A. Z. Chowdhury, and R. Hossen, "Optimization of 2 x 2 MZI electro-optic switch and its application as logic gate," 2015 18th International Conference on Computer and Information Technology (ICCIT), 294-299, Dec. 21-23, 2015.
27. Kumar Raghuwanshi, S., A. Kumar, and N.-K. Chen, "Implementation of sequential logic circuits using the Mach-Zehnder interferometer structure based on electro-optic effect," Opt. Commun., Vol. 333, 193-208, 2014. doi:10.1016/j.optcom.2014.07.066
28. Stegmaier, M., C. Rios, H. Bhaskaran, and W. H. P. Pernice, "Thermo-optical effect in phase-change nanophotonics," ACS Photonics, Vol. 3, 828-835, 2016. doi:10.1021/acsphotonics.6b00032
29. Chen, S., Y. Shi, S. He, and D. Dai, "Variable optical attenuator based on a reflective Mach-Zehnder interferometer," Opt. Commun., Vol. 361, 55-58, 2016. doi:10.1016/j.optcom.2015.10.041
30. David, E. R. and L. M. James, "Learning internal representations by error propagation," Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Foundations, 318-362, MIT Press, 1987.
31. Zhu, H. H., J. Zou, H. Zhang, Y. Z. Shi, S. B. Luo, N. Wang, H. Cai, L. X. Wan, B. Wang, X. D. Jiang, J. Thompson, X. S. Luo, X. H. Zhou, L. M. Xiao, W. Huang, L. Patrick, M. Gu, L. C. Kwek, and A. Q. Liu, "Space-efficient optical computing with an integrated chip diffractive neural network," Nat. Commun., Vol. 13, 1044, 2022. doi:10.1038/s41467-022-28702-0
32. Sludds, A., S. Bandyopadhyay, Z. Chen, Z. Zhong, J. Cochrane, L. Bernstein, D. Bunandar, P. B. Dixon, S. A. Hamilton, M. Streshinsky, A. Novack, T. Baehr-Jones, M. Hochberg, M. Ghobadi, R. Hamerly, and D. Englund, "Delocalized photonic deep learning on the internet's edge," Science, Vol. 378, 270-276, 2022. doi:10.1126/science.abq8271
33. Qian, H., S. Li, Y. Li, C.-F. Chen, W. Chen, S. E. Bopp, Y.-U. Lee, W. Xiong, and Z. Liu, "Nanoscale optical pulse limiter enabled by refractory metallic quantum wells," Science Advances, Vol. 6, eaay3456, 2020. doi:10.1126/sciadv.aay3456
34. Qian, H., S. Li, C.-F. Chen, S.-W. Hsu, S. E. Bopp, Q. Ma, A. R. Tao, and Z. Liu, "Large optical nonlinearity enabled by coupled metallic quantum wells," Light: Science & Applications, Vol. 8, 13, 2019. doi:10.1038/s41377-019-0123-4
35. Qian, H., Y. Xiao, and Z. Liu, "Giant Kerr response of ultrathin gold films from quantum size effect," Nat. Commun., Vol. 7, 13153, 2016. doi:10.1038/ncomms13153
36. Ma, H., D. Li, N. Wu, Y. Zhang, H. Chen, and H. Qian, "Nonlinear all-optical modulator based on non-Hermitian PT symmetry," Photonics Research, Vol. 10, 980-988, 2022. doi:10.1364/PRJ.450747
37. El-Ganainy, R., K. G. Makris, M. Khajavikhan, Z. H. Musslimani, S. Rotter, and D. N. Christodoulides, "Non-Hermitian physics and PT symmetry," Nat. Phys., Vol. 14, 11-19, 2018. doi:10.1038/nphys4323
38. Miri, M.-A. and A. Alu, "Exceptional points in optics and photonics," Science, Vol. 363, eaar7709, 2019. doi:10.1126/science.aar7709