volume | PIER Journals

Abstract

Dual-modality tomography integrates two different imaging technologies, allowing for the acquisition of more comprehensive sensing data. By combining information from both modalities, the accuracy of final imaging results is enhanced. However, due to the use of different physical sensitive field backgrounds by different measurement modalities, integrating information from different modalities with differing dimensions presents a challenge. To address this issue, a supervised DSCTFusion-ECA deep learning method is proposed. This method consists of four modules: initial imaging, feature extraction, feature fusion, and image reconstruction. In the feature extraction module, dense connections are utilized first to extract shallow cross-modal features, then two dual-branch feature extraction networks are utilized to separately capture modality-specific low-frequency global features and high-frequency local features for both modalities. The performance and robustness of multi-modality tomography can be effectively improved through the extraction of more comprehensive features. In the feature fusion module, Efficient Channel Attention is employed to capture channel dependencies and generate attention weights. The modal complementarity and the representation ability of key features have been enhanced, while avoiding information redundancy, thereby improving the discriminative power of the features. Simulation results show that the proposed network can fully extract and fuse features from EMT and UTT modalities, demonstrating strong robustness and generalization. Compared to the widely used U-Net network in tomography, DSCTFusion-ECA yields better reconstruction results.

1. Rahim, R. Abdul, M. H. Fazalul Rahiman, K. S. Chan, and S. W. Nawawi, "Non-invasive imaging of liquid/gas flow using ultrasonic transmission-mode tomography," Sensors and Actuators A: Physical, Vol. 135, No. 2, 337-345, 2007. Google Scholar

2. Peyton, A. J., M. S. Beck, A. R. Borges, J. E. De Oliveira, G. M. Lyon, Z. Z. Yu, M. W. Brown, and J. Ferrerra, "Development of electromagnetic tomography (EMT) for industrial applications. Part 1: Sensor design and instrumentation," 1st World Congress on Industrial Process Tomography, Vol. 3, 306-312, 1999.

3. Hayashi, Sanetomo, Shunsuke Yoshimoto, and Akio Yamamoto, "Noncontact 2D temperature imaging of metallic foils using electromagnetic tomography," IEEE Sensors Journal, Vol. 23, No. 16, 17942-17950, 2023. Google Scholar

4. Rahiman, Mohd Hafiz Fazalul, Ruzairi Abdul Rahim, Mohd Hezri Fazalul Rahiman, and Mazidah Tajjudin, "Ultrasonic transmission-mode tomography imaging for liquid/gas two-phase flow," IEEE Sensors Journal, Vol. 6, No. 6, 1706-1715, 2006. Google Scholar

5. Zhang, Ronghua, Hongying Fang, Qiong Zhang, Junyu Wang, Danqi Zhang, Junhua Cheng, and Wuliang Yin, "In situ damage monitoring of CFRPS by electromagnetic tomography with the compatible multitemplate supervised descent method," IEEE Transactions on Instrumentation and Measurement, Vol. 72, 1-12, 2023. Google Scholar

6. Zhang, Wei, Chao Tan, and Feng Dong, "Dual-modality tomography by ERT and UTT projection sorting algorithm," IEEE Sensors Journal, Vol. 20, No. 10, 5415-5423, 2020. Google Scholar

7. Liang, Guanghui, Ville Kolehmainen, Marko Vauhkonen, and Feng Dong, "Structural similarity driven joint reconstruction of conductivity and sound speed in EIT/UTT dual-modality tomography," Inverse Problems, Vol. 39, No. 10, 105010, 2023. Google Scholar

8. Steiner, Gerald, Manuchehr Soleimani, and Daniel Watzenig, "A bio-electromechanical imaging technique with combined electrical impedance and ultrasound tomography," Physiological Measurement, Vol. 29, No. 6, S63, 2008. Google Scholar

9. Jiang, Yandan, Manuchehr Soleimani, and Baoliang Wang, "Contactless electrical impedance and ultrasonic tomography: Correlation, comparison and complementarity study," Measurement Science and Technology, Vol. 30, No. 11, 114001, 2019. Google Scholar

10. Xu, Cong, Feng Dong, and Zhiqiang Zhang, "Dual-modality data acquisition system based on CPCI industrial computer," 2012 IEEE International Conference on Imaging Systems and Techniques Proceedings, 567-572, Manchester, UK, Jul. 2012.

11. Liang, Guanghui, Shangjie Ren, and Feng Dong, "Ultrasound guided electrical impedance tomography for 2D free-interface reconstruction," Measurement Science and Technology, Vol. 28, No. 7, 074003, 2017. Google Scholar

12. Zhang, Ronghua, Qi Wang, Huaxiang Wang, Mu Zhang, and Hongli Li, "Data fusion in dual-mode tomography for imaging oil-gas two-phase flow," Flow Measurement and Instrumentation, Vol. 37, 1-11, 2014. Google Scholar

13. Pusppanathan, Jaysuman, Ruzairi Abdul Rahim, Fatin Aliah Phang, Elmy Johana Mohamad, Nor Muzakkir Nor Ayob, Mohd Hafiz Fazalul Rahiman, and Chan Kok Seong, "Single-plane dual-modality tomography for multiphase flow imaging by integrating electrical capacitance and ultrasonic sensors," IEEE Sensors Journal, Vol. 17, No. 19, 6368-6377, 2017. Google Scholar

14. Yue, Shihong, Teresa Wu, Jian Pan, and Huaxiang Wang, "Fuzzy clustering based ET image fusion," Information Fusion, Vol. 14, No. 4, 487-497, 2013. Google Scholar

15. Malikov, Azamatjon Kakhramon ugli, Manuel Fernando Flores Cuenca, Beomjin Kim, Younho Cho, and Young H Kim, "Ultrasonic tomography imaging enhancement approach based on deep convolutional neural networks," Journal of Visualization, Vol. 26, No. 5, 1067-1083, 2023. Google Scholar

16. Shi, Qinzhen, Tianhua Zhou, Yifang Li, Chengcheng Liu, and Dean Ta, "Deep learning for TOF extraction in bone ultrasound tomography," IEEE Transactions on Computational Imaging, Vol. 8, 1063-1073, 2022. Google Scholar

17. Xiao, Jun, Ze Liu, Pengfei Zhao, Yong Li, and Jiwei Huo, "Deep learning image reconstruction simulation for electromagnetic tomography," IEEE Sensors Journal, Vol. 18, No. 8, 3290-3298, 2018. Google Scholar

18. Zhang, Wenbiao, Zexin Zhu, and Yijian Geng, "Simultaneous conductivity and permeability reconstructions for electromagnetic tomography using deep learning," IEEE Transactions on Instrumentation and Measurement, Vol. 72, 1-11, 2023. Google Scholar

19. Guo, Qi, Jiamin Ye, Chao Wang, and Xiao Liang, "An ill-conditioned optimization method and relaxation strategy of Landweber for EMT system based on TMR," IEEE Transactions on Instrumentation and Measurement, Vol. 70, 1-9, 2020. Google Scholar

20. Gan, Woon Siong, Acoustical Imaging: Techniques and Applications for Engineers, John Wiley & Sons, 2012.

21. Huang, Gao, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q. Weinberger, "Densely connected convolutional networks," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4700-4708, 2017.

22. Vaswani, A., N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, "Attention is all you need," 31st Conference on Neural Information Processing Systems (NIPS 2017), Vol. 30, Long Beach, CA, USA, 2017. Google Scholar

23. Dinh, Laurent, Jascha Sohl-Dickstein, and Samy Bengio, "Density estimation using Real NVP," ArXiv Preprint ArXiv:1605.08803, 2016. Google Scholar

24. Zhou, Man, Xueyang Fu, Jie Huang, Feng Zhao, Aiping Liu, and Rujing Wang, "Effective pan-sharpening with transformer and invertible neural network," IEEE Transactions on Geoscience and Remote Sensing, Vol. 60, 1-14, 2021. Google Scholar

25. Wang, Qilong, Banggu Wu, Pengfei Zhu, Peihua Li, Wangmeng Zuo, and Qinghua Hu, "ECA-Net: Efficient channel attention for deep convolutional neural networks," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 11534-11542, 2020.

26. Lee, Chen-Yu, Saining Xie, Patrick Gallagher, Zhengyou Zhang, and Zhuowen Tu, "Deeply-supervised nets," Artificial Intelligence and Statistics, 562-570, 2015.

Mr. Jinxun Le

Prof. Ronghua Zhang

Dr. Wenying Fu

Dr. Shuqing Jia

Dr. Xuefeng Bai

Dr. Boyang Li