volume | PIER Journals

Abstract

Alzheimer's disease (AD) is a degenerative disease of the nervous system that often occurs in the elderly. As magnetic resonance imaging (MRI) and positron emission tomography (PET) reflect the brain's anatomical changes and functional changes caused by AD, they are often used to diagnose AD. Multimodal fusion based on these two types of images can effectively utilize complementary information and improve diagnostic performance. To avoid the computational complexity of the 3D image and expand training samples, this study designed an AD diagnosis framework based on a 2.5D convolutional neural network (CNN) to fuse multimodal data. First, MRI and PET were preprocessed with skull stripping and registration. After that, multiple 2.5D patches were extracted within the hippocampus regions from both MRI and PET. Then, we constructed a multimodal 2.5D CNN to integrate the multimodal information from MRI and PET patches. We also utilized a training strategy called branches pre-training to enhance the feature extraction ability of the 2.5D CNN by pre-training two branches with corresponding modalities individually. Finally, the results of patches are used to diagnose AD and progressive mild cognitive impairment (pMCI) patients from normal controls (NC). The experiments were conducted with the ADNI dataset, and accuracies of 92.89% and 84.07% were achieved in the AD vs. NC and pMCI vs. NC tasks. The results are much better than using single modality and indicate that the proposed multimodal 2.5D CNN could effectively integrate complementary information from multi-modality and yield a promising AD diagnosis performance.

1. Todd, S., S. Barr, M. Roberts, and A. P. Passmore, "Survival in dementia and predictors of mortality: A review," International Journal of Geriatric Psychiatry, Vol. 28, No. 11, 1109-1124, 2013. Google Scholar

2. Markesbery, W. R. and M. A. Lovell, "Neuropathologic alterations in mild cognitive impairment: A review," Journal of Alzheimer's Disease, Vol. 19, No. 1, 221-228, 2010. Google Scholar

3. Tong, T., Q. Gao, R. Guerrero, C. Ledig, and D. Rueckert, "A novel grading biomarker for the prediction of conversion from mild cognitive impairment to Alzheimer's disease," IEEE Transactions on Biomedical Engineering, Vol. 64, No. 1, 1, 2016. Google Scholar

4. Eskilden, S. F., P. Coupé, D. García-Lorenzo, V. Fonov, J. C. Pruessner, and D. L. Collins, "Prediction of Alzheimer's disease in subjects with mild cognitive impairment from the ADNI cohort using patterns of cortical thinning," Neuroimage, Vol. 65, 511-521, 2013. Google Scholar

5. Tong, T., R. Wolz, Q. Gao, and R. Guerrero, "Multiple instance learning for classification of dementia in brain MRI," Medical Image Analysis, Vol. 18, No. 5, 808-818, 2014. Google Scholar

6. Drzezga, A., D. Altomare, C. Festari, J. Arbizu, S. Orini, K. Herholz, P. Nestor, F. Agosta, F. Bouwman, and F. Nobili, "Diagnostic utility of 18F-Fluorodeoxyglucose positron emission tomography (FDG-PET) in asymptomatic subjects at increased risk for Alzheimer's disease," European Journal of Nuclear Medicine and Molecular Imaging, Vol. 45, No. 9, 1487-1496, 2018. Google Scholar

7. Liu, M., D. Cheng, and W. Yan, "Classification of Alzheimer's disease by combination of convolutional and recurrent neural networks using FDG-PET images," Frontiers in Neuroinformatics, Vol. 12, 35, 2018. Google Scholar

8. Alberdi, A., A. Aztiria, and A. Basarab, "On the early diagnosis of Alzheimer's disease from multimodal signals: A survey," Artificial Intelligence in Medicine, Vol. 71, 1-29, 2016. Google Scholar

9. Mzoughi, H., I. Njeh, A. Wali, et al. "Deep multi-scale 3D convolutional neural network (CNN) for MRI gliomas brain tumor classification," Journal of Digital Imaging, Vol. 33, 903-915, 2020. Google Scholar

10. Gao, Y., Z. Li, C. Song, et al. "Automatic rat brain image segmentation using triple cascaded convolutional neural networks in a clinical PET/MR," Physics in Medicine and Biology, Vol. 66, No. 4, 04NT01, 2021. Google Scholar

11. Zhang, Q., Y. Liao, X. Wang, et al. "A deep learning framework for 18F-FDG PET imaging diagnosis in pediatric patients with temporal lobe epilepsy," European Journal of Nuclear Medicine and Molecular Imaging, Vol. 48, 2476-2485, 2021. Google Scholar

12. Zhang, R., C. Cheng, X. Zhao, and X. Li, "Multiscale mask R-CNN-based lung tumor detection using PET imaging," Molecular Imaging, Vol. 18, 1-8, 2019. Google Scholar

13. Zheng, H., L. Qian, Y. Qin, et al. "Improving the slice interaction of 2.5D CNN for automatic pancreas segmentation," Medical Physics, Vol. 47, 5543-5554, 2020. Google Scholar

14. Kitrungrotsakul, T., X. Han, Y. Iwamoto, et al. "A cascade of 2.5D CNN and bidirectional CLSTM network for mitotic cell detection in 4D microscopy image," IEEE-Acm Transactions on Computational Biology and Bioinformatics, Vol. 18, 396-404, 2021. Google Scholar

15. Li, A., F. Li, F. Elahifasaee, M. Liu, and L. Zhang, "Hippocampal shape and asymmetry analysis by cascaded convolutional neural networks for Alzheimer's disease diagnosis," Brain Imaging and Behavior, 1-10, 2021. Google Scholar

16. Jo, T., K. Nho, S. L. Risacher, and A. J. Saykin, "Deep learning detection of informative features in tau PET for Alzheimer's disease classification," BMC Bioinformatics, Vol. 21, No. Suppl 21, 496, 2020. Google Scholar

17. Gao, X., R. Hui, Z. Tian, et al. "Classification of CT brain images based on deep learning networks," Computer Methods and Programs in Biomedicine, Vol. 138, 49-56, 2017. Google Scholar

18. Suk, H. I., S. W. Lee, and D. Shen, "Latent feature representation with stacked auto-encoder for AD/MCI diagnosis," Brain Structure and Function, Vol. 220, No. 2, 841-859, 2015. Google Scholar

19. Zhang, D., Y. Wang, L. Zhou, H. Yuan, and D. Shen, "Multimodal classification of Alzheimer's disease and mild cognitive impairment," NeuroImage, Vol. 55, No. 3, 856-867, 2011. Google Scholar

20. Huang, Y., J. Xu, Y. Zhou, and T. Tong, "Diagnosis of Alzheimer's disease via multi-modality 3D convolutional neural network," Frontiers in Neuroscience, Vol. 13, 509, 2019. Google Scholar

21. Cheng, D. and M. Liu, "CNNs based multi-modality classification for AD diagnosis," 2017 10th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), 1-5, 2017. Google Scholar

22. Zhou, P., S. Jiang, L. Yu, Y. Feng, C. Chen, and F. Li, "Use of a sparse-response deep belief network and extreme learning machine to discriminate Alzheimer's disease, mild cognitive impairment, and normal controls based on amyloid PET/MRI images," Frontiers in Medicine, Vol. 7, 987, 2021. Google Scholar

23. Zhu, X., H. Suk, and D. Shen, "Low-rank dimensionality reduction for multi-modality neurodegenerative disease identification," World Wide Web, Vol. 22, No. 2, 907-925, 2019. Google Scholar

24. Lin, W., Q. Gao, J. Yuan, Z. Chen, and C. Feng, "Predicting Alzheimer's disease conversion from mild cognitive impairment using an extreme learning machine-based grading method with multimodal data," Frontiers in Aging Neuroscience, Vol. 12, 77, 2020. Google Scholar

25. Giorgio, A., L. Santelli, V. Tomassini, R. Bosnell, S. Smith, N. D. Stefano, and H. Johansen-Berg, "Age-related changes in grey and white matter structure throughout adulthood," NeuroImage, Vol. 51, No. 3, 943-951, 2010. Google Scholar

26. Dukart, J., M. L. Schroeter, and K. Muller, "Age correction in dementia-matching to a healthy brain," PloS One, Vol. 6, No. 7, e22193, 2011. Google Scholar

27. Lin, W., T. Tong, Q. Gao, D. Guo, X. Du, Y. Yang, G. Guo, M. Xiao, M. Du, and X. Qu, "Convolutional neural networks-based MRI image analysis for the Alzheimer's disease prediction from mild cognitive impairment," Frontiers in Neuroscience, Vol. 12, 777, 2018. Google Scholar

28. Roth, H. R., L. Lu, J. Liu, J. Yao, A. Seff, K. Cherry, L. Kim, and R. M. Summers, "Improving computer-aided detection using convolutional neural networks and random view aggregation," IEEE Transactions on Medical Imaging, Vol. 35, No. 5, 1170-1181, 2016. Google Scholar

29. Han, X., J. Jovicich, D. Salat, A. Kouwe, B. Quinn, S. Czanner, et al. "Reliability of MRI-derived measurements of human cerebral cortical thickness: The effects of eld strength, scanner upgrade and manufacturer," NeuroImage, Vol. 32, No. 1, 180-194, 2006. Google Scholar

30. Lin, M., Q. Chen, and S. Yan, Network in network, Proceedings of the IEEE International Conference on Learning Representations, 2014.

31. Szegedy, C., W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, et al. "Going deeper with convolutions," The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1-9, 2015. Google Scholar

32. He, K., X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770-778, 2016. Google Scholar

33. Ioffe, S. and C. Szegedy, "Batch normalization: Accelerating deep network training by reducing internal covariate shift," Proceedings of the 32nd International Conference on International Conference on Machine Learning, Vol. 37, 448-456, 2015. Google Scholar

34. Elahifasaee, F., F. Li, and M. Yang, "A classification algorithm by combination of feature decomposition and kernel discriminant analysis (KDA) for automatic MR brain image classification and AD diagnosis," Computational and Mathematical Methods in Medicine, Vol. 2019, 1-14, 2019. Google Scholar

35. Oh, K., Y. C. Chung, K. W. Kim, and I. S. Oh, "Classification and visualization of Alzheimer's disease using volumetric convolutional neural network and transfer learning," Scientific Reports, Vol. 9, No. 1, 18150-18165, 2019. Google Scholar

36. Salvatore, C., A. Cerasa, P. Battista, M. Gilardi, A. Quattrone, and I. Castiglioni, "Magnetic resonance imaging biomarkers for the early diagnosis of Alzheimer's disease: A machine learning approach," Frontiers in Neuroscience, Vol. 9, 307-319, 2015. Google Scholar

37. Liu, M., D. Cheng, K. Wang, et al. "Multi-modality cascaded convolutional neural networks for Alzheimer's disease diagnosis," Neuroinformatics, Vol. 16, No. 3-4, 295-308, 2018. Google Scholar

Dr. Xuyang Zhang

Dr. Weiming Lin

Dr. Min Xiao

Dr. Huazhi Ji