Multimodal Deep Learning Fusion Strategies for Alzheimer’s Disease Classification

Abstract

This study explores multimodal deep
learning models for diagnosing Alzheimer’s disease
(AD), integrating up to eight data modalities, including
clinical, imaging, and genetic information, using the
OASIS-3 dataset. Three fusion strategies (early, late,
and intermediate) are implemented and compared to
effectively combine heterogeneous data. Three case
studies are considered: (1) binary classification using
magnetic resonance imaging (MRI) and the Clinical
Dementia Rating (CDR) scale to distinguish between
cognitively normal individuals and those with AD
dementia; (2) binary classification using eight modalities
(MRI, CDR, CENTILOID, FAQ, NPI-Q, D1, C1, and
B8), improving predictive accuracy and robustness;
and (3) multiclass classification with the same eight
modalities to predict cognitive normality, uncertain
dementia, or other dementia subtypes. Experimental
results show that multimodal models consistently out-
perform unimodal approaches, demonstrating superior
classification performance and greater resilience to
uncertainty. Despite challenges in model interpretability
and dataset balancing, these findings underscore the
potential of multimodal fusion strategies to improve
computer-assisted diagnosis of Alzheimer’s disease.