A cross-modal attention-based multimodal deep learning framework for early prediction of adolescent mental health disorder

Abstract

Adolescent mental health conditions are frequently underdiagnosed due to the limitations of single diagnostic techniques and the reduced efficiency of procedures that fail to combine biological, psychological, and social aspects. We introduce a cross-modal attention-based multimodal deep learning framework that uses structural brain imaging and phenotypic data to identify anxiety disorders in the early stages. The proposed method incorporates the QTAB dataset to concurrently model T1-weighted magnetic resonance imaging (T1w MRI), behavioral questionnaire responses, and demographic variables, revealing related patterns of risk across modalities. The framework consists of three independently trained modality-specific encoders: a 3D convolutional neural network for structural MRI that generates 768- dimensional neuroanatomical representations, and two self-attention enhanced prototypical learning modules for behavioral and demographic data that generate discriminative metric embeddings of 32 and 64 dimensions, respectively. Cross-modal integration happens at the decision level, allowing for more reliable fusion in sample sizes that are limited. The experimental evaluation shows that the proposed framework obtains an AUC of 0.8935 for predicting anxiety disorders, representing a 15.1% improvement over the most robust unimodal baseline (questionnaire: AUC = 0.7766), with 85.7% sensitivity and 87.3% specificity. The best fusion weights were 23% MRI, 63% questionnaire, and 14% demographics, demonstrating the questionnaire data’s higher predictive signal. Results demonstrate that optimized weighted late fusion of well-calibrated modality predictions surpasses intricate learnt fusion techniques, underscoring the significance of proper weight optimization and calibration in small-sample multimodal psychiatric modelling. The results illustrate the efficacy of multimodal late fusion in the early detection of anxiety risk in adolescents and offer a clinically significant approach for scalable mental health screening systems.