Browsing by Author "Shatabda, Swakkhar"

Now showing 1 - 8 of 8

A multimodal approach to dementia detection using contrastive learning and LLM–VLM assisted reasoning with guided prompting
(BRAC University, 2025) Zaman, Fariha; Sanjana, Nowrin; Shatabda, Swakkhar
Early detection of dementia remains challenging due to the cost, limited accessibility and subjectivity of traditional clinical assessments. This study proposes a non-invasive multimodal dementia detection framework using spontaneous speech from the Pitt Corpus of DementiaBank by exploring contrastive learning using text–audio representation and reasoning-based foundation models, Large Language Models and Vision Language Models. Linguistic features capturing lexical diversity, syntactic complexity, coherence,etc were combined with acoustic features including pitch, jitter, shimmer, MFCC,etc with Contrastive Language Audio Pretraining (CLAP) based text and audio embeddings. Among classical machine learning classifiers, the best performing configuration using Random Forest with text, audio and handcrafted features achieved an accuracy of 90.83%, F1-score of 0.9083 and AUC of 0.9478, while LightGBM achieved 89.91% accuracy, 0.8989 F1-score and the highest AUC of 0.9675, demonstrating the effectiveness of multimodal fusion of text, audio and features over unimodal baselines. In addition, large language models were evaluated under instruction based inference without fine tuning, where GPT-OSS achieved the highest accuracy of 68.07% among the tested LLMs, outperforming Qwen3-4B and Mistral. Vision language models were further examined using different prompting techniques and the hybrid VLM + LLM reasoning pipeline consistently outperformed standalone VLM configurations, indicating that hierarchical reasoning enhances multimodal dementia classification. Overall, the findings show that contrastive multimodal learning achieves strong classification performance, while reasoning based LLM and VLM frameworks enhance interpretability highlighting the potential of AI assisted methods for early dementia screening.
An LSTM network-based model with attention techniques for predicting linear T-cell epitopes of the hepatitis C virus
(Scopus, 2024) Hosen, Md. Faruk; Mahmud, S. M. Hasan; Goh, Kah Ong Michael; Uddin, Muhammad Shahin; Nandi, Dip; Shatabda, Swakkhar; Shoombuatong, Watshara
: Hepatitis C virus (HCV) infection remains a significant global health challenge, often resulting in severe longterm physical complexity and even death. Since its discovery, HCV has exhibited substantial genetic variability, complicating vaccine development. Although some therapeutic approach have shown efficacy against certain HCV genotypes, a universally effective vaccine is still lacking. Recent research suggests that the body’s cellular immune response, particularly T cell epitopes of HCV (TCE-HCVs), plays a vital role in fighting the virus. Therefore, the precise and rapid identification of TCE-HCVs is essential for chronic HCV infection. In this work, we proposed a novel TCE-HCVs prediction model AttLSTM, which combines attention mechanism and long shortterm memory (LSTM). Specifically, we employed four robust feature encoding techniques: One-Hot Encoding, Global Vectors (GloVe), fastText, and Word2Vec to encode protein sequences. Additionally, k-mer embedding was utilized to help the model identify significant subsequence fragments within the protein sequences. To optimize the model’s performance, irrelevant features are eliminated using the SHapley Additive exPlanations (SHAP) approach. The resulting optimal feature subset was then fed into the AttLSTM model to identify TCEHCVs. The attention mechanism in this model dynamically captures the pairwise correlations of each neighboring target pair within a sliding window, thereby enhancing the understanding of the local environment of target residues. Extensive experiments showed that AttLSTM outperformed conventional machine learning (ML) classifiers in predictive performance. Notably, in k-fold cross validation, AttLSTM achieved superior performance compared to existing methods with accuracy of 80.77 %, MCC of 0.632, and AUC of 0.891. This exceptional performance indicates that AttLSTM has a strong predictive capability for identifying TCE-HCVs. We anticipate that AttLSTM will expedite the rapid identification of promising TCE-HCVs, aiding in the development of diagnostic and immunotherapeutic treatments for HCV in the future.
Benchmarking vision-language models for traffic scene understanding in South Asian traffic environments
(BRAC University, 2025-12) Choity, Naznin Sultana; Takmim, Samiha; Rahman, Abida; Shaid, Abdullah Al; Hossain, Md.Tanzim; Shatabda, Swakkhar
A substantial body of research has investigated methods to promote safer driving, yet many challenges persist, particularly in environments with complex traffic patterns. This study focuses on how drivers evaluate their surroundings and make safety-critical deci- sions, with a specific emphasis on the role of deep learning in enhancing safe driving. Deep learning enables rapid identification of hazardous situations by providing real-time feedback and alert mechanisms, thereby improving driving behavior and reducing risks. The objective of this work is to develop an AI-powered driving assistance system de- signed to enhance road safety, especially for inexperienced drivers. Real-world driving conditions were incorporated by collecting YouTube footage across diverse road types and traffic densities. Experts annotated the dataset by labeling key objects and identifying context-specific risk factors, decision-making cues, and hazard indicators. The system is powered by a custom-designed AI model capable of providing context- aware guidance, regulatory reminders, and hazard alerts in real time. By leveraging expert-annotated data and multimodal deep learning techniques, the system delivers per- sonalized and immediate support to increase driver situational awareness, confidence, and safety.
Diffusion models with graph attention for spatiotemporal EEG inpainting
(BRAC University, 2026-01) Dutta, Anindita; Joy, Angkon Dutta; Mondal, Plabon; Nisha, Humayra Anjum; Alam, Md. Golam Rabiul; Shatabda, Swakkhar
Contamination of artifacts, and loss of signal is a major impairment in Electroencephalography (EEG) recording reliability and standard methods of interpolation do not resolve the multichannel brain signal complex spatial-temporal dependencies. Current state-of-the-art approaches such as transformer or GAN based methods either do not consider EEG channels interaction or have high computational resources and cannot maintain critical frequency-domain properties. The synergistic combination of diffusion models with graph attention mechanisms and low-rank decomposition of the diffusion, which can both use the learned spatial relationships and specify mathematical priors, is still an open challenge in the field. In this paper, a new model that combines Denoising Diffusion Probabilistic Models (DDPM), Graph Attention Networks (GAT), and Singular Value Decomposition (SVD) to reconstruct corrupted EEG segments with 32 channels is suggested. The DEAP dataset (32 subjects, 40 emotion-elicitation trials) goes through preprocessing processes like baselines correction, channel-wise z-score normalization, percentile clipping, and 640-sample windowing with 50% overlap. SVD estimates first 16 orthogonal spatial-temporal components, which are used as priors and fused with raw features, and a hierarchical 1D U-Net with graph attention is then trained in 300 steps to reconstruct masked parts of the EEG. The DDPM+GAT+SVD model gives a Mean Squared Error (MSE) of 0.014153 ± 0.014911, Root Mean Squared Error (RMSE) of 0.110374 ± 0.044388, Mean Absolute Error (MAE) of 0.087305 ± 0.034363, Pearson Correlation Coefficient (PCC) of 0.8766 ± 0.1207 and Power Spectral Density (PSD) distance of 0.001293 ± 0.003648, ∼80.6% RMSE improvement over DEAP DIVE baseline. Ablation experiments bear out the synergistic contributions: DDPM+SVD (MSE 0.017276 ± 0.015000) and DDPM+GAT (MSE 0.026832 ± 0.020000) are worse by 22.1 percent and 47.2 percent, respectively, which confirms that mathematical decomposition, learned spatial attention and temporal diffusion are complements. Generalization without changes in architecture is proven by cross-dataset validation on PhysioNet Motor Imagery dataset (MSE 0.196824 ± 0.461074, RMSE 0.295641 ± 0.330788, PCC 0.8387 ± 0.1478). SVDs are also pre-computed in the process which saves training time which can be deployed with ease. These findings show that diffusion models structured hierarchically by incorporating graph attention and low-rank decomposition offer robust and physiologically plausible empirical findings on EEG reconstruction and are more effective in clinical diagnostic (as well as brain-computer interface and affective computing) tasks, compared to traditional artifact removal.
Generalized bridge pipeline for multimodal gene regulatory network discovery
(BRAC University, 2025-10) Alif, Abrar Sami Khan; Fahmid, Riyadus Salehin; Raihan, Mohammad Omar; Chowdhury, Omor Bin Amjad; Alam, Md. Golam Rabiul; Shatabda, Swakkhar
In this thesis, a computationally generalized pipeline where multimodal gene regulatory networks (GRNs) are built by combining transcriptomic data in RNA sequencing and functional dependency data in CRISPR knockout screens. Traditional forms of GRN rely on expression data as the only tool which can not detect causal or functional significant interactions. We attempted to solve this by constructing a contrastive bridge model, where both datasets are put in the same 128-dimensional latent space. We used Maximum Mean Discrepancy (MMD) loss and a diversitypreserving loss such that patterns of modality are aligned, and meaningful biological variation is not distorted. Using these embeddings, we built multimodal GRNs, combining evidence as provided by various outlets. In order to identify statistical and functional relationships, we demonstrated Spearman co-expression correlations, GENIE3 random forest importance scores, CRISPR dependency support, and cosine similarity of embedding vectors into a single edge-weight expression. The bridgefused networks have been steady in structure and introduced new cross-modal interactions (Bridge-fused vs GENIE3) when used in both hematopoietic and lung cell data. Top hub genes in these networks scored negative on the mean CRISPR dependency score which is an indication of important functional roles and Gene Ontology enrichment analysis scored significant representation of the immune activation and metabolic processes. The implications of these findings are that the bridge pipeline offers biologically meaningful, consistent and interpretable GRNs. Overall, this framework is a generalizable and data-driven framework to integrate heterogeneous genomic datasets, which can be applied in the process of identifying significant regulators and potential therapeutic targets in a broad variety of biological settings.
Information retrieval from tables in financial documents
(BRAC University, 2026-01) Chowdhury, Waseque; Abrar, Md. Nahid; Shah, Tanveer Ahmed; Chowdhury, Md. Sakib Hasan; Shatabda, Swakkhar
Structured financial data including balance sheets, income statements, risk metrics need tables in order to be presented within financial documents. Tables need correct infor- mation extraction for both financial analytic work and monitoring compliance together for generation of reports. The combination of complex table elements that include cells and merged headers as well as irregular layout structures creates problems for standard processing methods. This paper constructs an information retrieval framework which assesses machine learning, computer vision together with natural language processing, detection transformer, large language models, Vision language models, convolutional neu- ral network, optical character recognition, structured points of thought to find the most e!ective strategies for financial document table detection, table structure recognition and information extraction. The retrieved table relations enable clients to request financial in- formation, historical data and comparisons between di!erent table elements. The analysis of this research examines various tabular documents to understand interdisciplinary table detection methods through both positive and negative aspects as well as their practical utility in real-world contexts. From various case study reports, the e”ciency of di!erent models, datasets in processing multi-page financial documents, with robust cell detection and data extraction from complicated tabular layouts will define an optimized solution when addressing practical tasks. This research strengthens the connection between table detection methods and semantic retrieval technology to develop inexpensive automated platforms that retrieve financial data and perform analysis.
Multimodal deep learning for predicting mechanical ventilation duration from chest X-ray and clinical data
(BRAC University, 2026-01) Arnab, Abrar Shahrier; Fiyaz, Yeasin; Rahman, S M Mahidur; Alfa, Umma Souda; Shatabda, Swakkhar
In critical care facilities, invasive mechanical ventilation is now a key criterion for determining the severity of illness caused by the COVID-19 pandemic. Both clinical decision processes and the efficient use of critical care facilities can be improved by using the predicted duration of invasive ventilation. In order to predict the invasive ventilator days, the study proposes the use of a multimodal learning architecture that utilizes both clinical data and chest X-ray images.The Cancer Imaging Archive (TCIA) dataset included chest X-ray images and clinical data for patients. A total of 213 full instances were retained after rigorous data cleansing and mapping using patient identifiers. Z-score normalization was used to normalize the clinical data, and intensity normalization and scaling were used to normalize the chest X-ray DICOM image data. The study used a continuous regression model to predict the ventilation day outcome.The study used a variety of deep learning and machine learning models, including ResNet-18, DenseNet-121, Random Forest, Linear Regression, and Gradient Boosting. A multimodal attention-based fusion model was used to combine the image and clinical data. The study used MAE, RMSE, R², Pearson correlation, and Spearman correlation to evaluate the model’s performance. The results of the experiments show that multimodal deep-learning models have moderate performance, and the ensemble-based clinical models perform better than the linear ones. Besides, the results highlight the significant role of clinical data, and image data provides marginal performance improvements.
Zero-shot detection of jailbreaking attempts in LLMs
(BRAC University, 2025-10) Rahman, Md. Hasib Ur; Ankur, Arittra Paul; Zahin, Sadman; Fardin, Mominul Hoque; Shatabda, Swakkhar; Chakrabarty, Amitabha
The widespread deployment of Large Language Models (LLMs) has introduced significant safety challenges, notably the emergence of sophisticated ‘jailbreak’ attacks designed to bypass alignment measures and elicit harmful responses. While existing defenses often fail to generalize novel, zero-day attacks. we investigate the hypothesis that a classifier trained to a known distribution of attack patterns can achieve superior detection performance on entirely unseen adversarial prompts. We demonstrate that training on a specialized corpus of engineered safe prompts data that mirrors the structure and tonality of attacks—enhances the model’s ability to recognize conceptually similar yet novel threat vectors. When evaluated on a completely unseen challenge dataset of prompts confirmed to jailbreak state-of-theart models (including Grok-4, Grok-4 Heavy, and Gemini-2.5-Pro), our specialized detector improves accuracy from a baseline of 62.22% to 73.33%. These results, achieved with a compact training set, suggest that for rapidly evolving security threats like jailbreaking, targeted training with high-fidelity engineered data offers a more effective and resource-efficient defense mechanism than reliance on generalized, large-scale datasets.