An Optimized Machine Learning Framework for Heart Disease Prediction: Achieving a New State-of-the-Art in Accuracy and Explainability

Abstract

Cardiovascular Disease (CVD) is the major cause of mortality across the globe and requires predictive models that are highly precise, clinically transparent to ensure easyintegrationwithin the healthcare systems. The antecedent State-of-the-Art (SOTA) error on the target data was 92.0% (Random Forest). This thesis builds a machine learning model on top of an optimized framework, to exceed the current SOTA benchmark. The consolidated Heart Failure Prediction dataset was found to be analyzed comparatively on five different models (Random Forest, XGBoost, LightGBM, SVC, and MLP Classifier). IQR-based Outlier Removal and Standard Scaling of all the numeric features were some of the fundamental preprocessing steps involved in the methodology. The data was divided into 80 per cent of training and 20 per cent of test data. The Multi-Layer Perceptron (MLP) Classifier has a test set accuracy of 92.93% creating a new State-of-the-Art (SOTA) in this predictive task that is 0.93 points higher than the former benchmark. Most importantly, the Explainable AI (XAI) part, which has been verified through SHAP analysis of the probability output of the MLP, proves that the model decisions are indeed presupposed by clinically significant aspects: ST_Slope (the most significant impact) and Oldpeak (Asymptomatic Chest Pain). This study provides a clear, high-fidelity predictive architecture, which satisfies the most important requirement of reliable, explainable decision-making in Clinical Decision Support System (CDSS), and clearly and effectively improves the SOTA in predictive accuracy of heart diseases.