An Optimized Machine Learning Approach for Improved Sentiment Detection and Enhanced Recommendation Systems Using Drug Reviews

Abstract

Sentiment analysis, as a branch of Natural Language Processing (NLP) is becoming more useful in healthcare by helping understand patient feedback about medicines. This study aims to improve how we evaluate drug effectiveness by combining advanced NLP techniques and machine learning methods. We also propose creating a Drug Recommendation System to support healthcare professionals in choosing the right medicines. Our study takes this further by introducing five sentiment levels: Frustrated, Bad, Neutral, Good, and Excited, based on patient ratings. We utilized a dataset obtained from the UCI Machine Learning Repository for this research and collected additional data to balance the dataset. The text data is cleaned and prepared using NLP techniques such as breaking text tokenization involves dividing or cutting the text into small pieces and eliminating punctuation and unnecessary words, and converting words to the root or base forms (stemming and lemmatization). For understanding text better, we used methods like Bag of Words (BoW), Term Frequency-Inverse Document Frequency (TF-IDF), Word2Vec, and manual feature creation. To handle uneven data and improve results collected additional data and also we used SMOTE-SO-MAK, a technique that creates extra samples for less common sentiment categories. Among the different machine learning models tested, Logistic Regression (LR) gave the best results. We checked the system’s accuracy and performance using measures like precision, recall, F1- score, and overall accuracy. This study improves drug recommendation systems by integrating the latest NLP, machine learning algorithms and data balancing, and testing methods.