An Investigation and Evaluation of N-Gram, TF-IDF and Ensemble Methods in Sentiment Classification

Abstract

In the area of sentiment analysis and classification, the performance of the classification tasks can be varied based on the usage of text vectorization and feature extraction methods. This paper represents a detailed investigation and analysis of the impact on feature extraction methods to attain the highest classification accuracy of the sentiment from user reviews. Unigram, Bigram and Trigram are applied as n-gram vectorization models with TF-IDF features extraction method individually. Accuracy, misclassification rate, Receiver Operating Characteristics (ROC) and recall-precision are used in this study to evaluate which are counted as the most important performance measurement parameters in machine learning based approaches. Parameters are measured by the output obtained from Bagged Decision Tree (BDT), Random Forest (RF), Ada Boost (ADA), Gradient Boost (GB) and Extra Tree (ET). The outcomes of this study is to find out the best fitted combination of term frequency–inverse document frequency (TF-IDF) and n-grams for different data size.