Redefining Bangla Text Processing with a New Stemming Methodology

Abstract

Stemming is a basic NLP technique that normalizes the linguistic structure and improves the performance of text analysis by stripping words to their roots or base forms by removing suffixes which are quite vital in applications such as text normalization, information retrieval and keeping linguistic consistency. The Bangla language has a special stemming problem due to its extensive morphological structure with many inflectional changes and intricate grammatical rules. Hence, finding the root form of a word in Bangla involves much more complex affixation patterns and compound word creation than is evident in other languages with simpler grammatical systems. To handle the linguistic complexities of Bangla better than the previous approaches. In this work a new stemming method is developed for the language. It is more adaptable and practical for root word extraction in the real world because it has been equipped with the most updated methods to recognize and handle both the Bangla suffixes and morphological changes. Very encouraging results are obtained from the complex experiment carried out on a dataset containing 1,000 unique Bangla words. It’s extremely high F1-score of 85.66% and high accuracy rate of 87.2% do, in fact, justify its superior performance over traditional stemming algorithms. It is apparent from the present study that this newly proposed stemming strategy significantly enhances the efficacy and efficiency of all Bangla text processing systems, including search engines, information retrieval platforms, and all NLP applications. This study paves the way for further development in the Bangla Language Processing problem and emphasizes the cruciality of continued research in developing language-specific Natural Language Processing tools.