Conference Papers (Centre for Research on Bangla Language Processing)

Browse

Now showing 1 - 20 of 40

Morphological parsing of Bangla wods using PC-KIMMO
(BRAC University, 2004) Dasgupta, Sajib; Khan,Mumit
This paper describes Morphological parsing of Bangla words using PC-KIMMO, based on Kimmo Koskeniemil's model of two-level Morphology. There are three sections in the PC-KIMMO: rules section lexicon section and grammar section. We explain here how to write these sections in PC-KIMMO to do morphological analysis for Bangla.
Feature unification for morphological parsing in Bangla
(BRAC University, 2004) Dasgupta, Sajib; Khan, Dr. Mumit
This paper describes a Feature Unification Based Word Grammar model for the morphological parsing of Bangla words. While normal morphological parsing strategy is adequate to decompose a word into morphemes, it is not able directly to compute the part of speech of a derivationally complex word or return a word's inflectional features--precisely the information required for syntactic parsing. These deficiencies have now been remedied by adding a unification-based word grammar component which can provide parse trees and feature structures. In addition to that, feature unification lessens the number of lexicon classes (less space) and actually reduces the complexities regarding morphotactic analysis.
A double metaphone encoding for Bangla and its application in spelling checker
(BRAC University, 2005) Naushad UzZaman; Khan, Mumit
We present a Double Metaphone encoding for Bangla that can be used by spelling checkers to improve the quality of suggestions for misspelled words. The complex rules of Bangla spelling present a significant challenge in producing suggestions for a misspelled word when employing the traditional edit-distance methods; one must take phonetic similarity into account for the suggested alternatives to be reasonably accurate. We propose a Double Metaphone encoding for Bangla, taking into account the various context-sensitive rules, including those involving the large repertoire of consonant clusters in Bangla, and present a comparison with the traditional edit-distance based methods in producing suggestions for misspelled words.
Morphological analysis of inflecting compound words in Bangla
(BRAC University, 2005) Dasgupta, Sajib; Khan, Naira; Sarkar, Asif Iqbal; Pavel, Dewan Shahriar Hossain; Khan, Mumit
The addition of inflectional suffixes in Bangla com-pound words is fairly complex. A compound is a word that is formed by two or more different words acting as a single entity. One of the key distinguishing features of compounds is the absence of inflectional morphology between the constituents of a compound. In Bangla, however, the constituents may retain inflectional suffixes on either or both the constituents and the resultant compound may then be inflected further as a whole word. Such inflection creates ambiguities as context-free word grammar is unable to recognize whether the inflectional suffix is an inflectional property of the last constituent root-word or of the compound as a whole. We use a feature unification based morphological parser, which can successfully and efficiently parse compound words that retain such inflectional morphology and at the same time resolve such ambiguities.
A double metaphone encoding for approximate name searching and matching in Bangla
(BRAC University, 2005) Naushad UzZaman,; Khan, Mumit
Almost any word can be a Bangali name, and the name in turn is often spelled in many different ways, all of which are considered correct and interchangeable. The reason for the spelling complication is two-fold: (1) there is a large gap between the script and pronunciation in Bangla, largely attributed to the large scale Sanskritization process that started in the 12th century and continued throughout the middle ages, and (2) typical Bangla names have very different origins, from the indigenous names derived primarily from Sanskrit, to the imported Muslim names from Persian and Arabic, Christian names from Portuguese, and even the names from popular Western TV soap-operas. However, there is always a large degree of phonetic similarity in the spelling variants of a name, which is the key to searching and matching names in records. We present a Double Metaphone encoding for Bangla names, taking into account the various spelling and phonetic rules in use, which can be used by applications to search for and match names. We encode the spelling variants of a large number of names found in the literature to demonstrate that the encoding does indeed show that the variants of a name are equivalent. A name searching algorithm may employ various figures of merit to narrow the list of possibilities when searching for similar names; we demonstrate one such figure of merit using name encoding and edit distance that has shown good promise.
T12: an advanced text input system with phonetic support for mobile devices
(BRAC University, 2005) Naushad UzZaman, Khan Mumit
The popular T9 text input system for mobile devices uses a predictive dictionary-based disambiguation scheme, enabling a user to type in commonly-used words with low overhead. We present a new text input system called T12, which in addition to providing T9’s capabilities, also allows a user to cycle through the possible choices based on phonetic similarity, and to elaborate commonly used abbreviations, acronyms and other short forms. This ability to cycle through the possible choices acts as a spelling checker, which provides suggestions from the dictionary with similar pronunciation as the input word.
Teaching compiler development to undergraduates using a template based approach
(BRAC University, 2005) Islam, Md Zahurul; Khan, Mumit
Compiler Design remains one of the most dreaded courses in any undergraduate Computer Science curriculum, due in part to the complexity and the breadth of the material covered in a typical 14-15 week semester time frame. The situation is further complicated by the fact that most undergraduates have never implemented a large enough software package that is needed for a working compiler, and to do so in such a short time span is a challenge indeed. This necessitates changes in the way we teach compilers, and specifically in ways we set up the project for the Compiler Design course at the undergraduate level. We describe a template based method for teaching compiler design and implementation to the undergraduates, where the students fill in the blanks in a set of templates for each phase of the compiler, starting from the lexical scanner to the code generator. Compilers for new languages can be implemented by modifying only the parts necessary to implement the syntax and the semantics of the language, leaving much of the remaining environment as is. The students not only learn how to design the various phases of the compiler, but also learn the software design and engineering techniques for implementing large software systems. In this paper, we describe a compiler teaching methodology that implements a full working compiler for an imperative C-like programming language with backend code generators for MIPS, Java Virtual Machine (JVM) and Microsoft’s .NET Common Language Runtime (CLR).
Analysis of and observations from a Bangla News Corpus
(BRAC University, 2006) Majumder, Khair Md. Yasir Arafat
In this paper we present the compilation methodology and some statistical analysis on a Bangla news corpus-“Prothom-Alo”, which is the first of its kind for Bangla. We compare some of the statistics with the CIIL Bangla corpus and also present our observation of atypical behavior of Zipf’s curve for Prothom-Alo corpus.
N-gram based statistical grammar checker for Bangla and English
(Center for research on Bangla language processing (CRBLP), BRAC University, 2006) Alam, Md. Jahangir; UzZaman, Naushad; Khan, Mumit
This paper describes a statistical grammar checker, which considers the n-gram based analysis of words and POS tags to decide whether the sentence is grammatically correct or not. We employed this technique for both Bangla and English and also described limitation in our approach with possible solutions.
Comparion of different POS tagging technique (N-Gram, HMM and Brill's tagger) for Bangla
(BRAC University, 2006) Hasan, Fahim Muhammad; Naushad UzZaman; Khan, Mumit
There are different approaches to the problem of assigning each word of a text with a parts-of-speech tag, which is known as Part-Of-Speech (POS) tagging. In this paper we compare the performance of a few POS tagging techniques for Bangla language, e.g. statistical approach (n-gram, HMM) and transformation based approach (Brill’s tagger). A supervised POS tagging approach requires a large amount of annotated training corpus to tag properly. At this initial stage of POS-tagging for Bangla, we have very limited resource of annotated corpus. We tried to see which technique maximizes the performance with this limited resource. We also checked the performance for English and tried to conclude how these techniques might perform if we can manage a substantial amount of annotated corpus.
A proposed automated extraction procedure of Bangla text for corpus creation in unicode
(BRAC University, 2006) Pavel, Dewan Shahriar Hossain; Sarkar, Asif Iqbal; Khan, Mumit
This paper addresses the issue of automated Bangla corpus creation, which will significantly help the processes of lexicon development, morphological analysis, automatic parts of speech detection and automatic grammar extraction and machine translation. The plan is to collect all free Bangla documents on the world wide web and offline documents available and extract all the words in them to make a huge repository of text. This body of text or corpus will be used for several purposes of Bangla language processing after it is converted to Unicode text. The conversion process is also one of the associated and equally important research and development issue. Among several procedures our research focuses on a combination of font and language detection and Unicode conversion of retrieved Bangla text as a solution for automatic Bangla corpus creation and the methodology has been described in the paper.
JKimmo: A Multilingual computational mophology frame work for PC-KIMMO
(BRAC University, 2006) Islam, Md. Zahurul; Khan, Mumit
Morphological analysis is of fundamental interest in computational linguistics and language processing. While there are established morphological analyzers for mostly Western and a few other languages using localized interfaces, the same cannot be said for Indic and other less-studied languages for which language processing is just beginning. There are three primary obstacles to computational morphological analysis of these less-studied languages: the generative rules that define the language morphology, the morphological processor, and the computational interface that a linguist can use to experiment with the generative rules. In this paper, we present JKimmo, a multilingual morphological open-source framework that uses the PC-KIMMO two-level morphological processor and provides a localized interface for Bangla morphological analysis. We then apply Jkimmo to Bangla computational morphology, demonstrating both its recognition and generation capabilities. Jkimmo’s internationalization (i18n) frame-work allows easy localization in other languages as well, using a property file for the interface definitions and a transliteration scheme for the analysis.
Collaborative lexicon development for Bangla
(BRAC University, 2006) Pavel, Dewan Shahriar Hossain; Sarkar, Asif Iqbal; Shah, Faisal Muhammad; Khan, Mumit
This paper addresses the issue of building a Bangla lexicon with a collaborative effort through stand alone application and web based interface. The words in the lexicon will be annotated with a combination of tags addressing Parts-of-speech, syntactic, semantic and other grammatical features. Bangla words have been classified into several different parts – of – speech categories including various major word groups and subgroups. This paper aims to provide an integrated user – friendly software interface to the user to annotate a large existing Bangla word set and proposes a mechanism to collaboratively integrate linguists and other interested people into the lexicon build up process. The effort will be a significant progress towards development of a properly annotated lexicon. The outcome of the effort will significantly help in the processes of Morphological Analysis, Automatic grammar Extraction and machine translation for Bangla.
Skew angle detection of bangla script using radon transform
(BRAC University, 2006) Habib, S. M. Murtoza; Noor, Nawsher Ahamed; Khan, Mumit
Skew angle detection and correction an integral part of any OCR system. Without proper skew correction, the performance of an OCR will simply not be acceptable for most scanned images. We propose an innovative method for skew angle detection and correction for Bangla scripts using the Radon Transform. The basic idea is to identify the upper envelope by detecting the headline that accompanies most of the letters in the Bangla script, and then apply the Radon Transform to this upper envelope to get the skew angle. Once the angle is known, the correction is quite trivial to perform. While the current implementation handles only a single skew angle per text image, it can be extended to handle multiple skew angles by partitioning the document image.
GIS Based Real Time Traveler Information System: An Efficient Approach to Minimize Travel Time Using Available Media
(BRAC University, 2006) Hasnat, Md. Abul; Haque, Mohammad Mahmudul; Khan, Mumit
This paper addresses the issue of building a Bangla lexicon with a collaborative effort through stand alone application and web based interface. The words in the lexicon will be annotated with a combination of tags addressing Parts-of-speech, syntactic, semantic and other grammatical features. Bangla words have been classified into several different parts – of – speech categories including various major word groups and subgroups. This paper aims to provide an integrated user – friendly software interface to the user to annotate a large existing Bangla word set and proposes a mechanism to collaboratively integrate linguists and other interested people into the lexicon build up process. The effort will be a significant progress towards development of a properly annotated lexicon. The outcome of the effort will significantly help in the processes of Morphological Analysis, Automatic grammar Extraction and machine translation for Bangla.
History (Forward N-Gram) or future (Backward N-Gram)? Which model to consider for N-Gram analysis in Bangla?
(BRAC University, 2006) Khan, Naira; Habib, Md. Tarek; Alam, Md. Jahangir; Rahman, Rajib; UzZaman, Naushad; Khan, Mumit
This paper presents a directional advantage of n-gram modeling in terms of backward or forward n-gram modeling in Bangla. The most commonly used n-gram analysis is predominantly a forward n-gram. However in Bangla it appears that a backward n-gram is repeatedly more successful and yields more grammatical results than a forward n-gram. This paper hypothesizes that the rationale behind this success is the syntactic ordering of constituents in Bangla. Bangla is a head-final specifier-initial language as opposed to English, which is head-initial specifier-initial. Hence in Bangla, the head comes after its argument in a phrase. If an n-gram analysis begins with a head and moves backwards it will stretch to its own argument but if you move for-wards then you'll probably grab the argument of an-other head. As probability of occurrence of heads is higher, probability of depending on a head is also higher and hence a backward n-gram will probably have a greater chance of yielding grammatical results. We carried out several experiments to compare different directional results in different applications with an advantage in the backward direction. This will prove a useful linguistic insight in terms of n-gram based analysis depending upon variations of constituent analysis.
Analysis of N-Gram based text categorization for Bangla in a newspaper
(BRAC University, 2006) Mansur, Munirul; UzZaman, Naushad; Khan, Mumit
In this paper, we study the outcome of using ngram based algorithm for Bangla text categorization. To analyze the efficiency of this methodology we used one year Prothom-Alo news corpus. Our results show that n-grams of length 2 or 3 are the most useful for categorization. Using gram lengths more than 3reduces the performance of categorization.
Minimally segmenting performance Bangla optical character recognition using Kohonen network
(BRAC University, 2006) Shatil, Adnan Mohammad Shoeb; Khan, Mumit
This paper presents a method to use Kohonen neural network based classifier in Bangla Optical Character Recognition (OCR) system, providing much higher performance than the traditional neural network based ones. It describes how Bangla characters are processed, trained and then recognized with the use of a Kohonen network. While there have been significant efforts in using the various types of Artificial ,eural ,etworks (A,,) in optical character recognition, this is the first published account of using a segmentation-free optical character recognition system for Bangla using a Kohonen network. The methodology presented here assumes that the OCR pre-processor has minimally segmented the input words into easily segmentable chunks, and presenting each of these as images to the classification engine described here. The size and the font face used to render the characters are also significant in both training and classification. The images are first converted into grayscale and then to binary images; these images are then scaled to a fit a pre-determined area with a fixed but significant number of pixels. The feature vectors are then extracted from the rectangular pixel map, which in this case is simply a series of 0s and 1s of fixed length. Finally, a Kohonen neural network is chosen for the training and classification process. Although the steps are simple, and the simplest network is chosen for the training and recognition process, the resulting classifier is accurate to better than 98%, depending on the quality of the input images.
Bangla text input and rendering supports for short message sevice on Mobile devices
(BRAC University, 2006) Rownok, Tofazzal; Islam, Md. Zahurul; Khan, Mumit
Technology is the most important thing that involve in our everyday life. It is involving in almost every aspect of life like communication, work, shopping, recreation etc. Communication through mobile devices is the most effective and easy way now a day. It is faster, easier and you can communicate whenever you want from any-where. Mobile messaging or short message service is one of the popular ways to communicate using mobile devices. It is a big challenge to write and display Bangla characters on mobile devices. In this paper, we describe a Bangla text input method and rendering support on mobile devices for short message service.
Rule based automated pronunciation generator
(BRAC University, 2006) Mosaddeque, Ayesha Binte; UzZaman, Naushad; Khan, Mumit
This paper presents a rule based ronunciation generator for Bangla words. It takes a word and finds the pronunciations for the graphemes of the word. A grapheme is a unit in writing that cannot be analyzed into smaller components. Resolving the pronunciation of a polyphone grapheme (i.e. a grapheme that generates more than one phoneme) is the major hurdle that the Automated Pronunciation Generator (APG) encounters. Bangla is partially phonetic in nature, thus we can define rules to handle most of the cases. Besides, up till now we lack a balanced corpus which could be used for a statistical pronunciation generator. As a result, for the time being a rule-based approach towards implementing the APG for Bangla turns out to be efficient.