Bangla Dataset Generation for Natural Language Inference

Loading PDF preview...

Date

2023-05-30

Journal Title

Journal ISSN

Volume Title

Publisher

Department of Computer Science and Engineering(CSE), Islamic University of Technology(IUT), Board Bazar, Gazipur-1704, Bangladesh

Abstract

Understanding entailment and contradiction is fundamental to understanding nat ural language, and inference about entailment and contradiction is a valuable test ing ground for the development of semantic representations. However, machine learning research in this area has been dramatically limited by the lack of resources in Bangla. To address this, we propose to introduce our own corpus curated for natural language inference which is labeled pairs of sentences with a label that depicts their inner entailment. Our goal is to create a dataset that has over 30K instances and to do so we have now created a Bangla dataset by machine trans lating the SNLI corpus into Bangla. After that, we show that benchmark models can be used to evaluate and do the task of inference in Bangla . We hope that our dataset will catalyze research in Bangla sentence understanding by providing an informative standard evaluation task.For this we provided two baseline models which are both considered integral in the task of inference in any langauge.

Description

Supervised by Dr. Hasan mahmud, Associate Professor, Prof. Dr. Kamrul Hasan, Department of Computer Science and Engineering(CSE), Islamic University of Technology(IUT), Board Bazar, Gazipur-1704, Bangladesh Board Bazar, Gazipur, Bangladesh

Keywords

entailment, contradiction, neutral, natural language, inference, seman tic representations, machine learning, Bangla, corpus, labeled pairs of sentences, inner entailment, dataset, instances, SNLI corpus, machine translation, benchmark models, evaluation task, baseline models, sentence understanding

Citation

Collections

Endorsement

Review

Supplemented By

Referenced By