TRL : A Transformer Based Approach to Mental Health Counseling with Retrieval Augmented Generation and Low Rank Adaptation

Abstract

Mental health is a critical component of global well-being, yet access to timely and af fordable careremains limitedduetostigma,scarcityofprofessionals,andeconomicor geographic barriers. In Bangladesh, where prevalence rates of anxiety and depression among adolescents and university students are alarmingly high, these challenges are particularly severe. At the same time, advances in artificial intelligence (AI) and nat ural language processing (NLP) have created new opportunities to deliver accessible and scalable mental health support through conversational agents. This research investigates the development of an AI-powered counseling chatbot us ing textual transformers. Building on the strengths of large language models, we refine response generation through the integration of Retrieval-Augmented Gener ation (RAG) and Low-Rank Adaptation (LoRA). RAG enables dynamic access to rel evant external knowledge, while LoRA provides efficient fine-tuning, resulting in a lightweight yet effective model. Our approach emphasizes improving contextual ac curacy, mitigating hallucinations, and ensuring sensitivity to the nuanced nature of mental health conversations. The contributions of this work are threefold: (1) advancing transformer-based meth ods to generate precise, compassionate, and personalized counseling responses; (2) addressing key challenges such as dataset authenticity, preprocessing for sensitive text, and factual reliability; and (3) proposing a scalable framework for deploying mental health chatbots in resource-constrained contexts. Ultimately, this research aims to bridge the gap between individuals in need of psychological support and the care they are often unable to access, offering an ethical, efficient, and practical AI driven solution.