ADAB: A Culturally-Aligned Automated Response Generation Framework for Islamic App Reviews by Integrating ABSA and Hybrid RAG
| dc.contributor.author | Faruk, K.M.Tahlil Mahfuz | |
| dc.contributor.author | Talha, Mushfiqur Rahman | |
| dc.contributor.author | Ahamad, H.M.Kawsar | |
| dc.date.accessioned | 2026-06-24T06:36:16Z | |
| dc.date.available | 2026-07-04T15:45:14Z | |
| dc.date.issued | 2025-10-25 | |
| dc.description | Supervised by Mr. Syed Rifat Raiyan, Lecturer, Dr. KamrulHasan, Professor, Dr. Hasan Mahmud, Professor, Department of Computer Science and Engineering (CSE) Islamic University of Technology (IUT) Board Bazar, Gazipur, Bangladesh This thesis is submitted in partial fulfillment of the requirement for the degree of Bachelor of Science in Software Engineering, 2025 | |
| dc.description.abstract | Automated review response systems have advanced considerably, yet most fail to in corporate Islamic etiquette, values, and cultural norms, essential for meaningful en gagement with Muslim users. Prior research has shown that timely and thoughtful engagement with user reviews can improve user perception. However, managing re sponses at scale remains a significant challenge for developers, particularly when cul tural and religious considerations must be upheld. This research proposes ADAB, a framework forgenerating review responses that are culturally congruent with Islamic application contexts. The approach integrates a hybrid Retrieval-Augmented Gen eration (RAG) pipeline that employs agentic chunking and FAISS HNSW indexing to preserve contexts, with aspect-based sentiment analysis (ABSA) for fine-grained understanding of user feedback, and etiquette-aware prompt engineering to imbue responses with appropriate Islamic decorum. We also introduce a new open-source dataset of Islamic app reviews that supports the system’sdevelopmentandevaluation. Direct pairwise comparisons showed that ADAB’s responses were preferred in 40% of cases, compared to 15.3% for the baseline, with 44.7% ties. On average, our approach achieves an overall improvement of 9.9%, with the largest gain in application speci ficity (+30.39%). Wilcoxon signed-rank test confirmed significant improvements in accuracy (𝑝 = 0.0004), relevancy (𝑝 = 0.0417), and specificity (𝑝 = 8 × 10−9), while grammaticalcorrectnessshowed negligiblechange(𝑝 = 0.453). Theseresultsdemon strate that embedding cultural alignment in AI systems can foster trust and empathy, charting a path toward more respectful and human-centered response generation. | |
| dc.identifier.citation | [1] S.Agarwaletal.,“Gpt-oss-120b&gpt-oss-20bmodelcard,”arXivpreprintarXiv:2508.10925, 2025. [2] M.A.AlAsad, H. M.Imran, M. Alamin, T. A. Abdullah, and S. I. Chowdhury, “Sentiment and interest detection in social media using gpt-based large lan guage models,” in Proceedings of the 2023 6th International Conference on Ma chine Learning and Natural Language Processing, 2023, pp. 209–214. [3] M.Albuquerque, L. Barbosa, J. Moreira, A. da Silva, and T. Melo, “Fine-tuning open-source large language models for automated response to customer feed back,” in Symposium on Knowledge Discovery, Mining and Learning (KDMiLe), SBC, 2024, pp. 65–72. [4] N.Alturayeif, H. Aljamaan, andJ. Hassine, “Anautomatedapproachtoaspect based sentiment analysis of apps reviews using machine and deep learning,” Automated Software Engineering, vol. 30, no. 2, p. 30, 2023. [5] X.Amatriain,“Promptdesignandengineering:Introductionandadvancedmeth ods,” arXiv preprint arXiv:2401.14423, 2024. [6] N. Arabzadeh, X. Yan, and C. L. Clarke, “Predicting efficiency/effectiveness trade-offs for dense vs. sparse retrieval strategy selection,” in Proceedings of the 30th ACM International Conference on Information & Knowledge Management, 2021, pp. 2862–2866. [7] G. Azov, T. Pelc, A. F. Alon, and G. Kamhi, “Self-improving customer review response generation based on llms,” arXiv preprint arXiv:2405.03845, 2024. [8] A. Bhaskar, A. R. Fabbri, and G. Durrett, “Prompted opinion summarization with gpt-3.5,” arXiv preprint arXiv:2211.15914, 2022. [9] Y. CaoandF. H.Fard, “Pre-trained neural language models for automatic mo bile app user feedback answer generation,” in 2021 36th IEEE/ACM Interna tionalConferenceonAutomatedSoftwareEngineeringWorkshops(ASEW),IEEE, 2021, pp. 120–125. 51 [10] T.Chenetal.,“Densexretrieval: Whatretrieval granularity should we use?” In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024, pp. 15159–15177. [11] X. Chen, X. Chen, B. He, T. Wen, and L. Sun, “Analyze, generate and refine: Query expansion with llms for zero-shot open-domain qa,” in Findings of the Association for Computational Linguistics ACL 2024, 2024, pp. 11908–11922. [12] H.Déjean,S.Clinchant,andT.Formal,“Athoroughcomparisonofcross-encoders and llms for reranking splade,” arXiv preprint arXiv:2403.10407, 2024. [13] L.DiQuilioandF.Fioravanti,“Acomprehensiveframeworkforaspect-category sentimentanalysis,”inProceedingsoftheEighthWorkshoponNaturalLanguage for Artificial Intelligence(NL4AI2024)co-locatedwith23thInternationalConfer ence of the Italian Association for Artificial Intelligence (AI* IA 2024), CEUR-WS. org, 2024. [14] M.Douzeetal., “The faiss library,” arXiv preprint arXiv:2401.08281, 2024. [15] S.Ganesh,A.Purwar,etal., “Context-augmented retrieval: A novel framework for fast information retrieval based response generation using large language model,” arXiv preprint arXiv:2406.16383, 2024. [16] C. Gao, W. Zhou, X. Xia, D. Lo, Q. Xie, and M. R. Lyu, “Automating app re view response generation based on contextual knowledge,” ACM Transactions on Software Engineering and Methodology (TOSEM), vol. 31, no. 1, pp. 1–36, 2021. [17] Y. Gao et al., “Retrieval-augmented generation for large language models: A survey,” arXiv preprint arXiv:2312.10997, vol. 2, no. 1, 2023. [18] G. Greenheld, B. T. R. Savarimuthu, and S. A. Licorish, “Automating develop ers’ responses to app reviews,” in 2018 25th Australasian software engineering conference (ASWEC), IEEE, 2018, pp. 66–70. [19] J.Guetal.,“Asurveyonllm-as-a-judge,”arXivpreprintarXiv:2411.15594,2024. [20] S. Hassan, C. Tantithamthavorn, C.-P. Bezemer, and A. E. Hassan, “Studying the dialogue betweenusersanddevelopers of freeapps inthegoogle play store,” Empirical Software Engineering, vol. 23, pp. 1275–1312, 2018. [21] H.-L. Hsu and J. Tzeng, “Dat: Dynamic alpha tuning for hybrid retrieval in retrieval-augmented generation,” arXiv preprint arXiv:2503.23013, 2025. [22] R.Jagerman,H.Zhuang,Z.Qin,X.Wang,andM.Bendersky,“Queryexpansion by prompting large language models,” arXiv preprint arXiv:2305.03653, 2023. 52 [23] D.Jayakodyetal.,“Aspect-basedsentimentanalysistechniques:Acomparative study,” in 2024 Moratuwa Engineering Research Conference (MERCon), IEEE, 2024, pp. 205–210. [24] V.Karpukhinetal.,“Densepassageretrievalforopen-domainquestionanswer ing.,” in EMNLP (1), 2020, pp. 6769–6781. [25] T.KewandM.Volk,“Improvingspecificityinreviewresponsegeneration with data-driven data filtering,” in Proceedings of the Fifth Workshop on e-Commerce and NLP(ECNLP5), 2022, pp. 121–133. [26] T.S.Kim,Y.Lee,J.Shin,Y.-H.Kim,andJ.Kim,“Evallm:Interactiveevaluation of large language model prompts on user-definedcriteria,” in Proceedingsofthe 2024 CHI Conference on Human Factors in Computing Systems, 2024, pp. 1–21. [27] A. Kong et al., “Better zero-shot reasoning with role-play prompting,” arXiv preprint arXiv:2308.07702, 2023. [28] M.-C. Lee et al., “Hybgrag: Hybrid retrieval-augmented generation on textual and relational knowledge bases,” arXiv preprint arXiv:2412.16311, 2024. [29] X. H. Lù, “Bm25s: Orders of magnitude faster lexical search via eager sparse scoring,” arXiv preprint arXiv:2407.03618, 2024. [30] Y. A. Mustofa and I. S. K. Idris, “Ensemble approach to sentiment analysis of google play store app reviews,” Jambura Journal of Electrical and Electronics Engineering, vol. 6, no. 2, pp. 181–188, 2024. [31] T.Nguyenetal.,“Msmarco:Ahuman-generatedmachinereadingcomprehen sion dataset,” 2016. [32] G. Rosa et al., “In defense of cross-encoders for zero-shot retrieval, december 2022,” URL https://arxiv. org/abs/2212.06121, 2022. [33] P. Sahoo, A. K. Singh, S. Saha, V. Jain, S. Mondal, and A. Chadha, “A system atic survey of prompt engineering in large language models: Techniques and applications,” arXiv preprint arXiv:2402.07927, 2024. [34] S. Setty, H. Thakkar, A. Lee, E. Chung, and N. Vidra, “Improving retrieval for rag based question answering models on financial documents,” arXiv preprint arXiv:2404.07221, 2024. [35] K. Song, X. Tan, T. Qin, J. Lu, and T.-Y. Liu, “Mpnet: Masked and permuted pre-training for language understanding,” Advances in neural information pro cessing systems, vol. 33, pp. 16857–16867, 2020. 53 [36] N.Thakur,N.Reimers,A.R¨ "ucklé,A.Srivastava,andI.Gurevych,“Beir:Ahet erogenous benchmark for zero-shot evaluation of information retrieval mod els,” arXiv preprint arXiv:2104.08663, 2021. [37] L. Wang, N. Yang, and F. Wei, “Query2doc: Query expansion with large lan guage models,” arXiv preprint arXiv:2303.07678, 2023. [38] Y. Wang, G. Dai, S. Ke, and C. Zheng, “Evaluating sparse and dense retrieval in retrieval-augmented generation systems: A study,” in Proceedings of the 2024 10th International Conference on Communication and Information Processing, 2024, pp. 548–554. [39] M.Wankhade, A. C. S. Rao, and C. Kulkarni, “A survey on sentiment analysis methods, applications, and challenges,” Artificial Intelligence Review, vol. 55, no. 7, pp. 5731–5780, 2022. [40] H. Yang, C. Zhang, and K. Li, “Pyabsa: A modularized framework for repro ducible aspect-based sentiment analysis,” in Proceedings of the 32nd ACMinter nationalconferenceoninformationandknowledgemanagement,2023,pp.5117 5122. [41] H. Zhang et al., “Efficient and effective retrieval of dense-sparse hybrid vec tors using graph-based approximate nearest neighbor search,” arXiv preprint arXiv:2410.20381, 2024. [42] W. Zhang, W. Gu, C. Gao, and M. R. Lyu, “A transformer-based approach for improving app review response generation,” Software: Practice and Experience, vol. 53, no. 2, pp. 438–454, 2023. [43] W.Zhang,Y.Deng,B.Liu,S.J. Pan, and L. Bing, “Sentiment analysis in the era oflargelanguagemodels:Arealitycheck;2023,”URLhttps://arxiv.org/abs/2305.15005, [44] Y.Zhuetal.,“Largelanguagemodelsforinformationretrieval:Asurvey,”arXiv preprint arXiv:2308.07107, 2023 | |
| dc.identifier.other | https://repository.iutoic-dhaka.edu/server/api/core/items/9ca2b07b-36f7-4c61-9cbd-a3cea449e263 | |
| dc.identifier.uri | https://repository.iutoic-dhaka.edu/handle/123456789/2627 | |
| dc.language.iso | en | |
| dc.publisher | Department of Computer Science and Engineering(CSE), Islamic University of Technology(IUT), Board Bazar, Gazipur-1704, Bangladesh | |
| dc.source | IUT Institutional Repository | |
| dc.title | ADAB: A Culturally-Aligned Automated Response Generation Framework for Islamic App Reviews by Integrating ABSA and Hybrid RAG | |
| dc.type | Thesis |
Files
Original bundle
1 - 1 of 1
- Name:
- 38 Fulltext_ CSE_ADABACulturally-Aligned Automated Response Generation Framework_.pdf
- Size:
- 2.84 MB
- Format:
- Adobe Portable Document Format
