Pre-Processing the Prompt to generate an efficient Output for Unit Test Generation
Date
2025-10-25
Journal Title
Journal ISSN
Volume Title
Publisher
Department of Computer Science and Engineering(CSE), Islamic University of Technology(IUT), Board Bazar, Gazipur-1704, Bangladesh
Abstract
Large Language Models (LLMs) have shown remarkable performance across a wide
range of natural language processing tasks, yet their effectiveness remains highly sen
sitive to prompt formulation. Manual prompt engineering is often labor-intensive,
inconsistent, and difficult to scale. This thesis investigates the landscape of auto
matedpromptoptimization,analyzingrecentadvancementssuchasnaturallanguage
gradient-based refinement, Bayesian Optimization for discrete prompt tuning, evolu
tionary strategies, and joint fine-tuning approaches. Through a comprehensive com
parison of these techniques, the study identifies key trends, limitations, and opportu
nities in existing methods. Building on these insights, we propose a novel framework
that aims to combine the interpretability of natural language-based refinements with
the efficiency of automated search strategies. Experimental evaluations demonstrate
improvedtaskperformanceandpromptrobustnessacrossmultipleLLMbenchmarks.
This research contributes a unified perspective on automated prompt optimization
and paves the way for more adaptive and accessible prompt engineering methodolo
gies.
Description
Supervised by
Ms. Jibon Naher,
Lecturer,
Department of Computer Science and Engineering (CSE)
Islamic University of Technology (IUT)
Board Bazar, Gazipur, Bangladesh
This thesis is submitted in partial fulfillment of the requirement for the degree of Bachelor of Science in Software Engineering, 2025
Keywords
Citation
[1] S.Gaoetal.,Thepromptalchemist:Automatedllm-tailoredpromptoptimization for test case generation, 2025. arXiv: 2501.01329 [cs.SE]. [Online]. Available: https://arxiv.org/abs/2501.01329 [2] R.Lingampally,A.Gupta,andP.Jalote,“Amultipurposecodecoveragetoolfor java,” in 2007 40th Annual Hawaii International Conference on System Sciences (HICSS’07), 2007, 261b–261b. doi: 10.1109/HICSS.2007.24 [3] M.Mosbach, T. Pimentel, S. Ravfogel, D. Klakow, and Y. Elazar, Few-shot fine tuning vs. in-context learning: A fair comparison and evaluation, 2023. arXiv: 2305.16938 [cs.CL]. [Online]. Available: https://arxiv.org/abs/2305. 16938 [4] R. Pryzant, D. Iter, J. Li, Y. T. Lee, C. Zhu, and M. Zeng, Automatic prompt op timization with "gradient descent" and beam search, 2023. arXiv: 2305.03495 [cs.CL]. [Online]. Available: https://arxiv.org/abs/2305.03495 [5] A.Sabbatella, A. Ponti, I. Giordani, A. Candelieri, and F. Archetti, “Prompt op timization in large language models,” Mathematics, vol. 12, p. 929, Mar. 2024. doi: 10.3390/math12060929 [6] D. Soylu, C. Potts, and O. Khattab, Fine-tuning and prompt optimization: Two great steps that work better together,2024.arXiv:2407.10930[cs.CL].[Online]. Available: https://arxiv.org/abs/2407.10930 [7] W.Sunetal.,Abstractsyntaxtreefor programminglanguageunderstandingand representation: How far are we? 2023. arXiv: 2312.00413 [cs.SE]. [Online]. Available: https://arxiv.org/abs/2312.00413 [8] B. Wang et al., Towards understanding chain-of-thought prompting: An empir ical study of what matters, 2023. arXiv: 2212.10001 [cs.CL]. [Online]. Avail able: https://arxiv.org/abs/2212.10001 [9] H.YuandJ.Liu,Deepinsights into automated optimization with large language models and evolutionary algorithms, 2024. arXiv: 2410.20848 [cs.NE]. [On line]. Available: https://arxiv.org/abs/2410.20848 49 [10] Q.Zhang,Y.Shang,C.Fang,S.Gu,J.Zhou,andZ.Chen,Testbench:Evaluating class-level test case generation capability of large language models, 2024. arXiv: 2409.17561 [cs.SE]. [Online]. Available: https://arxiv.org/abs/2409. 17561 [11] T. Zheng et al., The curse of cot: On the limitations of chain-of-thought in in context learning,2025.arXiv:2504.05081[cs.CL].[Online].Available:https: //arxiv.org/abs/2504.05081