Date of Award
12-3-2024
Document Type
Thesis
Degree Name
Master of Science (MS)
Department
Mathematics and Statistics
First Advisor
Wei Zhang
Abstract
The recent advancements in large language models (LLMs), such as ChatGPT, offer promising potential for automating academic writing tasks. This study investigates the efficacy of prompt engineering techniques in guiding LLMs to generate high-quality research survey papers, specifically exploring their application in the fields of artificial intelligence (AI) in drug development and large language model customization for classification. Using a mixed-methods approach, this research assessed the impact of prompt engineering on the coherence, relevance, and overall quality of generated content through quantitative and qualitative analyses. Techniques including prompt refinement, contextual prompting, and iterative prompting were systematically applied to enhance the quality of generated outputs. The findings reveal that prompt engineering significantly improves the academic quality of LLM-generated papers, as evidenced by consistent high median scores for coherence, relevance, and coverage across evaluations by AI models simulating graduate students and professors. Mann-Whitney U tests, selected due to non-normal distribution of scores, showed no significant differences in median scores across key criteria between the two papers, supporting the robustness of prompt engineering techniques. Qualitative feedback further highlighted the clarity and logical flow of the generated papers, underscoring the effectiveness of prompt engineering in producing structurally coherent and contextually relevant content. Nonetheless, challenges such as hallucination—where LLMs generate unsupported or inaccurate information—indicate areas needing further research. Future studies should focus on developing advanced prompting techniques, integrating retrieval-augmented generation to reduce hallucinations, and expanding research across broader academic disciplines and human-AI collaborative frameworks. This study contributes valuable insights into optimizing LLMs for academic writing, offering practical implications for researchers, educators, and developers aiming to leverage AI in the production of scholarly content.
Recommended Citation
Awidi, Innocent Obed, "Investigating the Efficacy of Prompt Engineering Techniques for Research Survey Paper Generation Using Large Language Models" (2024). Theses and Dissertations. 1234.
https://research.ualr.edu/etd/1234
