اخبار و اعلانات

 آمار

تعداد نشریات44
تعداد شماره‌ها1,916
تعداد مقالات15,590
تعداد مشاهده مقاله46,914,124
تعداد دریافت فایل اصل مقاله19,067,459
mohammadi ghahderijani, Amirhossein, Moeinian, Ali, Razavizadeh, Zahra, Fatemi, Afsaneh, Ramezani, Reza. (1405). Evaluating Prompt Engineering Techniques for RAG in Small Language Models: A Multi-Hop QA Approach. , (), -. doi: 10.22108/jcs.2026.147609.1186
Amirhossein mohammadi ghahderijani; Ali Moeinian; Zahra Razavizadeh; Afsaneh Fatemi; Reza Ramezani. "Evaluating Prompt Engineering Techniques for RAG in Small Language Models: A Multi-Hop QA Approach". , , , 1405, -. doi: 10.22108/jcs.2026.147609.1186
mohammadi ghahderijani, Amirhossein, Moeinian, Ali, Razavizadeh, Zahra, Fatemi, Afsaneh, Ramezani, Reza. (1405). 'Evaluating Prompt Engineering Techniques for RAG in Small Language Models: A Multi-Hop QA Approach', , (), pp. -. doi: 10.22108/jcs.2026.147609.1186
mohammadi ghahderijani, Amirhossein, Moeinian, Ali, Razavizadeh, Zahra, Fatemi, Afsaneh, Ramezani, Reza. Evaluating Prompt Engineering Techniques for RAG in Small Language Models: A Multi-Hop QA Approach. , 1405; (): -. doi: 10.22108/jcs.2026.147609.1186

Evaluating Prompt Engineering Techniques for RAG in Small Language Models: A Multi-Hop QA Approach

Journal of Computing and Security
مقالات آماده انتشار، پذیرفته شده، انتشار آنلاین از تاریخ 17 آبان 1405
نوع مقاله: Research Article
شناسه دیجیتال (DOI): 10.22108/jcs.2026.147609.1186
نویسندگان
Amirhossein mohammadi ghahderijani1؛ Ali Moeinian* 1؛ Zahra Razavizadeh2؛ Afsaneh Fatemi3؛ Reza Ramezani3
1Department of Software Engineering, University of Isfahan, Isfahan, Iran
2Department of Software Engineering University of Isfahan, Isfahan, Iran
3Department of Software Engineering, Faculty of Computer Engineering, University of Isfahan, Isfahan, Iran
چکیده
Retrieval-Augmented Generation (RAG) supports small language models (SLMs) with external evidence, yet selecting effective prompt structures for multi-hop question answering remains challenging. This study presents a comparative evaluation of 24 prompt configurations: one standard RAG baseline, nine literature-derived templates, and fourteen manually constructed hybrid templates organized through a compositional framework of decomposition, evidence grounding, role framing, synthesis, self-verification, and output structuring. We evaluate Qwen2.5-3B-Instruct and Gemma3-4B-It on the complete 390-question HotpotQA test split distributed through RAGBench, using its supplied retrieved contexts. The experiment comprises 18,720 judged generations (390 questions × 24 templates × 2 models). Evaluation combines a GPT-4o-mini-based multidimensional quality score (SLLM), measured generation-call latency, and a quality-to-latency Efficiency Score. The highest mean SLLM values are 0.8305 for Qwen and 0.8455 for Gemma, representing absolute gains of 0.0623 and 0.0602 over their respective baselines. SLLM measures composite answer quality rather than conventional question-answering accuracy. Paired Wilcoxon tests with within-model Holm correction identify a significant best-hybrid improvement for Gemma, while the corresponding Qwen comparison does not reach significance. Both highest-scoring hybrids incur additional latency relative to their baselines, and complementary analyses reveal model specific prompt rankings and response patterns. The results provide practical guidance for selecting prompts according to quality and latency requirements. These findings are established for the two evaluated checkpoints and the RAGBench HotpotQA setting; broader applicability across models, retrieval pipelines, datasets, and domains remains a direction for further validation.
کلیدواژه‌ها
Retrieval Augmented Generation؛ Small Language Models؛ Prompt Template؛ Multi-Hop Question Answering
آمار
تعداد مشاهده مقاله: 1


سامانه مدیریت نشریات علمی. طراحی و پیاده سازی از سیناوب