Improving large language model applications in biomedicine with retrieval-augmented generation: a systematic review, meta-analysis, and clinical development guidelines

Checked 30 Sept 2026. Non-commercial or no-derivatives license: full text kept internally; only metadata and abstract are indexed and used.

OBJECTIVE: The objectives of this study are to synthesize findings from recent research of retrieval-augmented generation (RAG) and large language models (LLMs) in biomedicine and provide clinical development guidelines to improve effectiveness. MATERIALS AND METHODS: We conducted a systematic literature review and a meta-analysis. The report was created in adherence to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses 2020 analysis. Searches were performed in 3 databases (PubMed, Embase, PsycINFO) using terms related to "retrieval augmented generation" and "large language model," for articles published in 2023 and 2024. We selected studies that compared baseline LLM performance with RAG performance. We developed a random-effect meta-analysis model, using odds ratio as the effect size. RESULTS: Among 335 studies, 20 were included in this literature review. The pooled effect size was 1.35, with a 95% confidence interval of 1.19-1.53, indicating a statistically significant effect (P = .001). We reported clinical tasks, baseline LLMs, retrieval sources and strategies, as well as evaluation methods. DISCUSSION: Building on our literature review, we developed Guidelines for Unified Implementation and Development of Enhanced LLM Applications with RAG in Clinical Settings to inform clinical applications using RAG. CONCLUSION: Overall, RAG implementation showed a 1.35 odds ratio increase in performance compared to baseline LLMs. Future research should focus on (1) system-level enhancement: the combination of RAG and agent, (2) knowledge-level enhancement: deep integration of knowledge into LLM, and (3) integration-level enhancement: integrating RAG systems within electronic health records.

• Liu et al. 2025's systematic review and meta-analysis of 20 biomedical studies found that retrieval-augmented generation produces a statistically significant, though modest, performance improvement over baseline LLMs, with a pooled odds ratio of 1.35. Trace →
• The significant average benefit of RAG quantified by Liu et al. 2025 is consistent with domain-specific evaluations: Wan et al. 2025 report 77.8% exact-match accuracy for hybrid RAG in manufacturing QA, and Gaber et al. 2025 benchmarked an LLM workflow incorporating RAG on 2,000 MIMIC-derived medical cases for triage, referral, and diagnosis support. Trace →
• Liu et al. 2025 identify integrating RAG systems within electronic health records as a key future direction, which speaks directly to the persistent "last-mile" bottleneck in preparing research-grade clinical datasets that Chen et al. 2026 describe. Trace →

• RAG reliability and evaluation