Citation fabrication is contingent on informational terrain rather than a fixed model property: Linardon et al. 2025 show fabrication and error rates fall for topics with greater public awareness, an established evidence base, and broad scientific consensus—adding a within-field refinement to the 47–69% fabrication rates Walters & Wilder 2023 document across earlier studies.
E34 supports the contingency finding (fabrication and error rates less frequent for topics with greater public awareness, an established evidence base, and broad scientific consensus) and E10 supports the 47–69% fabrication range across earlier studies, but neither cited text states that Linardon's analysis was a 'within-field' design.
Written by Kimi K3 via Ollama Cloud · checked by GLM-5.3 via Ollama Cloud · 24 Oct, 04:56
Source chain
AEvery quote below was checked, without a model, to appear verbatim in its source.
- 01
“the reliability of LLM-generated citations is not fixed but contingent on the informational terrain they are asked to navigate”
Full-text passage · no page number · evidence E34
As these tools become more deeply integrated into research workflows [5], it is essential to equip users with the knowledge and skills to critically assess model-generated outputs. This includes instruction on how to identify hallucinated citations, validate bibliographic content, and appropriately disclose the use of generative artificial intelligence in scholarly outputs. Conclusions In conclusion, this research offers novel insights into how the characteristics of a topic influence the reliability of citations produced by LLMs. While we found that citation fabrication and error rates are common overall, these inaccuracies were less frequent when prompts addressed topics with greater public awareness, an established evidence base, and a broad scientific consensus. This suggests that the reliability of LLM-generated citations is not fixed but contingent on the informational terrain they are asked to navigate. Importantly, these findings have broader implications for the integration of LLMs into scholarly workflows. They indicate that improving citation accuracy is not solely a technical challenge but also a matter of strategic prompt design and topic selection.
Passage read from www.ebi.ac.uk, which may be a preprint rather than the published version.
- 02
“the proportion of fabricated citations is typically in the 47–69% range, with a higher rate in geography than in medicine”
Full-text passage · no page number · evidence E10
Third, an understanding of fabricated citations and errors can help AI developers improve the software and the methods used for development and testing. As we suggest in the “Discussion” section, citations are a special type of text for which predictive word choice, paraphrasing, and related techniques may be detrimental rather than useful. For instance, the replacement of a title word with a more common, or even more appropriate, synonym reduces the value of the title as a search string and as a unique identifier of a particular cited work. Previous research Several authors have noted that ChatGPT tends to cite works that do not actually exist12,16–25, and 6 studies have systematically investigated the extent of the problem13,15,26–29. For instance, Wagner and Ertl-Wagner15 used GPT-3.5 to answer 88 questions in 8 subspecialty areas of radiology, then evaluated the citations included in those responses. Of the 343 citations, 64% were fabricated (i.e., could not be found in PubMed or on the open web). As Table 1 shows, the proportion of fabricated citations is typically in the 47–69% range, with a higher rate in geography than in medicine.
Passage read from www.ebi.ac.uk, which may be a preprint rather than the published version.