Authors across the bundle flag risks of research automation: Lu et al. 2026 warn of taxing overwhelmed review systems and adding noise to the literature, Li et al. 2025 highlight hallucinations that appear valid but are false, and Resnik et al. 2026 enumerate ethical issues including increasing rates of biased, erroneous and deceptive research.
E6 warns of risks including taxing overwhelmed review systems and adding noise to the scientific literature, E47 highlights hallucinations that appear valid but are actually false, and E27 enumerates ethical issues including increasing rates of biased, erroneous and deceptive research.
Written by Kimi K3 via Ollama Cloud · checked by GLM-5.3 via Ollama Cloud · 1 Oct, 04:32
Source chain
AEvery quote below was checked, without a model, to appear verbatim in its source.
- 01
“there could be important risks, including taxing overwhelmed review systems and adding noise to the scientific literature”
Full-text passage · no page number · evidence E6
Passage read from www.ebi.ac.uk, which may be a preprint rather than the published version.
- 02
“AI models that learn patterns from training data may generate “hallucinations” that appear valid but are actually false or physically impossible”
Full-text passage · p. 1 · evidence E47
Unlike prior technologies that amplified human capability within existing paradigms, AI is redefining the very steps of scientific inquiry - from scientific hypothesis generation to experimental validation - and breaking down barriers that have long stymied progress across disciplines. AI has emerged as a transformative tool in scientific research, widely recognized for its contributions to groundbreaking achievements in highly complex domain-specific tasks. Nevertheless, beneath these remarkable successes, systemic vulnerabilities exist that threaten the authenticity of AI-enabled scientific research. Several interconnected challenges are particularly prominent: Large Language Models, now extensively employed for mining data from millions of research papers, face difficulties in extracting reliable information; AI models that learn patterns from training data may generate “hallucinations” that appear valid but are actually false or physically impossible; and both issues are amplified by a persistent lack of highquality experimental data. Addressing these challenges is not merely a technical necessity, but also a safeguard for the integrity of scientific research.
Passage read from f.oaes.cc, which may be a preprint rather than the published version.
- 03
“Increasing rate of biased, erroneous and deceptive research”
Full-text passage · no page number · evidence E27
Passage read from www.ebi.ac.uk, which may be a preprint rather than the published version.