Artificial intelligence and illusions of understanding in scientific research

Checked 30 Sept 2026. Closed or unknown license: metadata, DOI and short own annotations only.

Not shown: this source does not allow its abstract to be redistributed.

Highlights and notes from the curator's Zotero library. On papers that are not openly licensed, quotes are kept short.

• “taxonomy of scientists’ visions for AI, observing that their appeal comes from promises to improve productivity and objectivity by overcoming human shortcomings”

  p. 49

• “The proliferation of AI tools in science risks introducing a phase of scientific enquiry in which we produce more but understand less”

  Note: This is one of the comments I hear most often when speaking to colleagues about use of AI tools.

  p. 49

• “Philosophical questions • What is the nature of knowledge in predictive and generative AI? How does this knowledge differ from human knowledge? • Could AI systems be capable of understanding, and if so, how would it differ from human understanding? What kinds of epistemic dependence arise in human–…”trimmed

  Note: These questions would work as an excellent viewpoint when starting a new project or task. Do we have any new insights, thoughts, or ideas on these topics. Use as refelction material to get further.

  p. 55

• Beyond accuracy, several authors warn of an epistemic cost. Messeri & Crockett 2024 caution that AI may lead science to produce more but understand less. Bietti & Bangerter 2026 argue that outsourcing writing risks decoupling it from thinking and eroding core reasoning skills. Trace →
• Messeri & Crockett 2024 and Bietti & Bangerter 2026 raise complementary epistemic alarms about AI in science: the former warn of a phase of enquiry in which we produce more but understand less, while the latter argue that outsourcing writing to LLMs risks decoupling writing from thinking, since scientific writing is itself a form of thinking. Trace →
• Messeri & Crockett 2024 warn that the proliferation of AI tools in science risks a phase of scientific enquiry in which we produce more but understand less, anticipating Bietti & Bangerter 2026's contention that scientific writing is not merely communication but a form of thinking. Trace →
• Two perspective pieces converge on the epistemic stakes of AI-assisted science: Messeri & Crockett 2024 warn that proliferating AI tools risk a phase of enquiry in which we produce more but understand less, while Bietti & Bangerter 2026 argue that outsourcing writing to LLMs risks decoupling writing from thinking. Trace →
• Bietti & Bangerter 2026 and Messeri & Crockett 2024 articulate a shared epistemic worry: because scientific writing is itself a form of thinking, outsourcing it to LLMs risks a mode of science that produces more text while understanding less. Trace →
• Two commentaries converge on an epistemic risk of LLM adoption in science: Messeri & Crockett 2024 warn of a phase in which we produce more but understand less, and Bietti & Bangerter 2026 argue that outsourcing writing to LLMs risks decoupling writing from thinking. Trace →
• Both Binz et al. 2025 and Walters & Wilder 2023 caution that verification costs may erode LLM efficiency gains: the trust routinely placed in ordinary software is inappropriate for generative AI, and time saved in text generation may be offset by the time required to verify the output. Trace →

• Research integrity and LLMsanchor