General LLMs display uneven domain expertise outside mainstream biomedicine: Minasny et al. 2026 report they answered only up to 65% of advanced soil science exam questions correctly, and Burda et al. 2025 document 'plant blindness' and limited taxonomic diversity in botanical answers.
Minasny reports that LLMs could correctly answer only up to 65% of questions from advanced soil science examinations, and Burda documents that ChatGPT's outputs exhibit 'plant blindness' and limited taxonomic diversity when answering botanical questions.
Written by Kimi K3 via Ollama Cloud · checked by GLM-5.3 via Ollama Cloud · 4 Oct, 04:34
Source chain
AEvery quote below was checked, without a model, to appear verbatim in its source.
- 01
“on average, they could correctly answer only up to 65% of questions from advanced soil science examinations”
Full-text passage · p. 3 · evidence E28
While they demonstrate promising capabilities for conversational interaction, ensuring their reliability and depth of understanding in specialized soil science domains still requires signifi cant human validation and expertise. For example, Khanifar ( 34) assessed the performance of LLMs in answering soil science-related questions and found that, on average, they could correctly answer only up to 65% of questions from advanced soil science examinations. This existing foundation, with its strengths and weaknesses, sets the stage for the development of next-generation AI agents capable of more integrated, autonomou s, and collaborative scienti fic exploration and management in soil science. The vision of AI agents transforming scienti fic discovery is gaining momentum in fields such as chemistry, materials science, and biomedicine, where complex systems, vast datasets, and multidisciplinary knowledge must be integrated ( 5, 35, 36). The concept of an “AI scientist” or “co-scientist”— an AI system capable of skeptical learning, reasoning, and collaboration— has been proposed as a framework for generating novel scienti fich y p o t h e s e s aligned with researcher objectives ( 8, 37, 38).
Passage read from www.frontiersin.org, which may be a preprint rather than the published version.
- 02
“ChatGPT's outputs exhibit "plant blindness" and limited taxonomic diversity when answering botanical questions”
Full-text passage · p. 2 · evidence E69
The few existing studies which survey the use of AI in research environments, underscore both excitement and caution, highlighting how researchers leverage chat assistants to accelerate literature review and experiment planning, yet worry about uncontrolled hallucinations and biorisk implications7. Meanwhile, frontier systems demonstrate that agentic LLMs can achieve or exceed expert performance on literature-search and contradiction-detection tasks2, 8, 9, suggesting that with the right evaluation frameworks, models could reliably support scientific discovery. In agriculture there have been approaches to leverage farmer queries through LLM-powered systems10. For genomics, initial studies reveal feasibility but also highlight persistent domain-specific challenges: for example, ChatGPT’s outputs exhibit "plant blindness" and limited taxonomic diversity when answering botanical questions11; and LLMs, while extracting ecological data more than 50 times faster than human reviewers, require rigorous quality-control to avoid quantitative errors2.
Passage read from www.biorxiv.org, which may be a preprint rather than the published version.