Autonomous artificial intelligence, scientific research, and human values

Checked 30 Sept 2026. Open license (CC-BY, CC-BY-SA, CC0, public domain): full text indexed and used in synthesis.

During the initial stage of AI's incorporation into scientific research, AI systems have functioned predominantly as tools under direct human supervision and control. However, AI incorporation into scientific research is now entering a stage in which AI Agents perform research tasks with partial autonomy while remaining under human supervision and control. In the not-too-distant future, a third stage may arise when autonomous AI systems conduct their own research and generate knowledge without human supervision or control. While the second and third stages of AI-augmented research may offer substantial benefits for science and society, they also create novel ethical issues, including (1) Conducting immoral research that may harm humans and other forms of life; (2) Increasing rate of biased, erroneous and deceptive research; (3) Confidentiality challenges; (4) Overreliance on AI; (5) Diffusion of responsibility and accountability; (6) Deskilling; (7) Job losses; (8) AI-generated research beyond human comprehension; and (9) Erosion of trust. We suggest specific solutions to minimize the negative consequences of these issues and offer proposals for ensuring that incorporation of AI into scientific research supports human values.

• Both system builders and ethicists flag risks to the scientific literature itself: Lu et al. 2026 warn their own technology could tax overwhelmed review systems and add noise to the literature. Resnik et al. 2026 catalogue a broader set of ethical issues, including biased or deceptive research, overreliance on AI, and diffusion of responsibility. Trace →
• Authors across the bundle flag risks of research automation: Lu et al. 2026 warn of taxing overwhelmed review systems and adding noise to the literature, Li et al. 2025 highlight hallucinations that appear valid but are false, and Resnik et al. 2026 enumerate ethical issues including increasing rates of biased, erroneous and deceptive research. Trace →

• AI for science and agentic research