Field Kit Science · AI in research

Every claim traces backto its page.

What generative and agentic AI is doing in research, what it does to the evidence base, and one way of working with it: a corpus that grows every night behind a license gate, read by language models that must show their sources. Not a product to install: a way of working, built from open parts.

What is moving ↓See the building blocks ↓Present as slides ▶Open the live corpus →
Claim 18Supported

Beyond accuracy, several authors warn of an epistemic cost.

↳ Quote · p. 49 · curator's highlight

“The proliferation of AI tools in science risks introducing a phase of scientific enquiry in which we produce more but understand less”

↳ Paper
Messeri 2024Reddoi:10.1038/s41586-024-07146-0
The problem

Language models are already in the literature.Almost nobody says so.

These figures come from papers in this corpus. Each links to the claim that carries it, and from there to the sentence it was read from.

13.5%

of 2024 PubMed abstracts were, at a lower bound, processed with an LLM

Kobak 2025 · trace →

40%

the same lower bound in some disciplines, countries and journals

Kobak 2025 · trace →

~0.1%

of 75,000 papers published since 2023 disclosed any AI use

He 2026 · trace →

The trouble is not that researchers use these tools. It is that a fluent summary carries no address. You cannot tell which sentence in which paper it rests on, whether you were allowed to read that paper, or whether it has since been retracted.

Why it is possible now

Open metadata

OpenAlex (CC0), Crossref, Unpaywall and Europe PMC say, per paper and per version, where a legal copy is and under which license.

Retractions in the open

Since 2023 the Retraction Watch database is openly available through Crossref, so checking every paper every night costs nothing.

Models that can cite

Current models can write against numbered evidence and be checked by a second model. Agent protocols let you do it from a conversation.

What is moving

Generative and agentic AI in research,in three layers.

A map, not a ranking. Every paper here sits in this corpus behind the same license gate. Notes are the curator's reading. Classes come live from the license gate: on green papers a note can be checked against the full text indexed here, on the others it rests on the abstract.

Models of the language of life

Foundation models trained on sequences, structures, images and sound from across the tree of life. In evolution and biodiversity research, generative AI is already part of the method.

Agents that do research

Not a chatbot that answers, but systems that plan, call tools, run experiments and write up. Almost all of this is from the last three years.

What breaks

What it does to the evidence base.

Fabricated references, summaries that overreach, text nobody disclosed, and papers retracted for what a model wrote. The figures in the problem section come from the same bundle.

The idea

Don't trust the summary.Trace it.

Treat verification as a way of working, not a tool. The system reads at night; you judge in the morning, and every judgement has somewhere to stand.

01

Gate

Decide what you may do with a text before you store it. Unknown means closed.

02

Trace

Every claim points to a quote, a page and a DOI. A claim without a source cannot be saved.

03

Check

A second model judges each claim. A plain string match confirms every quote is really there.

04

Remember

Know when the system is saying the same thing again in new words.

Building blocks

What runs every night.

Nine parts, on open data and ordinary infrastructure. None is clever on its own. The discipline is in how they connect.

  1. 01 · any time Live

    One way in

    Zotero, Claude, the command line and the nightly search all do the same thing: put an identifier in a queue. One worker does the rest, so the rules apply in one place.

  2. 02 · on intake · 03:30 Live

    License gate

    Asks every source what a paper's license is. Green is stored and indexed, yellow stored but not indexed, orange and red kept as metadata only. Unknown counts as red; sources that disagree get the strictest class and a flag for a human.

  3. 03 · on intake · 03:30 Live

    Retraction check

    Crossref and Retraction Watch, on arrival and again every night. A retracted paper stays in the corpus, marked, and never reaches synthesis.

  4. 04 · on intake · hourly Live

    Full text with pages

    Structured text from Europe PMC, otherwise the PDF with its page breaks kept, so a quote can name its page. Open text is chunked and embedded with bge-m3 on our own server.

  5. 05 · 02:15 · 03:00 Live

    Discovery

    Each bundle starts from a few anchor papers. Every night OpenAlex is asked who cites them. Candidates close to the bundle's centre are queued, and no single anchor may bring in more than its share. A second lane looks only at the last 30 days, so new work gets in before anyone has cited it: about ten papers a week across the bundles.

  6. 06 · every 5 min Live

    The curator's reading

    Highlights and comments in a shared Zotero group become evidence with a page number. A short highlight and the curator's own comment can carry a closed paper where its text cannot.

  7. 07 · 04:30 Live

    Synthesis and verification

    One model reads a bundle's evidence and writes claims that cite numbered evidence. A different model judges each claim against its sources, and a string match with no model at all confirms each quote is in the text.

  8. 08 · after each run Live

    Claim memory

    Each claim is tied to the first claim that said the same thing, and marked new, repeat, or extended by a paper the line had not cited. The writing model is shown the standing claims and asked for what is new.

  9. 09 · on request Live

    Agent access

    An MCP server lets Claude add a paper, trace a claim or run a synthesis from a conversation. The public profile and this site can only read.

  10. 10 · planned Next

    The night's diff

    What is new, strengthened, qualified or unchanged since yesterday, and each claim's history: first seen on night 22, strengthened on night 25 by a new paper.

Last night

The same parts, in numbers.

Live from this instance, read as you load the page.

Papers
389
8 active bundles
Claims checked
524
429 supported
License conflicts
23
sent to a human
Retracted
2
kept, never synthesised
  • Green
    214 · 55%
  • Yellow
    86 · 22%
  • Orange
    21 · 5%
  • Red
    68 · 17%

Hover a class to see what the gate allows for it.

Latest synthesis: Kimi K3 writes, GLM-5.3 checks, via Ollama Cloud.

One claim, traced

One claim,all the way down.

Hand-picked, because it shows most of the parts at once: a closed paper and an open one, the curator's own reading, and a model that kept saying it.

Claim 18 · research-integrity · 30 Sept 2026

Beyond accuracy, several authors warn of an epistemic cost. Messeri & Crockett 2024 caution that AI may lead science to produce more but understand less. Bietti & Bangerter 2026 argue that outsourcing writing risks decoupling it from thinking and eroding core reasoning skills.

SupportedWritten by Claude Opus 5.5

Verifier: E8 contains the 'produce more but understand less' warning, and E15 frames writing as thinking and says outsourcing threatens core reasoning skills.

  1. E8
    Annotation · p. 49
    Red

    A closed paper. The system holds none of its text: this is the curator's own highlight from Zotero, with page and comment.

    “The proliferation of AI tools in science risks introducing a phase of scientific enquiry in which we produce more but understand less”

    Curator: This is one of the comments I hear most often when speaking to colleagues about use of AI tools.

    Messeri 2024: Artificial intelligence and illusions of understanding in scientific researchDOI ↗
  2. E15
    Full text
    Green

    An open paper, read from its full text, indexed on our own server. This text carries no page numbers.

    “it threatens the development and exercise of core cognitive skills involved in scientific reasoning”

    Bietti 2026: Will the widespread use of large language models in scientific writing undermine scientists’ critical thinking?DOI ↗
Memory

The synthesis has made this claim 19 times since 30 Sept 2026, each time in new words. Without memory each one looked like a finding. Claim memory ties them to one line, so a rewording is counted as a repeat and only a new paper extends it.

Open the full trace →

Where it chafes

Four things that went wrong,and what they taught.

Every machine check passed in each of these. That is the point.

The license was green. The PDF was not.

A paper was correctly green for its preprint. The PDF the curator had in Zotero was the publisher's version, through the library. It was stored and indexed, and six claims cited it.

What changed

The Zotero fallback now requires the gate itself to find the published version open, with no conflict. The text and its index were removed; the claims kept their other sources.

The lesson

A license belongs to a version, not to a DOI. The gate has to know which version it is holding.

Every check passed. The paper didn't belong.

In one bundle, 33 of 40 discovered papers came in because they cite a single, very widely cited anchor. Marketing, supply chains and the game of Go scored as close to the topic as the papers that belonged.

What changed

Each bundle can define its own centre, no anchor may bring in more than a share of a night's picks, and removals are recorded with a reason so they are not rediscovered.

The lesson

Machine checks verify the chain from claim to page. Whether a paper belongs is still a human judgement.

The synthesis kept saying the same thing.

Night after night the same claims came back in new words. One line, about the share of LLM-processed abstracts, was made eleven times. The model was fine; it was being shown nearly the same papers every night.

What changed

Claim memory links rewordings to one line. The evidence rotates, so papers that have waited longest are read next, and the model is told what has already been said.

The lesson

A fluent new paragraph is not new knowledge. Ask what is new since last night.

Every job was green. Nothing was coming in.

For a week the nightly jobs all reported success while almost no new paper arrived. Five of six bundles had reached their target, and discovery only looked at the most-cited works citing each anchor, so last week's papers could never qualify.

What changed

A second lane takes work from the last 30 days, newest first, about ten papers a week across the bundles. It does not count against a bundle's target.

The lesson

A green tick says a job ran, not that it did anything. Watch what arrives, not only what runs.

Boundaries

Rules that holdeven when the code is wrong.

A bug in the intake worker should not be able to leak a closed text or publish an unsourced claim. So the important rules live in the database too, not only in the code.

What it asks of supervision

Training researcherswhen their tools are agents.

Master's and PhD students already use these tools, often without saying so. The research on what it does to learning and to the supervisory relationship is young, and growing every week.

What to ask of a student's AI-assisted work

01

Show the chain

Every claim you bring me points to a quote, a page and a DOI. If it cannot, it is a lead, not a finding.

02

Say who checked

Which parts did a model write, which did a model check, and which did you read yourself?

03

Formulate the task first

Write down what you are asking before the agent does it. The jagged frontier is easiest to see from there.

The way of working

You don't need this code.You need the way of working.

Describe these building blocks to a coding model and build your own, around your own reading. This page is close to a specification. What matters more is what it asks of you once it runs.

What you build

  • Open metadata

    OpenAlex, Crossref with Retraction Watch, Unpaywall, Europe PMC, arXiv and bioRxiv. All have free APIs.

  • The reference manager you already use

    Zotero with a shared group: one collection per bundle, and your highlights become evidence.

  • A database that can search by meaning

    Postgres with pgvector, and an embedding model you host yourself (here bge-m3).

  • Two language models

    One writes, a different one checks. Here they are open-weight models at night, Claude in conversation.

  • A scheduler and an agent protocol

    Jobs that run while you sleep, and MCP so you can add a paper or trace a claim from a chat.

  • A small server

    This runs in about 1–1.5 GB of memory, next to other things on a modest box.

What it asks of you

  • Choose the anchors

    A bundle starts from a handful of papers you know well. They decide what the system goes looking for.

  • Settle the conflicts

    When sources disagree about a license, the gate flags it. You open the PDF and read the license line.

  • Read traces, not summaries

    Open the sentence before you repeat the claim. The trace is there so that you use it.

  • Prune, with a reason

    When a paper does not belong, take it out of the bundle and write down why. The record is part of the method.

  • Expect repetition

    Most nights add little. The useful question every morning is what changed, not what was said.

Where to start

01

One bundle

Ten papers you know well, on one question you care about.

02

The gate first

Before you store any text, decide what you are allowed to do with it.

03

The trace before the synthesis

Make it impossible to save a claim without a page and a DOI, then let a model write.

04

A month of nights

Run it every night and read the mornings before you trust it with anything.

Questions to take with you
  • Where would you not trust a connection like this in your own research?
  • Where is the line between a connection you would not have seen and one that should not be there?
  • How much should your own judgements steer what the system shows you next?
  • What would you have to do every week for this to stay honest?
Sources

Everything this page cites.

In the overview

Infrastructure

Explore the live corpus: Overview · Bundles · Licenses · Search