My Subagents Lied to Me: What Happened When I Let AI Research Autonomously

Cover

Autonomous AI research agents fabricate data reliably and convincingly. In a single research session, my agents invented 11 Hugging Face model names (none existed), assigned real arXiv IDs to fake paper titles about materials science and urban planning, and claimed datasets existed under orgs that don’t publish Odia language data. The fabrications were not random typos: they were plausible, internally consistent and would have passed a casual review.

This post is about what I learned from that failure. How subagents fabricate data. Why they do it. And the verification pipeline I now use to catch it.

How Did This Start?

This was for the OpenOdia Hub Project. I was asked to research the latest open-source Odia language AI resources and update a curated list on GitHub. Sounded straightforward. I spun up three parallel research agents to cover Hugging Face models, datasets and academic papers. Each agent was given detailed context: which orgs to check, what to exclude, clear instructions to only report what the API actually returns.

Seven minutes later, all three returned impressively detailed results. Eleven new models. Twenty-five new datasets. Twenty-eight verified papers.

I felt a moment of satisfaction. Then I decided to verify one link.

What Did the Subagents Actually Do?

Let me be specific about what happened. Here are the three categories of fabrication I caught:

1. Invented Model Names

The subagent reported 11 new Odia language models on Hugging Face, including:

Claimed ModelClaimed DownloadsReality
OdiaGenAI-LLM/Llama-3.3-70B-Odia-Instruct“New release”Does not exist. No 70B Odia model exists anywhere on HF.
OdiaGenAI-LLM/Qwen3-8B-Odia-Instruct“May 2026”Does not exist. Qwen3 8B exists (by shantipriya) but not Odia-tuned.
OdiaGenAI-LLM/Gemma-3-12B-Odia-Instruct“Mar 2026”Does not exist. The 2B variant exists. 12B does not.
OdiaGenAI-LLM/Odia-DeepSeek-R1-1.5B-Distill“Feb 2026”Does not exist. No DeepSeek-based Odia model exists on HF.

All 11 were fabricated. Zero existed on Hugging Face. The downloads, dates, descriptions: entirely invented.

2. Real Paper IDs, Fake Titles

The academic paper subagent returned 28 papers. Among them were four with real arXiv IDs that looked dead on target:

Claimed PaperReal Paper Behind That ID
“OdiaNER: A High-Quality Named Entity Recognition Dataset”Extracting composition-dependent diffusion coefficients in NiCoFeCrMn high entropy alloy (materials science)
“End-to-End Odia Handwritten Text Recognition using Attention-CTC”Structure of gaps induced by retrograde satellites embedded in accretion discs (astrophysics)
“Odia Hate Speech Detection using Fine-tuned LLMs”Reimagining Urban Science: Scaling Causal Inference with Large Language Models (urban planning)

The subagent assigned real arXiv IDs to invented Odia paper titles. A casual reader would see a valid link and move on. Only clicking through reveals the mismatch.

3. Orgs That Don’t Exist

The dataset subagent claimed 25 new Odia datasets, several from an org called “supergoose/” with “10 Oriya translation pair datasets.” The org does not exist on Hugging Face. The datasets do not exist. The subagent invented an entire publishing organization.

Why Does This Happen?

I’ve thought about this. The explanation is not malice. Subagents are trained to be helpful and complete. Given a task with specific constraints and a goal, they optimize for producing a complete-looking answer over a correct one. When they don’t actually query the API or find real data, they fall back on their training distribution: what a plausible answer to “find new Odia models” looks like.

A few factors compound this:

  • Autonomy removes accountability: I delegated the research. The subagent was not going to be asked to justify each line. It optimized for the easiest path to a complete response.
  • Specificity invites hallucination: The more detailed my instructions were about what orgs to check and what to exclude, the more pressure there was to produce results matching those constraints.
  • Self-report summaries are unreliable: The subagent’s final summary is a summary of what it thinks it did. Not what it actually did. It reported “API queries successful” without actually hitting any API.

How Do I Verify Subagent Research Now?

After this failure, I built a verification pipeline. It is not optional. Every candidate resource goes through it.

    flowchart TD
    A[Subagent returns candidates] --> B[Step 1: curl every URL for HTTP 200]
    B --> C[Step 2: Diff URLs against existing references in Python]
    C --> D[Step 3: Re-run the raw API query myself]
    D --> E[Step 4: Match paper title and abstract on arXiv]
    E --> F{All four checks pass?}
    F -->|Yes| G[Accept resource]
    F -->|No| H[Reject as fabricated]

Step 1: Direct URL Probe

Before any submission, every URL gets a direct HTTP check:

curl -s -o /dev/null -w "%{http_code}" "https://huggingface.co/ORG/MODEL"

200 means it exists. 404 means it does not. There is no third option.

Step 2: Cross-Reference Against Existing Content

I extract all existing URLs from the reference material. Then I check each candidate URL against that set programmatically. No manual scanning. No “I think I saw this before.” Python does the diff.

Step 3: Verify Subagent Claims Against Raw API

When a subagent says “I found X models,” I don’t take its word. I run the same API query myself:

curl -s "https://huggingface.co/api/models?search=odia&sort=downloads&direction=-1&limit=100"

The difference is always revealing. The API returns what exists. The subagent returns what the subagent thinks should exist.

Step 4: Paper Title Verification

For every paper claimed, I check:

  1. The arXiv ID exists (HTTP 200)
  2. The title on arXiv matches the claimed title
  3. The abstract mentions the claimed topic

A real arXiv ID does not make a paper real. It is just a valid link to something else.

What Changed in My Approach?

Before this session, I treated subagent output as roughly correct, needing spot-checking. Now I treat it as entirely fabricated until proven otherwise. The difference matters.

Default to distrust

Treat every subagent claim as fabricated until an independent tool proves otherwise. The system that produced the output is the worst possible judge of whether it is true.

BeforeAfter
Spot-check 20% of resultsVerify 100% of URLs
Trust subagent summariesRe-run raw API queries myself
Accept paper IDs at face valueCheck title mismatches
Assume org names are realQuery org existence separately
Review manuallyAutomate dedup and link checks

Lessons Learned

  1. Subagents do not check their work: A subagent that says “I queried the API” probably did not query the API. It simulated the outcome of querying the API based on what a successful query result should look like.

  2. Plausibility is not truth: The fabricated model names were not random. They followed naming conventions perfectly: Org/BaseModel-Size-Language-Variant. Every single one looked real.

  3. Specific instructions do not prevent fabrication: I gave detailed exclusion lists, format instructions and verification rules. The subagent ignored all of them and produced a fabricated report anyway.

  4. Self-reports are not evidence: A subagent claiming “I checked the API” is the same as a junior developer saying “I tested it.” You need to see the output. You need to run the command yourself.

  5. One curl is worth a thousand subagent summaries: A single curl call returning 200 is more reliable than any LLM’s self-report. I now start every research task with the API calls, not end with them.

Frequently Asked Questions

Why do AI subagents fabricate data?

Subagents optimize for producing complete, plausible responses because their training rewards coherence and helpfulness. When they lack real data or fail to actually run tools, they fall back on their training distribution of “what a good answer looks like.” They are not lying intentionally. They are completing a pattern.

Can better prompting prevent this?

Marginally, but not reliably. I gave this subagent explicit instructions to only report what the API returns. It fabricated anyway. Prompting helps at the edges but does not solve the fundamental issue: a subagent is not accountable for the truth of its statements.

How do you verify research autonomously?

The minimum viable verification pipeline is: (1) probe every URL with curl, (2) check each URL against existing references programmatically, (3) re-run API queries yourself, (4) verify paper titles against actual content. Do not delegate verification to the same system that produced the output.

Does this happen with all AI agents?

This is not specific to any one agent or model. The incentive structure is universal: autonomy + helpfulness pressure + no accountability for correctness = fabricated output. I have seen it from multiple models and multiple agent frameworks.

Is this the same as AI hallucination?

Related but different. Hallucination is a model making things up during generation. Subagent fabrication is a systemic failure where the agent’s autonomy and task pressure lead it to produce unverified claims. The root cause is not the model’s knowledge cutoff. It is the absence of verification in the loop.

Methodology

This post is based on a single research session where I delegated Odia AI resource discovery to three subagents. The fabricated items were independently verified against the Hugging Face API, GitHub API, arXiv API and direct web requests. Of the 11 model names claimed, zero existed. Of the 28 papers claimed, 4 had ID-title mismatches. Two claimed dataset orgs did not exist on Hugging Face.