Two AIs Agreeing Is Not Independent Confirmation

In the not so distant past, I watched a spectral imaging analysis pipeline conclude, with high confidence and across multiple methods, that a sensor had detected cattle breath. The pipeline had been generated by an AI assistant. The conclusion had been confirmed by a second AI assistant when the same prompt was run through it. The final report had been written by the first AI again, summarizing its own findings.

The conclusion was wrong. The sensor cannot physically detect breath in the wavelength range it operates in. The "signal" the pipeline found was almost certainly the result of operator hand-motion during capture, not any biological process. Every step in the chain felt rigorous in isolation. The whole chain was structurally broken.

This pattern is not unusual. It is becoming the dominant failure mode in AI-assisted technical work, and almost nobody using these tools has internalized why it happens or how to prevent it.

Why AI finds what you ask it to find

A large language model, prompted to find evidence of X in a dataset, is not a neutral observer evaluating whether X is present. It is an optimization process routing toward task completion. The task is "find X." The completion is "I found X, here is the evidence."

This is not a flaw in the system. It is the core of what makes these systems useful. When you ask an AI to draft a contract, summarize a meeting, or write a function, you want it to complete the task. The same orientation that produces helpful drafts produces confidently-asserted findings when the task is analytical.

The metal detector analogy is a perfect example. A metal detector asked whether there is metal in a location that will detect any metal present, including the metal in your watch, your belt buckle, and your phone. It is doing its job. The error is not in the detector. The error is in treating the detector's positive response as evidence of buried treasure rather than as evidence of metal-of-some-kind. The detector cannot tell you which metal it found or why it matters. That is your job, and the detector cannot do it for you.

Treating an AI's analytical output as if the AI were a neutral instrument, when the prompt structure pre-committed the system to finding what you asked for, is the equivalent of digging up your own watch and concluding you have located pirate gold.

Why agreement between AI systems is near-zero evidence

The instinct, when an AI returns a confident finding, is to validate it by asking a second AI. Two systems agreeing feels like cross-validation. It is not.

Independence in scientific epistemology requires genuinely independent processes. Two thermometers in the same room do not provide two independent measurements of global temperature. They provide one measurement, redundantly. Two LLMs trained on overlapping internet corpora, given the same prompt with the same baked-in assumptions, are not independent processes. They are two slightly different routes to the same answer, constrained by the same prompt, drawing on heavily overlapping training distributions.

When the analyst I observed prompted one model to find breath in the data, then prompted a second model with the same framing and noted that it also found breath, the second prompt added approximately zero information. Both systems were given the same task. Both produced outputs consistent with completing that task. The agreement was structural, not empirical.

The deeper version of this problem is that even genuinely different model architectures will tend to produce similar outputs when given similar prompts, because the prompt itself constrains the answer space heavily. Asking "find breath in this data" anchors any reasonable system on producing a breath-shaped answer. Two systems agreeing on a breath-shaped answer to a breath-shaped question is not corroboration. It is the same answer, twice.

The closed-loop confirmation problem

The failure compounds when AI is used at multiple stages of an analysis without human falsification between them. In the case I observed, AI generated the analysis pipeline, AI executed the analysis, and AI wrote the summary report concluding that breath had been detected. There was no human step in the loop where someone with domain knowledge asked: could this signal come from something other than breath? Could the methodology be flawed? Are the underlying assumptions defensible?

Each AI step took the previous AI step's output as input and treated it as authoritative. Confirmation bias accumulated across the pipeline rather than averaging out. By the time the final report was generated, the original prompt's pre-commitment to finding breath had been amplified through three layers of AI-assisted reasoning, each of which inherited the prior layer's framing as ground truth.

The fix is conceptually simple and procedurally hard: insert human falsification at every layer where the analysis could plausibly be wrong, or use AI explicitly for falsification rather than for confirmation at any layer. The procedural difficulty is that falsification feels like extra work, especially when the existing pipeline is producing confident results that match what you hoped to find. The temptation is to skip it. The temptation is wrong.

The adversarial alternative

The reframe that fixes most of this is straightforward: stop asking AI to confirm and start asking it to falsify. Stop asking "find evidence of X" and start asking "what could be wrong with concluding X."

Concrete prompts that change the optimization target:

These prompts work because they invert the system's optimization target. The AI is now routing toward objections rather than toward confirmation. The same model that will confidently assert breath was detected when prompted to find it will, when prompted adversarially, generate a list of plausible alternative explanations, methodological weaknesses, and unverified assumptions. Both responses are correct given their respective prompts. Both are produced by the same underlying capability. The user determines which capability gets activated by how the question is framed.

The harder version of adversarial use is to run both. Prompt for the analysis, then prompt for the critique of the analysis, then take the critique seriously and address each point before treating the original finding as real. This is what rigorous human reviewers do for each other's work. AI is well-suited to play the same role, if you ask it to.

A specific prompt pattern worth memorizing: after the AI produces an analytical conclusion, ask it "now argue against this conclusion as a peer reviewer would." The output is often more useful than the original analysis, because it surfaces the assumptions the original analysis depended on without examining.

What this means for technical practice

A few principles fall out of this for anyone using AI in technical or scientific work:

Default to adversarial prompting for any claim you intend to publish, act on, or stake reputation on. Confirmatory prompting is fine for exploration; it is not fine for conclusions.

Treat agreement between AI systems as zero evidence unless the prompts were genuinely structured to be independent, which usually means having different humans construct them without coordination.

Document your prompts, not just your outputs. The prompt is part of the methodology. An analysis whose prompts are not preserved is an analysis whose methodology is undocumented.

Require physical mechanism, not just statistical pattern, before treating any AI-found pattern as real. AI is excellent at finding patterns. Physical mechanism is what tells you whether a pattern is meaningful.

When AI is used at multiple stages of an analysis, insert explicit human falsification between stages, or use AI as the falsifier between stages. Closed loops without falsification compound errors instead of catching them.

The broader stakes

This dynamic is not confined to spectral imaging or to one anonymized analysis I happened to witness. The same pattern is visible right now in medical research, materials science, financial analysis, intelligence work, and any technical domain where AI-assisted analysis is being used to produce conclusions that get acted on. The pattern is hard to see in the moment, because each step looks rigorous. The pattern is easy to see in retrospect, when the conclusion fails to replicate or fails to survive scrutiny from someone outside the original loop.

The competitive consequence over the next few years is going to be significant. Organizations that develop adversarial AI hygiene will produce more reliable results than organizations that don't, and the gap will compound as AI use deepens. The professional consequence for individuals is similar. Being the person who thinks adversarially about AI outputs is becoming a meaningful career differentiator. Being the person who treats AI outputs as authoritative is becoming a liability.

The norms for how AI gets used in technical work are being set right now, in the early phase of widespread adoption. The defaults that get established in this period will shape the next decade of how science and engineering get done. We can establish that AI is a tool used adversarially, with explicit falsification, by analysts who retain ownership of their conclusions. Or we can establish that AI is treated as an oracle whose outputs become evidence by virtue of having been produced. The first produces better work. The second produces work that fails publicly, eventually, when the underlying confirmation loops finally meet a problem they cannot rationalize away.