Sense and decide
Disprove, Don't Confirm
Put every plausible explanation on the table and try to kill each one, instead of building support for the first or most confident.
Borrowed from Analysis of Competing Hypotheses, developed by CIA analyst Richards Heuer and taught in the CIA's Tradecraft Primer. Read the original source.
One session · Minutes to start · Anyone on the team can start it
Try this first
Force three explanations onto the table and ask of each: what would prove this wrong?
Use it when
The team is anchored on the AI's confident single answer.
Skip it when
Reserve it for consequential, uncertain questions. Do not use it on every decision; the extra analysis gets exhausting.
How to introduce it
Put the plausible explanations on the table together, then work to disprove each. The explanation that survives the strongest attempts to disprove it is the best-supported option so far, not a certainty. This inverts the usual habit of finding support for the first idea, which with AI is often the model's confident one.
How to show up
Push for at least three explanations, then hunt for what disproves each rather than building the case for the favorite. Guard hardest against treating the model's first answer as the thing to confirm.
How long it takes
Ten to thirty minutes on a consequential question.
What makes it hard
People produce two explanations and stop, or build toward the one they already hold. Push for more, and for real disconfirmation. A confident AI answer is easy to anchor on, so it takes effort to treat it as one hypothesis among several.
What it looks like when it's working
Several explanations get weighed and attacked, and sometimes the confident first answer dies. Gathering support for one answer and calling it done means the method is not in use.
How long until it sticks
Expect several uses before disproving rather than confirming becomes the team's instinct.
How you know it stuck
The reflex on an uncertain call is to lay out the options and try to break each.
The idea behind it
You can find support for almost any answer. What separates strong from weak is what survives an attempt to kill it.
Where it comes from
Intelligence analysis. A model hands you one confident answer. This forces at least three into the room and sends the team hunting for evidence against the favorite. Put every plausible explanation on the table at once, then go looking for what kills each one. Prefer the explanation that is least inconsistent with the evidence after you have tried seriously to disprove each one.