Sense and decide
Words of Estimative Probability
Agree what "likely" means in numbers
Agree what your confidence words mean in numbers, and separate how likely something is from how good the evidence is.
Borrowed from Sherman Kent, a CIA analyst who in 1964 proposed agreed meanings for words like "probable" and "likely." Read the original source.
Changes a rule or role · An hour or more to set up · Anyone on the team can start it
Try this first
Agree number ranges for the confidence words your team uses most.
Use it when
"Likely" and "probably" mean different things to different people, and AI output can sound more certain than the evidence warrants.
Skip it when
This matters where people trade probabilities and act on them. A team that rarely does can skip it. Keep it a rough shared scale, not a standard with a manual.
How to introduce it
Agree roughly what your confidence words mean, so likely does not mean thirty percent to one person and eighty to another. Then separate two things people blur: how probable the outcome is, and how solid the evidence behind the estimate is. Intelligence analysis learned this the hard way, and it matters again when acting on confident-sounding AI output.
How to show up
Get the shared scale agreed and keep it visible. Keep drawing the line between how likely and how sure we are of that.
How long it takes
A short session to agree the scale. Then it's applied.
What makes it hard
People are attached to their own sense of likely, so agreeing takes negotiation. The harder split is likelihood against evidence, because confident-sounding AI output can make a weak estimate feel better supported than it is.
What it looks like when it's working
Confidence words carry the same meaning across the team, and people separate probability from strength of evidence. If likely still means different things to different people, the vocabulary is not shared.
How long until it sticks
A few weeks of use before the shared vocabulary is shared.
How you know it stuck
The team's confidence words mean one thing, and people split likelihood from evidence without prompting.
The idea behind it
Vague probability words mean different things to different people and wreck shared understanding without anyone noticing. A common scale, and the split between likelihood and evidence, makes confidence something you can communicate.
Where it comes from
Intelligence analysis. Agree a shared vocabulary for confidence, and separate probability from analytic confidence. Vital when acting on AI outputs that sound certain. Standardize what "likely" means numerically, so "probably" isn't read as 30% by one person and 80% by another (Sherman Kent's fix).