Uncertainty & Calibration
Does a confidence score predict how often the result is actually correct?
Reference notes by Min · Reviewed October 15, 2026
Definition
Calibration describes the relationship between predicted confidence and observed correctness across comparable cases. Agent uncertainty management uses validated signals, missing-evidence checks and escalation rules to decide when the workflow should proceed, ask or abstain.
Confidence is useful only when its meaning is defined and tested. A token probability, a separate classifier score and a model’s written self-assessment are different signals. Avoid presenting them interchangeably as a probability that the whole workflow is safe.
What it covers
- Signal
- The score or evidence used to express uncertainty.
- Outcome
- The correctness definition being predicted.
- Population
- The cases for which validation applies.
- Decision
- The action taken at different uncertainty levels.
Specify the outcome being predicted. Correctly reading an invoice total does not imply the invoice is legitimate or payment is authorized. Calibrate a narrow measurable claim and keep other controls responsible for their own decisions.
In practice
Practical design patterns by Min.
- Define what the confidence score claims to predict.
- Compare scores with independently assessed outcomes.
- Keep authorization separate from estimated correctness.
Evaluate confidence on representative held-out cases and relevant subgroups. Examine changes after model or workflow updates. When the operating distribution changes, previously useful confidence relationships may no longer support the same threshold.
Research notes
Source-grounded details and editorial applications. Follow each link for the original document.
Calibration is an empirical relationship
Guo and colleagues examine confidence calibration in neural-network classification. Their methods and findings should not be assumed to validate language-model self-reported certainty or an entire agent workflow.
Source: Guo et al.: On Calibration of Modern Neural Networks (2017) ↗
Working example
An agent extracts invoice totals
Low-confidence extraction should trigger review before an approved payment process.
- Measure extraction error across score ranges.
- Inspect difficult formats and document classes separately.
- Escalate missing or inconsistent evidence even if the model sounds confident.
Why the controls matter
The invoice case needs explicit handling of missing pages, inconsistent currencies and contradictory totals. Those are evidence problems that can require review independently of a numerical score. The payment service still enforces approval and resource limits.
Common pitfalls
- Treating “I am 95% sure” as a calibrated probability.
- Using one threshold for tasks with different consequences.
- Confusing confidence with permission.
Abstention has costs too: excessive escalation can overwhelm reviewers or exclude difficult users. Measure both harmful automatic decisions and unnecessary deferrals. Choose thresholds with the responsible owner and document the tradeoff and uncertainty.
Questions for operational review
Use these questions to decide what evidence your team needs. They are editorial prompts, not a compliance checklist.
- What exact outcome does the score predict?
- Has the relationship been measured on this workflow?
- What evidence forces escalation regardless of confidence?
Articles linking here
References
Primary documents reviewed for this entry. Examples and implementation patterns are editorial synthesis.
- Guo et al.: On Calibration of Modern Neural Networks (2017) ↗Accessed October 15, 2026
- NIST AI RMF: Core functions and subcategories ↗Accessed October 15, 2026