Provable Limits and Certified Deferral for Verbalized Uncertainty in Small Language Models

Source: arXiv:2608.05064 · Published 2026-08-05 · By Jianru Shen

TL;DR

This paper addresses the challenge of deploying small open-weight language models (0.5B to 14B parameters) in risk-sensitive scenarios where autonomous model answers may be unsafe, making deferral to humans necessary. It investigates whether verbalized confidence—an integer score output by the model alongside its answer—can provide a calibrated and certified risk signal to enable principled threshold-based deferral. The authors provide three theoretical results delimiting calibration capabilities: strictly monotone calibration preserves the risk-coverage frontier and error-detection AUROC; temperature scaling cannot calibrate models whose confidence exceeds 0.5 but accuracy falls below 0.5; and a Clopper-Pearson statistical procedure can transform a finite calibration set into a valid finite-sample risk certificate under i.i.d. assumptions. Empirically, temperature scaling hits these theoretical limits on eight of 22 model-task pairs, while Platt scaling reduces expected calibration error (ECE) dramatically (as low as 0.02). However, certified autonomy at a 20% maximum risk budget is only granted to three model-task pairs, with none certified at 10%, underscoring the difficulty of safe autonomous use. The study audits eleven models from three families across two challenging benchmarks (ARC-Challenge, TruthfulQA) totaling 25,168 predictions, identifying and fixing an answer-ordering artifact in TruthfulQA to reduce positional bias. The result is a rigorous, practical framework combining verbalized confidence extraction, calibration, and certified threshold selection to determine safe autonomous usage of small LMs in critical settings.

Key findings

• Strictly monotone calibration (e.g., Platt scaling) preserves the risk–coverage frontier and error-detection AUROC (Proposition 2).
• Temperature scaling cannot calibrate models with all confidence scores > 0.5 and accuracy < 0.5; ECE bound exceeded in 8 of 22 model-task pairs (Corollary 1).
• Platt scaling reduces expected calibration error (ECE) from as high as 0.687 (uncalibrated) to as low as 0.02 on ARC-Challenge (Gemma3-1B model).
• Calibration quality improves with model size but median verbalized confidence remains nearly constant (0.80 to 1.00), implying calibration corrects a fixed confidence level drifting closer to accuracy with scale.
• Certified autonomy at 20% risk budget is achieved only by three model-task pairs (Qwen2.5-14B ARC: 99.8% coverage at 8.1% risk; Qwen2.5-7B ARC: 93.3% coverage at 10.8% risk; Qwen2.5-14B TruthfulQA: 43.6% coverage at 14.2% risk); none certified at 10% risk budget.
• Answer-ordering artifact on TruthfulQA was identified: originally correct answers always at first position, inflating positional accuracy baseline from ~25% to 100%. Randomizing options normalized letter distribution.
• Cross-task threshold transfer violates risk budgets by 1.3 to 30.6 percentage points, confirming per-task calibration and certification necessity.
• Error-detection AUROC based on verbalized confidence is modest (0.5 to 0.66) even for larger models, showing limited direct error signal quality.

Threat model

The adversary is an implicit probabilistic uncertainty scenario rather than a malicious attacker: the goal is to mitigate risk of incorrect predictions under i.i.d. deployment data by deferring low-confidence or high-risk cases to humans. The adversary cannot manipulate the input distribution or model outputs, nor cause distribution shift. The setting assumes access only to model answers and verbalized confidence scores via text interface without gradient or logit-level access.

Methodology — deep read

1. Threat Model and Assumptions: The adversary is not explicitly adversarial but deployment assumes an i.i.d. distribution between calibration and later data. The central problem is to determine when a small LM's answer is reliable enough to use autonomously versus deferral to a human is needed to keep errors below risk budget r_max. The model can only provide verbalized confidence along with its answer via a text interface; no white-box access or multiple output generations are assumed.

2. Data: Experiments use two multiple-choice benchmarks: ARC-Challenge (science questions, 1,471 test questions after 200 calibration held out) and TruthfulQA (adversarial questions, 817 total, split similarly). Each question has 2-5 answer choices. The calibration sets are 200 questions each, used for fitting post-hoc calibration maps and setting thresholds; test sets evaluate performance.

3. Architecture and Algorithms: Eleven open-weight instruction-tuned LMs from three families (Qwen2.5, Llama3.x, Gemma3) sized 0.5B to 14B parameters were studied. A single text prompt elicits an answer letter plus an integer confidence in a fixed "letter,confidence" format, parsed strictly or leniently. Verbalized confidence c is [0,1] after dividing integer by 100.

Post-hoc calibration maps are fit on the calibration set: temperature scaling (TS), Platt scaling (logistic regression on logit of c), and isotonic regression. Minimizing Expected Calibration Error (ECE) guides parameter fitting. Strict monotonicity of Platt and TS maps is important for theoretical guarantees.

4. Training Regime: There is no model training. Only calibration maps are fit on 200-question calibration sets per task-model pair. Temperature scaling optimizes temperature T ∈ [0.1,100], Platt fits logistic regression parameters (a,b), isotonic learns a non-decreasing step function.

5. Evaluation Protocol: Metrics include accuracy, Expected Calibration Error (ECE), AUROC for error detection based on confidence, and coverage at certified risk thresholds. Certified deferral thresholds are selected by inverting Clopper-Pearson binomial upper confidence bounds with union bound correction over possible thresholds, to provide finite-sample guarantees at 95% confidence (δ=0.05).

Threshold selection finds the smallest threshold τ such that empirical risk on retained predictions is below r_max, maximizing coverage. Data splits separate training (calibration) and test sets.

Theoretical results (Propositions 1-3, Corollary 1) underpin algorithmic guarantees and motivate choice of Platt scaling for calibration and certificate construction.

6. Reproducibility: The study was performed on a single laptop, running all models locally via OLLAMA. Calibration routines and thresholding procedures are standard and described in detail. Code release or frozen weights are not mentioned, so implementations may be custom but fully specified.

Example Workflow: For each question, the LM answers with a letter and integer confidence. The raw confidences are scaled by Platt calibration fit on 200 held-out questions to produce semantic probabilities. For a given risk budget (e.g., 20%), the Clopper-Pearson certificate is used to pick the lowest threshold with empirical risk within budget plus finite-sample confidence margin. At deployment, only predictions with calibrated confidence above threshold are accepted; others defer to human, ensuring error rate is controlled with statistical guarantee.

Technical innovations

• Formulation and proof that strictly monotone calibration maps preserve the risk–coverage frontier and error-detection AUROC (Proposition 2), linking calibration quality directly to deferral performance.
• Identification and theoretical characterization of an infeasibility bound showing temperature scaling cannot correct models whose verbalized confidence exceeds 0.5 when accuracy is below 0.5 (Corollary 1).
• Development of a finite-sample, distribution-free risk certificate based on the Clopper–Pearson procedure with union bound correction, ensuring calibrated thresholds yield statistically guaranteed error risk under i.i.d. assumptions (Proposition 3).
• Discovery and systematic correction of an answer-option ordering artifact in TruthfulQA's multiple-choice data that biased positional letter scoring and inflated accuracy estimates.
• Comprehensive empirical study combining verbalized uncertainty estimation, calibration methods, and certified deferral on eleven open-weight, instruction-tuned small LMs across two challenging benchmarks totaling over 25,000 prediction points.

Datasets

• ARC-Challenge — approximately 1,471 test questions, 200 calibration questions — public benchmark
• TruthfulQA (multiple-choice single-answer form) — 817 pooled calibration and test items — public benchmark

Baselines vs proposed

• Uncalibrated: ECE ranges up to 0.687 (Gemma3-1B ARC) vs Platt scaling reduces ECE to as low as 0.02
• Temperature scaling (TS): often hits infeasibility bound with optimal T* ≥ 100 on 8 of 22 model-task pairs, residual ECE at bound (e.g., 0.283 vs 0.276 predicted)
• Platt scaling: ECE improvement over TS, strictly monotone and maintains deferral sets
• Isotonic regression: sometimes marginally lower ECE than Platt but collapses confidence levels, coarsens deferral granularity
• Certified coverage at r_max=20% risk: Qwen2.5-14B ARC = 99.8% coverage vs Qwen2.5-7B ARC = 93.3%, others no certified threshold
• AUROC for error detection modest; largest models reach only 0.63–0.66 vs ~0.5 for small

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2608.05064.

Fig 1: Overview: a small LM emits an answer with verbalized confidence c; Platt

Limitations

• Benchmarks only cover multiple-choice tasks; open-ended generation scenarios remain untested for calibrated deferral applicability.
• Certification assumes i.i.d. calibration and deployment distributions; distribution shifts break the statistical guarantees.
• Calibration set sizes (200 questions) limit the fineness of risk guarantees; no certified thresholds found at stricter 10% risk budgets.
• Model sizes capped at 14B and only three model families considered; generality to larger or black-box models uncertain.
• Only two elicitation prompt formats tested on one dataset; robustness of verbalized confidence elicitation requires broader validation.
• Verbalized confidence signal alone shows weak discrimination power (low AUROC), limiting direct error detection.

Open questions / follow-ons

• How to extend calibrated deferral frameworks to open-ended, generative tasks rather than multiple-choice settings?
• How to maintain valid finite-sample risk certification guarantees under realistic distribution shifts at deployment?
• Can alternative uncertainty elicitation techniques beyond verbalized confidence improve error detection and certification utility?
• What is the impact of larger models or ensemble approaches on calibrated deferral performance and certification feasibility?

Why it matters for bot defense

For bot-defense and CAPTCHA practitioners, this work introduces a rigorous approach to leveraging calibrated verbalized confidence from small, locally running language models to manage risk in high-stakes, autonomous decisions by certified deferral. The finite-sample certification framework provides a principled method to set risk thresholds that guarantee error budgets are not exceeded with statistical confidence, a critical property when using AI models in security-sensitive pipelines where automated adversarial bypasses must be minimized. The findings also highlight the limits of temperature scaling and the benefits of Platt scaling for producing usable, interpretable confidence scores suitable for thresholding. Importantly, the demonstration of formal guarantees paired with transparent per-example justifications could inspire more auditable, risk-aware automated decision layers in bot mitigation systems. However, the modest error-detection AUROC reported implies verbalized confidence alone is insufficient for fine-grained error filtering at high accuracy, suggesting it should be combined with other signals or human-in-the-loop designs. Finally, the answer-ordering artifact audit provides a cautionary example that dataset biases can distort confidence calibration and evaluation in security applications, underscoring the need for rigorous input preprocessing.

Cite

@article{arxiv2608_05064,
  title={ Provable Limits and Certified Deferral for Verbalized Uncertainty in Small Language Models },
  author={ Jianru Shen },
  journal={arXiv preprint arXiv:2608.05064},
  year={ 2026 },
  url={https://arxiv.org/abs/2608.05064}
}

Read the full paper

• arXiv:2608.05064 (abstract)
• arXiv:2608.05064 (PDF)