Partition, Prompt, Aggregate: Statistical Self-Consistency in Language Models
Source: arXiv:2607.15277 · Published 2026-07-16 · By Patrik Wolf, Thomas Kleine Buening, Andreas Krause, Celestine Mendler-Dünner
TL;DR
This paper investigates whether large language models (LLMs) behave as conditional inference engines during in-context learning (ICL), specifically whether their probability estimates satisfy fundamental probability axioms like the law of total probability. The authors propose a systematic evaluation framework using binary conditioning trees to recursively partition a population into subpopulations, prompt LLMs with verbalized subpopulation descriptions, and aggregate those conditional estimates back to the full population level. This enables checking statistical self-consistency across partitions of varying granularity without requiring external labels. Applying this framework to persona prompting for U.S. population income data and other domains, the authors find that LLMs exhibit widespread violations of probabilistic self-consistency. Notably, the "macro fallacy" emerges: LLM estimates aggregated from finer subpopulation prompts are more aligned with human ground truth than direct estimates from coarse population-level prompts. This reveals that models possess relevant fine-grained knowledge but fail to propagate it reliably into aggregate predictions. The paper further proposes implicit "micro-to-macro" prompting to partially recover this benefit within a single prompt. Overall, statistical self-consistency is introduced as a novel, reference-free metric for evaluating LLM distributional reasoning capabilities beyond task-specific benchmarks.
Key findings
- Direct aggregate prompting at the root population level is systematically less aligned with ground truth than aggregates reconstructed from conditional LLM estimates at finer subpopulation levels, demonstrating the macro fallacy phenomenon.
- Node-wise conditional estimates generally improve in alignment at deeper tree levels, while subgroup priors estimated by LLMs degrade in quality with increasing partition granularity, creating a trade-off.
- The total variation distance between LLM-estimated subgroup priors and human survey priors increases significantly with tree depth, indicating harder estimation of small subgroup weights.
- Residual within-subgroup variance of the target variable clearly decreases as the population is partitioned more finely, while cross-group variance increases, exposing structure helpful for more precise estimation.
- Micro-to-macro prompting, which implicitly decomposes the population rather than explicitly partitioning it, partially recovers the aggregation benefit within a single prompt with minimal overhead, reducing absolute error relative to direct prompting across income thresholds.
- Statistical self-consistency metrics introduced, including split consistency and order consistency, reveal violations even in state-of-the-art models across problem domains.
- No correlation was observed between the proposed self-consistency scores and standard LLM benchmark alignment metrics, indicating these tests evaluate complementary aspects.
- Across various models (e.g., GPT-5.4, GPT-4o mini, Opus 4.7), tasks, and partitions, reconstructed aggregates improve alignment by up to around 50% relative error reduction (Fig 3).
Threat model
N/A – This work is an empirical and theoretical investigation of model probability estimation consistency, not a security paper.
Methodology — deep read
The authors start from the premise that in-context learning can be interpreted as performing conditional inference, where a prompt defines conditioning events, and the LLM output approximates the conditional probability of the answer given that context. Under this view, the law of total probability must hold: the marginal probability over the entire population equals the weighted sum of conditional probabilities over a valid partition.
To empirically test this, they construct binary conditioning trees (BCTs) which recursively partition a base population into subpopulations by conditioning on binary attributes. Each tree level corresponds to a valid partition. For example, using the 2024 American Community Survey (ACS) data on the US population, they partition by age and employment status.
Input data come from ACS, a large public survey (3.8 million respondents) with demographic and income attributes. Income distributions are discretized into bins. Tree splits are chosen using a regression-tree objective balancing variance reduction and subgroup weight.
For each node in the tree (subpopulation), the authors verbalize the conditioning attributes as a natural language prompt context — e.g., "A person between 31 and 68 years old, employed." They then ask the LLM a binary threshold question (e.g., "What is the probability their income is above $50k?") to obtain conditional probability estimates. The LLM is also separately asked multiple times to estimate subgroup prior probabilities, which are normalized and averaged.
Reconstructed marginal estimates at each tree level are computed by aggregating the LLM-elicited conditional estimates weighted by the corresponding subgroup priors. These are compared against the direct estimate obtained by prompting on the entire population without conditioning.
The alignment of estimates is measured by absolute error vs ground-truth ACS survey statistics. Additional metrics like total variation distance evaluate the quality of prior estimates. Aggregation gains are analyzed as relative improvements in alignment error over direct prompting.
To disentangle error sources, an oracle-prior experiment replaces LLM estimated priors with ground truth to isolate conditional estimate accuracy.
They also introduce an implicit micro-to-macro prompting scheme, where the model is tasked within one prompt to first reason about relevant subpopulations before returning an aggregate estimate.
Evaluation covers multiple models (including GPT-5.4, GPT-4o mini, Opus 4.7), multiple income thresholds, and variations in tree depth and attribute splits. Confidence intervals are computed via bootstrapping.
No model fine-tuning or retraining is performed; the study relies solely on zero-shot or few-shot prompting with natural language verbalizations derived from dataset attributes.
Overall, the methodology carefully tests a fundamental conditional inference assumption with an interpretable, partition-based prompting framework, blending statistical theory, public survey data, and natural language prompts.
Technical innovations
- Systematic use of binary conditioning trees to define hierarchical partitions for eliciting and aggregating LLM conditional probability estimates, enabling principled self-consistency evaluation.
- The "macro fallacy" observation that direct aggregate prompting underperforms reconstructed conditional aggregates, revealing incomplete propagation of subpopulation knowledge.
- Micro-to-macro implicit prompting that conditions the model internally on subpopulations before outputting an aggregate prediction, recovering aggregation benefits without explicit partitions.
- Introduction of local statistical self-consistency metrics (split consistency and order consistency) to evaluate probabilistic coherence across partition-based prompts without external labels.
- Application of rigorous decomposition of estimation errors into conditional estimate accuracy vs subgroup prior weighting, isolating sources of inconsistency.
Datasets
- 2024 American Community Survey (ACS) — 3.8 million respondents — public, US Census Bureau
Baselines vs proposed
- Direct aggregate prompting: alignment error = baseline vs reconstructed aggregate prompting: up to 50% relative error reduction (Fig 3)
- LLM-estimated subgroup priors TV distance to ground truth: increases with tree depth from near 0 to substantial values (Fig 4b)
- Micro-to-macro prompting vs direct prompting: absolute error reduced across 5 income thresholds and multiple models (Fig 5)
Figures from the paper
Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.15277.
Fig 6: Model comparison. We report the split consistency score SCε(A) and order consistency
Fig 2 (page 17).
Limitations
- The study focuses mainly on persona prompting and US population income data; generalization to other domains requires further validation.
- Subgroup priors are difficult to estimate reliably at deep partition levels, limiting benefits of very fine decompositions.
- The prompting templates and verbalizations rely on careful controlled attribute descriptions; applicability to free-form real-world prompts is unclear.
- No adversarial robustness or stress tests with deliberate prompt manipulations were performed.
- Only zero/few-shot prompting on pre-existing LLMs without fine-tuning; retraining or model modification effects remain unknown.
- Statistical self-consistency is necessary but not sufficient for correctness or utility; failures do not directly imply poor downstream task performance.
Open questions / follow-ons
- Can training or fine-tuning explicitly optimize for statistical self-consistency to improve LLM calibration and aggregate estimate accuracy?
- What is the impact of self-consistency violations on downstream reasoning, decision-making, or forecasting tasks relying on conditional inference?
- How do prompt formulations and natural language verbalizations affect the emergence and degree of the macro fallacy across diverse tasks?
- Can micro-to-macro or related implicit prompting approaches be generalized and standardized for practical, reliable aggregate inference with LLMs?
Why it matters for bot defense
For bot-defense and CAPTCHA practitioners, this paper underlines that LLM outputs often violate basic probability consistency constraints expected under a conditional inference interpretation. This means that relying on direct LLM probability estimates for decision thresholds or population-level risk assessments may yield misleading results. The macro fallacy finding suggests that careful decomposition of queries into finer-grained subpopulations, followed by aggregation, can yield more reliable estimates, a principle that could inform more robust challenge designs and risk scoring that depend on probabilistic user modeling. Furthermore, the micro-to-macro prompting method shows promise for extracting implicit statistical reasoning capabilities from LLMs within a single interaction, potentially enabling richer challenge logic and trust estimation without needing extensive partitioning. Overall, the evaluation framework introduced here offers a reference-free means to benchmark and understand LLM inference quality beyond conventional accuracy metrics, which could help security engineers gauge when and how to trust LLM signals in bot detection pipelines.
Cite
@article{arxiv2607_15277,
title={ Partition, Prompt, Aggregate: Statistical Self-Consistency in Language Models },
author={ Patrik Wolf and Thomas Kleine Buening and Andreas Krause and Celestine Mendler-Dünner },
journal={arXiv preprint arXiv:2607.15277},
year={ 2026 },
url={https://arxiv.org/abs/2607.15277}
}