CoGate: Confidence-Gated Co-Decoding for Secure Code Generation

Source: arXiv:2607.28529 · Published 2026-07-30 · By Minghao Hu, Lannan Luo, Allen Roush, Phillip Howard

TL;DR

This paper addresses a crucial limitation in existing co-decoding approaches for secure code generation with large language models (LLMs). While prior methods like CoSec+ combine a large base model with a smaller security expert model per decoding step, their acceptance rules do not consider the expert's confidence, which leads to pathological behaviors—particularly in out-of-distribution (OOD) contexts where the expert is uncertain and may introduce noise rather than secure guidance. The authors propose CoGate, a confidence-gated co-decoding mechanism that measures the expert's confidence (via max token probability or normalized entropy) and allows expert intervention only when confidence exceeds a threshold. This decouples the expert's absolute confidence from the relative token preference, mitigating misleading low-confidence guidance. Evaluated across six LLM backends (CodeGen, StarCoder, DeepSeek-Coder, Qwen2.5-Coder) on three benchmarks (HumanEval, Security Suite, CWEval), CoGate consistently outperforms previous methods, including CoSec+, particularly on the OOD, multi-language CWEval benchmark for simultaneous program correctness and security. It achieves up to 12.6% absolute improvement in the Func-Sec@10 metric. The core contribution is a theoretically motivated, training-free inference-time gating technique that selectively filters expert interventions to enhance secure and functionally correct code generation.

Key findings

• CoGate achieves up to 12.6% absolute gain on Func-Sec@10 on the CWEval benchmark compared to CoSec+ (Table 1).
• On StarCoder-7B, CoGate improves Pass@10 from 51.6% (CoSec+) to 54.5%, and Security Ratio from 77.2% to 82.6%.
• Confidence gating based on max-probability or normalized entropy effectively prevents low-confidence expert proposals from harming generation (Section 5.2, Fig 2).
• Optimal gate thresholds lie between 0.5 and 0.7 for max-probability signals, balancing false accept and false reject errors (Section 5.2).
• Normalized entropy gating better distinguishes OOD scenarios and provides superior gains on unseen CWEs in CWEval (Section 5.3).
• CoGate consistently outperforms LoRA fine-tuning of the base model and prior co-decoding method CoSec+ across six LLMs and three benchmarks (Table 1).
• As temperature increases, CoGate’s advantage over CoSec+ grows because CoGate gates out noisy low-confidence expert interventions (Section 5.4).
• The faulty acceptance behavior in CoSec+ that favors low-confidence expert tokens is formally analyzed and proven (Propositions 1 and 2).

Threat model

The adversary is implicit as distribution shifts or OOD inputs leading the expert model to produce uncertain token distributions. This uncertainty can mislead co-decoding acceptance rules by injecting insecure tokens. The adversary is not directly modeled but manifests as ambiguous or unseen code contexts where the base and expert differ significantly. The co-decoding framework assumes the base model cannot be modified or retrained to incorporate security knowledge; only inference-time interventions using a frozen expert model are feasible.

Methodology — deep read

The paper tackles the problem of secure code generation by combining a large base code LLM with a smaller security expert LLM at decoding time, a paradigm known as co-decoding. The key insight is that existing co-decoding acceptance rules (e.g., CoSec+) select expert tokens based on their relative probabilities compared to the base model without factoring in the expert’s absolute confidence, resulting in increased acceptance of low-confidence expert proposals that inject noise—particularly on OOD inputs.

Threat Model & Assumptions: The adversary is implicit via OOD or ambiguous inputs that cause the expert model to have low confidence in its next-token distribution. The models are pretrained on large public code datasets containing insecure patterns, so the goal is to steer generation towards secure, functionally correct code without retraining the base model.

Data: Evaluation uses three benchmarks: HumanEval (functional correctness), Security Suite (in-distribution CWE types seen by expert), and CWEval (multilingual functional correctness & security with OOD CWEs unseen during training). Multiple LLM families are tested: CodeGen (2.7B,6.1B), StarCoder (1B,7B), DeepSeek-Coder (6.7B), Qwen2.5-Coder (14B). Expert models are smaller versions (350M to 1.5B) trained by knowledge distillation from the base and further fine-tuned on security-relevant corpora.

Architecture / Algorithm: CoGate adds a confidence gate to existing co-decoding. At each decoding step t, given prefix x<t, both base model B and expert S produce next-token distributions Bt and St. The expert proposes token x̃ sampled from St. A confidence functional c(St) is computed, either maximum token probability or normalized (Shannon) entropy. The gate is open (gt = 1) if c(St) > τ, a threshold hyperparameter; otherwise, gt=0.

If gate is open, acceptance rule follows CoSec+: accept x̃ if a < min(1, Bt(x̃)/St(x̃)) with a fixed threshold a=0.3. If rejected or gate closed, sample from base Bt. This selects tokens where the expert is confident and relatively prefers the token, separating absolute confidence from relative preference.

Training Regime: Expert models derive from the base model via knowledge distillation and post-training on a security dataset as in CoSec+. Base models remain frozen. LoRA fine-tuning is applied to some baselines.

Evaluation Protocol: Metrics are Pass@1/10 for HumanEval; Security Ratio for Security Suite; Func-Sec@1/10 for CWEval (correct + secure). Results are averaged over 25 samples per prompt with fixed decoding hyperparameters (top-p=0.95, temperature=0.6, max 256 tokens). Threshold τ is chosen per model family on held-out splits maximizing CWEval Func-Sec@10. Token-level counterfactual intervention experiments compare CoGate, unconfident CoSec+, and base-only continuations.

Reproducibility: Exact code and models are not stated to be publicly released but evaluation uses open benchmark datasets. Model specifications and hyperparameters are detailed.

End-to-End Example: For a decoding step, given base Bt and expert St distributions, CoGate first calculates confidence c(St). If c(St) > τ, expert draws candidate token x̃ from St, which is accepted only if a < min(1, Bt(x̃)/St(x̃)); else falls back to base sampling. This removes expert interventions when expert confidence is low, thus preventing noisy guidance on uncertain inputs.

Technical innovations

• Identification and formal proof that existing co-decoding acceptance rules inversely correlate acceptance probability with expert confidence, causing noise injection on uncertain expert distributions.
• Introduction of a confidence gate mechanism that conditions expert intervention on an absolute confidence metric (max token probability or normalized entropy), separating reliability from token preference.
• Design of a lightweight inference-time gating rule that interpolates between always accepting expert proposals and never using the expert, controlled by a single threshold hyperparameter τ.
• Comprehensive token-level intervention utility analysis quantifying the trade-offs between false accept and false reject errors in this gated co-decoding framework.

Datasets

• HumanEval — ~164 programming problems with test suites — public
• Security Suite (from SVEN) — security-oriented code generation samples and CWE types — derived from expert training corpus
• CWEval — large multilingual (C, C++, Go, JS, Python) dataset with executable oracles evaluating correct and secure generation — public

Baselines vs proposed

• Original base models: For StarCoder-7B on CWEval Func-Sec@10 = 29.6% vs. CoGate (Entropy) = 35.2%
• LoRA fine-tuned base models: On StarCoder-7B Func-Sec@10 = 31.7% vs CoGate (Entropy) 35.2%
• CoSec+ prior co-decoding: On StarCoder-7B Func-Sec@10 = 32.8% vs CoGate (Entropy) 35.2%
• Qwen2.5-Coder-14B: Original Func-Sec@10 = 41.0%, CoSec+ = 44.2%, CoGate (Entropy) = 56.8%
• CodeGen-6.1B: CoSec+ Pass@10 = 49.8%, CoGate (Max-Prob) = 52.2%, CoGate (Entropy) = 52.7%
• CoGate outperforms CoSec+ and LoRA fine-tuning consistently across models and benchmarks.

Limitations

• CoGate depends on a heuristic confidence threshold τ that requires tuning per model family and task; sensitivity outside of studied ranges is unclear.
• The method assumes availability of a suitably secure and calibrated expert model, which may be costly to train or unavailable for new domains.
• Evaluation focuses notably on existing CWEs and benchmark suites; broader robustness to truly novel vulnerabilities or adversarial attacks is not assessed.
• Token-level gating may introduce latency due to per-step confidence calculations, though authors claim overhead is negligible.
• The approach does not address multi-token or sequence-level confidence correlations, potentially missing broader context signals.
• No public code release or pretrained expert models limits reproducibility and adoption.

Open questions / follow-ons

• How to automatically and dynamically tune the confidence threshold τ per instance or during decoding to further improve gating performance?
• Whether sequence-level confidence models can better capture expert reliability across multiple tokens rather than per-token gating.
• Extension of the confidence-gated co-decoding framework to adversarially crafted inputs or active attacks aiming to exploit the gating mechanism.
• Applying CoGate to other controllable generation tasks beyond secure code, such as fairness, bias, or toxicity steering.

Why it matters for bot defense

For bot-defense and CAPTCHA practitioners focusing on secure code generation and model robustness, CoGate offers a compelling inference-time method to mitigate risks of unreliable expert model guidance during generation. Its confidence-gated control mechanism elucidates the importance of explicitly measuring auxiliary model confidence rather than relying on relative token preferences alone. This insight can inform design of multi-model or multi-component defense pipelines, where the gating of signals from specialized expert subsystems (e.g., security analyzers or anomaly detectors) might mitigate failures caused by uncertainty or domain shifts. The approach also highlights the trade-offs between false accepts versus false rejects in gating auxiliary components, key to balancing bot-defense accuracy and user experience. Although the paper focuses on code generation, its emphasis on calibrated gating and selective intervention is broadly applicable wherever ensemble or co-decoding systems guide sensitive output under uncertainty.

Cite

@article{arxiv2607_28529,
  title={ CoGate: Confidence-Gated Co-Decoding for Secure Code Generation },
  author={ Minghao Hu and Lannan Luo and Allen Roush and Phillip Howard },
  journal={arXiv preprint arXiv:2607.28529},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.28529}
}

Read the full paper

• arXiv:2607.28529 (abstract)
• arXiv:2607.28529 (PDF)