Calibration Without Comprehension: Diagnosing the Limits of Fine-Tuning LLMs for Vulnerability Detection in Systems Software

Source: arXiv:2606.20502 · Published 2026-06-18 · By Arastoo Zibaeirad, Marco Vieira

TL;DR

This paper investigates whether fine-tuned large language models (LLMs) genuinely learn to reason about software vulnerabilities or simply calibrate their outputs without true comprehension. The authors introduce CWE-Trace, a rigorously curated dataset of 834 Linux kernel vulnerability samples spanning 74 CWE types, with a strict temporal split to separate pre-2025 historical data from post-cutoff leakage-free samples. They evaluate eight vanilla LLMs and 15 LoRA fine-tuned variants from three backbones across three tasks: non-targeted vulnerability detection, targeted detection, and CWE classification, using novel diagnostic metrics including the Directional Failure Index (DFI) and Hierarchical Distance and Direction (HDD). The study finds that data contamination does not give measurable gains, with 84% of nominally contaminated samples showing no usable memorization signals due to function-level distribution shift and label noise. Instead, stable backbone decision biases dominate model behavior, and fine-tuning mainly shifts output thresholds without changing underlying decision policies—termed "calibration without comprehension." Even the best detection accuracy barely exceeds chance (52.1%), while exact CWE classification remains below 1.3% Top-1 accuracy, demonstrating that current LLMs lack reliable security reasoning capabilities for systems software, regardless of fine-tuning.

Key findings

• The best vanilla LLM (CodeLlama) achieves only 52.1% detection accuracy, a mere +2.1 percentage points above random chance (50%).
• Directional Failure Index (DFI) values range widely from -85.5 to +94.8 percentage points, showing extreme bias toward false positives or false negatives stable across historical and leakage-free data.
• 84% of nominally contaminated samples have no usable memorization signal at the function level due to missing vulnerable functions or cross-mapping across datasets.
• ~31% of contaminated samples exhibit CWE misclassification, undermining contamination benefits.
• Fine-tuning mostly shifts the output threshold without changing the underlying decision policy, producing calibration without comprehension across backbones.
• DeepSeek-R1 backbone, despite weakest binary detection, gains the most from fine-tuning in coarse CWE classification, showing detection and understanding are decoupled.
• Exact CWE Top-1 classification accuracy remains below 1.3%, indicating negligible semantic depth improvement with fine-tuning.
• Context depth (multi-function vs single-function) does not significantly elevate detection performance, which remains near chance.

Threat model

The adversary corresponds to real-world software vulnerabilities embedded in Linux kernel source code, potentially unknown or unseen to the model. The LLMs attempt to detect and classify these vulnerabilities without reliance on memorized training data. Adversaries cannot directly manipulate input prompts or model weights in evaluation; the threat focuses on distinguishing genuine security reasoning from memorization or pattern matching.

Methodology — deep read

1. Threat Model and Assumptions: The paper does not define an explicit adversarial threat model but evaluates LLMs under zero-shot and fine-tuned conditions for vulnerability detection in Linux kernel code. The assumption is that the adversaries are malicious code flaws hidden in real systems software and models attempt to identify and classify them without relying on memorization of leaked training data.

2. Data: CWE-Trace is built from 834 manually curated samples extracted from Linux kernel CVE commits. These consist of 417 vulnerable–patched code pairs spanning 74 CWE types from the CWE-1000 hierarchy. The dataset is split temporally into a Primary Benchmark Dataset (PBD) with pre-2025 vulnerabilities (628 samples) and a Leakage-Free Dataset (LFD) with 2025+ CVEs (206 samples), ensuring strict temporal separation to avoid contamination. Samples preserve multi-function and multi-file context where fixes cross functions or files. Two reviewers validated each sample, removing non-security changes, contextual gaps, and singleton CWEs (no reliable classification signal). Sample-level details such as average lines and functions per sample are provided. This function-level granularity and context-awareness address shortcomings of prior datasets that isolate single functions or contain label noise.

3. Architecture / Algorithm: Eight vanilla LLMs are evaluated, covering code-specialized and general-purpose families, with parameter sizes from 4B to 32B and context windows from 16K to 1M tokens. Additionally, three of these backbones (Qwen3-4B, DeepSeek-R1-32B, Llama3.1-8B) are fine-tuned via parameter-efficient LoRA adapters on five state-of-the-art vulnerability datasets (Devign, LineVul, PrimeVul, MegaVul, VDISC). LoRA training applies low-rank updates (rank 16) to all attention and feedforward projections. The architecture remains fixed; novelty lies in the diagnostic evaluation framework (CWE-Trace) and metrics rather than modeling.

4. Training Regime: LoRA fine-tuning uses learning rate 2e-4 with cosine decay, 5% warmup, weight decay 0.01, gradient clipping 1.0, batch size 64, and zero dropout, running to convergence within 1 epoch. Oversampling is applied to vulnerable examples in imbalanced datasets. The same three-task objective is applied consistently: binary vulnerability detection, targeted detection with CWE hints, and CWE ranking when labels exist. Devign, lacking CWE labels, is trained only on binary detection.

5. Evaluation Protocol: Models are evaluated zero-shot (vanilla) or fine-tuned on PBD and LFD, measuring accuracy on three tasks: non-targeted detection, targeted detection, and CWE classification. Abstentions are allowed to reflect uncertainty. Metrics include standard accuracy (abstentions counted as errors), coverage, Directional Failure Index (DFI) to measure model bias toward false positive or negative errors, Contextual Gain Ratio (CGR) assessing benefit from CWE hints, Top-k accuracy and Mean Reciprocal Rank (MRR) for CWE classification, and Hierarchical Distance and Direction (HDD) capturing taxonomic error structure. The temporal split tests generalization to unseen future vulnerabilities without contamination. Ablations compare single-function (L1) vs multi-function (L2) samples, contaminated vs clean samples at function level, and label-space overlap with fine-tuning sets.

6. Reproducibility: The paper references pinned HuggingFace commits for each model checkpoint to ensure inference replicability. Fine-tuning details and hyperparameters are fully provided. The CWE-Trace dataset and evaluation scripts are presumably available (not explicitly stated). The benchmark and metrics are designed to be language-agnostic. However, some datasets used for fine-tuning are closed or partially accessible. Overall, the study emphasizes clean contamination controls and rigorous dataset curation.

Concrete example: A vulnerable Linux kernel sample containing a Use-After-Free (CWE-416) bug spanning multiple interacting functions and a header file is included in CWE-Trace. A model receives the unified multi-file code block zero-shot and predicts vulnerability presence and correct CWE label. Despite fine-tuning on overlapping datasets, the model displays a directional bias failing to move beyond chance performance, highlighting failure to comprehend underlying vulnerability mechanisms even with contextual information.

Technical innovations

• Introduction of CWE-Trace, a manually curated, context-aware vulnerability benchmark with strict temporal leakage control and multi-function/multi-file patches.
• Development of two diagnostic metrics: Directional Failure Index (DFI) quantifying directional decision biases and Hierarchical Distance and Direction (HDD) measuring semantic error structure using CWE-1000 hierarchy.
• Comprehensive multi-backbone LoRA fine-tuning study isolating effects of dataset contamination, label-space coverage, and backbone directional priors on vulnerability detection and classification.
• Function-level contamination and distribution shift analysis mechanistically explaining why CVE-level overlap does not translate into memorization or improved performance.

Datasets

• CWE-Trace — 834 samples (417 vulnerable–patched pairs) — Manually curated Linux kernel CVEs with temporal split and multi-function context
• Devign — 27k samples — Public vulnerability dataset (binary detection only)
• LineVul — 189k samples — Public vulnerability dataset with CWE labels
• PrimeVul — 235k samples — Public vulnerability dataset with CWE labels
• MegaVul — 339k samples — Public vulnerability dataset with CWE labels
• VDISC — 1.29m samples — Public vulnerability dataset with CWE labels

Baselines vs proposed

• Semgrep pattern matcher: recall = 0.3% (PBD), 0.0% (LFD) vs LLMs Top model 52.1% accuracy
• GraphCodeBERT (≤512 token subset): F1 = 61.5% (PBD), 62.5% (LFD) vs LLMs full long-context max ~52%
• Vanilla CodeLlama: Non-targeted detection accuracy = 52.1% vs chance 50%
• Fine-tuned Qwen3-4B: accuracy improvement +27.5 to +41.5 pp mainly from overcoming abstention, not semantic learning
• DeepSeek-R1 fine-tuning: detection accuracy often degrades by up to −29.1 pp, but improves coarse CWE classification
• Exact CWE-1000 Top-1 classification accuracy remains below 1.3% for all LLMs and fine-tuning variants

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2606.20502.

Fig 1: A unified diff of CVE-2024-41010. The vulnerability (Use-After-

Fig 2: Overview of the CWE-Trace framework. The pipeline begins with the extraction and manual pairing of 834 Linux kernel samples (417 pairs) (§III-A),

Fig 3: RQ1 mechanism analysis. Panel (a): fine-tuning vs. backbone detection

Fig 4: Fine-tuning ∆Root-Micro@1 for coarse CWE-1000 / root-taxonomy classification (relative to vanilla backbone). Each lollipop represents the change

Fig 5: RQ3 semantic-depth analysis. Panel (a): per-model MRR ranking on PBD and LFD. Panel (b): MRR vs. HDD mean distance for LFD fine-tuned

Limitations

• Limited to Linux kernel C code vulnerability samples; other domains/languages not evaluated yet.
• Focus on zero-shot prompting and LoRA fine-tuning; full fine-tuning or other PEFT methods are unexplored.
• Fine-tuning done with standard hyperparameters without sensitivity analysis for rank or learning rate.
• Interpretability limited to diagnostic metrics; no detailed layer-wise or attention analyses.
• Contextual prompting techniques like chain-of-thought or retrieval augmentation not evaluated and could alter results.
• Small leakage-free test set (206 samples) limits statistical power for some partitioned analyses.

Open questions / follow-ons

• Can chain-of-thought prompting, retrieval augmentation, or other structured prompts help overcome backbone directional biases identified here?
• How would full fine-tuning or alternative parameter-efficient finetuning methods (e.g., adapters, prefix tuning) impact vulnerability reasoning compared to LoRA?
• Can larger-scale, multi-language context-aware datasets extend CWE-Trace methodology and improve generalization of vulnerability detection?
• What internal model behaviors or representations underlie the observed directional failure modes, and can they be mitigated?

Why it matters for bot defense

For bot-defense and CAPTCHA practitioners, this study underlines the critical risk of overestimating LLMs' capabilities in detecting nuanced security vulnerabilities through fine-tuning alone. The findings emphasize that current LLMs, even when fine-tuned on large vulnerability datasets, tend to calibrate output probabilities without truly comprehending underlying semantic flaws or precise CWE classifications. This suggests caution when applying LLMs for automated vulnerability triage in production systems where reliable detection and correct classification are essential. The diagnostic metrics (DFI and HDD) introduced here could inform more nuanced evaluation frameworks in bot-defense contexts where understanding decision bias and hierarchical error patterns matter. Also, the strict temporal leakage controls and context-aware extraction provide design principles for building contamination-resistant evaluation benchmarks in adversarial settings. While the paper focuses on vulnerability detection, the demonstrated decoupling between detection and semantic understanding is a cautionary lesson for deploying LLMs in other security-sensitive automated triage or challenge-response tasks.

Cite

@article{arxiv2606_20502,
  title={ Calibration Without Comprehension: Diagnosing the Limits of Fine-Tuning LLMs for Vulnerability Detection in Systems Software },
  author={ Arastoo Zibaeirad and Marco Vieira },
  journal={arXiv preprint arXiv:2606.20502},
  year={ 2026 },
  url={https://arxiv.org/abs/2606.20502}
}

Read the full paper

• arXiv:2606.20502 (abstract)
• arXiv:2606.20502 (PDF)