LACUNA: A Testbed for Evaluating Localization Precision for LLM Unlearning

Source: arXiv:2607.02513 · Published 2026-07-02 · By Matteo Boglioni, Thibault Rousset, Siva Reddy, Marius Mosbach, Verna Dankers

TL;DR

This paper addresses the critical problem of unlearning sensitive memorized data, specifically personally identifiable information (PII), from large language models (LLMs). While prior state-of-the-art unlearning methods focus on output-level metrics (whether models stop producing undesired responses), they do not verify if the knowledge is truly erased at the parameter level or just obfuscated. To tackle this, the authors present LACUNA, the first testbed with ground-truth parameter-level localization of injected PII knowledge via a novel masked continual pretraining approach on 1B and 7B parameter OLMo models. LACUNA enables direct evaluation of unlearning methods for localization precision, revealing that despite strong output-level metrics, existing SOTA methods are imprecise and vulnerable to resurfacing attacks. The paper also demonstrates that given perfect localization knowledge, a simple gradient-based unlearning approach can robustly erase knowledge and resist resurfacing, underscoring the importance of precise localization-based unlearning. LACUNA is released as a benchmark and evaluation framework to push research toward more robust, truly privatizing unlearning methods.

Key findings

• LACUNA enables injection of PII into dedicated parameters covering 5% of model weights, grouped by six non-overlapping masks, injected via masked continual pretraining in 1B and 7B OLMo models.
• Masked continual pretraining with PII maintains model general utility, with downstream task accuracy fluctuating minimally around pretrained baselines (Figure 3a).
• Extraction rates of PII memorization exceed 50% for key fields in the 7B model (e.g. email addresses), validating successful localized memorization (Figure 3b).
• Conventional SOTA unlearning methods (SimNPO, AlphaEdit, MemFlex) achieve strong output-level forgetting but localization precision AUC barely exceeds 0.5, indicating edits are not targeted to PII-storing weights (Figure 4b).
• OracleGrad, an unlearning method constrained to the ground-truth forget mask, achieves localization precision AUC of 0.915, showing the upper bound for localized unlearning.
• OracleGrad maintains forgetting, retention, and utility better than or comparable to SOTA methods with far higher parameter-level precision (Figure 4a).
• Resurfacing attacks reveal that AlphaEdit and MemFlex leak a large fraction of forgotten profiles, SimNPO leaks fewer, while OracleGrad leaks least, confirming precise unlearning reduces relearning risk (Figure 5).
• Per-parameter masking and injection via gradient masking is feasible at 7B scale with negligible gradient distribution shifts, enabling scalable controlled localization.

Threat model

An adversary aims to extract remembered PII from the language model, including potentially reconstructing forgotten sensitive information by probing with prompts or applying resurfacing attacks. The attacker cannot alter the training or masking process but may fine-tune the model on external held-out PII data to attempt recovery of erased knowledge. The model owner must remove memorized PII robustly so the adversary cannot recover it through direct querying or fine-tuning attacks.

Methodology — deep read

The authors introduce a novel two-phase training and evaluation pipeline focused on localized knowledge injection and unlearning evaluation. 1) Threat Model & Assumptions: The adversary attempts to extract specific memorized PII from the model post-unlearning, potentially aided by resurfacing attacks. They cannot directly modify the training process after injection but can fine-tune on held-out PII to revive forgotten knowledge. 2) Data: They use 1,200 synthetic PII profiles from the PANORAMA dataset, comprising emails, birth cities, phone numbers, and driver's licenses, mixed with a 4.3B-token subset of the OLMo-2 pretraining corpus. QA pairs are generated via templated questions to improve memorization and extraction. A forget set and retain set of profiles are constructed from six disjoint groups using a 70/30 train/validation split, with 200 profiles per field in forget and retain. Additional paraphrased prompts enable evaluation on unseen formulations. 3) Architecture/Algorithm: Models are OLMo-2 1B and OLMo-3 7B parameter Transformers. Masked continual pretraining is used to inject PII into specific parameters targeted by binary masks covering 5% of the model’s parameters per forget/retain group. During backpropagation, only masked weights are updated for corresponding samples; other gradients are zeroed. Masks exclude embeddings and normalization layers and target feedforward and attention layers per-parameter granularity, packed efficiently into 32-bit integers. Instruction tuning (LoRA on last two layers) is done post injection to facilitate PII QA extraction. Three unlearning methods are compared: SimNPO (reference-free preference-based gradient-based unlearning), AlphaEdit (localization-based editing targeting FFN output projections with null-space projections to preserve knowledge), and MemFlex (gradient-based localization plus modular weight limiting). OracleGrad is introduced as an ideal oracle with perfect location knowledge, restricting updates to ground-truth masked weights using gradient difference objectives. 4) Training Regime: Continual masked pretraining trains the model on the mixture with gradient masking. Instruction tuning is done for 10 epochs with a 70/30 train/validation split for selecting best checkpoint. Gradient masking and localized updates are implemented efficiently for distributed training at 7B scale. 5) Evaluation Protocol: Output-level metrics include Exact Memorization (EM), Extraction Strength (ES), and Probability (Prob), measured on forget, retain, and utility sets with paraphrased prompts. Localization precision is evaluated as ROC AUC in discriminating in-mask weights from out-of-mask weights based on weight change scores induced by unlearning. Resurfacing attacks fine-tune the unlearned models on held-out PII, measuring leakage of forgotten profiles over many prompt attempts. Hyperparameter tuning is done on separate held-out profiles. 6) Reproducibility: LACUNA artifacts including models, masks, forget/retain splits, and evaluation metrics are released publicly. The masking and injection approach is novel and designed to scale to large models. The paper provides detailed algorithms and appendices on mask design, scoring, and evaluation.

Technical innovations

• A scalable masked continual pretraining technique that injects synthetic PII into predefined 5% parameter subsets via per-parameter binary masks in 1B and 7B OLMo models.
• Creation of LACUNA, the first unlearning testbed with ground-truth parameter-level localization enabling direct evaluation of unlearning localization precision.
• Introduction of a localization precision evaluation metric based on ROC AUC to quantify how specifically unlearning updates target the true PII-storing parameters.
• An oracle gradient-based unlearning method that restricts updates solely to parameters known to store forgotten knowledge, demonstrating gains in robustness and precision.

Datasets

• PANORAMA — 1,200 synthetic PII profiles focused on emails, birth cities, phone numbers, and driver’s licenses — publicly released synthetic dataset for PII memorization research
• OLMo-2 pretraining corpus subset — 4.3 billion tokens — internal OLMo pretraining data

Baselines vs proposed

• SimNPO: forget EM ~74%, localization precision AUC = 0.515 vs OracleGrad forget EM ~75%, localization precision AUC = 0.915
• AlphaEdit: forget EM ~13%, localization precision AUC = 0.500 vs OracleGrad forget EM ~75%, localization precision AUC = 0.915
• MemFlex: forget EM ~23%, localization precision AUC = 0.500 vs OracleGrad forget EM ~75%, localization precision AUC = 0.915
• Pretrained (no unlearning): utility ~baseline vs SimNPO utility slightly reduced
• OracleGrad maintains high retention and utility close to pretrained baseline

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.02513.

Fig 1: Overview of our pipeline. Phase 1: PII data is mixed with pretraining data and

Fig 2 (page 1).

Limitations

• LACUNA uses synthetic PII, which may differ in complexity and memorability from real-world sensitive data.
• Evaluation focuses on OLMo architectures and 1B/7B model scales; generalization to other LLMs and larger scales requires study.
• The resurfacing attack tested is a straightforward finetuning approach; stronger or adaptive attacks may yield different robustness insights.
• OracleGrad requires privileged knowledge of parameter localization, which is unavailable in practice, so applying localization methods to approximate this remains challenging.
• The masking approach targets feedforward and attention parameters but excludes embeddings and norms due to complexity.
• Instruction tuning on limited PII profiles may not fully reflect open-domain question answering about private data.

Open questions / follow-ons

• How to develop practical and accurate localization methods approximating the oracle’s privileged parameter knowledge for real-world unlearning?
• Can the masked continual pretraining injection method extend effectively to larger, more complex LLMs such as GPT-style generative models?
• What are the impacts of stronger or adaptive resurfacing attacks on localization-based unlearning robustness?
• How can localization precision metrics be generalized or improved to include embeddings, normalization layers, or cross-layer parameter interactions?

Why it matters for bot defense

This paper provides a rigorous framework for understanding the parameter-level effects of unlearning in LLMs, which is highly relevant for bot-defense practitioners tasked with ensuring models do not leak sensitive private data. By highlighting that output-level behavioral metrics are insufficient to verify true erasure, LACUNA encourages deploying evaluation methods that measure precise parameter updates, a critical insight for assessing false negatives in unlearning. It also demonstrates the vulnerability of current SOTA methods to resurfacing attacks, underscoring the risk of privacy leakage despite apparent forgetting. Captcha and bot-defense engineers should consider incorporating localization precision evaluation alongside output metrics to gain a more complete picture of unlearning success and robustness, guiding safer deployment of language models that hold user data or proprietary secrets.

Cite

@article{arxiv2607_02513,
  title={ LACUNA: A Testbed for Evaluating Localization Precision for LLM Unlearning },
  author={ Matteo Boglioni and Thibault Rousset and Siva Reddy and Marius Mosbach and Verna Dankers },
  journal={arXiv preprint arXiv:2607.02513},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.02513}
}

Read the full paper

• arXiv:2607.02513 (abstract)
• arXiv:2607.02513 (PDF)