Super Weights in LLMs and the Failure of Selective Training
Source: arXiv:2607.08733 · Published 2026-07-09 · By Shreyas Subramanian, Adewale Akinfaderin, Akarsha Sehwag
TL;DR
This work addresses the question of whether "Super Weights"—individual model parameters whose removal drastically harms language model performance—can be selectively fine-tuned in isolation for efficient adaptation. Prior work identified Super Weights by their criticality to model outputs, suggesting they might form a natural sparse update target for parameter-efficient fine-tuning (PEFT). However, this paper disproves this hypothesis through extensive experiments on OLMo-1B and OLMo-7B models. Training isolated Super Weights, even with expanded neighborhoods up to tens of thousands of parameters, consistently degrades model accuracy to random-guessing levels, despite decreasing training loss. In contrast, updating randomly selected sparse coordinates or applying LoRA-style low-rank full-layer updates achieves accuracy gains over the baseline. The study validates that Super Weights constitute stable structural properties across inputs and models but are not trainable effectively on their own. The key insight is that parameter importance (measured by pruning impact) does not imply trainability in isolation—effective fine-tuning requires coordinated, structured updates across entire layers.
Key findings
- Training only Super Weights (100 to 8,192 parameters) leads to ARC-Easy accuracy dropping to near random guessing (∼25%) on both OLMo-1B and OLMo-7B, with no improvement as more positions are added.
- Expanding training to local neighborhoods around Super Weights (up to 36,864 parameters) does not improve accuracy and sometimes causes unstable training (e.g., loss increasing to 202 on OLMo-1B).
- Training an equal number of randomly chosen positions in down proj layers (excluding Super Weights) yields 64.18% accuracy on ARC-Easy, exceeding the 60.65% baseline.
- LoRA updating all attention projection weights (∼0.16% parameters via rank-8 low-rank updates) achieves 66.88% accuracy on OLMo-1B and 77.3% on OLMo-7B, improving significantly over baseline.
- Freezing LoRA updates specifically at attention positions that correspond to Super Weights in down proj layers (freeze up to 6.8% of LoRA params) does not degrade performance significantly (p > 0.05 across 10 seeds).
- Freezing the top-1,000 highest-magnitude parameter positions in the trained LoRA updates yields statistically indistinguishable accuracy versus vanilla LoRA (62.77% vs 62.72% on OLMo-1B).
- Pruning Super Weights increases perplexity drastically (up to 4,400× on OLMo-7B) and reduces ARC-Easy accuracy to near chance, confirming their criticality but also showing this effect is model- and activation-spike specific.
- Super Weight positions change by 8.19% on average during direct training but only 0.15% in LoRA updates, indicating LoRA performs stable, coordinated changes while direct training causes disruptive, uncoordinated updates.
Threat model
The threat model assumes an adversary or practitioner with full access to the pretrained LLM parameters and the ability to identify Super Weights—parameters whose removal wreaks havoc on model performance. The adversary attempts parameter-efficient fine-tuning by restricting updates solely to these highly important parameters or their neighborhoods, under the assumption that such sparsity would yield effective adaptation. The adversary cannot perform full model fine-tuning or coordinated low-rank updates spanning entire layers, representing a constrained PEFT scenario focused on sparse coordinate updates at known critical positions.
Methodology — deep read
The study investigates the relationship between parameter importance (Super Weights) and trainability through six meticulously designed experiments on two large language models, OLMo-1B (1.28B parameters, 16 layers) and OLMo-7B (7B parameters, 32 layers).
The threat model considers an adversary or practitioner with the ability to identify Super Weights either via magnitude or activation-spike-based methods and asks whether selective training restricted to these coordinates improves or degrades performance. The authors assume knowledge of model parameters but restrict training updates to small subsets.
Data for evaluation consists mainly of ARC-Easy (around 2,370 test samples) as a multiple-choice benchmark and Winogrande for validation. Super Weights are identified primarily from down projection matrices (MLP layers) by analyzing activation spikes across 1,000 samples from WikiText-2. Pruning replication is performed across 10 diverse models for generalization.
The main architectural focus is on down proj layers where Super Weights concentrate, and attention projection matrices (q, k, v, o proj) targeted by LoRA. Training methods tested include direct sparse training of Super Weights (and neighborhoods), sparse random coordinate training controls, vanilla LoRA with rank-8 updates, and variants applying freezes on Super Weight positions within LoRA updates.
Optimization uses AdamW with a 1e-4 learning rate over 3 epochs, effective batch size 16. LoRA uses rank r=8 and alpha=16 for scaling, resulting in 0.16% trainable parameters. Ten random seeds assess statistical significance.
Evaluation metrics are exact match accuracy for multiple-choice tasks and perplexity for language modeling. Ablations compare single-seed pruning, direct training, neighborhood expansion, and full-layer LoRA variants.
Reproducibility is supported by parameter and seed details, but code and weights are not explicitly stated as released. The training steps are well-documented, with an end-to-end example showing isolated Super Weight training fails as measured by accuracy collapse and exploding validation perplexity, while LoRA training improves accuracy stably through coordinated low-rank updates.
In summary, the methodology rigorously controls for sparsity, module choice, and training budget, carefully contrasting isolated coordinate training against structured low-rank layer-wise adaptation to uncover the fundamental difference between parameter importance and trainability.
Technical innovations
- Demonstration that training isolated Super Weights—parameters critical under pruning—fails catastrophically, decoupling parameter importance from parameter trainability in isolation.
- Introduction of local neighborhood expansion around Super Weights as a control, showing that even contextually expanded sparse training fails to improve performance.
- Design of LoRA variants constraining updates at positions corresponding to Super Weights, experimentally confirming position-level freezing does not impair fine-tuning success.
- Theoretical analyses revealing that sparse coordinate-aligned updates only span a vanishing fraction of the fine-tuning subspace and that optimizer curvature suppresses updates at Super Weight positions.
- Empirical demonstration that full-layer, low-rank structured updates achieve superior adaptation by coordinating parameter changes across entire weight matrices rather than isolated elements.
Datasets
- ARC-Easy — 4,627 total QAs (2,251 train / 2,376 test) — Public benchmark
- Winogrande — 10,515 total QAs (9,248 train / 1,267 test) — Public benchmark
- WikiText-2 — 1,000 random samples for Super Weight consistency — Public benchmark
Baselines vs proposed
- Pretrained baseline (OLMo-1B): ARC-Easy accuracy = 60.65% vs direct Super Weight training (100 SW): ∼25%
- Random sparse training (k=4,096 in down proj): accuracy = 64.18% vs direct Super Weight training (k=4,096): ∼26%
- Vanilla LoRA (rank-8, attention projections): accuracy = 66.88% vs pretrained baseline: 60.65%
- LoRA-dproj-SW-freeze (scale s=0.0): accuracy = 66.50% vs vanilla LoRA: 66.88% (ns)
- LoRA-∆W-SW-freeze (freeze top 1,000 update positions): accuracy = 62.77% ± 0.62% vs vanilla LoRA: 62.72% ± 0.62%
- Pruning Super Weights on OLMo-7B: perplexity increases 4,400×; ARC-Easy accuracy drops from 73.3% to 60.5%
Figures from the paper
Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.08733.
Fig 1: The three LoRA variants. All operate on attention projections (q/k/v/o proj) and
Fig 2: Expanding from isolated Super Weights to neighborhoods of 100, 1,000, or 36,864
Fig 3: Per-seed accuracy for vanilla LoRA and LoRA-dproj-SW-freeze (OLMo-1B, seeds
Limitations
- Experiments primarily on ARC-Easy; results on harder datasets like MMLU or GSM8K were not conducted, limiting task generality.
- Random-position sparse training control only tested with a single seed and single parameter budget on OLMo-1B, limiting robustness claims.
- Activation-spike-based Super Weight identification used mainly from Yu et al. (2025); alternative identification methods and non-OLMo model families were less explored for training experiments.
- Evaluation is limited to PEFT methods centered on LoRA and direct training; other PEFT approaches like adapters or prompt tuning were not evaluated.
- Code and model checkpoints were not explicitly released, potentially hindering exact experimental reproduction.
- Seed ablation was conducted only for LoRA variants, not for sparse isolated training methods.
Open questions / follow-ons
- Can alternative PEFT methods beyond LoRA, such as adapters or prompt tuning, leverage Super Weights effectively or avoid the isolation failure observed here?
- How does the failure of isolated Super Weight training extend to larger models and more diverse downstream tasks, especially those requiring complex reasoning (e.g., MMLU, GSM8K)?
- Can improved sparse training schemes that introduce explicit coordination or adaptive neighborhood selection overcome the trainability limitations of isolated Super Weights?
- Do other identification methods for important parameters, beyond activation spikes or magnitude, yield different subsets of parameters that are trainable in isolation?
Why it matters for bot defense
For bot-defense engineers and CAPTCHA practitioners examining PEFT methods to efficiently adapt large language models for user interaction and bot-detection tasks, this paper highlights a critical insight: targeting only the individually most important parameters (Super Weights) for training is ineffective and degrades model performance severely. Efficient adaptation requires coordinated, structured updates distributed across entire layers rather than sparse, targeted tuning at critical coordinates. This understanding advises against simplistic sparse fine-tuning approaches that rely solely on importance scores for parameter selection. Instead, methods like LoRA that update low-rank factorizations spanning full layers provide robust, trainable, and stable model adaptation. When designing defense systems leveraging LLMs, practitioners should emphasize structured PEFT methods maintaining relational integrity over isolated sparse updates, as the latter risk catastrophic failure despite apparent parameter importance.
Cite
@article{arxiv2607_08733,
title={ Super Weights in LLMs and the Failure of Selective Training },
author={ Shreyas Subramanian and Adewale Akinfaderin and Akarsha Sehwag },
journal={arXiv preprint arXiv:2607.08733},
year={ 2026 },
url={https://arxiv.org/abs/2607.08733}
}