Neuron-Aware Active Few-Shot Learning for LLMs

Source: arXiv:2607.02423 · Published 2026-07-02 · By Zhuowei Chen, Liwei Chen, Christian Schunn, Raquel Coelho, Xiang Lorraine Li

TL;DR

This paper addresses the problem of selecting the most informative few-shot examples for adapting large language models (LLMs) to specialized domains with limited labeled data. Existing Active Few-Shot Learning (AFSL) approaches typically rely on output-level signals such as predictive entropy or external semantic embeddings to identify valuable samples from an unlabeled pool. However, these methods overlook rich internal model dynamics that can better reveal knowledge gaps and potential hallucinations. To overcome this, the authors propose NEUFS, a Neuron-Aware Active Few-Shot Learning framework that leverages neuron activation patterns within the feed-forward networks of LLMs. NEUFS employs a dual-criteria sample selection strategy combining clustering based on internal neuron activation patterns for diversity with a neuron consensus metric to identify hallucination-prone, informative samples. Through extensive experiments on three benchmark datasets spanning reasoning and text classification tasks, NEUFS consistently surpasses existing AFSL baselines across different LLM architectures and sizes, demonstrating higher accuracy and F1 scores. Ablations validate that internal neuron activations offer a more principled and effective selection signal than dense semantic embeddings or output-level uncertainty metrics.

Key findings

• NEUFS achieves highest accuracy of 0.452 on Qwen3-4B and 0.418 on Qwen3-8B on the MMLU-Pro reasoning benchmark, outperforming semantic and entropy-based methods (Table 1).
• On classification datasets Edu-Feedback and TREC with 8B models, NEUFS outperforms all baselines in macro-F1 and accuracy metrics, e.g. 0.862 accuracy on TREC with Qwen3-8B (Table 2).
• For smaller 3B and 4B models, NEUFS consistently improves F1 and accuracy, such as 0.692 F1 on Edu-Feedback with Qwen3-4B, beating external embedding-based approaches (Table 3).
• Neuron activation pattern representations outperform dense encoder (SimCSE) and decoder (Qwen embedding) based dense vectors in few-shot selection effectiveness (Table 4).
• NEUFS is robust to neuron sparsity hyperparameter k between 2000 and 10,000, with stable performance on larger 8B models (Figure 3).
• The weighting parameter τ that balances neuron consensus and cluster representativeness exhibits model size-dependent effects: smaller models benefit more from representativeness while larger models gain from combining both (Figure 4).
• Neuron consensus (number of unique activated neurons) inversely correlates with hallucination risk, guiding informative sample selection.
• Existing output-level uncertainty metrics like entropy are less reliable due to LLM overconfidence and misaligned objectives.

Threat model

The adversary scenario is neutral/non-applicable here, as the work focuses on improving sample selection for few-shot learning rather than adversarial attacks or defenses. The framework assumes access to the LLM's internal neuron activations and unembedding weights to extract activation patterns. The practitioner cannot access black-box API internals and must have white-box model availability.

Methodology — deep read

The core idea is to replace conventional output-level and external embedding-based signals for sample selection in AFSL with internal neuron activation patterns extracted from the LLM. The threat model assumes a practitioner with access to an open-weight LLM and a large unlabeled data pool from a specialized domain. The goal is to select few-shot samples to be annotated that best improve test-time performance with minimal cost.

Data: Experiments are conducted on three public datasets: MMLU-Pro (12,034 samples, multi-domain advanced reasoning), Edu-Feedback (1,799 training + 14,228 testing, binary edu feedback classification), and TREC (5,452 training + 500 testing, 6-way question classification).

Algorithm: For each candidate sample, neuron activations are harvested from feed-forward network (FFN) layers within the transformer's blocks during model generation. Using 'early unembedding,' neuron contribution to the predicted tokens is quantified by projecting neuron outputs back to vocabulary embedding space. Neurons are ranked by contribution score; top-k neurons are selected to represent sparse activation patterns for the sample, forming sets of activated neurons Nact.

These sets form discrete features used for two key components: 1) Neuron-Aware Sample Diversification via K-Medoids clustering with Jaccard distance on neuron activations to group samples covering diverse internal knowledge circuits, 2) Neuron Consensus Quantification by counting unique activated neurons, serving as a proxy for hallucination risk and informativeness. Lower consensus samples indicate higher hallucination propensity and thus more informative when selected.

A dual-criteria scoring balances cluster representativeness (distance to cluster medoid) and neuron consensus, controlled by hyperparameter τ, to rank and select samples from each cluster.

Training/Evaluation: Evaluations use instruction-tuned LLMs (Llama3 3B/8B, Qwen3 4B/8B). Different numbers of shots (5,10,20,30) are tested, averaged over three runs with varying seeds. Baselines include random, entropy-based uncertainty selection, semantic-only selection (TypiClust), combined semantic+uncertainty methods (Patron, VoteK), and graph-based diversity (FastVoteK).

Metrics: Accuracy, macro-F1 and overall aggregated performance are reported per dataset. Ablations examine neuron sparsity k and τ hyperparameters.

Reproducibility: Code is publicly released. The datasets used are public or contain train/test splits described. Neuron extraction and scoring use internal LLM weights, restricting applicability to open-weight models only. No adversarial or distribution shift evaluations are presented.

Concrete example: For a sample question, NEUFS retrieves neuron activations from FFNs, performs early unembedding to select top-k contributing neurons, forms a sparse activation pattern. Samples are clustered by these patterns. Each sample's unique activated neuron count is computed; combining this score with the cluster distance score yields a final rank used to select representative and hallucination-prone examples for annotation.

Technical innovations

• Introduction of using internal neuron activation patterns from FFNs as primary representations for active few-shot learning sample selection, rather than output or external embeddings.
• A dual-criteria sample selection strategy combining neuron activation pattern-based clustering for diversity with a neuron consensus metric to identify samples prone to hallucination and model errors.
• Early unembedding method to quantify neuron contribution scores to final predictions for selecting valid activated neurons relevant for knowledge invocation.
• Demonstration that neuron consensus (unique activated neuron count) inversely correlates with hallucination risk, enabling principled uncertainty estimation.
• Empirical validation on multiple datasets and LLM sizes showing neuron-based signals outperform traditional semantic or entropy uncertainty based AFSL methods.

Datasets

• MMLU-Pro — 12,034 samples — public reasoning benchmark spanning 14 domains
• Edu-Feedback — 16,027 samples (1,799 training, 14,228 testing) — educational feedback quality classification
• TREC — 5,952 samples (5,452 training, 500 testing) — 6-way question classification

Baselines vs proposed

• Random: accuracy = 0.388 vs NeuFS: 0.418 on MMLU-Pro with Qwen3-8B
• Patron (Semantic+Entropy): 0.416 vs NeuFS: 0.418 on MMLU-Pro with Qwen3-8B
• TypiClust (Semantic): 0.398 vs NeuFS: 0.418 on MMLU-Pro with Qwen3-8B
• Entropy Highest: 0.394 vs NeuFS: 0.418 on MMLU-Pro with Qwen3-8B
• Random: macro-F1 = 0.645 vs NeuFS: 0.660 on Edu-Feedback with Llama3 8B
• Patron: 0.643 vs NeuFS: 0.660 macro-F1 on Edu-Feedback with Llama3 8B
• Random accuracy = 0.823 vs NeuFS: 0.858 on TREC with Qwen3 8B
• Dense embeddings (Qwen embedding): 0.4131 average accuracy vs NeuFS (Neuron): 0.4178 on MMLU-Pro
• Dense encoder embeddings (SimCSE): 0.4013 average accuracy vs NeuFS: 0.4178 on MMLU-Pro

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.02423.

Fig 1: Neuron-Aware AFSL, which identifies infor-

Fig 2: Overview of the proposed NEUFS. For each candidate, we extract the activation values from the FFN in

Fig 3 (page 1).

Fig 4 (page 1).

Fig 5 (page 1).

Fig 6 (page 1).

Fig 7 (page 1).

Fig 8 (page 3).

Limitations

• Requires access to internal FFN activations and unembedding matrices in LLMs, limiting use to open-weight models, not black-box APIs.
• Two-stage filtering reduces overhead but computing neuron activations over large unlabeled pools remains computationally expensive compared to static embeddings.
• No evaluation under adversarial attacks or distribution shifts to verify robustness of neuron-based signals.
• Performance sensitivity to hyperparameters like neuron sparsity k and scoring weight τ varies especially for smaller models.
• Approximate interpretation of neuron consensus as hallucination proxy may not capture all failure modes or domains.
• Experiments confined to instruction-tuned Llama3 and Qwen3 models; generalizability to other LLM architectures unclear.

Open questions / follow-ons

• How well does neuron consensus generalize as a hallucination proxy across different LLM architectures and domains?
• Can neuron-aware AFSL methods be adapted for black-box LLM APIs without internal access via proxy models or other techniques?
• What are the computational efficiency trade-offs in scaling neuron activation extraction and clustering for very large unlabeled pools?
• Could combining neuron activations with external embeddings or fine-grained gradient signals further improve AFSL sample selection?

Why it matters for bot defense

This work offers a novel perspective for bot-defense and CAPTCHA practitioners interested in leveraging internal model signals for improved data selection and model adaptation. NEUFS’s neuron activation pattern-based framework could inform strategies to identify edge cases or uncertain inputs that models handle poorly, guiding annotation efforts more efficiently. The findings caution against relying solely on output-level uncertainty or external embeddings for selecting representative or challenging query samples. While NEUFS currently requires white-box access to model internals limiting applicability in black-box API scenarios common in production, the underlying principle of tapping into internal neuron dynamics presents a promising direction for better-informed active learning in robust CAPTCHA and bot detection systems. Captcha-related models focused on language understanding or classification could benefit from internal-signal-driven example selection to reduce annotation cost and improve coverage of failure cases. However, practical deployment would need adaptations to handle API constraints and evaluate computational overhead trade-offs.

Cite

@article{arxiv2607_02423,
  title={ Neuron-Aware Active Few-Shot Learning for LLMs },
  author={ Zhuowei Chen and Liwei Chen and Christian Schunn and Raquel Coelho and Xiang Lorraine Li },
  journal={arXiv preprint arXiv:2607.02423},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.02423}
}

Read the full paper

• arXiv:2607.02423 (abstract)
• arXiv:2607.02423 (PDF)