Batch-Invariant Spectral Intelligence for Robust and Explainable Insect Authentication
Source: arXiv:2606.26757 · Published 2026-06-25 · By Majharulislam Babor, Giacomo Rossi, Annalisa Altavilla, Oliver Schlüter, Marina M. -C. Höhne
TL;DR
This paper addresses the challenge of reliably authenticating insect species for edible insect production using near-infrared (NIR) spectroscopy under realistic industrial batch-to-batch variability. While NIR spectroscopy offers a fast, non-invasive method for insect species discrimination, spectral measurements vary substantially across production batches due to environmental, instrumental, and processing effects (e.g., blanching, plasma-activated water, ultrasound). These batch effects confound species identification models trained on limited batches, causing accuracy drops on unseen batches. The authors introduce the Batch-Invariant Spectral Network (BISN), an end-to-end deep learning framework that incorporates a learnable preprocessing module initialized with Savitzky-Golay filtering and uses an entropy-regularized adversarial training objective to remove batch-specific spectral variation before downstream feature extraction. BISN differs from prior domain-adversarial approaches by enforcing batch invariance upstream, directly on the spectral input representation. BISN is evaluated on a novel dataset of 2,700 samples of three edible insect species from three independent production batches, subjected to multiple processing treatments. It achieves a mean leave-one-batch-out classification accuracy of 0.93±0.04, outperforming the best baseline by 4% (p < 10^-6). Explainable AI techniques link BISN’s spectral attributions to lipid and protein absorption regions, grounding the model’s decisions in insect biochemistry. BISN thus demonstrates improved cross-batch robustness and biochemical interpretability for industrial-scale edible insect authentication.
Key findings
- BISN achieves mean leave-one-batch-out accuracy of 0.93 (std 0.04) on 2,700 spectra across 3 insect species and 3 batches.
- BISN outperforms the strongest baseline by 4% accuracy with p < 10^-6 significance.
- Batch-specific spectral variation is suppressed upstream via a learnable Savitzky–Golay-initialised convolution and entropy-maximizing adversarial loss, unlike DANN which applies domain adaptation after feature extraction.
- Blanching, plasma-activated water (PAW), and ultrasound treatments cause large spectral shifts overlapping lipid and protein NIR bands critical for species discrimination.
- Classical preprocessing methods reduce batch separability but do not fully remove batch effects, while BISN’s learned preprocessing yields more balanced species-relevant variance (Fig. 3d).
- Sparse attention in BISN identifies biologically meaningful wavelength regions contributing to classification decisions.
- Explainable AI via Integrated Gradients confirms BISN’s reliance on lipid C–H overtone and protein N–H combination regions consistently across folds.
- BISN jointly optimizes batch invariance and species discrimination without requiring batch labels at inference.
Threat model
The adversary is non-malicious environmental and instrumental batch variability that induces distribution shifts of NIR spectra across production runs, unknown during training and causing performance degradation. The model assumes no direct adversarial manipulation of inputs or labels, but aims to generalize species classification despite batch-induced spectral artefacts. BISN cannot defend against targeted spoofing beyond natural batch variation.
Methodology — deep read
The threat model assumes an industrial setting where the adversary is environmental/instrumental batch variation causing distribution shifts unseen during training. The goal is robust species authentication invariant to batch. BISN uses 2,700 spectra from three insect species (Acheta domesticus, Hermetia illucens, and Tenebrio molitor), each collected across three independent production batches spaced 30–60 days apart to include biological and instrumental variability. Samples undergo factorial treatments: reference (tap water), blanching at 70°C, plasma-activated water (PAW), with or without ultrasound exposure, yielding diverse spectral perturbations. Spectra were measured with a PerkinElmer Lambda 950 spectrometer over 700–2050 nm at 10 nm steps (136 wavelengths). Labels include species, batch, treatment, and ultrasound condition. Data splits used leave-one-batch-out (LOBO) cross-validation to test generalization on fully unseen production batches. BISN architecture has four components: (1) a learnable preprocessing 1D convolution initialized with Savitzky-Golay first derivative coefficients and followed by instance normalization, which outputs batch-invariant spectral representations; (2) a sparse attentive encoder generating an 8-dimensional latent embedding via a shared-weight feature transformer and sparsemax attention mask over wavelengths, regulated by attention entropy loss to distribute spectral focus; (3) a linear species classifier trained with cross-entropy loss; (4) an entropy-regularized adversarial batch branch connected by a gradient reversal layer that maximizes Shannon entropy over batch predictions (pushes batch prediction towards uniformity), thus removing batch-specific spectral cues from the learned representation. The overall loss combines species classification loss, batch entropy loss (with annealing), and attention sparsity regularizer. BISN contrasts with DANN-style approaches by enforcing batch invariance early at the spectral preprocessing stage rather than post-encoder. Classical preprocessing baselines (Savitzky–Golay, SNV, MSC, polynomial detrending) were optimized per training batch to suppress artefacts but remain fixed. Extensive baselines included LDA, GPC, PLSDA, domain adaptation models (DANN, ShapDA), and deep learning architectures (SpectraTr transformer, NIRCoreVision-MLP, TabPFN, TabNet). Models were evaluated by mean accuracy and F1 on held-out test batches under LOBO. Hyperparameters were tuned on internal validation sets stratified by batch and species. Explainability was performed using Integrated Gradients and region-constrained counterfactual optimizations to highlight spectrally important regions for decisions. Statistical significance of improvements was assessed (p < 10^-6). The source code and dataset are publicly released, improving reproducibility. As a concrete example, one LOBO fold holds out batch 3 during training on batches 1 and 2 with optimized BISN preprocessing and adversarial training. The learned preprocessing suppresses batch effect signatures, the sparse attentive encoder focuses on lipid and protein absorption bands, and the species classifier achieves 93% accuracy on batch 3 spectra, which classical preprocessing pipelines failed to classify robustly.
Technical innovations
- A novel batch-invariant spectral preprocessing module initialized with Savitzky–Golay filtering that is learnable and jointly optimized with classification.
- An entropy-regularized adversarial objective maximizes uncertainty over batch predictions, removing batch-specific spectral features upstream before feature extraction.
- Sparse wavelength attention with entropy regularization to ensure interpretability and robustness by focusing on complementary biochemical absorption regions.
- Integration of adversarial batch invariance loss with preprocessing convolution rather than applying domain adaptation after feature extraction, differing from DANN.
- Biochemically grounded explainability linking spectral attribution maps with known insect lipid and protein NIR absorption bands.
Datasets
- BISN edible insect NIR dataset — 2,700 spectra — three species across three independent industrial production batches, multiple treatments (public: https://github.com/majharB/bisn)
Baselines vs proposed
- Strongest baseline (not explicitly named, from Table 1 baselines): mean LOBO accuracy ≈ 0.89 vs BISN: 0.93 ± 0.04
- DANN domain-adversarial method: lower batch invariance and accuracy compared to BISN (exact numbers not specified)
- Classical preprocessing with logistic regression batch classification accuracy minimized to reduce batch effects but did not achieve BISN accuracy
- Transformer-based spectra method SpectraTr and CNN-based baselines underperform BISN under LOBO splits (specific metrics in supplementary)
Figures from the paper
Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2606.26757.
Fig 1: Architecture of the Batch-Invariant Spectral Network (BISN). Raw NIR spectra 𝑥enter an informed preprocessing
Fig 2: Spectral sensitivity analysis across insect species, processing treatments, and chemically defined NIR wavelength
Fig 3: NIR spectral profiles across batches, species, and preprocessing stages. Top row: mean spectra coloured by
Fig 4: BISN latent embedding visualisation across leave-one-batch-out (LOBO) folds. (a–c) PCA projections of the
Fig 5: Confusion matrices for BISN under leave-one-batch-out evaluation. (a) Batch 1 held out; (b) Batch 2 held out;
Fig 6: Spectral attribution and counterfactual analysis of BISN predictions. (a–c) Mean Integrated Gradients (IG)
Fig 7 (page 22).
Limitations
- Study limited to three insect species relevant to EU industrial applications; generalization to other species or more diverse insect biomass unknown.
- Only three production batches spaced 30–60 days apart, leaving longer-term temporal drift or different instruments untested.
- Batch adversarial training maximizes batch-entropy but does not test fully adversarial attackers or intentional batch spoofing strategies.
- The effect of noise and sample preparation inconsistencies beyond the studied processing treatments remains unassessed.
- Sparse attention interpretability relies on Integrated Gradients but causality of spectral regions in classification requires further validation.
Open questions / follow-ons
- How does BISN perform under larger-scale industrial variability, e.g. multiple instruments, geographical production sites, or longer time scales?
- Can batch-invariant processing be extended to handle adversarial attacks that intentionally mimic batch effects or manipulate spectral data?
- What is the effect of expanding the species set or including processed insect products beyond raw insect slurries?
- How can the sparse attention masks and explainability methods be further validated biologically or chemically to confirm causal relationships?
Why it matters for bot defense
For bot-defense or CAPTCHA practitioners, BISN provides a compelling example of an adversarially trained, domain-invariant preprocessing mechanism that operates early in the feature extraction pipeline to suppress unwanted batch-specific variation. This approach differs from typical domain-adversarial strategies applied at feature-level and demonstrates improved robustness to realistic distribution shifts. While the domain is edible insect spectroscopy, the principles of learnable domain-invariant preprocessing with entropy-maximizing adversarial loss could inform design of robust biometric or sensor-based authentication systems subject to batch or environmental shifts. BISN’s integration of explainability via sparse attention also offers a pathway to interpretable anomaly detection, critical for security use cases where understanding failure modes is essential. Practitioners should note BISN’s reliance on careful batch label supervision during training and that robustness guarantees extend only to natural batch variation, not adversarial tampering. Nonetheless, the idea of shifting domain invariance upstream to the preprocessing stage and enforcing high entropy batch prediction could be applicable beyond spectroscopy, including keyboard dynamics, network traffic features, or voice biometrics operating under distribution shift.
Cite
@article{arxiv2606_26757,
title={ Batch-Invariant Spectral Intelligence for Robust and Explainable Insect Authentication },
author={ Majharulislam Babor and Giacomo Rossi and Annalisa Altavilla and Oliver Schlüter and Marina M. -C. Höhne },
journal={arXiv preprint arXiv:2606.26757},
year={ 2026 },
url={https://arxiv.org/abs/2606.26757}
}