Generalist Vision-Language Models for Fast Radio Burst detection: a zero-shot benchmark against a specialized detector

Source: arXiv:2607.07382 · Published 2026-07-08 · By Raiff H. Santos, Amilcar R. Queiroz, Tharcisyo S. S. Duarte, K. E. L. de Farias, Rafael A. Batista

TL;DR

This paper investigates the potential for small, generalist Vision-Language Models (VLMs) to perform zero-shot classification of Fast Radio Bursts (FRBs) in dynamic radio spectra, compared against a state-of-the-art specialized deep learning model, SwinYNet. FRBs are millisecond-duration radio transients whose automated detection usually requires large, domain-specific datasets and dedicated training. The authors evaluate two open-weight, locally run VLMs (Gemma 4 2B and 4B) on a controlled, simulated benchmark of 3000 radio dynamic spectra including FRB signals, structured Radio Frequency Interference (RFI), and noise. Under a strictly zero-shot, prompt-only regime with no fine-tuning or labeled examples, Gemma 4 2B achieves 93.65% binary accuracy on a balanced 2000-sample subset, statistically indistinguishable from SwinYNet’s 92.90%. Notably, the VLM shows much lower false positives on RFI (6.4% vs 25.0%) and none on noise. SwinYNet maintains a perfect ranking ROC-AUC of 1.0, exceeding the VLM’s 0.9482, but the zero-shot VLM approaches this ceiling from general pretraining alone. By prompt engineering, the same VLMs perform a three-class FRB/RFI/noise classification over the full set with up to 86% accuracy and zero false FRB detections. This study demonstrates that generalist vision-language models pretrained on broad image-text data can already recognize astrophysical signal morphologies with no domain-specific training, offering a complementary tool with explainable output. However, domain-specialized detectors retain superior probabilistic discrimination and parameter estimation.

Key findings

• Gemma 4 2B zero-shot VLM achieves 93.65% binary FRB detection accuracy on 2000-sample balanced benchmark vs. SwinYNet’s 92.90%, difference not statistically significant.
• Gemma 4 2B false-positive rate on structured RFI is 6.4%, significantly lower than SwinYNet’s 25.0%.
• Gemma 4 2B produces zero false positives on pure noise samples, while SwinYNet has some false alarms.
• SwinYNet attains a perfect ROC-AUC of 1.0000 on this benchmark; Gemma 4 2B achieves 0.9482, indicating superior ranking calibration.
• Rewriting the prompt alone enables the VLMs to perform three-class FRB/RFI/noise classification with up to 86% accuracy on all 3000 samples without any false FRB detections.
• The VLMs return structured JSON output including probability and natural-language justification for each decision, enhancing interpretability compared to specialized black-box models.
• Gemma 4 2B and 4B are small, open-weight, locally runnable models (~2B+ parameters), highlighting feasibility for low-resource deployment.
• Simulated dynamic spectra span a range of realistic FRB dispersion measures (121–899 pc cm^-3), widths (3–5 ms), and flux densities, ensuring varied visual morphologies.

Threat model

Adversaries consist of terrestrial sources producing structured Radio Frequency Interference (RFI) and stochastic system noise that contaminate radio telescope data and can mimic or obscure fast radio bursts. These adversaries cannot modify the detection algorithms but challenge the classifiers by generating false positives. The model assumes no targeted adversarial attempts to fool the detection but must reliably discriminate dispersed astrophysical signals from diverse terrestrial artefacts.

Methodology — deep read

1. Threat model and assumptions: The adversary model is implicit—detection systems must separate genuine astrophysical FRBs from terrestrial interferences (RFI) and noise. The VLMs are evaluated for zero-shot recognition without exposure to labeled FRB domain data, testing transfer from general image-text pretraining. The specialized detector (SwinYNet) is trained end-to-end on FRB data, representing an upper bound on performance.

2. Data provenance, size, labels, splits, preprocessing: The dataset consists of 3000 synthetic L-band dynamic spectra simulated using simulateSearch software, modeling 2-second observations across 2048 frequency channels at 1.96608e-4 s sampling. The 3000 samples are evenly split into 1000 FRB, 1000 RFI (structured interference across 5 subtypes), and 1000 noise-only examples. A balanced binary benchmark subset of 2000 samples has 1000 FRB and 1000 non-FRB (500 RFI + 500 noise). Each instance is rendered as anonymized PNG images normalized and color-mapped, for VLM input, while SwinYNet operates on native PSRFITS files.

3. Architecture/algorithm: The generalist VLMs examined are Gemma 4 2B and Gemma 4 4B, small vision-language models with open weights designed for local execution, combining a vision encoder and a text decoder trained jointly on broad image-text datasets. They generate a natural-language classification and justification token-by-token in response to a prompt describing the classification task. The specialized baseline is SwinYNet, a supervised multi-task Swin Transformer trained to detect, segment, and parameterize FRBs in PSRFITS data. Unlike SwinYNet, the VLMs receive only a single PNG rendering and no fine-tuning or classifier head adaptation. The VLM decision is induced solely by the prompt, converting the task to zero-shot classification.

4. Training regime: Gemma models are fixed, no weights updated or fine-tuned during experiments. SwinYNet training details reference prior work, involving multi-epoch supervised optimization on labeled FRB data. VLM inference uses greedy decoding at a fixed prompt threshold 0.5.

5. Evaluation protocol: Metrics include accuracy, macro-F1 score, ROC-AUC, average precision, and false positive rates segmented by RFI and noise source classes. Paired comparisons on identical test samples examine per-sample correctness and error types. McNemar tests evaluate statistical differences in classification errors. Multi-class classification to separate FRB, RFI, and noise is also tested by prompt rewriting. Score distribution histograms and ROC curves illustrate calibration and ranking. No adversarial or real observational data tests are reported.

6. Reproducibility: The simulation pipeline uses a fixed global seed (42) and deterministic sample seeds, enabling exact regeneration of the dataset with simulateSearch. The Gemma VLMs are publicly accessible with fixed, open weights and run locally. SwinYNet is cited from prior published code. The image rendering settings and prompts are documented to support replication.

Concrete example: A single 2-second PSRFITS file is simulated with a dispersed FRB signal at DM ~300 pc cm^-3 and flux ~7 units. This file is converted to a 1024x768 PNG dynamic spectrum colored with a percentile normalization and viridis colormap. Gemma 4 2B receives the PNG plus a fixed natural-language prompt defining classes (FRB, RFI, noise). It autoregressively generates a JSON response with label "FRB", confidence score >0.9, and a textual justification referencing the characteristic ν^(-2) dispersion sweep pattern. SwinYNet processes the PSRFITS directly, producing a continuous probability score thresholded at 0.5. Both outputs are compared for correctness against ground truth. Over 2000 such paired samples the metrics and error rates described are aggregated.

Technical innovations

• Application of small, open-weight, local, generalist Vision-Language Models (Gemma 4 2B/4B) to astrophysical transient detection in zero-shot, prompt-only regime without any domain fine-tuning.
• Comparison of raw image-based VLM outputs with a specialized supervised deep detector (SwinYNet) using paired sample evaluation on controlled synthetic radio spectra.
• Usage of textual prompts to induce multi-class classification (FRB/RFI/noise) including natural-language structured explanations, integrating interpretability with classification.
• Demonstration that generalist visual transformers pretrained on broad image-text datasets can nearly match specialized domain model performance on a niche scientific detection task.

Datasets

• Simulated FRB dynamic spectra — 3000 samples (1000 FRB, 1000 RFI across 5 subtypes, 1000 noise) — generated with simulateSearch software
• Binary benchmark subset — 2000 samples (1000 FRB + 500 RFI + 500 noise) — balanced for classification evaluation

Baselines vs proposed

• SwinYNet: binary accuracy = 92.90% vs Gemma 4 2B zero-shot VLM: 93.65% (no statistically significant difference)
• SwinYNet false positive rate on RFI = 25.0% vs Gemma 4 2B: 6.4%
• SwinYNet ROC-AUC = 1.0000 vs Gemma 4 2B ROC-AUC = 0.9482
• Gemma 4 2B three-class task accuracy up to 86% with zero false FRB, SwinYNet results for this task not reported

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.07382.

Fig 2: Representative anonymized dynamic-spectrum images, one per class, as supplied to the VLM. Left: an FRB (a single

Fig 2 (page 9).

Fig 3 (page 9).

Limitations

• Use of fully synthetic simulated data limits assessment of real-world robustness and generalization to observational radio datasets with complex noise and interference.
• Zero-shot evaluation excludes any fine-tuning or adaptation that could improve VLM performance, setting a conservative lower bound rather than maximum capability.
• The VLMs operate on 2D image renderings rather than the native PSRFITS format, introducing an information bottleneck compared to specialized models accessing raw data.
• The study does not evaluate adversarial robustness or sensitivity to rare/unseen interference types in real telescope environments.
• Prompt sensitivity is acknowledged; minor changes in prompt wording could affect VLM output stability and accuracy.
• No assessment of computational efficiency or latency trade-offs between small VLMs and specialized detectors under real-time constraints.

Open questions / follow-ons

• How would prompt engineering and light fine-tuning (e.g. LoRA) of small VLMs impact FRB detection accuracy and robustness?
• Can the zero-shot VLM approach generalize effectively to real observational data with more complex RFI and noise distributions?
• What is the operational feasibility and runtime performance of deploying these small VLMs for real-time streaming transient detection near the instrument?
• How stable are VLM outputs to small changes in prompt phrasing or visual preprocessing steps in practice?

Why it matters for bot defense

This work illustrates a practical case where generalist vision-language foundation models pretrained on broad image-text corpora transfer effectively to a specialized scientific signal detection task without any domain-specific training or fine-tuning. For bot-defense and CAPTCHA practitioners, it suggests that generalist multimodal models could offer zero-shot or promptable detection capabilities for patterns resembling human activity or artifacts without needing large labeled datasets or retraining. The stable, structured natural-language outputs with justifications also point toward systems that can audit and explain classification outcomes transparently. However, the paper also highlights the continuing advantage of specialized supervised detectors for probabilistic ranking and fine-grained discrimination, emphasizing that foundation models could complement rather than replace task-specific pipelines. The zero-shot prompt-only experimental design offers a useful paradigm for rapidly assessing model transfer capability before committing to fine-tuning.

Cite

@article{arxiv2607_07382,
  title={ Generalist Vision-Language Models for Fast Radio Burst detection: a zero-shot benchmark against a specialized detector },
  author={ Raiff H. Santos and Amilcar R. Queiroz and Tharcisyo S. S. Duarte and K. E. L. de Farias and Rafael A. Batista },
  journal={arXiv preprint arXiv:2607.07382},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.07382}
}

Read the full paper

• arXiv:2607.07382 (abstract)
• arXiv:2607.07382 (PDF)