Panda: Unsupervised Pelvic Anomaly Detection for Real-Time MR Imaging

Source: arXiv:2607.24703 · Published 2026-07-27 · By Anika Knupfer, Maximilian Lindholz, Johanna Paula Müller, Jordina Aviles Verdera, Smiti Tripathy, Susanne Schulz-Heise et al.

TL;DR

This paper addresses the challenge of detecting pelvic anomalies in female patients using real-time MRI imaging. Pelvic diseases often have delayed diagnosis partly due to motion artifacts, tissue deformation, and scarcity of annotated pathological cases. Supervised methods are hampered by rare and diverse abnormalities with limited labeled data. The authors propose a novel unsupervised anomaly detection framework, termed Panda, adapted from the Dinomaly architecture, which leverages a frozen DINOv3 Vision Transformer encoder combined with a noisy MLP bottleneck and Linear Attention decoder. This setup prevents trivial identity mapping and remains computationally efficient for in-scanner use. The method detects deviations from learned normal pelvic appearance by measuring cosine distance between encoder and decoder representations at token-level, producing spatial anomaly maps and frame-level anomaly scores in near real-time.

Evaluated on a curated subset of the Uterine Myoma Dataset and additional low-field pelvic MR images, Panda achieves a high pixel-level AUROC of 88.06% and specificity of 95.45% at frame-level while processing 40.5 slices per second, meeting clinical real-time requirements. The unsupervised method provides localized anomaly maps for adaptive scanning and immediate radiologist feedback. Comparisons against a state-of-the-art ResVAE baseline reveal Panda’s superior spatial precision and specificity, though with a tradeoff of lower sensitivity particularly for small or low-contrast lesions. Ablations confirm robustness and highlight the frozen encoder's critical role. Overall, the work demonstrates an effective and practical approach for unsupervised pelvic anomaly detection in challenging MRI data with clinical deployment potential.

Key findings

• Pixel-level AUROC of 88.06% achieved on the Uterine Myoma Dataset (UMD) demonstrating strong anomaly detection ability.
• Frame-level specificity reaches 95.45% at a conservative threshold calibrated on healthy data, with precision of 71.88%.
• Frame-level sensitivity is low (11.61%) due to coarse token resolution (28×28) and conservative thresholding to avoid false alarms.
• Inference latency is 0.0247 seconds per slice (40.5 slices/s) on an NVIDIA RTX 3080, meeting real-time MRI workflow constraints.
• Compared to the ResVAE baseline, Panda shows higher pixel-level AUROC (87.85% vs 69.65%) and specificity (95.02% vs 64.43%) but lower sensitivity (38.39% vs 79.91%).
• Ablation removing dropout bottleneck slightly improves specificity; smaller ViT-S backbone reduces AUROC by 4.1% but doubles inference speed.
• Synthetic data inclusion provides small performance improvement; training on 50% of data reduces AUROC to 83.76%.
• The frozen DINOv3 encoder prevents co-adaptation and preserves a stable normative reference crucial for performance robustness.

Threat model

The adversary is any pathological or atypical pelvic anatomy appearing in MRI not observed during training. The model assumes no labeled anomaly examples and that anomalies induce deviations from normal anatomy captured by the frozen encoder’s representation space. Adversaries cannot control the imaging process but may cause subtle changes confounded by physiological motion and imaging artifacts. The model is not designed to defend against active adversarial attacks or deliberate image tampering.

Methodology — deep read

1. Threat Model & Assumptions: The adversary corresponds to anomalies or pathological deviations in pelvic MRI scans unknown to the model at training time. It is assumed that only healthy, normal pelvic MRI frames are available for training. No labeled anomaly data or pathology annotations are used, making the approach unsupervised. The adversary’s manifestations are heterogeneous, rare, and partially subtle compared to physiological motion and intensity variability.

2. Data: Training data (Dtrain) consists of 207 sagittal T2-weighted (T2w) healthy MRI volumes from clinical and research datasets, supplemented by 800 radiologist-validated synthetic images representing rare anatomical variants, all intensity normalized and spatially cropped around the uterus. The clinical data (156 volumes) comes from various field strength scanners with heterogeneous protocols; the research data (51 volumes) is from a low-field 0.55T scanner. Testing data (Dtest) combines 198 anomalous volumes from the publicly available Uterine Myoma Dataset (UMD) which includes pixel-level lesion annotations, plus 13 held-out low-field volumes. One slice per volume with visible anomalies is selected for evaluation, alongside 102 normal slices sampled to avoid proximity to lesion slices.

3. Architecture / Algorithm: The model adapts Dinomaly, a reconstruction-based anomaly detection architecture composed entirely of Transformer blocks. A pre-trained, frozen DINOv3 ViT-L/16 encoder extracts intermediate patch-level feature representations (28×28 grid), which serve as normative reference features. These encoder features pass through a noisy MLP bottleneck applying dropout to corrupt activations during training, preventing identity mapping and forcing the decoder to reconstruct normal features from incomplete information. The decoder uses four Linear Attention layers (with 8 heads and MLP ratio 2.097) rather than standard softmax attention to enable globally consistent reconstruction over the spatial token grid. Two complementary groups of encoder features—low-level (layers 3-6) and high-level semantic (layers 7-10)—are reconstructed independently and combined. The reconstruction loss is a group-to-group cosine similarity loss accentuated on hard tokens via gradient shrinking (33rd percentile threshold).

At inference, dropout is disabled for deterministic operation. Reconstruction error per token is quantified by cosine distance between the frozen encoder token features and decoder output features. Token-level anomaly scores are upsampled from 28×28 to 96×96 pixel maps to produce spatial anomaly heatmaps. A frame-level anomaly score aggregates token anomalies exceeding a healthy-calibrated threshold to flag abnormal slices.

4. Training Regime: The model trains for 120 epochs using AdamW optimizer with learning rate 4.24×10⁻⁴ and weight decay 10⁻⁴, batch size 32, on the training data described. Data augmentation is not applied. Hyperparameters were selected via Bayesian optimization over 40 trials maximizing pixel-level AUROC on Dtest without data leakage. Dropout rate in bottleneck is 13.6% during training. Gradient shrinking emphasizes hard tokens from the first epoch onward.

5. Evaluation Protocol: Evaluation uses pixel-level AUROC, accuracy, precision, sensitivity, specificity, and F1 score computed against expert lesion annotations on Dtest. Pixel-wise threshold is chosen to maximize Dice coefficient on Dtest. Frame-level metrics use a conservative healthy-calibrated threshold set at the 95th percentile of anomaly scores from only normal training slices, imposing a strict false-positive control suitable for clinical deployment. Inference latency is measured on a RTX 3080 GPU. Baseline comparison is performed using a re-trained ResVAE architecture evaluated on the same data and metrics.

6. Reproducibility: The paper does not explicitly mention public code release or frozen weights. The datasets include a mixture of public (UMD, synthetic) and institutional data with ethical approvals. Details of preprocessing, model architecture, training protocols, and hyperparameters are provided for replication, though full code or weights are not stated as released.

Example flow: A 96×96 sagittal T2w MRI slice is normalized, resized to 224×224, and passed through the frozen DINOv3 ViT encoder to extract 1024-dimensional patch embeddings over 28×28 token grid. These embeddings are corrupted by dropout in the MLP bottleneck and input to the Linear Attention decoder, which reconstructs two groups of features. The cosine distance between encoder and decoder outputs per token is computed and upscaled to a full-resolution anomaly map. Tokens exceeding a normality threshold contribute to a frame-level anomaly score indicating possible pathology. Processing occurs at 40.5 slices per second on GPU hardware.

Technical innovations

• Adaptation of the Dinomaly architecture for pelvic MRI anomaly detection using a frozen DINOv3 ViT-L/16 encoder to preserve normative semantic representations and avoid encoder-decoder co-adaptation.
• Introduction of a noisy MLP bottleneck with dropout during training to corrupt features and prevent trivial reconstruction of anomalies.
• Replacement of standard softmax attention with Linear Attention in the decoder to promote globally consistent reconstruction suitable for subtle contextual anomalies in pelvic MRI.
• Use of per-token cosine distance between encoder and decoder representations for spatially precise localization of anomalies at inference time.
• Healthy-calibrated anomaly thresholds derived solely from normal training data to prioritize high specificity and reduce false positives in clinical real-time deployment.

Datasets

• Clinical Healthy Pelvic MRI Dataset — 156 volumes — institutional / approved under ethics 24-304-Br, 23-444-Bm
• Research Healthy Pelvic MRI Dataset — 51 volumes at 0.55T low field — institutional
• SynthUterus Synthetic Dataset — 800 images — publicly available
• Uterine Myoma Dataset (UMD) — 198 anomalous volumes with pixel annotations — publicly available

Baselines vs proposed

• ResVAE baseline pixel-level AUROC = 69.65% vs Panda pixel-level AUROC = 87.85%
• ResVAE pixel-level sensitivity = 79.91% vs Panda sensitivity = 38.39%
• ResVAE pixel-level specificity = 64.43% vs Panda specificity = 95.02%
• Frame-level specificity (Panda) = 95.45% at threshold calibrated on healthy data
• Inference latency: ResVAE not reported vs Panda 0.0247s per slice (40.5 slices/s)

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.24703.

Fig 1: (a) Schematic and T2w MRI pelvic illustration (pink—uterus; light

Fig 2: Proposed framework for low-latency pelvic anomaly detection in MRI.

Fig 4: illustrates the complementary nature of the two approaches across two

Fig 3: Representative results on Dtest. Each row presents from left to right: T2w

Fig 5 (page 8).

Limitations

• Low frame-level sensitivity (11.61%) due to coarse 28×28 token grid, limiting detection of small or low-contrast lesions.
• Relatively low pixel-level precision (17.31%) and F1 score (23.87%) caused by high class imbalance and small pathological regions.
• No explicit adversarial or targeted distribution shift robustness evaluation presented.
• Model performance depends heavily on frozen pretrained encoder; alternatives or fine-tuning could impact results.
• Synthetic data inclusion impact is modest and clinical generalizability beyond uterine myomas remains unvalidated.
• Fine delineation of lesion boundaries remains challenging due to token-level resolution limits.

Open questions / follow-ons

• Can fusion of reconstruction-based (Panda) and generative approaches (ResVAE) combine high specificity and sensitivity for improved anomaly detection?
• How can token grid resolution or multi-scale architectures be enhanced to improve detection of small or low-contrast lesions?
• What is the performance and generalizability of the approach on broader pelvic pathologies and in interventional MRI settings?
• How robust is the method under significant distribution shifts such as different scanners, protocols, or patient populations not covered in training?

Why it matters for bot defense

While not directly related to CAPTCHA or bot defense, the methodology of unsupervised anomaly detection using a stable frozen encoder and reconstruction-based spatial anomaly localization can inspire techniques for detecting anomalous input patterns or adversarial manipulations in other domains. The use of token-level cosine distances and linear attention to balance reconstruction fidelity with anomaly sensitivity may inform novel architectural choices for bot detection models requiring generalization to rare or unseen attack types. Additionally, the healthy-calibrated thresholding prioritizing high specificity to reduce false positives resonates with the real-world operational constraints in bot-defense, where false alarms reduce trust and usability.

Practitioners in bot defense and CAPTCHA design may find value in exploring representation-level reconstructions coupled with principled noise injection to prevent trivial identity mappings, analogous to the noisy bottleneck here, enabling the detection of subtle but meaningful deviations without supervised anomaly labeling. Real-time inference speed optimizations detailed are also relevant for latency-sensitive interactive systems such as CAPTCHA challenges.

Cite

@article{arxiv2607_24703,
  title={ Panda: Unsupervised Pelvic Anomaly Detection for Real-Time MR Imaging },
  author={ Anika Knupfer and Maximilian Lindholz and Johanna Paula Müller and Jordina Aviles Verdera and Smiti Tripathy and Susanne Schulz-Heise and Jana Hutter },
  journal={arXiv preprint arXiv:2607.24703},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.24703}
}

Read the full paper

• arXiv:2607.24703 (abstract)
• arXiv:2607.24703 (PDF)