DynoDINO: Harnessing Dynamic Latent Information from DINO Features for Multi-Phase Medical Image Segmentation

Source: arXiv:2607.29568 · Published 2026-07-31 · By Yu-Pu Hsu, Jen-Jee Chen, Yu-Chee Tseng

TL;DR

DynoDINO addresses key challenges in multi-phase medical image segmentation, specifically for contrast-enhanced CT (CECT) used in liver cancer diagnosis. Unlike traditional approaches that either treat each phase independently or fuse phases with naive channel concatenation, DynoDINO explicitly models temporal contrast enhancement dynamics across phases. It first performs slice-level anatomical alignment using an FFT-accelerated zero-mean normalized cross-correlation (ZNCC) method to align multi-phase data despite respiratory motion and incomplete acquisitions. Then, a novel multi-phase fusion module with a Mix-Attention mechanism and an Adaptive Gating module selectively preserves diagnostically relevant temporal variations while suppressing misalignment artifacts. Evaluations on three large-scale diverse datasets (LiTS, PLC-CECT, and WAW-TACE) demonstrate DynoDINO consistently improves boundary delineation, structural fidelity, and robustness to missing or shifted phases compared to state-of-the-art baselines including nnU-Net, SegFormer, Dino U-Net, and MedDINOv3.

Key findings

• DynoDINO reduces global 95th percentile Hausdorff Distance (HD95) on LiTS from MedDINOv3 baseline 13.85mm to 13.76mm.
• On the PLC-CECT multi-phase dataset, DynoDINO reduces global HD95 by 15.2% from 28.72mm (nnU-Net multi-phase) to 24.35mm.
• DynoDINO decreases average minimum detectable tumor volume (Avg. Vmin) on PLC-CECT from 90,145.65 mm3 (nnU-Net) to 21,871.95 mm3 (multi-phase input).
• When confronted with missing phases in the WAW-TACE dataset, DynoDINO maintains stable performance, unlike nnU-Net which suffers a macro-averaged Global DSC drop from 79.04% to 77.75% transitioning from single- to multi-phase input.
• The Mix-Attention mechanism decreases memory footprint by 4.1% (about 7.08M fewer parameters) compared to standard cross-attention, enabling high-resolution multi-phase fusion with FlashAttention.
• Applying the FFT-accelerated ZNCC alignment method allows effective slice-to-slice matching despite inter-phase misalignment caused by motion.
• Adaptive Gating mechanism prevents feature degradation and stabilizes training by suppressing residual misalignment artifacts during difference-based residual learning.
• DynoDINO outperforms CNN and transformer baselines on boundary-centric metrics (NSD, HD95) critical for clinical lesion characterization, despite sometimes lower Dice similarity values.

Threat model

The adversary corresponds to practical clinical challenges: anatomical misalignment across phases due to respiratory motion and acquisition variability, incomplete or missing phases due to clinical imaging protocol heterogeneity, and intensity variations from contrast agent kinetics. The system assumes no intentional tampering but must robustly segment lesions despite these distortions and inconsistent temporal information.

Methodology — deep read

1. Threat Model & Assumptions: The framework assumes an adversarial scenario typical of clinical multi-phase imaging, where inter-phase slices suffer from anatomical misalignment due to respiratory motion, patient movement, or incomplete scanning. The adversary is effectively the variability in acquisition protocols and physiology rather than a malicious attacker. The system must robustly segment lesions despite these practical imperfections without exact voxel-level registration.

2. Data: Three datasets were used: LiTS (131 patients, single-phase liver tumor CT from MSD challenge adapted for multi-phase by padding missing phases), PLC-CECT (278 cancer patients + 83 controls with multi-phase CT including Early, Mid, Late phases and 50,560 lesion-bearing 2D slices), and WAW-TACE (233 post-treatment HCC patients with incomplete phases and 377 tumor labels). Pre-contrast phases were excluded. Data preprocessing involved resizing slices to 512x512 and intensity clipping consistent with MedDINOv3.

3. Architecture / Algorithm: DynoDINO uses MedDINOv3 as the backbone encoder and extracts multi-scale latent features from layers 3, 6, 9, 12. Inputs from 3 phases are concatenated along batch then reshaped for encoding. The core is a Fusion Module applying N iterations (e.g., 4) of a novel Mix-Attention (MA) mechanism computing a shared bidirectional attention map between the Late Phase (LP) as reference and Early/Mid phases as targets. This reduces parameters and memory compared to separate cross-attention maps. Post-fusion, a difference-based residual learning module performs layer normalization and subtraction of fused features to highlight temporal contrast changes. An adaptive gating mechanism then weights these differences spatially to suppress noise/artifacts before adding back to Early Phase features, preserving phase-specific enhancement patterns while rejecting residual misalignment errors.

4. Training Regime: Models were trained using AdamW optimizer with weight decay 0.05, a SequentialLR scheduler (10-epoch linear warmup, 90-epoch cosine annealing), and a compound soft Dice plus cross-entropy loss. Training used NVIDIA RTX 4090 GPUs within MedDINOv3 official settings. Missing phases in WAW-TACE were zero-padded to avoid crashing CNN baselines. Batch size and seed details are not specified.

5. Evaluation Protocol: Metrics included Dice Similarity Coefficient (DSC), Normalized Surface Distance (NSD), and 95th percentile Hausdorff Distance (HD95), with emphasis on boundary-relevant NSD and HD95 to reflect clinical utility. Baselines included nnU-Net, SegFormer, Dino U-Net, and MedDINOv3, evaluated under single-phase (EP only) and multi-phase inputs. Robustness under missing phases and phase shifts was also assessed across the datasets.

6. Reproducibility: The framework builds on publicly available MedDINOv3. The PLC-CECT and WAW-TACE datasets are from open Science Data Bank and Zenodo respectively. Preprocessing and alignment code details are explained but no explicit public code or pretrained weights are mentioned in the source.

Technical innovations

• FFT-accelerated zero-mean normalized cross-correlation (ZNCC) based slice alignment method for robust inter-phase anatomical matching tolerant to intensity variations and motion artifacts.
• Mix-Attention mechanism computing a single shared bidirectional attention map between reference and target phases, reducing memory and parameter overhead compared to vanilla cross-attention.
• Difference-based residual learning module that explicitly models temporal contrast dynamics by layer-normalized subtraction of fused multi-phase features.
• Adaptive Gating mechanism applying a learned spatial weight to difference features, enhancing training stability and suppressing artifacts from residual misalignment.

Datasets

• LiTS — 131 patients, 201 3D CT volumes adapted to multi-phase by padding — public (MSD challenge)
• PLC-CECT — 278 liver cancer patients + 83 controls, 50,560 2D slices multi-phase CT with ground-truth — public (Science Data Bank)
• WAW-TACE — 233 treatment-naive HCC patients, 29,037 2D slices, incomplete multi-phase data with tumor labels — public (Zenodo)

Baselines vs proposed

• nnU-Net (multi-phase): Global HD95 = 28.72 mm vs DynoDINO (multi-phase): 24.35 mm (15.2% improvement) on PLC-CECT
• nnU-Net (single-phase): Global DSC = 79.04% vs DynoDINO (single-phase): comparable but DynoDINO better HD95 on WAW-TACE
• MedDINOv3 (single-phase): HD95 = 13.85 mm vs DynoDINO (single-phase): 13.76 mm on LiTS
• SegFormer: Liver DSC = 94.06% vs DynoDINO: 96.16% on LiTS
• Dino U-Net: Tumor DSC = 55.79% vs DynoDINO: 60.12% on LiTS

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.29568.

Fig 1: The DynoDINO framework. Multi-phase CT slices are first aligned using the ZNCC matching method (here,

Fig 2: The Fusion Module, which performs N iterations of Mix-Attention followed by a difference-based residual

Fig 3: The Mix-Attention Layer computes a shared bidirectional attention map (Attmap) between the reference

Fig 4: Qualitative comparison of segmentation results on the LiTS dataset. Results are shown in red.

Fig 5: Qualitative comparison of segmentation results on the PLC-CECT dataset.

Fig 6: Qualitative comparison of segmentation results on the WAW-TACE dataset.

Fig 7: Analysis of training dynamics and convergence stability on the WAW-TACE dataset. Blue and red curves

Fig 8: Visualization of the FFT-ZNCC alignment (Case P0001, PLC-CECT). The green and red masks represent the

Limitations

• DynoDINO’s Vision Transformer backbone sacrifices pixel-level local resolution resulting in slightly larger minimum detectable tumor volumes compared to CNN baselines.
• The study does not report robustness under adversarial attacks or deliberate tampering of input images.
• Lack of detailed ablation studies on the impact of each module (ZNCC alignment, Mix-Attention, Adaptive Gating) individually across datasets.
• Training regime details such as batch sizes, number of seeds for statistical validation, and run-to-run variance are not extensively reported.
• The method relies on phase presence assumptions (EP mandatory), limiting application to protocols missing early reference phases.
• No explicit external clinical validation or user studies with radiologists evaluating segmentation utility in workflows.

Open questions / follow-ons

• Can the proposed Mix-Attention and Adaptive Gating mechanisms be generalized to other multi-temporal medical imaging modalities beyond CECT, such as dynamic MRI or PET?
• What is the quantitative impact of each individual component (ZNCC alignment, Mix-Attention, Residual Difference Learning, Adaptive Gating) on final segmentation performance via ablations?
• How does DynoDINO perform under adversarial conditions or simulated severe motion artifacts beyond natural clinical variability?
• Can the method be extended to 3D volumetric alignment and fusion to leverage spatial context instead of independent 2D slice matching?

Why it matters for bot defense

For bot-defense and CAPTCHA practitioners, DynoDINO presents an instructive example of leveraging advanced multi-phase temporal fusion combined with robust alignment to handle noisy, incomplete, or misaligned inputs effectively. The principles of efficient fused representations via Mix-Attention and gating mechanisms to suppress artifacts have analogs in signal processing for noisy channel inputs. While medical imaging and CAPTCHA operate in distinct domains, the approach's emphasis on adaptive fusion and artifact suppression is relevant when designing multi-modal human verification systems where inputs may arrive asynchronously or corrupted. Furthermore, the FFT-accelerated slice alignment offers a scalable method for cross-frame correspondence that could inspire preprocessing for multi-frame challenge-response stimuli in bot detection. Overall, this architecture highlights the benefit of modeling temporal dynamics explicitly and robustly over naive aggregation—a lesson applicable for time-distributed verification data in CAPTCHA or bot-detection contexts.

Cite

@article{arxiv2607_29568,
  title={ DynoDINO: Harnessing Dynamic Latent Information from DINO Features for Multi-Phase Medical Image Segmentation },
  author={ Yu-Pu Hsu and Jen-Jee Chen and Yu-Chee Tseng },
  journal={arXiv preprint arXiv:2607.29568},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.29568}
}

Read the full paper

• arXiv:2607.29568 (abstract)
• arXiv:2607.29568 (PDF)