Multi-Resolution Feature Stem for Diabetic Retinopathy lesion segmentation

Source: arXiv:2607.08679 · Published 2026-07-09 · By Indranil Dutta, Taehee Jeong

TL;DR

This paper addresses a fundamental challenge in automated diabetic retinopathy (DR) lesion segmentation: the extreme variation in lesion sizes from tiny microaneurysms (MAs) to large hemorrhages (HE) and exudates (EX). Prior approaches typically use a fixed input resolution for segmentation models, which fails to accommodate the conflicting demands imposed by differently sized lesions. Through systematic evaluation of multiple architectures including U-Net, UNet++, Vision Transformers, and DeepLabV3+ at two input resolutions (512×512 and 1024×1024), the authors identify a counter-intuitive finding that higher resolution inputs improve detection of fine-grained lesions like MAs but can degrade performance on larger lesions like hemorrhages. To overcome this trade-off, they propose a novel Multi-Resolution Feature Stem integrated with a UNet++ backbone. This stem creates an input pyramid at three scales (1024, 512, 256), applies shared-weight convolutional blocks to each, fuses the features, and feeds them to UNet++. Experimental results on the DDR dataset demonstrate that their multi-resolution approach improves mean Intersection over Union (mIoU) by 15.0% and mean Average Precision (mAP) by 7.8% over baselines, with notable gains in segmenting small and medium lesions, albeit with some degradation on hemorrhages. The work provides new empirical insight into resolution-dependent behavior in DR lesion segmentation and a parameter-efficient architecture that balances fine detail and contextual information.

Key findings

• Increasing input resolution from 512×512 to 1024×1024 improves mIoU of U-Net from 0.1288 to 0.3136 and mAP from 0.2982 to 0.5378 on validation set.
• U-Net++ mIoU improves from 0.1433 to 0.3513 and mAP from 0.3951 to 0.5220 when resolution increases from 512×512 to 1024×1024.
• DeepLab-v3+ performance degrades with higher resolution: mIoU decreases from 0.2098 to 0.1722 and mAP from 0.3848 to 0.3094 when increasing resolution.
• Proposed Multi-Resolution Feature Stem architecture achieves +25.58% AP and +39.33% IoU improvement on Exudates, and +14.2% AP and +7.0% IoU for Microaneurysms over U-Net++ baseline at 1024×1024.
• Model underperforms on Hemorrhages with -15.9% AP and -10.66% IoU compared to U-Net++ at 1024×1024.
• Soft Exudates segmentation AP improves dramatically by +255.68% and IoU by +103.04% with the proposed model versus U-Net++.
• Model achieves a 1.27× inference time overhead compared to baseline U-Net++ while providing 11-14% performance gains on challenging lesions.
• Combining focal loss (γ=2.0, α=0.75) with Dice loss in training balances class imbalance and spatial overlap optimization.

Threat model

n/a — The paper focuses on medical image segmentation challenges related to lesion scale variability rather than security adversaries or attacks. The implicit 'threat' is clinical risk from missed or inaccurate lesion detection.

Methodology — deep read

The study begins by establishing the threat model implicit in medical image segmentation: the challenge is accurately detecting lesions of vastly differing scales in retinal fundus images to aid diabetic retinopathy diagnosis, rather than adversarial security threats.

Data was sourced from the publicly available Diabetic Retinopathy Dataset (DDR), partitioned into Training (383 images), Validation (149 images), and Testing (225 images) subsets. Each image comes with pixel-level annotations for four lesion types: Exudates (EX), Hemorrhages (HE), Microaneurysms (MA), and Soft Exudates (SE). Ground truth masks for each lesion type are binary and stacked into a 4-channel tensor.

Images were resized to two fixed input resolutions: 512×512 and 1024×1024 using bilinear interpolation for images and nearest-neighbor for masks to preserve label integrity. Aspect ratios were preserved with center cropping. Extensive data augmentation was applied to combat the small dataset size, including flips, rotations, scaling, color jitter, elastic deformations, and Gaussian blur with prescribed probabilities.

The core architecture innovation is the Multi-Resolution Feature Stem (MRFS) integrated with a UNet++ backbone. From a 1024×1024 input image, a 3-level image pyramid is created at 1024, 512, and 256 resolutions. Each scale is processed in parallel by shared-weight convolutional blocks consisting of two 3×3 convolutions, batch normalization, and ReLU activation, producing 64-channel feature maps. The lower resolution feature maps are upsampled via bilinear interpolation to match 1024×1024 resolution and concatenated (resulting in 192 channels). A 1×1 convolution reduces this to 64 channels, which is fed as input to UNet++ encoder, replacing the initial 3-channel input layer.

The shared-weight design encourages scale-invariant feature learning. The MRFS therefore enables the model to encode fine-grained features from high resolution alongside broader contextual cues from downsampled images simultaneously at the input stage.

Training used a combined loss of focal loss (γ=2.0, α=0.75) and Dice loss weighted 0.6 and 0.4 respectively, addressing class imbalance and spatial segmentation overlap. Models were trained primarily on an Apple MacBook Pro M4 Max GPU accelerated with Metal Performance Shaders, and for heavier runs on Google Colab Pro with A100 GPUs utilizing CUDA and mixed-precision. Batch size, epochs, and optimizer specifics were not explicitly detailed.

Evaluation metrics included mean Average Precision (mAP), mean Intersection over Union (mIoU), and per-class Average Precision (AP) and IoU for each lesion type. Baselines tested were U-Net, UNet++, DeepLab-v3+, and instances of the proposed MRFS model. Experiments tested sensitivity to input resolution (512 vs 1024) for baselines and performance gains using MRFS vs best baseline on test dataset.

An example workflow: a 1024×1024 RGB fundus image is downsampled to form the image pyramid. Each scale passes through shared conv blocks producing feature maps. These are upsampled as needed, concatenated, fused, and input to UNet++ encoder. The network predicts a 4-channel segmentation mask. Loss is computed by focal plus dice losses against ground truth lesion masks. Performance is then quantitatively compared to baselines on validation and test sets.

Code and training details are publicly available, supporting reproducibility, though dataset is closed-source beyond DDR. Some hyperparameters and training regime details are inferred from standard practices but not exhaustively reported.

Technical innovations

• Discovery of the opposing effects of increasing input resolution on segmentation performance of different DR lesion types, with higher resolution benefiting microaneurysms but degrading hemorrhage detection.
• Multi-Resolution Feature Stem design that creates an input-level image pyramid processed in parallel with shared-weight convolutional blocks to extract scale-invariant features before backbone encoding.
• Fusion of multi-scale features via channel-wise concatenation and 1×1 convolution to produce a rich, unified feature map integrating fine details and context for downstream segmentation.
• Integration of the multi-resolution stem with UNet++ by replacing the initial input layer with one accepting the 64-channel fused multi-scale representation, enabling seamless end-to-end training.

Datasets

• Diabetic Retinopathy Dataset (DDR) — 757 images total (383 training, 149 validation, 225 test) — public but specialized medical dataset

Baselines vs proposed

• U-Net 512x512: mAP=0.2982, mIoU=0.1288 vs U-Net 1024x1024: mAP=0.5378, mIoU=0.3136
• UNet++ 512x512: mAP=0.3951, mIoU=0.1433 vs UNet++ 1024x1024: mAP=0.5220, mIoU=0.3513
• DeepLab-v3+ 512x512: mAP=0.3848, mIoU=0.2098 vs DeepLab-v3+ 1024x1024: mAP=0.3094, mIoU=0.1722
• Baseline UNet++ (1024) vs Proposed Multi-Resolution U-Net++ (1024): mAP 0.5220 vs 0.5656 (+7.8%), mIoU 0.3513 vs 0.4040 (+15.0%)
• Per Lesion - Exudates AP: baseline 0.5225 vs proposed 0.6562 (+25.58%), IoU: baseline 0.3391 vs proposed 0.4725 (+39.33%)
• Microaneurysms AP: baseline 0.1700 vs proposed 0.1942 (+14.2%), IoU: baseline 0.1532 vs proposed 0.1639 (+7.0%)
• Soft Exudates AP: baseline 0.0848 vs proposed 0.3016 (+255.68%), IoU: baseline 0.1092 vs proposed 0.2217 (+103.04%)
• Hemorrhages AP: baseline 0.5253 vs proposed 0.4419 (-15.9%), IoU: baseline 0.3170 vs proposed 0.2832 (-10.66%)

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.08679.

Fig 1: Fundus photographs illustrating various DR lesions

Fig 2 (page 1).

Fig 3 (page 1).

Fig 4 (page 1).

Fig 5 (page 1).

Fig 6 (page 1).

Fig 2: Multi-Resolution Feature Stem architecture

Fig 3: Segmentation example for Exudates

Limitations

• Performance degradation on hemorrhages indicates the multi-resolution stem struggles to differentiate hemorrhages from retinal blood vessels due to shared-weight conv feature extraction.
• Limited dataset size (383 training images) restricts generalizability and may bias results despite augmentation.
• Training regime details such as number of epochs, optimizer parameters, and random seeds are incompletely described, limiting exact reproducibility.
• No adversarial or distribution shift evaluations were conducted to assess robustness in varied clinical settings or imaging protocols.
• Computational overhead of the multi-resolution stem increases inference time by 27% compared to baseline UNet++, which may restrict use in real-time applications.
• The approach is evaluated only on the DDR dataset, without cross-dataset validation to confirm broader applicability.

Open questions / follow-ons

• Can model adaptations improve hemorrhage segmentation performance without sacrificing gains on other lesion types?
• How would the multi-resolution stem perform under domain shifts such as different imaging devices or populations?
• Could adaptive, lesion-type-specific fusion or weighting mechanisms outperform the current shared-weight design?
• What is the trade-off between computational cost and segmentation accuracy in large-scale deployment scenarios?

Why it matters for bot defense

Although focused on medical image segmentation, the findings about input resolution trade-offs and multi-scale feature integration are relevant to bot defense and CAPTCHA learning architectures facing multi-scale visual challenges. The demonstrated limitation of uniformly increasing resolution might encourage designers to implement multi-resolution input stems or pyramids to balance fine detail and context for improved recognition accuracy. The shared-weight convolutional block approach could inspire lightweight multi-scale feature extraction in resource-limited CAPTCHA models. However, practical implications like computational overhead versus accuracy gains must be carefully considered in latency-sensitive environments such as CAPTCHAs. The methodological rigor in systematic resolution and architecture ablations offers valuable experimental design lessons.

Cite

@article{arxiv2607_08679,
  title={ Multi-Resolution Feature Stem for Diabetic Retinopathy lesion segmentation },
  author={ Indranil Dutta and Taehee Jeong },
  journal={arXiv preprint arXiv:2607.08679},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.08679}
}

Read the full paper

• arXiv:2607.08679 (abstract)
• arXiv:2607.08679 (PDF)