Counterfactual Explainability Framework With CycleGAN And Counterfactual-Classifier Alignnment Score for Retinal Disease Classification

Source: arXiv:2607.21068 · Published 2026-07-23 · By Kritanu Chattopadhyay, Sayanjit Singha Roy, Soumya Chatterjee

TL;DR

This paper addresses explainability challenges in deep learning models for retinal disease classification from fundus images, a clinically important task for early diagnosis but hindered by the black-box nature of neural networks. The authors propose CounterFundus, a novel counterfactual explainability framework using a CycleGAN to generate visually plausible healthy (normal) counterparts of diseased retinal fundus images. By comparing the original pathological image to its generated counterfactual, they derive difference maps that highlight disease-relevant retinal regions. Complementing this, a new Counterfactual-Classifier Alignment Score (CCAS) is introduced to quantitatively measure spatial agreement between these difference maps and classifier saliency maps (EigenCAM), combining rank correlation, IoU, and pointing accuracy into a robust single metric. Results demonstrate that CounterFundus achieves strong disease classification performance with an EfficientNet-B5 backbone (95.38% accuracy, 99.69% AUC) and produces counterfactual explanations that are both visually plausible and well-aligned with classifier-relevant retinal areas.

Key findings

• EfficientNet-B5 classifier achieved 95.38% accuracy, 95.31% macro F1-score, and 99.69% AUC on a held-out test set of 4217 fundus images.
• The CycleGAN-based counterfactual generator produced healthy counterparts with average SSIM of 0.783 and PSNR of 21.63 dB.
• Classifier confidence on disease classes decreased by ~0.67 points after counterfactual translation, while normal-class confidence increased by the same margin, indicating semantic disease-to-normal image translation.
• The proposed CCAS metric yielded strong spatial agreement between difference maps and EigenCAM saliency, e.g., overall Spearman correlation of 0.95, [email protected] of 0.563, and pointing accuracy of 0.547.
• EigenCAM outperformed GradCAM, GradCAM++, and LIME in CCAS evaluation, with respective IoUs of 0.563, 0.22, 0.187, and 0.2.
• Augmenting classifier training with CCAS-filtered counterfactual samples improved accuracy from 95.38% to 97.51% while reducing synthetic data by 28%.
• External validation on the RFMiD dataset showed generalizability of counterfactual explanations with CCAS Spearman ρ = 0.48 and IoU = 0.49 without adaptation.
• Class-wise, diabetic retinopathy showed perfect precision, recall, and F1-score of 100%, reflecting strong localization and classifier performance.

Threat model

n/a - This work does not explicitly consider adversarial threats or security adversaries. The focus is on building explainability methods to improve trust in retinal disease classification models.

Methodology — deep read

1. Threat Model & Assumptions: The adversary is not explicitly modeled as this is an explainability framework. The focus is on deep learning classification of retinal disease images with the assumption that model interpretability and spatial alignment between explanation and classifier saliency are critical for clinical trust.

2. Data: The dataset comprises 4217 RGB fundus images categorized into normal, diabetic retinopathy, glaucoma, and cataract classes (balanced roughly 1000 images each). The data was split 80/20 stratified holdout for training/testing, with 5-fold stratified cross-validation on the training portion.

3. Architecture / Algorithm:

• Classification backbone: EfficientNet-B5 with a modified classification head (two-layer MLP with dropout and ReLU).
• Counterfactual generator: CycleGAN with two ResNet-based generators translating diseased to normal (GD→N) and vice versa, plus PatchGAN discriminators. Losses combine adversarial, cycle-consistency (L1), and identity losses with hyperparameters λcyc=7 and λidt=0.3.
• Difference map: Absolute channel-wise difference between original diseased image and counterfactual, weighted by radial mask to suppress boundary noise.
• CCAS metric: Combines Spearman rank correlation (ρ), binary IoU, and pointing accuracy (PA) to quantify spatial alignment between difference maps and EigenCAM saliency.

4. Training Regime:

• Classifier trained with AdamW optimizer, learning rate 1e-4, weight decay 1e-4, batch size 16, over 35 epochs with cosine annealing.
• Label smoothing (ϵ=0.1) and mixup augmentation (α=0.4) were applied to improve generalization.
• CycleGAN trained for 150 epochs with Adam optimizer (β1=0.5, β2=0.999), learning rate 2e-4 decayed linearly after epoch 75.

5. Evaluation Protocol:

• Metrics: classification accuracy, F1-score, ROC-AUC for disease detection; SSIM, PSNR, and FID for counterfactual quality; CCAS components for spatial alignment.
• Baselines: GradCAM, GradCAM++, and LIME saliency methods compared against EigenCAM.
• Ablations: Effect of adding all vs CCAS-filtered synthetic counterfactual images on classifier performance.
• External validation performed without retraining on RFMiD dataset.

6. Reproducibility:

• The paper does not state availability of code or weights publicly.
• Used publicly available Kaggle multi-class fundus dataset for training.

Example end-to-end flow: A diseased fundus image is passed through the CycleGAN generator to produce a healthy counterfactual image. The pixel-wise difference map between the original and counterfactual localizes pathology. Simultaneously, the classifier saliency map (EigenCAM) is computed from the EfficientNet-B5 model. The CCAS metric then quantitatively evaluates spatial overlap between the two maps, confirming that the counterfactual focuses on classifier-relevant retinal changes.

Technical innovations

• Use of CycleGAN-driven disease-to-normal retinal fundus image translation to generate visually plausible counterfactuals for explainability.
• Introduction of the Counterfactual-Classifier Alignment Score (CCAS), combining Spearman correlation, IoU, and pointing accuracy to quantitatively assess spatial agreement between counterfactual difference maps and classifier saliency maps.
• Application of CCAS as a filtering mechanism for selecting high-quality synthetic counterfactual images for data augmentation, improving classification performance while reducing synthetic data volume.
• Demonstration that EigenCAM provides stronger spatial alignment with counterfactual explanations than commonly used gradient-based saliency methods like GradCAM and GradCAM++ in retinal disease detection.

Datasets

• Kaggle Multi-class Fundus Dataset — 4217 images — publicly available online
• RFMiD Retinal Fundus Multi-disease Image Dataset — size not specified here — publicly available as external validation

Baselines vs proposed

• GradCAM: IoU = 0.22 vs EigenCAM: IoU = 0.563
• GradCAM++: IoU = 0.187 vs EigenCAM: IoU = 0.563
• LIME: IoU = 0.2 vs EigenCAM: IoU = 0.563
• Classifier baseline (A1): Accuracy = 95.38% vs + all CF augmentation (A2): 97.99%
• • CCAS-filtered CF augmentation (A3): Accuracy = 97.51% vs A2: 97.99%

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.21068.

Fig 1: Fundus images of (a) normal, (b) diabetic retinopathy,

Fig 2: Proposed CounterFundus pipeline.

Fig 3: The EfficientNet-B5 classifier is observed to converge

Fig 4: Disease-to-normal counterfactual translation results for

Fig 5: t-SNE projection of EfficientNet-B5 features.

Fig 6: CCAS visualization for (1) diabetic retinopathy, (2)

Fig 7: Qualitative comparison of saliency maps across repre-

Fig 8 (page 6).

Limitations

• The study uses a single 4-class Kaggle fundus dataset primarily, limiting diversity; external validation on RFMiD shows weaker performance (Spearman 0.48) indicating possible domain shift issues.
• Only retinal fundus imaging is considered; generalization to other ophthalmic or medical imaging modalities remains untested.
• No adversarial robustness evaluations or tests under deliberately perturbed inputs to examine counterfactual stability or reliability under attack.
• The CCAS metric, while novel, is evaluated mainly with EigenCAM saliency; further validation against expert lesion annotations or other saliency methods could strengthen claims.
• Code and trained model weights were not publicly released, which limits reproducibility outside the reported experiments.
• The potential clinical usability and interpretability by practitioners while promising remain unvalidated through user studies.

Open questions / follow-ons

• How well does the CCAS metric correlate with expert clinician annotations or lesion masks to verify clinical relevance of counterfactual explanations?
• Can the CounterFundus framework be adapted or extended for other medical imaging domains such as OCT, dermoscopy, or chest X-rays as proposed for future work?
• What are the effects of adversarial perturbations or distribution shifts on counterfactual explanation fidelity and classifier alignment?
• How might model uncertainty or multi-modal approaches (e.g., combining fundus with patient metadata) improve counterfactual generation and interpretability?

Why it matters for bot defense

For bot-defense and CAPTCHA practitioners, this paper highlights how counterfactual explanation techniques combined with generative models (CycleGAN) can yield spatially meaningful interpretations aligning with classifier decision regions. This spatial alignment measurement approach (CCAS) offers a quantitative evaluation protocol that can bridge the gap between visual plausibility and model relevance in explanations. Such methodology could inspire analogous approaches to explainability and robustness validation in CAPTCHA-solving models or bot detection classifiers, particularly when interpretability needs to be grounded in domain-specific features. Moreover, the use of counterfactual data augmentation filtered by spatial alignment metrics suggests a pathway to improve model robustness and generalization, which is crucial for bot-defense systems exposed to adversarial probing. However, the domain difference means direct application requires adaptation, and the focus here is on medical imaging rather than security attacks.

Cite

@article{arxiv2607_21068,
  title={ Counterfactual Explainability Framework With CycleGAN And Counterfactual-Classifier Alignnment Score for Retinal Disease Classification },
  author={ Kritanu Chattopadhyay and Sayanjit Singha Roy and Soumya Chatterjee },
  journal={arXiv preprint arXiv:2607.21068},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.21068}
}

Read the full paper

• arXiv:2607.21068 (abstract)
• arXiv:2607.21068 (PDF)