FaithIR: Rethinking Infrared Image Super-Resolution from Perceptual Sharpness to Task Relevant Fidelity

Source: arXiv:2608.03106 · Published 2026-08-04 · By Axi Niu, Zhenguo Wu, Kang Zhang, Qingsen Yan, Jinqiu Sun, Yanning Zhang

TL;DR

This paper addresses the problem of infrared image super-resolution (IISR), which is critical for downstream machine perception tasks such as object detection and semantic segmentation. Existing methods typically optimize for perceptual sharpness, often producing artificial textures, over-sharpened edges, and spurious high-frequency details that distort the authentic thermal structures of infrared images. These artifacts can negatively impact downstream task reliability despite visual improvements. To tackle this, the authors propose FaithIR, a novel framework that emphasizes task-relevant fidelity instead of perceptual sharpness. FaithIR consists of a patch-level conditioning branch extracting global thermal and structural context, and a pixel-level restoration branch performing dense local reconstruction guided by this global information. It operates directly in the pixel domain via conditional flow matching, preserving infrared-specific thermal patterns and semantic structures. Extensive experiments on FLIR-IISR, M3FD, and FMB datasets demonstrate that FaithIR achieves strong reconstruction fidelity, robust cross-dataset generalization, and notably improves downstream object detection and semantic segmentation compared to prior art. The results highlight the importance of faithful infrared structure preservation for reliable machine perception rather than appealing but misleading sharpness maximization.

Key findings

• FaithIR achieves the highest PSNR (24.91 dB) and SSIM (0.8121) on FLIR-IISR test set, surpassing prior IISR methods including Real-IISR and DifIISR (Table 1).
• FaithIR reduces FID to 74.28, outperforming the closest baseline Real-IISR (81.58), demonstrating better distributional consistency.
• In object detection on FLIR-IISR, FaithIR yields 594 detected instances and a 72.10% detection rate, outperforming all baselines by large margins (Table 1).
• On M3FD for ground-truth object detection, FaithIR achieves the highest mAP50 of 0.661 and superior class-wise recall compared to InfraFFN, CoRPLE, and DifIISR (Table 5).
• For semantic segmentation on FLIR-IISR, FaithIR attains lowest semantic deviation (3.66%), indicating more faithful preservation of task-relevant semantic cues (Table 1).
• On FMB semantic segmentation, FaithIR leads with 49.0% mIoU and consistently improved class-wise IoU (Table 6).
• Ablation shows pixel-domain restoration outperforms latent-space alternatives by +0.21 dB PSNR and -10.98 FID, with consistent gains in downstream detection and segmentation (Table 7).
• Patch-Level Conditioning Branch and pixel-wise AdaLN modulation both contribute significant improvements; removing or simplifying these components degrades performance noticeably.

Threat model

The paper does not target security threats or adversarial attacks. Its threat model assumes a standard supervised setting where the low-resolution infrared images and their high-resolution ground truths are known during training, and no malicious adversaries or perturbations are modeled.

Methodology — deep read

The core goal is to reconstruct high-resolution infrared images from low-resolution inputs in a way that faithfully preserves thermal structures and semantic information relevant to downstream tasks. The threat model assumes availability of paired low-high resolution infrared images; no adversarial threat is considered. The authors criticize existing methods relying on natural image priors that induce unnatural textures unsuitable for IR domain faithful reconstruction.

Data: FaithIR is trained from scratch on the FLIR-IISR dataset (1,192 training pairs; 265 testing pairs) containing 1024×768 HR and 256×192 LR infrared images. Additional evaluation is conducted on M3FD and FMB without fine-tuning to test cross-dataset generalization. LR inputs for M3FD and FMB were generated via 4× bicubic downsampling.

Architecture: FaithIR uses a dual-branch transformer-based framework operating directly in the pixel domain. A Patch-Level Conditioning Branch extracts global thermal and structural information from upsampled LR patches via adaptive layer normalization (AdaLN), multi-head self-attention, and gated residual MLP blocks. It outputs position-aware conditioning tokens encoding long-range dependencies and object layouts.

A Pixel-Level Restoration Branch takes the noisy corrupted HR estimate concatenated with the upsampled LR input and performs dense local reconstruction. It embeds pixel tokens per patch and modulates them with the conditioning tokens using pixel-wise AdaLN, enabling position-specific refinement inside each patch. Compact global attention tokenizes pixel tokens into a smaller set for scalable global interaction before pixel reconstruction.

Training: The framework is trained with a conditional flow matching objective, a continuous generative modeling approach. Given a noised intermediate image state controlled by a timestep, the model predicts a velocity field matching the ground truth high-res image minus noise. AdamW optimizer is used on 4 A100 GPUs with random 64×64 LR and corresponding 256×256 HR crops. Pretrained natural image models or tokenizers are not used to prevent cross-domain biases.

Evaluation: Quantitative full-reference metrics include PSNR, SSIM, LPIPS, and FID on FLIR-IISR, M3FD, and FMB. No-reference metrics CLIPIQA, MANIQA, MUSIQ, and NIQE supplement visual quality assessment. Downstream evaluation involves fixed YOLOv8n object detectors for detection metrics (mAP50 and recall) and SegFormer-B2 for semantic segmentation (mIoU and class IoU). Semantic deviation metric measures prediction consistency on FLIR-IISR. Ablation studies isolate the impact of pixel-domain restoration, patch-level conditioning, and pixel-wise modulation.

End-to-end example: At inference, starting from Gaussian noise, FaithIR integrates the velocity field learned during training from t=0 to t=1 via Euler steps, conditioned on the upsampled low-resolution IR image patches, producing the super-resolved high-resolution infrared output with preserved thermal patterns and contours.

Code release and pretrained weights are not mentioned, so reproducibility beyond dataset and training details remains uncertain.

Technical innovations

• Proposal of a dual-branch architecture combining patch-level global structural conditioning with pixel-level dense restoration tailored to infrared thermal image characteristics.
• Introduction of pixel-wise AdaLN modulation enabling position-specific conditioning within each image patch, improving fine-grained local detail recovery over patch-shared methods.
• Direct pixel-domain restoration formulation using conditional flow matching, avoiding potential distortion from latent image encodings prevalent in prior SR methods.
• Empirical demonstration that faithful infrared structure preservation improves downstream perception tasks more than simply enhancing perceptual sharpness or natural image priors.

Datasets

• FLIR-IISR — 1,457 paired infrared HR-LR images — public
• M3FD — size unspecified, infrared images with detection labels — public
• FMB — size unspecified, infrared images with segmentation labels — public

Baselines vs proposed

• PFT-SR: PSNR=23.67 vs FaithIR: 24.91 on FLIR-IISR
• SinSR: PSNR=22.93 vs FaithIR: 24.91 on FLIR-IISR
• HAT: PSNR=23.72 vs FaithIR: 24.91 on FLIR-IISR
• InfraFFN: mAP50=0.643 vs FaithIR: 0.661 on M3FD detection
• CoRPLE: mIoU=48.6 vs FaithIR: 49.0 on FMB segmentation
• Latent-Space DiT variant: PSNR=24.70 vs FaithIR pixel-domain: 24.91

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2608.03106.

Fig 1: Overview of the proposed FaithIR.

Fig 2: The proposed (a) Patch-Level Conditioning Block

Fig 3: Qualitative comparison on the FLIR-IISR test set.

Fig 4: Downstream task results on the FLIR-IISR test set. The first two rows show object detection, and the last two rows

Fig 5: Object detection results on M3FD using the same fixed detector. Green, red, and blue boxes indicate correct detections,

Fig 6: Semantic segmentation results on FMB using the same fixed segmentation model. Highlighted regions are enlarged

Fig 7: Qualitative comparison on the M3FD test set.

Fig 8: Qualitative comparison on the FMB test set.

Limitations

• No adversarial robustness or active attack scenarios examined; the model’s behavior under malicious inputs is unknown.
• Cross-dataset generalization tested only on two other datasets without fine-tuning; further domain shifts or real-world conditions might affect performance.
• Reproducibility is limited due to lack of public code or pretrained models mentioned.
• Focus is on paired supervised training; unsupervised or weakly supervised settings not explored.
• No explicit analysis on computational cost, latency, or scalability for real-time embedded deployment.
• The approach’s applicability to more severely degraded or real-world noisy infrared imagery is unclear.

Open questions / follow-ons

• Can the FaithIR pixel-domain conditional flow matching approach be extended to handle real-world uncontrolled infrared image degradation or noise?
• How does FaithIR perform under adversarial or perturbation attacks targeting downstream perception tasks, and can it be made robust?
• Is it possible to incorporate unsupervised or self-supervised training to reduce need for paired HR-LR infrared datasets?
• What are the latency and resource requirements for deploying FaithIR in real-time infrared sensing systems such as UAVs or autonomous vehicles?

Why it matters for bot defense

For bot-defense and CAPTCHA practitioners, the lessons from FaithIR highlight the importance of preserving task-relevant fidelity in image restoration rather than optimizing purely for human perceptual quality or sharpness. In scenarios where infrared images or other sensor modalities are used as inputs for automated recognition or verification systems, degradation or artificial alteration of authentic thermal or structural cues can undermine robustness and security.

FaithIR’s dual-branch architecture and pixel-level conditional flow matching suggest a promising direction for designing restoration pipelines that improve machine reliability without introducing misleading artifacts. This principle can extend to other imaging modalities used for bot detection challenges, including adapting restoration models to maintain semantic and structural integrity. FaithIR also demonstrates the need to evaluate super-resolution or enhancement models not only by traditional perceptual metrics but also on downstream task performance metrics critical to security-sensitive automation.

Cite

@article{arxiv2608_03106,
  title={ FaithIR: Rethinking Infrared Image Super-Resolution from Perceptual Sharpness to Task Relevant Fidelity },
  author={ Axi Niu and Zhenguo Wu and Kang Zhang and Qingsen Yan and Jinqiu Sun and Yanning Zhang },
  journal={arXiv preprint arXiv:2608.03106},
  year={ 2026 },
  url={https://arxiv.org/abs/2608.03106}
}

Read the full paper

• arXiv:2608.03106 (abstract)
• arXiv:2608.03106 (PDF)