InfantFace: Detecting infant faces in neonatal clinical environments

Source: arXiv:2606.20449 · Published 2026-06-18 · By Abdullah Bin-Obaid, Maria M. Cobo, Rebeccah Slater, Lionel Tarassenko, Mauricio Villarroel

TL;DR

This paper addresses the critical challenge of accurately detecting infant faces in neonatal clinical environments, which is foundational for non-contact assessments such as pain expression analysis and physiological monitoring in neonates. Standard general-purpose face detectors struggle due to factors unique to NICU settings, including cluttered backgrounds, occlusions from medical devices, variable illumination, and infant fragility. The authors propose InfantFace, a one-stage YOLOv11m-based model specifically adapted for neonatal face detection. They train the model initially on large publicly available face datasets, then fine-tune it on a curated neonatal research dataset containing over 228 videos of 113 infants. The results demonstrate that InfantFace outperforms three state-of-the-art general face detectors before fine-tuning (achieving AP50=0.87 on neonatal data), and achieves significantly improved performance after domain adaptation (AP50=0.96). The study highlights the importance of clinical-domain adaptation for robust neonatal face detection and emphasizes the need for larger publicly accessible neonatal datasets to support future work in this area.

Key findings

• InfantFace model achieved an AP50 of 0.87 on neonatal research dataset before domain adaptation (Table 2), outperforming MTCNN (AP50 < 0.87), SCRFD, and DSFD models.
• After fine-tuning on neonatal research data, InfantFace improved AP50 to 0.96 and AP50:95 to 0.67 on neonatal dataset (Table 4).
• InfantFace reached an AP50 of 0.99 on VGGFace2 and FDDB datasets before clinical adaptation; WIDER FACE dataset had lowest pre-adaptation AP50 of 0.80 (Table 2).
• Fine-tuning on WIDER FACE yielded best generalization to neonatal research dataset with AP50 of 0.90 and AP50:95 of 0.47 (Table 3).
• AP50:95 metrics were consistently lower than AP50 across datasets, indicating bounding box localisation remains a challenge.
• Five-fold cross-validation with subject-separated splits was employed to avoid data leakage and robustly evaluate generalization.
• Data augmentation included horizontal flips, rotations ±90°, translation, shear, and multi-scale training with images resized 64×64 to 192×192.
• Standardising and refining annotations for single-face images via consensus of three detectors improved data quality and consistency.

Threat model

The threat model involves natural domain shift and image degradation challenges in neonatal clinical video data rather than malicious attackers. The task is to reliably detect infant faces despite occlusions, poor lighting, clutter, and motion associated with medical equipment and clinical interventions. There is no explicit adversary attempting to evade detection, but the model must be robust to intrinsic clinical environment variability and non-ideal imaging conditions.

Methodology — deep read

The threat model assumes the task is automated detection of infant faces from video frames recorded in neonatal clinical environments, requiring robustness to occlusions, illumination changes, and clutter. The adversary is effectively the complex clinical setting and image quality degradation, not a malicious attacker.

The data provenance includes five datasets: four public general face datasets (VGGFace2 with 3.16 million images; CelebA with 202k images; FDDB with 2,845 images; WIDER FACE with 32,203 images) and one neonatal research dataset comprising 228 videos from 114 sessions involving 113 infants collected in UK hospitals between 2012-2023. The neonatal data involves varied clinical settings including NICU and postnatal wards, and videos capture acute pain procedures and controls.

Images were processed to obtain single-face annotations by combining the bounding box predictions of three face detectors (MTCNN, SCRFD, DSFD) and retaining images where all three detectors agreed with IoU > 0.60. For neonatal videos, nine frames per video were extracted (one per 5-second interval), downsampled, and manually annotated.

The core architecture is based on YOLOv11m, a one-stage object detector pretrained on COCO (80 classes). The detection head was modified for single-class detection (face) and trained with standard YOLO modules: a backbone network, neck with spatial pyramid pooling, and multi-scale detection head. Losses combined Complete IoU loss for bounding box regression, binary cross-entropy classification loss, and Distribution Focal Loss for localization precision, with weighted sums emphasizing tight bounding boxes (λ_box=8.0).

Training used five-fold cross-validation with subject-separated splits to avoid leakage, stratified by identity or image category depending on dataset. Images were resized to 128×128 with zero padding, and multiple augmentations were applied including horizontal flips, ±90° rotations, translations, shearing, perspective warps, multi-scale training with sizes from 64×64 to 192×192 pixels, and HSV color augmentation.

Models were initially trained on VGGFace2 (base training), fine-tuned separately on CelebA, FDDB, and WIDER FACE for benchmark evaluation, and subsequently fine-tuned again on neonatal research data to adapt to clinical domain. Evaluation metrics were average precision at IoU thresholds (AP50 and AP50:95) to measure detection and localization; these metrics were reported on held-out test splits subject-separated from training.

Baselines included three state-of-the-art face detectors: MTCNN, SCRFD, and DSFD. Ablation evaluated the effect of fine-tuning on different datasets and on neonatal domain adaptation. Statistical significance tests were not detailed. Code and annotations were released for reproducibility, though neonatal data remains private due to privacy.

A concrete example: starting with COCO-pretrained YOLOv11m, model was retrained on consensus single-face images from VGGFace2 with 5-fold cross-validation, applying augmentations; then fine-tuned on WIDER FACE training images; finally fine-tuned on neonatal frames annotated manually. The full model showed AP50 improvements from 0.87 to 0.96 on neonatal test samples after domain adaptation, demonstrating effective transfer and specialization.

Technical innovations

• Use of consensus annotations from three state-of-the-art face detectors to curate a high-quality single-face training set from large, noisy public datasets.
• Clinical domain adaptation via fine-tuning on a novel neonatal research dataset of video frames from NICU environments to handle real-world occlusions and lighting challenges.
• Application and tuning of YOLOv11m for single-class infant face detection with tailored loss weighting emphasizing bounding box precision (λ_box=8.0), improving localisation metrics.
• Systematic evaluation across multiple benchmark datasets and neonatal clinical data with a balanced training scheme and five-fold cross-validation to robustly assess generalization and domain transfer.

Datasets

• VGGFace2 — 3,163,185 images — public
• CelebA — 202,599 images — public
• FDDB — 2,845 images — public
• WIDER FACE — 32,203 images (16,106 publicly accessible) — public
• Neonatal research dataset — 2,052 frames extracted from 228 videos from 113 infants — private clinical data

Baselines vs proposed

• MTCNN: AP50 on neonatal research dataset = below 0.87 vs InfantFace pre-adaptation = 0.87
• SCRFD: lower AP50 than InfantFace pre-adaptation (exact numbers unspecified)
• DSFD: lower AP50 than InfantFace pre-adaptation (exact numbers unspecified)
• InfantFace (pre-domain-adaptation) on VGGFace2: AP50 = 0.99 vs on WIDER FACE: AP50 = 0.80
• InfantFace (fine-tuned on WIDER FACE) on neonatal data: AP50 = 0.90 vs InfantFace (neonatal-adapted): AP50 = 0.96
• InfantFace (neonatal-adapted) AP50:95 on neonatal dataset improved from 0.40 (pre-adaptation) to 0.67

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2606.20449.

Fig 1: Example detections from the evaluated face detectors across public datasets.

Fig 2 (page 7).

Fig 3 (page 8).

Fig 4 (page 9).

Fig 5 (page 9).

Fig 2: Schematic of the YOLOv11m architecture. It highlights the three main components of

Fig 7 (page 28).

Fig 8 (page 29).

Limitations

• Neonatal dataset size is limited (2,052 annotated frames from 113 infants), constraining robustness evaluation.
• Neonatal data is not publicly available, hindering reproducibility and external benchmarking on clinical data.
• No reported adversarial robustness tests or evaluations under extreme occlusions or unpredictable device placements.
• Performance trade-offs observed where domain adaptation improved neonatal detection but decreased some general dataset AP50:95 metrics.
• Lack of publicly accessible neonatal datasets limits broader research and method comparison.
• The model focuses mainly on neonates in acute pain-related clinical procedures, which may bias data and limit generalizability.

Open questions / follow-ons

• How well does the model generalize to other neonatal units with different equipment, lighting, or demographics?
• Can synthetic data augmentation or generative approaches effectively simulate neonatal occlusions and improve training?
• What are effective strategies to maintain both neonatal domain specialization and general face detection performance simultaneously?
• Could multimodal sensor fusion (e.g., thermal combined with RGB) further improve face detection under occlusion and low light?

Why it matters for bot defense

For bot-defense and CAPTCHA practitioners, this work illustrates the critical importance of domain adaptation when deploying face detection or recognition in specialized environments. Just as general face detectors perform poorly in neonatal clinical settings without adaptation, CAPTCHA or fraud detection systems relying on vanilla detectors may likewise degrade under domain shifts, occlusions, or environmental complexities. The paper’s methodology—combining large-scale public datasets with smaller domain-specific data and leveraging consensus annotations for high-quality labels—provides a blueprint for adapting detection systems to specialized contexts. Practitioners can apply similar fine-tuning pipelines and validate using cross-domain metrics emphasizing both detection presence (AP50) and accurate localization (AP50:95) to ensure reliable face or object detection under challenging operational conditions. The neonatal dataset scarcity also highlights the ongoing necessity for ethically sourced, privacy-compliant domain data to improve robustness against sophisticated evasion tactics involving occlusions or low-quality imaging.

Cite

@article{arxiv2606_20449,
  title={ InfantFace: Detecting infant faces in neonatal clinical environments },
  author={ Abdullah Bin-Obaid and Maria M. Cobo and Rebeccah Slater and Lionel Tarassenko and Mauricio Villarroel },
  journal={arXiv preprint arXiv:2606.20449},
  year={ 2026 },
  url={https://arxiv.org/abs/2606.20449}
}

Read the full paper

• arXiv:2606.20449 (abstract)
• arXiv:2606.20449 (PDF)