Occlusion-Aware Panoptic Segmentation with Joint Position Embedding and Occlusion-Level Attention

Source: arXiv:2607.18112 · Published 2026-07-20 · By Wenbo Wei, Jun Wang, Shan Raza, Abhir Bhalerao

TL;DR

This paper addresses the challenge of panoptic segmentation in scenes with occlusions, a common problem where current transformer-based methods still struggle despite advances. The authors identify that prior work neglects to explicitly model occlusion, limiting accuracy when objects partially obstruct each other. To tackle this, they propose PEMOLA, a lightweight occlusion-aware module that modulates position embeddings by integrating occlusion priors extracted via an occlusion-level attention map and a learnable occlusion label embedding. They train a dedicated occlusion classifier on the COCO-OLAC dataset with coarse image-level occlusion labels (low, medium, high) to generate spatial occlusion cues, which then guide the position embedding adaptation in transformer-based panoptic segmentation models. Furthermore, the authors annotate Cityscapes with a consistent occlusion labeling scheme (Cityscapes-OLAC) to evaluate generalization.

Extensive experiments on COCO-OLAC and Cityscapes-OLAC confirm that integrating PEMOLA consistently improves segmentation quality across occlusion severity levels and model architectures (Mask2Former, Mask DINO). Gains of up to +3.3 PQ on low occlusion COCO-OLAC and +0.8 PQ on Cityscapes demonstrate robustness with minimal computational overhead. Ablations verify that both occlusion-level attention and occlusion label embedding contribute additively. Visualizations show the modulated position embeddings better capture occlusion patterns to disambiguate object boundaries. Overall, the work highlights the importance and effectiveness of incorporating occlusion priors for panoptic segmentation in complex scenes.

Key findings

• PEMOLA integrated into Mask2Former yields up to +1.4 PQSt, +2.4 APThpan, +3.0 mIoUpan on low occlusion images in COCO-OLAC (Table III).
• PEMOLA integrated into Mask DINO shows +3.3 PQ, +4.2 PQTh, +2.0 PQSt improvements under low occlusion and +1.0 PQ, +1.5 PQTh, +1.9 mIoUpan under high occlusion (Table III).
• Overall PQ improvement on COCO-OLAC is +0.8 over Mask2Former baseline and Mask DINO baseline (Table IV).
• On Cityscapes-OLAC, PEMOLA improves Mask2Former by +0.8 PQ, +1.4 PQTh, +3.3 APThpan, +1.3 mIoUpan, showing cross-dataset generalization (Table V).
• Occlusion classifier trained on COCO-OLAC achieves 78.2% top-1 accuracy on val black-background images with Swin-L backbone (Table I).
• Occlusion classifier generalizes well on Cityscapes-OLAC with 86.2% top-1 acc on black-background val set (Table II).
• Removal of occlusion-level attention or occlusion label embedding causes PQ drops of 0.6 and 0.5 respectively in ablations, confirming each is valuable (Table VI).
• Applying PEMOLA modulation to pixel decoder improves PQ by +0.8 versus no change on transformer decoder insertion (Table VIII).

Threat model

The adversary is implicit: the challenge is partial occlusion of objects causing segmentation errors. The model must identify and separate occluded instances despite incomplete visual information. The system assumes no adversarial manipulation or intentionally deceptive occluders, only natural occlusion patterns. The occlusion classifier has limited knowledge in the form of image-level occlusion labels but no access to explicit occludee masks or depth order.

Methodology — deep read

1. Threat Model & Assumptions: The method assumes access to image-level occlusion severity labels (low, medium, high) rather than instance-level occlusion masks. The occlusion classifier is trained under supervised conditions on these labels to generate occlusion priors. The transformer-based panoptic segmentation model processes images where occlusion is common but unknown at training except via these labels.

2. Data:

• COCO-OLAC: Extension of COCO dataset with image-level occlusion annotations.
• Cityscapes-OLAC: Newly annotated Cityscapes images with occlusion labels following COCO-OLAC protocol (2,975 train images, 500 val). Images are preprocessed to blacken non-object regions for the occlusion classifier to focus on foreground. Train/val splits use 90%/10% splits on COCO-OLAC training and official val sets for testing.

3. Architecture & Algorithm:

• Occlusion Classifier: Trained to classify images into low, medium, or high occlusion. Tested variants from ResNet-50/101 to Swin Transformers (Swin-L chosen). Uses standard binary cross-entropy loss.
• PEMOLA Module: Modulates the position embedding of the panoptic segmentation transformer's pixel decoder. -- Occlusion-Level Attention: Obtained via Grad-CAM on the occlusion classifier, producing a spatial map highlighting occluded regions. -- Occlusion Label Embedding: Learnable vector embedding for the occlusion severity label encoding channel-wise modulation. -- Joint Position Embedding Modulation: The standard position embedding tensor Epos is elementwise multiplied by (1 + (Occlusion-Level Attention ⊗ Occlusion Label Embedding)) as a residual modulation. This preserves base position info while adaptively scaling features where occlusions occur.

4. Training Regime:

• Occlusion classifier trained with blackened background images for foreground focus.
• Panoptic segmentation models pretrained and fine-tuned with PEMOLA added without altering architecture.
• Learning rates adjusted per model; experiments on ResNet-50 backbones for fair comparisons.
• Hardware mentioned includes University of Warwick's HPC cluster but detailed training epochs and batch sizes are not explicitly specified in the paper summary.

5. Evaluation Protocol:

• Metrics: Panoptic Quality (PQ), PQ for "thing" (PQTh) and "stuff" (PQSt) classes, instance average precision APThpan, semantic mIoUpan.
• Baselines: Mask2Former, Mask DINO, Panoptic FPN, Panoptic FCN, Panoptic-DeepLab, YOSO.
• Ablation slices explore removing occlusion priors and Grad-CAM smoothing.
• Cross-dataset evaluation performed on Cityscapes-OLAC.
• No explicit adversarial or distribution shift tests beyond cross-dataset generalization.

6. Reproducibility:

• The authors release code and datasets at https://github.com/wenbo-wei/PEMOLA.
• The Cityscapes-OLAC occlusion annotations are newly created and publicly shared.
• Full details on exact training hyperparameters and seeds are sparse but sufficient for reproduction.

Concrete Example: An input image from COCO-OLAC is preprocessed (non-object pixels blacked out), passed through the Swin-L based occlusion classifier which outputs a confidence distribution over occlusion levels. Grad-CAM produces an occlusion-level attention heatmap highlighting likely occluded regions. This heatmap is interpolated and modulated channel-wise with a learnable embedding vector for the predicted occlusion level. The modulation is applied elementwise as a residual to the position embedding tensor of the panoptic segmentation pixel decoder, improving spatial awareness of occlusion locations and severity. The panoptic segmentation network then uses these modulated position embeddings to better disambiguate overlapping and occluded objects in the output mask predictions. Experimental results confirm improved PQ and APThpan across occlusion severities.

Technical innovations

• The introduction of PEMOLA, a lightweight module that modulates transformer position embeddings with occlusion priors, improving spatial representation under occlusion.
• Use of Grad-CAM derived occlusion-level attention maps from a dedicated occlusion classifier to provide spatial occlusion cues for position embedding modulation.
• Joint modulation of position embedding by combining spatial occlusion attention with a learnable channel-wise occlusion label embedding to adaptively scale positional information.
• Manual annotation of Cityscapes-OLAC with image-level occlusion labels consistent with COCO-OLAC to enable cross-dataset occlusion-aware evaluation.

Datasets

• COCO-OLAC — ~10k images (subset of COCO with occlusion severity labels) — public
• Cityscapes-OLAC — 2,975 train + 500 val images with manual occlusion severity labels — newly created by authors and public

Baselines vs proposed

• Mask2Former baseline on COCO-OLAC: PQ = 40.7 vs +PEMOLA: PQ = 41.5
• Mask DINO baseline on COCO-OLAC: PQ = 44.0 vs +PEMOLA: PQ = 44.8
• Mask2Former baseline on Cityscapes-OLAC: PQ = 61.5 vs +PEMOLA: PQ = 62.3
• Occlusion classifier (Swin-L) on COCO-OLAC black-background validation images: 78.2% top-1 accuracy
• Occlusion classifier (Swin-L) on Cityscapes-OLAC black-background validation images: 86.2% top-1 accuracy

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.18112.

Fig 1: Examples of occlusion-induced segmentation failures from COCO

Fig 2: Overview of the proposed PEMOLA module. PEMOLA modulates position embedding using two occlusion priors: the occlusion-level attention extracted

Fig 3 (page 1).

Fig 4 (page 1).

Fig 3: Visualisation of position embedding before and after PEMOLA.

Fig 6 (page 3).

Fig 7 (page 3).

Fig 8 (page 3).

Limitations

• Occlusion annotation is at coarse image-level only, not instance-level, limiting fine-grained occlusion reasoning.
• Only panoptic segmentation models based on current transformer architectures tested; other backbones or methods unexplored.
• Ablation studies do not extensively test robustness under large domain shifts beyond Cityscapes-OLAC.
• Computational overhead claimed minimal but exact impact on latency or memory reported only qualitatively.
• No adversarial or real-world deployment evaluations under controlled occlusion manipulation.
• Details on hyperparameters, training epochs, and hardware for full reproduction are not comprehensive.

Open questions / follow-ons

• Can PEMOLA be extended to instance-level or amodal occlusion reasoning for finer-grained segmentation?
• How does PEMOLA perform on video panoptic segmentation tasks with temporal occlusion dynamics?
• What is the impact of alternative occlusion prior sources beyond Grad-CAM, e.g., learned occlusion maps or depth cues?
• Can the module adapt dynamically to unseen occlusion types beyond the training distribution without performance degradation?

Why it matters for bot defense

Occlusion handling is critical in bot defense scenarios where visual challenges such as CAPTCHAs may contain overlapping or partially hidden objects. This work provides a promising avenue to improve segmentation and recognition robustness by incorporating explicit occlusion priors into transformer-based image models, enabling more reliable parsing of complex scenes. The lightweight PEMOLA module could be integrated with existing transformer visual backbones used in CAPTCHA analysis pipelines, improving accuracy especially in cases of object overlap or deliberate occlusion attacks. Additionally, the cross-dataset evaluation on Cityscapes-OLAC indicates reasonable generalization, a desirable property for real-world defense applications where data distributions vary. However, since the method relies on supervised occlusion labels and only image-level occlusion awareness, further research would be needed to adapt it fully for adversarial or intentionally deceptive CAPTCHA scenarios where occlusion statistics differ substantially.

Cite

@article{arxiv2607_18112,
  title={ Occlusion-Aware Panoptic Segmentation with Joint Position Embedding and Occlusion-Level Attention },
  author={ Wenbo Wei and Jun Wang and Shan Raza and Abhir Bhalerao },
  journal={arXiv preprint arXiv:2607.18112},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.18112}
}

Read the full paper

• arXiv:2607.18112 (abstract)
• arXiv:2607.18112 (PDF)