Fast Object Removal Attacks on Safety-Critical Video-based Perception Systems

Source: arXiv:2608.02806 · Published 2026-08-03 · By Mohammad Imtiaz Hasan, M Sabbir Salek, Nathan Jones, Mashrur Chowdhury, Rong Ge

TL;DR

This paper investigates a novel, near-real-time semantic object removal attack targeting video-based perception systems commonly used in intelligent transportation systems (ITS). The attack manipulates video frames from infrastructure-mounted cameras by removing specific target objects (e.g., pedestrians) and replacing them with visually coherent background patches drawn from temporally adjacent frames. This approach preserves high visual similarity to original frames while suppressing the detections of targeted objects, threatening safety-critical ITS applications that rely on accurate object detection. The authors validate their attack framework on real-world video data captured at an intersection within the South Carolina Connected Vehicle Testbed (SC-CVT). They achieve high attack success rates, suppressing up to 97.59% of pedestrian detections and maintaining frame-level image similarity metrics above PSNR 40 dB and SSIM 0.996. The average runtime per frame is under 0.2 seconds on GPU, indicating near-real-time applicability. Forensic tamper-detection models struggled to distinguish manipulated frames from authentic ones, highlighting the stealthy nature of the attack. This emphasizes a concerning gap in the security of video perception for ITS and motivates future development of robust defenses against semantic object removal.

Key findings

• Object removal attacks reduce pedestrian detections by up to 97.59% using YOLO26 on real-world SC-CVT video frames.
• Frame-level attack success rate reached 94.48%, indicating most frames had the targeted object effectively removed from detection outputs.
• Reconstructed manipulated frames maintain high global visual fidelity with PSNR values >40 dB and SSIM values >0.996 at two tested resolutions (1080p and 720p).
• Mean execution time for the attack pipeline ranges from 0.074 to 0.172 seconds per frame on NVIDIA A100 GPU, enabling near-real-time performance.
• Both one-stage (YOLO26) and two-stage (Faster R-CNN) detectors show strong vulnerability with consistent object detection suppression.
• Seam-loss based adaptive padding blending improves visual smoothness and reduces boundary artifacts in reconstructed frames.
• Pretrained tamper-detection models (DeCLIP, PSCC-net, MVSS-net) exhibit limited ability to detect the manipulated frames, with ROC-AUC scores close to random chance.
• The attack exploits temporal coherence in fixed infrastructure cameras by copying coherent background patches from the most recent earlier frames without target objects.

Threat model

The adversary is assumed to have unauthorized access to the video stream or edge computing device processing video frames before object detection occurs. They can manipulate pixels in target object regions with the goal of removing these objects visually and semantically from detection outputs, without affecting other image regions. The attacker does not require priori knowledge of the object's location but relies on integrated real-time object detectors for localization. They cannot perform full-frame manipulation or cause gross image artifacts, as the attack must maintain visual plausibility to evade human and forensic detection. The adversary aims for stealthy, targeted object removal in a fixed-camera surveillance environment.

Methodology — deep read

The study assumes an adversary with unauthorized access to video frames at the edge computing device before the perception system processes them. The adversary lacks prior spatial knowledge of target objects but uses an integrated object detector (YOLO26 or Faster R-CNN) to identify target bounding boxes automatically in near real-time within each frame.

The attack pipeline consists of four sequential stages:

1. Target-Region Localization: Each video frame is processed by an object detector to produce bounding boxes for target objects (pedestrians). These define spatial regions for removal.

2. Temporal Clean-Patch Retrieval: For each target bounding box in the current frame, the algorithm searches backward through previous frames to find the most recent frame where the corresponding spatial region is free of any detected objects. The pixel patch from this clean historical frame is extracted and bilinearly resized if resolution varies. This exploits fixed camera viewpoints and temporal coherence.

3. Context-Aware Alpha Blending: The retrieved patch is inserted into the target region of the current frame, but a surrounding padded region is blended using a smooth alpha mask to reduce visible seams or boundary artifacts. A cubic smoothstep function shapes the alpha decay blending weights.

4. Frame Reconstruction: The blended patches for all target objects are sequentially inserted to generate the final attacked frame. A seam-loss function evaluates boundary smoothness for multiple padding widths, choosing the padding minimizing boundary intensity discontinuity.

The attack objective minimizes aggregate detection confidence scores of the targeted objects in the manipulated frame while preserving visual plausibility to evade human and forensic detection.

Data originates from 28 real-world videos from the SC-CVT testbed collected at an intersection, sampled at 0.5 fps and downscaled to 1080p and 720p. A total of 191 frames per resolution were processed.

Experiments run on NVIDIA A100 GPU with half-precision (YOLO26) and single-precision (Faster R-CNN) inference. The restored frames are quantitatively evaluated for PSNR and SSIM both frame-wide and within patched regions, attack success rate (reduction of detections), computational latency, and forensic tamper detection using multiple pretrained model checkpoints.

Comparisons include two detector architectures and two seam-loss types (L1 vs L2) to demonstrate attack robustness and reconstruction quality tradeoffs.

A concrete example: For pedestrian detected by YOLO26 with bounding box Bt(k) in frame It, the prior frames It-1, It-2 ... are checked for absence of any detected object overlapping Bt(k). The nearest frame with a clean patch Pt(k) is found, resized, alpha-blended with padding p to generate Rt(k)*. The attacked frame It^ is updated in that ROI. This pipeline executes for all detected pedestrians in the frame at <0.2 seconds each, producing a visually coherent but detection-suppressed frame.

Technical innovations

• A near-real-time video object removal attack that leverages temporally coherent background patches from recent frames instead of expensive deep inpainting.
• Context-aware alpha blending with adaptive padding optimized via seam-loss minimization for smooth integration of inserted patches.
• Detector-agnostic attack framework compatible with both one-stage and two-stage object detectors using only bounding box detection outputs.
• Evaluation of stealthiness through forensic tamper detection models demonstrating limited detection ability on manipulated frames.

Datasets

• South Carolina Connected Vehicle Testbed (SC-CVT) — 28 videos, 1920x1080 original resolution, 191 frames sampled per resolution at 0.5 fps — real-world ITS intersection data

Baselines vs proposed

• YOLO26 detector: baseline object detection rate vs attack reduces detection by up to 97.59%
• Faster R-CNN detector: similar mitigation in detection but slightly lower attack suppression metrics than YOLO26
• L1-based seam-loss vs L2-based seam-loss: patch-level PSNR higher and smoother boundary transitions with L2 loss
• Pretrained tamper detectors DeCLIP, PSCC-net, MVSS-net: ROC-AUC near 0.5 (random guessing) on identifying attacked frames vs clean frames

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2608.02806.

Fig 1: Effect of targeted object removal attack on ITS safety-critical functions showing 1)

Fig 3: (a) SSIM and (b) PSNR scores for different configurations

Fig 4: Qualitative comparison of original frames captured at an intersection and reconstructed

Fig 5: Object detection reduction rate and 𝐀𝐒𝐑𝐟𝐫𝐚𝐦𝐞 score for different object detection models,

Fig 6: Object detection time and frame reconstruction time for different object detection models,

Fig 7: (a) Accuracy, (b) F1-score, and (c) ROC-AUC value for DeCLIP model checkpoints

Fig 8: Accuracy, F1-score, and ROC-AUC value for different PSCC-net model

Fig 9: (a) Accuracy, (b) F1-score, and (c) ROC-AUC score for MVSS-net plus model checkpoints

Limitations

• Attack effectiveness depends on availability of spatially and temporally consistent background patches in previous frames, limiting use to static or near-static camera systems.
• The approach assumes unauthorized edge device access, which may not represent all realistic threat scenarios.
• No evaluation under shifting environmental conditions with large illumination or weather changes impacting patch retrieval quality.
• Real-time performance reported on high-end NVIDIA A100 GPUs; feasibility on lower-end edge devices remains unclear.
• Limited to pedestrian target class evaluation; applicability to other object types or multiple simultaneous targets needs further study.
• No analysis of adaptive defenses or retraining the detector to resist repetition of attack signatures.

Open questions / follow-ons

• How resilient are modern object detectors retrained on patched or attack-augmented datasets to this attack vector?
• Can temporal consistency-based defenses detect or mitigate these stealthy patch-insertion attacks in live video streams?
• To what extent can this attack be extended to moving-camera scenarios where viewpoint varies significantly between frames?
• How does environmental variability (lighting, weather, occlusion) affect patch retrieval reliability and attack success?

Why it matters for bot defense

For bot-defense and CAPTCHA practitioners, this work highlights a compelling semantic attack vector on video-based perception systems that rely on real-time object detection. The demonstrated object removal attack compromises the foundational assumption that digital input streams reflect physical scenes without tampering — a crucial assumption in many video-based verification and authentication mechanisms. This attack's near-real-time nature and high visual fidelity expose the risk that malicious actors could invisibly remove or alter targeted entities (e.g., humans) from image streams processed for safety or verification.

Practitioners should consider that detection-based bot defenses leveraging video data channels may be vulnerable to manipulation via temporal patch replacement attacks rather than classical adversarial perturbations. Incorporating temporal coherence checks, detector ensemble strategies, and tamper-evident video encoding could form a defensive baseline against such removal attacks. Benchmarking detection efficacy not just on static images but on adversarially reconstructed video frames is also advisable. Overall, this paper underscores the need to robustly verify spatial-temporal consistency of video streams in security-critical perception contexts.

Cite

@article{arxiv2608_02806,
  title={ Fast Object Removal Attacks on Safety-Critical Video-based Perception Systems },
  author={ Mohammad Imtiaz Hasan and M Sabbir Salek and Nathan Jones and Mashrur Chowdhury and Rong Ge },
  journal={arXiv preprint arXiv:2608.02806},
  year={ 2026 },
  url={https://arxiv.org/abs/2608.02806}
}

Read the full paper

• arXiv:2608.02806 (abstract)
• arXiv:2608.02806 (PDF)