EgoPolice: A Benchmark for Egocentric Video Understanding in High-Stakes Police Body-Worn Camera Footage

Source: arXiv:2607.06468 · Published 2026-07-07 · By Max Gonzalez Saez-Diez, Jihoon Chung, Adam D. Wolsky, Gregory Lanzalotto, Dean Knox, Jonathan Mummolo et al.

TL;DR

EgoPolice is a novel benchmark dataset designed for egocentric video understanding in real-world police body-worn camera (BWC) footage. The dataset consists of over 180 hours of police-civilian interactions sourced from multiple U.S. law enforcement agencies, with detailed per-second annotations of critical police actions relevant for social science and policing research. EgoPolice captures challenging high-stakes scenarios characterized by rapid, irregular camera motion, dense human interactions, occlusions, and rare but important events like weapon draws and handcuffing. Prior egocentric datasets primarily focus on benign or scripted daily activities, whereas EgoPolice presents more visually uniform scenes that require fine-grained contextual reasoning due to minimal background cues.

The authors benchmark a suite of state-of-the-art video and vision-language models on supervised action classification and zero-shot multiple-choice question answering tasks using EgoPolice. Despite advances in video models like VideoMAE V2 and VLMs such as Gemini 2.5 Pro, performance remains moderate especially on complex or rare actions with noisy video conditions, demonstrating a substantial performance gap for real-world deployment. Beyond serving as a research benchmark, EgoPolice is already supporting ongoing human-in-the-loop applications for large-scale police video review, evidencing practical value for facilitating police oversight through AI-assisted analysis.

Key findings

• EgoPolice comprises 2,684 body-worn camera videos totaling 184 hours, with 130,739 seconds of annotated police-civilian interactions across 9 key action classes.
• Severe egocentric camera motion is prevalent; optical flow magnitudes during action segments are significantly higher than prior egocentric datasets like Ego4D (Fig 3b), introducing visual degradation such as blur and rapid viewpoint changes.
• Visual context cues are less discriminative: CLIP embedding similarity distributions show that intra-class and inter-class frames in EgoPolice have almost overlapping distributions (TVD=0.150), unlike Kinetics (TVD=0.573) or ActivityNet (TVD=0.452), implying reliance on fine-grained motion and contextual features.
• Linear probing experiments reveal that video-pretrained models (VideoMAE V2 with mean F1 up to 58.4% on 1-min windows) outperform image-based models like CLIP and DINOv2, underlining the benefit of motion-aware representations (Table 4).
• Performance varies greatly by class: models score well (>60% F1) on visually salient classes like Civilian-On-Ground but poorly (<20% F1) on rare or fine-grained classes like Any Officer-Handcuffing or BWC Running.
• Zero-shot video-language models struggle especially small/open-source versions (e.g., LLaVA-Mini at ~15%) while proprietary models like Gemini 2.5 Pro reach only ~77% accuracy on 1-min clips (Table 5), insufficient for autonomous deployment.
• Common failure modes include low-light conditions, motion blur, rapid short-duration events, object misidentification (e.g., flashlights confused with weapons), and action misattribution in crowded scenes (Fig 4).
• The dataset’s rigorous annotation pipeline achieved high inter-annotator agreement (Krippendorff’s alpha ~79.4%), supported by specific objective definitions to minimize subjective bias.

Threat model

The adversary is effectively the inherent visual and temporal noise present in body-worn camera footage—characterized by rapid, irregular camera motion, occlusions, variable lighting, and crowded scenes—which challenge reliable action recognition. The adversary cannot directly manipulate input labels but causes distortions that impede model reliability. The model must robustly identify critical police-civilian actions despite these complex conditions without relying on strong global context or large in-domain training data.

Methodology — deep read

The authors begin by defining a threat model centered on the challenges posed by real-world noise and occlusions in police BWCs, where an adversary is effectively the difficult visual conditions and the rapid, unpredictable actions making video understanding error-prone. The adversary’s capabilities include rapid camera motion and complex scene composition, while the models cannot rely on extensive labeled data or clear global contextual cues.

Data provenance includes 2,684 videos totaling approximately 185 hours from multiple U.S. police departments, notably Chicago PD’s COPA footage which includes high-stakes incidents like shootings and injuries, supplemented by Pasadena, Dallas, LA, San Antonio PDs, and news network sources. The videos are unedited or minimally edited for COPA and Pasadena but heavily edited for others. Videos are split into cases representing incidents, each with multiple officers recorded.

Annotation was a two-stage process: Stage 1 used multiple annotators to identify candidate interaction windows with a 10-second buffer, followed by Stage 2 where annotators labeled 9 police-civilian action classes precisely at second-level granularity using objective instructions to avoid inferring intent. Annotations accounted for roles (BWC wearer, other officers, civilians). The authors developed a custom annotation tool optimized for efficiency with looping 1-sec clips and single-action labeling to reduce fatigue. They hired 33 trained annotators for ~5,400 hours with ongoing monitoring and mental health precautions due to the sensitive content.

For modeling, the authors used pretrained video and vision-language models as frozen feature extractors, training lightweight linear probing classification heads on EgoPolice. They trained on the COPA dataset in 6-fold cross-validation with temporal and geographic OOD splits (2024 data and Pasadena PD). Videos from other sources were used as auxiliary training data. Models were evaluated on 1-second, 10-second, and 1-minute temporal windows, with predictions aggregated via max pooling.

Separately, they devised a zero-shot multiple-choice video question answering task with 12,000 questions querying 9 classes plus 'None'. Evaluations covered image-to-text and video-language models without fine-tuning, testing abilities for semantic and temporal reasoning.

Metrics reported include class-averaged F1 for classification and accuracy for MCQ. They conducted qualitative error analyses identifying domain-specific failure types. The dataset, annotation tool source, and benchmarking protocols are released openly for reproducibility. Full end-to-end evaluation showcases real-world feasibility but reveals substantial challenges.

One concrete example is classifying the 'BWC Weapon Out' action under severe camera motion and low-light conditions: VideoMAE V2 attains only ~40% per-second F1, highlighting difficulties due to occlusion and rapid camera movement. Zero-shot VLM Gemini 2.5 Pro reaches ~77% accuracy on longer clips but still misidentifies key events, underlining the necessity for specialized training and multimodal fusion.

Technical innovations

• Creation of a large-scale, real-world egocentric police BWC dataset annotated at second-level granularity on critical, high-stakes police-civilian interactions.
• Rigorous multi-stage annotation pipeline using objective action definitions and a custom efficient web tool designed to minimize annotator fatigue and bias in a distressing content domain.
• Systematic benchmarking of both supervised video classification and zero-shot video-language models highlighting the domain-specific challenges of high camera motion and minimal scene context.
• Quantitative analysis showing extreme visual noise and low inter-class visual separability in BWC footage that challenge reliance on global scene cues, motivating the need for fine-grained contextual understanding.

Datasets

• EgoPolice — 2,684 videos, 184h 57m total — public police body-worn camera footage from 6 US police departments and news sources (COPA Chicago, Pasadena, Dallas, LA, San Antonio, DC, miscellaneous news)
• COPA Chicago PD subset — 600 videos, 109h 15m — main evaluation and cross-validation source, minimally edited
• Pasadena PD subset — 76 videos, 10h 44m — held-out geographic OOD test set

Baselines vs proposed

• Random baseline classification F1: ~7.9% (1 sec), 20.2% (1 min) vs VideoMAE V2: 37.9% (1 sec), 58.4% (1 min) on ID splits
• CLIP classification F1: 39.2% (1 sec) vs VideoMAE V2 37.9% (1 sec), but VideoMAE better on motion-dependent classes
• Zero-shot VLM GPT-4.1-Nano accuracy: ~18.6% (1 sec) below random 20%; Gemini 2.5 Pro achieves 76.9% (1 min)
• Open-source VLMs like LLaVA-Mini and LLaMA-VID perform worse (~15-40%) than proprietary models, showing room for improvement

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.06468.

Fig 1: Sample frames of EgoPolice. We show representative frames for Easy (Top,

Fig 2 (page 2).

Fig 3 (page 2).

Fig 4 (page 2).

Fig 5 (page 2).

Fig 6 (page 2).

Fig 7 (page 2).

Fig 8 (page 2).

Limitations

• EgoPolice overrepresents high-stakes events, not reflecting the full distribution of police activity or everyday interactions.
• Annotation subjectivity remains difficult to eliminate fully despite objective definitions; some action boundaries are inherently ambiguous.
• Most auxiliary data (non-COPA/Pasadena) are heavily edited, limiting their use to training augmentation but not valid evaluation splits.
• Current benchmarks do not include adversarial attack or robustness testing under intentional perturbations.
• Zero-shot evaluations rely on existing VLMs not explicitly trained on BWC or policing contexts, which disadvantages them in understanding domain-specific visual cues.
• Some failure modes such as spurious correlations and temporal misalignment remain unsolved and require novel modeling approaches.

Open questions / follow-ons

• How can models better integrate temporal dynamics and fine-grained motion cues to detect brief, rapid police actions in highly noisy egocentric footage?
• What architectures or multi-modal approaches can reduce spurious correlations and improve attribution for complex multi-agent scenes in BWCs?
• How to incorporate uncertainty quantification or human-in-the-loop feedback to mitigate risk of misclassification in safety-critical policing applications?
• Can self-supervised or weakly supervised learning leverage large-scale uncurated BWC video repositories to close the performance gap?

Why it matters for bot defense

From a bot-defense and CAPTCHA perspective, EgoPolice highlights the challenges of reliable, high-stakes video understanding in the wild where motion blur, occlusions, and minimal discriminative context exist. Models must distinguish subtle, context-dependent actions under severe video noise—a useful analogue for designing CAPTCHA systems that resist adversarial automated video or motion analysis by bots. The dataset and insights can guide defenses relying on fine-grained temporal cues or anomalous human motion patterns. However, the difficulty in robustly parsing first-person video underscores the limitations of current vision-language models in complex dynamic settings, cautioning against overreliance on automated classification without human review. The embodied egocentric setting and need for second-level granularity suggest that layered security mechanisms combining multiple sensory inputs and temporal reasoning will be critical for next-generation bot defenses or verification tasks in video-rich environments.

Cite

@article{arxiv2607_06468,
  title={ EgoPolice: A Benchmark for Egocentric Video Understanding in High-Stakes Police Body-Worn Camera Footage },
  author={ Max Gonzalez Saez-Diez and Jihoon Chung and Adam D. Wolsky and Gregory Lanzalotto and Dean Knox and Jonathan Mummolo and Brandon M. Stewart and Olga Russakovsky },
  journal={arXiv preprint arXiv:2607.06468},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.06468}
}

Read the full paper

• arXiv:2607.06468 (abstract)
• arXiv:2607.06468 (PDF)