Motion-Conditioned Multi-View Fusion for Myocardial Infarction Localization from Echocardiography

Source: arXiv:2607.15268 · Published 2026-07-16 · By Guang Yang, Wentian Xu, Siyu Wang, Betty Raman, Lei Li, Vicente Grau

TL;DR

This paper addresses the challenge of segment-level myocardial infarction (MI) localization from echocardiography (Echo) videos, which is crucial for clinical assessment but complicated by view-dependent ambiguity and limited motion annotations. Prior methods either relied on dense supervision for motion estimation or operated on single views, limiting spatial precision. The authors propose MCF-Net, a motion-conditioned multi-view fusion framework that leverages a pretrained echocardiography foundation model (EchoPrime) combined with extremely sparse cardiac motion cues derived from a single annotated template frame transferred across videos. Motion-derived soft masks serve as spatial priors to enhance feature extraction, while a novel fusion mechanism integrates motion and visual features across apical two- and four-chamber views to improve localization.

MCF-Net demonstrates superior segment-level MI localization performance on the public HMC-QU dataset, achieving 72.4% F1 score and 84.9% accuracy, outperforming strong baselines including motion-only, vision-only, and prior fusion methods by up to +8.1% F1. Ablation studies confirm the complementary benefits of motion-guided soft refinement and motion-conditioned multi-view fusion components. The method’s sparse annotation requirement and strong empirical gains suggest practical potential for scalable, robust MI localization in routine echocardiography without cumbersome labeling.

Key findings

• MCF-Net achieves 72.4% ± 3.8 F1 score and 84.9% ± 2.3 accuracy on segment-level MI localization, outperforming motion-only, vision-only, and prior fusion baselines.
• Compared to EchoPrime foundation model alone (64.3% F1), MCF-Net improves F1 by +8.1% and accuracy by +4.8%.
• Sparse motion modeling requires only one annotated template frame per dataset, transferred with a patch matching and CoTracker3 point tracking to extract dense motion trajectories across videos.
• Motion Guided Soft Refinement (MSR) improves the baseline EchoPrime F1 from 64.3% to 66.8%, while Motion Conditioned Fusion (MCF) boosts F1 to 70.3%; the full model combining MSR+MCF reaches 72.4%.
• Moderate motion mask strength λ=0.3 yields peak performance; excessive masking (λ≥0.5) degrades localization.
• Multi-head cross-view attention fuses motion-conditioned visual embeddings from A2C and A4C views, improving anatomical coherence and reducing false positives compared to single-view or naïve fusion methods.
• Paired patient-level bootstrap confirms F1 improvement over EchoPrime with statistical significance (Delta=0.08, 95% CI=[0.01,0.15], p<0.05).
• Qualitative results show MCF-Net produces compact and anatomically plausible infarct segment maps, avoiding overestimation or fragmented patterns typical of vision-only or motion-only approaches.

Threat model

N/A – the paper focuses on medical image analysis for myocardial infarction localization rather than adversarial threat or attack scenarios.

Methodology — deep read

The authors propose MCF-Net, a framework for segment-level MI localization from paired apical two-chamber (A2C) and four-chamber (A4C) echocardiographic video views. The task is to infer binary infarct presence for each of the 12 myocardial segments visible across the two views, using the AHA 17-segment model.

Threat Model & Assumptions: The adversary setting is implicit—accurate and reproducible localization under real-world Echo noise, artifacts, and view ambiguity is the key challenge. No dense frame-level or cross-view anatomical correspondences are assumed, and only one annotated template frame is required for the entire dataset to sparsely supervise motion.

Data: The public HMC-QU dataset comprises 162 A4C and 160 A2C videos from 160 patients with paired views. Data is split into 112 training, 16 validation, and 32 test patients. Videos are resized to 224x224 and uniformly sub-sampled to 16 frames for processing. The dataset features segment-level MI annotations on six segments per view.

Architecture & Algorithm: The pipeline consists of three key modules:

1. Sparse Motion Modeling: Starting from one reference A4C video annotated with 32 myocardial wall landmarks (covering six segments), localized template matching (NCC within 41x41 windows) transfers landmark positions to each target video’s first frame. Landmarks with low confidence fallback to prior positions to ensure robust structural mapping. Then, CoTracker3 models dense temporal trajectories for each landmark across T frames independently in A2C and A4C views.

2. Motion Guided Soft Refinement (MSR): Tracked landmarks induce segment-aware Gaussian mixture masks forming soft spatial attention over myocardial regions across time. The original video frames are modulated element-wise (1 + λ * mask) to enhance edges and intensities near the myocardial wall without distorting surrounding contexts.

3. Motion Conditioned Fusion (MCF): The refined video is encoded by a frozen EchoPrime foundation model producing 512D visual embeddings per view. Landmark trajectories are summarized by an MLP into 128D motion embeddings. Each visual embedding is modulated via FiLM conditioning with motion latent vectors (element-wise scale and shift). Conditioned embeddings from both views are concatenated, and multi-head attention with the concatenated motion embeddings queries aggregates cross-view complementary features. The fused representation passes through a linear classifier head to predict 12 segment infarct probabilities.

Training: Models are trained for 50 epochs with batch size 16, using AdamW optimizer with 1e-3 learning rate and 1e-4 weight decay on an NVIDIA RTX A4000 GPU. EchoPrime backbone remains frozen during training to preserve pretrained spatial priors.

Evaluation: Metrics include AUROC, precision-recall AUC (PR-AUC), F1, and accuracy, averaged over three random seeds with patient-level splits. Models compared include motion-only baselines (active polynomials APs, APsML), vision-only backbones (InceptionV3, R(2+1)D, EchoPrime), and prior fusion methods (EchoAna, EchoPrime non-fusion). Ablation studies test MSR and MCF modules individually and combined.

Reproducibility: The HMC-QU dataset is publicly available, and the paper states implementation in PyTorch, but code release or frozen weights are not explicitly mentioned. The use of a publicly published foundation model EchoPrime supports reproducibility.

Technical innovations

• A sparse cardiac motion modeling approach requiring only a single annotated template frame, transferred across videos with patch matching and CoTracker3 tracking.
• Motion Guided Soft Refinement module creating segment-aware, Gaussian mixture soft masks from sparse trajectories to modulate echocardiographic input frames before feature encoding.
• Motion Conditioned Fusion strategy applying FiLM-based motion gating to visual embeddings per view combined with multi-head cross-view attention for complementary evidence integration.
• Use of pretrained echocardiography foundation model (EchoPrime) frozen encoder combined with learned motion embeddings for fine-grained localized infarct detection.

Datasets

• HMC-QU — 160 paired A2C/A4C echocardiogram videos from 160 patients — public

Baselines vs proposed

• APs (motion-only): F1 = 57.7% ± 5.8, Accuracy = 79.0% ± 2.6 vs MCF-Net F1 = 72.4% ± 3.8, Accuracy = 84.9% ± 2.3
• APsML (motion-only): AUROC = 83.0% ± 2.1, PR-AUC = 70.4% ± 7.2, F1 = 59.9% ± 6.0 vs MCF-Net AUROC = 87.6% ± 2.4, PR-AUC = 74.7% ± 5.6
• InceptionV3 (vision-only): F1 = 27.0% ± 2.8, Accuracy = 61.5% ± 1.1 vs MCF-Net F1 = 72.4% ± 3.8, Accuracy = 84.9% ± 2.3
• R(2+1)D (vision-only): F1 = 45.6% ± 9.8, Accuracy = 73.3% ± 5.4 vs MCF-Net F1 = 72.4% ± 3.8, Accuracy = 84.9% ± 2.3
• EchoPrime (vision-only FM): AUROC = 84.9% ± 2.6, F1 = 64.3% ± 3.8, Accuracy = 80.1% ± 2.1 vs MCF-Net F1 = 72.4% ± 3.8, Accuracy = 84.9% ± 2.3
• EchoPrimeNF (naïve fusion): F1 = 59.3% ± 5.6, Accuracy = 78.8% ± 1.7 vs MCF-Net F1 = 72.4% ± 3.8, Accuracy = 84.9% ± 2.3
• EchoAna (fusion): F1 = 61.7% ± 4.6, Accuracy = 80.9% ± 2.8 vs MCF-Net F1 = 72.4% ± 3.8, Accuracy = 84.9% ± 2.3

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.15268.

Fig 1: Overview of MCF-Net. (1) Sparse Motion Modeling extracts motion em-

Fig 2 (page 3).

Fig 3 (page 3).

Fig 4 (page 3).

Fig 5 (page 3).

Fig 6 (page 3).

Fig 7 (page 3).

Fig 8 (page 3).

Limitations

• The method relies on good landmark initialization via template matching, which may fail in very poor quality echoes despite fallback mechanism.
• Cross-view anatomical correspondence is not explicitly modeled, potentially missing tighter inter-view spatial constraints.
• Evaluation is limited to a single public dataset (HMC-QU) without multi-center or prospective validation.
• No adversarial testing or robustness analysis under perturbations is reported.
• The frozen foundation model encoder prevents end-to-end fine-tuning that might further improve localization performance.
• Quantitative segment-level evaluation uses binary classification rather than precise pixel-wise segmentation, limiting granularity.

Open questions / follow-ons

• How would incorporating explicit cross-view anatomical correspondence or registration improve localization accuracy?
• Can the sparse motion modeling be extended or automated further to require no manual template annotation?
• How robust is MCF-Net under different clinical acquisition settings, echo machines, or varying image quality?
• Would end-to-end fine-tuning of the foundation encoder with motion conditioning yield additional gains?

Why it matters for bot defense

The proposed approach exemplifies how sparse motion priors can be effectively integrated with pretrained foundation models via conditioning and multi-view fusion to improve fine-grained localization in challenging, ambiguity-prone imaging modalities. For bot-defense or CAPTCHA practitioners, this work illustrates the value of combining domain-specific sparse dynamic cues with large pretrained visual backbones to resolve ambiguous spatial regions and improve robustness without requiring dense manual labeling. The motion guided soft refinement approach can inspire selective attention or feature enhancement strategies that incorporate external priors in multi-view video or sequential data settings. Moreover, the motion conditioned fusion using FiLM and cross-attention demonstrates an effective mechanism to gate and fuse complementary modality signals, a technique potentially generalizable to integrating diverse automated bot evidence streams or sensor inputs for robust, fine-grained bot detection.

Cite

@article{arxiv2607_15268,
  title={ Motion-Conditioned Multi-View Fusion for Myocardial Infarction Localization from Echocardiography },
  author={ Guang Yang and Wentian Xu and Siyu Wang and Betty Raman and Lei Li and Vicente Grau },
  journal={arXiv preprint arXiv:2607.15268},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.15268}
}

Read the full paper

• arXiv:2607.15268 (abstract)
• arXiv:2607.15268 (PDF)