Automatic Echocardiography Segmentation via Transition Probability Correlation for Stable Semantic Extraction

Source: arXiv:2607.07580 · Published 2026-07-08 · By Xinran Chen, Xiyuan Wang, Guangquan Zhou, Chuan Chen

TL;DR

This paper addresses the challenge of accurately segmenting echocardiographic videos, which is complicated by speckle noise, low signal-to-noise ratio, blurred anatomical boundaries, and cardiac motion. The authors propose a semi-supervised framework combining a novel Spatio-temporal Local Self-Similarity Fusion (STLSF) module and a Frequency-aware Denoising (FD) pre-training strategy. The STLSF module leverages local transition probability correlations across short video frame windows to correct semantic inconsistencies caused by noise and temporal discontinuities, while also guiding texture enhancement from stable semantic cues to improve boundary precision. The FD pre-training tailors encoder features to ultrasound-specific frequency domain characteristics by injecting controlled noise in the spectral domain and denoising, improving robustness to echocardiography artifacts. Evaluated on two public large-scale benchmarks, CAMUS and EchoNet-Dynamic, the approach achieves state-of-the-art Dice scores of 93.87% and 92.62% respectively, with substantial improvements in boundary accuracy (HD95 reductions of up to 1.16 mm compared to strongest baselines).

Key findings

• The proposed method achieves 93.87% Dice on CAMUS and 92.62% Dice on EchoNet-Dynamic, outperforming prior state-of-the-art models including XMem++ and EchoONE.
• STLSF’s Transition Semantic Guidance (TSG) module alone improves Dice by +0.26% and reduces HD95 by 0.37 mm relative to baseline without STLSF.
• Semantic-to-Texture Guidance (STG) module alone increases Dice by +0.24% and lowers HD95 by 0.33 mm compared to baseline.
• Combining both TSG and STG modules results in a +0.50% Dice improvement and 1.04 mm HD95 reduction, indicating synergistic effect.
• Frequency-aware Denoising (FD) pre-training surpasses alternatives like super-resolution, deblurring, and masked frequency modeling by 0.16-0.32% Dice and 0.6-0.7 mm HD95 on average.
• Model inference runs at 15.07 clips/s (150 frames/s) on a single NVIDIA RTX 4090 GPU with 78.62 GFLOPs and 38.5 MB memory footprint.
• Qualitative comparisons show smoother segmentation boundaries and more temporally consistent masks compared to strong baselines susceptible to speckle noise artifacts.

Threat model

Not a security paper. The 'threat' is effectively the inherent ultrasound speckle noise, low imaging quality, and temporal discontinuities that degrade semantic consistency in echocardiography video segmentation.

Methodology — deep read

The authors begin by defining the threat of noise and ambiguous semantics in echocardiography, focusing on the challenges speckle noise and rapid cardiac motion pose to deep segmentation models. Their adversary is essentially poor image quality and temporal discontinuity limiting feature fidelity.

Data comprises the public CAMUS dataset (with 7:1:2 train/val/test split) and EchoNet-Dynamic (official splits followed), two well-known echocardiography video benchmarks. Videos are sampled in clips of 10 frames, but only two frames (end diastolic and systolic) have segmentation annotations. Frames are resized to 224x224 (CAMUS) or 128x128 (EchoNet-Dynamic) and standard augmentations (rotation, flips, brightness) are applied.

The architecture uses a convolutional backbone inspired by the OverLoCK design: a three-stage encoder extracts hierarchical spatial features with a base-overview-focus paradigm. Basic blocks capture shallow local features, while dynamic blocks incorporate an overview feature to extract coarse deep semantics globally.

The core innovation is the STLSF module, which utilizes local spatio-temporal self-similarity across a small neighborhood window around each pixel to model transition probabilities reflecting local cardiac motion. This captures the temporal coherence of semantics spatially and across frames. The STLSF has two main parts: the Transition Semantic Guidance (TSG) first computes transition affinity between a query pixel and its neighbors using independent linear projections followed by softmax normalization to form a transition probability distribution. An attention mechanism then uses this distribution to rectify deep semantic features by aligning based on stable transition patterns. Next, the Semantic-to-Texture Guidance (STG) fuses these refined deep features with shallow texture layers via a semantic-guided cross-attention within a local dilated spatio-temporal neighborhood. This filters noise and enhances boundary details informed by the stable semantic cues.

Separately, the Frequency-aware Denoising (FD) pre-training corrupts input images by applying controlled ultrasound speckle noise in the frequency domain (via DCT), then trains the encoder to denoise and reconstruct frequency features. This enforces the encoder to learn ultrasound imaging priors, enhancing robustness to noise. The encoder backbone uses basic convolutional blocks followed by dynamic blocks guided by an overview semantic feature. Pre-training runs for 500 epochs with batch size 32 and learning rate 1e-3.

For downstream segmentation, the encoder weights are initialized from the FD pre-training. Training uses AdamW optimizer with learning rate 5e-3, weight decay 0.05, cosine annealing scheduler, and a joint loss combining Dice and BCE applied only on annotated frames. Semi-supervised temporal constraints are enforced on unlabeled frames to maintain spatiotemporal consistency. Fine-tuning runs for 100 epochs.

Evaluation measures Dice coefficient for overlap plus HD95 and ASSD for boundary accuracy. Results are benchmarked against 8 SOTA baselines including XMem++, MemSAM, H2Former and EchoONE, reporting statistically significant gains in Dice and boundary metrics.

They also conduct ablation experiments isolating STLSF components and various pre-training strategies to validate the contribution of each design choice. All experiments run on a single NVIDIA RTX 4090 GPU. The combination of local transition probabilities, semantic-guided texture refinement, and frequency-domain denoising pre-training is shown end-to-end effective in improving segmentation quality and temporal coherence under noisy echocardiographic video conditions.

Code availability or frozen weights are not explicitly mentioned, indicating partial reproducibility pending authors' release.

Technical innovations

• Introduction of Spatio-temporal Local Self-Similarity Fusion (STLSF) module that leverages local pixel-level transition probability correlations across frames to correct semantic inconsistencies.
• Dual-path STLSF design with Transition Semantic Guidance for semantic rectification and Semantic-to-Texture Guidance for noise-adaptive texture enhancement.
• Frequency-aware Denoising (FD) pre-training strategy injecting realistic ultrasound speckle noise in the frequency domain to teach noise-robust anatomical feature extraction.
• Base-Overview-Focus convolutional encoder architecture combining local detail extraction and global semantic overview to enable effective spatiotemporal feature learning.

Datasets

• CAMUS — ~450 patients (245 training) — Public cardiac ultrasound dataset
• EchoNet-Dynamic — 10,030 echocardiogram videos — Public clinical ultrasound dataset

Baselines vs proposed

• XMem++: CAMUS Dice = 93.45%, Proposed = 93.87%; EchoNet-Dynamic Dice = 87.72%, Proposed = 92.62%
• VideoMamba: CAMUS Dice = 91.26%, Proposed = 93.87%; EchoNet-Dynamic Dice = 90.01%, Proposed = 92.62%
• H2Former: CAMUS Dice = 92.34%, Proposed = 93.87%; EchoNet-Dynamic Dice = 91.89%, Proposed = 92.62%
• EchoONE: CAMUS Dice = 93.07%, Proposed = 93.87%; EchoNet-Dynamic Dice = 92.12%, Proposed = 92.62%
• PKEcho-Net: CAMUS Dice = 93.23%, Proposed = 93.87%; EchoNet-Dynamic Dice = 91.89%, Proposed = 92.62%
• MemSAM: CAMUS Dice = 92.88%, Proposed = 93.87%; EchoNet-Dynamic Dice = 92.26%, Proposed = 92.62%
• SimLVSeg: CAMUS Dice = 92.88%, Proposed = 93.87%; EchoNet-Dynamic Dice = 92.11%, Proposed = 92.62%
• NCMNet: CAMUS Dice = 93.56%, Proposed = 93.87%; EchoNet-Dynamic Dice = 92.30%, Proposed = 92.62%

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.07580.

Fig 1: Overall architecture of the FD pre-training framework and CISR-Net.

Fig 2: Visual comparison with representative baselines on CAMUS and EchoNet-

Limitations

• The approach relies on semi-supervised training with only two annotated frames per clip, limiting annotation density and possibly temporal accuracy.
• Evaluation focuses on public datasets only; robustness under real-world imaging variations or different ultrasound machines is not assessed.
• No adversarial or targeted robustness testing against motion artifacts or extreme noise scenarios reported.
• Reproducibility may be constrained due to lack of publicly available code or trained weights at present.
• The local window size hyperparameters and their sensitivity to varying cardiac motion dynamics are not deeply explored.
• The method trades some architectural complexity for inference speed, but generalization to clinical deployment scenarios with diverse data remains to be validated.

Open questions / follow-ons

• How does the STLSF module generalize to other ultrasound modalities or non-cardiac anatomical segmentation tasks?
• Can the frequency-aware denoising pre-training be extended or adapted to other noise types or imaging modalities beyond ultrasound?
• What is the impact of increasing annotation density on temporal frames for semi-supervised training—would performance improve further?
• How sensitive is the model to varying window sizes or neighborhood definitions in the transition probability computations?

Why it matters for bot defense

For bot-defense and CAPTCHA practitioners focusing on image/video semantic stability and noise robustness, this work illustrates a novel use of local self-similarity and transition probability modeling to improve semantic extraction in noisy, low SNR conditions. Analogously, difficult image challenges involving temporal consistency under noise could benefit from transition probability-inspired feature rectification or frequency-domain denoising strategies to enhance stable semantic recognition. Although aimed at medical imaging, the underlying principles of combining local spatiotemporal affinities to suppress noise-induced semantic errors and distill consistent semantic-texture representations could inform the design of CAPTCHA systems resistant to adversarial degradation or automated spoofing under challenging visual conditions.

Cite

@article{arxiv2607_07580,
  title={ Automatic Echocardiography Segmentation via Transition Probability Correlation for Stable Semantic Extraction },
  author={ Xinran Chen and Xiyuan Wang and Guangquan Zhou and Chuan Chen },
  journal={arXiv preprint arXiv:2607.07580},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.07580}
}

Read the full paper

• arXiv:2607.07580 (abstract)
• arXiv:2607.07580 (PDF)