A Flexible Sparsity-Aware FPGA Accelerator with Column-Wise Compression for Efficient CNN Inference

Source: arXiv:2607.19248 · Published 2026-07-21 · By Amirhossein Zarei, Shervin Vakili

TL;DR

This paper addresses the challenge of accelerating convolutional neural networks (CNNs) on resource-constrained FPGA platforms in the presence of irregular sparsity patterns in model weights. Unstructured sparsity achieves higher model accuracy but is difficult to efficiently map on hardware due to irregular memory and computation patterns, while structured sparsity simplifies hardware design at the cost of model flexibility and potential accuracy loss. The authors present SparHiXcel-v2, a flexible and configurable FPGA accelerator architecture that balances sparsity support flexibility with hardware efficiency. The design features a scalable 2D MAC array and introduces a novel column-wise kernel compression method that shifts and eliminates zero-value columns to minimize hardware underutilization. To complement the hardware, a hardware-algorithm co-design is proposed including filter ordering optimization and an iterative structured pruning and revival technique tailored to the microarchitecture. Extensive evaluation on VGG16 and ResNet18 shows SparHiXcel-v2 achieves up to 2.5 TOPS and 210 GOPS/W on VGG16 and 1.1 TOPS and 72 GOPS/W on ResNet18 using an AMD Kintex UltraScale+ FPGA, with only modest accuracy degradation. The design effectively exploits irregular and structured sparsity patterns, enabling substantial improvements in throughput, PE utilization, and energy efficiency compared to baselines and prior accelerators.

Key findings

• SparHiXcel-v2 achieves 9.9% and 8.8% average speedup at 50% sparsity for ResNet18 and VGG16 respectively on a 15x15 PE array; speedups improve to 31.6% and 30.6% at 70% sparsity.
• Effective PE utilization efficiency can exceed 100%, indicating completion of dense-equivalent operations in fewer cycles due to sparsity exploitation (Fig. 6).
• Structured sparsity mode achieves over 2.5 TOPS and 210 GOP/s/W for VGG16 and over 1.1 TOPS and 72 GOP/s/W for ResNet18 on AMD Kintex UltraScale+ FPGA.
• The novel column-wise kernel compression eliminates all-zero columns after shifting nonzero weights leftwards within kernel rows, reducing the number of PE columns required without losing information (Fig. 1).
• The PE array assignment flexibly maps kernels with variable column sizes from dense n×n down to n×1 PEs depending on sparsity, improving utilization and reducing hardware costs.
• The vertical partial sum reduction network with V-Nodes enables efficient partial result accumulation across channels without centralized bottlenecks (Fig. 2b, Fig. 3a).
• The iterative hardware-aware structured pruning and revival algorithm further boosts throughput and compression while maintaining accuracy.
• The microprogrammed control signals generated by the scheduler statically govern all PE assignments, multiplexing, and weight column indexing, simplifying runtime control.

Threat model

The adversary corresponds to the intrinsic challenge of irregular and input-dependent sparsity patterns causing inefficiencies and overheads in FPGA CNN inference accelerators. The threat is inefficient hardware utilization, load imbalance, and increased energy consumption resulting from poor mapping of sparse models. The system assumes the sparsity patterns are trustworthy and does not consider malicious attacks or adversarial input beyond this operational context.

Methodology — deep read

The paper's threat model considers accelerating CNN inference with sparse kernels on FPGA hardware, where the adversary is the complexity of irregular sparsity inducing hardware inefficiencies and load imbalance. There is no explicit adversarial attack model, focusing instead on design trade-offs between sparsity flexibility and hardware overhead.

Data: The authors use pretrained CNN models from Torchvision – specifically VGG16, ResNet18, and EfficientNetV2-S trained on ImageNet. Random pruning is applied at 50% and 70% levels to generate sparse models.

Architecture: SparHiXcel-v2 is built around a scalable 2D PE array (multiply-accumulate units) adopting a weight-stationary dataflow. The key technical innovation is a column-wise kernel compression that shifts nonzero weights leftward within rows to remove zero-valued columns, preserving kernel rectangularity for simpler hardware mapping. This compression information is stored as control signals encoding original column indices.

Each PE contains a multiplier, adders to accumulate partial sums vertically and horizontally within PE groups, and control logic to support flexible delay and synchronization to handle column shifts. Partial sums flow vertically then horizontally within PE groups, then across filter channels through a configurable network of vertical nodes (V-Nodes), implementing distributed accumulation without central bottlenecks.

A microprogrammed control scheme is used: a Python-based scheduler tool statically optimizes PE assignments, filter ordering, V-Line mappings, and control multiplexer sizing. The scheduler pre-generates all microinstructions and control signals, which are loaded with weights into the PEs at runtime.

Training regime: The paper develops a multi-phase structured pruning and revival algorithm tailored to the architecture to produce hardware-friendly constrained sparsity patterns with minimal accuracy loss. The pruning operates iteratively with fine-tuning. Details on epochs or hyperparameters for pruning are not explicitly provided.

Evaluation: They measure speedup and effective PE utilization (the ratio of dense equivalent MAC ops over cycles times PE count) for each convolutional layer of benchmark models. Performance is modeled cycle-accurately using the scheduler tool, including pipeline timing and synchronization. Experiments compare dense, 50%, and 70% sparse models on PE arrays of various sizes (e.g. 15×15, 7×15). Additionally, FPGA implementation results on AMD Kintex UltraScale+ report throughput (TOPS) and energy efficiency (GOP/s/W).

Reproducibility: Source code including scheduler and accelerator RTL is released on GitHub. The datasets are standard ImageNet-pretrained models. FPGA bitstreams or full hardware designs are not explicitly stated as released.

Concrete example: For a 3×3 kernel with partial sparsity, compression shifts nonzero weights into leftmost columns, reducing kernel width and thus PE columns assigned. In the PE array, PEs in a group corresponding to kernel columns receive inputs delayed and synchronized to align partial sums properly for accumulation. Partial results propagate vertically and horizontally within each PE group, then vertically across groups using V-Nodes connected in vertical lines, enabling flexible and efficient reductions despite irregular sparsity patterns.

Technical innovations

• Column-wise compression method that shifts and removes zero-valued kernel columns while preserving rectangular kernel shape and all nonzero weights.
• Configurable 2D PE array microarchitecture with flexible horizontal PE allocation dynamically adjusted to compressed kernel width.
• Distributed vertical partial sum reduction network (V-Nodes and V-Lines) that enable scalable accumulation of multi-channel partial results without centralized bottlenecks.
• Hardware-algorithm co-design involving a microprogrammed scheduler that statically generates control signals including weight column indexing and filter scheduling.
• Multi-phase structured pruning and revival algorithm tailored specifically for SparHiXcel-v2 hardware constraints.

Datasets

• ImageNet-pretrained VGG16 — standard Torchvision models used for evaluation
• ImageNet-pretrained ResNet18 — standard Torchvision models used for evaluation
• ImageNet-pretrained EfficientNetV2-S — standard Torchvision models used for evaluation

Baselines vs proposed

• Dense baseline (no sparsity): Effective PE efficiency approx. 100% by definition
• At 50% sparsity - ResNet18: speedup = 9.9% vs dense baseline; at 70% sparsity speedup = 31.6%
• At 50% sparsity - VGG16: speedup = 8.8%; at 70% sparsity speedup = 30.6%
• Structured sparsity mode on SparHiXcel-v2: VGG16 throughput > 2.5 TOPS and 210 GOP/s/W vs unstructured baseline lower (exact numbers not given)
• Structured sparsity mode on SparHiXcel-v2: ResNet18 throughput > 1.1 TOPS and 72 GOP/s/W

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.19248.

Fig 1: The proposed compression method applied to three example kernels

Fig 2: (a) Three example sparse filters (F1 to F3) with two channels (Ch1 and

Fig 3: (a) V-Node internal circuit, (b) FSUM-Store circuit.

Fig 4: Internal architecture of PEs.

Fig 5: An example illustrating PE assignment schedule in multiple rounds

Fig 6: Performance results of SparHiXcel (15 × 15 PE array) at three sparsity levels, showing effective PE efficiency for (a) ResNet18, (b) VGG16, and (c)

Fig 7: Performance results using 15 × 15 and 7 × 15 PE arrays.

Fig 8: presents the performance results of these configu-

Limitations

• Performance evaluation is based mostly on cycle-accurate modeling and FPGA implementations but lacks evaluation on ASIC or more widely deployed platforms.
• The pruning algorithm details such as training epochs, exact hyperparameters, and accuracy vs sparsity trade-offs are not fully elaborated.
• No explicit adversarial or robustness evaluation for corrupted input or malicious attacks against sparsity encoding/control mechanisms.
• The scheduler and control strategy rely on static pre-generated signals; dynamic input-dependent sparsity or runtime adaptability is not addressed.
• The evaluation focuses on a limited set of CNN architectures (VGG16, ResNet18, EfficientNetV2-S) without demonstrating generalization across more complex or transformer-like models.
• There is no end-to-end inference latency or system-level evaluation including memory subsystems or batch processing effects.

Open questions / follow-ons

• How does SparHiXcel-v2 performance and efficiency scale with more complex modern CNNs or transformer-based vision models exhibiting different sparsity structures?
• Can the compression and mapping techniques be extended to dynamically adapt to input-dependent sparsity patterns at runtime rather than static scheduling?
• What are the trade-offs of implementing similar sparsity-aware accelerators on ASIC platforms versus FPGA in terms of area, power, and flexibility?
• How does the structured pruning and revival algorithm impact end-to-end model accuracy across diverse datasets and tasks, especially beyond ImageNet classification?

Why it matters for bot defense

For bot-defense and CAPTCHA practitioners, SparHiXcel-v2's approach offers a detailed study in designing hardware accelerators capable of efficiently handling sparse CNN models common in vision tasks. The paper highlights key challenges in balancing sparsity flexibility and hardware cost, which are relevant when deploying CNNs for image-based bot detection or CAPTCHA analysis in low-power or embedded environments. The column-wise compression and hardware-aware pruning could inform design decisions to optimize inference latency and energy consumption for on-device bot detection models. However, the reliance on static scheduling and pruning tailored to specific hardware implies some limits in adaptability to rapidly evolving bot traffic patterns or model updates. Understanding this hardware-algorithm co-design offers insights into how sparsity can be exploited in constrained environments without sacrificing model effectiveness, a tradeoff important in security-focused edge deployments.

Cite

@article{arxiv2607_19248,
  title={ A Flexible Sparsity-Aware FPGA Accelerator with Column-Wise Compression for Efficient CNN Inference },
  author={ Amirhossein Zarei and Shervin Vakili },
  journal={arXiv preprint arXiv:2607.19248},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.19248}
}

Read the full paper

• arXiv:2607.19248 (abstract)
• arXiv:2607.19248 (PDF)