Scalable Variational Quantum Optimization via Pauli Correlation Encoding: Application to Large-Scale Power Demand Portfolio Optimization

Source: arXiv:2607.24722 · Published 2026-07-27 · By Takuya Yoshioka, Keita Sasada, Riku Usuki, Yuichiro Nakano, Keisuke Fujii

TL;DR

This paper addresses the critical scalability challenge in variational quantum algorithms (VQAs) for combinatorial optimization, specifically targeting large-scale quadratic unconstrained binary optimization (QUBO) problems that exceed the qubit resources of current quantum hardware. The authors introduce an encoding framework called Pauli Correlation Encoding (PCE), which represents classical binary variables as expectation values of k-body Pauli correlation operators on a quantum state. This approach enables a combinatorial increase in effective problem size relative to the number of qubits, via a continuous relaxation that encodes multi-qubit correlations compactly. The paper applies PCE to electric power demand portfolio optimization, a practically motivated, fully connected QUBO with thousands of variables and dense quadratic couplings. Through extensive numerical simulations for problem sizes from 18 to over 10,000 variables, the method achieves normalized cost gaps on the order of 10^-4 relative to certified optimal solutions. The optimization performance benefits from an interplay between continuous variable relaxation and discrete decoding, with larger systems providing more consistent improvement. Finally, real-device experiments on a trapped-ion quantum processor demonstrate robustness to noise and finite sampling, recovering high-quality near-optimal solutions. This establishes PCE as a physically motivated, qubit-efficient, and scalable quantum optimization framework bridging variational quantum circuit expressivity and classical combinatorial complexity.

Key findings

• Pauli Correlation Encoding (PCE) enables representing up to m = 3 * (n choose k) variables with n qubits and correlation order k, where k = n/2 maximizes m (Fig. 1).
• Numerical simulations achieve normalized cost gaps (∆CT/WT) on the order of 10^-4 compared to certified optima from the Gurobi solver, even at problem sizes m = 10,296.
• PCE optimization outperforms greedy baselines initialized from random or all-zero solutions across all tested sizes (Fig. 2), indicating effective binary initializations.
• The continuous relaxation employed, based on sigmoid-transformed Pauli correlators, provides a smooth optimization landscape but requires careful tuning of hyperparameters (αsc, β).
• Effective resolution and distribution of Pauli correlators govern performance: larger instances show more variables with correlators close to ±1, facilitating reliable binary decoding (Sec. IV B).
• QPU experiments on trapped-ion devices maintain superior solution quality over greedy random baselines up to m = 756; beyond this size, performance approaches a random baseline due to noise and finite sampling limits.
• The proposed two-stage hybrid approach—time-averaged portfolio optimization feeding into time-resolved optimization—improves convergence and solution quality.
• Greedy post-processing after decoding significantly improves solution cost relative to raw thresholded solutions, justifying the multi-stage procedure.

Threat model

The adversary is not explicitly modeled; the setting assumes limited, noisy near-term quantum hardware and classical optimizer constraints. No assumptions about adversarial attackers tampering with data or hardware are made; the focus is on inherent noise and statistical sampling fluctuations limiting solution quality on real quantum devices.

Methodology — deep read

1. Threat Model & Assumptions: The work assumes the adversarial challenges common in quantum optimization—limited qubit counts and noisy hardware—but does not explicitly consider malicious adversaries. The primary assumption is the constrained quantum resources and noise inherent in near-term devices.

2. Data: The power demand portfolio problem is constructed from an open dataset with 2,143 consumer-level hourly electricity consumption time series over 61 days. For m ≤ 2,143, random subsets of consumers are sampled; for m > 2,143, synthetic consumers are generated via mixup data augmentation (convex combinations of profiles) preserving statistical properties, enabling large-scale QUBO instances up to m = 10,296.

3. Architecture / Algorithm: The core quantum representation uses n qubits and k-body Pauli operators Π_i^(k) acting with Pauli X, Y, or Z on subsets of k qubits (k = n/2). The total encoded variables m = 3 * combinatorial(n, k). A layered parameterized quantum circuit of depth 5 prepares states |ψ(θ)⟩, comprising layers of Ry, Rz single-qubit rotations followed by fully connected Rzz two-qubit entangling gates, totaling O(l n^2) gates. The variational parameters θ control the prepared state.

Binary variables x_i ∈ {0,1} are relaxed to continuous variables y_i(θ) = sigmoid(2 α ⟨Π_i^(k)⟩_θ), where ⟨Π_i^(k)⟩_θ are the expectation values of the Pauli correlators on the state |ψ(θ)⟩ and α is a sharpness parameter. This continuous relaxation enables gradient-based optimization.

Loss functions correspond to quadratic portfolio costs (Model 1: time-averaged, Model 2: time-resolved), plus a regularization term weighted by β. The loss is minimized over θ using the classical BFGS optimizer with gradients computed exactly via the quantum simulator Qulacs.

Encoding and decoding use three measurement bases (all-X, all-Y, all-Z) to efficiently estimate all correlators despite the combinatorial scaling of encoded variables, keeping measurement settings fixed at three global bases for scalability.

4. Training Regime: Variational optimization uses noiseless state-vector simulation. Each Model 1 instance is run for five independent random initializations with 5-layer circuits, selecting the best minimum decoded binary cost solution. The optimized parameters initialize Model 2 (time-resolved) optimization without restarting. Hyperparameters αsc and β are grid-searched to optimize performance, with α scaling as αsc * n^{floor(k/2)}.

5. Evaluation Protocol: Performance is evaluated by normalized cost gaps ∆CT/WT relative to certified optima from Gurobi with a 10^-4 gap tolerance. Comparisons include greedy baselines with all-zero or random initializations plus local greedy post-processing. Hardware experiments on trapped-ion QPU verify robustness to sampling noise and errors. Key metrics include cost gap, binary decoding accuracy, correlator distributions, and overlap between simulation and QPU solutions. Results across problem sizes from m=18 to 10,296 quantify scalability.

6. Reproducibility: Code and datasets are not explicitly released; classical simulation uses Qulacs simulator. Hardware results are obtained on a trapped-ion system, without publicly available quantum device parameters or noise profiles. Detailed optimizer parameters and hyperparameter selection strategies are described, enabling partial replication.

Example Workflow: For a problem with m=756 variables encoded into n=14 qubits with k=7-body correlators, the circuit parameters θ initialized randomly are optimized using BFGS to minimize the loss function comprising quadratic portfolio cost plus regularization. The resulting continuous correlators ⟨Π_i^(k)⟩_θ are sigmoid-transformed to continuous variables, then thresholded by sign to obtain a binary portfolio selection x. After greedy post-processing, normalized cost gaps below 10^-4 are observed relative to Gurobi-certified optimal solutions. This optimized parameter set bootstraps the subsequent time-resolved Model 2 optimization, with final evaluation conducted on QPU hardware to test robustness under noise.

Technical innovations

• Application of Pauli Correlation Encoding (PCE) with k-body correlators (k=n/2) to achieve combinatorial scaling of encoded variables relative to qubit count, enabling optimization problems with over 10,000 variables on ~14 qubits.
• Two-stage hybrid optimization workflow combining time-averaged portfolio problem for PCE parameter initialization with subsequent time-resolved optimization for improved solution quality and stability.
• Efficient measurement strategy using only three global Pauli bases (all-X, all-Y, all-Z) regardless of encoded problem size to estimate exponentially many correlators, minimizing measurement overhead.
• Detailed characterization of the interplay between continuous relaxation via sigmoid-transformed Pauli correlators and discrete binary decoding, elucidating how effective resolution impacts optimization performance and scalability.

Datasets

• Energy Management System Open Data platform — 2,143 consumer electricity consumption time series over 61 days — publicly available
• Synthetic consumer profiles via mixup data augmentation — custom generated convex combinations of experimental data — non-public

Baselines vs proposed

• Greedy (all-zero): normalized cost gap ∆CT/WT ≈ 10^-2 (size-independent)
• Greedy (random, 1000 runs average): normalized cost gap improves with size, minimum ~10^-3 for large m
• PCE simulation: normalized cost gap ∆CT/WT ≈ 10^-4 consistently across 18 ≤ m ≤ 10,296
• PCE QPU experiments (up to m=756): normalized cost gap close to simulation, outperforming greedy random baseline; for m>756 QPU results approach greedy random baseline
• Gurobi commercial solver: reference certified optimal solution with gap ≤10^-4

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.24722.

Fig 8: Histograms of the sets {⟨Π(k)

Fig 9: Power procurement profiles for the time-resolved formulation (Model 2) with m = 60 consumers. Solutions obtained

Fig 5: For the largest instance (m = 10,296), a similar stacked

Fig 10: Same as Fig. 9, but for the instance with m = 756 consumers.

Fig 5 (page 17).

Fig 6 (page 17).

Limitations

• No adversarial robustness evaluation beyond noise and finite sampling on QPU; security against active attack vectors not studied.
• Quantum hardware experiments limited to moderate size (m=756), with performance degrading for largest instances due to noise and sampling constraints.
• No systematic investigation of circuit depth scaling beyond fixed depth l=5 or exploration of ansatz expressivity trade-offs.
• Lack of public code and data releases limits reproducibility and independent verification.
• Continuous relaxation may result in some binary variables decoded near threshold, leading to instability under finite shot noise in practical hardware.
• No ablation studies isolating individual components (e.g., effect of post-processing or regularization β) on larger problem instances presented in detail.

Open questions / follow-ons

• How does circuit depth and ansatz expressivity affect PCE performance and scalability beyond the fixed depth of five used here?
• Can PCE be extended to other combinatorial optimization problems beyond power demand portfolio optimization while retaining similar scalability and performance?
• What error mitigation or noise-resilient strategies can improve QPU performance for the largest problem sizes?
• How does PCE interact with alternative quantum optimization algorithms such as QAOA or quantum annealing in terms of representational efficiency and solution quality?

Why it matters for bot defense

For bot-defense and CAPTCHA practitioners interested in quantum-enabled combinatorial optimization, this work shows that variational quantum algorithms can be made scalable via Pauli Correlation Encoding by compressing many binary variables into qubit correlators, which may be relevant for optimization of test design or adaptive challenge policies at scales previously out of reach. The continuous relaxation and decoding interplay highlighted here underscores the importance of carefully designing discretization steps in quantum-classical optimization workflows, relevant when translating continuous quantum outputs to discrete defenses or challenge parameters.

While quantum hardware remains noisy and limited in scale, the demonstrated robustness and near-optimal performance on classical simulators for large problem sizes provides a promising direction for future quantum-assisted bot-defense optimization tasks. Researchers should note the critical role of measurement strategies minimizing overhead and the hybrid classical-quantum parameter training framework, which align with emerging hardware constraints. However, this approach still requires mature hardware and careful parameter tuning, limiting immediate production deployment but offering a valuable conceptual foundation for long-term quantum-enhanced CAPTCHA optimization.

Cite

@article{arxiv2607_24722,
  title={ Scalable Variational Quantum Optimization via Pauli Correlation Encoding: Application to Large-Scale Power Demand Portfolio Optimization },
  author={ Takuya Yoshioka and Keita Sasada and Riku Usuki and Yuichiro Nakano and Keisuke Fujii },
  journal={arXiv preprint arXiv:2607.24722},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.24722}
}

Read the full paper

• arXiv:2607.24722 (abstract)
• arXiv:2607.24722 (PDF)