Benchmarks & Metrics
Cross-Entropy Benchmarking
A method for estimating how closely a quantum computer's output matches the ideal result of a random circuit, used in quantum supremacy experiments.
Definition
Cross-entropy benchmarking, or XEB, checks whether a quantum computer is really running a circuit correctly when the circuit is too large to verify directly. You run a random circuit many times, record the bit strings that come out, and compute how likely those strings are according to the ideal circuit. If the machine works, it produces strings that the ideal circuit says are likely much more often than chance. The score between 0 and 1 measures how faithful the hardware is.
In depth
XEB was proposed by Boixo and colleagues at Google in 2018 as a way to demonstrate quantum supremacy and certify that a device works as claimed. For random circuits, the ideal output probabilities follow a Porter-Thomas distribution, with a few very likely strings and many unlikely ones. A noisy device produces a mixture of the ideal distribution and uniform noise, and the cross-entropy score measures the weight of the ideal part. In Google's 2019 Sycamore experiment, the total XEB fidelity was about 0.2 percent on 53 qubits, small but statistically unambiguous with millions of samples.
The method relies on being able to compute ideal probabilities. For the largest circuits, that is infeasible, so experiments estimate fidelity by extrapolation from smaller circuits and by per-gate error models, and the claim for the full circuit rests on the assumption that errors multiply simply. This is a known weakness: classical spoofing algorithms, including tensor network methods and approaches that exploit noise, have tried to reproduce high XEB scores with less than ideal simulation, and results have been debated.
Beyond supremacy experiments, XEB is used as a calibration tool. Google applies it to individual gates and layers to measure speckle-based errors including coherent ones, and uses it to optimize gate parameters. Because it works on random circuits with entangling gates across the whole chip, it captures crosstalk and correlated errors that single-qubit benchmarking misses.
Technical Definition
The linear XEB fidelity is F = 2^n times the average over measured bit strings x_i of the ideal probability P(x_i), minus 1, where n is the number of qubits. For a perfect device sampling from a Porter-Thomas distribution, F is about 1, and for a uniformly random output F is 0. The estimate requires computing ideal probabilities P(x_i) classically, which is feasible for smaller circuits.
Visual Explanation: An Analogy
Imagine checking a student's answers to a huge exam you cannot fully grade. You grade a sample of questions you can check, and see whether their answers match the right ones far more than random guessing would. The score on the sample estimates the score on the whole exam.
Key facts
| Item | Detail |
|---|---|
| Formula | F equals 2^n times the average ideal probability of observed strings, minus 1 |
| Range | 0 for uniform noise to about 1 for a perfect device |
| Sycamore (2019) | Total XEB fidelity about 0.2 percent for 53 qubits and 20 cycles |
| Requirement | Classical computation of ideal probabilities, feasible only for smaller circuits |
| Proposed | Boixo and colleagues, 2018 |
Where the idea comes from
XEB came from Google's quantum supremacy program. It was introduced in 2018, used in the 2019 Sycamore result, extended in the 2021 and 2022 experiments by USTC and Google, and used again for the 2024 Willow random circuit sampling benchmark.
Real-World Use Cases
- Verifying the output of random circuit sampling experiments.
- Calibrating gates and layers on full-chip circuits.
- Detecting crosstalk and correlated errors.
- Comparing devices on a common random-circuit benchmark.
How it differs from related ideas
- Randomized benchmarking measures average gate error, and XEB measures whole-circuit fidelity on random circuits.
- Quantum volume tests heavy output generation, which is related to XEB but defined with a threshold.
- XEB can be spoofed by classical algorithms that exploit noise, so a high score alone is not a proof of advantage.
Common Misconceptions
- A low XEB number such as 0.002 does not mean the experiment failed. It can still be statistically significant with enough samples.
- XEB does not verify every aspect of the computation. It tests faithfulness on random circuits.
- It does not apply easily to circuits with structured outputs.
Frequently asked questions
What is cross-entropy benchmarking?
A way to estimate the fidelity of a quantum computer on random circuits by checking how likely its outputs are under the ideal circuit.
Why is it used for supremacy experiments?
Because those circuits cannot be fully verified classically, so fidelity is estimated from samples and simulations of smaller circuits.
What are its limits?
It needs classical simulation for the ideal probabilities and can be approached by classical spoofing methods.
Related Terms
Keep exploring
- Algorithm: Random Circuit Sampling (Quantum Supremacy Experiments)
- Hardware: Google Sycamore
- Hardware: Google Willow