Variational Shadow Quantum Learning¶

Copyright (c) 2021 Institute for Quantum Computing, Baidu Inc. All Rights Reserved.

Overview¶

In this tutorial, we will discuss the workflow of Variational Shadow Quantum Learning (VSQL) [1] and accomplish a binary classification task using VSQL. VSQL is a hybird quantum-classical framework for supervised quantum learning, which utilizes parameterized quantum circuits and classical shadows. Unlike commonly used variational quantum algorithms, the VSQL method extracts "local" features from the subspace instead of the whole Hilbert space.

Background¶

We consider a $k$-label classification problem. The input is a set containing $N$ labeled data points $D={\left\{\right(x^i,y^i\left)\right\}}_{i=1}^N$, where $x^i\in R^m$ is the data point and $y^i$ is a one-hot vector with length $k$ indicating which category the corresponding data point belongs to. Representing the labels as one-hot vectors is a common choice within the machine learning community. For example, for $k=3$, $y^a=[1,0,0{]}^{\text{T}},y^b=[0,1,0{]}^{\text{T}}$, and $y^c=[0,0,1{]}^{\text{T}}$ indicate that the $a^{\text{th}}$, the $b^{\text{th}}$, and the $c^{\text{th}}$ data points belong to class 0, class 1, and class 2, respectively. The learning process aims to train a model $F$ to predict the label of every data point as accurately as possible.

The realization of $F$ in VSQL is a combination of parameterized local quantum circuits, known as shadow circuits, and a classical fully-connected neural network (FCNN). VSQL requires preprocessing to encode classical information into quantum states. After encoding the data, we convolutionally apply a parameterized local quantum circuit $U\left(\theta \right)$ to qubits in each encoded quantum state, where $\theta$ is the vector of parameters in the circuit. Then, expectation values are obtained via measuring local observables on these qubits. After the measurement, there is an additional classical FCNN for postprocessing.

We can write the output of $F$, which is obtained from VSQL, as ${\tilde{y}}^i=F\left(x^i\right)$. Here ${\tilde{y}}^i$ is a probability distribution, where ${\tilde{y}}_j^i$ is the probability of the $i^{\text{th}}$ data point belonging to the $j^{\text{th}}$ class. In order to predict the actual label, we calculate the cumulative distance between ${\tilde{y}}^i$ and $y^i$ as the loss function $L$ to be optimized:

where $W$ and $b$ are the weights and the bias of the one layer FCNN. Note that this loss function is derived from cross-entropy [2].

Pipeline¶

Here we give the whole pipeline to implement VSQL.

1. Encode a classical data point $x^i$ into a quantum state $|x^i⟩$.
2. Prepare a parameterized local quantum circuit $U\left(\theta \right)$ and initialize its parameters $\theta$.
3. Apply $U\left(\theta \right)$ on the first few qubits. Then, obtain a shadow feature via measuring a local observable, for instance, $X\otimes X\cdots \otimes X$, on these qubits.
4. Sliding down $U\left(\theta \right)$ one qubit each time, repeat step 3 until the last qubit has been covered.
5. Feed all shadow features obtained from steps 3-4 to an FCNN and get the predicted label ${\tilde{y}}^i$ through an activation function. For multi-label classification problems, we use the softmax activation function.
6. Repeat steps 3-5 until all data points in the data set have been processed. Then calculate the loss function $L(\theta ,W,b)$.
7. Adjust the parameters $\theta$, $W$, and $b$ through optimization methods such as gradient descent to minimize the loss function. Then we get the optimized model $F$.

Since VSQL only extracts local shadow features, it can be easily implemented on quantum devices with topological connectivity limits. Besides, since the $U\left(\theta \right)$ used in circuits are identical, the number of parameters involved is significantly smaller than other commonly used variational quantum classifiers.

Paddle Quantum Implementation¶

We will apply VSQL to classify handwritten digits taken from the MNIST dataset [3], a public benchmark dataset containing ten different classes labeled from '0' to '9'. Here, we consider a binary classification problem for the prupose of demonstration, in which only data labeled as '0' or '1' are used.

First, we import the required packages.

Data preprocessing¶

Each image of a handwritten digit consists of $28\times 28$ grayscale pixels valued in $[0,255]$. We first flatten the $28\times 28$ 2D matrix into a 1D vector $x^i$, and use amplitude encoding to encode every $x^i$ into a 10-qubit quantum state $|x^i⟩$. To do amplitude encoding, we first normalize each vector and pad it with zeros at the end.

Building the shadow circuit¶

Now, we are ready for the next step. Before diving into details of the circuit, we need to clarify several parameters:

• $n$: the number of qubits encoding each data point.
• $n_{qsc}$: the width of the quantum shadow circuit . We only apply $U\left(\theta \right)$ on consecutive $n_{qsc}$ qubits each time.
• $D$: the depth of the circuit, indicating the repeating times of a layer in $U\left(\theta \right)$.

Here, we give an example where $n=4$ and $n_{qsc}=2$.

We first apply $U\left(\theta \right)$ to the first two qubits and obtain the shadow feature $O_1$.

Then, we prepare a copy of the same input state $|x^i⟩$, apply $U\left(\theta \right)$ to the two qubits in the middle, and obtain the shadow feature $O_2$.

Finally, we prepare another copy of the same input state, apply $U\left(\theta \right)$ to the last two qubits, and obtain the shadow feature $O_3$. Now we are done with this data point!

In general, we will need to repeat this process for $n-n_{qsc}+1$ times for each data point. One thing to point out is that we only use one shadow circuit in the above example. When sliding the shadow circuit $U\left(\theta \right)$ through the $n$-qubit Hilbert space, the same parameters $\theta$ are used. You can use more shadow circuits for complicated tasks, and different shadow circuits should have different parameters $\theta$.

Below, we will use a 2-local shadow circuit, i.e., $n_{qsc}=2$ for the MNIST classification task, and the circuit's structure is shown in Figure 5.

The circuit layer in the dashed box is repeated for $D$ times to increase the expressive power of the quantum circuit. The structure of the circuit is not unique. You can try to design your own circuit.

When $n_{qsc}$ is larger, the $n_{qsc}$-local shadow circuit can be constructed by extending this 2-local shadow circuit. Let's print a 4-local shadow circuit with $D=2$ to find out how it works.

Shadow features¶

We've talked a lot about shadow features, but what is a shadow feature? It can be seen as a projection of a state from Hilbert space to classical space. There are various projections that can be used as a shadow feature. Here, we choose the expectation value of the Pauli $X\otimes X\otimes \cdots \otimes X$ observable on selected $n_{qsc}$ qubits as the shadow feature. In our previous example, $O_1=⟨X\otimes X⟩$ on the first two qubits is the first shadow feature we extracted with the shadow circuit.

Classical postprocessing with FCNN¶

After obtaining all shadow features, we feed them into a classical FCNN. We use the softmax activation function so that the output from the FCNN will be a probability distribution. The $i^{\text{th}}$ element of the output is the probability of this data point belonging to the $i^{\text{th}}$ category, and we predict this data point belongs to the category with the highest probability. In order to predict the actual label, we calculate the cumulative distance between the predicted label and the actual label as the loss function to be optimized:

Let's take a look at the actual training process, which takes about eight minutes:

Conclusion¶

VSQL is a hybrid quantum-classical algorithm based on classical shadows, which extracts local features in a convolution way. Combing parameterized circuit $U\left(\theta \right)$ with a classical FCNN, VSQL demonstrates a good performance in binary classification tasks. In the quantum classifier tutorial, we have introduced a commonly used classifier using a parameterized quantum circuit. In that framework, the parameterized circuit is applied to all qubits, and the optimization process searches through the whole Hilbert space to find the optimized $\theta$. Unlike that method, in VSQL, the parameterized circuit $U\left(\theta \right)$ is only applied to a few selected qubits each time. The number of parameters of VSQL for $k$-label classification is the sum of parameters in $U\left(\theta \right)$ and parameters in FCNN, which is in total $n_{qsc}D+\left[\right(n-n_{qsc}+1)+1]k$. Compared to commonly used variational quantum classifiers that need $nD$ parameters in the parameterized quantum circuit, VSQL only has $n_{qsc}D$ parameters in the parameterized local quantum circuit. As a result, the amount of quantum resources (the number of quantum gates) required has been significantly reduced.

References¶

[1] Li, Guangxi, Zhixin Song, and Xin Wang. "VSQL: Variational Shadow Quantum Learning for Classification." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 35. No. 9. 2021.

[2] Goodfellow, Ian, et al. Deep learning. Vol. 1. No. 2. Cambridge: MIT press, 2016.

[3] LeCun, Yann. "The MNIST database of handwritten digits." http://yann.lecun.com/exdb/mnist/ (1998).