Leveraging Data Locality in Quantum Convolutional Classifiers

Abstract

Quantum computing (QC) has opened the door to advancements in machine learning (ML) tasks that are currently implemented in the classical domain. Convolutional neural networks (CNNs) are classical ML architectures that exploit data locality and possess a simpler structure than a fully connected multi-layer perceptrons (MLPs) without compromising the accuracy of classification. However, the concept of preserving data locality is usually overlooked in the existing quantum counterparts of CNNs, particularly for extracting multifeatures in multidimensional data. In this paper, we present an multidimensional quantum convolutional classifier (MQCC) that performs multidimensional and multifeature quantum convolution with average and Euclidean pooling, thus adapting the CNN structure to a variational quantum algorithm (VQA). The experimental work was conducted using multidimensional data to validate the correctness and demonstrate the scalability of the proposed method utilizing both noisy and noise-free quantum simulations. We evaluated the MQCC model with reference to reported work on state-of-the-art quantum simulators from IBM Quantum and Xanadu using a variety of standard ML datasets. The experimental results show the favorable characteristics of our proposed techniques compared with existing work with respect to a number of quantitative metrics, such as the number of training parameters, cross-entropy loss, classification accuracy, circuit depth, and quantum gate count.

1. Introduction

The choice of an appropriate machine learning model for specific applications requires consideration of the size of the model since it is linked to the performance [1]. Considering the aforementioned factor, convolutional neural networks (CNNs)s are preferable to multi-layer perceptrons (MLPs)s because of their smaller size and reduced training time while maintaining high accuracy [2,3]. Preserving the spatiotemporal locality of data allows CNNs to reduce unnecessary data connections and therefore reduces their memory requirements [2,3]. This phenomenon reduces the number of required training parameters and thus incurs less training time [2,3].

In the context of quantum computing, great emphasis has been given to quantum-based machine learning, and, in recent years, various techniques have been devised to develop this field [4]. The contemporary quantum machine learning (QML) techniques can be considered as hybrid quantum–classical variational algorithms [5,6,7,8,9]. Generally, variational quantum algorithms (VQAs) utilizes parameterized rotation gates in fixed quantum circuit structures, usually called ansatz, and is optimized using classical techniques like gradient descent [5,6,7,8,9]. However, like MLPs, preserving data locality is challenging for QML algorithms. For instance, the multidimensionality of input datasets is ignored in contemporary QML algorithms and are flattened into one-dimensional arrays [5,6,7,8,9]. Furthermore, the absence of a generalizable technique for quantum convolution limits the capability of QML algorithms to directly adapt CNN structures.

In this work, we present an multidimensional quantum convolutional classifier (MQCC) to address the shortcomings of the existing CNN implementations in reconciling the locality of multidimensional input data. The proposed VQA technique leverages quantum computing to reduce the number of training parameters and time complexity compared with classical CNN models. Similar to the CNN structures, MQCC contains a sequence of convolution and pooling layers for multifeature extraction from multidimensional input data and a fully connected layer for classification.

The subsequent sections of this paper are organized in the following structure. Section 2 discusses fundamental background information regarding different basic and complex quantum operations. Section 3 highlights existing works that are related to the proposed techniques. The proposed methodology is introduced in Section 4 with details given to the constituent parts. The experimental results and the explanation of the used verification metrics are presented in Section 5. Further discussion about the obtained results is provided in Section 6. Finally, Section 7 concludes this work with potential future directions.

2. Background

In this section, we present background information pertaining to quantum computing and quantum machine learning. Here, we present the quantum gates and operations that are utilized in the proposed multidimensional quantum convolutional classifier (MQCC). In addition, interested readers may find fundamental details related to quantum information and computing in Appendix A.

2.1. Quantum Measurement and Reset

The quantum measurement operation of a qubit is usually and informally referred to as a measurement “gate”. The measurement gate is a nonunitary operation that can project the quantum state of a qubit |$\psi$〉 to the |0〉 or |1〉 basis states [10]. The likelihood of measuring any basis state can be obtained by taking the squared magnitude of their corresponding basis state coefficient. For an n-qubit register |$\psi$〉 with $2^n$ possible basis states, the probability of measuring each qubit in any particular basis state |j〉, where $0\le j<2^n$ is given by $|c_j{|}^2$ [11]. The classical output of n-qubit amplitude-encoded [12] data can be decoded as ${\psi }_{\mathrm{decoded}-\mathrm{data}}^{\mathrm{classical}}$. This classical output vector can be reconstructed by the square root of the probability distribution $\sqrt{P\left(\right|\psi \rangle )}$, as shown in (1), (2), and Figure 1. When amplitude-encoding [12] is used for encoding positive real classical data, the coefficients of the corresponding quantum pure state [10]|$\psi$〉 are also positive real, i.e.,  $c_j\in {\mathbb{R}}^{+}$, where $0\le j<2^n$. Thus, the amplitudes of |$\psi$〉 are numerically equal in values to the coefficients of ${\psi }_{\mathrm{decoded}-\mathrm{data}}^{\mathrm{classical}}$, i.e., $|\psi \rangle ={\psi }_{\mathrm{decoded}-\mathrm{data}}^{\mathrm{classical}}$. Therefore, the quantum state |$\psi$〉 can be completely determined from the measurement probability distribution such that $|\psi \rangle =\sqrt{P\left(\right|\psi \rangle )}$ only when the amplitudes of the quantum state are all of positive real values. Moreover, the probability distribution $P\left(\right|\psi \rangle )$ can be reconstructed by repeatedly measuring (sampling) the quantum state |$\psi$〉. In general, an order of $2^n$ measurements is required to accurately reconstruct the probability distribution. In order to reduce the effects of quantum statistical noise, it is recommended to gather as many circuit samples (shots) [13] as possible.

$$P\left(|\psi \rangle \right)=\left[\begin{array}{c}{p}_0\\ p_1\\ ⋮\\ p_j\\ ⋮\\ p_{N-2}\\ p_{N-1}\end{array}\right]=\left[\begin{array}{c}|c_0{|}^2\\ |c_1{|}^2\\ ⋮\\ |c_j{|}^2\\ ⋮\\ |c_{N-2}{|}^2\\ |c_{N-1}{|}^2\end{array}\right],\mathrm{where}{p}_j={\left|c_j\right|}^2,\mathrm{and}0\le j<N$$

(1)

$${\psi }_{\mathrm{decoded}-\mathrm{data}}^{\mathrm{classical}}=\sqrt{P\left(|\psi \rangle \right)}=\left[\begin{array}{c}|c_0|\\ |c_1|\\ ⋮\\ |c_j|\\ ⋮\\ |c_{N-2}|\\ |c_{N-1}|\end{array}\right]$$

(2)

The reset operation sets the state of qubits to |0〉. This operation consists of a midcircuit measurement gate followed by a conditional X gate [14,15] such that the bit-flip operation is applied when the measured qubit is in state |1〉. The reset gate and its equivalent circuit are both shown in Figure 2 [15].

2.2. Classical-to-Quantum (C2Q)

There are a number of quantum data encoding techniques [12,16], each of which uses different methods to initialize quantum circuits from the ground state. Among the many methods, this work leverages the classical-to-quantum (C2Q) arbitrary state synthesis [12,16] operation to perform amplitude encoding and initialize an n-qubit state $|{\psi }_0\rangle$, see Figure 3. The C2Q operation employs a pyramidal structure of multiplexed $R_y$ and $R_z$ gates. It should be noted that the $R_z$ gates are only required for positive real data. Thus, for positive real data, the circuit depth is $2·2^n-n-2$, while for complex data, the circuit depth is $3·2^n-n-4$ [12].

2.3. Convolutional Neural Networks (CNNs)

CNNs are one of the most widely used types of deep neural networks for image classification [17]. It consists of convolutional, pooling, and fully connected layers. The convolutional layer applies multiple filters to the input to create feature maps. The pooling layer reduces the dimensionality of each feature map but retains the most important information. Some of the most important pooling techniques include max-pooling, average pooling, and sum pooling. Fully connected layers are in the last layers of the CNN network, and their inputs correspond to the flattened one-dimensional matrix generated by the last pooling layer. Activation functions, which are essential for handling complex patterns, are used throughout the network. Finally, a softmax prediction layer is used to generate probability values for each of the possible output labels. The label with the highest probability is selected as the final prediction.

Figure 3. The quantum circuit for classical-to-quantum (C2Q) arbitrary state synthesis [16].

Figure 3. The quantum circuit for classical-to-quantum (C2Q) arbitrary state synthesis [16].

2.4. Quantum Machine Learning with Variational Algorithms

Variational quantum algorithms are a type of quantum–classical technique that facilitates the implementations of machine learning on noisy intermediate-scale quantum (NISQ) machines [5,7]. Current quantum devices are not able to maintain coherent states for sufficient periods, preventing current algorithms from performing meaningful optimization on the machine learning model. Thus, VQAs combine classical optimization algorithms with parameterized quantum circuits, or ansatz. Here, the ansatz takes on the role of the model [5]. One specific type of VQA is the variational quantum classifier (VQC), which is used for classification problems. Existing VQCs [6,8,9] have been shown to be effective for classifying datasets with high accuracy and few training parameters in both simulation and current quantum processors.

3. Related Work

In this section, we discuss the existing related works with an emphasis on quantum machine learning. Our discussion focuses on commonly used data encoding techniques, existing implementations of the convolution and quantum convolution algorithms, and related quantum machine learning (QML) algorithms. Moreover, we also discuss existing quantum convolutional classification algorithms that leverage data locality.

3.1. Data Encoding

For encoding classical image data into the quantum domain, the commonly used methods are Flexible Representation of Quantum Images (FRQI) [18] and Novel Enhanced Quantum Representation (NEQR) [19]. In FRQI, positional and color information is encoded as amplitude encoding and angle encoding, respectively. In NEQR, positions of the pixels are encoded using amplitude encoding but color information is encoded using basis encoding, where q represents the number of bits/qubits allocated for color data. For $N=2^n$ data points, in terms of circuit width and depth, FRQI incurs $n+1$ and $\mathcal{O}\left(4^n\right)$, respectively, while NEQR incurs $n+q$ and $\mathcal{O}\left(qn{2}^n\right)$, respectively [20]. Although these techniques are employed in the existing quantum convolution techniques, their disadvantages are discussed below.

3.2. Convolution

We now discuss existing implementations of convolution and discuss their associated time complexity. These implementations consist of various classical and quantum techniques. In addition, we consider the shortcomings of existing quantum convolution methods.

3.2.1. Classical Convolution

Classical implementations of convolution are usually implemented directly, through general matrix multiplication (GEMM) or through the Fast Fourier transform (FFT). A data size N, running the direct implementation on CPUs, has complexity $\mathcal{O}\left(N^2\right)$ [21], while the complexity of an FFT-based implementation is $\mathcal{O}(NlogN)$ [21]. On GPUs, FFT-based convolution incurs a similar $\mathcal{O}(NlogN)$ complexity [22], while the direct approach requires $\mathcal{O}\left(N_{K}N\right)$ FLOPS [23,24], where $N_K$ is the filter size.

3.2.2. Quantum Convolution

The existing quantum convolution techniques [25,26,27,28,29] rely on fixed filter sizes and support only specific filters at a time, e.g., edge detection. They do not contain methods for implementing a general filter. Additionally, these techniques have a quadratic circuit depth, i.e., $\mathcal{O}\left(n^2\right)$, where $n=\lceil {log}_{2}N\rceil$ is the number of qubits and N is the size of the input data. While these methods appear to show quantum advantage, these results do not include overhead incurred from data encoding. The related methods employ the FRQI and NEQR data encoding methods, leading to inferior performance compared with classical methods once the additional overhead is factored in. The authors in [30] propose an edge-detection technique based on quantum wavelet transform QWT and amplitude encoding, named quantum Hadamard edge detection (QHED), which is not generalizable for multiple convolution kernels or multidimensional data. Thus, their algorithm loses parallelism, increases circuit depth, and is difficult to generalize beyond capturing 1-D features. In [20], the authors have developed a quantum convolution algorithm that supports single feature/kernel and multidimensional data. In this work, we leverage the convolution method from [20] and generalize it to support multiple features/kernels in our proposed MQCC framework.

3.3. Quantum Machine Learning

There exist two primary techniques for quantum convolutional classification that may leverage data locality through convolution: quantum convolutional neural network (QCNN)s [31] and quanvolutional neural networks [32]. QCNNs are inspired by classical convolutional neural networks, employing quantum circuits to perform convolutions, and quanvolutional neural networks replace classical convolutional layers with quantum convolutional (or quanvolutional) layers.

3.3.1. Quantum Convolutional Neural Networks

The QCNN [31] and our proposed multidimensional quantum convolutional classifier (MQCC) are both VQAs models with structures inspired by CNNs. However, QCNNs borrow the superficial structure of CNNs without considering the underlying purpose. Specifically, the QCNN’s quantum ansatz is designed so that its “convolution” and “pooling” operations exploit the locality of qubits in the circuit (rather than the locality of data). However, unlike data locality, qubit locality does not have a practical purpose for machine learning in terms of isolating relevant input features. Moreover, by considering input data as 1-D, QCNNs do not leverage the dimensionality of datasets, which constitute a primary advantage of CNNs. MQCC, on the other hand, faithfully implements CNN operations in quantum circuits, offering performance improvements in circuit execution time (based on circuit depth) classification accuracy over contemporary implementations of QCNNs on classical computers.

3.3.2. Quanvolutional Neural Networks

Quanvolutional neural networks [32] are a hybrid quantum–classical algorithm named eponymously after the quanvolutional layer added to a conventional CNN. These quanvolutional layers serve to decimate a 2D image, which is then sequentially fed into a quantum device. In this manner, the quanvolutional layer effectively exploits data locality. Yet, the model’s dependency on classical operations, specifically the decimation of input data and the repeated serial data I/O transfer, vastly increases compute time. In contrast, the required convolution operation is incorporated into our proposed MQCC, reducing classical–quantum data transfer. Moreover, MQCC takes advantage of parallelism inherent to quantum computers, while quanvolutional neural networks do not. Together, this allows the MQCC to apply convolutional filters to window data in parallel.

4. Materials and Methods

In this section, we describe the materials and methods associated with the proposed multidimensional quantum convolutional classifier (MQCC). The proposed method mainly uses generalized quantum convolution, quantum pooling based on the quantum Haar transform (QHT) and partial measurement [11], and a quantum fully connected layer that is illustrated in this section. To the best of our knowledge, this work is the first to carry out the following:

• Develop a generalizable quantum convolution algorithm for a quantum-convolution-based classifier that supports multiple features/kernels.
• Design a scalable MQCC that uses multidimensional quantum convolution and pooling based on the QHT. This technique reduces training parameters and time complexity compared with other classical and quantum implementations.
• Evaluate the MQCC model in a state-of-the-art QML simulator from Xanadu using a variety of datasets.

4.1. Quantum Fully Connected Layer

A fully connected classical neural network constitutes a collection of layers that each perform a linear transformation on $N_{\mathrm{in}}$ input features $\mathbf{x}\in {\mathbb{R}}^{N_{\mathrm{in}}}$ to generate $N_{\mathrm{out}}$-feature output $\mathbf{y}\in {\mathbb{R}}^{N_{\mathrm{out}}}$ [2,3]. Each layer can be represented in terms of a multiply-and-accumulate (MAC) operation and an addition operation, as shown in (3), where $\mathbf{W}\in {\mathbb{R}}^{N_{\mathrm{out}}\times N_{\mathrm{in}}}$ and $\mathbf{b}\in {\mathbb{R}}^{N_{\mathrm{out}}}$ represent the trainable weight and bias parameters, respectively. Here, we use bold symbols to represent classical quantities, e.g., vectors, while the Dirac notation to represent their quantum counterparts.

$$\mathbf{y}={\mathbf{W}}^T\mathbf{x}+\mathbf{b}$$

(3)

The particular weights and biases that generate the $j^{\mathrm{th}}$ feature of the output, $y_j$, can be isolated by taking the $j^{\mathrm{th}}$ column of $\mathbf{W}$, ${\mathbf{w}}_j$, and the $j^{\mathrm{th}}$ term of $\mathbf{b}$, $b_j$, as shown in (4), which can be directly implemented using quantum circuits. Section 4.1.1 discusses the quantum circuits for a single-feature output, and Section 4.1.2 generalizes the proposed technique for an arbitrary amount of features in the output.

$$\begin{array}{c}{y}_j=({\mathbf{w}}_j·\mathbf{x})+b_j,\mathrm{where}0\le j<N_{\mathrm{out}}\end{array}$$

(4)

4.1.1. Single-Feature Output

For a single-feature output neural network, the weight parameters can be represented as a vector $\mathbf{w}\in {\mathbb{R}}^{N_{\mathrm{in}}}$. Here, $\mathbf{w}$ can be expressed as a quantum state |w〉, as shown in (5), similar to the process of C2Q data encoding; see Section 2.2.

$$\begin{array}{cc}|w\rangle & =\frac{1}{\left|\right|\mathbf{w}\left|\right|}\left[\begin{array}{c}\mathbf{w}\\ 0\\ ⋮\\ 0\end{array}\right]\begin{array}{c}\phantom{w}↕N_{\mathrm{in}}\\ \phantom{\begin{array}{c}0\\ ⋮\\ 0\end{array}}\end{array}↕2^{n_{\mathrm{in}}},\mathrm{where}{n}_{\mathrm{in}}=\lceil {log}_2{N}_{\mathrm{in}}\rceil \hfill \end{array}$$

(5)

Similarly, for a single-feature output, Dirac notation of the MAC operation follows from (4), as shown in (6), where |$\psi$〉 corresponds to the input data.

$$y-b=\langle w|\psi \rangle$$

(6)

However, it is necessary to obtain a quantum operator to perform a parameterized unitary linear transformation from the weights vector |w〉 on the input data |$\psi$〉 using an inverse C2Q operation as shown in Figure 4 and described by (7).

$$\begin{array}{cc}{\mathrm{U}}_{\mathrm{MAC}}\left(\right|w\rangle )& =\left[\begin{array}{c}\langle w|\\ \langle \times |\\ ⋮\\ \langle \times |\end{array}\right]↕2^{n_{\mathrm{in}}}\hfill \\ & ={\mathrm{U}}_{\mathrm{C}2\mathrm{Q}}^{†}\left(\right|w\rangle )\hfill \end{array}$$

(7)

4.1.2. Multifeature Output

A multifeature output can be implemented in a naive approach using Single-Feature Output (7) for an $N_{\mathrm{out}}$-feature output, where $N_{\mathrm{out}}\le N_{\mathrm{in}}$, which can be obtained by encoding each weight vector ${\mathbf{w}}_j:0\le j<N_{\mathrm{out}}$ as a normalized row in ${\mathrm{U}}_{\mathrm{MAC}}$. However, the result might yield a nonunitary operator as the weight vectors can be arbitrary. ${\mathrm{U}}_{\mathrm{MAC}}$ is unitary when each row is orthogonal to all other rows in the matrix, mathematically, $\langle W_i|W_j\rangle ={\delta }_{ij}:\forall i,j\in [0,N_{\mathrm{out}})$. As described in Appendix A.4, independently defined weights can be supported for each feature of the output by multiplexing ${\mathrm{U}}_{\mathrm{MAC}}$. Now, the generic fully connected operation, ${\mathrm{U}}_{\mathrm{FC}}$, can be generated as shown in (8), where $n_{\mathrm{out}}=\lceil {log}_2{N}_{\mathrm{out}}\rceil$.   

$${\mathrm{U}}_{\mathrm{FC}}=\left[\begin{array}{ccc}{\mathrm{U}}_{\mathrm{MAC}}\left(\right|w_0\rangle )& & \\ & \ddots & \\ & & {\mathrm{U}}_{\mathrm{MAC}}\left(\right|w_{2^{n_{\mathrm{out}}}-1}\rangle )\end{array}\right]$$

(8)

By generating $N_{\mathrm{out}}$ replicas of the initial state, $|{\psi }_0\rangle$, the operation can be parallelized, i.e., ${\mathrm{U}}_{\mathrm{MAC}}\left(\right|w_0\rangle )\dots {\mathrm{U}}_{\mathrm{MAC}}\left(\right|w_{2^{n_{\mathrm{out}}}-1}\rangle )$ transformations from (8).

Replication:

To replicate the initial state, $|{\psi }_0\rangle$ $n_{\mathrm{out}}$ qubits, which extends the state vector to a total size of $2^{n_{\mathrm{in}}+n_{\mathrm{out}}}$, see (9) and Figure 5.

$$|{\psi }_1{\rangle =|0\rangle }^{\otimes n_{\mathrm{out}}}\otimes |{\psi }_0\rangle =\left[\begin{array}{c}|{\psi }_0\rangle \\ 0\\ ⋮\\ 0\end{array}\right]\begin{array}{c}\phantom{|{\psi }_0\rangle }↕2^{n_{\mathrm{in}}}\\ \phantom{\begin{array}{c}0\\ ⋮\\ 0\end{array}}\end{array}↕2^{n_{\mathrm{in}}+n_{\mathrm{out}}}$$

(9)

By applying an $n_{\mathrm{out}}$-qubit Hadamard (see Appendix A.3) operation to the relevant qubits the replicas can be obtained through superposition (see (10) and Figure 5), which generates the desired replicas scaled by a factor of $\frac{1}{\sqrt{2^{n_{\mathrm{out}}}}}$ to maintain normalization $\langle {\psi }_2|{\psi }_2\rangle =1$.

$$\begin{array}{c}|{\psi }_2\rangle =\left(H^{\otimes n_{\mathrm{out}}}\otimes I^{\otimes n_{\mathrm{in}}}\right)|{\psi }_1\rangle =\frac{1}{\sqrt{2^{n_{\mathrm{out}}}}}\left[\begin{array}{c}|{\psi }_0\rangle \\ ⋮\\ |{\psi }_0\rangle \end{array}\right]\begin{array}{c}\phantom{|{\psi }_0\rangle }↕2^{n_{\mathrm{in}}}\\ \phantom{⋮}\phantom{\updownarrow }⋮\\ \phantom{|{\psi }_0\rangle }↕2^{n_{\mathrm{in}}}\end{array}↕2^{n_{\mathrm{in}}+n_{\mathrm{out}}}\end{array}$$

(10)

Applying the ${\mathrm{U}}_{\mathrm{FC}}$ Filter:

${\mathrm{U}}_{\mathrm{FC}}$ can perform the MAC operation for the entire $N_{\mathrm{out}}$-feature output in parallel with the set of replicas of $|{\psi }_0\rangle$; see (11) and Figure 5.

$$\begin{array}{cc}|{\psi }_3\rangle ={\mathrm{U}}_{\mathrm{FC}}|{\psi }_2\rangle & =\left[\begin{array}{c}{\mathrm{U}}_{\mathrm{MAC}}\left(\right|w_0\rangle \left)\right|{\psi }_0\rangle \\ ⋮\\ {\mathrm{U}}_{\mathrm{MAC}}\left(\right|w_j\rangle \left)\right|{\psi }_0\rangle \\ ⋮\\ {\mathrm{U}}_{\mathrm{MAC}}\left(\right|w_{2^{n_{\mathrm{out}}}-1}\rangle \left)\right|{\psi }_0\rangle \end{array}\right]\begin{array}{c}\phantom{{\mathrm{U}}_{\mathrm{MAC}}\left(\right|w_0\rangle \left)\right|{\psi }_0\rangle }↕2^{n_{\mathrm{in}}}\\ \phantom{⋮}\phantom{\updownarrow }⋮\\ \phantom{{\mathrm{U}}_{\mathrm{MAC}}\left(\right|w_j\rangle \left)\right|{\psi }_0\rangle }↕2^{n_{\mathrm{in}}}\\ \phantom{⋮}\phantom{\updownarrow }⋮\\ \phantom{{\mathrm{U}}_{\mathrm{MAC}}\left(\right|w_{2^{n_{\mathrm{out}}}-1}\rangle \left)\right|{\psi }_0\rangle }↕2^{n_{\mathrm{in}}}\end{array}↕2^{n_{\mathrm{in}}+n_{\mathrm{out}}}\hfill \\ & =\left[\begin{array}{c}\langle w_0|{\psi }_0\rangle \\ \times \\ ⋮\\ \times \\ ⋮\\ \langle w_j|{\psi }_0\rangle \\ \times \\ ⋮\\ \times \\ ⋮\\ \langle w_{2^{n_{\mathrm{out}}}-1}|{\psi }_0\rangle \\ \times \\ ⋮\\ \times \end{array}\right]\begin{array}{c}\phantom{\begin{array}{c}\langle w_0|{\psi }_0\rangle \\ \times \\ ⋮\\ \times \end{array}}↕2^{n_{\mathrm{in}}}\\ \phantom{⋮}\phantom{\updownarrow }⋮\\ \phantom{\begin{array}{c}\langle w_j|{\psi }_0\rangle \\ \times \\ ⋮\\ \times \end{array}}↕2^{n_{\mathrm{in}}}\\ \phantom{⋮}\phantom{\updownarrow }⋮\\ \phantom{\begin{array}{c}\langle w_{2^{n_{\mathrm{out}}}-1}|{\psi }_0\rangle \\ \times \\ ⋮\\ \times \end{array}}↕2^{n_{\mathrm{in}}}\end{array}↕2^{n_{\mathrm{in}}+n_{\mathrm{out}}}\hfill \end{array}$$

(11)

Data Rearrangement:

The data rearrangement operation can be performed by applying perfect-shuffle gates; see Appendix A.6. It simplifies the output-feature extraction by gathering them into $N_{\mathrm{out}}$ data points at the top of the state vector; see (12) and Figure 5.

$$\begin{array}{cc}|{\psi }_4\rangle =\prod _{i=0}^{n_{\mathrm{out}}-1}\left(\mathrm{RoL}(n_{\mathrm{in}}+n_{\mathrm{out}}-i)\otimes I^{\otimes i}\right)|{\psi }_3\rangle & =\left[\begin{array}{c}\langle w_0|{\psi }_0\rangle \\ ⋮\\ \langle w_j|{\psi }_0\rangle \\ ⋮\\ \langle w_{2^{n_{\mathrm{out}}}-1}|{\psi }_0\rangle \\ \times \\ ⋮\\ \times \end{array}\right]\begin{array}{c}\phantom{\begin{array}{c}\langle w_0|{\psi }_0\rangle \\ ⋮\\ \langle w_j|{\psi }_0\rangle \\ ⋮\\ \langle w_{2^{n_{\mathrm{out}}}-1}|{\psi }_0\rangle \end{array}}↕2^{n_{\mathrm{out}}}\\ \phantom{\begin{array}{c}\times \\ ⋮\\ \times \end{array}}\end{array}↕2^{n_{\mathrm{in}}+n_{\mathrm{out}}}\hfill \end{array}$$

(12)

It is worth mentioning that instead of applying the auxiliary qubits at the most significant position, as shown in the decomposed and simplified fully connected circuit in Figure 6, auxiliary qubits can be applied at the least significant position to avoid perfect-shuffle permutations.

4.1.3. Circuit Depth of the Quantum Fully Connected Layer

As discussed in Section 2.2, ${\mathrm{U}}_{\mathrm{MAC}}$ operation is implemented by applying the C2Q/arbitrary state synthesis operation with a depth of $\left(3·2^{n_{\mathrm{in}}}-n_{\mathrm{in}}-4\right)$ fundamental single-qubit and CNOT gates. The depth is expected to increase by a factor of $2^{n_{\mathrm{out}}}$ when multiplexing ${\mathrm{U}}_{\mathrm{MAC}}$ to implement an $N_{\mathrm{out}}$-feature output [33], see (13).

$$\begin{array}{cc}{\Delta }_{{\mathrm{U}}_{\mathrm{FC}}}(n_{\mathrm{in}},n_{\mathrm{out}})& =2^{n_{\mathrm{out}}}·{\Delta }_{{\mathrm{U}}_{\mathrm{MAC}}\left(n_{\mathrm{in}}\right)}\hfill \\ & =2^{n_{\mathrm{out}}}\left(3·2^{n_{\mathrm{in}}}-n_{\mathrm{in}}-4\right)\hfill \\ & =\mathcal{O}\left(2^{n_{\mathrm{in}}+n_{\mathrm{out}}}\right)\hfill \end{array}$$

(13)

Figure 6. Decomposed and simplified quantum fully connected layer.

Figure 6. Decomposed and simplified quantum fully connected layer.

4.2. Generalized Quantum Convolution

The most significant part of the MQCC framework is the generalized quantum convolution operation with support for arbitrary, parameterized filters. Compared with the classical convolution operation, the convolution operation in the quantum domain achieves exponential improvement in time complexity due to its innate parallelism. The convolution operation consists of stride, data rearrangement, and multiply-and-accumulate (MAC) operations.

Stride:

The first step of quantum convolution is generating shifted replicas of the input data. Quantum decrementers controlled by additional qubits, called “filter” qubits are used for this purpose. The ${\mathrm{U}}_{\mathrm{shift}}^{-1}$ operator shown in Figure A8b shifts the replica by a single stride.

The stride operation, ${\mathrm{U}}_{\mathrm{stride}}$, is composed of controlled quantum decrementers, where each ${\mathrm{U}}_{\mathrm{shift}}^{-1}$ operation has a quadratic depth complexity; see (A15). Thus, the depth of the controlled quantum decrementer can be derived according to (14), where n corresponds to the number of qubits the decrementer is applied to and c reflects the number of control qubits.

$$\begin{array}{cc}{\Delta }_{{\mathrm{cU}}_{\mathrm{shift}}^{\pm 1}}(n,c)& =\sum _{i=c}^{n+c-1}{\Delta }_{\mathrm{MCX}}\left(i\right)\hfill \\ & =\sum _{i=c}^{n+c-1}(48i-196)\hfill \\ & =\sum _{i=0}^{n-1}48i+48c-196n\hfill \\ & =24n^2+48c-220n\hfill \\ & =\mathcal{O}\left(n^{2}c\right)\mathrm{for}\mathrm{large}n\hfill \end{array}$$

(14)

Multiply-and-Accumulate (MAC):

Kernels are applied to the strided replicas of the input data in parallel using the MAC operation; see Figure 4. In the MAC operations, kernels are applied to the contiguous set of data with the help of the inverse arbitrary state synthesis operation. One benefit achieved by using this MAC operation is the superposition of multiple kernels. The superposition of the kernel can be helpful for the classification of multiple features.

Data Rearrangement:

Data rearrangement is required to coalesce the output pieces of the MAC steps and create one contiguous piece of output. This step is performed using perfect shuffle permutation (PSP) operations described in Appendix A.6.

4.2.1. One-Dimensional Multifeature Quantum Convolution

The one-dimensional quantum convolution operation, with a kernel of size $N_K$ terms, requires generating $N_K$ replicas of the input data in a range of possible strides between $0\le k<N_K$. Therefore, a total of $N_{K}N$ terms need to be encoded into a quantum circuit, including the $n_k=\lceil {log}_2{N}_K\rceil$ additional auxiliary qubits, denoted as “kernel” qubits, which are allocated the most significant qubits to maintain data contiguity.

Necessary $N_K$ replicas of the input vector are created by using Hadamard gates; see Figure 7. Convolution kernels can be implemented using multiply-and-accumulate (MAC) operations; as such, it is possible to leverage ${\mathrm{U}}_{\mathrm{MAC}}$, as defined in Section 4.1, for implementing quantum convolution kernels. Given a kernel $\mathbf{K}\in {\mathbb{R}}^{N_K}$, the corresponding kernel operation ${\mathrm{U}}_K$ can be constructed from the normalized kernel $|K\rangle$, as shown in (15).

$${\mathrm{U}}_K={\mathrm{U}}_{\mathrm{MAC}}\left(\right|K\rangle ),\mathrm{where}|K\rangle =\widehat{\mathbf{K}}=\frac{\mathbf{K}}{\left|\right|\mathbf{K}\left|\right|}$$

(15)

When applied to the $n_k$ lower qubits of the state vector, ${\mathrm{U}}_K$ applies the kernel K to all data windows in parallel. However, in CNNs, convolution layers typically must support multiple convolution kernels/features. Fortunately, one major advantage of the proposed quantum convolution technique is that multiple features can be supported by multiplexing only the MAC operations—the stride and data rearrangement operations do not need to be multiplexed; see Figure 7. Accordingly, for $N_F$ features, $n_f=\lceil {log}_2{N}_F\rceil$ must be added to the circuit and placed in superposition using Hadamard gates, similar to the process in (9). Thus, the depth complexity of ${\mathrm{U}}_{\mathrm{stride}}$ can be expressed in terms of ${\Delta }_{{\mathrm{cU}}_{\mathrm{shift}}^{-1}}(n-j,c)$, as described by (16), where $c=1$ for all $0\le j<n_k$; see Figure 7. Similarly, the depth complexity of ${\mathrm{U}}_K$ can be expressed by (17). Finally, The depth of the proposed multifeature 1D quantum convolution can be obtained as (18).   

$$\begin{array}{cc}{\Delta }_{U_{\mathrm{stride}}}(n,n_k)& =\sum _{j=0}^{n_k-1}{\Delta }_{U_{\mathrm{shift}}^{-1}}(n-j,1)=\sum _{j=0}^{n_k-1}\left[24{(n-j)}^2+48-220(n-j)\right]\hfill \\ & \le 24n_k{n}^2-24n_k^{2}n-196n_{k}n+8n_k^3-98n_k^2-58n_k\hfill \\ & =\mathcal{O}\left(n_k{n}^2-n_k^{2}n+n_k^3\right)\mathrm{for}\mathrm{large}n\gg n_k\hfill \end{array}$$

(16)

$$\begin{array}{cc}{\Delta }_{U_K}(n_k,n_f)& =2^{n_f}·{\Delta }_{U_{\mathrm{mac}}\left(n_k\right)}\hfill \\ & \le 2^{n_f}\left(3·2^{n_k}-n_k-4\right)\hfill \\ & =\mathcal{O}\left(2^{n_f+n_k}\right)\mathrm{for}\mathrm{large}{n}_f,n_k\hfill \end{array}$$

(17)

$$\begin{array}{cc}{\Delta }_{1\mathrm{D}\mathrm{conv}}(n,n_k,n_f)& ={\Delta }_H+{\Delta }_{U_{\mathrm{stride}}}(n,n_k)+{\Delta }_{U_K}(n_k,n_f)+{\Delta }_{\mathrm{SWAP}}\hfill \\ & =1+24n_k{n}^2-24n_k^{2}n-196n_{k}n+8n_k^3-98n_k^2-58n_k\hfill \\ & +2^{n_f}\left(3·2^{n_k}-n_k-4\right)+3\hfill \\ & =\mathcal{O}\left(n_k{n}^2-n_k^{2}n+2^{n_f+n_k}\right),\mathrm{where}n\gg n_f,n_k\hfill \end{array}$$

(18)

Figure 7. One-dimensional quantum convolution circuit.

Figure 7. One-dimensional quantum convolution circuit.

4.2.2. Multidimensional Multifeature Quantum Convolution

Multidimensional quantum convolution can be implemented by stacking multiple one-dimensional quantum circuits, as shown in Figure 8. A d-dimensional quantum convolution circuit can be constructed with a stacked kernel of 1-dimensional convolution circuits only when the multidimensional kernels are outer products of d instances of 1-dimensional kernels. The depth of d-D quantum convolution can be obtained as (19).

$$\begin{array}{cc}\hfill {\Delta }_{d-\mathrm{D}\mathrm{conv}}(n,n_k,n_f)& =\mathcal{O}\left(n_{k_{\max }}{n}_{\max }^2-n_{k_{\max }}^2{n}_{\max }+2^{n_k+n_f}\right),\mathrm{where}\hfill \\ \hfill n_{\max }& =\underset{i=0}{\overset{d-1}{max}}\left(n_i\right),n_{k_{\max }}=\underset{i=0}{\overset{d-1}{max}}\left(n_{k_i}\right),\mathrm{and}{n}_{\max }\gg n_f,n_{k_{\max }}\hfill \end{array}$$

(19)

4.3. Quantum Pooling

A critical part of CNNs is the pooling operation or downsampling of the feature maps. One widely used method is average pooling, where only the average of the adjacent pixels in the feature map is preserved, creating a smoothing effect [34].

4.3.1. Quantum Average Pooling using Quantum Haar Transform

The average pooling operation can be implemented using the quantum wavelet transform (QWT) [11], which has the advantage of preserving data locality using wavelet transform decomposition. It is a commonly used technique for dimension reduction in image processing [11]. In this work, we utilize the simplest and first wavelet transform, quantum Haar transform (QHT) [11] to implement quantum pooling operation. This operation is executed in two steps: the Haar wavelet operation and data rearrangement.

Haar Wavelet Operation:

For separating the high- and low-frequency components from the input data, H gates are applied in parallel. The number of H gates applied in QHT is equal to the levels of decomposition.

Data Rearrangement:

After separating the high- and low-frequency components, quantum rotate-right (RoR) operations are applied to group them accordingly. As mentioned before, the proposed framework is highly parallelizable regardless of the dimensions of the data, as the QHT operation can be applied to multiple dimensions of data in parallel.

As shown in Figure 9a, for a single-level of decomposition, H gates are applied on one qubit (the least significant qubit) per dimension, and for ℓ-level decomposition, shown in Figure 9b, l number of H gates are applied per dimension. Each level of decomposition reduces the size of the corresponding dimension by a factor of 2.

4.3.2. Quantum Euclidean Pooling using Partial Measurement

In machine learning applications, the average and maximum (max) pooling [34] are the most commonly used pooling schemes for dimension reduction. The two schemes differ in the sharpness of data features. On one hand, max pooling yields a sharper definition of input features, which makes it preferable for edge detection and certain classification applications [34]. On the other hand, average pooling offers a smoother dimension reduction that may be preferred in other workloads [34]. Thus, to accompany our implementation of quantum averaging pooling using QHT (see Section 4.3.1), it would be beneficial to have an implementation of quantum max pooling. However, such an operation would be nonunitary, creating difficulty for the implementation of quantum max pooling [35]. Therefore, instead of max pooling, we utilize an alternative pooling technique we denote as quantum Euclidean pooling.

Mathematically, average and Euclidean pooling are special cases of the p-norm [36], where for a vector of size N elements, the p-norm or ${\ell }^p$ norm of a vector $\mathbf{x}\in {\mathbb{C}}^N$ is given by (20) for $p\in \mathbb{Z}$ [36]. The average pooling occurs for the 1-norm ($p=1$) and Euclidean pooling occurs for the 2-norm ($p=2$). A notable benefit of the Euclidean pooling technique is its zero-depth circuit implementation by leveraging partial measurement [35].

$${\left|\right|\mathbf{x}\left|\right|}_p={\left(\sum _{i=0}^N{x}_i^p\right)}^{\frac{1}{p}}$$

(20)

This work leverages the multilevel, d-dimensional quantum Euclidean pooling circuit presented in [35]; see Figure 10. Here, for each dimension i, ${\ell }_i$ is the number of decomposition levels for dimension where $0\le i<d$ [35].

4.4. Multidimensional Quantum Convolutional Classifier

The proposed multidimensional quantum convolution classifier framework, see Figure 11, resembles a CNN [2] structures. After a sequence of convolution pooling (CP) pairs, the model is finally connected to a fully connected layer, see Figure 6 and Figure 11. The total number of layers in the proposed model can be expressed in terms of CP pairs as $2\lambda +1$, where $\lambda$ is the number of CP pairs.

Figure 10. Multilevel, d-D Euclidean pooling circuit.

Figure 10. Multilevel, d-D Euclidean pooling circuit.

Figure 11. High-level overview of the MQCC architecture.

Figure 11. High-level overview of the MQCC architecture.

It is worth mentioning that there is no advantage in changing the number of features among convolution/pooling layers in the MQCC because of the implementation constraints. Therefore, the total number of kernel features can be estimated globally instead of layer-by-layer.

The circuit width of the MQCC (21) can be derived from the number of convolution layers, pooling layers, and the fully connected layer. Input data are encoded using n qubits, and each convolution operation adds $n_f=\lceil {log}_2{N}_F\rceil$ qubits for $N_F$ features and $n_k=\lceil {log}_2{N}_K\rceil$ qubits per layer for kernels. In addition, the fully connected operation contributes $n_c=\lceil {log}_2{N}_C\rceil$ qubits to encode $N_C$ output features/classes. On the other hand, each Euclidean pooling layer frees ℓ qubits, which can then be reused by other layers.

$$\begin{array}{cc}{n}_{\mathrm{MQCC}}^{\mathrm{Avg}}& =n+n_f+\lambda n_k+n_c\hfill \\ n_{\mathrm{MQCC}}^{\mathrm{Euclidean}}& =n+n_f+n_k+max(0,(\lambda -1)n_k-\lambda \ell +n_c)\hfill \end{array}$$

(21)

The MQCC can be further parallelized in terms of circuit depth between the interlayer convolution/fully connected layers. This parallelism can be achieved by performing (multiplexed) MAC operations from the quantum convolution and fully connected layers in parallel with the stride operation from the previous layer(s) of quantum convolution. The circuit depth of MQCC can be derived as shown in (22).

$$\begin{array}{cc}{\Delta }_{\mathrm{MQCC}}& =\underset{i=0}{\overset{d-1}{max}}\Delta {\mathrm{U}}_{\mathrm{stride}}(n_i,n_{k_i})+\lambda ({\Delta }_H+{\Delta }_{\mathrm{SWAP}})\hfill \\ & +\sum _{j=0}^{\lambda -2}max\left(\underset{i=0}{\overset{d-1}{max}}{\Delta }_{{\mathrm{U}}_{\mathrm{stride}}}(n_i-j{\ell }_i,n_{k_i}),{\Delta }_{{\mathrm{U}}_K}(n_k,n_f)\right)\hfill \\ & +max\left({\Delta }_{{\mathrm{U}}_{\mathrm{FC}}}(n-(\lambda -1)\ell ,n_c),{\Delta }_{{\mathrm{U}}_K}(n_k,n_f)\right)\hfill \end{array}$$

(22)

4.5. Optimized MQCC

Figure 12 presents a width-optimized implementation of MQCC, which we refer to as Quantum-Optimized Multidimensional Quantum Convolutional Classifier (MQCC Optimized). To reduce the required number of qubits, the convolution and pooling operations are swapped, which allows kernel qubits to be trimmed for each convolution layer, see Section 4.2. To achieve higher processing efficiency, trimmed qubits are reassigned to later layers of dimension reduction and run in parallel. Furthermore, only Euclidean pooling with partial measurements is used because of the inherent circuit depth efficiency. The circuit width of MQCC Optimized is shown in (23), where n is the number of qubits corresponding to the data size, $n_f$ is the number of qubits corresponding to the features, and $n_c$ is the number of qubits corresponding to the classes. If necessary, additional pooling operations can be applied to keep the circuit width at or below the absolute minimum number of qubits n by excluding qubits dedicated to features and classes. It should be noted that reordering convolution and pooling operations reduces the maximum number of convolution operations by 1.

$$n_{\mathrm{MQCC}\mathrm{Optimized}}=n+n_f+n_c$$

(23)

Accordingly, the depth of MQCC Optimized can be expressed as shown in (24).

$$\begin{array}{cc}{\Delta }_{\mathrm{MQCC}}^{\mathrm{opt}}& =\underset{i=0}{\overset{d-1}{max}}\Delta {\mathrm{U}}_{\mathrm{stride}}(n_i-{\ell }_i,n_{k_i})+\lambda ({\Delta }_H+{\Delta }_{\mathrm{SWAP}})\hfill \\ & +\sum _{j=1}^{\lambda -1}max\left(\underset{i=0}{\overset{d-1}{max}}{\Delta }_{{\mathrm{U}}_{\mathrm{stride}}}(n_i-j{\ell }_i,n_{k_i}),{\Delta }_{U_K}(n_k,n_f)\right)\hfill \\ & +max\left({\Delta }_{{\mathrm{U}}_{\mathrm{FC}}}(n-\lambda \ell ,n_c),{\Delta }_{{\mathrm{U}}_K}(n_k,n_f)\right)\hfill \end{array}$$

(24)

To further reduce the depth of MQCC Optimized, we investigated replacing inverse-C2Q operations for MAC operations with different parameterized ansatz. More specifically, a common ansatz in QML, namely NLocal operation in Qiskit [37] or BasicEntanglerLayer in Pennylane [38], was utilized; see Figure 13. The depth of this ansatz is linear with respect to the data qubits (see (25)), which is a significant improvement over arbitrary state synthesis, which has a circuit depth of $\mathcal{O}\left(2^n\right)$ for an n-qubit state. Although the ansatz could potentially reduce circuit depth, its structure lacks theoretical motivation or guarantees for high fidelity when modeling convolution kernels.

$${\Delta }_{\alpha }(n,\ell )=\ell ·n+1$$

(25)

Figure 12. Quantum-Optimized Multidimensional Quantum Convolutional Classifier (MQCC Optimized).

Figure 12. Quantum-Optimized Multidimensional Quantum Convolutional Classifier (MQCC Optimized).

Figure 13. Alternate ansatz $\left({\mathrm{U}}_{\mathrm{ansatz}}\right)$ option, distinct from $U_{\mathrm{MAC}}$.

Figure 13. Alternate ansatz $\left({\mathrm{U}}_{\mathrm{ansatz}}\right)$ option, distinct from $U_{\mathrm{MAC}}$.

5. Experimental Work

In this section, we first detail our experimental setup, followed by the results for the proposed MQCC technique. Experiments were conducted using real-world, multidimensional image data to test both the individual and composite components of our techniques.

5.1. Experimental Setup

The MQCC methodology was validated by first evaluating its most important component, namely convolution using the metric of fidelity and then evaluating the complete technique, i.e., MQCC and MQCC optimized by conducting classifications experiments using 1D, 2D, and 3D datasets.

For the convolution experiments on 1D data, we used audio material published by the European Broadcasting Union for sound quality assessment [39]. Using a preprocessing step, the data were converted into a single channel, with the data size varying from $2^8$ data points to $2^{20}$ data points, sampled at a rate of $44.1$ kHz.

For conducting 2D convolution experiments, we used 2D images that are either black and white or color Jayhawks [40], as shown in Figure 14. These images range in size from $(8\times 8)$ pixels to $(512\times 512\times 3)$ pixels. For the 3D image experiments, we used hyperspectral images from the Kennedy Space Center (KSC) dataset [41]. The images were preprocessed and resized, with the sizes ranging from $(8\times 8\times 8)$ pixels to $(128\times 128\times 128)$ pixels. Simulations of quantum convolution operation were run using Qiskit SDK (v0.45.0) from IBM Quantum [13] over the given data. To demonstrate the effect of statistical noise on the fidelity (26), both noise-free and noisy (with 1,000,000 circuit samples/shots) simulation environments were evaluated.

$$\mathrm{Fidelity}\left(\mathbf{X},\mathbf{Y}\right)=\frac{\langle \mathbf{X},\mathbf{Y}\rangle }{{\left|\right|\mathbf{X}\left|\right|}_F{\left|\right|\mathbf{Y}\left|\right|}_F}$$

(26)

To evaluate the performance of the complete MQCC and the MQCC-optimized technique, they were tested against CNNs, QCNNs, and quanvolutional neural networks by their capabilities of binary classification on real-world datasets, such as MNIST [42], FashionMNIST [43], and CIFAR10 [44]. The classical components in these trials were run using PyTorch (v2.1.0) [45], while the quantum circuits used Pennylane (v0.32.0), a Xanadu QML-focused framework [46].

The experiments were performed on a cluster node at the University of Kansas [47]. The node consisted of a 48-Core Intel Xeon Gold 6342 CPU, three NVIDIA A100 80 GB GPUs (CUDA version 11.7) with PCIe 4.0 connectivity and 256 GB of 3200 MHz DDR4 RAM. To account for initial parameter variance in ML or noise in noisy simulations, experiments were repeated for 10 trials, with the median being displayed in graphs.

5.2. Configuration of ML Models

The different techniques fundamentally being ML models meant that they could share some parameters and metrics during their testing. For example, the log loss and the Adam optimizer [48] were shared by all the techniques, and the “feature-count” metric was shared between the CNN and MQCC, which have 4 features per convolution layer. The parameters that were unique to each model are discussed next.

Convolutional Neural Networks: In Figure 15, we show the classification accuracy of the CNN model on ($16\times 16$) and ($28\times 28$) FashionMNIST datasets, using average pooling, max pooling, and Euclidean pooling. The plots show the obtained accuracy with and without ReLU [49], which is an optional layer that can be appended to each pooling layer in a CNN. Based on the results, which show Max Pooling without ReLU to be the configuration with the best accuracy, we chose it to be the baseline configuration for CNN in our tests.

Quanvolutional Neural Networks: While quanvolutional neural networks were initially introduced without a trainable random quantum circuit in the quanvolutional layer, later work has suggested implementing parameterized and trainable quanvolutional layers. We, therefore, test both the trainable and nontrainable quanvolutional techniques, and Figure 16 demonstrates that the trainable variant outperforms the other method in the ($16\times 16$) FashionMNIST dataset, although the differences are negligible on the ($28\times 28$) dataset. This is used as evidence behind our decision to use the trainable variant of the quanvolutional neural network as the baseline for comparison with the other models.

Quantum Convolutional Neural Networks: We based our implementation of the QCNN on [50]; however, some modifications were made to the technique to work around limitations present in the data encoding method. When encoding data that are not of size $2^n$ in each dimension, the original method flattens (vectorizes) the input before padding with zeros, as opposed to padding each dimension. However, this sacrifices the advantage of multidimensional encoding, where each dimension is mapped to a region of qubits. To ensure a level field between QCNN and MQCC, the ($16\times 16$) and ($28\times 28$) FashionMNIST datasets were tested both for the original (1D) and a corrected (2D) data encoding configuration of the QCNN, the results of which are shown in Figure 17. As expected, we see a clear improvement on the ($28\times 28$) dataset, and based on this, we chose the corrected (2D) data encoding method as our baseline QCNN for comparison against other ML models.

5.3. Results and Analysis

We first present the results of the quantum convolution operations on data with varying dimensionalities. Then, we compare the fidelity results of the quantum convolution under a noisy simulation environment with reference to classical convolution implementation. Finally, we present the results for MQCC.

Quantum Convolution Results

The fidelity of the quantum convolution technique was tested in both a noise-free and noisy environment against a classical implementation using common $(3\times 3)$ and $(5\times 5)$ filter kernels. These kernels, as described in (27)–(33), include the Averaging $F_{\mathrm{avg}}$, Gaussian blur $F_{\mathrm{blur}}$, Sobel edge-detection $F_{\mathrm{Sx}}$/$F_{\mathrm{Sy}}$, and Laplacian of Gaussian blur (Laplacian) $F_{\mathcal{L}}$ filters. To enable a quantum implementation of these kernels, a classical preprocessing step zero-padded each kernel until the dimensions were an integer power of two. As negative values may occur in classical convolution, the magnitudes of the output values were cast into a single-byte range $[0,255]$ in a classical postprocessing step.

$$F_{\mathrm{avg}}^{3\times 3}=\frac{1}{9}\left[\begin{array}{ccc}1& 1& 1\\ 1& 1& 1\\ 1& 1& 1\end{array}\right],$$

(27)

$$F_{\mathrm{avg}}^{5\times 5}=\frac{1}{25}\left[\begin{array}{ccccc}1& 1& 1& 1& 1\\ 1& 1& 1& 1& 1\\ 1& 1& 1& 1& 1\\ 1& 1& 1& 1& 1\\ 1& 1& 1& 1& 1\end{array}\right]$$

(28)

$$F_{\mathrm{blur}}^{3\times 3}=\frac{1}{16}\left[\begin{array}{ccc}1& 2& 1\\ 2& 4& 2\\ 1& 2& 1\end{array}\right],$$

(29)

$$F_{\mathrm{blur}}^{5\times 5}=\frac{1}{273}\left[\begin{array}{ccccc}1& 4& 7& 4& 1\\ 4& 16& 26& 16& 4\\ 7& 26& 41& 26& 7\\ 4& 16& 26& 16& 4\\ 1& 4& 7& 4& 1\end{array}\right]$$

(30)

$$F_{\mathrm{Sx}}=\frac{1}{4}\left[\begin{array}{ccc}1& 0& -1\\ 2& 0& -2\\ 1& 0& -1\end{array}\right],$$

(31)

$$F_{\mathrm{Sy}}=\frac{1}{4}\left[\begin{array}{ccc}1& 2& 1\\ 0& 0& 0\\ -1& -2& -1\end{array}\right]$$

(32)

$$F_{\mathcal{L}}^{3\times 3}=\frac{1}{6}\left[\begin{array}{ccc}1& 1& 1\\ 1& -8& 1\\ 1& 1& 1\end{array}\right],$$

(33)

$$F_{\mathcal{L}}^{5\times 5}=\frac{1}{20}\left[\begin{array}{ccccc}1& 1& 1& 1& 1\\ 1& 1& 1& 1& 1\\ 1& 1& -24& 1& 1\\ 1& 1& 1& 1& 1\\ 1& 1& 1& 1& 1\end{array}\right]$$

(34)

A 1D averaging kernel of sizes ($1\times 3$) and ($1\times 5$) was applied to audio files after preprocessing, described in Section 5.1, with data size ranging from $2^8$ to $2^{20}$ data points.

Table 1. The 1D convolution (averaging) filters applied to 1D audio samples [39].

Data Size (No. of Sample Points)	(3) Averaging Classical/ Noise-Free	(3) Averaging Noisy (${10}^6$ Shots)	(5) Averaging Classical/ Noise-Free	(5) Averaging Noisy (${10}^6$ Shots)
256
4096
65,536
1,048,576

presents the reconstructed output data of this operation, with Figure 18 displaying the associated calculated fidelity.

The 2D averaging, Gaussian blur, Sobel edge-detection, and Laplacian kernels were applied to 2D black-and-white (B/W) and 3D RGB Jayhawk images, see Figure 14, ranging from ($8\times 8$) to ($512\times 512$) pixels and ($8\times 8\times 3$) pixels to ($512\times 512\times 3$) pixels, respectively.

The reconstruction from convolution operations in classical, noise-free, and noisy environments on ($128\times 128$)- and ($128\times 128\times 3$)-pixel input images can be seen in

Table 2. Two-dimensional convolution kernels applied to a ($128\times 128$) bw image [40].

Kernel	($3\times 3$) Kernel Classical/Noise-Free	($3\times 3$) Kernel Noisy (${10}^6$ Shots)	($5\times 5$) Kernel Classical/Noise-Free	($5\times 5$) Kernel Noisy (${10}^6$ Shots)
Average
Gaussian
Sobel-X
Sobel-Y
Laplacian

and

Table 3. Two-dimensional convolution kernels applied to a ($128\times 128\times 3$) rgb image [40].

Kernel	($3\times 3$) Kernel Classical/Noise-Free	($3\times 3$) Kernel Noisy (${10}^6$ Shots)	($5\times 5$) Kernel Classical/Noise-Free	($5\times 5$) Kernel Noisy (${10}^6$ Shots)
Average
Gaussian
Sobel-X
Sobel-Y
Laplacian

, respectively.

Finally, a 3D averaging kernel of sizes ($3\times 3\times 3$) and ($5\times 5\times 5$) was applied to hyperspectral images from the KSC dataset [41]. The images were preprocessed and resized to a power of two, ranging from ($8\times 8\times 8$) pixels to ($128\times 128\times 128$) pixels in size.

Table 4. The 3D convolution (averaging) filters applied to 3D hyperspectral images (bands 0, 1, and 2) [41].

Data Size (${\mathit{N}}_{\mathbf{rows}}\times {\mathit{N}}_{\mathbf{cols}}\times {\mathit{N}}_{\mathbf{bands}}$)	($3\times 3\times 3$) Averaging Classical/ Noise-Free	($3\times 3\times 3$) Averaging Noisy (${10}^6$ Shots)	($5\times 5\times 5$) Averaging Classical/ Noise-Free	($5\times 5\times 5$) Averaging Noisy (${10}^6$ Shots)
($8\times 8\times 8$)
($16\times 16\times 16$)
($32\times 32\times 32$)
($64\times 64\times 64$)
($128\times 128\times 128$)

shows the reconstructed output images from convolution operations in classical, noise-free, and noisy environments.

Compared with the expected, classically generated results, the noise-free quantum results tested at 100% fidelity across all trials. Therefore, in a noise-free environment, given the same inputs, the proposed convolution techniques have no degradation compared with classical convolution. The fidelity of noisy simulations using 1D audio, 2D B/W, 2D RGB, and 3D hyperspectral data are presented in Figure 18, Figure 19, Figure 20, and Figure 21, respectively. The fidelity degradation in these figures is due to the statistical noise where the constant shot count (number of circuit samples) becomes less and less sufficient to describe the increasing data size. We could improve this reduction from noise by increasing the number of shots, but our experiments were limited to 1,000,000 shots in simulation, which is the maximum number of shots allowed by the simulator.

6. Discussion

In this section, we discuss the results of our experiments with MQCC and compare them against the other models in terms of the number of required training parameters, the accuracy of the model, and the circuit depth of the implemented model. The number of qubits required by MQCC can be easily calculated using (23). The number of qubits required for data encoding and filter implementation can be obtained from the dimensions of the data and filter respectively, i.e., $n=\lceil {log}_{2}128\rceil +\lceil {log}_{2}128\rceil +\lceil {log}_{2}3\rceil =16$ for $(128\times 128\times 3)$ data, $n_f=\lceil {log}_{2}4\rceil =2$ qubits for four features. Similarly, the number of qubits required for feature classes can be calculated as, for example, $n_c=\lceil {log}_{2}2\rceil =1$ for 2 classes. All together, for input data encoded into n qubits, the optimized MQCC requires $n+n_f+n_c=19$ qubits.

6.1. Number of Parameters

Among the classical ML models evaluated, MQCC had the fewest trainable parameters; see Figure 22. This implies potential advantages such as reduced memory usage and faster performance when using classical gradient descent. Although the reduction in parameter decreases from (MLP to CNN) and then further from (CNN to MQCC), parameter reduction diminishes from (MLP to CNN) and further from (CNN to MQCC), and there is still a significant $83.62\%$ decrease in parameter count.

6.2. Loss History and Accuracy

ML-based classifiers aim to maximize the accuracy of their classifications, measured by a loss function during training to estimate the accuracy that may be exhibited when deployed. Hence, Figure 23 and Figure 24 depict the performance of the ML models across the experimental datasets in their plotting of log-loss history and classification accuracy. The MNIST [42] dataset is not complex enough to effectively distinguish models; however, differences begin to emerge in the FashionMNIST [43] and CIFAR10 [44] datasets. MLP consistently achieves the highest accuracy across trials due to its larger parameter count, allowing for greater flexibility in adapting to nuances in input. The CNN showcases its ability to select relevant input features using convolution and data locality, demonstrating the second-highest accuracy. Among the tested models, QCNN generally performs the poorest, displaying its inability to properly leverage data locality. However, comparing the accuracy of MQCC and quanvolutional neural networks is inconclusive. Quanvolutional neural networks performed better on FashionMNIST, whereas MQCC performed better on CIFAR10.

6.3. Gate Count and Circuit Depth

Although comparing MQCC, MQCC Optimized, and QCNN with quantum metrics like gate count and circuit depth is viable, see Figure 25, it is challenging to include quanvolutional neural networks in the comparison due to their significant differences from the other models. These differences are due to the quantum component within quanvolutional neural networks constituting a small fraction of the entire algorithm, bringing it closer to a classical algorithm than a quantum algorithm. Meanwhile, comparing the techniques of MQCC and QCNN in Figure 25 highlights the rationale behind developing MQCC Optimized. Initially, MQCC performed worse than QCNN in gate count and circuit depth. However, after optimizations, MQCC matched the performance of QCNN and even outperformed it in the best-case scenarios. While the QCNN architecture appears more suitable for shallower quantum circuits than MQCC, it is because the high parallelization of each QCNN layer halves the active qubits.

Despite QCNN using half the active qubits per layer than MQCC, MQCC utilizes the extra qubits for weights and features, with each pooling layer reducing the qubit count by a constant amount, $n_k$. However, as QCNN structures are motivated by the classical convolution operation, they usually need more complex and deeper “convolution” and “pooling” ansatz to attain a higher accuracy.

Figure 25. Gate count and circuit depth of MQCC vs. QCNN: (a) gate count; (b) circuit depth.

Figure 25. Gate count and circuit depth of MQCC vs. QCNN: (a) gate count; (b) circuit depth.

6.4. Complexity Comparison with Classical Models

The proposed method can also be compared with the classical models in terms of temporal complexity. The temporal/depth complexity of MQCC Optimized can be easily derived using Figure 11 and Figure 12, as well as (24), as shown in (35), where ${\mathcal{O}}_{stride}$ and ${\mathcal{O}}_{{\mathrm{U}}_{\mathrm{k}}}$ are combined to obtain ${\mathcal{O}}_{conv}$.

$$\begin{array}{cc}{\mathcal{O}}_{MQCC}^{opt}(n,n_k,n_f,n_c)& ={\mathcal{O}}_{stride}(n-\ell ,n_k)+{\mathcal{O}}_{{\mathrm{U}}_{\mathrm{k}}}(n_k,n_f)+{\mathcal{O}}_{FC}(n-\lambda \ell ,n_c)\hfill \\ & ={\mathcal{O}}_{conv}(n,n_k,n_f)+{\mathcal{O}}_{FC}(n-\lambda \ell ,n_c)\hfill \end{array}$$

(35)

For a MAC-based fully connected layer, we can consider ${\mathrm{U}}_{\mathrm{FC}}={\mathrm{U}}_{\mathrm{MAC}}$ to obtain the complexity using (13) and (35), as shown in (36). Similarly, for an ansatz-based fully connected layer, we can consider ${\mathrm{U}}_{\mathrm{FC}}={\mathrm{U}}_{\mathrm{ansatz}}$ to obtain the complexity using (25) and (35), as shown in (37).

$$\begin{array}{cc}{\mathcal{O}}_{MQC{C}_{MAC}}^{opt}& =\mathcal{O}(n_k{n}^2-n_k^{2}n+2^{n_k+n_f})+\mathcal{O}\left(2^{n+n_c-\lambda \ell }\right)\hfill \\ & =\mathcal{O}\left(n^2\right)+\mathcal{O}\left(2^n\right)=\mathcal{O}\left(2^n\right)=\mathcal{O}\left(N\right)\approx {\mathcal{O}}_{FC}\hfill \end{array}$$

(36)

$$\begin{array}{cc}{\mathcal{O}}_{MQC{C}_{ansatz}}^{opt}& =\mathcal{O}(n_k{n}^2-n_k^{2}n+2^{n_k+n_f})+\mathcal{O}\left(n\right)\hfill \\ & =\mathcal{O}\left(n^2\right)+\mathcal{O}\left(n\right)=\mathcal{O}\left({\left({log}_{2}N\right)}^2\right)\approx {\mathcal{O}}_{conv}\hfill \end{array}$$

(37)

The general expression for calculating the temporal complexity of classical CNNs is shown in (38). It can be broadly divided into three parts: the complexity of the convolutional layers, the complexity of the pooling layers, and the complexity of the fully connected layers. The complexity of the combined layers of convolution and pooling is primarily determined by the number of filters, the size of the filters, and the dimensions of the input feature maps, while the complexity of fully connected layers depends on the number of neurons in the layers. It is worth mentioning that in classical CNNs, convolutional layers dominate both the pooling and fully connected layers in the overall execution time, as expressed by (38).

$$\begin{array}{cc}{\mathcal{O}}_{CNN}^{Classical}& ={\mathcal{O}}_{conv}+{\mathcal{O}}_{pooling}+{\mathcal{O}}_{fully\_connected\_layer}\approx {\mathcal{O}}_{conv}\hfill \end{array}$$

(38)

A comparison of the depth/temporal complexity of MQCC Optimized against the classical method is shown in

Table 5. Comparison of depth/time complexity of proposed MQCC-optimized against classical CNN.

a Depth complexity of C2Q [12] data encoding (I/O) technique
	${\mathcal{O}}_{\mathrm{I}/\mathrm{O}}\left(2^n\right)={\mathcal{O}}_{\mathrm{I}/\mathrm{O}}\left(N\right)$
b Complexity of proposed technique compared with classical convolutional neural network
MQCC Optimized with MAC-based fully connected layer	MQCC Optimized with ansatz-based fully connected layer	Direct (CPU) [21]	FFT (CPU/GPU) [21,22]	GEMM (GPU) [23,24]
$\begin{array}{c}{\mathcal{O}}_{\mathrm{alg}}\left(2^n)\right.\equiv {\mathcal{O}}_{\mathrm{alg}}\left(N\right)\hfill \\ {\mathcal{O}}_{\mathrm{alg}}+\mathrm{I}/\mathrm{O}\left(2^n+2^n)\right.\equiv \hfill \\ {\mathcal{O}}_{\mathrm{alg}}+\mathrm{I}/\mathrm{O}\left(N\right)\hfill \end{array}$	$\begin{array}{c}{\mathcal{O}}_{\mathrm{alg}}\left(n^2)\right.\equiv {\mathcal{O}}_{\mathrm{alg}}\left({(logN)}^2\right)\hfill \\ {\mathcal{O}}_{\mathrm{alg}}+\mathrm{I}/\mathrm{O}\left(n^2+2^n)\right.\equiv \hfill \\ {\mathcal{O}}_{\mathrm{alg}}+\mathrm{I}/\mathrm{O}\left(N\right)\hfill \end{array}$	$\begin{array}{c}{\mathcal{O}}_{\mathrm{alg}}\left(4^n\right)\equiv \hfill \\ {\mathcal{O}}_{\mathrm{alg}}\left(N^2\right)\hfill \end{array}$	$\begin{array}{c}{\mathcal{O}}_{\mathrm{alg}}\left(n{2}^n\right)\equiv \hfill \\ {\mathcal{O}}_{\mathrm{alg}}\left(NlogN\right)\hfill \end{array}$	$\begin{array}{c}{\mathcal{O}}_{\mathrm{alg}}\left(2^{(n+n_k)}\right)\equiv \hfill \\ {\mathcal{O}}_{\mathrm{alg}}\left(N_{K}N\right)\hfill \end{array}$

. It details the depth complexities of C2Q data encoding (I/O overhead), two variants of MQCC Optimized, i.e., MAC-based fully connected layer and ansatz-based fully connected layer, and three variants of classical CNN implementation. Among the classical CNN variants, i.e., Direct (CPU), FFT (CPU/GPU), and GEMM (GPU), we consider the Direct (CPU) case as the worst case and GEMM (GPU) as the best case. The MQCC-optimized algorithm with a MAC-based fully connected layer matches the best case. Considering the depth of the I/O circuit, which, in this case, is the depth of the C2Q method, the overall complexity of MQCC is still better than the worst case in classical methods. The MQCC-optimized algorithm, with an ansatz-based fully connected layer, has the least complexity among all the compared models. Although the I/O overhead represents the worst-case scenario for our proposed technique (see Table 5), the complexity of our proposed MQCC technique, including the I/O overhead, matches the best case in classical methods. It is worth emphasizing that the I/O overhead is not intrinsic to our proposed technique; rather, it is a general consideration for any data-intensive quantum application like quantum machine learning (QML) classification. Moreover, our proposed MQCC method provides two additional advantages over classical CNNs, being highly parallelizable and requiring fewer training parameters, see Figure 22, which ultimately leads to fewer resource requirements than classical CNNs.

7. Conclusions

In this paper, we presented a multidimensional quantum convolutional classifier (MQCC) that consists of quantum convolution, quantum pooling, and a quantum fully connected layer. We leveraged existing convolution techniques to support multiple features/kernels and utilized them in our proposed method. Furthermore, we proposed a novel width-optimized quantum circuit that reuses freed-up qubits from the pooling layer in the subsequent convolutional layer. The proposed MQCC additionally preserves data locality in the input data which has shown to improve data classification in convolutional classifiers. The MQCC methodology is generalizable for any arbitrary multidimensional filter operation and pertinent for multifeature extraction. The proposed method can also support data of arbitrary dimensionality since the underlying quantum pooling and convolution operations are generalizable across data dimensions. We experimentally evaluated the proposed MQCC on various real-world multidimensional images, utilizing several filters through simulations on state-of-the-art quantum simulators from IBM Quantum and Xanadu. In our experiments, MQCC achieved higher classification accuracy over contemporary QML methods while having a reduced circuit depth and gate count. In our future work, we are planning on expanding MQCC with additional convolution capabilities, such as arbitrary striding and dilation, and further optimizing it for deployment on real-world quantum processors. In addition, we will investigate using our proposed quantum techniques for real-life applications such as medical imaging and classification.