F-Divergences and Cost Function Locality in Generative Modelling with Quantum Circuits

Abstract

Generative modelling is an important unsupervised task in machine learning. In this work, we study a hybrid quantum-classical approach to this task, based on the use of a quantum circuit born machine. In particular, we consider training a quantum circuit born machine using f-divergences. We first discuss the adversarial framework for generative modelling, which enables the estimation of any f-divergence in the near term. Based on this capability, we introduce two heuristics which demonstrably improve the training of the born machine. The first is based on f-divergence switching during training. The second introduces locality to the divergence, a strategy which has proved important in similar applications in terms of mitigating barren plateaus. Finally, we discuss the long-term implications of quantum devices for computing f-divergences, including algorithms which provide quadratic speedups to their estimation. In particular, we generalise existing algorithms for estimating the Kullback–Leibler divergence and the total variation distance to obtain a fault-tolerant quantum algorithm for estimating another f-divergence, namely, the Pearson divergence.

1. Introduction

One of the most challenging technological questions of our time is whether existing quantum computers can achieve quantum advantage in tasks of practical interest. Variational quantum algorithms (VQAs), which are well suited to the constraints imposed by existing devices, have emerged as the leading strategy for achieving such a quantum advantage [1,2,3,4].

In VQAs, a problem-specific cost function, which typically consists of a functional of the output of a parameterised quantum circuit, is efficiently evaluated using a quantum computer. Meanwhile, a classical optimiser is leveraged to train the circuit parameters in order to minimise the cost function. This hybrid quantum-classical approach is robust to the limited connectivity and qubit count of existing devices, and, by restricting the circuit depth, also provides an effective strategy for error mitigation.

Given their flexibility, VQAs have been proposed for a vast array of applications. Of particular relevance are applications of VQAs to machine learning problems, including classification [5,6,7,8,9,10], data compression [11,12,13], clustering [14], generative modelling [15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32], and inference [33].

In this paper, we focus on a hybrid quantum-classical approach to generative modelling using a born machine [34]. We adopt an adversarial framework to this task, in which a born machine (the ‘generator’) generates samples from the target distribution, while a binary classifier (the ‘discriminator’) attempts to distinguish between generated samples and true samples. This is sometimes referred to in the literature as a quantum generative adversarial network.

In a generalisation of existing approaches, we consider training the born machine with respect to any f-divergence as a cost function. Well-known examples of f-divergences include the Kullback–Leibler divergence (KL), the Jensen–Shannon divergence (JS), the squared Hellinger distance (${\mathrm{H}}^2$), the total variation distance (TV), and the Pearson divergence (${\chi }^2$). In the adversarial framework, it is straightforward to estimate the f-divergence: any such divergence is defined in terms of the density ratio of the target distribution and model distribution, which can be estimated using standard techniques via the output of the binary classifier [35]. On this basis, we propose a heuristic for training the born machine, based on the idea of dynamically switching the f-divergence during training in order to optimise the rate of convergence and utilise favourable qualities of each one. We also propose a second heuristic, based on introducing locality into the f-divergence, motivated by the now well-established connection between locality and barren plateaus in VQA training landscapes [36,37]. For both heuristics, we provide numerical evidence to suggest that they can lead to (sometimes significant) performance improvements, particularly in under- and over-parameterised circuits.

We conclude this paper with a discussion of the longer-term implication of quantum devices for computing the f-divergences between two probability distributions. In particular, we discuss the existence of quadratic speedups for the estimation of TV and KL shown by [38,39,40] and extend these results to an algorithm for estimating ${\chi }^2$, assuming access to a fault-tolerant quantum computer.

The remainder of this paper is organised as follows. In Section 2, we begin by introducing generative modelling, Born machines, and f-divergences. In Section 3, we then introduce the two training heuristics for the born machine. In Section 4, we provide numerical results to demonstrate the performance of the heuristics. In Section 5, we discuss the long-term implications of quantum devices for computing f-divergences. Finally, in Section 6, we offer some concluding remarks.

2. Background

2.1. Generative Modelling

Generative modelling is an unsupervised machine learning task in which the goal is to learn the probability distribution which generates a given data set. More precisely, given access to i.i.d. samples ${\mathit{x}}_1,\dots ,{\mathit{x}}_m\stackrel{\mathrm{i}.\mathrm{i}.\mathrm{d}.}{\sim }p\left(\mathit{x}\right)$ in ${\mathbb{R}}^p$, the objective of generative modelling is to learn a model $q_{\mathbf{\theta }}\left(\mathit{x}\right)$, typically parameterised by a d dimensional parameter vector, $\mathbf{\theta }\in {\mathbb{R}}^d$, which closely resembles $p\left(\mathit{x}\right)$. Generative models find applications in a wide range of problems, ranging from the typical modalities of machine learning such as text [41], image [42] and graph [43] analysis, to problems in active learning [44], reinforcement learning [45], medical imaging [46], physics [47], and speech synthesis [48].

Broadly speaking, one can distinguish between two main categories of generative model: prescribed models and implicit models [49,50]. Prescribed models provide an explicit parametric specification of the distribution of the observed random variable $\mathit{x}$, directly specifying the density $q_{\mathbf{\theta }}\left(\mathit{x}\right)$. An example of a prescribed model is the ubiquitous multivariate Gaussian distribution. Implicit models, on the other hand, specify only the stochastic procedure which generates samples. An example of an implicit model is a complex computer simulation of some physical phenomenon, for which the likelihood function cannot be computed. Since, in this case, one no longer models $q_{\mathbf{\theta }}\left(\mathit{x}\right)$ directly, valid objectives can now only involve quantities (e.g., expectation values) which can be estimated efficiently using samples.

In the last three decades, a number of generative models, both explicit and implicit, have been proposed in the machine learning literature. These include autoregressive models [51,52], normalising flows [53,54,55], variational autoencoders [56,57], Boltzmann machines [58,59,60], generative stochastic networks [61], generative moment matching networks [62,63], and generative adversarial networks [64]. These models are classically implemented using deep neural network architectures. In recent years, however, hybrid quantum-classical approaches based on parameterised quantum circuits have also gained traction [15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32].

2.2. Born Machines as Implicit Generative Models

By directly exploiting born’s probabilistic interpretation of quantum wave functions [65], it is possible to model the probability distribution of classical data using a pure quantum state. Such models are referred to as born machines [34]. We are particularly interested in born machines for which the quantum state is obtained via a parameterised quantum circuit (as opposed to, say, a continuous time Hamiltonian evolution). These are known as quantum circuit born machines (QCBMs) [15,16].

The use of QCBMs as generative models is in large part motivated by their expressiveness. Indeed, it is now well established that born machines have greater expressive power than classical models, including neural networks [20] and partially matrix product states [66] (see also [19]). This means, in particular, that QCBMs can efficiently represent certain distributions which are classically intractable to simulate (e.g., [67,68,69]). These include those recently used in a demonstration of quantum supremacy [70].

Let us consider a binary vector $\mathit{x}\in {\{0,1\}}^n$, with n the number of qubits. A QCBM takes a product state ${|0\rangle }^{\otimes n}$ as input and evolves it into a normalised output state $|\Psi (\mathbf{\theta })\rangle$ via a parameterised quantum circuit $U\left(\mathbf{\theta }\right)$. One can generate n-bit strings according to

$$\mathit{x}\sim q_{\mathbf{\theta }}\left(\mathit{x}\right)={\left|\langle \mathit{x}|\Psi \left(\mathbf{\theta }\right)\rangle \right|}^2,$$

(1)

where $|\mathit{x}\rangle$ are computational basis states; sampling from this distribution then consists of a simple measurement. Since we only have access to $\mathit{x}\sim q_{\mathbf{\theta }}\left(\mathit{x}\right)$ and not the probabilities, $q_{\mathbf{\theta }}\left(\mathit{x}\right)$ themselves, the born machine can be regarded as an implicit generative model. We consider parameterised quantum circuits $U\left(\mathbf{\theta }\right)$ of the form

$$U\left(\mathbf{\theta }\right)=\prod _{i=1}^D{W}_i{U}_i\left({\theta }_i\right),$$

(2)

where ${\left\{W_i\right\}}_{i=1}^D$ is a set of fixed unitaries, ${\left\{U_i\left({\theta }_i\right)\right\}}_{i=1}^D$ is a set of parameterised unitaries, and D is the depth of circuit. We also assume that $U_i\left({\theta }_i\right)=e^{-i{\theta }_i{V}_i}$ are rotations through angles ${\theta }_i$, generated by Hermitian operators $V_i$ with eigenvalues $\pm 1$. In this case, one can compute partial derivatives of $q_{\mathbf{\theta }}\left(\mathit{x}\right)$ using the parameter-shift rule [71], which reads

$${\partial }_{{\theta }_i}{q}_{\mathbf{\theta }}\left(\mathit{x}\right)=q_{{\mathbf{\theta }}_i^{+}}\left(\mathit{x}\right)-q_{{\mathbf{\theta }}_i^{-}}\left(\mathit{x}\right),$$

(3)

where ${\mathbf{\theta }}_{i^{\pm }}=\mathbf{\theta }\pm \frac{\pi }{4}{\mathit{e}}_i$, with ${\mathit{e}}_i$ a unit vector in the ith direction. More generally, this formula allows one to express the first-order partial derivative of an expectation of a function h as

$${\partial }_{{\mathbf{\theta }}_i}{\mathbb{E}}_{\mathit{x}\sim q_{\mathbf{\theta }}\left(\mathit{x}\right)}\left[h\left(\mathit{x}\right)\right]={\mathbb{E}}_{\mathit{x}\sim q_{{\mathbf{\theta }}_i^{+}}\left(\mathit{x}\right)}\left[h\left(\mathit{x}\right)\right]-{\mathbb{E}}_{\mathit{x}\sim q_{{\mathbf{\theta }}_i^{+}}\left(\mathit{x}\right)}\left[h\left(\mathit{x}\right)\right].$$

(4)

The major challenge in using any implicit generative model is designing a suitable objective function. As noted before, one cannot compute $q_{\mathbf{\theta }}\left(\mathit{x}\right)$ directly, and thus valid objectives can only involve statistical quantities (e.g., expectations) which can be efficiently computed using samples. For generative models based on QCBMs, various objectives have been proposed, including moment-matching, maximum mean discrepancy, Stein and Sinkhorn divergences, and adversarial objectives based on the Kullback–Leibler divergence. In this paper, we propose a more general class of objective functions—f-divergences—for training QCBMs.

2.3. Adversarial Generative Modelling with f-Divergences

Let $f:(0,\infty )\to \mathbb{R}$ be a convex function with $f\left(1\right)=0$ and strict convexity at 1. Suppose that $p\left(\mathit{x}\right)=0$ whenever $q_{\mathbf{\theta }}\left(\mathit{x}\right)=0$. The f-divergence, or Csiszár divergence [72,73], between $q_{\mathbf{\theta }}$ and p is defined as

$$D_f(p\parallel q_{\mathbf{\theta }})={\mathbb{E}}_{\mathit{x}\sim q_{\mathbf{\theta }}\left(\mathit{x}\right)}\left[f\left(\frac{p\left(\mathit{x}\right)}{q_{\mathbf{\theta }}\left(\mathit{x}\right)}\right)\right].$$

(5)

Suppose instead that $q_{\mathbf{\theta }}\left(\mathit{x}\right)=0$ whenever $p\left(\mathit{x}\right)=0$. Then the f-divergence can be written in terms of $f^{*}$, the convex conjugate of f, as

$$D_f(p\parallel q_{\mathbf{\theta }})={\mathbb{E}}_{\mathit{x}\sim p\left(\mathit{x}\right)}\left[f^{*}\left(\frac{q_{\mathbf{\theta }}\left(\mathit{x}\right)}{p\left(\mathit{x}\right)}\right)\right].$$

(6)

In what follows, we will generally prefer this formulation, as it leads to simpler expressions.

The function f is called the generator of the divergence. For different choices of f, one obtains well-known divergences such as TV, KL, and ${\chi }^2$. In this paper, we investigate the effect of this choice on the training of a QCBM. To ensure a fair comparison, we assume that the generators are standardised and normalised such that $f^{\prime }\left(1\right)=0$ and $f^{″}\left(1\right)=1$ [74]. This ensures that $D_f(p\parallel q_{\mathbf{\theta }})\ge 0$ with equality if and only if $p\equiv q_{\mathbf{\theta }}$, even if p and $q_{\mathbf{\theta }}$ are unnormalised. Note that one can normalise and standardise

We minimise the f-divergence using gradient-based methods. We thus require the derivative of $D_f$ with respect to ${\mathbf{\theta }}_i$. Using the chain and the parameter-shift rules, it is straightforward to compute

$$\begin{array}{ccc}\hfill {\partial }_{{\mathbf{\theta }}_i}{D}_f(p\parallel q_{\mathbf{\theta }})& =& \sum _{\mathit{x}}p\left(\mathit{x}\right){\partial }_{{\theta }_i}{f}^{*}\left(\frac{q_{\mathbf{\theta }}\left(\mathit{x}\right)}{p\left(\mathit{x}\right)}\right)\hfill \end{array}$$

(7)

$$\begin{array}{ccc}& =& \sum _{\mathit{x}}p\left(\mathit{x}\right)f^{*\prime }\left(\frac{q_{\mathbf{\theta }}\left(\mathit{x}\right)}{p\left(\mathit{x}\right)}\right)\frac{1}{p\left(\mathit{x}\right)}{\partial }_{{\theta }_i}{q}_{\mathbf{\theta }}\left(\mathit{x}\right)\hfill \end{array}$$

(8)

$$\begin{array}{ccc}& =& \sum _{\mathit{x}}{f}^{*\prime }\left(\frac{q_{\mathbf{\theta }}\left(\mathit{x}\right)}{p\left(\mathit{x}\right)}\right)\left(q_{{\mathbf{\theta }}_i^{+}}\left(\mathit{x}\right)-q_{{\mathbf{\theta }}_i^{-}}\left(\mathit{x}\right)\right)\hfill \end{array}$$

(9)

$$\begin{array}{ccc}& =& {\mathbb{E}}_{\mathit{x}\sim q_{{\mathbf{\theta }}_i^{+}}\left(\mathit{x}\right)}\left[f^{*\prime }\left(\frac{q_{\mathbf{\theta }}\left(\mathit{x}\right)}{p\left(\mathit{x}\right)}\right)\right]-{\mathbb{E}}_{\mathit{x}\sim q_{{\mathbf{\theta }}_i^{-}}\left(\mathit{x}\right)}\left[f^{*\prime }\left(\frac{q_{\mathbf{\theta }}\left(\mathit{x}\right)}{p\left(\mathit{x}\right)}\right)\right].\hfill \end{array}$$

(10)

We summarise some well-known f-divergences, the convex conjugates of their generators, and their parameter-shift rules, in

Table 1. A summary of well-known f-divergences, including the definition, the convex conjugate of the generator $f^{*}$, and the corresponding parameter-shift rule in terms of the ratio $r\left(\mathit{x}\right)=\frac{q_{\mathbf{\theta }}\left(\mathit{x}\right)}{p\left(\mathit{x}\right)}$. The $\parallel$ symbol indicates that the divergence is asymmetric, while a comma indicates that it is symmetric. Interestingly, one can construct symmetric f-divergences for every asymmetric one (see Table 2).

f-Divergence	Definition	${\mathit{f}}^{*}$	Parameter-Shift
total variation	$\mathrm{TV}(p,q_{\mathbf{\theta }})=\frac{1}{2}\sum |p\left(\mathit{x}\right)-q_{\mathbf{\theta }}\left(\mathit{x}\right)|$	$\frac{1}{2}|r-1|$	$\frac{1}{2}{\mathbb{E}}_{q_{{\mathbf{\theta }}^{+}}}\left[\mathrm{sgn}\left(r\right(\mathit{x})-1)\right]-\frac{1}{2}{\mathbb{E}}_{q_{{\mathbf{\theta }}^{-}}}\left[\mathrm{sgn}\left(r\right(\mathit{x})-1)\right]$
squared Hellinger	${\mathrm{H}}^2(p,q_{\mathbf{\theta }})=\sum {(\sqrt{p\left(\mathit{x}\right)}-\sqrt{q_{\mathbf{\theta }}\left(\mathit{x}\right)})}^2$	$2{(\sqrt{r}-1)}^2$	$-2{\mathbb{E}}_{q_{{\mathbf{\theta }}^{+}}}\left[\frac{1}{\sqrt{r\left(\mathit{x}\right)}}\right]+2{\mathbb{E}}_{q_{{\mathbf{\theta }}^{-}}}\left[\frac{1}{\sqrt{r\left(\mathit{x}\right)}}\right]$
Kullback–Leibler (type I, forward)	$\mathrm{KL}(p\parallel q_{\mathbf{\theta }})={\mathbb{E}}_p\left[log\frac{p\left(\mathit{x}\right)}{q_{\mathbf{\theta }}\left(\mathit{x}\right)}\right]$	$-logr+r-1$	$-{\mathbb{E}}_{q_{{\mathbf{\theta }}^{+}}}\left[\frac{1}{r\left(\mathit{x}\right)}\right]+{\mathbb{E}}_{q_{{\mathbf{\theta }}^{-}}}\left[\frac{1}{r\left(\mathit{x}\right)}\right]$
Kullback–Leibler (type I, reverse)	$\mathrm{KL}(q_{\mathbf{\theta }}\parallel p)={\mathbb{E}}_{q_{\mathbf{\theta }}}\left[log\frac{q_{\mathbf{\theta }}\left(\mathit{x}\right)}{p\left(\mathit{x}\right)}\right]$	$rlogr-r+1$	${\mathbb{E}}_{q_{{\mathbf{\theta }}^{+}}}\left[logr\left(\mathit{x}\right)\right]-{\mathbb{E}}_{q_{{\mathbf{\theta }}^{-}}}\left[logr\left(\mathit{x}\right)\right]$
Kullback–Leibler (type II, forward)	$\mathrm{KL}\left(p\right|\frac{p+q_{\mathbf{\theta }}}{2})={\mathbb{E}}_p\left[log\frac{2p\left(\mathit{x}\right)}{p\left(\mathit{x}\right)+q_{\mathbf{\theta }}\left(\mathit{x}\right)}\right]$	$4log\frac{2}{r+1}+2(r-1)$	$-4{\mathbb{E}}_{q_{{\mathbf{\theta }}^{+}}}\left[\frac{1}{r\left(\mathit{x}\right)+1}\right]+4{\mathbb{E}}_{q_{{\mathbf{\theta }}^{-}}}\left[\frac{1}{r\left(\mathit{x}\right)+1}\right]$
Kullback–Leibler (type II, reverse)	$\mathrm{KL}(q_{\mathbf{\theta }}\parallel \frac{p+q_{\mathbf{\theta }}}{2})={\mathbb{E}}_{q_{\mathbf{\theta }}}\left[log\frac{2q_{\mathbf{\theta }}\left(\mathit{x}\right)}{p\left(\mathit{x}\right)+q_{\mathbf{\theta }}\left(\mathit{x}\right)}\right]$	$4rlog\frac{2r}{r+1}+2(1-r)$	$4{\mathbb{E}}_{q_{{\mathbf{\theta }}^{+}}}\left[log\frac{r\left(\mathit{x}\right)}{r\left(\mathit{x}\right)+1}+\frac{1}{r\left(\mathit{x}\right)+1}\right]$             $-4{\mathbb{E}}_{q_{{\mathbf{\theta }}^{-}}}\left[log\frac{r\left(\mathit{x}\right)}{r\left(\mathit{x}\right)+1}+\frac{1}{r\left(\mathit{x}\right)+1}\right]$
Pearson (forward)	${\chi }^2(p\parallel q_{\mathbf{\theta }})=\sum \frac{{(p\left(\mathit{x}\right)-q_{\mathbf{\theta }}\left(\mathit{x}\right))}^2}{p\left(\mathit{x}\right)}$	$\frac{{(r-1)}^2}{2}$	${\mathbb{E}}_{q_{{\mathbf{\theta }}^{+}}}\left[r\left(\mathit{x}\right)\right]-{\mathbb{E}}_{q_{{\mathbf{\theta }}^{-}}}\left[r\left(\mathit{x}\right)\right]$
Pearson (reverse)	${\chi }^2(q_{\mathbf{\theta }}\parallel p)=\sum \frac{{(p\left(\mathit{x}\right)-q_{\mathbf{\theta }}\left(\mathit{x}\right))}^2}{q_{\mathbf{\theta }}\left(\mathit{x}\right)}$	$\frac{{(r-1)}^2}{2r}$	$-\frac{1}{2}{\mathbb{E}}_{q_{{\mathbf{\theta }}^{+}}}\left[\frac{1}{r{\left(\mathit{x}\right)}^2}\right]+\frac{1}{2}{\mathbb{E}}_{q_{{\mathbf{\theta }}^{-}}}\left[\frac{1}{r{\left(\mathit{x}\right)}^2}\right]$

and

Table 2. A summary of the symmetric f-divergences corresponding to some well-known asymmetric f-divergences, including the definition, and the parameter-shift rule.

f-Divergence	Definition	Parameter-Shift
symmetric Kullback–Leibler (type I, Jeffrey)	$\mathrm{J}(p,q_{\mathbf{\theta }})=\mathrm{KL}(p\parallel q_{\mathbf{\theta }})+\mathrm{KL}(q_{\mathbf{\theta }}\parallel p)$	$\frac{1}{2}{\mathbb{E}}_{q_{{\mathbf{\theta }}^{+}}}\left[logr\left(\mathit{x}\right)-\frac{1}{r\left(\mathit{x}\right)}\right]-\frac{1}{2}{\mathbb{E}}_{q_{{\mathbf{\theta }}^{-}}}\left[logr\left(\mathit{x}\right)-\frac{1}{r\left(\mathit{x}\right)}\right]$
symmetric Kullback–Leibler (type II, Jensen–Shannon)	$\mathrm{JS}(p,q_{\mathbf{\theta }})=\mathrm{KL}(p\parallel \frac{p+q_{\mathbf{\theta }}}{2})+\mathrm{KL}(q_{\mathbf{\theta }}\parallel \frac{p+q_{\mathbf{\theta }}}{2})$	$2{\mathbb{E}}_{q_{{\mathbf{\theta }}^{+}}}\left[log\frac{r\left(\mathit{x}\right)}{1+r\left(\mathit{x}\right)}\right]-2{\mathbb{E}}_{q_{{\mathbf{\theta }}^{-}}}\left[log\frac{r\left(\mathit{x}\right)}{1+r\left(\mathit{x}\right)}\right]$
symmetric Pearson	${\overline{\chi }}^2(p,q_{\mathbf{\theta }})={\chi }^2(p\parallel q_{\mathbf{\theta }})+{\chi }^2(q_{\mathbf{\theta }}\parallel p)$	$\frac{1}{4}{\mathbb{E}}_{q_{{\mathbf{\theta }}^{+}}}\left[2r\left(\mathit{x}\right)-\frac{1}{r{\left(\mathit{x}\right)}^2}\right]-\frac{1}{4}{\mathbb{E}}_{q_{{\mathbf{\theta }}^{-}}}\left[2r\left(\mathit{x}\right)-\frac{1}{r{\left(\mathit{x}\right)}^2}\right]$

. We also plot some of the convex conjugate generators in Figure 1.

Returning to Equation (10), it is clear that the problem of computing the gradient reduces to that of estimating the probability ratio $r\left(\mathit{x}\right)=\frac{q_{\mathbf{\theta }}\left(\mathit{x}\right)}{p\left(\mathit{x}\right)}$. We choose to define $r\left(\mathit{x}\right)$ in this way since it is more natural when one is interested with writing the f-divergence in terms of $f^{*}$, as we do here. Note that in some literature the ratio is defined in the reverse manner by switching the probabilities. We can estimate the probability ratio from the output of a binary classifier [35]. Suppose we assign samples $\mathit{x}$∼$q_{\mathbf{\theta }}\left(\mathit{x}\right)$ to one class, and samples $\mathit{x}$∼$p\left(\mathit{x}\right)$ to another class. Suppose, in addition, that one has access to an exact binary classifier $d_{*}\left(\mathit{x}\right)$, which outputs the probability that the sample $\mathit{x}$ originated from $q_{\mathbf{\theta }}\left(\mathit{x}\right)$. Then, assuming uniform prior probabilities for the two classes, it is straightforward to show via Bayes’ theorem that (see Section 2.2 in [50]).

$$\begin{array}{c}\hfill r\left(\mathit{x}\right)=\frac{d_{*}\left(\mathit{x}\right)}{1-d_{*}\left(\mathit{x}\right)}.\end{array}$$

(11)

In practice, we do not have access to the exact classifier $d_{*}\left(\mathit{x}\right)$. However, under the assumption that we can efficiently sample from both distributions, we can train a classifier $d_{\mathbf{\varphi }}\left(\mathit{x}\right)$, parameterised by $\mathbf{\varphi }$, to distinguish between the two distributions. One can use any proper scoring rule to train the classifier [50]. A typical choice is the negative cross entropy, given by

$$\mathcal{L}(\mathbf{\varphi };\mathbf{\theta })=-{\mathbb{E}}_{\mathit{x}\sim q_{\mathbf{\theta }}\left(\mathit{x}\right)}\left[log{d}_{\mathbf{\varphi }}\left(\mathit{x}\right)\right]-{\mathbb{E}}_{\mathit{x}\sim p\left(\mathit{x}\right)}\left[log(1-d_{\mathbf{\varphi }}\left(\mathit{x}\right))\right].$$

(12)

The classifier seeks to minimise this objective, which corresponds to low classification errors. We emphasise that, in this objective, $\mathbf{\theta }$ is fixed at the current QCBM parameters. The resulting classifier approximates the probability ratio for the current QCBM as

$$r\left(\mathit{x}\right)\approx \frac{d_{\mathbf{\varphi }}\left(\mathit{x}\right)}{1-d_{\mathbf{\varphi }}\left(\mathit{x}\right)}.$$

(13)

This can be plugged into Equation (10) to approximate the gradient. With this in mind, we define the cost function for the QCBM as

$$\mathcal{J}(\mathbf{\theta };\mathbf{\varphi })={\mathbb{E}}_{\mathit{x}\sim q_{\mathbf{\theta }}\left(\mathit{x}\right)}\left[f^{*\prime }\left(\frac{d_{\mathbf{\varphi }}\left(\mathit{x}\right)}{1-d_{\mathbf{\varphi }}\left(\mathit{x}\right)}\right)\right],$$

(14)

where now the parameters of the classifier $\mathbf{\varphi }$ are fixed and the argument of the expectation value is independent of $\mathbf{\theta }$. The adversarial generative modelling can be regarded as the following optimisation problem

$$\begin{array}{ccc}\hfill {\mathbf{\theta }}^{*}& =& \underset{\mathbf{\theta }}{arg\; min}\mathcal{J}(\mathbf{\theta };\mathbf{\varphi }),\hfill \end{array}$$

(15)

$$\begin{array}{ccc}\hfill {\mathbf{\varphi }}^{*}& =& \underset{\mathbf{\varphi }}{arg\; min}\mathcal{L}(\mathbf{\varphi };\mathbf{\theta }),\hfill \end{array}$$

(16)

where the required expectation values are estimated from samples. In principle, the classifier can be trained to optimality in order to provide the best possible ratio for the generative model. Alternatively, the two objective functions can be optimised in tandem, using alternating gradient descent steps or a two-timescale gradient descent scheme [75].

3. Training Heuristics

3.1. Switching f-Divergences

In this Section, we describe a heuristic for dynamically switching between f-divergences throughout the training process of our generative model (specifically the QCBM).

To motivate this heuristic, we examine how $D_f(p\parallel q_{\mathbf{\theta }})$ varies with respect to values of $r\left(\mathit{x}\right)=\frac{q_{\mathbf{\theta }}\left(\mathit{x}\right)}{p\left(\mathit{x}\right)}$. We begin by noting that all f-divergences which can be standardised agree on the divergence between nearby distributions [76], but can otherwise exhibit very different behaviours. In particular, we focus on their initial rates of convergence.

One may rationalise the different rates of convergence for each divergence at the beginning of training by considering the following argument [50,64,77]. Consider n qubits, such that there are $2^n$ different values of $r\left(\mathit{x}\right)$. For a successful training, all these values need to converge towards 1 (which implies our goal that $q_{\mathbf{\theta }}\equiv p$). Now suppose we were to estimate the divergence in Equation (6) using a set of samples from the target distribution ${\mathit{x}}_1,\dots ,{\mathit{x}}_m\stackrel{\mathrm{i}.\mathrm{i}.\mathrm{d}.}{\sim }p\left(\mathit{x}\right)$. At the beginning of training, $q_{\mathbf{\theta }}$ is initialised at random and is therefore expected to be far from the target. This means that $q_{\mathbf{\theta }}\left({\mathit{x}}_i\right)\ll p\left({\mathit{x}}_i\right)$ for most of the samples. In other words, at the beginning of training most of the samples yield probability ratios $r\left({\mathit{x}}_i\right)\ll 1$.

It is evident from the left panel of Figure 1 that some divergences, including TV, vary slowly in the region where $r\ll 1$, and are therefore more liable to saturation in the initial stages of training. Other divergences, such as forward KL and reverse ${\chi }^2$, generate strong gradients in this region. In the limiting case where p and $q_{\mathbf{\theta }}$ have disjoint supports, TV and JS saturate, whereas forward KL diverges [78]. This problem is well known within the context of training generative adversarial networks; since an idealised formulation optimises JS, several alternative cost functions have been proposed to mitigate its slow initial convergence [64,77,78,79].

Though we can only apply this logic to the particular regime where p and $q_{\mathbf{\theta }}$ are far apart, it is also evident from Figure 1 that the f-divergences exhibit a wide diversity of behaviours throughout most of training. We propose to exploit this with the following heuristic. At every optimisation step, we choose an f-divergence for each direction in parameter space that generates the highest gradient in said direction. This requires no additional quantum circuit evaluations since we only need to evaluate Equation (10) for the different generators. Concretely, the heuristic can be written as follows. For each step, to update parameter ${\mathbf{\theta }}_i$, we choose the f-divergence labelled j, $D_{f_j}$, which obeys:

$$\left|{\partial }_{{\mathbf{\theta }}_i}{D}_{f_j}\right|>\left|{\partial }_{{\mathbf{\theta }}_i}{D}_{f_k}\right|\forall k\in \mathcal{F}.$$

(17)

For simplicity, in this paper, we restrict the set $\mathcal{F}$ to only contain those f-divergences illustrated in Figure 1. We call this heuristic f-switch.

3.2. Local Cost Functions

In this Section, we outline an alternative heuristic for training the QCBM, based on introducing locality into the cost function. Let us briefly provide some motivation for this approach. One of the most fundamental challenges associated with hybrid quantum-classical algorithms is the barren plateau phenomenon, whereby the gradient of the cost function vanishes exponentially in the number of qubits [36,37,80,81,82,83,84,85,86,87,88]. This phenomenon can arise due to deep unstructured ansätze [80], large entanglement [83,84], high levels of noise [88], and global cost functions [36,37]. As such, it is a rather general phenomenon in many quantum machine learning applications, including generative models. In the presence of barren plateaus, exponential precision (i.e., an exponential number of samples) is required in order to resolve against finite sampling noise and determine a minimising direction in the cost function landscape. Since the standard objective of quantum algorithms is to achieve a polynomial scaling in the system size (as opposed to the exponential scaling of classical algorithms), barren plateaus can destroy any hope of a variational quantum algorithm achieving quantum advantage.

Although, in this paper, we do not directly analyse the emergence of barren plateaus in the QCBM, we are nonetheless motivated by existing results on barren plateaus. We focus, in particular, on the connection between barren plateaus and global cost functions (i.e., cost functions defined in terms of global observables), given that such cost functions naturally arise in hybrid quantum-classical generative models. The connection between trainability and locality was first established by Cerezo et al. [36], who proved that cost functions defined in terms of global observables exhibit barren plateaus for all circuit depths in circuits composed of random two-qubit gates which act on alternating pairs of qubits (i.e., blocks forming local 2-designs). Meanwhile, local cost functions do not exhibit barren plateaus for shallow circuits; in this case, cost function gradients vanish at worst polynomially in the number of qubits.

On the basis of this result, there is clear motivation to seek a local cost function (i.e., a cost function defined in terms of local observables) for the hybrid quantum-classical generative model introduced in Section 2.3. We now attempt to make some progress towards this goal.

We write $q_{\mathbf{\theta }}^i\left({\mathit{x}}_i\right)$ to denote the marginal distribution of the $i^{\mathrm{th}}$ element of the bit-string $\mathit{x}=({\mathit{x}}_1,\cdots ,{\mathit{x}}_n)$. Using Jensen’s inequality on Equation (6), it can be shown that the f-divergence between joint distributions is larger than the f-divergence between marginal distributions. Thus, we have

$$D_f(p\left(\mathit{x}\right)\parallel q_{\mathbf{\theta }}\left(\mathit{x}\right))\ge \frac{1}{n}\sum _{i=1}^n{D}_f(p^i\left({\mathit{x}}_i\right)\parallel q_{\mathbf{\theta }}^i\left({\mathit{x}}_i\right)).$$

(18)

Our heuristic consists of minimising the right-hand side of this inequality. Even though this is a lower-bound to the original cost, it is a fully local cost function. Later, we show how to generalise this approach allowing for a trade off between trainability and accuracy. We call this heuristic f-local.

Let us show the difference between the global cost function (left-hand side of the inequality) and the local cost function (right-hand side) by means of an example. For ease of exposition, we assume in this discussion that the f-divergence of interest is the reverse KL with generator $f^{*}\left(r\right)=rlogr-r+1$. We emphasise, however, that the methodology is generic to any f-divergence. We begin by rewriting the expression in Equation (1) as

$$q_{\mathbf{\theta }}\left(\mathit{x}\right)=\langle \mathbf{0}|U^{†}\left(\mathbf{\theta }\right)H_{\mathit{x}}U\left(\mathbf{\theta }\right)|\mathbf{0}\rangle ,$$

(19)

where we have defined $H_{\mathit{x}}:=|\mathit{x}\rangle \langle \mathit{x}|$. We can thus write the reverse KL in the form of a generic cost function (see, e.g., [3]) as

$$\begin{array}{c}\hfill \mathrm{KL}(q_{\mathbf{\theta }}\parallel p)=\sum _{\mathit{x}}{q}_{\mathbf{\theta }}\left(\mathit{x}\right)log\frac{q_{\mathbf{\theta }}\left(\mathit{x}\right)}{p\left(\mathit{x}\right)}=\sum _{\mathit{x}}{g}_{\mathit{x}}\left(\langle \mathbf{0}\left|U^{†}\left(\mathbf{\theta }\right)H_{\mathit{x}}U\left(\mathbf{\theta }\right)\right|\mathbf{0}\rangle \right),\end{array}$$

(20)

where we define $g_{\mathit{x}}\left(q_{\mathbf{\theta }}\right):=q_{\mathbf{\theta }}log\frac{q_{\mathbf{\theta }}}{p\left(\mathit{x}\right)}$. This cost function is clearly global, since the observables, $H_{\mathit{x}}$, act on all qubits.

Now, rewriting Equation (20) in terms of the adversarial approximation in Equation (14), we have

$$\begin{array}{c}\hfill \mathcal{J}(\mathbf{\theta };\mathbf{\varphi })=\sum _{\mathit{x}}{q}_{\mathbf{\theta }}\left(\mathit{x}\right)logit\left(d_{\mathbf{\varphi }}\left(\mathit{x}\right)\right)=\sum _{\mathit{x}}{h}_{\mathit{x}}\left(\langle \mathbf{0}\left|U^{†}\left(\mathbf{\theta }\right)H_{\mathit{x}}U\left(\mathbf{\theta }\right)\right|\mathbf{0}\rangle \right),\end{array}$$

(21)

where $h_{\mathit{x}}\left(q_{\mathbf{\theta }}\right):=q_{\mathbf{\theta }}logit\left(d_{\mathbf{\varphi }}\left(\mathit{x}\right)\right)$, and $logit\left(y\right):=log\frac{y}{1-y}$. It is interesting to note that the global observable $H_{\mathit{x}}$ only enters into $h_{\mathit{x}}\left(q_{\mathbf{\theta }}\right)$ via the first term, namely $q_{\mathbf{\theta }}\left(\mathit{x}\right)$. It is arguable, however, that the second term in $h_{\mathit{x}}\left(q_{\mathbf{\theta }}\right)$, namely $logit\left(d_{\mathbf{\varphi }}\left(\mathit{x}\right)\right)$ should also be regarded as a global quantity.

We now consider the fully local cost function in the right-hand side of Equation (18). Applying the adversarial approximation to each of the n probability ratios, the QCBM objective is

$$\begin{array}{c}\hfill {\mathcal{J}}^L(\mathbf{\theta };\mathbf{\varphi })=\frac{1}{n}\sum _{\phantom{{\mathit{x}}_i\in \{0,1\}}i=1}^n\sum _{{\mathit{x}}_i\in \{0,1\}}^{\phantom{n}}{q}_{\mathbf{\theta }}^i\left({\mathit{x}}_i\right)logit\left(d_{\mathbf{\varphi }}^i\left({\mathit{x}}_i\right)\right)=\frac{1}{n}\sum _{\phantom{{\mathit{x}}_i\in \{0,1\}}i=1}^n\sum _{{\mathit{x}}_i\in \{0,1\}}{h}_{{\mathit{x}}_i}^L\left(\langle \mathbf{0}\left|U^{†}\left(\mathbf{\theta }\right)H_{{\mathit{x}}_i}^{L}U\left(\mathbf{\theta }\right)\right|\mathbf{0}\rangle \right),\end{array}$$

(22)

where we have replaced the global observable $H_{\mathit{x}}$ in Equation (21) by the set of local observables

$$H_{{\mathit{x}}_i}^L={|\mathit{x}\rangle \langle \mathit{x}|}_i\otimes {𝟙}_{\tilde{i}}.$$

(23)

Here, ${|\mathit{x}\rangle \langle \mathit{x}|}_i$ is a projector on the computational basis for qubit i, and ${𝟙}_{\tilde{i}}$ denotes the identity on all qubits except qubit i. We have also replaced the ‘global’ function $h_{\mathit{x}}\left(q_{\mathbf{\theta }}\right)$ in Equation (21) by the set of local functions

$$h_{{\mathit{x}}_i}^L\left(p_{\mathbf{\theta }}^i\right)=q_{\mathbf{\theta }}^{i}logit\left(d_{\mathbf{\varphi }}^i\left({\mathit{x}}_i\right)\right).$$

(24)

Here, ${\left\{d_{\mathbf{\varphi }}^i\left({\mathit{x}}_i\right)\right\}}_{i=1}^n$ is a set of n ‘local’ classifiers, which act only on the marginal distribution corresponding to the ith qubit. That is to say, $d_{\mathbf{\varphi }}^i$ are trained to distinguish between samples ${\mathit{x}}_i\sim q_{\mathbf{\theta }}^i\left({\mathit{x}}_i\right)$ and samples ${\mathit{x}}_i\sim p^i\left({\mathit{x}}_i\right)$. One may ask why it is not sufficient to simply make only the observable, $H_{\mathit{x}}$, local as is done in other literature addressing the barren plateau problem [36]. In our case, it turns out that if one does not also make the functions $h_{\mathit{x}}$ local, in other words by keeping the classifier ‘global’, the cost function becomes intractable to compute due to a need to explicitly compute joint probabilities from the circuit, $q_{\mathbf{\theta }}$. This hints at the subtlety that appears when attempting to address barren plateaus in generative modelling, that does not necessarily exist in other variational algorithms.

We are, of course, interested in whether the local cost function is faithful to the original cost function. Recall that we are minimising the lower bound in Equation (18). It is clear that, if the local cost function is minimised, so that $D_f(q_{\mathbf{\theta }}^i\parallel p^i)=0$ for all $i\in \{1,\cdots ,n\}$, and all of the marginals coincide, there is still no guarantee that the joint distributions will be identical. This observation suggests that, while this cost function may be more trainable than the original cost function on account of its locality, it may also be significantly less accurate. In an attempt to remedy this, we can instead consider a more general k-local cost function which acts on subsets of k qubits. In particular, by defining ${\mathit{x}}_{i:j}:=({\mathit{x}}_i,\cdots ,{\mathit{x}}_j)$, we can introduce

$$\begin{array}{cc}\hfill {\mathcal{J}}^{L\left(k\right)}(\mathbf{\theta };\mathbf{\varphi })& =\frac{1}{n-k+1}\sum _{\phantom{{\mathit{x}}_{i:i+k-1}\in {\{0,1\}}^k}i=1}^{n-k+1}\sum _{{\mathit{x}}_{i:i+k-1}\in {\{0,1\}}^k}^{\phantom{n}}{q}_{\mathbf{\theta }}^{i:i+k-1}\left({\mathit{x}}_{i:i+k-1}\right)logit\left(d_{\mathbf{\varphi }}^{i:i+k-1}\left({\mathit{x}}_{i:i+k-1}\right)\right)\hfill \end{array}$$

(25)

$$\begin{array}{cc}\hfill & =\frac{1}{n-k+1}\sum _{\phantom{{\mathit{x}}_{i:i+k-1}\in {\{0,1\}}^k}i=1}^{n-k+1}\sum _{{\mathit{x}}_{i:i+k-1}\in {\{0,1\}}^k}^{\phantom{n}}{h}_{{\mathit{x}}_{i:i+k-1}}^{L\left(k\right)}\left(\langle \mathbf{0}\left|U^{†}\left(\mathbf{\theta }\right)H_{{\mathit{x}}_{i:i+k-1}}^{L\left(k\right)}U\left(\mathbf{\theta }\right)\right|\mathbf{0}\rangle \right),\hfill \end{array}$$

(26)

where

$$\begin{array}{cc}\hfill H_{{\mathit{x}}_{i:i+k-1}}^{L\left(k\right)}& ={|\mathit{x}\rangle \langle \mathit{x}|}_{i:i+k-1}\otimes {𝟙}_{\widetilde{i:i+k-1}},\hfill \end{array}$$

(27)

$$\begin{array}{cc}\hfill h_{{\mathit{x}}_{i:i+k-1}}^{L\left(k\right)}\left(q_{\mathbf{\theta }}^{i:i+k-1}\right)& =q_{\mathbf{\theta }}^{i:i+k-1}logit\left(d_{\mathbf{\varphi }}^{i:i+k-1}\left({\mathit{x}}_{i:i+k-1}\right)\right),\hfill \end{array}$$

(28)

and where ${\left\{d_{\mathbf{\varphi }}^{i:i+k-1}\left({\mathit{x}}_{i:i+k-1}\right)\right\}}_{i=1}^{n-k+1}$ is a set of $n-k+1$ ‘k-local’ classifiers, defined in an obvious fashion. This k-local cost function now approximates the sum of the reverse KL between the k-marginals (of neighbouring qubits) of the target distribution $p\left(\mathit{x}\right)$, and the variational distribution $q_{\mathbf{\theta }}\left(\mathit{x}\right)$.

Arguing as before, it is clear that the k-local cost function will admit additional global minima in comparison to the global cost function for any $1\le k<n$. In particular, when the k-local cost function is minimised, the k-nearest neighbour marginals of $p\left(\mathit{x}\right)$ and $q_{\mathbf{\theta }}\left(\mathit{x}\right)$ coincide. One can expect, however, that as the value of k is increased, not only will the number of additional minima decrease, but the disparity between the joint distributions of the target and the model at these global minima will decrease. This suggests that in order to achieve a ‘sweet spot’ between trainability and accuracy, a reasonably approach is to start by optimising the k-local cost function with a small value of k (promoting trainability), before iteratively increasing the value of k (promoting accuracy) until $k=n$, thus recovering the global cost function.

We should remark that, while for ease of notation we have defined the k-local cost function in terms of marginals with respect to neighbouring qubits $({\mathit{x}}_i,\cdots ,{\mathit{x}}_{i+k-1})$, one can in theory choose any sets of qubits of size at most k (e.g., nearest neighbours, all possible combinations, and randomly sampled). In general, for a fixed value of k, this choice will influence the accuracy of the objective function, as well as its computational cost, and should be made on a case-by-case basis on the basis of the available computational resources.

4. Numerical Results

In this Section, we present numerical results to illustrate the performance of the training heuristics proposed in Section 3.

Preliminaries

Throughout this Section, we utilise a QCBM composed of alternating layers of single qubit gates and entangling gates (see Figure 2). We implement the quantum circuit using pytket [89] and execute the simulations with Qiskit [90]. The parameters of the QCBM are updated using stochastic gradient descent with a constant learning rate, which is tuned to each of the simulations.

Regarding the classical component of the adversarial generative model (i.e., the binary classifier), we use either a fully connected feed-forward neural network with ReLU neurons (NN), or a support vector machine with RBF kernel (SVM). Indeed, one rather surprising byproduct of our numerical investigation is that the training performance of the adversarial generative model could be improved, at times significantly, by using a SVM in place of a NN for this component (see Figure 3). This, in itself, should be of some interest to practitioners. Not only can SVMs be faster to train, but they depend on significantly fewer hyper-parameters than NNs, whose performance is often highly dependent on careful tuning of the number of hidden layers, the number of neurons in each hidden layer, the learning rate, the batch size, etc. While we do not suggest that SVMs will always outperform NNs in this setting, this does indicate that SVMs may represent a viable alternative. We implement the NNs using PyTorch [91], while the SVMs are implemented with scikit-learn [92]. The particular hyper-parameters used in each simulation are specified below.

In the majority of our numerical simulations, we consider a QCBM with 3 qubits. This corresponds to a discrete target distribution p which takes $2^3$ values. We generally also assume that the target distribution corresponds to a particular instantiation of the QCBM, for a fixed number of layers, $D_p$. By varying the number of layers, $D_{q_{\mathbf{\theta }}}$ used to train the generative model, we can then investigate different parameterisation regimes of interest. In the case that the number of layers used to generate the target is greater than the number of layers used in the model ($D_p>D_{q_{\mathbf{\theta }}}$), the model is under-parameterised (or severely under-parameterised). Meanwhile, when the number of layers used to generate the target and the number of layers used in the model are equal ($D_p=D_{q_{\mathbf{\theta }}}$), the model is said to be exactly parameterised. In these cases, a solution to the learning problem is guaranteed to exist: there exists $\mathbf{\theta }={\mathbf{\theta }}_0$ such that $p\equiv q_{{\mathbf{\theta }}_{\mathbf{0}}}$. Finally, when the number of layers used to generate the target is less than the number used in the model ($D_p<D_{q_{\mathbf{\theta }}}$), the model is over-parameterised (or severely over-parameterised). We provide a more precise definition of these different cases, as applied to our numerics, in

Table 3. The different parameterisation regimes used in the 3 qubit numerical simulations.

	Severely over Parameterised (OO)	over Parameterised (O)	Exactly Parameterised (E)	under Parameterised (U)	Severely under Parameterised (UU)
Number of parameters (layers) used to generate the target p	12 parameters (1 layer)	12 parameters (1 layer)	12 parameters (1 layer)	30 parameters (4 layers)	30 parameters (4 layers)
Number of parameters (layers) used for the model $q_{\mathbf{\theta }}$	30 parameters (4 layers)	24 parameters (3 layers)	12 parameters (1 layer)	18 parameters (2 layers)	12 parameters (1 layer)

.

For each of the settings (i.e., choice of circuit depth for the target and model, choice of heuristic, number of qubits) explored, we train the generative model using nine independent parameter initialisations. We then use a bootstrapping procedure to provide a more robust estimate of the median cost at each training epoch. We first take samples of size nine from the outcome of the nine independent experiments, 10,000 times with replacement. We then compute the median cost across each set of samples to obtain a distribution of 10,000 medians. Using this distribution, we compute the median and obtain error bars from the 5th and 95th percentiles, corresponding to a $90\%$ confidence interval.

4.1. Switching f-Divergences

We begin by considering the performance of the heuristic introduced in Section 3.1. The f-divergences that can be standardised locally behave as KL to second order [76]. Notably, TV cannot be standardised; indeed, it is straightforward to show that TV provides an upper bound for all other f-divergences with $f^{″}\left(1\right)=1$ in this regime. For this reason, we evaluate both the exact TV and the exact KL to measure performance.

We begin by reporting the results obtained using an exact classifier, for each of the parameterisation regimes given in Table 3. The generator is trained using 1000 samples per iteration. The results are given in

Table 4. Performance of the QCBM trained using the TV and the f-divergence heuristic for 3 qubits in over-, under-, and exactly parameterised regimes. We show the bootstrapped median of the TV (top two rows) and the KL (bottom two rows) after 500 epochs. The asterisk (*) on some of the experiments indicates that the cost is still converging. The bold indicates the regimes where f-switch significantly outperforms the other methods.

${\mathit{D}}_{\mathit{f}}$ Evaluated	${\mathit{D}}_{\mathit{f}}$ Used in Training	OO (12, 30)	O (12, 24)	E (12, 12)	U (30, 18)	UU (30, 12)
TV	TV	$\left(1.12\begin{array}{c}+0.45\\ -0.28\end{array}\right)\times {10}^{-2}$	$\left(8.4\begin{array}{c}+1.2\\ -1.0\end{array}\right)\times {10}^{-3}$	$\left(1.00\begin{array}{c}+1.51\\ -0.12\end{array}\right)\times {10}^{-2}$	$\left(1.06\begin{array}{c}+0.26\\ -0.23\end{array}\right)\times {10}^{-2}$	$\left(1.4\begin{array}{c}+2.4\\ -0.7\end{array}\right)\times {10}^{-2}$
TV	f-switch	$\left(\mathbf{0.6}\begin{array}{c}+\mathbf{3.8}\\ -\mathbf{0.5}\end{array}\right)\times {\mathbf{10}}^{-\mathbf{5}}$ *	$\left(\mathbf{2.5}\begin{array}{c}+\mathbf{2.5}\\ -\mathbf{2.1}\end{array}\right)\times {\mathbf{10}}^{-\mathbf{3}}$ *	$\left(3.1\begin{array}{c}+1.8\\ -1.9\end{array}\right)\times {10}^{-2}$	$\left(0.65\begin{array}{c}+0.27\\ -0.51\end{array}\right)\times {10}^{-2}$	$\left(1.8\begin{array}{c}+2.9\\ -0.9\end{array}\right)\times {10}^{-2}$
KL	TV	$\left(3.5\begin{array}{c}+2.1\\ -1.3\end{array}\right)\times {10}^{-4}$	$\left(2.0\begin{array}{c}+0.6\\ -0.4\end{array}\right)\times {10}^{-4}$	$\left(2.6\begin{array}{c}+14.8\\ -2.3\end{array}\right)\times {10}^{-3}$	$\left(3.7\begin{array}{c}+1.7\\ -92.6\end{array}\right)\times {10}^{-4}$	$\left(0.6\begin{array}{c}+24.3\\ -0.4\end{array}\right)\times {10}^{-3}$
KL	f-switch	$\left(\mathbf{0.0182}\begin{array}{c}+\mathbf{1.383}\\ -\mathbf{0.012}\end{array}\right)\times {\mathbf{10}}^{-\mathbf{8}}$	$\left(1.8\begin{array}{c}+20.9\\ -1.7\end{array}\right)\times {10}^{-5}$ *	$\left(3.5\begin{array}{c}+9.1\\ -2.0\end{array}\right)\times {10}^{-3}$	$\left(2.4\begin{array}{c}+1.6\\ -2.4\end{array}\right)\times {10}^{-4}$	$\left(1.8\begin{array}{c}+4.3\\ -1.5\end{array}\right)\times {10}^{-3}$

.

Our results indicate that the heuristic is able to outperform TV when the QCBM is (severely) over-parameterised. This may be due to the extra degrees of freedom in the model. These allow for more discrepancies between the loss landscapes of the f-divergences, which the heuristic is able to exploit. In Figure 4 and Figure 5, we provide a more detailed illustration of the training performance of the f-switch heuristic in this regime. Figure 4 corresponds to an exact classifier: in this case, use of the heuristic significantly improves the convergence of the QCBM. Figure 5 corresponds to a trained classifier, trained on 1000 samples per iteration: in this case, use of the heuristic can lead to marginal performance improvements with respect to TV (left-hand figure). The remaining results in this Section are all reported for an exact classifier.

The average performance of the heuristic is similar to TV in the exactly and under-parametrised regimes. There are, however, initial parameter configurations within these regimes for which the heuristic significantly outperforms TV. In Figure 6, we plot the median losses obtained throughout the training of the QCBM in the under-parametrised U(30, 18) regime. The best-performing experiment in this regime is also presented in Figure 7, alongside all the other f-divergences considered in Figure 1. After 200 epochs, the training method that solely uses TV has converged, but all the other divergences, including the heuristic, continue to converge exponentially quickly to smaller losses. In the under-parameterised regime, the ansatz is not guaranteed to contain the true solution. However, after reaching a KL of $\sim {10}^{-3}$, these f-divergences traverse similar landscapes. Since the f-switch heuristic is shown to reach a KL of $\sim {10}^{-5}$, we can assume that all of these f-divergences will converge to the global minimum, with the heuristic arriving first.

Finally, in Figure 8, we illustrate the mechanics of the f-switch heuristic. In particular, we plot which f-divergence is ‘activated’ for each direction in the parameter space, at each epoch of the training in Figure 7.

We remark that as the number of qubits is increased, the randomly initialised model and the target distributions are expected to be increasingly further apart. The heuristic can pick the divergence that provides the highest initial learning signal. For this reason, we expect the heuristic to become particularly useful as the number of qubits is increased.

4.2. Local Cost Functions

We now turn our attention to the heuristic introduced in Section 3.2, incorporating locality in the cost function, dubbed f-local. In this Section, the target distribution is a discretised Gaussian. All classifiers are neural networks with 1 hidden layer made of 10 k ReLU neurons, where k is the locality parameter. The number of layers in the QCBM equals the number of qubits, $D=n$. All expectation values are estimated using 500 samples. In Figure 9, we plot the training performance of the QCBM using the global cost function and several k-local cost functions, for $n=4$, 5, and 6 qubit experiments. For 4 and 5 qubits, we show the bootstrapped median for the first 500 training epochs, as well as $90\%$ confidence intervals. For 6 qubits, we plot an illustrative training example for the first 1000 training epochs.

Let us make several remarks. Firstly, it would appear that the use of a k-local cost function can indeed improve the convergence (rate) of the training procedure, particularly during the initial stages. This improvement is increasingly evident as the number of qubits is increased. As such, this approach could be regarded as a potential strategy for tackling barren plateaus in higher-dimensional problems. However, we leave a thorough study of this phenomenon to future work.

Secondly, it is clear that the use of any k-local cost function will eventually prohibit convergence to the true target distribution. As discussed in Section 3.2, the k-local cost function is minimised whenever the k-marginal distributions of the target and the model coincide, which does not necessarily imply that their joint distributions are equal. The smaller the value of k, the greater the possible disparity between two distributions whose k-marginals coincide. This is clearly visualised in Figure 9: as the value of k decreases, the asymptotic reverse KL achieved during training with the k-local cost function plateaus at increasingly larger values.

As remarked previously, this suggests that an optimal training strategy may be to start the training procedure with a small value of k, before iteratively increasing the value of k as training proceeds. For example, let us consider the 5 qubit experiment in Figure 9b. Initially, the 3-local cost function (red) appears to yields the greatest convergence rate. After approximately 150 epochs, the 4-local cost function (purple) now seems to be favourable. Asymptotically, one can imagine that the global cost function (blue) will be preferable. One observes similar behaviour in the 6 qubit experiment in Figure 9c.

In practice, of course, it is not possible to compute the reverse KL directly, and thus another tractable metric is required in order to determine the optimal moment for switching between the k-local cost functions. Alternatively, one can simply increase the locality of the cost function after a set number of epochs.

5. Estimation of f-Divergences on Fault-Tolerant Quantum Computers

The above discussion is purely heuristic in nature and suitable for near-term quantum computers, but we can also address f-divergences from the other end of the spectrum; using fault-tolerant devices. In particular, we can leverage a recent line of study into quantum property testing of distributions. The key question here is whether or not a particular probability distribution has a certain property.

The work of [38] was one of the first to provide such an answer, demonstrating a quadratic speedup for determining whether two distributions over $\left[n\right]$ were close or $\epsilon$-far in TV. These quantum algorithms typically work in the oracle model, and we measure run time relative to the number of queries to such an oracle (query complexity). In the classical case, we define oracle access to a distribution over $\left[n\right]$, $p={\left\{p_i\right\}}_{i=1}^n$ as $O_p:\left[S\right]\to \left[n\right],S\in \mathbb{N}$. The oracle is a mechanism to translate a uniform distribution over $\left[S\right]$ to the true distribution over $\left[n\right]$. In the quantum case, such an oracle is replaced by a unitary operator, ${\widehat{O}}_p$ acting on a state encoding $s\in \left[S\right]$, along with an ancillary register to ensure reversibility and defined as: ${\widehat{O}}_p|s\rangle |0\rangle =|s\rangle |O_p\left(s\right)\rangle \forall s\in \left[S\right]$.

We begin our discussion with the TV. The authors of [38] produced a quantum property testing algorithm for the TV via an algorithm which actually estimates the TV quadratically faster. The analysis in [38] resulted in an algorithm to estimate the TV up to additive error $\epsilon$, with probability of success of $1-\delta$, using $O(\sqrt{n}/{\epsilon }^8{\delta }^5)$ samples. This was later improved by [39] to the following

Theorem 1

(Section 4, Montanaro [39]). Assume $p,q$ are two distributions on $\left[n\right]$. Then there is a quantum algorithm that approximates $TV(p,q)$ up to an additive error $\epsilon >0$, with probability of success $1-\delta$, using $O(\sqrt{n}{\epsilon }^{-3/2}/log\left(1/\delta \right))$ quantum queries.

These ideas were extended in [40] to also give an algorithm for computing the (forward) KL quadratically faster than possibly classically (and also computing certain entropies of distributions). Due to the existence of the ratio $p_i/q_i$ in the expression for the KL, we must make a further assumption, which was not necessary in the case of the TV distance in Theorem 1. This assumption will also be necessary when considering many of the other divergences in Table 1. In particular, we must assume the two distributions are such that: $p_i/q_i\le g\left(n\right),\forall i\in \left[n\right]$, for some $g:\mathbb{N}\to {\mathbb{R}}^{+}$. (This assumption is appropriate when one defines the KL in terms of the generator f and the ratio $r=p/q$. Conversely, when one defines the KL in terms of the convex conjugate $f^{*}$ and the ratio $r=q/p$, then the appropriate assumption would instead be that $q_i/p_i\le g\left(n\right)$, $\forall i\in \left[n\right]$.) This assumption is also necessary in the classical case. With this, we then have

Theorem 2

(Theorem 4.1, Li and Wu [40]). Assume $p,q$ are two distributions on $\left[n\right]$ satisfying $p_i/q_i\le g\left(n\right),\forall i\in \left[n\right]$ for some $a:\mathbb{N}\to {\mathbb{R}}^{+}$. Then there is a quantum algorithm that approximates $KL(p\parallel q)$ within an additive error $\epsilon >0$ with probability of success at least $2/3$ using $\tilde{O}(\sqrt{n}/{\epsilon }^2)$ quantum queries to p and $\tilde{O}(\sqrt{n}g\left(n\right)/{\epsilon }^2)$ quantum queries to q. (The notation $\tilde{O}(·)$ ignores factors that are polynomial in $logn$ and $log1/\epsilon$.)

These results cover two of the f-divergences we use above (see Table 1). In particular, the latter algorithm provide a quantum speedup since it is known that one requires $\Omega \left(n/log\left(n\right)\right),\Omega \left(ng\left(n\right)/log\left(n\right)\right)$ classical queries to p and q respectively to estimate the KL [93]. On the other hand, we get a speedup for the former algorithm since it is known one requires $\Theta \left(n^{2/3}{\epsilon }^{-4/3}\right)$ [94] queries to test if two distributions are near or far in TV classically, which is an easier problem than estimating the metric directly.

The key idea behind both of these algorithms is to use a subroutine known as quantum probability estimation or quantum counting, which is adapted from quantum amplitude estimation. This provides a quadratic speedup in producing estimates ${\tilde{p}}_i,{\tilde{q}}_i$, of probabilities $p_i,q_i$ from the distributions $p,q$, which are specified via a quantum oracle. Once the estimates of ${\tilde{p}}_i,{\tilde{q}}_i$ have been produced via the quantum subroutine, both of the above algorithms reduce to simple classical post-processing. This post processing involves constructing a random variable, y, whose expectation value gives exactly the divergence we require. For TV and KL estimation, this random variable is given by

$$\begin{array}{ccc}\hfill y_i^{\mathrm{TV}}& :=& \frac{\left|p_i-q_i\right|}{p_i+q_i},\hfill \end{array}$$

(29)

$$\begin{array}{ccc}\hfill y_i^{\mathrm{KL}}& :=& log\frac{p_i}{q_i}=log{p}_i-log{q}_i.\hfill \end{array}$$

(30)

By sampling this random variable according to another distribution $r:={\left(r_i\right)}_{i=1}^n$ (to be defined below), the quantity of interest is exactly given as an expectation value, namely

$$\begin{array}{ccc}\hfill \sum _i{r}_i^{\mathrm{TV}}{y}_i^{\mathrm{TV}}& =& \mathbb{E}\left[y^{\mathrm{TV}}\right]=\mathrm{TV}(p,q),\hfill \end{array}$$

(31)

$$\begin{array}{ccc}\hfill \sum _i{r}_i^{\mathrm{KL}}{y}_i^{\mathrm{KL}}& =& \mathbb{E}\left[y^{\mathrm{KL}}\right]=\mathrm{KL}(p\parallel q).\hfill \end{array}$$

(32)

One can check [38,40] that the suitable random variables are given by

$$\begin{array}{ccc}\hfill r_i^{\mathrm{TV}}& =& \frac{1}{2}\left(p_i+q_i\right),\hfill \end{array}$$

(33)

$$\begin{array}{ccc}\hfill r_i^{\mathrm{KL}}& =& q_i.\hfill \end{array}$$

(34)

Due to the probabilistic nature of quantum mechanics, one cannot obtain the exact values of the probabilities required to compute these expectation values. We must settle instead for approximations of $p,q$, namely $\tilde{p},\tilde{q}$. These estimates are achieved using the quantum approximate counting lemma, which is an application of quantum amplitude estimation [95]. The work in [40] considered two versions of this algorithm, called EstAmp and EstAmp’. The only difference between these two algorithms is the behavior when one of the probabilities, $q_i$, is sufficiently close to zero. This is problematic in the case of the KL estimation (and indeed entropy estimation) in [40] since the relevant quantities diverge as $q_i\to 0$. The same is true in our case, as $q_i^{-1}$ appears in many f-divergences.

Theorem 3

(Theorem 13, Brassard et al. [95] and Theorem 2.3, Li and Wu [40]). For any $k,M\in \mathbb{N}$, there is a quantum algorithm (named EstAmp) with M queries to a boolean function, $\chi :\left[S\right]\to \{0,1\}$ that outputs $\tilde{a}={sin}^2\left(\frac{l\pi }{M}\right)$ for some $l\in \{0,\cdots ,M-1\}$ such that

$$Pr\left[\tilde{a}={sin}^2\left(\frac{l\pi }{M}\right)\right]=\frac{{sin}^2(M\Delta \pi )}{M^2{sin}^2(\Delta \pi )}\le \frac{1}{{(2M\Delta )}^2},$$

(35)

where $\Delta =|\omega -l/M|$. This promises $|\tilde{a}-a|\le 2\pi k\frac{\sqrt{a(1-a)}}{M}+k^2\frac{{\pi }^2}{M^2}$ with probability at least $8/{\pi }^2$ for $k=1$ and with probability greater than $1-\frac{1}{2(k=1)}$ for $k\ge 2$. If $a=0$ then $\tilde{a}=0$.

The modified algorithm (EstAmp’) outputs ${sin}^2\left(\frac{\pi }{2M}\right)$ when EstAmp outputs 0, and outputs the same as EstAmp otherwise. Now that we have a mechanism for estimating the probabilities, we need a final ingredient, which is the generic speedup of Monte Carlo methods from [39]

Theorem 4

(Theorem 5, Montanaro [39]). Let $\mathcal{A}$ be a quantum algorithm with output X such that $Var\left[X\right]\le {\sigma }^2$. Then for ε where $0<\epsilon <4\sigma$, by using $O((\sigma /\epsilon ){log}^{3/2}(\sigma /\epsilon )loglog(\sigma /\epsilon ))$ executions of $\mathcal{A}$ and ${\mathcal{A}}^{-1}$, Algorithm 3 in [39] outputs an estimate $\tilde{\mathbb{E}}\left[X\right]$ of $\mathbb{E}\left[X\right]$ such that

$$Pr\left[|\tilde{\mathbb{E}}\left[X\right]-\mathbb{E}\left[X\right]|\ge \epsilon \right]\le 1/5.$$

(36)

Using these results, we now extend Theorems 1 and 2 to cover another f-divergence in Table 1: the forward Pearson divergence, ${\chi }^2(p\parallel q)$. The convex conjugate of the generator for this divergence is given by $f^{*}\left(r\right)=\frac{1}{2}{(r-1)}^2$ or, equivalently, $f^{*}\left(r\right)=\frac{1}{2}(r^2-1)$. The equivalence of these two generators is straightforward to demonstrate. In particular, we have -4.6cm0cm

$${\mathbb{E}}_p\left[{\left(\frac{q_i}{p_i}-1\right)}^2\right]=\sum _i{p}_i{\left(\frac{q_i}{p_i}\right)}^2-2\sum _i{p}_i\frac{q_i}{p_i}+\sum _i{p}_i=\sum _{\mathit{x}}{p}_i{\left(\frac{q_i}{p_i}\right)}^2-1={\mathbb{E}}_p\left[{\left(\frac{q_i}{p_i}\right)}^2-1\right].$$

(37)

In fact, in what follows, we make use of the following representation:

$${\chi }^2(p\parallel q):=\frac{1}{2}\sum _i{p}_i\left[{\left(\frac{q_i}{p_i}\right)}^2-1\right]=\sum _i{q}_i\left(\frac{q_i}{p_i}-1\right):=\sum _i{r}_i^{FP}{y}_i^{FP},$$

(38)

where we have identified $r_i^{FP}=q_i$ and $y_i^{FP}=\frac{1}{2}(\frac{q_i}{p_i}-1)$. Using this representation, we develop the following Algorithm 1 for estimating the forward Pearson divergence.

Algorithm 1: Estimate the forward Pearson divergence of $p={\left(p_i\right)}_{i=1}^n$ and ${\left(q_i\right)}_{i=1}^n$ on $\left[n\right]$.

The query complexity of this Algorithm is contained in the following theorem. We defer the proof of this result, which is largely a technical extension of the proof(s) in [40], to Appendix A.

Theorem 5.

Assume $p,q$ are two distributions on $\left[n\right]$ satisfying $q_i/p_i\le g\left(n\right),\forall i\in \left[n\right]$ for some $a:\mathbb{N}\to {\mathbb{R}}^{+}$. Then there is a quantum algorithm that approximates ${\mathcal{X}}^2(p\parallel q)$ within an additive error $\epsilon >0$ with probability of success at least $2/3$ using $\tilde{O}(\sqrt{n}g\left(n\right)/{\epsilon }^2)$ quantum queries to q and $\tilde{O}(\sqrt{n}g{\left(n\right)}^2/{\epsilon }^2)$ quantum queries to p.

6. Discussion

Each f-divergence, with its unique operational meaning, finds application in information theory, statistics, and machine learning. In this paper, we showed that a generative model called quantum circuit born machine can be trained by efficiently minimising anyf-divergence. The key observation is that a probabilistic classifier can be trained adversarially to provide an approximation to such divergences.

Building on this, we developed heuristics aimed at improving convergence of the generative training. The first heuristic, f-switch, lets each parameter minimise a different f-divergence. Numerical results with an ideal exact classifier show that this heuristic can converge faster and to better minima than when using a single f-divergence. However, in a more realistic setting where the classifier is trained adversarially, f-switch yields results similar to those obtained by minimising a single f-divergence.

The second training heuristic, f-local, consists of using a single f-divergence approximated by local cost functions. Numerical results show that, as the number of qubits increases, this strategy yield improved convergence of the generative training than when using a global cost function. To the best of our knowledge this is the first proposal of cost functions for generative modelling that can interpolate between trainability and accuracy. Extensive numerical simulations will be needed to confirm whether f-local can alleviate the barren plateau problem in generative modelling.

Interestingly, our local cost functions approximate the f-divergence using an ensemble of local binary classifiers. If the target probability distribution is known to have a particular conditional independence structure (e.g., it is defined by a Bayesian network or a Hidden Markov model), this information could be used to inform the choice of local classifiers.

One interesting research direction is to adapt the above heuristics to work with other families of distance measures. Of particular interest, integral probability metrics (IPMs) include the maximum mean discrepancy, the Dudley metric and the Wasserstein distance. While f-divergences are defined in terms of probability ratios, IPMs are defined in terms of probability differences. However, it is know that under suitable constraints margin-based classifiers yield estimators for IPMs [96]. This suggests that an extension of our heuristics to IPMs could be possible.

In this work, we also discussed the possibility of estimating certain f-divergences on a fault-tolerant quantum computer, therefore avoiding the use of classifiers. Previously published work has proven quadratic quantum speedups for the estimation of total variation [38,39] and forward Kullback–Leibler (KL) of type I [40]. Using these algorithms a quadratic speedups is achievable for the reverse KL of type I, and thus for the symmetric KL of type I (also known as Jeffrey divergence). It is plausible that with some refinements these algorithms can provide quadratic speedups for the KL of type II as well.

We contributed to this topic with an algorithm for estimating Pearson ${\chi }^2$ divergences and by providing its query complexity. Interestingly, high-order Pearson divergences (also known as Vajda divergences) can be used to approximate any other f-divergence via Taylor expansion [97]. Generalising our quantum algorithm to Vajda divergences would therefore provide a way to estimate all other f-divergences on a fault-tolerant quantum computer.