Quantum Mean-Field Games with the Observations of Counting Type

Abstract

Quantum games and mean-field games (MFG) represent two important new branches of game theory. In a recent paper the author developed quantum MFGs merging these two branches. These quantum MFGs were based on the theory of continuous quantum observations and filtering of diffusive type. In the present paper we develop the analogous quantum MFG theory based on continuous quantum observations and filtering of counting type. However, proving existence and uniqueness of the solutions for resulting limiting forward-backward system based on jump-type processes on manifolds seems to be more complicated than for diffusions. In this paper we only prove that if a solution exists, then it gives an $ϵ$-Nash equilibrium for the corresponding N-player quantum game. The existence of solutions is suggested as an interesting open problem.

1. Introduction

In [1], two recently developed branches of game theory, quantum games and mean field games (MFGs), were merged, creating quantum MFGs. MFGs represent a very popular recent development in game theory. It was initiated in [2,3]. For recent developments one can consult monographs [4,5,6,7] and numerous references therein. Quantum games were initiated by Meyer [8], Eisert, Wilkens and Lewenstein [9], and Marinatto and Weber [10], and were dealt with afterwards in numerous publications, see, e.g., surveys [11,12], and Chapter 13 of textbook [13].

Using approaches from [9,10], one can transform any game to a new quantum version. This transformation modifies in a systematic way all properties of the games: equilibria, their stability, etc. For instance, stability of the equilibria of the transformed Replicator Dynamics for two-player two-action games was analyzed in [14]. ESS (evolutionary stable strategies) for the transformed Rock-Paper-Scissors game was analyzed in [15], and for 3 player games in [16]. The transformations of the simplest cooperative games were analyzed in [17]. In [18] the EWL (Eisert, Wilkens and Lewenstein) protocol was applied to the Battle of Sexes, in [19] to the general prisoner’s dilemma and in [20] to the three player quantum Prisoner’s dilemma. Peculiar behavior and remarkable phase transitions were found. The extension of EWL protocol for games with continuous strategy space was suggested in [21].

For application of related quantum concepts (including quantum probability) to cognitive sciences we refer to [22,23] and references therein.

The main accent in all these developments was made on stationary or repeated games, see, e.g., [24,25] for the latter, and [26] for their interpretation in economics. Not only for games, but generally for quantum control the main stream of quantum control research is based on open loop controls, with a rare appearance of a feedback control, see, e.g., [27] and [28].

The present paper initiates the study of the truly dynamic theory with observations of counting type and with the strategies chosen by players in real time. Since direct continuous observations are known to destroy quantum evolutions (so-called quantum Zeno paradox) the necessary new ingredient for quantum dynamic games must be the theory of non-direct observations and the corresponding quantum filtering. This theory is usually performed in two forms: diffusive (or homodyne) type and counting type. In paper [1] the author developed quantum MFGs based on diffusive type filtering. In the present paper quantum MFGs are built for counting type quantum observations and filtering.

As a part of the construction we show that the limiting behavior of mean field interacting controlled quantum particles (or N-player quantum game) can be described by certain classical MFG forward-backward system of jump-type equations on manifolds, the forward part being given by a new kind of nonlinear jump-type stochastic Schrödinger equations. One of the objectives of the paper is to draw the attention of game theorists to this type of games and this type of forward-backward systems, which were not studied before, and no results even on the existence of solutions are available. These objects are fully classical, but represent the limit of quantum games.

The main result states that any solution to this forward-backward system represents an approximate $N^{-1/4}$-Nash equilibrium for the initial N-player dynamic quantum game.

The content of the paper is as follows. In the next section we recall the basic theory of quantum continuous measurement and filtering. In Section 3, as a warm-up, we discuss briefly an example of a two-player quantum dynamic game on a qubit with observation and feedback control of counting type. In Section 4 the new nonlinear equations are introduced for the case of controlled counting detection and the convergence of N-particle observed quantum evolutions to the decoupled system of these equations is obtained, together with explicit rates of convergence. In Section 5 the MFG limits for quantum N-player games are introduced and it is proven that solutions for the limiting MFG equations specify $ϵ$-Nash equilibria for N-player quantum game, with $ϵ$ of order $N^{-1/4}$. The limiting MFGs can be also looked at as classical MFGs, though complex-valued and evolving in infinite-dimensional manifolds. In the final section we state the problem of existence of the solutions, even in the simplest case of the control problem on a qubit.

2. Quantum Filtering of Counting Type

The general theory of quantum non-demolition observation, filtering and resulting feedback control was built essentially in papers [29,30,31]. For alternative simplified derivations of the main filtering equations given below (by-passing the heavy theory of quantum filtering) we refer to [32,33,34,35,36] and references therein. For the technical side of organising feedback quantum control in real time, see, e.g., [37,38,39].

We shall describe briefly the main result of this theory.

The non-demolition measurement of quantum systems can be organised in two versions: photon counting and homodyne detection. As was stressed above, here we shall deal only with counting measurements. In this case the main equation of quantum filtering takes the form

$$d{\gamma }_t=-i[H,{\gamma }_t]dt+\sum _j(-\frac{1}{2}\{L_j^{\ast }{L}_j,{\gamma }_t\}+\mathrm{tr}\left(L_j{\gamma }_t{L}_j^{\ast }\right){\gamma }_t)dt+\sum _j\left(\frac{L_j{\gamma }_t{L}_j^{\ast }}{\mathrm{tr}\left(L_j{\gamma }_t{L}_j^{\ast }\right)}-{\gamma }_t\right)d{N}_t^j,$$

(1)

in terms of the density matrices ${\gamma }_t$, where H is the Hamiltonian of the free (not observed) motion of a quantum system, the operators $\left\{L_j\right\}$ define the coupling of the system with the measurement devices, and the (counting) observed Poisson processes $N_t^j$ are independent and have the position dependent intensities $\mathrm{tr}\left(L_j^{\ast }{L}_j{\gamma }_t\right)$, so that the compensated processes $M_t^j=N_t^j-{\int }_0^t\mathrm{tr}\left(L_j^{\ast }{L}_j{\gamma }_s\right)ds$ are martingales. By $\{A,B\}$ we denote the anticommutator of two operators: $\{A,B\}=AB+BA$. In terms of the compensated processes $M_t^j$ Equation (1) rewrites as

$$d{\gamma }_t=-i[H,{\gamma }_t]dt+\sum _j(L_j\gamma L_j^{\ast }-\frac{1}{2}\{L_j^{\ast }{L}_j,{\gamma }_t\})dt+\sum _j\left(\frac{L_j{\gamma }_t{L}_j^{\ast }}{\mathrm{tr}\left(L_j{\gamma }_t{L}_j^{\ast }\right)}-{\gamma }_t\right)d{M}_t^j.$$

(2)

In this paper we shall deal only with the simplest case when the operators L are unitary. In this case $d{M}_t^j=d{N}_t^j-dt$ and Equations (1) and (2) become linear and take the form

$$d{\gamma }_t=-i[H,{\gamma }_t]dt+\sum _j\left(L_j{\gamma }_t{L}_j^{\ast }-{\gamma }_t\right)d{N}_t^j=-i[H,{\gamma }_t]dt+\sum _j\left(L_j{\gamma }_t{L}_j^{\ast }-{\gamma }_t\right)(d{M}_t^j+dt).$$

(3)

This dynamics preserves the set of pure states. Namely, if $\varphi$ satisfies the equation

$$d{\varphi }_t=-iH{\varphi }_{t}dt+\sum _j(L_j-\mathbf{1}){\varphi }_{t}d{N}_t^j=-iH{\varphi }_{t}dt+\sum _j(L_j-\mathbf{1}){\varphi }_t(d{M}_t^j+dt),$$

(4)

then ${\gamma }_t={\varphi }_t\otimes {\overline{\varphi }}_t$ satisfies Equation (3).

The theory of quantum filtering reduces the analysis of quantum dynamic control and games to the controlled version of evolutions (1). Two types of control can be naturally considered (see [40]). The players can control the Hamiltonian H, say, by applying appropriate electric or magnetic fields to the atom, or the coupling operators $L_j$. Thus (3) extends to the equation

$$\begin{array}{cc}\hfill d{\gamma }_t& =-i[H+u\widehat{H},{\gamma }_t]dt+\sum _j\left(L_j\left(v\right){\gamma }_t{L}_j^{\ast }\left(v\right)-{\gamma }_t\right)d{N}_t^j\hfill \\ \hfill & =-i[H+u\widehat{H},{\gamma }_t]dt+\sum _j\left(L_j\left(v\right){\gamma }_t{L}_j^{\ast }\left(v\right)-{\gamma }_t\right)(d{M}_t^j+dt),\hfill \end{array}$$

(5)

with some self-adjoint $\widehat{H}$, control u and a family of unitary operators $L\left(v\right)$ depending on a control parameter v.

It is seen from Equation (5) that its evolution preserves traces of matrices. One can also show that these evolutions preserve positivity of matrices $\gamma$ (see, e.g., [36]).

3. Example of a Quantum Dynamic Two-Player Game

Let us stress again that the whole physics of quantum dynamic games with a feedback control of a finite number of players is incorporated into the stochastic filtering Equation (1), so that the quantum dynamic games are reduced to the stochastic games with jumps governed by this equation with operators H and L that may depend on control. As a warm-up before the mean-field setting let us consider the simple example of a zero-sum quantum dynamic two-player game on a qubit, where a complete analytic solution can be found.

Working with a qubit means that the Hilbert space of the quantum system is two dimensional. Let L be fixed and the Hamiltonian be the sum of two parts, controlled by the first and the second player respectively. Stochastic filtering Equation (1) simplifies to the equation (omitting index t)

$$d\psi =-iH\psi dt+(L\psi -\psi )d{N}_t,H=u{H}_1+v{H}_2,$$

(6)

$u,v$ being control parameters of players I and II. Assume $u\in [-U,U]$, $v\in [-V,V]$ with some positive $U,V$. Moreover, $\psi$ has only two coordinates: $\psi =({\psi }_0,{\psi }_1)$. Using Ito’s rule $d{N}_{t}d{N}_t=d{N}_t$ we find the equation for ${\psi }_0^{-1}$:

$$d{\psi }_0^{-1}=\frac{i}{{\psi }_0^2}{\left(H\psi \right)}_{0}dt-\frac{{\left(L\psi \right)}_0-{\psi }_0}{{\psi }_0{\left(L\psi \right)}_0}d{N}_t.$$

Consequently, again by Ito’s rule, we find the equation for $w={\psi }_1/{\psi }_0$:

$$dw=i[w{\left(HW\right)}_0-{\left(HW\right)}_1]dt+[{\left(LW\right)}_1-w{\left(LW\right)}_0]d{N}_t,$$

(7)

where $W=(w_0,w_1)=(1,w)$.

Let us choose the simplest possible L: $L={\sigma }_3$—the third Pauli matrix (diagonal with diagonal elements 1 and $-1$). Then Equation (7) simplifies to

$$dw=i[w{\left(\widehat{H}W\right)}_0-{\left(\widehat{H}W\right)}_1]dt-2wd{N}_t.$$

(8)

The payoffs in quantum setting are given by certain operators, that is, they have the form

$$P(t,W;u(.),v(.))={\int }_t^T({\psi }_s,J{\psi }_s)ds+({\psi }_T,F{\psi }_T),$$

(9)

where J and F are some self-adjoint operators. They may depend on the control parameters, but we shall look for the case when they do not. In terms of w this payoff rewrites as

$$P(t,W;u(.),v(.))={\int }_t^T\frac{(W_s,J{W}_s)}{1+|w_s{|}^2}ds+\frac{(W_T,F{W}_T)}{1+|w_T{|}^2}.$$

(10)

Thus the zero-sum quantum dynamic two-player game (with a feedback control) with a fixed horizon T in this setting is the stochastic dynamic game with the state space $\mathbf{C}$, with the evolution described by the jump-type stochastic Equation (8) and payoff (10). The aim of the first player is to maximise the expectation of (10) using an appropriate feedback strategies $u(.)=u(t,W_t)$. The second player tries to minimise it using an appropriate feedback strategies $v(.)=v(t,W_t)$.

The remarkable feature of this game is that the possible jumps are only of type $w\to -w$. Consequently, in the coordinates $r=\sqrt{x^2+y^2}$ and $\xi =y/x$ (where $w=x+iy$), the dynamics is deterministic. Therefore, if the operators J and F of current and terminal payoffs are invariant under the transformation $w\to -w$, the game can be reduced to a deterministic differential game. This game is still very complicated.

Let us consider now the most trivial example of commuting operators $H_1$ and $H_2$ controlled by two players. To be concrete, let us chose $H_1$ diagonal with diagonal elements 1 and 0, and $H_2$ diagonal with elements 0 and 1. Then Equation (8) becomes linear in w:

$$dw=i\left[(u-v)w\right]dt-2wd{N}_t,$$

(11)

and then the modulus $\rho ={\left|w\right|}^2$ becomes the integral of motion: $d\left(\right|w{|}^2)=0$. Choosing $\rho =1$ for definiteness we get the equation for the angle $\varphi$ on the circle $\rho =1$:

$$dcos\varphi =(u-v)sin\varphi dt-2cos\varphi d{N}_t.$$

(12)

If J and F are invariant under the transformation $w\to -w$, we can identify points when $cos\varphi$ differ only by a sign (so that possible jumps $cos\varphi \to -cos\varphi$ become irrelevant), and the evolution on a circle, given by the set $\varphi \in [-\pi /2,\pi /2]$ with identified endpoints, becomes deterministic:

$$\frac{d}{dt}cos\varphi =(u-v)sin\varphi ⟺\dot{\varphi }=-(u-v),$$

(13)

that is a simple rotation. Choosing $F=0$ and the simplest nontrivial J with zero diagonal elements and real numbers j as non-diagonal terms. The payoff (10) for $\rho =\left|w\right|=1$ simplifies to

$$P(t,W;u(.),v(.))=j{\int }_t^{T}cos{\varphi }_{s}ds.$$

(14)

The HJB-Isaacs equation takes the form

$$\frac{\partial S}{\partial t}+\underset{u}{max}\left(-u\frac{\partial S}{\partial \varphi }\right)+\underset{v}{min}\left(v\frac{\partial S}{\partial \varphi }\right)+jcos\varphi =0.$$

Assuming for definiteness that $U>V$, so that the first player has an edge in this game, the equation rewrites as

$$\frac{\partial S}{\partial t}+(U-V)\left|\frac{\partial S}{\partial \varphi }\right|+jcos\varphi =0.$$

(15)

This is HJB of a pure maximisation problem. It can be solved via the method of viscosity solutions. For instance, let us find a stationary solution describing the average winning of the first player per unit of time in a long lasting game. For this one searches for a solution to (15) in the form $S=\lambda (T-t)+S_0\left(\varphi \right)$ with a constant $\lambda$. Then $S_0\left(\varphi \right)$ (obviously defined up to a constant multiplier, so that we can set $S_0\left(0\right)=0$) satisfies the equation

$$-\lambda +(U-V)\left|\frac{\partial S_0}{\partial \varphi }\right|+jcos\varphi =0.$$

(16)

To guess the right solution one can derive from the meaning of this equation that $S_0$ must be an even function of $\varphi$ with maximum at $\varphi =0$, decreasing on $[0,\pi /2]$. Hence $(\partial S_0/\partial \varphi )\left(0\right)=0$ and thus $\lambda =j$ and Equation (16) on $[0,\pi /2]$ becomes

$$-(U-V)\frac{\partial S_0}{\partial \varphi }=j(1-cos\varphi ),$$

(17)

so that

$$S_0\left(\varphi \right)=-\frac{j}{U-V}(\varphi -sin\varphi ).$$

This function (considered as periodically continued with period $\pi$ to the whole line) is smooth outside points $(2k+1)\pi /2$, where it has convex kinks. Hence this is really the viscosity solution to (16) confirming that our educated guess above was correct and that $\lambda =j$ is the income per unit of time to the first player for a long lasting game.

Another example for the case of quantum control (without games) was given in [28].

4. Controlled Limiting Stochastic Equation

Let X be a Borel space with a fixed Borel measure that we denote $dx$.

For a linear operator O in $L^2\left(X\right)$ we shall denote by $O_j$ the operator in $L^2\left(X^N\right)$ that acts on functions $f(x_1,\cdots ,x_N)$ as O acting on the variable $x_j$. For a linear operator A in $L^2\left(X^2\right)$ we shall denote by $A_{ij}$ the operator in $L^2\left(X^N\right)$ that acts as A on the variables $x_i,x_j$.

Let H and $\widehat{H}$ be two self-adjoint operators in $L^2\left(X\right)$ and A a self-adjoint integral operator in $L^2\left(X^2\right)$ with the kernel $A(x,y;x^{\prime },y^{\prime })$ that acts on the functions of two variables as

$$A\psi (x,y)={\int }_{X^2}A(x,y;x^{\prime },y^{\prime })\psi (x^{\prime },y^{\prime })d{x}^{\prime }d{y}^{\prime }.$$

It is assumed that A is symmetric in the sense that it takes symmetric functions $\psi (x,y)$ (symmetric with respect to permutation of x and y) to symmetric functions.

Let us consider the quantum evolution of N particles driven by the interaction Hamiltonian

$$H_u\left(N\right)f(x_1,\cdots ,x_N)=\sum _{j=1}^N(H_j+u_j(t,{\Gamma }_N^{\left(j\right)}){\widehat{H}}_j)f(x_1,\cdots ,x_N)+\frac{1}{N}\sum _{i<j\le N}{A}_{ij}f(x_1,\cdots ,x_N).$$

(18)

Here continuous functions $u_j(t,\gamma )$ describe the controls of jth agent, who is supposed to have access to the jth subsystem, namely to the partial trace ${\Gamma }_{N,t}^{\left(j\right)}$ (with respect to all other variables but j) of the state ${\Gamma }_{N,t}$. All $u_j$ are taken from a bounded interval $[-U,U]$.

In order to be able to carry out a feedback control we assume further that this quantum system is observed via coupling with the collection of (possibly controlled) identical one-particle unitary families $L\left(v\right)$. That is, we consider the filtering Equation (3) of the type

$$d{\Psi }_{N,t}=-i{H}_u\left(N\right){\Psi }_{N,t}dt+\sum _{j=1}^N(L_j\left(v_j(t,{\Gamma }_N^{\left(j\right)})\right)-\mathbf{1}){\Psi }_{N,t}d{N}_t^j.$$

(19)

The corresponding density matrix ${\Gamma }_{N,t}={\Psi }_{N,t}\otimes \overline{{\Psi }_{N,t}}$ satisfies the equation of type (5):

$$d{\Gamma }_{N,t}=-i[H_u\left(N\right),{\Gamma }_{N,t}]dt+\sum _j(L_j\left(v_j(t,{\Gamma }_N^{\left(j\right)})\right){\Gamma }_{N,t}{L}_j^{\ast }\left(v_j(t,{\Gamma }_N^{\left(j\right)})\right)-{\Gamma }_{N,t})d{N}_t^j.$$

(20)

The main ingredient in the construction of quantum MFG theory is the quantum law of large numbers that states that as $N\to \infty$, the limiting evolution of each particle (precise conditions are given in the theorem below) is described by the nonlinear stochastic equation

$$d{\psi }_{j,t}=-i[H+u(t,{\gamma }_{j,t})\widehat{H}+A^{{\overline{\eta }}_t}]{\psi }_{j,t}dt+(L\left(v_j(t,{\gamma }_{j,t})\right){\psi }_{j,t}-{\psi }_{j,t})d{N}_t^j,$$

(21)

where $A^{{\overline{\eta }}_t}$ is the integral operator in $L^2\left(X\right)$ with the integral kernel

$$A^{{\overline{\eta }}_t}(x;y)={\int }_{X^2}A(x,y;x^{\prime },y^{\prime })\overline{{\eta }_t(y,y^{\prime })}dyd{y}^{\prime }$$

and

$${\eta }_t(y,z)=\mathbf{E}\left({\psi }_{j,t}\left(y\right){\overline{\psi }}_{j,t}\left(z\right)\right).$$

The equation for the corresponding density matrix ${\gamma }_{j,t}={\psi }_{j,t}\otimes {\overline{\psi }}_{j,t}$ writes down as

$$\begin{array}{cc}\hfill d{\gamma }_{j,t}& =-i[H+u(t,{\gamma }_{j,t})\widehat{H},{\gamma }_{j,t}]dt-i[A^{{\overline{\eta }}_t},{\gamma }_{j,t}]dt\hfill \\ \hfill & +(L\left(v_j(t,{\gamma }_{j,t})\right){\gamma }_{j,t}{L}^{\ast }-{\gamma }_{j,t})d{N}_t^j,\hfill \\ \hfill {\eta }_t(y,z)& =\mathbf{E}\left({\psi }_{j,t}\left(y\right){\overline{\psi }}_{j,t}\left(z\right)\right)=\mathbf{E}{\gamma }_{j,t}(y,z).\hfill \end{array}$$

(22)

For the analysis of the limiting behavior we use an approach from [41,42], where the main measures of the deviation of the solutions ${\Psi }_{N,t}$ to N-particle systems from the product of the solutions ${\psi }_t$ to the Hartree equations are the following positive numbers from the interval $[0,1]$:

$${\alpha }_N\left(t\right)=1-({\psi }_t,{\Gamma }_{N,t}{\psi }_t).$$

In the present stochastic case, these quantities depend not just on the number of particles in the product, but on the concrete choice of these particles. The proper stochastic analog of the quantity ${\alpha }_N\left(t\right)$ is the collection of random variables

$${\alpha }_{N,j}\left(t\right)=1-({\psi }_{j,t},{\Gamma }_{N,t}{\psi }_{j,t})=1-\mathrm{tr}\left({\gamma }_{j,t}{\Gamma }_{N,t}\right)=1-\mathrm{tr}\left({\gamma }_{j,t}{\Gamma }_{N,t}^{\left(j\right)}\right),$$

(23)

where the latter equation holds by the definition of the partial trace. Here ${\gamma }_{j,t}$ is identified with the operator in $L^2\left(X^N\right)$ acting on the jth variable and ${\Gamma }_{N,t}^{\left(j\right)}$ denotes the partial trace of ${\Gamma }_{N,t}$ with respect to all variables except for the jth.

Since evolutions (20) preserve the set of operators with the unit trace, (23) rewrites as

$${\alpha }_{N,j}\left(t\right)=\mathrm{tr}\left((\mathbf{1}-{\gamma }_{j,t}){\Gamma }_{N,t}\right)=\mathrm{tr}\left((\mathbf{1}-{\gamma }_{j,t}){\Gamma }_{N,t}^{\left(j\right)}\right).$$

(24)

Assuming that all controls $u_j$ and $v_j$ are given by identical feedback functions $u(t,\gamma )$, $v(t,\gamma )$ and that the initial conditions for Equation (19) is the tensor product of i.i.d. random vectors, the expectations $\mathbf{E}{\alpha }_N\left(t\right)=\mathbf{E}{\alpha }_{N,j}\left(t\right)$ are well defined (they do not depend on a particular choice of particles).

Expressions ${\alpha }_{N,j}$ can be linked with the traces by the following inequalities, due to Knowles and Pickl:

$${\alpha }_{N,j}\left(t\right)\le \mathrm{tr}|{\Gamma }_{N,t}^{\left(j\right)}-{\gamma }_{j,t}|\le 2\sqrt{2{\alpha }_{N,j}\left(t\right)},$$

(25)

see Lemma 2.3 from [42].

Theorem 1.

Let $H,\widehat{H}$ be self-adjoint operators in $L^2\left(X\right)$, with $\widehat{H}$ bounded, and $L\left(v\right)$ be a family of unitary operators depending Lipschitz continuously on v:

$$\parallel L\left(v_1\right)-L\left(v_2\right)\parallel \le {\varkappa }_L|v_1-v_2|.$$

(26)

Let A be a symmetric self-adjoint integral operator A in $L^2\left(X^2\right)$ with a Hilbert-Schmidt kernel, that is a kernel $A(x,y;x^{\prime },y^{\prime })$ such that

$${\parallel A\parallel }_{HS}^2={\int }_{X^4}{\left|A(x,y;x^{\prime },y^{\prime })\right|}^{2}dxdyd{x}^{\prime }d{y}^{\prime }<\infty ,$$

(27)

$$A(x,y;x^{\prime }{y}^{\prime })=A(y,x;y^{\prime },x^{\prime }),A(x,y;x^{\prime },y^{\prime })=\overline{A(x^{\prime },y^{\prime };x,y)}.$$

(28)

Let the functions $u(t,\gamma )$ and $v(t,\gamma )$ with values in bounded intervals $[-U,U]$ and $[-V,V]$ respectively be Lipschitz in the sense that

$$|u(t,\gamma )-u(t,\tilde{\gamma })|\le \varkappa \mathrm{tr}|\gamma -\tilde{\gamma }|,|v(t,\gamma )-v(t,\tilde{\gamma })|\le \varkappa \mathrm{tr}|\gamma -\tilde{\gamma }|.$$

(29)

Let ${\psi }_{j,t}$ be solutions to Equation (21) with i.i.d. initial conditions ${\psi }_{j,0}$, $\parallel {\psi }_{j,0}\parallel =1$. Let ${\Psi }_{N,t}$ be the solution to the N-particle Equation (19) with $H_u\left(N\right)$ given by (18) and with the initial condition

$${\Psi }_{N,0}(x_1,\cdots ,x_N)=\prod {\psi }_{j,0}\left(x_j\right).$$

Then

$$\mathbf{E}{\alpha }_N\left(t\right)\le {(exp\{(7\parallel A\parallel }_{HS}+12\varkappa (\parallel \widehat{H}\parallel +{\varkappa }_L+{\varkappa }_L^2\varkappa \left)\right)t\}-1)\frac{1}{\sqrt{N}}.$$

(30)

Proof. 

By Ito’s product rule for counting processes,

$$d{\alpha }_{N,j}\left(t\right)=-\mathrm{tr}\left(d{\Gamma }_{N,t}{\gamma }_{j,t}\right)-\mathrm{tr}\left({\Gamma }_{N,t}d{\gamma }_{j,t}\right)-\mathrm{tr}\left(d{\Gamma }_{N,t}d{\gamma }_{j,t}\right),$$

(31)

with the Ito product rule being $d{N}_t^{j}d{N}_t^i={\delta }_i^{j}d{N}_t^j$.

Let us denote by I and II the parts of the differential $d{\alpha }_{N,j}\left(t\right)$ that contain $L_j$ and, respectively, not.

Starting with II we obtain, denoting $A_j^{{\overline{\eta }}_t}$ the operator $A^{{\overline{\eta }}_t}$ acting on the jth variable, that

$$\begin{array}{cc}\hfill II& =i\mathrm{tr}\left([H_j+u_j(t,{\gamma }_{j,t}){\widehat{H}}_j+A_j^{{\overline{\eta }}_t},{\gamma }_{j,t}]{\Gamma }_{N,t}\right)dt+i\mathrm{tr}\left({\gamma }_{j,t}[H\left(N\right),{\Gamma }_{N,t}]\right)dt\hfill \\ & =i\mathrm{tr}\left([H_j+u_j(t,{\gamma }_{j,t}){\widehat{H}}_j+A_j^{{\overline{\eta }}_t},{\gamma }_{j,t}]{\Gamma }_{N,t}\right)dt+i\mathrm{tr}\left([{\gamma }_{j,t},H\left(N\right)]{\Gamma }_{N,t}\right)dt\hfill \\ & =-i\mathrm{tr}\left([H_j+u_j(t,{\gamma }_{j,t}){\widehat{H}}_j+A_j^{{\overline{\eta }}_t},q_{j,t}]{\Gamma }_{N,t}\right)dt+i\mathrm{tr}\left([H\left(N\right),q_{j,t}]{\Gamma }_{N,t}\right)dt\hfill \\ & =i\mathrm{tr}\left([H\left(N\right)-H_j-u_j(t,{\gamma }_{j,t}){\widehat{H}}_j-A_j^{{\overline{\eta }}_t},q_{j,t}]{\Gamma }_{N,t}\right)dt=I{I}_1+I{I}_2,\hfill \end{array}$$

with

$$I{I}_1=i\mathrm{tr}\left([\frac{1}{N}\sum _{m\ne j}{A}_{mj}-A_j^{{\overline{\eta }}_t},q_{j,t}]{\Gamma }_{N,t}\right)dt$$

and

$$I{I}_2=i\mathrm{tr}\left([(u_j(t,{\Gamma }_{N,t}^{\left(j\right)})-u_j(t,{\gamma }_{j,t})){\widehat{H}}_j,q_{j,t}]{\Gamma }_{N,t}\right)dt=B_{j,t}dt,$$

with

$$|B_{j,t}|\le 2|u_j(t,{\Gamma }_{N,t}^{\left(j\right)})-u_j(t,{\gamma }_{j,t})|\parallel \widehat{H}\parallel \parallel q_{j,t}{\Psi }_{N,t}\parallel$$

$$\le 2\varkappa \mathrm{tr}|{\Gamma }_{N,t}^{\left(j\right)}-{\gamma }_{j,t}|\parallel \widehat{H}\parallel \sqrt{{\alpha }_{N,j}\left(t\right)}$$

$$\le 4\varkappa \sqrt{2{\alpha }_{N,j}\left(t\right)}\parallel \widehat{H}\parallel \sqrt{{\alpha }_{N,j}\left(t\right)}=4\sqrt{2}\varkappa \parallel \widehat{H}\parallel {\alpha }_{N,j}\left(t\right),$$

where for the last inequality we used (25).

The term $I{I}_1$ was dealt with in [1] (proof of Theorems 3.1) yielding the estimate

$$\mathbf{E}|I{I}_1{|\le 7\parallel A\parallel }_{HS}\left(\mathbf{E}{\alpha }_N\left(t\right)+\frac{1}{\sqrt{N}}\right).$$

(32)

Let us turn to I. We have

$$\begin{array}{cc}\hfill I& =-\mathrm{tr}\left[(L\left(v_j(t,{\gamma }_{j,t})\right){\gamma }_{j,t}{L}^{\ast }\left(v_j(t,{\gamma }_{j,t})\right)-{\gamma }_{j,t}){\Gamma }_{N,t}\right]d{N}_t^j,\hfill \\ & -\mathrm{tr}\sum _k\left[(L_k\left(v_k(t,{\Gamma }_{N,t}^{\left(k\right)})\right){\Gamma }_{N,t}{L}_k^{\ast }\left(v_k(t,{\Gamma }_{N,t}^{\left(k\right)})\right)-{\Gamma }_{N,t}){\gamma }_{j,t}\right]d{N}_t^k\hfill \\ & -\mathrm{tr}\left[(L_j\left(v_j(t,{\Gamma }_{N,t}^{\left(j\right)})\right){\Gamma }_{N,t}{L}_j^{\ast }\left(v_j(t,{\Gamma }_{N,t}^{\left(j\right)})\right)-{\Gamma }_{N,t})(L_j\left(v_j(t,{\gamma }_{j,t})\right){\gamma }_{j,t}{L}_j^{\ast }\left(v_j(t,{\gamma }_{j,t})\right)-{\gamma }_{j,t})\right]d{N}_t^j.\hfill \end{array}$$

Since ${\gamma }_{j,t}$ and $L_k$ with $k\ne j$ commute, it follows that all terms with $k\ne j$ cancel. Taking into account other cancelation (arising from the unitarity of $L_j$) we obtain

$$I=\mathrm{tr}[{\Gamma }_{N,t}{\gamma }_{j,t}-L_j\left(v_j(t,{\Gamma }_{N,t}^{\left(j\right)})\right){\Gamma }_{N,t}{L}_j^{\ast }\left(v_j(t,{\Gamma }_{N,t}^{\left(j\right)})\right)L_j\left(v_j(t,{\gamma }_{j,t})\right){\gamma }_{j,t}{L}_j^{\ast }\left(v_j(t,{\gamma }_{j,t})\right)]d{N}_t^j.$$

If $L_j$ would be constant, this expression would vanish. In the present controlled version, some work is required. First of all, writing ${\gamma }_{j,t}=\mathbf{1}-q_{j,t}$ we obtain

$$I=C_{j,t}d{N}_t^j=C_{j,t}(d{M}_t^j+dt)$$

with

$$C_{j,t}=\mathrm{tr}[{\Gamma }_{N,t}{q}_{j,t}-L_j\left(v_j(t,{\Gamma }_{N,t}^{\left(j\right)})\right){\Gamma }_{N,t}{L}_j^{\ast }\left(v_j(t,{\Gamma }_{N,t}^{\left(j\right)})\right)L_j\left(v_j(t,{\gamma }_{j,t})\right)q_{j,t}{L}_j^{\ast }\left(v_j(t,{\gamma }_{j,t})\right)].$$

To make the calculations more transparent, let us omit indices at $v,\gamma ,q,\Gamma$. Thus

$$\begin{array}{cc}\hfill C_{j,t}& =\mathrm{tr}[\Gamma q-L\left(v(t,{\Gamma }_{N,t}^{\left(j\right)})\right)\Gamma L^{\ast }\left(v(t,{\Gamma }_{N,t}^{\left(j\right)})\right)L\left(v(t,\gamma )\right)q{L}^{\ast }\left(v(t,\gamma )\right)]\hfill \\ & =\mathrm{tr}[L\left(v(t,\gamma )\right)\Gamma L^{\ast }\left(v(t,\gamma )\right)L\left(v(t,\gamma )\right)q{L}^{\ast }\left(v(t,\gamma )\right)]\hfill \\ & -\mathrm{tr}[L\left(v(t,{\Gamma }_{N,t}^{\left(j\right)})\right)\Gamma L^{\ast }\left(v(t,{\Gamma }_{N,t}^{\left(j\right)})\right)L\left(v(t,\gamma )\right)q{L}^{\ast }\left(v(t,\gamma )\right)]\hfill \\ & =\mathrm{tr}\left[\left(L\left(v(t,\gamma )\right)\Gamma L^{\ast }\left(v(t,\gamma )\right)-L\left(v(t,{\Gamma }_{N,t}^{\left(j\right)})\right)\Gamma L^{\ast }\left(v(t,{\Gamma }_{N,t}^{\left(j\right)})\right)\right)L\left(v(t,\gamma )\right)q{L}^{\ast }\left(v(t,\gamma )\right)\right]\hfill \\ & =C_{j,t}^1+C_{j,t}^2,\hfill \end{array}$$

where

$$\begin{array}{cc}\hfill C_{j,t}^1& =\mathrm{tr}[(L\left(v(t,\gamma )\right)-L\left(v(t,{\Gamma }_{N,t}^{\left(j\right)})\right))\Gamma L^{\ast }\left(v(t,\gamma )\right)L\left(v(t,\gamma )\right)q{L}^{\ast }\left(v(t,\gamma )\right)]\hfill \\ & =\mathrm{tr}[(L\left(v(t,\gamma )\right)-L\left(v(t,{\Gamma }_{N,t}^{\left(j\right)})\right))\Gamma q{L}^{\ast }\left(v(t,\gamma )\right)]\hfill \\ & =\mathrm{tr}[\Gamma q{L}^{\ast }\left(v(t,\gamma )\right)(L\left(v(t,\gamma )\right)-L(v(t,{\Gamma }_{N,t}^{\left(j\right)})))],\hfill \\ \hfill C_{j,t}^2& =\mathrm{tr}\left[L\right(v(t,{\Gamma }_{N,t}^{\left(j\right)})\Gamma (L^{\ast }\left(v(t,\gamma )\right)-L^{\ast }\left(v(t,{\Gamma }_{N,t}^{\left(j\right)})\right)L\left(v(t,\gamma )\right)q{L}^{\ast }\left(v(t,\gamma )\right)].\hfill \end{array}$$

We can now estimate $C_{j,t}^1$ as $I{I}_2$ above yielding

$$\begin{array}{cc}\hfill |C_{j,t}^1|& \le |\left({\Psi }_{N,t}q,L^{\ast }\left(v(t,\gamma )\right)(L\left(v(t,\gamma )\right)-L\left(v(t,{\Gamma }_{N,t}^{\left(j\right)})\right)){\Psi }_{N,t}\right)|\hfill \\ & \le \parallel q{\Psi }_{N,t}\parallel \parallel L\left(v(t,\gamma )\right)-L\left(v(t,{\Gamma }_{N,t}^{\left(j\right)})\right)\parallel \le \sqrt{{\alpha }_{N,j}\left(t\right)}{\varkappa }_L\varkappa \mathrm{tr}|\gamma -{\Gamma }_{N,t}^{\left(j\right)}|\hfill \\ & \le 2\sqrt{2}{\varkappa }_L\varkappa {\alpha }_{N,j}\left(t\right).\hfill \end{array}$$

With $C_{j,t}^2$ yet another add-and-subtract manipulation is required. Namely,

$$C_{j,t}^2=\mathrm{tr}\left[\right(L^{\ast }\left(v(t,\gamma )\right)-L^{\ast }\left(v(t,{\Gamma }_{N,t}^{\left(j\right)})\right)L\left(v(t,\gamma )\right)q{L}^{\ast }\left(v(t,\gamma )\right)L(v(t,{\Gamma }_{N,t}^{\left(j\right)})\Gamma ]=C_{j,t}^{21}+C_{j,t}^{22}$$

with

$$\begin{array}{cc}\hfill C_{j,t}^{21}& =\mathrm{tr}\left[\right(L^{\ast }\left(v(t,\gamma )\right)-L^{\ast }\left(v(t,{\Gamma }_{N,t}^{\left(j\right)})\right)L\left(v(t,\gamma )\right)q{L}^{\ast }\left(v(t,\gamma )\right)L(v(t,\gamma )\Gamma ]\hfill \\ & =\mathrm{tr}[(L^{\ast }\left(v(t,\gamma )\right)-L^{\ast }\left(v(t,{\Gamma }_{N,t}^{\left(j\right)})\right)L\left(v(t,\gamma )\right)q\Gamma ],\hfill \\ \hfill C_{j,t}^{22}& =\mathrm{tr}\left[\right(L^{\ast }\left(v(t,\gamma )\right)-L^{\ast }\left(v(t,{\Gamma }_{N,t}^{\left(j\right)})\right)L\left(v(t,\gamma )\right)q{L}^{\ast }\left(v(t,\gamma )\right)(L(v(t,{\Gamma }_{N,t}^{\left(j\right)})-L\left(v(t,\gamma )\right))\Gamma ].\hfill \end{array}$$

The first term is estimated as above yielding

$$|C_{j,t}^{21}|\le 2\sqrt{2}{\varkappa }_L\varkappa {\alpha }_{N,j}\left(t\right).$$

And the second one is estimated as

$$|C_{j,t}^{22}|\le {\varkappa }_L^2{|v(t,{\Gamma }_{N,t}^{\left(j\right)})-v(t,\gamma )|}^2\le 8{\varkappa }_L^2{\varkappa }^2{\alpha }_{N,j}\left(t\right).$$

Thus

$$d{\alpha }_{N,j}\left(t\right)=I{I}_1+(B_{j,t}+C_{j,t})dt+C_{j,t}d{M}_t^j.$$

Therefore, since $M_t^j$ is a martingale and its differential does not contribute to the expectation, it follows that

$$d\mathbf{E}{\alpha }_N\left(t\right)\le {7\parallel A\parallel }_{HS}\left(\mathbf{E}{\alpha }_N\left(t\right)+\frac{1}{\sqrt{N}}\right)dt+(4\sqrt{2}\varkappa \parallel \widehat{H}\parallel +8\sqrt{2}{\varkappa }_L\varkappa +8{\varkappa }_L^2{\varkappa }^2)\mathbf{E}{\alpha }_N\left(t\right)dt.$$

Applying Gronwall’s inequality yields (30). □

5. Quantum MFG

Let us consider the quantum dynamic game of N players, where the dynamics of the density matrix ${\Gamma }_{N,t}$ is given by the controlled dynamics of type (20):

$$\begin{array}{cc}\hfill d{\Gamma }_{N,t}& =-i\sum _j[H_j+u_j(t,{\Gamma }_{N,t}^{\left(j\right)}){\widehat{H}}_j,{\Gamma }_{N,t}]-\frac{i}{N}\sum _{l<j\le N}[A_{lj},{\Gamma }_{N,t}]\hfill \\ & +\sum _j(L_j\left(v_j(t,{\Gamma }_{N,t}^{\left(j\right)})\right){\Gamma }_{N,t}{L}_j^{\ast }\left(v_j(t,{\Gamma }_{N,t}^{\left(j\right)})\right)-{\Gamma }_{N,t})d{N}_t^j.\hfill \end{array}$$

(33)

Assume as above that controls $u_j$ and $v_j$ of each jth player can be chosen from some bounded closed intervals $[-U,U]$ and $[-V,V]$ respectively, that the initial matrix is the product of iid states,

$${\Gamma }_{N,0}(x_1,\cdots ,x_n;y_1,\cdots ,y_N)=\prod _{j=1}^N{\psi }_j\left(x_j\right)\overline{{\psi }_j\left(y_j\right)},$$

and that the payoff of each player on the interval $[t,T]$ is given by the expression

$$P_j(t,W;u(.))={\int }_t^T\left(\mathrm{tr}\left(J_j{\Gamma }_{N,s}\right)-\frac{c}{2}{u}_j^2\left(s\right)\right)ds+\mathrm{tr}\left(F_j{\Gamma }_{N,T}\right),$$

(34)

where J and F are some operators in $L^2\left(X\right)$ expressing the current and the terminal costs of the agent, $J_j$ and $F_j$ denote their actions on the jth variable, constants $c\ge 0$ measure the cost of applying control u.

Remark 1. 

(i) We choose the simplest payoff function. Of course more general dependence on u, v is possible. As long as payoff is convex in u and v the results below are still valid. (ii) Also everything remains in force if only H or only L is controlled, that is either u or v is not present in all formulas.

Notice that by the property of the partial trace, the payoff (34) rewrites as

$$P_j(t,W;u(.))={\int }_t^T\left(\mathrm{tr}\left(J_j{\Gamma }_{N,s}^{\left(j\right)}\right)-\frac{c}{2}{u}_j^2\left(s\right)\right)ds+\mathrm{tr}\left(F_j{\Gamma }_{N,T}^{\left(j\right)}\right),$$

(35)

so that it really depends explicitly only on the individual partial traces ${\Gamma }_{N,t}^{\left(j\right)}$, which can be considered as quantum analogs of the positions of classical particles.

Let us stress again that, after all equations arising from physics are written, our quantum dynamic N-player game can be formulated in fully classical terms. Namely, the goal of each jth player is to maximise the expectation of payoff (35) under the evolution (33) depending on all controls $u=\left(u_j\right)$. The information available to the jth player is the ‘position’ of jth player, which is the partial trace ${\Gamma }_{N,t}^{\left(j\right)}$, and thus the actions of jth player are chosen among the feedback strategies $u_j$ that are measurable functions $u_j(t,{\Gamma }_{N,t}^{\left(j\right)})$. An additional technical assumption that we are using in the analysis below is that the class of feedback strategies is reduced to Lipschitz continuous functions of partial traces. Therefore both the information setting and technical assumptions are slightly different from the simpler setting of two-player game of Section 3, where players were assumed to define their strategies on the basis of the whole state (not a partial trace). The restriction to partial traces is necessary to uncouple the dynamics in the limit of $N\to \infty$.

The limiting evolution of each player can be expected to be described by the equations

$$\begin{array}{cc}\hfill d{\gamma }_{j,t}& =-i[H+u_j(t,{\gamma }_{j,t})\widehat{H},{\gamma }_{j,t}]dt-i[A^{\overline{{\eta }_t}},{\gamma }_{j,t}]dt\hfill \\ \hfill & +(L\left(v_j(t,{\gamma }_{j,t})\right){\gamma }_{j,t}{L}^{\ast }\left(v_j(t,{\gamma }_{j,t})\right)-{\gamma }_{j,t})d{N}_t^j,\hfill \end{array}$$

(36)

with

$${\eta }_t(x,y)=\underset{N\to \infty }{lim}\frac{1}{N}\sum _{j=1}^N{\gamma }_{j,t}(x,y),$$

and with payoffs given by

$$P_j(t,W;u(.))={\int }_t^T\left(\mathrm{tr}\left(J{\gamma }_{j,s}\right)-\frac{c}{2}{u}_j^2\left(s\right)\right)ds+\mathrm{tr}\left(F{\gamma }_{j,T}\right).$$

(37)

For pure states ${\gamma }_{j,t}={\psi }_{j,t}\otimes {\overline{\psi }}_{j,t}$ this payoff turns to

$$P_j(t,W;u(.))={\int }_t^T\left(({\psi }_{j,t},J{\psi }_{j,t})-\frac{c}{2}{u}_j^2\left(s\right)\right)ds+({\psi }_{j,T},F{\psi }_{j,T}).$$

(38)

Let us say that the pair of functions ${(u,v)}_t^{MFG}\left(\gamma \right)={(u,v)}^{MFG}(t,\gamma )$ with $t\in [0,T]$ and $\gamma$ from the set of density matrices in $L^2\left(X\right)$, and ${\eta }_t^{MFG}(x,y)$ with $x,y\in X$, $t\in [0,T]$, solve the limiting MFG problem if (i) ${(u,v)}_t\left(\gamma \right)$ is an optimal feedback strategy for the stochastic control problem (36), (37) under the fixed function ${\eta }_t={\eta }_t^{MFG}$ and (ii) ${\eta }_t^{MFG}$ arises from the solution of (36) under fixed ${(u,v)}_t={(u,v)}_t^{MFG}$.

Theorem 2.

Let the conditions on $H,L,A$ from Theorem 1 hold. Assume that the pair ${(u,v)}_t^{MFG}\left(\gamma \right)$ and ${\eta }_t^{MFG}(x,y)$ solves the limiting MFG problem and moreover $u_t^{MFG}$ is Lipschitz in the sense of inequality (29). Then the strategies

$${(u,v)}_j(t,{\Gamma }_{N_t})={(u,v)}_t^{MFG}\left({\Gamma }_{N,t}^{\left(j\right)}\right),$$

form a symmetric ϵ-Nash equilibrium for the N-agent quantum game described by (33) and (34), where strategies of all players are sought among measurable controls $(u,v)(t,\gamma )$ that depend Lipschitz in γ in the sense of inequality (29), with $ϵ=C\left(T\right)N^{-1/4}$, $C\left(T\right)$ depending on ${\parallel A\parallel }_{HS}$, $\parallel \widehat{H}\parallel$, ϰ, ${\varkappa }_L$.

Proof. 

Assume that all players, except for one of them, say the first one, are playing according to the MFG strategy ${(u,v)}^{MFG}(t,{\Gamma }_{N,t}^{\left(j\right)})$, $j>1$, and the first player is following some other strategy $(\tilde{u},\tilde{v})(t,{\Gamma }_{N,t}^{\left(1\right)})$. By the law of large numbers (which is not affected by a single deviation), all ${\eta }_t^j$ are equal and are given by the formula ${\eta }_t=\mathbf{E}{\gamma }_{j,t}$ for all $j>1$. Moreover, $\mathbf{E}{\alpha }_{N,j}\left(t\right)=\mathbf{E}{\alpha }_N\left(t\right)$ are the same for all $j>1$.

Following the proof of Theorem 1 we obtain

$${\dot{\alpha }}_{N,j}\left(t\right)=I+I{I}_1+I{I}_2$$

(39)

with the same $I,I{I}_1,I{I}_2$, as in the proof of Theorem 1, though $(u,v)$ being ${(u,v)}^{MFG}(t,{\Gamma }_{N,t}^{\left(j\right)})$, $j>1$, and $(\tilde{u},\tilde{v})(t,{\Gamma }_{N,t}^{\left(1\right)})$ for $j=1$. Looking first at $j>1$ we note that up to an additive correction of magnitude not exceeding ${4\parallel A\parallel }_{HS}/N$ expression $I{I}_1$ can be substituted by the expression

$$i\mathrm{tr}\left([\frac{1}{N}\sum _{m\ne j,1}{A}_{mj}-A_j^{{\overline{\eta }}_t},q_{j,t}]{\gamma }_{N,t}\right),$$

which is then dealt with exactly as in the proof of Theorem 1 (with $N-1$ instead of N) yielding the same estimate (30) (with a corrected multiplier) for $\mathbf{E}{\alpha }_N\left(t\right)=\mathbf{E}{\alpha }_{N,j}\left(t\right)$, $j>1$, that is

$$\mathbf{E}{\alpha }_N\left(t\right)\le {(exp\{(7\parallel A\parallel }_{HS}+12\varkappa (\parallel \widehat{H}\parallel +{\varkappa }_L+{\varkappa }_L^2\varkappa \left)\right)t\}-1)\frac{1}{\sqrt{N}}{(1+4\parallel A\parallel }_{HS}).$$

(40)

The same estimate is obtained for $\mathbf{E}{\alpha }_{N,1}\left(t\right)$ (even without the correcting term ${4\parallel A\parallel }_{HS}$) yielding

$$\mathbf{E}{\alpha }_{N,j}\left(t\right)\le C\left(T\right)N^{-1/2}$$

for all j and a constant $C\left(T\right)$ depending on ${\parallel A\parallel }_{HS},\varkappa ,{\varkappa }_L,\parallel \widehat{H}\parallel$.

We can now compare the expected payoffs (35) received by the players in the N-player quantum game with the expected payoff (37) received in the limiting game. For each jth player the difference is bounded by

$$\mathbf{E}{\int }_t^T|\mathrm{tr}\left(J({\Gamma }_{N,s}^{\left(j\right)}-{\gamma }_{j,s})\right)|ds+\mathbf{E}|\mathrm{tr}\left(F({\Gamma }_{N,T}^{\left(j\right)}-{\gamma }_{j,T})\right)|.$$

Since,

$$|\mathrm{tr}\left(J({\Gamma }_{N,s}^{\left(j\right)}-{\gamma }_{j,s})\right)|\le \parallel J\parallel \mathrm{tr}|{\Gamma }_{N,s}^{\left(j\right)}-{\gamma }_{j,s}|,$$

and by (25),

$$\mathrm{tr}|{\Gamma }_{N,s}^{\left(j\right)}-{\gamma }_{j,s}|\le 2\sqrt{2{\alpha }_{N,j}\left(s\right)},$$

it follows that the expectation of the difference of the payoffs is bounded by

$$2\sqrt{2}(\parallel J\parallel T+\parallel F\parallel )\underset{t}{sup}\mathbf{E}\sqrt{{\alpha }_{N,j}\left(t\right)}$$

$$\le 2\sqrt{2}(\parallel J\parallel T+\parallel F\parallel )\underset{t}{sup}\sqrt{\mathbf{E}{\alpha }_{N,j}\left(t\right)}\le (\parallel J\parallel T+\parallel F\parallel )C\left(T\right)N^{-1/4},$$

with a constant $C\left(T\right)$ depending on ${\parallel A\parallel }_{HS},\varkappa ,{\varkappa }_L,\parallel \widehat{H}\parallel$.

But by the assumption of the Theorem, ${(u,v)}_t^{MFG}$ is the optimal choice for the limiting optimization problem. Hence the claim of the theorem follows. □

6. Discussion

The problem of proving existence or uniqueness for the solution of the limiting MFG on manifold seems to be nontrivial. We suggest it as an interesting open problem.

Let us give a bit more detail for the simplest case of two-dimensional Hilbert space (a qubit), as in Section 3.

When there is no control v (that is, operator L is constant) and there is no free (uncontrolled) part of the Hamiltonian, the limiting Equation (21) simplify to the equation (omitting indices j and t for simplicity)

$$d\psi =-i[u\widehat{H}+A^{\overline{\eta }}]\psi dt+(L\psi -\psi )d{N}_t.$$

(41)

Moreover, $\psi$ has only two coordinates: $\psi =({\psi }_0,{\psi }_1)$. Using Ito’s rule as in Section 3, we find the equation for $w={\psi }_1/{\psi }_0$:

$$dw=i[u(w{\left(\widehat{H}W\right)}_0-{\left(\widehat{H}W\right)}_1)+w{\left(A^{\overline{\eta }}W\right)}_0-{\left(A^{\overline{\eta }}W\right)}_1]dt+({\left(LW\right)}_1-w{\left(LW\right)}_0)d{N}_t,$$

(42)

where $W=(w_0,w_1)=(1,w)$.

The most common interaction operator between qubits is the operator describing the possible exchange of photons, $A=a_1^{\ast }{a}_2+a_2^{\ast }{a}_1$, with the annihilation operators $a_1$ and $a_2$ of the two atoms. This interaction is given by the tensor $A(j,k;m,n)$ such that $A(1,0;0,1)=A(0,1;1,0)=1$ with other elements vanishing. Hence $A_{10}^{\eta }={\eta }_{01}$, $A_{01}^{\eta }={\eta }_{10}$, with other elements vanishing. Let us take also the simplest possible L: $L={\sigma }_3$—the third Pauli matrix. Then Equation (42) simplifies to

$$dw=i[u(w{\left(\widehat{H}W\right)}_0-{\left(\widehat{H}W\right)}_1)+{\overline{\eta }}_{10}{w}^2-{\overline{\eta }}_{01}]dt-2wd{N}_t.$$

(43)

If $\widehat{H}$ is diagonal with diagonal elements $h_0,h_1$, this turns to

$$dw=i[uw(h_0-h_1)+{\overline{\eta }}_{10}{w}^2-{\overline{\eta }}_{01}]dt-2wd{N}_t.$$

(44)

In this simplest case, choosing $h_0-h_1=1$ and $c=0$ in payoff (38), we obtain the HJB equation for the individual control in the form

$$\begin{array}{c}\frac{\partial S}{\partial t}+\underset{u}{max}\left(ux\frac{\partial S}{\partial y}-uy\frac{\partial S}{\partial x}\right)+\frac{(W,JW)}{1+{\left|w\right|}^2}\\ +\frac{\partial S}{\partial y}Re({\overline{\eta }}_{10}{w}^2-{\overline{\eta }}_{01})]-\frac{\partial S}{\partial x}Im({\overline{\eta }}_{10}{w}^2-{\overline{\eta }}_{01})]+(S(-x,-y)-S(x,y))=0.\end{array}$$

(45)

Already this equation on the complex plane $\mathbf{C}$, describing optimal control for the individual quantum feedback control in a qubit, is quite nonstandard. And to deal with the corresponding forward-backward system one needs not only its well-posedness in a certain generalized sense, but some continuous dependence on parameters. May be some method from [43] or [28] can be used to get insight into this problem.

As a future research direction it is worth mentioning the general development of the theory of the limiting classical mean-field games, which are mean-field games on infinite dimensional curvilinear manifolds based on Markov processes with jumps, highly fascinating and nontrivial objects. Of course usual questions of classical mean-field games on the connection between stationary and time dependent solutions are fully open here, as well as the theory of the corresponding master equation. On the other hand, quantum dynamic games of finite number of players (touched upon in Section 3) lead to new nonlinear functional-differential equations on manifolds of Hamilton-Jacobi or Isaacs type, which are also worthy of proper analysis.