A Branch-and-Bound Algorithm for Polymatrix Games ϵ-Proper Nash Equilibria Computation

Abstract

When several Nash equilibria exist in the game, decision-makers need to refine their choices based on some refinement concepts. To this aim, the notion of a $ϵ$-proper equilibria set for polymatrix games is used to develop 0–1 mixed linear programs and compute $ϵ$-proper Nash equilibria. A Branch-and-Bound exact arithmetics algorithm is proposed. Experimental results are provided on polymatrix games randomly generated with different sizes and densities.

1. Introduction

For quite a long time, Nash equilibrium solutions have been depicted as the most desirable outcome for games [1]. Avis and Fukuda [2], Audet et al. [3], and Audet, Belhaiza, and Hansen [4] proposed algorithmic approaches to enumerate the extreme Nash equilibria of two-player games. Daskalakis et al. [5] and Hazan and Krauthgamer [6] focused on the complexity of Nash equilibrium computation for two-player games. Etessami and Yannakis [7] focused on the complexity of approximated Nash equilibria computation for three and more player finite games.

Yanovskaya [8] used the Complementary Pivoting Method to compute Nash equilibria for polymatrix games. Howson [9], Eaves [10], and Howson and Rosenthal [11] used a similar approach. In particular, Quintas [12] demonstrated that a Nash equilibria set is a union of a finite number of convex polytopes for polymatrix games. Miller and Zucker [13] reduced the polymatrix game equilibrium computation problem to a copositive-plus linear complementarity problem (LCP) that requires a single-time application of Lemke’s algorithm [14]. On the same line, Wilson [15] adapted the Lemke and Howson algorithm [16] originally designed for two-player games to n-player games. Govindan and Wilson [17] developed polymatrix game sequences to approximate n-player games Nash equilibrium solutions. The enumeration of all polymatrix game extreme Nash equilibria was addressed in [4]. Strekalovskii and Ekhbat [18] reduced the search for Nash equilibria to a simple optimization problem. They also brought a simple proof on the existence of a Nash equilibrium for polymatrix games.

Meanwhile, an important concern emerged as decision-makers facing multiple Nash equilibria needed to refine their choices using more strict criteria. Although many refinement concepts were introduced, results on polymatrix games are quite limited. Papadimitriou and Roughgarden [19] demonstrated that the computation of a polymatrix game correlated equilibrium may be achieved in polynomial time. Belhaiza [20] demonstrated that perfect equilibria sets for polymatrix games are also unions of finite numbers of convex polytopes, possibly disjoint.

The proper refinement of the Nash equilibrium concept was introduced by Myerson [21] with the idea that reasonable players should try much harder to avoid the most costly mistakes than they should try to avoid the less costly ones. In fact, the proper refinement defines more stability conditions related to small rationality imperfections also called “trembling-hand perfection”. Myerson [21] demonstrated that there is always at least one proper equilibrium for any game in strategic form. For bimatrix games, Belhaiza et al. [22] introduced the concept of a set of $ϵ$-proper equilibria. They used sequences of quadratic programs to validate or reject the properness of a given Nash equilibrium point.

This paper brings theoretical and computational results on proper Nash equilibria for polymatrix games. To the best of our knowledge, no such results were published before. The sections of this paper are ordered as follows. Section 2 gives a formal description of polymatrix games. Section 3 recalls the definition of $ϵ$-proper equilibria and proposes a novel formulation of the $ϵ$-properness conditions. Section 4 proposes mixed 0–1 quadratic and linear formulations to detect $ϵ$-proper and non-proper Nash equilibria, and shows on a three-player polymatrix game how $ϵ$-proper sequences of equilibria can be analytically generated. Section 5 presents a Branch-and-Bound algorithm for polymatrix games $ϵ$-proper equilibria computation. The algorithm is based on a mixed 0–1 linear program combining Nash equilibrium primal and dual conditions and the $ϵ$-properness conditions. Finally, computational results on polymatrix games randomly generated are presented in Section 6.

2. Polymatrix Games

A polymatrix is a strategic interaction of $n\ge 2$ players in a non-cooperative and normal setting, where the payoff of each player is the total sum of a series of partial payoffs. Particularly, when $n=2$, such interaction defines a bimatrix game. With $N=\{1,\dots ,n\}$ as the set of all players, every player $i\in N$ formulates his choice over a set of strategies $S_i=\{s_i^1,....,s_i^{m_i}\}$, where $|S_i|=m_i$ is finite and all element strategies are pure. A given player i’s partial payoff matrix with respect to a player j’s strategic decisions is a $m_i\times m_j$ matrix denoted by $A_{ij}=(a_{ij}^{kl})$. A partial payoff $a_{ij}(s_i^k,s_j^l)$ is rewarded to player i, if player i selects his strategy $s_i^k$ and player j selects his strategy $s_j^l$. We define $A_{i.}=[A_{i1}..A_{ij}..A_{in}]$ the matrix of player i’s payoffs with respect to all players. The overall payoff for player i with respect to any choice $(s_1^k,\dots ,s_n^l)$ in pure strategies of the n players, is

$$A_i(s_1^k,\dots ,s_n^l)=\sum _{j\ne i}{a}_{ij}(s_i^k,s_j^l).$$

Hence, a given player i’s payoff with respect to player j’s decisions is not related to the remaining players’ choices. Every player i sets a mixed strategy vector $X_i$ over his set of pure strategies and seeks to maximize his total game payoff. The vector $X_i$ is such that ${\left(X_i\right)}^T=(x_i^1,....,x_i^{m_i})$, where for every $k\in \{1,\dots ,m_i\}$, $x_i^k$ represents the probability that player i selects his strategy $s_i^k\in S_i$. A given player i’s mixed strategies vector $X_i$ belongs to the set $\widetilde{S_i}=\{X_i:e_{m_i}^t{X}_i=1,X_i\ge 0\},$ where $e_{m_i}^t$ is a one-row matrix with all entries set to 1. The game total payoff for player i is

$${\alpha }_i\left(X\right)={\left(X_i\right)}^T\sum _{j\ne i}{A}_{ij}{X}_j=\sum _{j\ne i}\sum _{k=1}^{m_i}\sum _{l=1}^{m_j}{a}_{ij}^{kl}{x}_i^k{x}_j^l.$$

Polymatrix games are used in some applications where the players’ payoff matrices are additive. For instance, Belhaiza et al. [23] used a polymatrix game to model a manager–controller–board of directors’ conflict. As for other strategic form games, a polymatrix game has indeed at least one Nash equilibrium as shown in [1]. We can define a Nash equilibrium $\widehat{X}=({\widehat{X}}_1,\dots ,{\widehat{X}}_n)$ as a n-tuple of mixed strategies such that for any other n-tuple $X=({\widehat{X}}_1,..,{\widehat{X}}_{i-1},X_i,{\widehat{X}}_{i+1},..,{\widehat{X}}_n)$ the following condition is satisfied

$${({\widehat{X}}_i)}^T\sum _{j\ne i}{A}_{ij}{\widehat{X}}_j\ge {\left(X_i\right)}^T\sum _{j\ne i}{A}_{ij}{\widehat{X}}_j,\mathrm{for}i\in N.$$

Differently stated, a Nash equilibrium is a situation where a given player i’s payoff with respect to all remaining players is maximized, and similarly for all other players in N. A polymatrix game set of Nash equilibria $NE$ is the union of a finite number of polytopes, also referred to as maximal Nash subsets. The set of extreme Nash equilibria is defined as the set of all different vertices of all the maximal Nash subsets. Any extreme Nash equilibrium is an extreme point of at least one of the maximal Nash subsets. Hence, a subset $T\subset \mathit{NE}$ is considered as a Nash subset, if and only if for every pair of elements $({\widehat{X}}_i,{\widehat{Y}}_i)$, such that $({\widehat{X}}_1,..,{\widehat{X}}_i,..,{\widehat{X}}_n)\in T$ and $({\widehat{Y}}_1,..,{\widehat{Y}}_i,..,{\widehat{Y}}_n)\in T$, we have $({\widehat{X}}_1,..,{\widehat{Y}}_i,..,{\widehat{X}}_n)\in T$ and $({\widehat{Y}}_1,..,{\widehat{X}}_i,..,{\widehat{Y}}_n)\in T$, for all $i\in N$. Hence, ${\widehat{X}}_i$ and ${\widehat{Y}}_i$ are interchangeable. A given Nash subset T is considered as maximal if it is shown that it is not properly contained in any another Nash subset [24].

3. Set of $\mathbf{ϵ}$-Proper Equilibria

The proper refinement originates from the assumption that reasonable players should try much harder to avoid more costly mistakes than they should try to avoid less costly ones. While it is well known that any proper equilibrium is always perfect, a perfect Nash equilibrium may be non-proper. Given a finite game in strategic form, Myerson [21] defines a proper equilibrium as the limit of an infinite sequence of $ϵ$-proper equilibria, when $ϵ$ converges to zero. We here apply Myerson’s definition of the $ϵ$-proper equilibrium to polymatrix games. Let us first note $A_{i·}^k$ and $A_{i·}^q$ as the $k^{th}$ and $q^{th}$ rows of the payoff matrix $A_{i·}$ of player i, respectively.

Definition 1.

A polymatrix game profile $\widehat{X}=({\widehat{X}}_1,\dots ,{\widehat{X}}_n)$ is ϵ-proper, for $ϵ>0$, if for all $i\in N$:

$$\mathit{if}\sum _{j\ne i}{A}_{ij}^k{\widehat{X}}_j<\sum _{j\ne i}{A}_{ij}^q{\widehat{X}}_j,\mathit{then}$$

$${\widehat{x}}_{ik}\le ϵ{\widehat{x}}_{iq},k,q\in \left\{1,2,\dots ,m_i\right\},$$

(1)

$$\mathit{and}{\widehat{x}}_{ik}>0,k\in \left\{1,2,\dots ,m_i\right\}.$$

(2)

For a given $ϵ\ge 0$ and $\sigma \ge 0$, we now introduce the following set ${\Omega }_{ϵ}^{\sigma }$ such that:

$$\begin{array}{cc}{\Omega }_{ϵ}^{\sigma }=\hfill & \{X=(X_1,\dots ,X_n):\forall i\in N,\exists u_i\mathrm{and}L\mathrm{such}\mathrm{that}\hfill \\ & e_{m_i}^t{X}_i=1,\hfill \\ & \sigma \le x_{ik},\forall k\in \left\{1,2,\dots ,m_i\right\},\hfill \\ & \mathrm{and}\forall k,q\in \left\{1,2,\dots ,m_i\right\},k\ne q:\hfill \\ & {\displaystyle \sum _{j\ne i}{A}_{ij}^k{X}_j\le \sum _{j\ne i}{A}_{ij}^q{X}_j+L{u}_{qk}^i,}\hfill \\ & x_{ik}+u_{kq}^i\le ϵx_{iq}+1,\hfill \\ & u_{kq}^i+u_{qk}^i\le 1,\mathrm{with}k<q,\hfill \\ & u_{kq}^i\in \left\{0,1\right\}.\}\hfill \end{array}$$

In the above, $u_{kq}^i$ is a binary variable, and the parameter $L\in {\Re }^{+}$ is set to a sufficiently large value. The following Proposition 1 gives a novel formulation of the $ϵ$-properness conditions and shows that each element of ${\Omega }_{ϵ}^{\sigma }$ is an $ϵ$-proper equilibrium.

Proposition 1.

If $X=(X_1,\dots ,X_n)\in {\Omega }_{ϵ}^{\sigma }$, for $ϵ>0$ and $\sigma >0$, then X is an ϵ-proper equilibrium.

Proof. 

Assume that $X=(X_1,\dots ,X_n)\in {\Omega }_{ϵ}^{\sigma }$, for $ϵ>0$ and $\sigma >0$. Let k and q be indices in $\left\{1,2,\dots ,m_i\right\}$, such that $k\ne q$. The inequalities $u_{kq}^i+u_{qk}^i\le 1$ ensure that the binary combination $u_{kq}^i=1$ and $u_{qk}^i=1$ is impossible. Moreover,

-

if $u_{kq}^i=0$ and $u_{qk}^i=0$, then ${\displaystyle \sum _{j\ne i}{A}_{ij}^k{X}_j=\sum _{j\ne i}{A}_{ij}^q{X}_j}$;

-

if $u_{kq}^i=1$, then $ϵx_{ik}\le x_{ik}\le ϵx_{iq}\le x_{iq}$ implies that $x_{iq}\ge ϵx_{ik}$ and $u_{qk}^i=0$, thus ${\displaystyle \sum _{j\ne i}{A}_{ij}^k{X}_j\le \sum _{j\ne i}{A}_{ij}^q{X}_j}$.

It follows that conditions (1) are satisfied. Finally, with $0<\sigma \le x_{ik}$, for all $k\in \left\{1,2,\dots ,m_i\right\}$, conditions (2) are satisfied. □

Proposition 2 states that every $ϵ$-proper equilibrium belongs to ${\Omega }_{ϵ}^{\sigma }$, for every sufficiently small value of $\sigma$.

Proposition 2.

If $X=(X_1,\dots ,X_n)$ is an ϵ-proper equilibrium for a given $ϵ>0$, there exists a real number $\overline{\sigma }>0$, such that $(X_1,\dots ,X_n)\in {\Omega }_{ϵ}^{\sigma }$, for $0\le \sigma \le \overline{\sigma }$.

Proof. 

If a profile $X=(X_1,\dots ,X_n)$ is an $ϵ$-proper equilibrium for some $ϵ>0$, for all $i\in N$, conditions (1) can be reformulated using the binary variables $u_{kq}^i$, for all $k\ne q\in \left\{1,2,\dots ,m_i\right\}$:

$$\begin{array}{c}\mathrm{If}\left\{\begin{array}{c}\sum _{j\ne i}{A}_{ij}^k{X}_j<\sum _{j\ne i}{A}_{ij}^q{X}_j,\hfill \\ x_{ik}\le ϵx_{iq},\hfill \end{array}\right.\hfill \end{array}\mathrm{then}\left\{\begin{array}{c}{\displaystyle \sum _{j\ne i}{A}_{ij}^k{X}_j\le \sum _{j\ne i}{A}_{ij}^q{X}_j+L{u}_{qk}^i,}\hfill \\ x_{ik}+u_{kq}^i\le ϵx_{iq}+1,\hfill \\ u_{qk}^i=0,\hfill \\ u_{kq}^i=1.\hfill \end{array}\right.$$

$$\mathrm{If}\left\{\begin{array}{c}{\displaystyle \sum _{j\ne i}{A}_{ij}^q{X}_j<\sum _{j\ne i}{A}_{ij}^k{X}_j,}\hfill \\ x_{iq}\le ϵx_{ik},\hfill \end{array}\right.\mathrm{then}\left\{\begin{array}{c}{\displaystyle \sum _{j\ne i}{A}_{ij}^q{X}_j\le \sum _{j\ne i}{A}_{ij}^k{X}_j+L{u}_{kq}^i,}\hfill \\ x_{iq}+u_{qk}^i\le ϵx_{ik}+1,\hfill \\ u_{kq}^i=0,\hfill \\ u_{qk}^i=1.\hfill \end{array}\right.$$

$$\mathrm{If}\left\{\begin{array}{c}{\displaystyle \sum _{j\ne i}{A}_{ij}^k{X}_j=\sum _{j\ne i}{A}_{ij}^q{X}_j}\hfill \\ x_{ik}\le 1,\hfill \\ x_{iq}\le 1,\hfill \end{array}\right.\mathrm{then}\left\{\begin{array}{c}{\displaystyle \sum _{j\ne i}{A}_{ij}^k{X}_j\le \sum _{j\ne i}{A}_{ij}^q{X}_j+L{u}_{qk}^i,}\hfill \\ x_{ik}+u_{kq}^i\le ϵx_{iq}+1,\hfill \\ \\ {\displaystyle \sum _{j\ne i}{A}_{ij}^q{X}_j\le \sum _{j\ne i}{A}_{ij}^k{X}_j+L{u}_{kq}^i,}\hfill \\ x_{iq}+u_{qk}^i\le ϵx_{ik}+1,\hfill \\ \\ u_{kq}^i=0,\hfill \\ u_{qk}^i=0.\hfill \end{array}\right.$$

Finally, conditions (2) ensure the existence of a $\overline{\sigma }>0$, such that $\overline{\sigma }\le x_{ik}$, for all $k\in \left\{1,2,\dots ,m_i\right\}$ and for all $i\in N$. Then, for every $\sigma$ such that $0<\sigma \le \overline{\sigma }$:

$$\begin{array}{cc}\sigma \le x_{ik},\hfill & forallk\in \left\{1,2,\dots ,m_i\right\}.\hfill \end{array}$$

Thus, $X=(X_1,\dots ,X_n)\in {\Omega }_{ϵ}^{\sigma }$ for any $\sigma$, such that $0<\sigma \le \overline{\sigma }$. □

The set ${\Omega }_{ϵ}^{\sigma }$ can now be called a set of $ϵ$-proper equilibria. Myerson [21] also showed that for every finite game in strategic form the set of proper equilibria is non-empty. Hence, such a sequence of $ϵ$-proper equilibria exists for every polymatrix game. Finding such sequences is not an easy task. While we proposed a 0–1 mixed quadratic programming approach to generate such sequences for bimatrix games [22], we now extend our approach to polymatrix games.

4. Computation of $\mathbf{ϵ}$ Proper Equilibria

Trying analytically to obtain such sequences of $ϵ$-proper equilibria would be unsuccessful if the equilibrium is non-proper. Therefore, one should try to detect equilibrium profiles that could help generate such sequences, and eliminate equilibrium profiles offering no possibility. To do so, we propose 0–1 mixed quadratic formulations that help detect the existence of $ϵ$-proper equilibria. To overcome the numerical noise that may appear while performing quadratic optimization with infinitesimal $ϵ$, we propose a 0–1 mixed linear formulation and implement it using exact arithmetics. We also show how the analytical computation of $ϵ$-proper equilibria could be performed, provided that $ϵ$-proper equilibria close to a Nash equilibrium are detected.

4.1. Mixed 0–1 Quadratic Formulations

We now propose a sequence of parametrized 0–1 mixed quadratic programs with optimal solutions defining a sequence of $ϵ$-proper equilibria while $\sigma$ converges to 0.

Proposition 3.

A Nash equilibrium $\widehat{X}=({\widehat{X}}_1,\dots ,{\widehat{X}}_n)$ is proper if and only if the following 0–1 mixed quadratic program $\left({\mathfrak{Q}}_{⸧}\right)$ is feasible for all $\overline{\sigma }>0$, and if ${\displaystyle \underset{\sigma \to 0^{+}}{lim}ϵ\left(\sigma \right)=0}$.

$$\left({\mathfrak{Q}}_{⸧}\right)\begin{array}{cc}ϵ\left(\sigma \right)=minϵ\hfill & \\ subjectto\hfill & \\ {\widehat{x}}_{ik}-ϵ\le x_{ik}\le {\widehat{x}}_{ik}+ϵ,\forall i\in N,k\in \left\{1,2,\dots ,m_i\right\},\hfill & \\ (X_1,\dots ,X_n)\in {\Omega }_{ϵ}^{\sigma },\hfill & \\ 0\le ϵ\le 1.\hfill & \end{array}$$

Proof. 

Assume that $(X_1\left(\sigma \right),\dots ,X_n\left(\sigma \right),ϵ\left(\sigma \right))$ is the optimal solution of $\left({\mathfrak{Q}}_{⸧}\right)$ for a known Nash equilibrium $\widehat{X}=({\widehat{X}}_1,\dots ,{\widehat{X}}_n)$. Proposition 1 asserts that $(X_1\left(\sigma \right),\dots ,X_n\left(\sigma \right))$ is an $ϵ\left(\sigma \right)$-proper equilibrium. In addition, the convergence conditions are here reformulated through the minimization of $ϵ$ such that ${\widehat{x}}_{ik}-ϵ\le x_{ik}\le {\widehat{x}}_{ik}+ϵ,\&k\in \left\{1,2,\dots ,m_i\right\},\&i\in N,$ to make sure that the $ϵ$-proper equilibrium converges towards $({\widehat{X}}_1,\dots ,{\widehat{X}}_n)$.

Hence, if the 0–1 mixed quadratic program $\left({\mathfrak{Q}}_{⸧}\right)$ is feasible for all $\sigma >0$ and converging to 0, we may conclude from proposition (2) that there always exists an $ϵ$-proper equilibrium $(X_1,\dots ,X_n)\in {\Omega }_{ϵ}^{\sigma }$.

In addition, if the Nash equilibrium $\widehat{X}=({\widehat{X}}_1,\dots ,{\widehat{X}}_n)$ is proper, the optimal objective value $ϵ\left(\sigma \right)$ converges necessarily towards 0, when $\sigma >0$ converges to 0, to force the solution $(X_1\left(\sigma \right),\dots ,X_n\left(\sigma \right))$ to converge towards $({\widehat{X}}_1,\dots ,{\widehat{X}}_n)$ simultaneously. We can also add that $ϵ\left(0\right)=0$ in this case.

Otherwise, if $ϵ\left(\sigma \right)$ does not converge towards 0, when $\sigma >0$ converges to 0, such $ϵ$-proper sequence $(X_1\left(\sigma \right),\dots ,X_n\left(\sigma \right))$ does not even exist, when $ϵ$ converges to 0. In this case, it is obvious that the given Nash equilibrium $\widehat{X}$ cannot be proper. □

In conclusion, if $ϵ\left(\sigma \right)$ converges to 0, while $\sigma >0$ converges to 0, it is possible to obtain a sequence $(X_1\left(\sigma \right),\dots ,X_n\left(\sigma \right))$$ϵ$-proper converging to $({\widehat{X}}_1,\dots ,{\widehat{X}}_n)$, when $ϵ\left(\sigma \right)$ converges to 0. Therefore, we have to compute the values of $ϵ\left(\sigma \right)$ for some small values of $\sigma$.

Corollary 1.

If $(X_1\left(\sigma \right),\dots ,X_n\left(\sigma \right),ϵ\left(\sigma \right))$ is the optimal solution to the 0–1 mixed quadratic program (${\mathfrak{Q}}_{⸧}$), for some $\sigma >0$, then $(X_1\left(\sigma \right),\dots ,X_n\left(\sigma \right))$ is an ${ϵ}_{\sigma }$-proper equilibrium, and if $0<{\sigma }^{\prime }<{\sigma }^{″}$, then $0\le ϵ\left({\sigma }^{\prime }\right)\le ϵ\left({\sigma }^{″}\right)$.

Proof. 

If $0<{\sigma }^{\prime }<{\sigma }^{″}$, then the 0–1 mixed quadratic program (${\mathfrak{Q}}_{{⸧}^{\prime }}$), for ${\sigma }^{\prime }>0$, is a relaxation of the same 0–1 mixed quadratic program (${\mathfrak{Q}}_{{⸧}^{″}}$), but for ${\sigma }^{″}>0$. The main difference between the two programs lies in fact in the conditions of ${\Omega }_{ϵ}^{{\sigma }^{\prime }}$ and ${\Omega }_{ϵ}^{{\sigma }^{″}}$:

$\begin{array}{ccc}{\sigma }^{\prime }\le x_{ik},\hfill & k\in \left\{1,2,\dots ,m_i\right\},\hfill & i\in N,\hfill \end{array}$$and\begin{array}{ccc}{\sigma }^{\prime \prime }\le x_{ik},\hfill & k\in \left\{1,2,\dots ,m_i\right\},\hfill & i\in N,\hfill \end{array}$

$$\Rightarrow \begin{array}{ccc}{\sigma }^{\prime }<{\sigma }^{\prime \prime }\le x_{ik},\hfill & k\in \left\{1,2,\dots ,m_i\right\},\hfill & i\in N.\hfill \end{array}$$

Thus, ${\Omega }_{ϵ}^{{\sigma }^{\prime \prime }}\subseteq {\Omega }_{ϵ}^{{\sigma }^{\prime }}$ and $0\le ϵ\left({\sigma }^{\prime }\right)\le ϵ\left({\sigma }^{″}\right)$. □

Hence, there exists only two probable outcomes when computing $ϵ\left(\sigma \right)$ for tiny values of $\sigma$. The first possible outcome corresponds to the case where $ϵ\left(\sigma \right)$ is found to be bounded from below by a strictly positive value $\overline{ϵ}$. The second possible outcome corresponds to the case where $ϵ\left(\sigma \right)$ converges to zero.

Case 1: $ϵ\left(\sigma \right)\ge \overline{ϵ}$ When $ϵ\left(\sigma \right)$ is found to be bounded from below by an $\overline{ϵ}$ strictly positive indicates that there are no $ϵ$-proper equilibria close to $({\widehat{X}}_1,\dots ,{\widehat{X}}_n)$, for $ϵ$ values less than $\overline{ϵ}$. Therefore, $\widehat{X}=({\widehat{X}}_1,\dots ,{\widehat{X}}_n)$ would not be proper.

Now, let us assume that $ϵ\left(\sigma \right)$ converges to some value $\overline{ϵ}>0$, when $\sigma >0$ converges to 0 as in (${\mathfrak{Q}}_{⸧}$). We then define a 0–1 mixed quadratic program under the same conditions of ${\Omega }_{ϵ}^{\sigma }$, but with $ϵ\le \overline{ϵ}/2$ and seeking to maximize $\sigma$. When the optimal objective value of this program is found to be equal to $zero$, we can obviously conclude that it is not possible to obtain a sequence $(X_1(\sigma ),\dots ,X_n(\sigma ))$ of $ϵ$-proper equilibria converging to the Nash equilibrium $\widehat{X}$. Hence, the Nash equilibrium $\widehat{X}=({\widehat{X}}_1,\dots ,{\widehat{X}}_n)$ is not proper.

Proposition 4.

Consider the following 0–1 mixed quadratic program:

$$\begin{array}{ccc}\left({\mathfrak{Q}}_{\overline{•}}\right)& \underset{ϵ,\sigma }{max}\sigma \hfill & \\ & subjectto\hfill & \\ & {\widehat{x}}_{ik}-ϵ\le x_{ik}\le {\widehat{x}}_{ik}+ϵ,\hfill & k\in \left\{1,2,\dots ,m_i\right\},i\in N,\hfill \\ & 0\le ϵ\le \overline{ϵ}/2,\hfill & \\ & (X_1,\dots ,X_n)\in {\Omega }_{ϵ}^{\sigma }.\hfill & \end{array}$$

If the optimal objective value of $\left({\mathfrak{Q}}_{\overline{•}}\right)$ is zero for a given $\overline{ϵ}>0$, the Nash equilibrium $\widehat{X}=({\widehat{X}}_1,\dots ,{\widehat{X}}_n)$ is not proper.

Proof. 

If the optimal objective value of (${\mathfrak{Q}}_{\overline{•}}$) is equal to 0, it is not possible to obtain a sequence $(X_1\left(\sigma \right),\dots ,X_n\left(\sigma \right))$$ϵ$-proper converging to $({\widehat{X}}_1,\dots ,{\widehat{X}}_n)$. Therefore, the Nash equilibrium $\widehat{X}=({\widehat{X}}_1,\dots ,{\widehat{X}}_n)$ is not proper. □

At this stage, a numerical noise may appear with infinitesimal values of $ϵ$. It is risky to rely just on the nonlinear optimization results to detect $ϵ$-properness of a Nash equilibrium. The following mixed 0–1 linear formulations are proposed to overcome this problem.

4.2. Mixed 0–1 Linear Formulation

It is possible to resolve all numerical issues due to the non-linear formulation if $ϵ$ is set to a fixed value. The following Corollary 2 can be formulated. It establishes a 0–1 mixed linear formulation from Proposition 4.

Corollary 2.

For a given $ϵ>0$, consider the following 0–1 mixed linear program

$$\begin{array}{cc}\left({\mathfrak{P}}_{•}\right)max\sigma \hfill & \\ & subjectto\hfill & \\ & {\widehat{x}}_{ik}-ϵ\le x_{ik}\le {\widehat{x}}_{ik}+ϵ,\hfill & k\in \left\{1,2,\dots ,m_i\right\},i\in N,\hfill \\ & (X_1,\dots ,X_n)\in {\Omega }_{ϵ}^{\sigma }.\hfill & \end{array}$$

If the optimal objective value of $\left({\mathfrak{P}}_{•}\right)$ is equal to 0, or $\left({\mathfrak{P}}_{\overline{•}}\right)$ is unfeasible, the Nash equilibrium $\widehat{X}=({\widehat{X}}_1,\dots ,{\widehat{X}}_n)$ is not proper.

The proof of this corollary is trivial (see Proof of Proposition 3). Hence, we can rely on the 0–1 mixed linear optimization results to certify that an equilibrium is non-proper. Computationally, this can be done using exact arithmetics. Exact arithmetics perform operations on real numbers defined as rationals instead of floats or doubles. A rational number consists of two very large integers, namely a numerator and a denominator. Overloading the elementary operators to handle algebraic operations on these rationals is fundamental. A Greatest Common denominator function is useful to reduce the size of the stored Big Integers. Exact arithmetics was successfully used by Audet et al. [25] and Avis et al. [26] in the context of two-person games equilibrium enumeration and refinement. The automatic detection of non-proper Nash equilibria can then safely be carried out for polymatrix games.

Case 2: $ϵ\left(\sigma \right)$ converges to zero If $ϵ\left(\sigma \right)$ converges towards zero, we can conclude on the existence of at least one sequence of $ϵ$-proper equilibria necessarily close to the given Nash equilibrium $\widehat{X}$ we are testing for properness. Computationally speaking, unless exact arithmetics is used here, this may not be enough to conclude on the properness of the given Nash equilibrium. Therefore, it is imperative to use exact arithmetics to get consistent results.

As an extension of Corollary 2, the following Corollary 3 can be formulated, when $ϵ$ is set to a fixed value.

Corollary 3.

For some $ϵ>0$, consider the mixed 0–1 linear program $\left({\mathfrak{P}}_{•}\right)$ of corollary 2:

$$\begin{array}{ccc}\left({\mathfrak{P}}_{•}\right)\hfill & \sigma \left(ϵ\right)=max\sigma \hfill & \\ & subjectto\hfill & \\ & {\widehat{x}}_{ik}-ϵ\le x_{ik}\le {\widehat{x}}_{ik}+ϵ,\hfill & k\in \left\{1,2,\dots ,m_i\right\},i\in N,\hfill \\ & (X_1,\dots ,X_n)\in {\Omega }_{ϵ}^{\sigma }.\hfill & \end{array}$$

If the optimal objective value $\sigma \left(ϵ\right)$ of $\left({\mathfrak{P}}_{•}\right)$ converges to 0 and remains strictly positive, when ϵ converges to 0, then $({\widehat{X}}_1,\dots ,{\widehat{X}}_n)$ is a proper equilibrium.

Proof. 

Let $(X_1\left(ϵ\right),\dots ,X_n\left(ϵ\right),\sigma \left(ϵ\right))$ be the optimal solution to (4), for some given equilibrium profile $({\widehat{X}}_1,\dots ,{\widehat{X}}_n)$.

If $\sigma \left(ϵ\right)$ is strictly positive, Proposition (1) ensures that $(X_1(ϵ),\dots ,X_n(ϵ))$ is an $ϵ$-proper equilibrium.

Hence, if the optimal objective value $\sigma \left(ϵ\right)$ converges to 0 and remains strictly positive when $ϵ$ converges to 0, $(X_1(ϵ),\dots ,X_n(ϵ))$ remains an $ϵ$-proper equilibrium and converges to the equilibrium $({\widehat{X}}_1,\dots ,{\widehat{X}}_n)$. Therefore, $({\widehat{X}}_1,\dots ,{\widehat{X}}_n)$ is a proper equilibrium. □

Since it is based on a linear formulation, this corollary resolves all numerical issues due to the nonlinear formulation of Proposition 3. First, we illustrate our results with a polymatrix game where one of the extreme Nash equilibria is not perfect. With the following example, we show how the Corollary 3 can be applied.

Example Consider the three players polymatrix game $(4\times 3\times 2)$, where the following matrices $A_{1.}$, $A_{2.}$ and $A_{3.}$ detail the payoffs of players I, II, and III.

$$A_{1.}=\left(\begin{array}{ccc}1& 1& 3\\ 1& 2& 0\\ 3& 4& 1\\ 2& 3& 2\end{array}|\begin{array}{cc}1& 1\\ 4& 1\\ 3& 2\\ 1& 2\end{array}\right)\hspace{1em}\hspace{1em}\hspace{1em}{A}_{2.}=\left(\begin{array}{cccc}5& 2& 2& 3\\ 2& 5& 3& 4\\ 1& 4& 2& 1\end{array}|\begin{array}{cc}2& 5\\ 1& 2\\ 4& 1\end{array}\right)$$

$$A_{3.}\left(\begin{array}{cccc}1& 1& 2& 3\\ 2& 1& 2& 1\end{array}|\begin{array}{ccc}5& 4& 3\\ 2& 1& 3\end{array}\right)$$

Using exact arithmetic, the E$\chi$-MIP algorithm [4,25] enumerated three different extreme Nash equilibria for this game, as detailed in

Table 1. Extreme Nash equilibria.

Eq	${\mathit{\alpha }}_1$	${\mathit{\alpha }}_2$	${\mathit{\alpha }}_3$	${\mathit{X}}_1$	${\mathit{X}}_2$	${\mathit{X}}_3$
1	4	6	5	$(0,0,1,0)$	$(0,0,1)$	$(1,0)$
2	4	8	4	$(0,1,0,0)$	$(0,0,1)$	$(1,0)$
3	$\frac{35}{9}$	$\frac{31}{6}$	$\frac{89}{18}$	$(\frac{1}{6},0,\frac{5}{6},0)$	$(\frac{1}{18},0,\frac{17}{18})$	$(\frac{7}{9},\frac{2}{9})$

.

The optimization results in

Table 2. Optimization results.

Eq.	Perfect	$\mathit{ϵ}$-Proper	$\mathit{ϵ}$	$\mathit{\sigma }\left(\mathit{ϵ}\right)$
1	yes	yes	$\frac{1}{1000}$	$\frac{1}{100,100,100}$
2	yes	no	$\frac{1}{1000}$	0
3	yes	yes	$\frac{1}{1000}$	$\frac{1}{5,998,996}$

(as shown in [20], a triplet of linear programs was used to conclude on the perfectness of the extreme Nash equilibria) and

Table 3. Optimization results for equilibrium 3.

$\mathit{ϵ}$	$\mathit{\sigma }\left(\mathit{ϵ}\right)$	${\mathit{X}}_1\left(\mathit{ϵ}\right)$	${\mathit{X}}_2\left(\mathit{ϵ}\right)$	${\mathit{X}}_3\left(\mathit{ϵ}\right)$
$\frac{1}{2}$	$\frac{1}{36}$	$(\frac{11}{36},\frac{1}{18},\frac{11}{18},\frac{1}{36})$	$(\frac{1}{18},\frac{1}{36},\frac{11}{12})$	$(\frac{49}{54},\frac{5}{54})$
$\frac{1}{4}$	$\frac{1}{90}$	$(\frac{17}{90},\frac{2}{45},\frac{34}{45},\frac{1}{90})$	$(\frac{2}{45},\frac{1}{90},\frac{17}{18})$	$(\frac{211}{270},\frac{59}{270})$
$\frac{1}{8}$	$\frac{1}{333}$	$(\frac{56}{219},\frac{7}{219},\frac{17}{24},\frac{7}{1752})$	$(\frac{8}{333},\frac{1}{333},\frac{36}{37})$	$(\frac{8}{9},\frac{1}{9})$
$\frac{1}{9}$	$\frac{1}{473}$	$(\frac{81}{473},\frac{9}{473},\frac{382}{473},\frac{1}{473})$	$(\frac{76}{1419},\frac{1}{473},\frac{1340}{1419})$	$(\frac{100}{129},\frac{29}{129})$
$\frac{1}{10}$	$\frac{1}{586}$	$(\frac{50}{293},\frac{5}{293},\frac{475}{586},\frac{1}{586})$	$(\frac{5}{1758},\frac{1}{586},\frac{830}{879})$	$(\frac{1363}{1758},\frac{395}{1758})$
$\frac{1}{100}$	$\frac{1}{59,896}$	$(\frac{1250}{7487},\frac{25}{14,974},\frac{49,795}{59,896},\frac{1}{59,896}$)	($\frac{9995}{179,688},\frac{1}{59,896},\frac{84845}{89,844})$	($\frac{139,693}{179,688},\frac{39,995}{179,688}$)
$\frac{1}{1000}$	$\frac{1}{5,998,996}$	$(\frac{250,000}{1,499,749},\frac{250}{1,499,749},\frac{4,997,995}{5,998,996},\frac{1}{5,998,996})$	$(\frac{999,995}{17,996,988},\frac{1}{5,998,996},\frac{274,145}{290,274})$	$(\frac{13,996,993}{17,996,988},\frac{3,999,995}{17,996,988})$
$\frac{1}{10,000}$	$\frac{1}{599,989,996}$	$(\frac{25,000,000}{149,997,499},\frac{2500}{149,997,499},\frac{499,979,995}{599,989,996},\frac{1}{599,989,996})$	$(\frac{99,999,995}{1,799,969,988},\frac{1}{599,989,996},\frac{849,984,995}{899,984,994})$	$(\frac{1,399,969,993}{1,799,969,988},\frac{399,999,995}{1,799,969,988})$
$\frac{1}{100,000}$	$\frac{1}{59,999,899,996}$	$(\frac{2,500,000,000}{14,999,974,999},\frac{25,000}{14,999,974,999},\frac{49,999,799,995}{59,999,899,996},\frac{1}{59,999,899,996})$	$(\frac{9,999,999,995}{179,999,699,988},\frac{1}{59,999,899,996},\frac{84,999,849,995}{89,999,849,994})$	$(\frac{139,999,699,993}{179,999,699,988},\frac{39,999,999,995}{179,999,699,988})$
$\frac{1}{1,000,000}$	$\frac{1}{5,999,998,999,996}$	$(\frac{250,000,000,000}{1,499,999,749,999},\frac{250,000}{1,499,999,749,999},\frac{4,999,997,999,995}{5,999,998,999,996},\frac{1}{5,999,998,999,996})$	$(\frac{999,999,999,995}{17,999,996,999,988},\frac{1}{5,999,998,999,996},\frac{8,499,998,499,995}{8,999,998,499,994})$	$(\frac{13,999,996,999,993}{17,999,996,999,988},\frac{3,999,999,999,995}{17,999,996,999,988})$

point towards the existence of $ϵ$-proper equilibria sequences close to extreme Nash equilibria 1 and 3.

The optimization results in Table 3 show how the value of $\sigma \left(ϵ\right)$ decreases while $ϵ$ converges to 0 and the quest of $ϵ$-proper equilibria close to extreme Nash equilibrium 3 is successful.

The following figure (Figure 1) illustrates how the value of $\sigma \left(ϵ\right)$ converges to 0, when $ϵ$ converges to 0, for the extreme Nash equilibrium 3.

4.3. Analytical Computation

We now use the optimization results in Table 2 and Table 3 to analytically generate sequences of $ϵ$-proper equilibria from extreme Nash equilibrium 3. In the extreme Nash equilibrium 3 strategy profile, player I randomizes on $x_{11}$ and $x_{13}$. A sequence of $ϵ$-proper equilibria should then take into account that player II is indifferent between his first and third strategies. One should think that if player I is not indifferent between his first and third strategies, he would have to assign to the less interesting strategy a probability less than $ϵ$ times the probability of the other strategy.

Therefore, $3x_{21}+4x_{22}+x_{23}+3x_{31}+2x_{32}=x_{21}+x_{22}+3x_{23}+x_{31}+x_{32}.$

Thus, we have

$$\begin{array}{c}2x_{23}=2x_{21}+3x_{22}+2x_{31}+x_{32}\\ \iff \end{array}$$

$$x_{23}=\frac{1}{2}+x_{21}+\frac{3}{2}{x}_{22}+\frac{1}{2}{x}_{31}.$$

(3)

One should also note that this is satisfied by equilibrium 3. Hence, player II would have to assign a probability larger than $\frac{1}{2}$ to his third strategy.

Simultaneously, a sequence of $ϵ$-proper equilibria should take into account that player II is indifferent between his first and third strategies. Therefore, $5x_{11}+2x_{12}+2x_{13}+3x_{14}+2x_{31}+5x_{32}=x_{11}+4x_{12}+2x_{13}+x_{14}+4x_{31}+x_{32}.$

Thus, we have

$$2x_{11}+x_{14}+3x_{32}=x_{12}+1.$$

(4)

This condition is satisfied by equilibrium 3. Taking into account that $x_{12}$ and $x_{14}$ would receive very small but strictly positive probabilities, player I would have to assign a probability less than $\frac{1}{2}$ to his first strategy, and player III would have to assign a probability less than $\frac{1}{3}$ to his second strategy.

Finally, a sequence of $ϵ$-proper equilibria would take into account that player III is indifferent between his first and second strategies. Therefore, $x_{11}+x_{12}+2x_{13}+3x_{14}+5x_{21}+4x_{22}+3x_{23}=2x_{11}+x_{12}+2x_{13}+x_{14}+2x_{21}+x_{22}+3x_{23}.$

Thus, we have

$$2x_{14}+3x_{21}+2x_{22}=x_{11}.$$

(5)

This condition is also satisfied by equilibrium 3. Player II would have to assign a probability less than $\frac{1}{6}$ to his first strategy.

We here detail some of the steps to generate a sequence of $ϵ$-proper equilibria converging to the extreme Nash equilibrium 3.

We first set $x_{31}=\frac{7}{9}-\alpha ϵ$, $x_{32}=\frac{2}{9}+\alpha ϵ$, $x_{21}=\frac{1}{18}+\lambda ϵ$, $x_{22}=\frac{1}{36}ϵ$ and $x_{23}=\frac{17}{18}+\rho ϵ$, where $\lambda +\rho +\frac{1}{36}=0$.

With conditions (3), we obtain $\lambda =\frac{36\alpha -5}{144}$ and $\rho =\frac{1-36\alpha }{144}$.

Since $x_{32}$ needs to be less than $\frac{1}{3}$, we have

$$\alpha ϵ\le \frac{1}{9}.$$

(6)

For simplification purposes, we consider that player I would be indifferent between his second and fourth strategies, which yields $x_{14}=x_{12}$. This assumption does not eliminate the possibility for player I to have a marked preference for one of these two strategies.

We then set $x_{14}=x_{12}$ and substitute condition (5) in (4).

We obtain

$$x_{12}=x_{14}=\frac{7-324\alpha }{284}\times ϵ.$$

(7)

Since $x_{12}\le \frac{1}{6}ϵ$, because player I prefers his first strategy to his second, we obtain a lower bound on $\alpha$:

$$\alpha \ge -\frac{121}{972}.$$

Then, by (5) we have $x_{11}=\frac{1}{6}+(2\beta +3\lambda +\frac{1}{8})ϵ$ and $x_{13}=\frac{5}{6}-(4\beta +3\lambda +\frac{1}{18})ϵ$, where

$$\beta =\frac{7-324\alpha }{284}.$$

It has now been made clear that we obtained a sequence of $ϵ$-proper equilibria converging to the extreme Nash equilibrium 3 when $ϵ$ converges to 0.

5. Branch-and-Bound Algorithm for $\mathbf{ϵ}$-Proper Equilibrium Computation

We now propose a Branch-and-Bound algorithm for $ϵ$- proper equilibrium computation. The algorithm is based on a 0–1 mixed linear program which provides at optimality an $ϵ$-proper equilibrium to a polymatrix game. The 0–1 mixed linear program combines Nash equilibrium primal and dual conditions detailed in [4] and the $ϵ$-properness conditions. Note that $ϵ$ here is set to a fixed rational value which is strictly positive, but very close to $0.$

5.1. The 0–1 Mixed Linear Master Program

The following 0–1 mixed linear program $\left({\mathfrak{L}}_{•}\right)$ combines the Nash equilibrium primal and dual conditions, for polymatrix games, and the $ϵ$-properness conditions aforementioned. The optimization of $\left({\mathfrak{L}}_{•}\right)$ using exact arithmetics, provides at optimality a Nash equilibrium $\widehat{X}$ and an $ϵ$-proper equilibrium X, when $ϵ$ is set to a fixed value.

Proposition 5.

Given a polymatrix game, for some $ϵ>0,$ consider the 0–1 mixed linear program $\left({\mathfrak{L}}_{•}\right)$:

$$\begin{array}{cc}\hfill \sigma \left(ϵ\right)=Maximiz{e}_{\widehat{X},X,V,\sigma }\sigma & \end{array}$$

(8)

$$\begin{array}{ccc}\mathit{Subject}\mathit{to}\hfill & \hfill e_m^t{\widehat{X}}_i=1,\hfill & i\in N,\hfill \end{array}$$

(9)

$$\begin{array}{ccc}& {\widehat{X}}_i+V_i\le 1,\hfill & \hfill i\in N,\end{array}$$

(10)

$$\begin{array}{ccc}& {\alpha }_i{e}^i-{\sum }_{j=1,j\ne i}^n{A}_{ij}{\widehat{X}}_j\ge 0,& \hfill i\in N\end{array}$$

(11)

$$\begin{array}{ccc}& {\alpha }_i{e}^i-{\sum }_{j=1,j\ne i}^n{A}_{ij}{\widehat{X}}_j-L{V}_i\le 0,& \hfill i\in N\end{array}$$

(12)

$$\begin{array}{ccc}& {\widehat{x}}_{ik}-ϵ\le x_{ik}\le {\widehat{x}}_{ik}+ϵ,\forall k\in 1,\dots ,m_i,& \hfill i\in N,\end{array}$$

(13)

$$\begin{array}{cc}& (X_1,\dots ,X_n)\in {\Omega }_{ϵ}^{\sigma }\end{array}$$

(14)

$$\begin{array}{ccc}& {\widehat{X}}_i\ge 0,\hfill & i\in N,\hfill \end{array}$$

(15)

$$\begin{array}{ccc}& V_i\in {\{0,1\}}^{m_i},\hfill & i\in N.\hfill \end{array}$$

(16)

This 0–1 mixed linear program $\left({\mathfrak{L}}_{•}\right)$ is always feasible for any $ϵ>0$ and its optimal objective value $\sigma \left(ϵ\right)$ is always strictly positive.

The mixed strategies vector $\widehat{X}=({\widehat{X}}_1,\dots ,\widehat{X}n)$ is a Nash equilibrium of the given polymatrix game and the completely mixed strategies vector $X=(X_1,\dots ,X_n)$ is an ϵ-proper equilibrium.

Proof. 

Constraints (9)–(12) assert that $\widehat{X}=({\widehat{X}}_1,\dots ,{\widehat{X}}_n)$ is a Nash equilibrium of the given polymatrix game (see [4]). Constraints (13) and (14) assert that $X=(X_1,\dots ,X_n)$ is an $ϵ$-proper equilibrium as shown earlier. For any given polymatrix game, the set of Nash equilibria and the set of proper equilibria are non-empty. Therefore, for any $ϵ>0,$ there exists a pair of vectors $\widehat{X}$ and X such that $\widehat{X}$ is a Nash equilibrium, and X is an $ϵ$-proper equilibrium. Hence, the optimal objective value $\sigma \left(ϵ\right)$ is always strictly positive. □

The mixed 0–1 linear program $\left({\mathfrak{L}}_{•}\right)$ serves as the master program in the proposed $ϵ$-proper Branch-and-Bound algorithm.

5.2. The $ϵ$-Proper Branch-and-Bound Algorithm

Let P be defined as a master linear program obtained after relaxation of the 0–1 conditions in $\left({\mathfrak{L}}_{•}\right)$. The proposed $ϵ$-proper Branch-and-Bound algorithm follows a principal branching tree scheme and many secondary branching trees. At every vertex on the principal tree, a binary branching is performed on one of the 0–1 variables. At every vertex of the principal tree, the algorithm solves a master linear program P with at most two constraints fixing one or more binary variables and cutting one or more binary variables combinations. Each time a Nash equilibrium $\widehat{X}$ is obtained, the objective function value $\sigma$ is compared to a recorded maximum $\overline{\sigma }$. In the case where the optimal objective $\sigma$ is equal to zero or lower than $\overline{\sigma }$, the given principal tree vertex and its sons cannot provide any $ϵ$-proper equilibrium with a larger $\sigma$ than $\overline{\sigma }$. The current $\overline{\sigma }$ stored serves as a bound to shorten the algorithm’s execution.

At every vertex of a secondary tree node, a linear sub-program Q is solved. This linear sub-program Q results from the master linear program P, with one combination of binary variables fixing constraints and at most two constraints fixing one or more continuous x variables. If a Nash equilibrium $\widehat{X}$ is obtained, the objective function value $\sigma$ is compared to a recorded maximum $\overline{\sigma }$. Again, in the case where the optimal objective $\sigma$ is equal to zero or lower than $\overline{\sigma }$, the given secondary tree vertex and its sons cannot provide any $ϵ$-proper equilibrium with a larger $\sigma$ than $\overline{\sigma }$.

The $ϵ$-proper Branch-and-Bound algorithm is formally stated as follows.

Step 1: Initialization.

Define:

-

  P; Master linear program defined by the relaxation of the 0–1 conditions in $\left({\mathfrak{L}}_{•}\right).$

-

  $Q;$ Sub-program defined by a secondary linear program where the U and V variables in P are fixed.

-

  $\widehat{X}$, X; ${\mathfrak{L}}_{•}$’s real decision variables.

-

  $U,V$; ${\mathfrak{L}}_{•}$’s binary decision variables.

-

  $E=\varnothing$; List of Nash equilibria visited.

-

  $E_{ϵ}=\varnothing$; List of $ϵ$-proper Nash equilibria obtained.

-

  $D=0$; Depth level reached in the principal tree.

-

  $CR$; Principal tree root node.

-

  C; Current node.

-

  ${\widehat{x}}_{ik}$, $x_{ik}$; Real decision variables on the $k^{th}$ strategy of player i$(i=1,\dots ,n).$

-

  $v_i^q$; Binary decision variables on the $q^{th}$ strategy of player i$(i=1,\dots ,n).$

-

  $u_{kq}^i$; Binary decision variables on the $k^{th}$ and $q^{th}$ strategies of player i$(i=1,\dots ,n).$

-

  $\overline{\sigma }$; Best value of $\sigma$ found.

Set C = CR, $\overline{\mathbf{\sigma }}=\mathbf{0}$ and Go to step 2.

  Step 2: Solving master program.

If $D\le |\widehat{X}|+|U|$, solve current node’s program P.

-

  If the program P is infeasible or the optimal objective $\sigma$ is equal to 0; STOP.

-

  Else;

  -

  If $D<|\widehat{X}|+|U|$, Go to step 3.

  -

  Else, Let $S=(\widehat{X};X;V)$ be the solution obtained;

  -

  If the optimal objective $\sigma$ of P is less than $\overline{\sigma }$, STOP.

  -

  Else, if the optimal objective $\sigma$ of P is ≥$\overline{\sigma }$,

Update$\overline{\sigma }=\sigma$, AddX to E and $X_{\sigma }$ to $E_{ϵ}$ and Go to step 4.

  Step 3: Principal Branching.

Select a binary variable $v_i^q\in V$ or $u_{kq}^i\in U$ over which no principal branching was performed.

Let$p=D+1:$

If $p\le |\widehat{X}|+|U|$ set$D=p$ and

-

  Set $v_i^q\in V$ or $u_{kq}^i\in U$ selected to 1 in P and Go to step 2.

-

  Set $v_i^q\in V$ or $u_{kq}^i\in U$ selected to 0 in P and Go to step 2.

Step 4: Secondary Branching.

Set binary variable vectors V and U at their values in $S.$

For every ${\widehat{x}}_{ik}>0\in \widehat{X}$: Set${\widehat{x}}_{ik}=0$ and Go to step 5.

  Step 5: Solving subprogram.

Solve $Q(\widehat{X})$ to optimality.

-

  If the program Q is infeasible or the optimal objective $\sigma$ is equal to 0; STOP.

-

  Else, if the optimal objective $\sigma$ of $Q(\widehat{X})$ is less than $\overline{\sigma }$, STOP.

-

  If the optimal objective $\sigma$ of $Q(\widehat{X})$ is $\ge \overline{\sigma }$, Update$\overline{\sigma }$ = $\sigma$, Add $\widehat{X}$ to E and X to $E_{ϵ}$. Go to step 4.

The sets E and $E_{ϵ}$ are non-empty and contain respectively all Nash equilibria and all $ϵ$-proper equilibria encountered. The maximum value of $\sigma$ is $\sigma \left(ϵ\right)=\overline{\sigma }.$ The last element entering $E_{ϵ}$ corresponds to $\sigma \left(ϵ\right).$

Theorem 1.

The ϵ-proper Branch-and-Bound algorithm generates at least one ϵ-proper equilibrium for any polymatrix game.

Proof. 

The algorithm searches for all possible ways to satisfy the Nash equilibrium, complementary slackness, and properness conditions through the principal branching tree and the secondary trees. If $X\left(ϵ\right)$ is the unique $ϵ$-proper equilibrium, there exists at least one path in the algorithm’s generated branching tree leading to it. □

The complexity of the $ϵ$-proper Branch-and-Bound algorithm is certainly non-polynomial. It increases exponentially when the size of the polymatrix game increases.

6. Experimental Results

Table 4. $ϵ$-proper experimental results.

$\mathit{ϵ}={10}^{-4}$	$\mathit{d}=0.125$		$\mathit{d}=0.25$		$\mathit{d}=0.5$		$\mathit{d}=0.75$		$\mathit{d}=1.0$
Size	ne	Time	ne	Time	ne	Time	ne	Time	ne	Time
2 × 2 × 2	5.1	11.7	2.5	9.3	1.8	7.9	1.5	7.8	1.0	6.7
3 × 2 × 2	7.3	22.3	4.9	20.3	3.5	18.5	1.6	8.7	1.2	8.1
3 × 3 × 2	8.6	35.2	5.3	24.3	2.5	21.2	1.8	20.8	1.5	19.5
3 × 3 × 3	11.0	47.8	7.6	35.6	2.9	27.4	2.0	27.0	1.6	26.7
4 × 4 × 4	13.7	72.7	8.5	61.8	3.9	56.4	3.7	45.2	2.6	41.5
5 × 4 × 3	15.2	87.7	9.4	86.5	4.7	82.4	4.2	79.9	2.8	74.6
5 × 4 × 4	16.8	109.4	10.0	97.1	6.2	93.6	5.3	81.4	3.2	77.1
5 × 5 × 5	18.2	123.6	10.3	115.4	6.7	106.9	5.9	91.1	4.7	84.5

presents the computational results obtained on randomly generated sets of three-player polymatrix games with different sizes and density. The size of the polymatrix games generated goes from $(2\times 2\times 2)$ to $(5\times 5\times 5).$ The density of the polymatrix games generated goes from $12.5$% to 100 %. The entries of the polymatrix games generated are integers. The optimization of the master and secondary linear programs P and Q uses a C++ implementation of the Simplex algorithm in exact arithmetics.

In Table 4, column (Size) lists the different numbers of strategies for each player before elimination of strategies strongly dominated. The column (d) lists the different densities of the partial payoff matrices generated. The column ($ϵ$) lists the values of the parameter $ϵ$ used. The column ($ne$) indicates the average number of $ϵ$-proper equilibria encountered over 10 polymatrix games randomly generated. Finally, the value (Time) indicates the average time in seconds required to find an $ϵ$-proper equilibrium with a maximum value of $\sigma$. Finally, the experimental results are obtained under MS Windows on workstations equipped with 3.3 GHz Intel Core i5 vPro processors and 3.2 GB RAM.

In particular, one can observe that reducing the partial payoff matrices density increases the computing time required to find an $ϵ$-proper equilibrium as it also increases the total number of $ϵ$-proper equilibria detected for the polymatrix game.

7. Conclusions

In this paper we have shown how to compute $ϵ$-proper Nash equilibria for polymatrix games. The notion of a set of $ϵ$-proper equilibria for polymatrix games was used to develop a class of 0–1 mixed linear programs to detect proper and non-proper Nash equilibria. Consequently, a 0–1 mixed linear program was proposed to compute an $ϵ$-proper Nash equilibrium for a polymatrix game. We have finally presented our experimental results on randomly generated polymatrix games. A future extension of this work should naturally tackle proper equilibrium computation for more general forms of n-player games.