Stationary Bayesian–Markov Equilibria in Bayesian Stochastic Games with Periodic Revelation

Abstract

I consider a class of dynamic Bayesian games in which types evolve stochastically according to a first-order Markov process on a continuous type space. Types are privately informed, but they become public together with actions when payoffs are obtained, resulting in a delayed information revelation. In this environment, I show that there exists a stationary Bayesian–Markov equilibrium in which a player’s strategy maps a tuple of the previous type and action profiles and the player’s current type to a mixed action. The existence can be extended to K-periodic revelation. I also offer a computational algorithm to find an equilibrium.

1. Introduction

In the parable of the blind men and the elephant1, individuals attempt to comprehend the concept of an elephant solely through the sense of touch. Each person, limited to exploring only a specific part of the elephant, such as the wriggling trunk, flapping ears, or swinging tail, reaches their own conclusions about the shape of an elephant. One perceives the elephant as resembling a snake, another as a fan, and yet another as a rope. The parable serves as an analogy to the situation where economic agents have incomplete perceptions about the true state of the world. Players choose actions based on their limited perception of the state of the economy, but, eventually, the true state of the world is revealed. Players’ actions accordingly yield rewards or penalties.2

To incorporate the above aspect of reality into a formal model, this paper considers a class of dynamic Bayesian games in which types evolve stochastically according to a first-order Markov process on a continuous type space (“Bayesian stochastic games”). It is well known that equilibria of stochastic games with the continuous state space are elusive. This paper overcomes the challenge with the Bayesian feature. Dynamic Bayesian games with serially correlated types, however, are notorious for the curse of dimensionality that the dimension of players’ beliefs grows over time, and, thus, equilibria of such games are generally not tractable. This paper introduces delayed revelation of private information which is referred to as “periodic revelation” to overcome the dimensionality issue of beliefs. Types remain private when players choose actions, but they are revealed alongside actions when payoffs are obtained. In this framework, there exist a class of stationary Markov perfect equilibria. The game structure and equilibrium concept can be applied to the analyses of dynamic oligopoly with asymmetric information or many other economic environments with a delay of information revelation.

Suppose, at each time t, a player has a limited piece of information about the true state of the world. It is defined as the type of the player in the t-stage Bayesian game. The type profile of all the players is the true state of the economy, and it is hidden when players choose actions. But type and action profiles are eventually revealed to everyone at the end of the stage when individual payoffs of the stage Bayesian game are obtained. Toward the next period, the type profile stochastically evolves according to a first-order Markov process based on the revealed type and action profiles. Especially, the previous type and action profiles $\left(s_{t-1},a_{t-1}\right)$, are publicly known at the beginning of time t, and all the players know the probability distribution over the new type profile $Pr\left(s_t\mid s_{t-1},a_{t-1}\right)$ as common knowledge.

To see the structure in a familiar setting, consider a duopoly playing a dynamic Cournot competition. At time t, firm i learns its cost type $c_i$, which is drawn from a compact interval in the real line $[0,\alpha ]$. However, firm i does not know firm j’s cost type, and vice versa. Firm i is aware that the cost type profile, which is the state of the economy, evolves stochastically over time according to a first-order Markov process, and that the stochastic process is based on the previous type and action profiles. Suppose the previous stage types and actions are publicly known, say, after the financial statements of firms are revealed. Then, firm i can have a belief over firm j’s current cost type based on the previous type profile and action profile. Conditional on their private cost types and beliefs about each other, firms choose optimal quantities to produce. Although similar approaches have appeared in the dynamic Bayesian games literature frequently, to the best of my knowledge, the models have suffered from the curse of dimensionality due to the history-dependent beliefs.3 This paper, by contrast, allows players to have time-invariant beliefs as long as the previous actions and types are the same under periodic revelation. Also, the models have never been treated in a continuous type space under a first-order Markov process. This paper shows existence of a class of stationary Markov perfect equilibria in this environment, which are termed “stationary Bayesian–Markov equilibria”.

Developing a framework for analyses of dynamic oligopolies with asymmetric information has been an open question for a long time due to the difficulty of dealing with the beliefs. Fershtman and Pakes [4] propose a framework for dynamic games with asymmetric information over a discrete type space focusing on empirical tractability. Their theoretical equilibrium concept is history-dependent, but they cleverly detour the dimensionality issue with the assumption that accumulated data include all the information about the history.4 Cole and Kocherlakota [5] and Athey and Bagwell [6], respectively, suggest new equilibrium concepts in a class of dynamic Bayesian games in which players’ beliefs are a function of the full history of public information.5 Hörner, Takahashi, and Vieille [7] characterize a subset of equilibrium payoffs, focusing on the case where players report their private information truthfully in dynamic stochastic Bayesian games.6

The key distinction of this paper from existing literature in dynamic Bayesian games is to add periodic revelation to mitigate the dimensionality problem. In the literature, it is conventional to formulate that private information remains hidden and never becomes public. If the game is under the serially correlated type evolution, without periodic revelation, players formulate their beliefs about other players’ current type based on the history of available past information. Then, the dimension of each player’s beliefs exponentially grows over time.7 Periodic revelation in this paper, however, enables players to have common prior for the current stage type distribution based on the revealed information, and, thus, players have time-invariant beliefs as long as the revealed information is the same regardless of the calendar time, which are consistent with the underlying type evolution. This helps to establish the stationary equilibrium concept even when types are serially correlated. Practically, it captures the economic environments where private information is periodically disclosed, either by legal requirements or voluntarily. Therefore, the model can be applied to dynamic oligopolies under the periodic disclosure requirement on the firms’ financial performance, e.g., Form 10-K in the U.S., or to the setting of Barro and Gordon [8] to analyze the effects of release of the Federal Open Market Committee (FOMC) transcripts with a five-year delay (See Ko and Kocherlakota [9]).

Levy and McLennan [10] show that, for stochastic games with complete information in a continuous state space, existence of stationary Markov equilibria is not guaranteed. In such environments, Nowak and Raghavan [11] show that there exists a stationary correlated equilibrium. Duggan [12] shows that there exists a stationary Markov equilibrium in cases where the state variable has an additional random component. The random component, so-called noise, can be viewed as an embedded randomization device for each state. In this perspective, a stationary Markov equilibrium in noisy stochastic games can be seen isomorphic to a stationary correlated equilibrium by Nowak and Raghavan [11]. Barelli and Duggan [13] formulate a dynamic stochastic game in which players’ strategies depend on the past and current information to show that Nowak and Raghavan’s stationary correlated equilibrium can be uncorrelated. By contrast, this paper considers the case where players have private information: the state is defined by a type profile of the game of incomplete information. As the players compute their interim payoff by integrating possible scenarios over their beliefs, the convexity of the set of interim payoffs is naturally obtained. It is more realistic that the convexity is obtained from the Bayesian structure, compared with previous approaches that utilize public randomization devices (Nowak and Raghavan [11]) or random noise (Duggan [12]).

The concept of the stationary Bayesian–Markov equilibrium is closely related to a stationary correlated equilibrium in Nowak and Raghavan [11]. That is, in a Bayesian game, players have beliefs about the type profile of their game, which is the state of the economy. Players then choose actions to maximize their expected payoffs considering their individual beliefs about the state of the economy. According to Aumann [14], a resulting Bayesian equilibrium can be regarded as a correlated equilibrium distribution, utilizing the collection of beliefs as a randomization device.

To prove existence of stationary Markov equilibria in conventional stochastic games, it is common to consider an induced game of the original stochastic games. The induced game is defined by a stage game that is indexed by the state of the world and a profile of continuation value functions. Then, the next step is to find a fixed point of the expected continuation value function in the induced game. Finally, one may apply the generalized implicit function theorem and a measurable selector theorem to extract an equilibrium strategy profile. During this process, it is crucial to have a convex set of the expected continuation value functions in order to ensure existence of a fixed point. Therefore, the key of proof is convexification of the set of continuation values in each state of the economy. Nowak and Raghavan [11] introduce a public randomization device, a so-called sunspot, as a means to convexification. In this paper, the idea of convexification is extended to the case where each player obtains private information according to common prior that depends on the past information from periodic revelation.

The model of a Bayesian stochastic game with periodic revelation is formally described in Section 2. Section 3 contains the existence theorem and the proof. Section 4 is devoted to a computational algorithm. Section 5 sketches the extension of proof to K-periodic revelation and concludes. In the Supplementary Materials, I present an application, specifically an incomplete information version of an innovation race between two pharmaceutical companies with periodic revelation.

2. The Basic Model

2.1. The Primitives

I use superscript “$+($resp. $-)$” to denote the next period (resp. the previous period). A discounted Bayesian stochastic game with periodic revelation is a tuple,

$$\begin{array}{c}\hfill (\mathcal{I},{((S_i,{\mathfrak{S}}_i),X_i,A_i,u_i,{\delta }_i,\mu ,\eta )}_{i\in \mathcal{I}},\tau ),\end{array}$$

(1)

such that

• $\mathcal{I}=\{1,\dots ,n\}$ is a finite set of n players, and for each $i\in \mathcal{I}$,
• $(S_i,{\mathfrak{S}}_i)$ is a measurable space of player i’s types, regardless of the calendar time, i.e., $S_i^{+}=S_i$,
• $(X_i,{\mathfrak{X}}_i)$ is a measurable space of player i’s actions,
• $A_i:S_i\rightrightarrows X_i$ is the feasible action correspondence,
• $u_i:S\times X\to \mathfrak{R}$ is the payoff function, where $S={\prod }_{i\in \mathcal{I}}{S}_i$ and $X={\prod }_{i\in \mathcal{I}}{X}_i$,
• ${\delta }_i\in [0,1)$ is the discount factor,
• $\mu :S\times X\times {\mathfrak{S}}_i\to [0,1]$ is a transition function8,
• $\eta :S\times X\times S_i\times {\mathfrak{S}}_{-i}\to [0,1]$ is a transition function,
• $\tau :S\times X\times \mathfrak{S}\to [0,1]$ is a transition function,
• It is an infinite horizon game.

Let $\Delta (·)$ denote the set of probability measures. I assume that

($\mathcal{A}1$)

For each $i\in \mathcal{I}$, $S_i$ is a Borel subset of a complete separable metric space, and ${\mathfrak{S}}_i$ is its Borel $\sigma$-algebra. Endowed with the product topology, the Cartesian product S is a Borel subset of a complete separable metric space. A product of $\sigma$-algebras $\mathfrak{S}={\mathfrak{S}}_1\times \cdots \times {\mathfrak{S}}_n$ is its Borel $\sigma$-algebra (M10, Billingsley [16], p. 254).

($\mathcal{A}2$)

For each $i\in \mathcal{I}$, there is an atomless probability measure ${\varphi }_i$ such that $(S_i,{\mathfrak{S}}_i,{\varphi }_i)$ is a complete measure space of player i’s types; $\varphi$ is a product probability measure such that $\varphi ={\varphi }_1\times \cdots \times {\varphi }_n$.9

($\mathcal{A}3$)

For each $i\in \mathcal{I}$, $X_i$ is a compact metric space; ${\mathfrak{X}}_i$ is its Borel-$\sigma$ algebra. Endowed with the product topology, the finite Cartesian product X is a compact metric space and $\mathfrak{X}$ is its Borel-$\sigma$ algebra. A typical element is denoted by $a\in X$. There is a measure $\kappa$ such that $(X,\mathfrak{X},\kappa )$ is a complete measure space.

($\mathcal{A}4$)

For each i, define $T_i\equiv S\times X\times S_i$. A typical element is denoted by $(s^{-},a^{-},s_i)$ or $t_i$. Notice that $T_i$ is a complete separable metric space. Let ${\mathfrak{T}}_i$ be its Borel-$\sigma$ algebra. There is an atomless probability measure ${\lambda }_i$ such that $(T_i,{\mathfrak{T}}_i,{\lambda }_i)$ is complete measure space. Endowed with the product topology, the Cartesian product T is also a complete separable metric space and a product of $\sigma$-algebras $\mathfrak{T}$ is its Borel-$\sigma$ algebra’ $\lambda$ is a product probability measure such that $\lambda ={\lambda }_1\times \cdots \times {\lambda }_n$.

($\mathcal{A}5$)

For each $i\in \mathcal{I}$, $A_i$ is nonempty, compact valued, and lower measurable.10

($\mathcal{A}6$)

The expression $u_i(·,·)$ is bounded; there exists $C_i\in {\mathfrak{R}}_{+}$ such that for each $(s,a)$, $\mid u_i(s,a)\mid \le C_i$. For each $a\in X$, $u_i(·,a)$ is measurable; for each $s\in S$, $u_i(s,·)$, continuous.

($\mathcal{A}7$)

For each $(s^{-},a^{-})$, there is a prior distribution for Nature, $\tau (·\mid s^{-},a^{-})\in \Delta \left(S\right)$ about the current type s. For each $Z\in \mathfrak{S}$, $\tau (Z\mid ·,·)$ is jointly measurable; $\tau (·\mid s^{-},a^{-})$ is absolutely continuous with respect to the atomless measure $\varphi$.

($\mathcal{A}8$)

For each $(s^{-},a^{-},s_i)$, there are beliefs $\eta (·\mid s^{-},a^{-},s_i)\in \Delta \left(S_{-i}\right)$ about the other players’ current types. This is the $s_i$-section of $\tau (·\mid s^{-},a^{-})$. Given $(s^{-},a^{-})$ and for each $s_i$, the mapping $(s^{-},a^{-},s_i)\mapsto \eta (·\mid s^{-},a^{-},s_i)$ is a regular conditional probability on $S_{-i}$. For each $Z_{-i}$, $\eta (Z_{-i}\mid ·,·,·)$ is jointly measurable; $\eta (·\mid s^{-},a^{-},s_i)$ is absolutely continuous with respect to the atomless product measure ${\varphi }_1\times \cdots \times {\varphi }_{i-1}\times {\varphi }_{i+1}\times \cdots \times {\varphi }_n$.

($\mathcal{A}9$)

For each $(s,a)$, for each player i, there is an anticipation $\mu (·\mid s,a)\in \Delta \left(S_i\right)$ about the future type of player i oneself. This is a marginal distribution derived by $\tau (·\mid s,a)$ (Note that $S_i^{+}=S_i$).11

For each $Z_i\in {\mathfrak{S}}_i$, $\mu (Z_i\mid s,a)$ is jointly measurable in $(s,a)$; $\mu (·\mid s,a)$ is absolutely continuous with respect to the complete, atomless measure ${\varphi }_i$. For $\varphi$-almost all s, the mapping $a\mapsto \mu (·\mid s,a)$ is norm-continuous.

($\mathcal{A}10$)

For each i, $\tau$ is decomposed into $\mu$ and $\eta$: for each $(s,a)\in S\times X$, and all $Z^{+}\in \mathfrak{S}$, I have the following:

$$\begin{array}{c}\hfill \tau (Z^{+}\mid s,a)={\int }_{s_i^{+}}{\int }_{s_{-i}^{+}}{\mathcal{I}}_Z(s_i^{+},s_{-i}^{+})\eta (d{s}_{-i}^{+}\mid s,a,s_i^{+})\mu (d{s}_i^{+}\mid s,a),\end{array}$$

(2)

Example 1.

Consider a dynamic Bayesian Cournot competition as follows. There are two firms and these firms face unknown production cost types each period. Other than the cost type structure, the other components of Cournot competition are applied as usual. The price of the good is determined by a market demand function:

$$\begin{array}{c}\hfill p(q_1,q_2)=\alpha -q_1-q_2{q}_1,q_2\in [0,\alpha ]and\alpha >1.\end{array}$$

(3)

The condition $q_1,q_2\in [0,\alpha ]$ helps the price remain above 0. The payoff functions $u_1$ and $u_2$ are as follows:

$$\begin{array}{c}\hfill u_1(c_1,c_2,q_1,q_2)=(p-c_1)·q_{1}and{u}_2(c_1,c_2,q_1,q_2)=(p-c_2)·q_2.\end{array}$$

(4)

The key distinction of the model is at the cost type structure. Assume cost type space of player i is given by $[0,\alpha ]$. Consider an area that is given by a square made of four points, $(0,0)$, $(0,\alpha )$, $(\alpha ,0)$, and $(\alpha ,\alpha )$, and called Region Z. Now, the cost types of firms are drawn from a joint probability distribution that is determined by the previous cost levels and the previous production levels of the firms: The shape of the joint probability distribution is determined by $c_1^{-},c_2^{-},q_1^{-}$ and $q_2^{-}$ over Region Z.12 For a specific example, assume the cost types are assumed to follow a joint distribution over Region Z:

$$\begin{array}{c}\hfill (c_1,c_2)\sim F^{c_1^{-},c_2^{-},q_1^{-},q_2^{-}}\left(RegionZ\right).\end{array}$$

(5)

Any joint probability measure with respect to $(c_1,c_2)$ is allowed as a cost type structure in this sample model if the following conditions are satisfied: (1) the joint probability measure has $[0,\alpha ]\times [0,\alpha ]$ (Region Z) as its support; (2) it is differentiable with respect to $c_i$ over the type space $[0,\alpha ]$; and (3) it is continuous with respect to $q_i$ over the action space $[0,\alpha ]$. For example, for each i, consider a concave quadratic function $f_i(c_i\mid c_1^{-},c_2^{-},q_1^{-},q_2^{-})\subset {\mathfrak{R}}_{+}$, which is peaked at $\alpha ·{(q_1^{-}+q_2^{-})}^2/2-(c_1^{-}+c_2^{-})/2$ and goes through zeros at $c_i=0$ or $c_i=\alpha$, i.e., $f_i(0\mid c_1^{-},c_2^{-},q_1^{-},q_2^{-})=0$ or $f_i(\alpha \mid c_1^{-},c_2^{-},q_1^{-},q_2^{-})=0$, as well as of which area is one (1). Compose a joint probability distribution by the product of the aforementioned functions $f(c_1\mid c_1^{-},c_2^{-},q_1^{-},q_2^{-})\times f(c_2\mid c_1^{-},c_2^{-},q_1^{-},q_2^{-})$.13 Then, the support of the joint distribution is the square made of four points, $(0,0)$, $(0,\alpha )$, $(\alpha ,0)$, and $(\alpha ,\alpha )$, and it satisfies $\left(\mathcal{A}8\right)$. The joint probability distribution can be more general, not necessarily being a product of two marginal distributions. A uniform joint distribution over Region Z is also allowed.

In this example, the set of player indices $\mathcal{I}=\left\{1,2\right\}$. The costs of firms $c_1,c_2$ are considered as types $s_i$ in the above model setup, and the type space for each firm $S_i$ is given by a closed interval on the real line $[0,\alpha ]$. Then, it satisfies the condition $\left(\mathcal{A}1\right)$ of a complete separable metric space. The produced quantities $q_1,q_2$ are actions $a_i$ in the above model setup. The action space for each firm is given by a compact metric space $q_1,q_2\in [0,\alpha ]$ on the real line. As the type and action spaces are all on the real line, $\left(\mathcal{A}2\right)$, $\left(\mathcal{A}3\right)$, and $\left(\mathcal{A}4\right)$ are fulfilled. Assumption $\left(\mathcal{A}5\right)$ is satisfied as long as, for example, a pure action strategy $q_1:c_1\mapsto [0,\alpha ]$ and $q_2:c_2\mapsto [0,\alpha ]$ are continuous in $c_1$ and $c_2$, respectively. As $q_1$ and $q_2$ are bounded, the market price $p=\alpha -q_1-q_2$ is bounded. Then, it is clear that payoff function for firm i, for each $(q_1,q_2)$, $u_i(·,q_1,q_2)$, is bounded and continuous. Also, given $(c_1,c_2,)$, $u_i(c_1,c_2,·)$ is integrable.

The probability measures $\tau (·),\mu (·),\eta (·)$ in this example are as follows: from player 1’s perspective,14

$$\begin{array}{ccc}\hfill \tau (·\mid s^{-},a^{-})\in \Delta \left(S\right)& \iff & (c_1,c_2)\sim F^{c_1^{-},c_2^{-},q_1^{-},q_2^{-}}(·),\hfill \\ \hfill \eta (·\mid s^{-},a^{-},s_i)\in \Delta \left(S_{-i}\right)& \iff & c_2^e\sim F^{c_1^{-},c_2^{-},q_1^{-},q_2^{-}}\left({\widehat{c}}_1\right),\hfill \\ \hfill \mu (·\mid s^{-},a^{-})\in \Delta \left(S_i\right)& \iff & c_1\sim {\int }_{c_2}{F}^{c_1^{-},c_2^{-},q_1^{-},q_2^{-}}d{c}_2,\hfill \end{array}$$

(6)

where $c_2^e$ is what player 1 believes as player 2’s type when player 1’s cost type is ${\widehat{c}}_1$. Since the model is on the real line, assumptions $(\mathcal{A}7-10)$ are satisfied as long as the probability function (1) has Region Z as its support, (2) is differentiable with respect to $c_i$ over $[0,\alpha ]$, and (3) is continuous with respect to $q_i$ over $[0,\alpha ]$. By contrast, if Region Z were a triangle made of three points of $(0,0)$, $(0,\alpha )$, and $(\alpha ,0)$, it would violate $\left(\mathcal{A}8\right)$ because the conditional distribution for $c_i$ given $c_{-i}$ has a jump on the cost type space $[0,\alpha ]$. I keep using this example to describe the equilibrium concept and the sketch of the proof.

2.2. Timing

Suppose the game reaches the middle of the stage of time $(t-1)$. Players choose actions $a^{-}$ given their own type without knowing others. However, the payoff function for player i depends on the types and actions of all players. At the end of the previous period, the type profile and action profile $(s^{-},a^{-})$ are revealed and the payoff $u_i(s^{-},a^{-})$ is given to player i for all i. Next, the time-t stage game proceeds as follows (Figure 1 depicts the timeline):

• Nature moves to draw each player’s type based on the Markov process $\tau (·\mid s^{-},a^{-})$ for all i.
• For each i, player i whose type is $s_i$ chooses actions based on their beliefs $\eta (·\mid s^{-},a^{-},s_i)$, maximizing their discounted sum of expected payoffs.
• At the end of the current period, the realized type profile and action profile $(s,a)$ are revealed, and player i earns payoff $u_i(s,a)$. Nature moves to draw each player’s next period type based on the Markov process $\tau (·\mid s,a)$ for all i.

2.3. Stationary Bayesian–Markov Equilibrium

A stationary Bayesian–Markov strategy for player i is a measurable mapping ${\sigma }_i:S\times X\times S_i\to \Delta \left(X_i\right)$. For each $(s^{-},a^{-},s_i)$, a probability measure ${\sigma }_i(s^{-},a^{-},s_i)$ assigns probability one to $A_i\left(s_i\right)$. Let ${\mathrm{\Sigma }}_i$ denote the set of stationary Bayesian–Markov strategies:

$$\begin{array}{c}\hfill {\mathrm{\Sigma }}_i=\{{\sigma }_i\mid {\sigma }_i\in \mathcal{M}(S\times X\times S_i,\Delta \left(X_i\right)),{\sigma }_i(s^{-},a^{-},s_i)\left(A_i\left(s_i\right)\right)=1\}.\end{array}$$

(7)

For each $s\in S$, let $\sigma (s^{-},a^{-},s)$ denote the product probability measure ${\sigma }_1(s^{-},a^{-},s_1)\times \cdots \times {\sigma }_n(s^{-},a^{-},s_n)$.15 In addition, $\sigma$ denotes a profile of mappings $({\sigma }_1,\cdots ,{\sigma }_n)$ and $\mathrm{\Sigma }$ denotes the set of stationary Bayesian–Markov strategy profiles $\sigma$.

For each $\sigma$, player i’s interim expected continuation value function $v_i(·\mid \sigma ):S\times X\times S_i\to \mathfrak{R}$ is a measurable function. To define the interim expected continuation value function $v_i$, remember that player i with the current type $s_{i,t}$ only has beliefs over the current realization $(s_t,a_t)$ and the future type over $s_{i,t+1}$. Then, player i will again imagine, in the next period $(t+1)$, if they learn their type is $s_{i,t+1}$, they will have new beliefs over the type and action profile realization $(s_{t+1},a_{t+1})$ and the future type in period $(t+2)$, $s_{i,t+2}$. That is, the interim expected continuation value function relies on the expectation over the current beliefs of player i and their conjecture over the future beliefs conditional on the current beliefs. Therefore, the value function demonstrates a fractal structure in which there is an expectation within an expectation ad infinitum. To see clearly, denote $E_{s_{-i,t},a_t}\left[u_i(s_t,a_t)\right]$ as follows when player i’s belief over the others’ type and realized action profile $(s_{-i,t},a_t)$ is given by $\eta (s_{-i,t}\mid s_{t-1},a_{t-1},s_i)\times \sigma (s_{t-1},a_{t-1},s_t)$:

$$\begin{array}{c}{E}_{s_{-i,t},a_t}\left[u_i\left(s_t,a_t\right);s_{i,t},\eta ,\sigma \right]\hfill \\ ={\int }_{s_{-i,t}}{\int }_{a_t}{u}_i(s_t,a_t)\sigma (s_{t-1},a_{t-1},s_t)\left(da\right)\eta (d{s}_{-i,t}\mid s_{t-1},a_{t-1},s_{i,t})\hfill \end{array}$$

(8)

In addition, denote $E_{s_{i,t+1},s_{-i,t+1},a_{t+1}}\left[u_i\left(s_{t+1},a_{t+1}\right);\mu ,\eta ,\sigma \right]$ as follows to express ex ante expected payoff at the beginning of time $(t+1)$, which is before player i learns their type $s_{i,t+1}$:

$$\begin{array}{c}{E}_{s_{i,t+1},s_{-i,t+1},a_{t+1}}\left[u_i\left(s_{t+1},a_{t+1}\right);\mu ,\eta ,\sigma \right]\hfill \\ ={\int }_{s_{i,t+1}}{\int }_{s_{-i,t+1}}\left[\begin{array}{c}{\int }_{a_{t+1}}{u}_i(s_{t+1},a_{t+1})\sigma (s_t,a_t,s_{i,t+1})\left(da\right)\hfill \\ \times \eta (d{s}_{-i,t+1}\mid s_t,a_t,s_{i,t+1})\mu \left(d{s}_{i,t+1}\mid s_t,a_t\right)\hfill \end{array}\right]\hfill \\ ={\int }_{s_{i,t+1}}{E}_{s_{-i,t},a_t}\left[u_i(s_{t+1},a_{t+1});\eta ,\sigma \right]\mu \left(d{s}_{i,t+1}\mid s_t,a_t\right)\hfill \end{array}$$

(9)

Then, the interim expected continuation value function is, for each $(s_{t-1},a_{t-1},s_{i,t})$,16

$$\begin{array}{c}{v}_i(s_{\tau -1},a_{\tau -1},s_{i,\tau }\mid \sigma )\hfill \\ =E_{s_{-i,t},a_t}\left\{\begin{array}{c}(1-{\delta }_i)u_i(s_t,a_t)\hfill \\ +{\delta }_i{E}_{s_{i,t+1},s_{-i,t+1},a_{t+1}}\left[\begin{array}{c}(1-{\delta }_i)u_i(s_{t+1},a_{t+1})\hfill \\ +{\delta }_i{E}_{s_{i,t+2},s_{-i,t+2},a_{t+2}}\left(\begin{array}{c}(1-{\delta }_i)u_i(s_{t+2},a_{t+2})\hfill \\ +\cdots \hfill \end{array}\right)\hfill \end{array}\right]\hfill \end{array}\right\}\hfill \end{array}$$

(10)

$$\begin{array}{c}=E_{s_{-i,t},a_t}\left\{\begin{array}{c}(1-{\delta }_i)u_i(s_t,a_t)\hfill \\ +{\delta }_i{\int }_{s_{i,t+1}}{E}_{s_{-i,t+1},a_{t+1}}\left[\begin{array}{c}(1-{\delta }_i)u_i(s_{t+1},a_{t+1})\hfill \\ +\cdots \hfill \end{array}\right]\mu \left(d{s}_{i,t+1}\mid s_{t,}{a}_t\right)\hfill \end{array}\right\}\hfill \end{array}$$

(11)

As Equation (11) is a fractal, it is more convenient to express by recursion: For each $(s^{-},a^{-},s_i)$,

$$\begin{array}{c}{v}_i(s^{-},a^{-},s_i\mid \sigma )\hfill \\ ={\int }_{s_{-i}}{\int }_a\left[\begin{array}{c}(1-{\delta }_i)u_i(s,a)\hfill \\ +{\delta }_i{\int }_{s_i^{+}}{v}_i(s,a,s_i^{+}\mid \sigma )\mu (d{s}_i^{+}\mid s,a)\hfill \end{array}\right]\sigma (s^{-},a^{-},s)\left(da\right)\eta (d{s}_{-i}\mid s^{-},a^{-},s_i).\hfill \end{array}$$

(12)

A profile of stationary Bayesian–Markov strategies $\sigma$ is a stationary Bayesian–Markov equilibrium if, for each $(s^{-},a^{-},s)$, each player i’s strategy ${\sigma }_i$ maximizes i’s interim expected continuation values. That is, given $(s^{-},a^{-})$, for each $s_i$, $\sigma (s^{-},a^{-},s_i)$ puts probability one on the set of solution to

$$\begin{array}{c}\underset{a_i\in A_i\left(s_i\right)}{max}{\int }_{s_{-i}}{\int }_{a_{-i}}\left[\begin{array}{c}(1-{\delta }_i)u_i(s,a)\hfill \\ +{\delta }_i{\int }_{s_i^{+}}{v}_i(s,a,s_i^{+}\mid a_i,{\sigma }_{-i})\mu (d{s}_i^{+}\mid s,a)\hfill \end{array}\right]\hfill \\ \times {\sigma }_{-i}(s^{-},a^{-},s_{-i})\left(d{a}_{-i}\right)\eta (d{s}_{-i}\mid s^{-},a^{-},s_i).\hfill \end{array}$$

(13)

By the one-shot deviation principle, every stationary Bayesian–Markov equilibrium is subgame perfect.

Example 2.

Continued from the above dynamic Bayesian Cournot competition, the strategies for each player can be any mapping from a cost type of player i to a probability distribution over a subset of the action space, $q_i\in [0,\alpha ]$, e.g., $c_i\mapsto \Delta \left(A_i\right)$, where $A_i\in [0,\alpha ]$, given previous $c_1^{-},c_2^{-},q_1^{-},q_2^{-}$. Among possible strategies, the equilibrium strategy is obtained by maximization of the interim expected continuous value function. Recall the stage payoff function is $u_i(c_1,c_2,q_1,q_2)=(\alpha -q_1-q_2)·q_1-c_1·q_1$, and assume $A\left(c_i\right)$ is a singleton, that is, $q_i\left(c_i\right)$ is a pure action strategy. Solving the following dynamic optimization, equilibrium strategy for player 1 is given by

$$\begin{array}{c}{q}_1(c_1^{-},c_2^{-},q_1^{-},q_2^{-},c_1)\hfill \\ =argmax{\int }_{c_2}{\int }_{q_2}\left[\begin{array}{c}(1-{\delta }_1)(\alpha -q_1-q_2)·q_1-c_1·q_1\hfill \\ +{\delta }_1{\int }_{c_1^{+}}\left[(\alpha -q_1^{+}-q_2^{+})·q_1^{+}-c_1^{+}·q_1^{+}\right]\mu (d{c}_1^{+}\mid c_1,c_2,q_1,q_2)\hfill \\ +{\delta }_1^2{\int }_{c_1^{++}}\left[(\alpha -q_1^{++}-q_2^{++})·q_1^{++}-c_1^{++}·q_1^{++}\right]\mu (d{c}_1^{++}\mid c_1^{+},c_2^{+},q_1^{+},q_2^{+})\hfill \\ +\cdots \hfill \end{array}\right]\hfill \\ \times {\mathbb{1}}_{q\left(c_2\right)}\left(d{q}_2\right)\times \eta (d{c}_2\mid c_1^{-},c_2^{-},q_1^{-},q_2^{-},c_1).\hfill \end{array}$$

Then, by taking derivatives and solving the system of equations, for each $q_i$

$$\begin{array}{c}\hfill q_i(c_i\mid c_1^{-},c_2^{-},q_1^{-},q_2^{-})={\displaystyle \frac{1}{3(1-\delta )}}\left[\begin{array}{c}2\left(1-\delta \right)\left(\alpha -c_1\right)-\left(1-\delta \right)\left(\alpha -\mathbb{E}{c}_2\right)\hfill \\ +\delta \mathbb{E}\left[{\displaystyle \frac{\partial \mathbb{E}\left(c_2^{+}·q_2^{+}\right)}{\partial q_2}}\right]-2\delta {\displaystyle \frac{\partial \mathbb{E}\left(c_1^{+}·q_1^{+}\right)}{\partial q_1}}+{\displaystyle \frac{\left(1-\delta \right)}{2}}\left(c_1-\mathbb{E}{c}_1\right)\hfill \\ +{\displaystyle \frac{\left(1-\delta \right)}{2}}\left({\displaystyle \frac{\partial \mathbb{E}\left(c_1^{+}·q_1^{+}\right)}{\partial q_1}}-\mathbb{E}\left[{\displaystyle \frac{\partial \mathbb{E}\left(c_1^{+}·q_1^{+}\right)}{\partial q_1}}\right]\right)\hfill \end{array}\right]\end{array}$$

(14)

Observe that the equilibrium strategy is forward-looking: $q_i$ takes into account the variation in future cost type distribution due to changes in $q_i$. However, it only cares about the next period because the optimization problem is recursive. Using this recursiveness, the existence theorem for the continuous space dynamic stochastic game is presented in the next section.

3. Existence Theorem

Theorem 1

(Existence Theorem). For every Bayesian stochastic game with periodic revelation, there exists a stationary Bayesian–Markov equilibrium.

First, I construct the set V of interim expected continuation value function profiles in the following two paragraphs.17 Fix $i\in \mathcal{I}$. Let $L^{\infty }(T_i,{\lambda }_i)$ be the collection of ${\lambda }_i$-equivalence classes of ${\lambda }_i$-essentially bounded, measurable extended real-valued functions from $S\times X\times S_i$ to $\mathfrak{R}$; $L^{\infty }(T_i,{\lambda }_i)$ is equipped with the usual norm ${\Vert ·\Vert }_{L^{\infty }}$, which gives the smallest essential upper bound; that is, ${\Vert f\Vert }_{L^{\infty }}=infM$ if $\mid f(s^{-},a^{-},s_i)\mid \le M$ for ${\lambda }_i$-almost all $(s^{-},a^{-},s_i)$; $L^1(T_i,{\lambda }_i)$ is the collection of ${\lambda }_i$-equivalence classes of integrable functions from $T_i=S\times X\times S_i$ to $\mathfrak{R}$; $L^1(T_i,{\lambda }_i)$ is equipped with ${\Vert ·\Vert }_{L^1}$ such that ${\Vert g\Vert }_{L^1}={\int }_{T_i}\mid g\mid d{\lambda }_i={\int }_{S_i}{\int }_X{\int }_S\mid g\mid d\varphi d\kappa d{\varphi }_i$.18 By the Riesz representation theorem (see Royden–Fitzpatrick ([19], hereafter RF, p. 400), $L^{\infty }(T_i,{\lambda }_i)$ is the dual space of $L^1(T_i,{\lambda }_i)$, and it is endowed with weak-* topology. By Proposition 14.21 of RF (p. 287), $L^{\infty }(T_i,{\lambda }_i)$ is a locally convex Hausdorff topological vector space. Let $L^{\infty }(T,\lambda )$ denote the Cartesian product $L^{\infty }(T_1,{\lambda }_1)\times \cdots \times L^{\infty }(T_n,{\lambda }_n)$. Then, endowed with product topology, $L^{\infty }(T,\lambda )$ is a locally convex Hausdorff topological vector space.

Let $V_i$ consist of functions $v_i\in L^{\infty }(T_i,{\lambda }_i)$ with $\Vert v_i{\Vert }_{L^{\infty }}\le C_i$; that is, $\mid v_i(s^{-},a^{-},s_i)\mid \le C_i$ for ${\lambda }_i$-almost all $(s^{-},a^{-},s_i)$. The constant $C_i\in {\mathfrak{R}}_{+}$ is of which $\mid u_i(s,a)\mid \le C_i$ for all $(s,a)$. Clearly, $V_i$ is nonempty and convex. Then, the Cartesian product $V=V_1\times \cdots \times V_n$ is also non-empty and convex. By Alaoglu’s theorem (RF, p. 299), $V_i$ is compact. The finite product V is also compact (see Theorem 26.7, Munkres [20], p. 167).

Now I show V is metrizable in the weak-* topology. Since $T_i$ is separable metric space, its Borel $\sigma$-algebra is countably generated. $L^1(T_i,{\lambda }_i)$ is separable.19 By Corollary 15.11 of RF (p. 306), $V_i$ is metrizable in the weak-* topology. The Cartesian product V is metrizable in the product weak-* topology, so it is compact if and only if sequentially compact (see Theorem 28.2, Munkres [20], p. 179).

In order to see if the correspondence $v\mapsto M_v$ is nonempty, closed graph, and convex valued, I consider the following induced game. Eventually I want to show that the set of interim expected payoffs profiles $E_v^{(s^{-},a^{-})}\left(s\right)$ from the induced game is in fact equivalent to its convexification $co{E}_v^{(s^{-},a^{-})}\left(s\right)$. Let ${\mathrm{\Gamma }}_v^{(s^{-},a^{-})}\left(s\right)$ denote a state-s induced game of a Bayesian stochastic game given a profile of continuation value functions v and the realized state-action profiles $(s^{-},a^{-})$ in the previous period:

$$\begin{array}{c}\hfill {\mathrm{\Gamma }}_v^{(s^{-},a^{-})}\left(s\right)=(\mathcal{I},{((S_i,{\mathfrak{S}}_i),X_i,A_i\left(s_i\right),{\widehat{U}}_i^{(s^{-},a^{-})}(·\mid v),\mu (·\mid ·,·),\eta (·\mid s^{-},a^{-},s_i))}_{i\in \mathcal{I}}).\end{array}$$

(15)

Here, ${\widehat{U}}_i^{(s^{-},a^{-})}(·\mid v):S_i\times \mathrm{\Sigma }\to \mathfrak{R}$ is defined as follows: for each $s_i$, each $\sigma$,

$$\begin{array}{c}{\widehat{U}}_i^{(s^{-},a^{-})}(s_i,\sigma \mid v)\hfill \\ ={\int }_{s_{-i}}{\int }_a\left[\begin{array}{c}(1-{\delta }_i)u_i(s,a)\hfill \\ +{\delta }_i{\int }_{s_i^{+}}{v}_i(s,a,s_i^{+})\mu (d{s}_i^{+}\mid s,a)\hfill \end{array}\right]\sigma (s^{-},a^{-},s)\left(da\right)\eta (d{s}_{-i}\mid s^{-},a^{-},s_i).\hfill \end{array}$$

(16)

In the interim stage of a general one-shot Bayesian game, suppose players use behavioral strategies. Knowing their realized type ($s_i$), player i exerts a mixed action (${\beta }_i\left(s_i\right)\in \Delta \left(X_i\right)$), induced by their behavioral strategy (a measurable mapping ${\beta }_i:S_i\to \Delta \left(X_i\right)$). In the induced game of the Bayesian stochastic game in this paper, players use stationary Bayesian–Markov strategies, and it is essentially the same to the behavioral strategy in a one-shot Bayesian game. The mixed actions induced by stationary Bayesian–Markov strategy profile $\sigma =({\sigma }_1,\dots ,{\sigma }_n)$ of all players determine a product probability measure $\sigma (s^{-},a^{-},s)={\sigma }_1(s^{-},a^{-},s_1)\times \cdots \times {\sigma }_n(s^{-},a^{-},s_n)$. The difference between a stationary Bayesian–Markov strategy and a behavioral strategy is that, in the former, beliefs are given by the consequence of the previous stage game, and, thus, mixed actions depend on the previously realized type and action profiles as well as the current own type of player i oneself.20

The space $\Delta \left(X_1\right)\times \cdots \times \Delta \left(X_n\right)$ and $\Delta \left(A_1\left(s_1\right)\right)\times \cdots \times \Delta \left(A_n\left(s_n\right)\right)$ are endowed with the product weak topology. By Theorem 2.8 of Billingsley ([16], p. 23), ${\alpha }^m\to \alpha$ in $\Delta \left(X_1\right)\times \cdots \times \Delta \left(X_n\right)$ if and only if ${\alpha }_i^m\to {\alpha }_i$ in $\Delta \left(X_i\right)$ for each i.

Lemma 1.

Given $(s^{-},a^{-})$, for each v and each σ, ${\widehat{U}}_i^{(s^{-},a^{-})}(·,\sigma \mid v)$ is measurable in $s_i$. Then, ${\widehat{U}}_i^{(·,·)}(·,\sigma \mid v)$ is measurable in $(s^{-},a^{-},s_i)$. Given $(s^{-},a^{-})$, for each $s_i$, ${\widehat{U}}_i^{(s^{-},a^{-})}(s_i,·\mid ·)$ is jointly continuous in $(\sigma ,v)$.

Proof. 

Define

$$\begin{array}{c}{U}_i^{(s^{-},a^{-})}(s,\sigma \mid v)\hfill \\ ={\int }_a\left[(1-{\delta }_i)·u_i(s,a)+{\delta }_i{\int }_{s_i^{+}}{v}_i(s,a,s_i^{+})·\mu (d{s}_i^{+}\mid s,a)\right]·\sigma (s^{-},a^{-},s)\left(da\right)\hfill \end{array}$$

(17)

and

$$\begin{array}{c}\hfill {\widehat{U}}_i^{(s^{-},a^{-})}(s_i,\sigma \mid v)={\int }_{s_{-i}}{U}_i^{(s^{-},a^{-})}(s,\sigma \mid v)\eta (d{s}_{-i}\mid s^{-},a^{-},s_i).\end{array}$$

(18)

For each $Z_i\in {\mathfrak{S}}_i$, $\mu (Z_i\mid s,a)$ is measurable in $s_i$; $v_i(s,a,s_i^{+})$ is bounded, then Theorem 19.7 of Aliprantis and Border ([22], hereafter AB, p. 627) implies ${\int }_{s_i^{+}}{v}_i(s,a,s_i^{+})\mu (d{s}_i^{+}\mid s,a)$ is measurable in $s_i$. Recall that, given $(s^{-},a^{-})$, for each $Z_{-i}\in {\mathfrak{S}}_{-i}$, $\eta (Z_{-i}\mid s^{-},a^{-},·)$ is measurable. Since $U_i^{(s^{-},a^{-})}(s,\sigma \mid v)$ is bounded, ${\widehat{U}}_i^{(s^{-},a^{-})}(·,\sigma \mid v)$ is measurable. Similarly, since for each $Z_{-i}\in {\mathfrak{S}}_{-i}$, $\eta (Z_{-i}\mid ·,·,·)$ is measurable, ${\widehat{U}}_i^{(·,·)}(·,\sigma \mid v)$ is also measurable.

Fix $(s^{-},a^{-})$. Consider a sequence $\left\{(a^m,v^m)\right\}\to (a,v)$, where for each i, $v_i^m\to v_i$ in weak-* topology. I have

$$\begin{array}{c}\left|{\widehat{U}}_i^{(s^{-},a^{-})}(s_i,a^m\mid v^m)-{\widehat{U}}_i^{(s^{-},a^{-})}(s_i,a\mid v)\right|\hfill \\ =\left|{\int }_{s_{-i}}\left[\begin{array}{c}(1-{\delta }_i)u_i(s,a^m)-(1-{\delta }_i)u_i(s,a)\hfill \\ +{\delta }_i{\int }_{s_i^{+}}{v}_i^m(s,a^m,s_i^{+})\mu (d{s}_i^{+}\mid s,a^m)\hfill \\ -{\delta }_i{\int }_{s_i^{+}}{v}_i(s,a,s_i^{+})\mu (d{s}_i^{+}\mid s,a)\hfill \end{array}\right]\times \eta (d{s}_{-i}\mid s^{-},a^{-},s_i)\right|.(\forall i)\hfill \end{array}$$

(19)

Since $u_i(s,·)$ is continuous, $\left|(1-{\delta }_i)u_i(s,a^m)-(1-{\delta }_i)u_i(s,a)\right|\to 0$, and

$$\begin{array}{c}\left|{\int }_{s_i^{+}}{v}_i^m(s,a^m,s_i^{+})\mu (d{s}_i^{+}\mid s,a^m)-{\int }_{s_i^{+}}{v}_i(s,a,s_i^{+})\mu (d{s}_i^{+}\mid s,a)\right|\hfill \\ \le \left|{\int }_{s_i^{+}}{v}_i^m(s,a^m,s_i^{+})\mu (d{s}_i^{+}\mid s,a^m)-{\int }_{s_i^{+}}{v}_i^m(s,a^m,s_i^{+})\mu (d{s}_i^{+}\mid s,a)\right|\hfill \\ +\left|{\int }_{s_i^{+}}{v}_i^m(s,a^m,s_i^{+})\mu (d{s}_i^{+}\mid s,a)-{\int }_{s_i^{+}}{v}_i(s,a,s_i^{+})\mu (d{s}_i^{+}\mid s,a)\right|\hfill \\ \le C_i·\left||\mu (·\mid s,a^m)-\mu (·\mid s,a)\right||\hfill \\ +\left|{\int }_{s_i^{+}}[v_i^m(s,a^m,s_i^{+})-v_i(s,a,s_i^{+})]\mu (d{s}_i^{+}\mid s,a)\right|\hfill \end{array}$$

(20)

The first inequality is by the triangle inequality. The second inequality is by the fact that $v_i^{(s^{-},a^{-})}$ is essentially bounded by $C_i$ and that $|\int fd\mu |\le \int \left|f\right|d\mu$.

As $v_i^m\to v_i$ in weak-* topology, by Proposition 3.13 of Brezis ([23], p. 63), for any given point of $(s,a,s_i^{+})\in T_i$, $v_i^m(s,a,s_i^{+})\to v_i(s,a,s_i^{+})$. This induces ${\int }_{S_i^{+}}{v}_i^m(·,a^m,s_i^{+})\mu (d{s}_i^{+}\mid s,a)\to {\int }_{T_i}{v}_i(·,a^m,s_i^{+})\mu (d{s}_i^{+}\mid s,a)$. Moreover, as $a_m\to a$ in X, $v_i(s,a^m,s_i^{+})\to v_i(s,a,s_i^{+})$, because $u_i(s,·)$ for given s is continuous, and $v_i$ is a continuous linear functional of $u_i$.21 This gives us $\left|{\int }_{s_i^{+}}[v_i^m(s,a^m,s_i^{+})-v_i(s,a,s_i^{+})]\mu (d{s}_i^{+}\mid s,a)\right|\to 0$. Then, combined with norm continuity of $\mu (·\mid s,a)$ in a, I have the RHS of the last inequality converging to 0. Now, continuity of ${\widehat{U}}_i^{(s^{-},a^{-})}(s_i,\sigma \mid v)$ follows from that given s, the family of real valued functions $\mathcal{U}\equiv {\left\{{\widehat{U}}_i^{(s^{-},a^{-})}(s_i,·\mid v^m)\right\}}_{m\in N}$ is equicontinuous at each a, and the result of Rao [24] under the absolutely continuous information structure holds. □

Fix $(s^{-},a^{-})$ and v. For each s, let $B_v^{(s^{-},a^{-})}\left(s\right)$ be the set of mixed action profiles induced by Bayesian–Nash equilibria of ${\mathrm{\Gamma }}_v^{(s^{-},a^{-})}$ (s). Then

$$\begin{array}{c}{B}_v^{(s^{-},a^{-})}\left(s\right)\hfill \\ \left\{\begin{array}{c}\sigma (s^{-},a^{-},s)\hfill \\ \in \Delta \left(A_1\left(s_1\right)\right)\times \cdots \times \Delta \left(A_n\left(s_n\right)\right)\hfill \end{array}\left|\begin{array}{c}\sigma \in \mathrm{\Sigma };\mathrm{for}\mathrm{each}j\in \mathcal{I},\hfill \\ \mathrm{and}\mathrm{for}\mathrm{each}i\in \mathcal{I}\setminus \left\{j\right\}\hfill \\ {\sigma }_j\mathrm{satisfies}\mathrm{payoff}-\mathrm{indifference}\hfill \\ \mathrm{among}i’\mathrm{s}\mathrm{actions}\hfill \\ {\widehat{U}}_i^{(s^{-},a^{-})}(h_i,s_i,\sigma \mid v)\hfill \\ {\displaystyle =\underset{\forall a_i\in A_i\left(s_i\right)}{max}{\widehat{U}}_i^{(s^{-},a^{-})}(h_i,s_i,a_i,{\sigma }_{-i}\mid v)}\hfill \end{array}\right.\right\}.\hfill \end{array}$$

(21)

The condition holds if and only if player j’s equilibrium mixed action makes all the other player i’s choice of pure actions $a_i$ that belongs to the support of ${\sigma }_i\left(s_i\right)$ indifferent for player i in terms of the interim expected continuation value. This condition induces player i to mix their actions. As a result, player i’s mixed action gives the same payoff (in terms of the interim expected continuation value) to any pure action from the support of ${\sigma }_i\left(s_i\right)$.

Now define

$$\begin{array}{c}\hfill {\widehat{\xi }}^{(s^{-},a^{-})}(s,\sigma ;v)=\sum _{i=1}^n\left[{\widehat{U}}_i^{(s^{-},a^{-})}(s_i,\sigma ;v)-\underset{a_i\in A_i\left(s_i\right)}{max}{\widehat{U}}_i^{(s^{-},a^{-})}(s_i,a_i,{\sigma }_{-i};v)\right].\end{array}$$

(22)

Then, for all $\sigma (s^{-},a^{-},s)\in B_v^{(s^{-},a^{-})}\left(s\right),{\widehat{\xi }}^{(s^{-},a^{-})}(s,\sigma ;v)=0$. Below I follow Duggan’s [12] approach.

Lemma 2.

For each v, $(s^{-},a^{-},s)\mapsto B_v^{(s^{-},a^{-})}\left(s\right)$ is nonempty, compact valued, and lower measurable.

Proof. 

A correspondence $s\mapsto \Delta \left(A_1\left(s_1\right)\right)\times \cdots \times \Delta \left(A_n\left(s_n\right)\right)$ is lower measurable, nonempty, and compact valued. To see this, I want, for each open subset G of X, the lower inverse image of G to be measurable; $A^l\left(G\right)=\{s\in S\mid A\left(s\right)\cap G\ne \varnothing \}\in \mathfrak{S}$. Since $\mathfrak{S}={\mathfrak{S}}_1\times \cdots \times {\mathfrak{S}}_n$ is generated by measurable rectangles, $A^l\left(G\right)\in \mathfrak{S}$ if and only if $A_i^l\left(G_i\right)\in {\mathfrak{S}}_i$, where $pro{j}_i^{-1}\left(G_i\right)$ is a subbasis element for the product topology on X. From $\left(\mathcal{A}4\right)$, $A_i$ is lower measurable for each i, it is clear that $s\mapsto \left(A_1\left(s_1\right)\right)\times \cdots \times \left(A_n\left(s_n\right)\right)$ is lower measurable. It is also nonempty and compact valued because so is each $A_i$. Then, $s\mapsto \Delta \left(A_1\left(s_1\right)\right)\times \cdots \times \Delta \left(A_n\left(s_n\right)\right)$ is lower measurable, nonempty, and compact valued by Himmelberg and Van Vleck [25].

Notice that

$$\begin{array}{c}{\widehat{\xi }}^{(s^{-},a^{-})}(s,\sigma ;v)\hfill \\ ={\displaystyle \sum _{i=1}^n}\left[{\widehat{U}}_i^{(s^{-},a^{-})}(s_i,\sigma ;v)-\underset{a_i\in A_i\left(s_i\right)}{max}{\widehat{U}}_i^{(s^{-},a^{-})}(s_i,a_i,{\sigma }_{-i};v)\right]\hfill \\ ={\displaystyle \sum _{i=1}^n}\left[\begin{array}{c}{\int }_{s_{-i}}{U}_i(s,\alpha ;v)\eta (d{s}_{-i}\mid s^{-},a^{-},s_i)\hfill \\ -\underset{a_i\in A_i\left(s_i\right)}{max}{\int }_{s_{-i}}{U}_i(s,a_i,{\alpha }_{-i});v)\eta (d{s}_{-i}\mid s^{-},a^{-},s_i)\hfill \end{array}\right]\hfill \\ ={\displaystyle \sum _{i=1}^n}{\int }_{s_{-i}}\left[U_i(s,\alpha ;v)-\underset{a_i\in A_i\left(s_i\right)}{max}{U}_i(s,a_i,{\alpha }_{-i});v)\right]\eta (d{s}_{-i}\mid s^{-},a^{-},s_i)\hfill \\ ={\displaystyle \sum _i}{\int }_{S_i}{\int }_s{\displaystyle \sum _{i=1}^n}\left[U_i(s,\alpha ;v)-\underset{a_i\in A_i\left(s_i\right)}{max}{U}_i(s,a_i,{\alpha }_{-i};v)\right]\tau (ds\mid s^{-},a^{-})d{\delta }_{s_i},\hfill \end{array}$$

(23)

where, given $(s^{-},a^{-},s)$, $\alpha =\sigma (s^{-},a^{-},s)$, and $U_i(s,\alpha ;v)=\left[\begin{array}{c}(1-{\delta }_i)u_i(s,\alpha )\hfill \\ +{\delta }_i{\int }_{s_i^{+}}{v}_i(s,\alpha ,s_i^{+})\mu (d{s}_i^{+}\mid s,\alpha )\hfill \end{array}\right]$. Similarly, $U_i(s,a_i,{\alpha }_{-i};v)=\left[\begin{array}{c}(1-{\delta }_i)u_i(s,a_i,{\alpha }_{-i})\hfill \\ +{\delta }_i{\int }_{s_i^{+}}{v}_i(s,a_i,{\alpha }_{-i},s_i^{+})\mu (d{s}_i^{+}\mid s,a_i,{\alpha }_{-i})\hfill \end{array}\right]$, and ${\delta }_{s_i}$ is the Dirac delta measure concentrated at $s_i$. Define

$$\begin{array}{c}\hfill \xi (s,\alpha ;v)\equiv \sum _{i=1}^n\left[U_i(s,\alpha )-\underset{a_i\in A_i\left(s_i\right)}{max}{U}_i(s,a_i,{\alpha }_{-i})\right].\end{array}$$

Then, $\xi (s,\alpha ;v)=0$ implies ${\widehat{\xi }}^{(s^{-},a^{-})}(s,\sigma ;v)=0$.22 By Lemma 2 of Duggan [12] and measurability of $\tau (Z\mid ·,·)$ for each $Z\in \mathfrak{S}$, $(s^{-},a^{-},s)\mapsto B_v^{(s^{-},a^{-})}\left(s\right)$ is lower measurable. Given $(s^{-},a^{-})$, Balder [18] gives us nonemptiness of $B_v^{(s^{-},a^{-})}\left(s\right)$ for each s. Recall that the interim expected payoff function ${\widehat{U}}_i^{(s^{-},a^{-})}(s_i,·;v)$ is continuous, and the finite product of ${\times }_i\Delta \left(A_i\left(s_i\right)\right)$ is compact. By the theorem of maximum (Theorem 17.31, AB, p. 570), $B_v^{(s^{-},a^{-})}\left(s\right)$ is compact subset of $\Delta \left(A_1\left(s_1\right)\right)\times \cdots \times \Delta \left(A_n\left(s_n\right)\right)$ for each s. □

Given s as realized type profile, define the set of realized payoffs for player i from $B_v^{(s^{-},a^{-})}\left(s\right)$ as $P_{v,i}^{(s^{-},a^{-})}\left(s\right)=U_i(s,B_v^{(s^{-},a^{-})}\left(s\right);v)$. Then, $(s^{-},a^{-},s)\mapsto P_{v,i}^{(s^{-},a^{-})}\left(s\right)$ is nonempty, compact valued, and lower measurable, since $U_i(s,·;v)$ is continuous. By the Kuratowski–Ryll-Nardzewski measurable selection theorem (see Theorem 18.13, AB, p.600), it admits a measurable selector. Then, the correspondence is integrable. The set of interim expected payoffs for player i is denoted as $E_{v,i}^{(s^{-},a^{-})}\left(s_i\right)={\int }_{s_{-i}}{P}_{v,i}^{(s^{-},a^{-})}\left(s\right)\eta (d{s}_{-i}\mid s^{-},a^{-},s_i)$. Let $E_v^{(s^{-},a^{-})}\left(s\right)$ denote the Cartesian product $E_{v,1}^{(s^{-},a^{-})}\left(s_1\right)\times \cdots \times E_{v,n}^{(s^{-},a^{-})}\left(s_n\right)$.

Lemma 3.

For each v, each $(s^{-},a^{-},s)$, $E_v^{(s^{-},a^{-})}\left(s\right)=co{E}_v^{(s^{-},a^{-})}\left(s\right)$.

Proof. 

By Theorem 18.10 of AB (p. 598), the correspondence for each player i’s realized payoffs $(s^{-},a^{-},s)\mapsto P_{v,i}^{(s^{-},a^{-})}\left(s\right)$ is, in fact, measurable. For any $({\tilde{s}}^{-},{\tilde{a}}^{-},{\tilde{s}}_i)$, consider the $({\tilde{s}}^{-},{\tilde{a}}^{-},{\tilde{s}}_i)$-section of the correspondence, $({\tilde{s}}^{-},{\tilde{a}}^{-},{\tilde{s}}_i,s_{-i})\mapsto P_{v,i}^{({\tilde{s}}^{-},{\tilde{a}}^{-})}({\tilde{s}}_i,s_{-i})$; it is clearly measurable. By Theorem 4 of Hildenbrand ([26], p. 64),

$$\begin{array}{c}\hfill {\int }_{s_{-i}}{P}_{v,i}^{({\tilde{s}}^{-},{\tilde{a}}^{-})}({\tilde{s}}_i,s_{-i})\eta (d{s}_{-i}\mid {\tilde{s}}^{-},{\tilde{a}}^{-},{\tilde{s}}_i)={\int }_{s_{-i}}co{P}_{v,i}^{({\tilde{s}}^{-},{\tilde{a}}^{-})}({\tilde{s}}_i,s_{-i})\eta (d{s}_{-i}\mid {\tilde{s}}^{-},{\tilde{a}}^{-},{\tilde{s}}_i).\end{array}$$

(24)

Hence, for each $(s^{-},a^{-},s_i)$, $E_{v,i}^{(s^{-},a^{-})}\left(s_i\right)=co{E}_{v,i}^{(s^{-},a^{-})}\left(s_i\right)$. Since Cartesian product of convex sets of $\mathfrak{R}$ is convex in ${\mathfrak{R}}^n$, $E_v^{(s^{-},a^{-})}\left(s\right)=co{E}_v^{(s^{-},a^{-})}\left(s\right)$. □

Lemma 4.

The mapping $(s^{-},a^{-},s)\mapsto E_v^{(s^{-},a^{-})}\left(s\right)$ is lower measurable, nonempty, compact, and convex valued.

Proof. 

Notice that for each $(s^{-},a^{-},s_i)$, $E_{v,i}^{(s^{-},a^{-})}\left(s_i\right)$ is nonempty and convex. Recall that $(s^{-},a^{-},s)\mapsto P_{v,i}^{(s^{-},a^{-})}\left(s\right)$ admits a measurable selector. Let ${\left\{f_{i,k}\right\}}_k$ denote the set of measurable selectors from the correspondence. Note that ${\left\{f_{i,k}(s^{-},a^{-},s)\right\}}_k\subset \mathfrak{R}$. Since $P_{v,i}^{(s^{-},a^{-})}\left(s\right)$ is compact, for each $f_{i,k}$, there is a family of functions that converge pointwise to $f_{i,k}$ at each $(s^{-},a^{-},s)$. Notice that $U_i(s,\sigma \left(s\right);v)$ is bounded, so a family of functions that converges pointwise to $f_{i,k}$ is uniformly integrable. Recall that each ${\lambda }_i$ is a probability measure. Then, $S\times X\times S_i$ is a set of finite measure; the finite Cartesian product $S\times X\times S$ is also of finite measure. Obviously, the aforementioned family of functions is tight. Applying the Vitali convergence theorem (RF, p. 377), I obtain compactness of ${\int }_{s_{-i}}{P}_{v,i}^{(s^{-},a^{-})}({\overline{s}}_i,s_{-i})\eta (d{s}_{-i}\mid s^{-},a^{-},{\overline{s}}_i)$ for each i. The finite product $E_v^{(s^{-},a^{-})}\left(s\right)$ is therefore compact at each $(s^{-},a^{-},s)$. Applying Tonelli’s theorem (RF, p. 420), I have ${\int }_{s_{-i}}{f}_{i,k}(s^{-},a^{-},{\overline{s}}_i,s_{-i})\eta (d{s}_{-i}\mid {\overline{s}}^{-},a^{-},s_i)$ measurable. Thus, I also get the lower measurability of $(s^{-},a^{-},s)\mapsto {\int }_{s_{-i}}{P}_{v,i}^{(s^{-},a^{-})}({\overline{s}}_i,s_{-i})$$\eta (d{s}_{-i}\mid {\overline{s}}^{-},a^{-},s_i)$. Then, by Proposition 2.3 (ii) of Himmelberg ([27], p. 67), $(s^{-},a^{-},s)\mapsto E_v^{(s^{-},a^{-})}\left(s\right)$ is lower measurable. □

By the Kuratowski–Ryll-Nardzewski measurable selection theorem (Theorem 18.13, AB, p. 600), $(s^{-},a^{-},s)\mapsto E_v^{(s^{-},a^{-})}\left(s\right)$ has a measurable selector. Given v, define $M_v$ to be the set of all $\lambda$-equivalence classes of measurable selectors of $(s^{-},a^{-},s)\mapsto E_v^{(s^{-},a^{-})}\left(s\right)$.

Lemma 5.

The mapping $v\mapsto M_v$ is nonempty, closed graph, and convex valued.

Proof. 

By construction, for each v, $M_v$ is nonempty, closed, and convex. Recall that the interim expected payoff function ${\widehat{U}}_i^{(s^{-},a^{-})}(s_i,\sigma ;·)$ is continuous. By the theorem of maximum, $v\mapsto B_v^{(s^{-},a^{-})}\left(s\right)$ is upper hemicontinuous. Suppose a sequence $(v^m,{\sigma }^m(s^{-},a^{-},s))\to (v,\sigma (s^{-},a^{-},s)$, where ${\sigma }^m(s^{-},a^{-},s)\in B_{v^m}^{(s^{-},a^{-})}\left(s\right)$ and $\sigma (s^{-},a^{-},s)\in B_v^{(s^{-},a^{-})}\left(s\right)$. Lemma 1 tells us that for each player i23,

$$\begin{array}{c}\hfill d_i^m{\left(s_i\right)}^{(s^{-},a^{-})}\equiv \underset{\alpha \in \Delta \left(A\right(s\left)\right)}{max}\mid {\widehat{U}}_i^{(s^{-},a^{-})}(s_i,\alpha ;v^m)-{\widehat{U}}_i^{(s^{-},a^{-})}(s_i,\alpha ;v)\mid \to 0.\end{array}$$

(25)

Now I proceed similarly to Lemma 7 of Nowak and Raghavan [11]. Suppose $g^m\in M_{v^m}$ and $g^m\to g$ in the product weak-* topology on V. I want to show that $g\in M_v$. By Mazur’s theorem and Alaoglu’s theorem (Brezis [23], p. 61 and p. 66), there is sequence $\left\{y^m\right\}$ made up of convex combinations of the $g^m$s that satisfies $\left||y^m-g\right|{|}_{L^{\infty }}\to 0$ as $m\to \infty$. This implies that $\left\{y^m\right\}$ converges to g pointwise almost everywhere on T. Let $y^m\left(t\right)\to g\left(t\right)$ for all $t\in T-T_0$ where $\lambda \left(T_0\right)=0$. Recall that $T\equiv S\times X\times S$. Then, for each $t\in T-T_0$, for each m, if $y^m(s^{-},a^{-},s)\in E_{v^m}^{(s^{-},a^{-})}\left(s\right)$, then $g\left(s\right)\in E_{v^m}^{(s^{-},a^{-})}\left(s\right)$; since $d_i^m{\left(s_i\right)}^{(s^{-},a^{-})}\to 0$, $E_{v^m}^{(s^{-},a^{-})}\left(s\right)\to E_v^{(s^{-},a^{-})}\left(s\right)$; thus, $g(s^{-},a^{-},s)\in E_v^{(s^{-},a^{-})}\left(s\right)$ for each $(s^{-},a^{-},s)=t\in T-T_0$ and clearly $g\in M_v$. Hence, $v\mapsto M_v$ is upper-hemicontinuous. In addition, I observe that for each m, $M_{v^m}$ is closed and $M_v$ is closed. By the closed graph theorem for correspondences (Theorem 17.11, AB, p. 561), $v\mapsto M_v$ is closed graph. □

Proof of Existence Theorem.

Obviously, $M_v\subset V$. By the Kakutani–Fan–Glicksberg theorem (see Theorem 17.55, AB, p. 583), I have a fixed point of $v\mapsto M_v$. Then, I have $w:S\times X\times S\to {\mathfrak{R}}^n$ such that given $(s^{-},a^{-})$, $w(s^{-},a^{-},s)\in E_v^{(s^{-},a^{-})}\left(s\right)$ for all $s\in S-S_0$, $\varphi \left(S_0\right)=0$. Recall that ${\widehat{U}}_i^{(s^{-},a^{-})}(s_i,\alpha ;v)$ is measurable in $(s^{-},a^{-},s_i)$, continuous in $\alpha$ (Lemma 1). Now, by Filippov’s implicit function theorem (see Theorem 18.17, AB, p. 603), there exists a measurable mapping $f:S\times X\times S\to \Delta \left(X_1\right)\times \cdots \times \Delta \left(X_n\right)$ such that, given $(s^{-},a^{-})$, for each s, $f(s^{-},a^{-},s)\in B_v^{(s^{-},a^{-})}\left(s\right)$ and for all i, each $s_i\in S_i-pro{j}_i\left(S_0\right)$,

$$\begin{array}{c}\hfill w_i(s^{-},a^{-},s_i)={\int }_{s_{-i}}\left[\begin{array}{c}(1-{\delta }_i)u_i(s,f(s^{-},a^{-},s))\hfill \\ +{\delta }_i{\int }_{s_i^{+}}{w}_i(s,a,s_i^{+})\mu (d{s}_i^{+}\mid s,f(s^{-},a^{-},s))\hfill \end{array}\right]\eta (d{s}_{-i}\mid s^{-},a^{-},s_i).\end{array}$$

(26)

For each i, $(s^{-},a^{-},s)\in S\times X\times (S-S_0)$, $(s^{-},a^{-},s)\mapsto B_v^{(s^{-},a^{-})}\left(s\right)$ admits a measurable selector by the Kuratowski–Ryll-Nardzewski measurable selection theorem (Theorem 18.13, AB, p. 600). Let $f_0$ be any measurable selector of $(s^{-},a^{-},s)\mapsto B_v^{(s^{-},a^{-})}\left(s\right)$. Given $(s^{-},a^{-})$, put

$$\begin{array}{c}\hfill f^{*}(s^{-},a^{-},s)=\left\{\begin{array}{cc}{f}_0(s^{-},a^{-},s)\hfill & \mathrm{if}s\in S_0\hfill \\ f(s^{-},a^{-},s)\hfill & \mathrm{if}s\in S-S_0\hfill \end{array}\right..\end{array}$$

(27)

Then, $f^{*}(s^{-},a^{-},s)\in B_v^{(s^{-},a^{-})}\left(s\right)$ for all s, and

$$\begin{array}{c}\hfill w_i^{*}(s^{-},a^{-},s_i)={\int }_{s_{-i}}\left[\begin{array}{c}(1-{\delta }_i)u_i(s,f^{*}(s^{-},a^{-},s))\hfill \\ +{\delta }_i{\int }_{s_i^{+}}{w}_i^{*}(s,a,s_i^{+})\mu (d{s}_i^{+}\mid s,f^{*}(s^{-},a^{-},s))\hfill \end{array}\right]\eta (d{s}_{-i}\mid s^{-},a^{-},s_i).\end{array}$$

(28)

Therefore, f is the stationary Bayesian–Markov equilibrium strategy profile. □

4. Computational Algorithm

The flow of the proof in Section 3 can be used as a computational algorithm to find an equilibrium. The algorithm is especially useful for the environment of heterogeneous beliefs.24 First, find a fixed point in a set of interim expected continuous value functions, and second, extract an equilibrium strategy that generates the fixed point value function. In this sense, the computational algorithm for the dynamic Bayesian games in a continuous type space with periodic revelation is close to the value function iteration in macroeconometrics.25 The difference between the popular macroeconometrics technique and the following algorithm is, however, that there are multiple agents who have heterogeneous beliefs about the others’ types in dynamic Bayesian games. That is, the highlight of the computational algorithm in this section is to find a fixed point for a array of interim expected continuous value functions given the heterogeneous beliefs.

Start the algorithm by approximating interim expected continuous value function $v_i(·)$ for player i by a polynomial, preferably a Chebyshev orthogonal polynomial.26 Accordingly, the type space and action space are given by grids of Chebyshev nodes. In addition, build the beliefs and transition probability distribution over different sets of grids for numerical integration, for example, a type space with equidistant nodes $S_i=\{s_1,\cdots ,s_k,\cdots ,s_N\}$ and an action space with equidistant nodes, $X_i=\{a_1,\cdots ,a_k,\cdots ,a_N\}$ for all i. Over the grids, construct the probability distribution over the collection of types for the current stage, $\tau (·\mid s^{-},a^{-})\in \Delta \left(S\right)$, given any previous type and action profiles, $(s^{-},a^{-})$. From this $\tau (·\mid s^{-},a^{-})$, beliefs for the other players $\eta (·\mid s^{-},a^{-},s_i)$ and the future type distribution $\mu (·\mid s,a)$ can be obtained.

Then, the main algorithm is given by the following backward iteration process: start with the jth guess of $v_i^j(·)$ and plug it into the right-hand side. After numerical integration using $\mu (·\mid s,a)$ and $\eta (·\mid s^{-},a^{-},s_i)$, the result from the right-hand side is fed to the loop as the $(j+1)$th guess of $v_i^{j+1}(·)$. Repeat the iteration until $v_i(·)$ converges. By the existence theorem, a fixed point for $v_i(·)$ exists, and, thus, for large enough j, $v_i^j(·)$ is guaranteed to be sufficiently close to $v_i^{j+1}(·)$.

$$\begin{array}{c}{v}_i^{j+1}(s^{-},a^{-},s_i)\hfill \\ =\underset{f\in \Delta \left(X\right)}{max}{\int }_{s_{-i}}\left[\begin{array}{c}(1-{\delta }_i)u_i(s,f(s^{-},a^{-},s))\hfill \\ +{\delta }_i{\int }_{s_i^{+}}{v}_i^j(s,a,s_i^{+})\mu (d{s}_i^{+}\mid s,f(s^{-},a^{-},s))\hfill \end{array}\right]\eta (d{s}_{-i}\mid s^{-},a^{-},s_i).\hfill \end{array}$$

(29)

Define an operator $T(·)$ that implements Equation (29). That is, ${\overrightarrow{v_i}}^{j+1}=T\left({\overrightarrow{v_i}}^j\right)$ and, more precisely, it is an operation for coefficients of Chebyshev polynomials. As there are multiple players $i\in \{1,\dots ,n\}$, by stacking all the vectors of ${\overrightarrow{v_i}}^j$ for all i, in the jth iteration,

$$\left[\begin{array}{c}{\overrightarrow{v_1}}^{j+1}\\ {\overrightarrow{v_2}}^{j+1}\\ ⋮\\ {\overrightarrow{v_n}}^{j+1}\end{array}\right]=T\left(\left[\begin{array}{c}{\overrightarrow{v_1}}^j\\ {\overrightarrow{v_2}}^j\\ ⋮\\ {\overrightarrow{v_n}}^j\end{array}\right]\right).$$

(30)

The sequence of the vectors ${\left\{{\overrightarrow{v_i}}^j\right\}}_j$ for all i is guaranteed to converge as $T(·)$ is a contraction mapping. Denote the solution as ${\overrightarrow{v_i}}^{*}$. Then, the associated equilibrium stationary Bayesian strategy profile $f(·)$ in Equation (29) can be obtained by plugging in the solution ${\overrightarrow{v_i}}^{*}$ to the both sides.

5. Conclusions and Extension

In reality, economic agents rarely have a true understanding about the state of the economy. Often they have their own biased perceptions which reflect only part of the true state of the economy, and their perceptions may remain private. Moreover, such biased perceptions tend to be serially correlated. To reflect such aspects of reality more closely, dynamic games with asymmetric information can be a better modeling choice than ones with symmetric information. This paper constructs a class of dynamic Bayesian games where types evolve stochastically according to a first-order Markov process depending on the type and action profiles from the previous period. In this class of dynamic Bayesian games, however, players’ beliefs exponentially grow over time. To mitigate the curse of dimensionality in the dynamic Bayesian game structure, this paper considers the environment where the asymmetric information in the past becomes symmetric with a delay (periodic revelation). Specifically, this paper considers the case where the previous type profile is revealed as public information with a one-period delay. That is, type profile remains hidden when players choose actions, but type and action profiles are revealed when players obtain their payoffs. As common prior for the next stage Bayesian game is pinned down by the type and action profiles, players’ beliefs do not explode over time.

Theoretically, in the dynamic Bayesian game of this paper, the type space is a complete separable metric space (Polish space) and the action space is a compact metric space. A stationary Bayesian–Markov strategy is constructed as a measurable mapping that maps a tuple of the previous type profile, action profile, and player i’s current type, i.e., $(s^{-},a^{-},s_i)$, to a probability distribution over actions. Then, there exists a stationary Bayesian–Markov equilibrium. This stationary Bayesian–Markov equilibrium concept is related to the stationary correlated equilibrium concept in Nowak and Raghavan [11] through Aumann [14].

Similarly, the proof can be extended to a game with K-period lagged revelation, i.e., in each period t, players observe type and action profiles up to period $(t-K)$, whereas for periods from $(t-K+1)$ to t, they only observe the history of their own types, their own actions, and their own payoffs. Then, existence of a stationary Bayesian–Markov equilibrium with K-period lagged revelation can be proved once players’ heterogeneous beliefs $\eta (s_{-i}\mid ·)$ are adjusted judiciously: The unrevealed history from period $(t-K+1)$ to period t plays the role of individual player’s new “type”, and beliefs are assumed to be induced by a common prior conditional on the lagged information $(s_{-K},a_{-K})$. Figure 2 depicts the timeline. See the Supplementary Materials for the formal proof.