Dynamic Mechanism Design for Repeated Markov Games with Hidden Actions: Computational Approach

Abstract

This paper introduces a dynamic mechanism design tailored for uncertain environments where incentive schemes are challenged by the inability to observe players’ actions, known as moral hazard. In these scenarios, the system operates as a Markov game where outcomes depend on both the state of payouts and players’ actions. Moral hazard and adverse selection further complicate decision-making. The proposed mechanism aims to incentivize players to truthfully reveal their states while maximizing their expected payoffs. This is achieved through players’ best-reply strategies, ensuring truthful state revelation despite moral hazard. The revelation principle, a core concept in mechanism design, is applied to models with both moral hazard and adverse selection, facilitating optimal reward structure identification. The research holds significant practical implications, addressing the challenge of designing reward structures for multiplayer Markov games with hidden actions. By utilizing dynamic mechanism design, researchers and practitioners can optimize incentive schemes in complex, uncertain environments affected by moral hazard. To demonstrate the approach, the paper includes a numerical example of solving an oligopoly problem. Oligopolies, with a few dominant market players, exhibit complex dynamics where individual actions impact market outcomes significantly. Using the dynamic mechanism design framework, the paper shows how to construct optimal reward structures that align players’ incentives with desirable market outcomes, mitigating moral hazard and adverse selection effects. This framework is crucial for optimizing incentive schemes in multiplayer Markov games, providing a robust approach to handling the intricacies of moral hazard and adverse selection. By leveraging this design, the research contributes to the literature by offering a method to construct effective reward structures even in complex and uncertain environments. The numerical example of oligopolies illustrates the practical application and effectiveness of this dynamic mechanism design.

1. Introduction

Brief Review

In recent decades, theoretical literature has increasingly emphasized the mathematical modeling and analysis of incentive schemes, also referred to as principal-player relationships [1], particularly in uncertain environments. In such contexts, parties involved face constraints arising from four main cases: (a) risk-sharing with symmetric information, (b) hidden type with asymmetric information, (c) hidden type with partially observable states involving asymmetric information, and (d) hidden action due to the inability to observe each player’s actions. The second and fourth constraints, known in economic theory as adverse selection and moral hazard, respectively, are of particular interest. The central challenge posed by these constraints revolves around designing mechanisms that effectively address adverse selection and moral hazard while maximizing rewards [2,3,4]. Adverse selection refers to situations where one party possesses information not available to others, potentially leading to suboptimal outcomes. Moral hazard, on the other hand, arises when one party’s actions are not observable, resulting in risks that are not fully accounted for in the incentive structure. Addressing these issues requires a sophisticated mechanism design that aligns the interests of all parties involved and mitigates the adverse effects of asymmetric information. By focusing on the design of mechanisms that incentivize truthful disclosure of information and discourage opportunistic behavior, researchers aim to develop strategies that maximize rewards while ensuring efficiency and fairness in economic interactions. This involves creating incentive schemes that align individual actions with desirable outcomes, effectively balancing the risks and asymmetries in information. The objective is to establish a framework where parties are motivated to act in ways that lead to mutually beneficial results, even in the presence of hidden information and actions.

The classical approach to hidden action, as outlined by Cvitanic [5], pertains to situations where a player’s actions remain unobservable to the principal. This lack of observability leads to a reduction in the principal’s expected profit, as the consequences of the player’s hidden actions may not align with the principal’s objectives. Real-world applications are abundant where principals cannot infer players’ actions due to the high cost of observation or inherent impossibility. Consider the insurance market, where the expectations placed on insurance companies are heavily influenced by the moral and cultural values of insured individuals. Moral hazard manifests when an insured individual exerts less effort in maintaining their health, leading to increased costs for the insurer. This happens because the insured party does not bear the full financial consequences of their actions, therefore incentivizing behavior that raises costs for the insurer, such as unnecessary medical services. Adverse selection, on the other hand, relates to inherent characteristics of the insured individual, such as genetic predispositions. In this scenario, individuals with higher genetic risks are more likely to seek insurance, creating a situation where the pool of insured individuals is disproportionately composed of higher-risk individuals. This leads to adverse outcomes for the insurer, as they may face higher-than-expected costs due to the concentration of high-risk individuals in their customer base. Overall, both moral hazard and adverse selection underscore the challenges faced by principals in designing effective incentive mechanisms and managing risk in environments where player actions are hidden or difficult to observe. Addressing these challenges requires innovative approaches that account for asymmetric information and align incentives to achieve desired outcomes. This involves creating mechanisms that not only incentivize desired behaviors but also manage the risks associated with unobservable actions and hidden information.

When actions are unobservable or non-contractable, the principal encounters difficulty in selecting the desired actions for the player. Instead, the principal must identify the optimal actions for the player and devise a contract that encourages desirable behavior. This process revolves around incentives, with the principal seeking to influence the player’s choices by crafting an appropriate contract. This situation epitomizes moral hazard, wherein players may act contrary to the principal’s interests. This underscores the significance of creating effective incentive mechanisms to mitigate such risks and ensure alignment between the player’s actions and the principal’s objectives.

In numerous instances documented in the literature, the principal lacks private information, while players possess private information and can take actions hidden from the principal. This scenario gives rise to both adverse selection and moral hazard. Conversely, when players lack private information, moral hazard arises with ex ante (risk-sharing with) symmetric uncertain information. The distinction between these scenarios is crucial for comprehending the dynamics of incentive structures. Furthermore, the state of the system is categorized as exogenous if players’ actions do not directly influence the payoff. Conversely, if players’ actions affect the payoff, the state is deemed endogenous. These categorizations, elucidated by Garrett et al. [6], Board et al. [7], and Halac et al. [8], offer a framework for analyzing the interplay between information asymmetry, strategic behavior, and the design of incentive mechanisms. Understanding these dynamics is pivotal for devising effective strategies to mitigate adverse effects such as moral hazard and adverse selection in various economic and organizational contexts.

Fudenberg et al. [9] pioneered a dynamic principal-player model centered on a stochastic process with hidden actions, advocating for the breakdown of optimal long-term contracts into easily computable short-term contracts. Plambeck and Zenios [10] extended this study by integrating the principal-player model with the physical structure of a Markov decision process, providing a dynamic model that captures the intricate interplay between hidden actions and stochastic processes. Cole and Kocherlakota [11] further advanced this field by proposing a solution for dynamic Markov games where each player’s actions remain unobservable to others, and these actions can influence an unobservable state variable. This enriched the understanding of strategic interactions in dynamic settings.

In the context of Markov games, Doepke and Townsend [12] developed recursive algorithms to solve optimal contracts in dynamic principal-player scenarios with concealed states and hidden actions, demonstrating the feasibility of employing recursive techniques to derive optimal contracts. Their work sheds light on effective strategies for managing information asymmetry and strategic behavior and contributes to the broader research on optimal contracting in the presence of moral hazard concerns.

Chen et al. [13] devised a framework addressing moral hazard concerns encompassing both observable and unobservable actions. Their paradigm offers insights into designing incentive mechanisms that account for the complexities arising from hidden actions, therefore enhancing our understanding of how to mitigate moral hazard in various economic contexts. These research contributions underscore the importance of dynamic modeling and algorithmic approaches in analyzing principal-player interactions with hidden actions. They provide valuable guidance for designing effective incentive mechanisms and managing moral hazard concerns in dynamic and uncertain environments.

The paper addresses real-world situations where players are unable to observe each other’s actions due to cost or impracticality. It offers insights into dynamic modeling, optimal contract design, and algorithmic solutions for managing strategic interactions in such environments. These findings hold practical implications for designing effective incentive mechanisms and decision-making strategies in settings where player actions are concealed.

This work introduces a dynamic mechanism design tailored for uncertain environments where parties encounter constraints due to the inability to observe players’ actions. It proposes incentive schemes to tackle these constraints within a multiplayer Markov game framework, where outcomes are influenced by both the payoff state and players’ actions. The involved parties are bound by either moral hazard or adverse selection, complicating decision-making. To maximize expected returns, the mechanism incentivizes truthful state revelation, aligning players’ incentives with desired outcomes. The paper establishes essential relationships to determine optimal variables, including the mechanism, methods, and distribution vector. By addressing the challenges posed by hidden actions and information asymmetry, the proposed framework offers valuable insights for designing effective incentive mechanisms in complex, uncertain environments, ultimately enhancing decision-making processes and strategic interactions.

A numerical example concerning oligopolies emphasizes the effectiveness of our proposed method. By applying our approach to a specific scenario, we illustrate its practical utility and efficacy in achieving desirable outcomes. Through numerical analysis and simulation, we offer concrete evidence of the method’s effectiveness in addressing real-world challenges. This example validates our approach, demonstrating its ability to deliver optimal solutions and inform its potential applications in various contexts.

The paper is structured into distinct sections to systematically address dynamic mechanism design in uncertain environments. In Section 2, the formulation of the moral hazard approach is explored, providing insights into theoretical foundations and presenting a model tailored to address this challenge. Section 3 shifts focus to mechanism design and the recovery of variables of interest, elaborating on strategies employed to mitigate the effects of moral hazard through effective incentive mechanisms. Convergence analysis is conducted in Section 4, rigorously examining the stability and convergence properties of the proposed approach. Section 5 utilizes a numerical example to illustrate the practical utility and efficacy of the proposed method, offering concrete evidence of its effectiveness in real-world scenarios. Finally, Section 6 concludes with remarks and outlines for future research, encapsulating key findings and suggesting potential areas for further exploration and refinement of the dynamic mechanism design framework.

2. Repeated Game and Moral Hazard

The main properties of the game, considered below, are as follows:

• The player l is informed privately of his type $\theta$ for each time $t\ge 1$ from the finite set ${\mathsf{\Theta }}^l$, and takes an action (make decisions) $a_t^l$ from the finite set $A^l$ [14,15];
• all players, being in states $\mathsf{\Theta }={\times }_{l\in \mathcal{N}}{\mathsf{\Theta }}^l,$$\mathcal{N}:=\{1,...,n\},$ take actions simultaneously from the set $A={\times }_{l\in \mathcal{N}}{A}^l$;
• each player has a valuation function $v^l(a_t^l,{\theta }_t^l)\ge 0$ (utility function) that establishes current utility value for the player $l\in \mathcal{N}$.
• the state dynamics of player l is defined by the transition distribution

  $$p^l\left({\theta }_{t+1}^l\right|{\theta }_t^l,a_t^l,{\theta }_{t-1}^l,a_{t-1}^l\dots ,{\theta }_0^l,a_0^l)=p^l\left({\theta }_{t+1}^l\right|{\theta }_t^l,a_t^l),$$

  (1)

  that is the probability to move from type ${\theta }_t^l$ to type ${\theta }_{t+1}^l$ applying action $a_t^l$.
• denoting by $\Delta \left({\mathsf{\Theta }}^l\right)$ the set of statedistribution${\mu }_t^l\left({\theta }_t^l\right)$ over ${\mathsf{\Theta }}^l$, namely ${\mu }_t^l\left({\theta }_t^l\right)\in \Delta \left({\mathsf{\Theta }}^l\right)$, let us suppose that the state dynamics of the players are mutually independent, each chain is ergodic, and ${\mu }^l$ is its unique invariant distribution, i.e.,

  $${\mu }_t^l\left({\theta }_t^l\right)\underset{t\to \infty }{\to }{\mu }^l\left({\theta }^l\right).$$

We associate each player’s message $m_t^l$ with its type, where $m_t^l\in M^l\subseteq {\mathsf{\Theta }}^l$, and denote the overall message as $m_t=(m_t^1,...,m_t^n)$. Here, $M^l\subseteq {\mathsf{\Theta }}^l$ represents the set of messages such that at each time t, player l sends a message $m_t^l$ to all other players, who respond with $a_t=(a_t^1,a_t^2,\dots ,a_t^n)$.

Remark 1. 

In partially observable Markov processes, the set of observable states is external, distinct from ${\mathsf{\Theta }}^l$. Conversely, in Bayes–Markov processes, the set of observable states is internal, corresponding to the subset $M^l$. This distinction highlights the difference in how information is perceived and processed by players in different types of Markov processes, impacting their decision-making and strategies within the game.

Reward Function

The utility function $v^l(a_t^l,{\theta }_t^l,m_t^l)$ captures the losses incurred by player l when applying action $a_t^l\in A^l$ based on the message $m_t^l\in M^l$, generated in the type (state) ${\theta }_t^l\in {\mathsf{\Theta }}^l$. This function quantifies the impact of player l’s actions on their overall performance, considering both the current state of the system and the message conveyed. By incorporating these factors, the utility function provides valuable insights into player l’s decision-making process and aids in optimizing their strategies within the game.

A mechanism $\phi$ in the above system is the conditional distribution $\phi \left(a\right|m)$ of possible joint actions $a\in A=\underset{l\in \mathcal{N}}{\otimes }{A}^l$ that respond to the joint message $m\in M=\underset{l\in \mathcal{N}}{\otimes }{M}^l$.

Denote by ${\mathsf{\Phi }}_{adm}$ the set of possible mechanisms:

$${\mathsf{\Phi }}_{adm}=\left\{\phi \left({\alpha }_t\right|m_t)\ge 0\mid \sum _{{\alpha }_t\in A}\phi \left({\alpha }_t\right|m_t)=1,m_t\in M\right\}.$$

(2)

A (behavioral) strategy ${\sigma }^l\left(m_t^l\right|{\theta }_t^l)$ for player l is given by ${\sigma }^l:{\mathsf{\Theta }}^l\to M^l$, which associates the given type ${\theta }_t^l$ with the message $m_t^l$.

The set of feasible policies is given by

$${\mathcal{S}}_{adm}^l=\left\{{\sigma }^l\left(m_t^l\right|{\theta }_t^l)\ge 0\mid \sum _{m_t^l\in {\mathsf{\Theta }}^l}{\sigma }^l\left(m_t^l\right|{\theta }_t^l)=1,{\theta }_t^l\in {\mathsf{\Theta }}^l\right\}.$$

(3)

and

$${\mathcal{S}}_{adm}={\mathcal{S}}_{adm}^1\times ...\times {\mathcal{S}}_{adm}^n.$$

The relationship ${\xi }^l\left(a_t^l\right|{\alpha }_t^l)$ for player l is given by ${\xi }^l:{\mathcal{A}}^l\to \Delta \left(A^l\right)$. The set of actions that are feasible is defined by

$$\begin{array}{c}{\mathsf{\Xi }}_{adm}^l=\left\{{\xi }^l\left(a_t^l\right|{\alpha }_t^l)\ge 0\mid {\displaystyle \sum _{a_t^l\in A_t^l}}{\xi }^l\left(a_t^l\right|{\alpha }_t^l)=1,{\alpha }_t^l\in A_t^l\right\}.\end{array}$$

(4)

Remark 2. 

${\xi }^l\left(a_t^l\right|{\alpha }_t^l)$ represents the likelihood with which a player l believes that ${\alpha }_t^l$ is an action $a_t^l$.

The average reward $U^l(\phi ,\sigma )$ (in the stationary ergodic regime) of the player l is the summed utility $W^l(a_t^l,{\theta }_t^l,m_t^l)$ given by employing the mechanism $\phi$, the strategy $\sigma$ and ${\xi }^l\left(a_t^l\right|{\alpha }_t^l)$:

$$\begin{array}{c}{U}^l(\phi ,\sigma ,\xi )={\displaystyle \sum _{{\theta }_t^l\in {\mathsf{\Theta }}^l}}{\displaystyle \sum _{m_t\in M}}{\displaystyle \sum _{a_t\in A}}{W}^l(a_t^l,{\theta }_t^l,m_t^l){\displaystyle \prod _{i\in \mathcal{N}}}\phi \left({\alpha }_t\right|m_t){\sigma }^i\left(m_t^i\right|{\theta }_t^i){\xi }^i\left(a_t^i\right|{\alpha }_t^i){\mu }^i\left({\theta }_t^i\right),\end{array}$$

(5)

where

$$W^l(a_t^l,{\theta }_t^l,m_t^l)=\sum _{{\theta }_{t+1}^l\in {\mathsf{\Theta }}^l}{v}^l(a_t^l,{\theta }_t^l,m_t^l)p^l\left({\theta }_{t+1}^l\right|{\theta }_t^l,a_t^l),$$

The following series of events occurs in time t and player $l\in \mathcal{N}$ given a mechanism $\phi$:

• Every player privately observes his current type ${\theta }_t^l$ obtained from ${\mu }^l\left({\theta }_t^l\right)$.
• Every player sends a message $m_t^l\in M_t^l$ according to ${\sigma }^l\left(m_t^l\right|{\theta }_t^l)$.
• An alternative ${\alpha }_t\in A_t$ is selected considering $\phi \left({\alpha }_t\right|m_t)\in {\mathsf{\Phi }}_{adm}$.
• Every player selects an $a_t\in A_t$ given ${\alpha }_t^l\in {\mathcal{A}}_t^l$ and ${\xi }^l\left(a_t^l\right|{\alpha }_t^l)$.
• The allocation is realized, and players obtain a reward.
• At last, ${\theta }_{t+1}^l$ is obtained from $p^l\left({\theta }_{t+1}^l\right|{\theta }_t^l{a}_t^l)$ given ${\theta }_t^l$ and $a_t^l$.

3. Mechanism and Equilibrium

Following [14,15] define the extended variable

$$c^l\left(m_t^l{\theta }_t^l{a}_t^l{\alpha }_t^l\right)=\phi \left({\alpha }_t\right|m_t){\sigma }^l\left(m_t^l\right|{\theta }_t^l){\xi }^l\left(a_t^l\right|{\alpha }_t^l){\mu }^l\left({\theta }_t^l\right)\in C_{adm}^l$$

(6)

with

$$\begin{array}{c}{C}_{adm}^l:=\left\{c^l\left(m_t^l{\theta }_t^l{a}_t^l{\alpha }_t^l\right)\ge 0\left|{\displaystyle \sum _{{\theta }_t^l\in {\mathsf{\Theta }}_t^l}\sum _{m_t^l\in {\mathsf{\Theta }}_t^l}\sum _{a_t^l\in A_t^l}\sum _{{\alpha }_t^l\in {\mathcal{A}}_t^l}}{c}^l\left(m_t^l{\theta }_t^l{a}_t^l{\alpha }_t^l\right)=1,\right.\right.\\ {\displaystyle \sum _{m_t^l\in {\mathsf{\Theta }}_t^l}\sum _{a_t^l\in A_t^l}\sum _{{\alpha }_t^l\in {\mathcal{A}}_t^l}}{c}^l\left(m_t^l{\theta }_t^l{a}_t^l{\alpha }_t^l\right)={\mu }^l\left({\theta }_t^l\right)>0,\\ \left.{\displaystyle \sum _{m_t^l\in {\mathsf{\Theta }}_t^l}\sum _{a_t^l\in A_t^l}\sum _{{\theta }_t^l\in {\mathsf{\Theta }}_t^l}\sum _{{\theta }_{t+1}^l\in {\mathsf{\Theta }}_{t+1}^l}\sum _{{\alpha }_t^l\in {\mathcal{A}}_t^l}}[{\delta }_{{\theta }_t^l{\theta }_{t+1}^l}-p^l\left({\theta }_{t+1}^l\right|{\theta }_t^l{a}_t^l)]c^l\left(m_t^l{\theta }_t^l{a}_t^l{\alpha }_t^l\right)=0\right\},\end{array}$$

(7)

where ${\delta }_{\tilde{\theta },{\theta }^l}$ is Kroneker’s symbol, ${\theta }_{t+1}^l\in {\mathsf{\Theta }}_t^l$ and $c\left({\theta }_t^l{m}_t^l{a}_t^i{\alpha }_t^l\right)=$$(c^1\left({\theta }_t^1{m}_t^1{a}_t^1{\alpha }_t^1\right)$$,\dots ,$$c^n\left({\theta }_t^n{m}_t^n{a}_t^n{\alpha }_t^n\right))$. Notice that

$$\begin{array}{ccc}\sum _{{\alpha }_t^l\in A_t}\phi \left({\alpha }_t\right|m_t)=1,& \sum _{m_t^l\in {\mathsf{\Theta }}_t}{\sigma }^l\left(m_t^l\right|{\theta }_t^l)=1,\sum _{{\theta }_t^l\in {\mathsf{\Theta }}_t^l}{\mu }^l\left({\theta }_t^l\right)=1,& \sum _{a_t^l\in {\mathcal{A}}_t^l}{\xi }^l\left(a_t^l\right|{\alpha }_t^l)=1\end{array}$$

such that $c^l\in S^l$ satisfies

$$\begin{array}{c}{S}^l:=\left\{c^l\left(m_t^l{\theta }_t^l{a}_t^l{\alpha }_t^l\right)\ge 0\left|{\displaystyle \sum _{{\theta }_t^l\in {\mathsf{\Theta }}_t^l}\sum _{m_t^l\in {\mathsf{\Theta }}_t^l}\sum _{a_t^l\in A_t^l}\sum _{{\alpha }_t^l\in {\mathcal{A}}_t^l}}{c}^l\left(m_t^l{\theta }_t^l{a}_t^l{\alpha }_t^l\right)=1,\right.\right.\\ \left.{\displaystyle \sum _{m_t^l\in {\mathsf{\Theta }}_t^l}\sum _{a_t^l\in A_t^l}\sum _{{\alpha }_t^l\in {\mathcal{A}}_t^l}}{c}^l\left(m_t^l{\theta }_t^l{a}_t^l{\alpha }_t^l\right)={\mu }^l\left({\theta }_t^l\right)>0\right\}.\end{array}$$

(8)

The individual aim (5) in c-variables looks as

$$\begin{array}{c}{U}^l(\phi ,\sigma ,\xi )={\displaystyle \sum _{l\in \mathcal{N}}}{\overline{U}}^l\left(c\right)\to {\displaystyle \underset{c\in C_{adm}}{max}}\end{array}$$

(9)

$$\begin{array}{c}{\overline{U}}^l\left(c\right)={\displaystyle \sum _{{\theta }_t^l\in {\mathsf{\Theta }}_t^l}\sum _{m_t^l\in {\mathsf{\Theta }}_t^l}\sum _{a_t^l\in A_t^l}\sum _{{\alpha }_t^l\in {\mathcal{A}}_t^l}}{W}^l(a_t^l,{\theta }_t^l,m_t^l){\displaystyle \prod _{i\in \mathcal{N}}}{c}^i\left(m_t^i{\theta }_t^i{a}_t^i{\alpha }_t^i\right)\end{array}$$

(10)

where $c^l\left(m_t^l{\theta }_t^l{a}_t^l{\alpha }_t^l\right)$ is given by (6) and ${\mathcal{C}}_{adm}={\times }_l{C}_{adm}^l$. The variable $c^{l*}$ denotes the solution of the problem given in Equation (9).

Find a mechanism $\phi \left({\alpha }_t\right|m_t)$, the Bayesian strategies ${\sigma }^{*}$ and the action kernel ${\xi }^{*}$, that realizes the rule given by and solves the following nonlinear programming problem

$$({\phi }^{*},{\sigma }^{*},{\xi }^{*})=arg\underset{\phi \in {\mathsf{\Phi }}_{adm},}{max}\sum _{l\in \mathcal{N}}{U}^l(\phi ,{\sigma }^{*},{\xi }^{*}).$$

(11)

where for a given mechanism $\phi \left(a_t\right|m_t)$ the strategies ${\sigma }_{\phi }^{*}$ and ${\xi }_{\phi }^{*}$ fulfills the Bayesian–Nash equilibrium satisfying

$$U^l(\phi ,{\sigma }_{\phi }^{*},{\xi }_{\phi }^{*})\ge U^l(\phi ,{\sigma }^l,{\sigma }^{-l*},{\xi }^l,{\xi }^{-l*}).$$

(12)

such that $\phi$ is unique and ${\sigma }^{*}=\left({\sigma }^{1*},...,{\sigma }^{n*}\right)$ is known as the Bayesian–Nash equilibrium where ${\sigma }^{-l*}=\left({\sigma }^{1*},...,{\sigma }^{l-1*},{\sigma }^{l+1,*},...,{\sigma }^{n*}\right)$ and ${\xi }^{-l*}=\left({\xi }^{1*},...,{\xi }^{l-1*},{\xi }^{l+1,*},...,{\xi }^{n*}\right)$.

The following Lemma explains how we can compute the mechanism ${\phi }^{*}\left({\alpha }_t\right|m_t)$ given $c^{l*}\left({\theta }_t^l{m}_t^l{a}_t^l{\alpha }_t^l\right)$,

Let us assume that (9) is solved, then the variable ${\phi }^{*}\left({\alpha }_t\right|m_t)$ can be recovered from $c^{l*}\left({\theta }_t^l{m}_t^l{a}_t^l{\alpha }_t^l\right)$ as follows:

$${\phi }^{*}\left({\alpha }_t\right|m_t)={\displaystyle \frac{{\displaystyle \sum _{l\in \mathcal{N}}\sum _{{\theta }_t^l\in {\mathsf{\Theta }}_t^l}\sum _{a_t^l\in A_t^l}}{c}^{l*}\left({\theta }_t^l{m}_t^l{a}_t^l{\alpha }_t^l\right)}{\sum _{l\in \mathcal{N}}\sum _{{\theta }_t^l\in {\mathsf{\Theta }}_t^l}\sum _{a_t^l\in A_t^l}\sum _{{\gamma }_t^l\in {\mathcal{A}}_t^l}{c}^{l*}\left({\theta }_t^l{m}_t^l{a}_t^l{\gamma }_t^l\right)}}$$

(13)

The variables can be retrieved using the formulas shown below.

To recover ${\sigma }^{l*}\left(m_t^l\right|{\theta }_t^l)$ note that

$${\sigma }^{l*}\left(m_t^l\right|{\theta }_t^l)={\displaystyle \frac{{\displaystyle \sum _{a_t^l\in A_t^l}\sum _{{\alpha }_t^l\in {\mathcal{A}}_t^l}}{c}^{l*}\left({\theta }_t^l{m}_t^l{a}_t^l{\alpha }_t^l\right)}{\sum _{{\varrho }_t^l\in {\mathsf{\Theta }}_t^l}\sum _{a_t^l\in A_t^l}\sum _{{\alpha }_t^l\in {\mathcal{A}}_t^l}{c}^{l*}\left({\theta }_t^l{\varrho }_t^l{a}_t^l{\alpha }_t^l\right)}}.$$

(14)

For the distribution ${\overline{\mu }}^{l*}\left(m_t^l\right)$ the resulting relation is given by

$${\overline{\mu }}^{l*}\left(m_t^l\right)={\displaystyle \sum _{{\theta }_t^l\in {\mathsf{\Theta }}_t^l}\sum _{a_t^l\in A_t^l}\sum _{{\alpha }_t^l\in {\mathcal{A}}_t^l}}{c}^{l*}\left({\theta }_t^l{m}_t^l{a}_t^l{\alpha }_t^l\right),l\in \mathcal{N}$$

(15)

The actions observer ${\xi }_{\alpha |a}^{*}$ can be recovered employing

$${\xi }^{l*}\left(a_t^l\right|{\alpha }_t^l)={\displaystyle \frac{{\displaystyle \sum _{{\theta }_t^l\in {\mathsf{\Theta }}_t^l}\sum _{m_t^l\in {\mathsf{\Theta }}_t^l}}{c}^{l*}\left({\theta }_t^l{m}_t^l{a}_t^l{\alpha }_t^l\right)}{\sum _{{\theta }_t^l\in {\mathsf{\Theta }}_t^l}\sum _{m_t^l\in {\mathsf{\Theta }}_t^l}\sum _{{\gamma }_t^l\in A_t^l}{c}^{l*}\left({\theta }_t^l{m}_t^l{a}_t^l{\gamma }_t^l\right)}}.$$

(16)

If the solutions of the problem (9) given by ${\sigma }^{l*}\left(m_t^l\right|{\theta }_t^l)$, ${\phi }^{*}\left({\alpha }_t\right|m_t)$ and ${\xi }^{l*}\left({\alpha }_t^l\right|a_t^l)$) comply with the Bayesian–Nash equilibrium presented in Equation (12) for all $l\in \mathcal{N}$, then variables $c^{l*}\left({\theta }_t^l{m}_t^l{a}_t^l{\alpha }_t^l\right)$ fulfill the ergodicity constraints given by:

$$\begin{array}{c}{\displaystyle \sum _{{\theta }_t^l\in {\mathsf{\Theta }}_t^l}\sum _{m_t^l\in {\mathsf{\Theta }}_t^l}\sum _{a_t^l\in A_t^l}\sum _{{\alpha }_t^l\in {\mathcal{A}}_t^l}}[{\delta }_{{\theta }_t^l{\theta }_{t+1}^l}-p^l\left({\theta }_{t+1}^l\right|{\theta }_t^l{a}_t^l)]c^{l*}\left({\theta }_t^l{m}_t^l{a}_t^l{\alpha }_t^l\right),{\theta }_{t+1}^l\in {\mathsf{\Theta }}_t^l,\end{array}$$

(17)

Lemma 1. 

The mechanism ${\phi }^{*}\left({\alpha }_t\right|m_t)$, the actions kernel ${\xi }^{l*}\left(a_t^l\right|{\alpha }_t^l)$ and the strategies ${\sigma }^{l*}\left(m_t^l\right|{\theta }_t^l)$ coincide with the Bayesian–Nash equilibrium presented in Equation (12).

Proof. 

It results straightforward from:

$$\begin{array}{c}{\displaystyle \underset{c\in C_{adm}}{max}}\overline{U}\left(c\right)=\overline{U}\left(c^{*}\right)={\displaystyle \sum _{l\in \mathcal{N}}}{\overline{U}}^l\left(c^{*}\right)={\displaystyle \sum _{l\in \mathcal{N}}}{U}^l({\phi }^{*},{\sigma }_{{\phi }^{*}}^{*},{\xi }_{{\phi }^{*}}^{*})=\\ {\displaystyle \sum _{l\in \mathcal{N}}}\left({\displaystyle \sum _{{\theta }_t^l\in {\mathsf{\Theta }}_t^l}\sum _{m_t^l\in {\mathsf{\Theta }}_t^l}\sum _{a_t^l\in A_t^l}\sum _{{\alpha }_t^l\in {\mathcal{A}}_t^l}}{W}^l(a_t^l,{\theta }_t^l,m_t^l){\left({\phi }^{*}\left({\alpha }_t\right|m_t)\right)}^n{\sigma }^{l*}\left(m_t^{l*}\right|{\theta }_t^{l*})\right.\\ \left.{\xi }^{l*}\left(a_t^{l*}\right|{\alpha }_t^{l*}){\mu }^{l*}\left({\theta }_t^{l*}\right){\displaystyle \prod _{\iota \ne l\in \mathcal{N}}}{\sigma }^{\iota *}\left(m_t^{\iota *}\right|{\theta }_t^{\iota *}){\xi }^{\iota *}\left({\alpha }_t^{\iota *}\right|a_t^{\iota *}){\mu }^{\iota *}\left({\theta }_t^{\iota *}\right)\right)=\\ {\displaystyle \sum _{l\in \mathcal{N}}}\left({\displaystyle \underset{{\sigma }^l\in {\mathcal{S}}_{adm}^l,{\xi }^l\in {\mathsf{\Xi }}_{adm}^l}{max}\sum _{m_t^l\in {\mathsf{\Theta }}_t^l}\sum _{{\alpha }_t^l\in A_t^l}}{\left({\phi }^{*}\left({\alpha }_t\right|m_t)\right)}^n{\displaystyle \sum _{{\theta }_t^l\in {\mathsf{\Theta }}_t^l}\sum _{a_t^l\in {\mathcal{A}}_t^l}}{W}^l(a_t^l,{\theta }_t^l,m_t^l){\sigma }^l\left(m_t^l\right|{\theta }_t^l)\right.\\ \left.{\xi }^l\left({\alpha }_t^l\right|a_t^l){\mu }^l\left({\theta }_t^l\right){\displaystyle \prod _{\iota \ne l\in \mathcal{N}}}{\sigma }^{\iota *}\left(m_t^{\iota *}\right|{\theta }_t^{\iota *}){\xi }^{\iota *}\left({\alpha }_t^{\iota *}\right|a_t^{\iota *}){\mu }^{\iota *}\left({\theta }_t^{\iota *}\right)\right)\\ \ge {\displaystyle \sum _{l\in \mathcal{N}}}\left({\displaystyle \sum _{m_t^l\in {\mathsf{\Theta }}_t^l}\sum _{{\alpha }_t^l\in A_t^l}}{\left({\phi }^{*}\left({\alpha }_t\right|m_t)\right)}^n{\displaystyle \sum _{{\theta }_t^l\in {\mathsf{\Theta }}_t^l}\sum _{a_t^l\in {\mathcal{A}}_t^l}}{W}^l(a_t^l,{\theta }_t^l,m_t^l){\sigma }^l\left(m_t^l\right|{\theta }_t^l)\right.\\ \left.{\xi }^l\left({\alpha }_t^l\right|a_t^l){\mu }^l\left({\theta }_t^l\right){\displaystyle \prod _{\iota \ne l\in \mathcal{N}}}{\sigma }^{\iota *}\left(m_t^{\iota *}\right|{\theta }_t^{\iota *}){\xi }^{\iota *}\left({\alpha }_t^{\iota *}\right|a_t^{\iota *}){\mu }^{\iota *}\left({\theta }_t^{\iota *}\right)\right)=\\ {\displaystyle \sum _{l\in \mathcal{N}}}{U}^l(\phi ,{\sigma }^l,{\sigma }^{-l*},{\xi }^l,{\xi }^{-l*})\end{array}$$

As a result,

$$\begin{array}{c}{\displaystyle \sum _{l\in \mathcal{N}}}\left(U^l({\phi }^{*},{\sigma }_{\phi *}^{*},{\xi }_{\phi *}^{*})-U^l({\phi }^{*},{\sigma }^l,{\sigma }^{-l*},{\xi }^l,{\xi }^{-l*})\right)\ge 0.\end{array}$$

(18)

Given that Equation (18) is valid for all the feasible strategies $\sigma$ and $\xi$, it is satisfied for ${\sigma }^j={\sigma }^{j*}$ and ${\xi }^j={\xi }^{j*}$ for $j\ne l$, with

$$\begin{array}{c}{U}^l({\phi }^{*},{\sigma }_{\phi *}^{*},{\xi }_{\phi *}^{*})-U^l({\phi }^{*},{\sigma }^l,{\sigma }^{-l*},{\xi }^l,{\xi }^{-l*})\ge 0,\end{array}$$

(19)

that satisfies Equation (12) for $\phi ={\phi }^{*}$. □

4. Convergence Analysis

In our study, we illustrate the convergence of our proposed strategy to a Bayesian–Nash equilibrium, as documented in previous works [16,17,18]. To ensure this convergence, we incorporate a regularization parameter into our strategy. This parameter acts as a stabilizing factor, guiding the system towards a single equilibrium point. Through empirical analysis and simulations, we validate the effectiveness of our approach in achieving convergence to the desired equilibrium, highlighting the importance of the regularization parameter in enhancing stability and optimality in strategic interactions.

The game comprises a set $\mathcal{N}=1,...,n$ of players indexed by $l\in \mathcal{N}$. each player l’s joint strategy variable is denoted as

$$c^l\left(m_t^l{\theta }_t^l{a}_t^l{\alpha }_t^l\right)=\phi \left({\alpha }_t\right|m_t){\sigma }^l\left(m_t^l\right|{\theta }_t^l){\xi }^l\left(a_t^l\right|{\alpha }_t^l){\mu }^l\left({\theta }_t^l\right).$$

This variable encapsulates the interplay of various components, including message, type, action, and belief functions, shaping player l’s strategic decisions within the game.

In the game, variables $x^l\in {\mathcal{X}}^l$ are considered, where ${\mathcal{X}}^l$ represents a convex and compact set. Here, $x^l:=\mathrm{col}\left(c^l\left(m_t^l\theta t^{l}a{t}^l{\alpha }_t^l\right)\right)$, where $\mathrm{col}$ denotes the column operator. This formulation aggregates player l’s joint strategy variable into a column vector, facilitating mathematical representation and analysis of the game dynamics. Let $x={(x^1,...,x^n)}^{\top }\in \mathcal{X}$ be the joint strategy of the players and let $x^{-l}:={\left(x^1,...,x^{l-1},x^{l+1},...,x^n\right)}^{\top }\in {\mathcal{X}}^{-l}$ be the strategy of the complement of the players. Let us consider a Nash equilibrium problem with n players and denote by $x=(x^l,x^{-l})\in {\mathbb{R}}^n$ the vector representing the x-th player’s strategy where $x\in {\mathcal{X}}_{adm}$ and ${\mathcal{X}}_{adm}={\mathcal{X}}_{adm}^l\times {\mathcal{X}}_{adm}^{-l}$ is a bounded set given by

$$\begin{array}{c}{\mathcal{X}}_{adm}:=\left\{x\in {\mathbb{R}}^n:x\ge 0,A_{eq}x=b_{eq}\in {\mathbb{R}}^{m_0},A_{ineq}x\le b_{ineq}\in {\mathbb{R}}^{m_1}\right\}.\end{array}$$

Let $g^l:{\mathbb{R}}^n\to \mathbb{R}$ represent the l-th player’s reward function (or cost function). Players aim to reach a non-cooperative equilibrium by finding a strategy $x^{*}=\left(x^{1*},...,x^{n*}\right)$ where for any $x^l$ and any $l\in \mathcal{N}$:

$$\tilde{U}\left(x\right):=\sum _{l\in \mathcal{N}}\left[\left(\underset{x^l\in {\mathcal{X}}^l}{max}{g}^l\left(x^l,x^{-l}\right)\right)-g^l\left(x^l,x^{-l}\right)\right].$$

(20)

Let

$$\tilde{U}\left(x\right):=\sum _{l\in \mathcal{N}}\left[g^l\left({\overline{x}}^l,x^{-l}\right)-g^l\left(x^l,x^{-l}\right)\right],$$

(21)

and

$${\overline{x}}^l\in \underset{x^l\in {\mathcal{X}}^l}{Argmax}{g}^l\left(x^l,x^{-l}\right)$$

(22)

where

$$g^l\left({\overline{x}}^l,x^{-l}\right)-g^l\left(x^l,x^{-l}\right)\ge 0$$

(23)

for any $x^l\in {\mathcal{X}}^l$ and $l\in \mathcal{N}$. The vector $x^{*}\in \mathcal{X}$ is a Nash equilibrium if

$$x^{*}\in Arg\underset{x\in \mathcal{X}}{max}\left\{\tilde{U}\left(x\right)\right\}.$$

(24)

Lagrange Regularization Approach

In our analysis, we denote by ${\mathcal{X}}^{*}\subseteq {\mathcal{X}}_{adm}$ the set encompassing all potential solutions of the game [19,20]. To ensure the attainment of a unique solution within this set, we employ the Lagrange function. This mathematical tool enables the incorporation of constraints into the optimization framework by integrating the objective function with penalty terms for constraint violations. By doing so, the Lagrange function facilitates the identification of an optimal solution that simultaneously satisfies both the objective and constraints. Through the careful utilization of the Lagrange function, we navigate the complexities inherent in the game and ensure the achievement of a unique solution within the designated solution space. This approach allows us to address the intricacies of the problem and obtain a solution that optimally balances the competing objectives and constraints involved. To reach a unique solution, we consider the Lagrange function given by

$$\begin{array}{c}\mathbb{L}\left(x,{\lambda }_{eq},{\lambda }_{ineq}\right):=-\omega \tilde{U}\left(x\right)-{\lambda }_{eq}^{⊺}\left(A_{eq}x-b_{eq}\right)-{\lambda }_{ineq}^{⊺}\left(A_{ineq}x-b_{ineq}\right)\\ -{\displaystyle \frac{\delta }{2}}\left({∥x∥}^2-{∥{\lambda }_{eq}^{⊺}∥}^2-{∥{\lambda }_{ineq}^{⊺}∥}^2\right)\end{array}$$

(25)

where the parameters $\omega$ and $\delta$ are positive, the Lagrange vector-multipliers ${\lambda }_{ineq}\in {\mathbb{R}}^{m_1}$ are non-negative and the components of ${\lambda }_{eq}\in {\mathbb{R}}^{m_0}$ may have any sign. As a result, the nonlinear problem

$${\mathbb{L}}_{\omega ,\delta }\left(x,{\lambda }_{eq},{\lambda }_{ineq}\right)\to \underset{x\in {\mathcal{X}}_{adm}}{min}\underset{{\lambda }_{eq},{\lambda }_{ineq\ge 0}}{max}$$

(26)

has a unique saddle point on z since the optimized Equation () is strongly convex if the parameters $\omega$ and $\delta >0$ provide the condition

$$\frac{{\partial }^2}{\partial x\partial x^{⊺}}{\mathbb{L}}_{\omega ,\delta }\left(x,{\lambda }_{eq},{\lambda }_{ineq}\right)>0\forall x\in {\mathcal{X}}_{adm}\subset {\mathbb{R}}^n$$

(27)

and it is strongly concave on the Lagrange multipliers ${\lambda }_{eq},{\lambda }_{ineq}$ for any $\delta >0$. Given the properties of Equation (25) it has a unique saddle point

$$\left(x^{*}\left(\delta \right),{\lambda }_{eq}^{*}\left(\omega ,\delta \right),{\lambda }_{ineq}^{*}\left(\omega ,\delta \right)\right)$$

(28)

for which the following inequalities hold:

$$\begin{array}{c}{\mathbb{L}}_{\omega ,\delta }\left(x\left(\delta \right),{\lambda }_{eq}^{*}\left(\omega ,\delta \right),{\lambda }_{ineq}^{*}\left(\omega ,\delta \right)\right)\ge {\mathbb{L}}_{\omega ,\delta }\left(x^{*}\left(\delta \right),{\lambda }_{eq}^{*}\left(\omega ,\delta \right),{\lambda }_{ineq}^{*}\left(\omega ,\delta \right)\right)\ge \\ {\mathbb{L}}_{\omega ,\delta }\left(x^{*}\left(\delta \right),{\lambda }_{eq}\left(\omega ,\delta \right),{\lambda }_{ineq}\left(\omega ,\delta \right)\right)\end{array}$$

(29)

for any ${\lambda }_{eq},{\lambda }_{ineq}$ with non-negative components and any $x\in {\mathbb{R}}^n$.

Theorem 1. 

The nonlinear programming problem

$${\mathbb{L}}_{\omega ,\delta }\left(x,{\lambda }_{eq},{\lambda }_{ineq}\right)\to \underset{x\in {\mathcal{X}}_{adm}}{min}\underset{{\lambda }_{eq},{\lambda }_{ineq\ge 0}}{max}$$

has a unique equilibrium point at x.

Remark 3. 

This theorem asserts that the given nonlinear programming problem possesses a single equilibrium point located at x, indicating a stable solution within the optimization framework.

Proof. 

Considering that the functional is minimizing, we prove that the Hessian matrix $\mathcal{H}:={\displaystyle \frac{{\partial }^2}{\partial x\partial x^{⊺}}}{\mathbb{L}}_{\omega ,\delta }\left(x,{\lambda }_{eq},{\lambda }_{ineq}\right)$ is strictly positive definite, $\mathcal{H}>0$, for all $x\in {\mathbb{R}}^n$ and for some $\omega$,$\delta >0.$ Developing $\mathcal{H}$ one has that

$$\begin{array}{c}{\displaystyle \frac{{\partial }^2}{\partial x^2}}{\mathbb{L}}_{\omega ,\delta }\left(x,{\lambda }_{eq},{\lambda }_{ineq}\right)=\omega {\displaystyle \frac{{\partial }^2}{\partial x^2}}U\left(x\right)+\delta I_{n\times n}\ge \delta \left(1+{\displaystyle \frac{\omega }{\delta }}\zeta \right)I_{n\times n}>0\forall \delta >\omega \left|\zeta \right|\\ \zeta :={\displaystyle \underset{x\in {\mathcal{X}}_{adm}}{min}}{\zeta }_{min}\left({\displaystyle \frac{{\partial }^2}{\partial x^2}}U\left(x\right)\right)\end{array}$$

where $\zeta$ is an eigenvalue that fulfills the condition $\mathcal{H}>0$ if $\delta >\omega \left|\zeta \right|$. Then, the Lagrange in Equation (25) has a unique minimal point $x^{*}$. □

Theorem 2. 

Under the assumption that the bounded set ${\mathcal{X}}^{*}$ of all solutions of the game is not empty and there exists an equilibrium point $\tilde{x}\in {\mathcal{X}}_{adm}$ such that

$$A_{ineq}\tilde{x}<b_{ineq}$$

(30)

that is, Slater’s condition holds; furthermore, that the parameters ω and δ are time-varying, i.e.,

$$\omega ={\omega }_i,\delta ={\delta }_i\left(i=0,1,2,....\right)$$

such that

$$0<{\omega }_i\downarrow 0,\frac{{\omega }_i}{{\delta }_i}\downarrow 0\mathrm{when}i\to \infty ,$$

(31)

one obtains

$$\left.\begin{array}{c}{x}_i^{*}:=x^{*}\left({\omega }_i,{\delta }_i\right)\underset{i\to \infty }{\to }{x}^{**}\\ {\lambda }_{eq}^{*}\left({\omega }_i,{\delta }_i\right)\underset{i\to \infty }{\to }{\lambda }_{eq}^{**}\\ {\lambda }_{ineq}^{*}\left({\omega }_i,{\delta }_i\right)\underset{i\to \infty }{\to }{\lambda }_{ineq}^{**}\end{array}\right\}$$

(32)

where $x^{**}\in {\mathcal{X}}^{*},$$\left({\lambda }_{eq}^{**},{\lambda }_{ineq}^{**}\right)\in {\mathsf{\Xi }}^{*}$ define the solution of the game with the minimal norm given by

$$\left.\begin{array}{c}{∥x^{**}∥}^2+{∥{\lambda }_{eq}^{**}∥}^2+{∥{\lambda }_{ineq}^{**}∥}^2\le {∥x^{*}∥}^2+{∥{\lambda }_{eq}^{*}∥}^2+{∥{\lambda }_{ineq}^{*}∥}^2\\ \\ \mathrm{for}\mathrm{all}{x}^{*}\in {\mathcal{X}}^{*},\left({\lambda }_{eq}^{*},{\lambda }_{ineq}^{*}\right)\in {\mathsf{\Xi }}^{*}\end{array}\right\}$$

(33)

Remark 4. 

Theorem 2 presents conditions under which a unique solution of the game can be attained. First, we assume that the bounded set ${\mathcal{X}}^{*}$ of all solutions of the game is not empty and there exists an equilibrium point $\tilde{x}\in {\mathcal{X}}_{adm}$ such that Slater’s condition (30) holds. This condition ensures the existence of a feasible point in the interior of the inequality constraints. Additionally, the parameters ω and δ are considered to be time-varying, satisfying (31) as i tends to infinity.

Under these assumptions, the theorem asserts the convergence of the solution sequences $x_i^{*}$, ${\lambda }_{eq}^{*}$, and ${\lambda }_{ineq}^{*}$ to $x^{**}$, ${\lambda }_{eq}^{**}$, and ${\lambda }_{ineq}^{**}$, respectively, as i tends to infinity (32). Here, $x^{**}\in {\mathcal{X}}^{*}$ and $\left({\lambda }_{eq}^{**},{\lambda }_{ineq}^{**}\right)\in {\mathsf{\Xi }}^{*}$ define the solution of the game with the minimal norm (33).

In essence, Theorem 2 establishes conditions under which a unique equilibrium solution of the game can be achieved, ensuring convergence of the solution sequences to a solution with minimal norm. These conditions are crucial for guaranteeing the stability and optimality of the solution in the dynamic environment described by the nonlinear programming problem.

Proof. 

See Appendix A. □

5. Numerical Example

5.1. Description of the Oligopoly Problem

In the typical structure of an oligopolistic market, only a handful of producers offer similar or identical products, resulting in homogeneous goods [21]. This market arrangement is characterized by a known demand function, with consumers playing a passive role. In this scenario, the producers themselves act as the players in the market dynamics. To address the challenges posed by oligopoly, producers often agree to split the market among themselves, recognizing that fragmenting the market further into smaller companies would be counterproductive. The price each producer receives for their product is influenced by the quantity of goods produced, and producers react to each other’s production adjustments until an equilibrium is reached. This equilibrium, known as a Nash equilibrium, occurs when no producer can improve their payoff by unilaterally changing their strategy, given the strategies of the other producers. In essence, producers in an oligopolistic market strive to balance their production decisions and pricing strategies to achieve a stable market outcome. This strategic interaction among producers is a defining characteristic of oligopoly, shaping market dynamics and outcomes.

5.2. Resulting Values

Applying our method yields a resulting mechanism that effectively optimizes strategic interactions. This mechanism orchestrates player behaviors to converge towards desired outcomes, promoting stability and efficiency within the game environment. Through iterative refinement and adjustment, it guides players towards actions that maximize collective welfare while respecting constraints and objectives.

$$\begin{array}{c}{\phi }_{\alpha |m}^{*}=\left[\begin{array}{ccc}0.3287& 0.3287& 0.3426\\ 0.2637& 0.2637& 0.4727\\ 0.2302& 0.2302& 0.5396\\ 0.3111& 0.3110& 0.3779\end{array}\right]\end{array}$$

The resulting behavior strategies emerge from our method, encapsulating players’ optimal actions within the game. These strategies are finely tuned to navigate the complexities of strategic interactions, aiming to achieve desirable outcomes while balancing competing objectives and constraints effectively.

$$\begin{array}{c}{\sigma }_{m|\theta }^{1*}=\left[\begin{array}{cccc}0.1918& 0.3711& 0.1921& 0.2450\\ 0.2500& 0.2500& 0.2500& 0.2500\\ 0.2503& 0.2499& 0.2495& 0.2502\\ 0.2423& 0.2422& 0.2733& 0.2422\end{array}\right]\end{array}\begin{array}{c}{\sigma }_{m|\theta }^{2*}=\left[\begin{array}{cccc}0.1124& 0.3184& 0.4568& 0.1124\\ 0.2650& 0.2416& 0.2383& 0.2551\\ 0.2597& 0.2384& 0.2389& 0.2630\\ 0.2413& 0.2466& 0.2715& 0.2407\end{array}\right]\end{array}$$

$$\begin{array}{c}{\sigma }_{m|\theta }^{3*}=\left[\begin{array}{cccc}0.1757& 0.1760& 0.4518& 0.1964\\ 0.2425& 0.2418& 0.2746& 0.2411\\ 0.2436& 0.2436& 0.2436& 0.2693\\ 0.2500& 0.2500& 0.2500& 0.2500\end{array}\right]\end{array}$$

The resulting action kernels represent the distribution of players’ actions based on their observed information and strategic considerations. These kernels encapsulate the strategic decision-making process within the game, guiding players towards actions that maximize their utility while considering the actions of others and the overall game dynamics.

$$\begin{array}{c}{\xi }_{a|\alpha }^{1*}=\left[\begin{array}{ccc}0.3286& 0.3286& 0.3427\\ 0.3333& 0.3333& 0.3333\\ 0.2324& 0.2324& 0.5352\end{array}\right]\end{array}\begin{array}{c}{\xi }_{a|\alpha }^{2*}=\left[\begin{array}{ccc}0.3254& 0.3254& 0.3491\\ 0.3172& 0.3172& 0.3656\\ 0.1760& 0.1760& 0.6480\end{array}\right]\end{array}$$

$$\begin{array}{c}{\xi }_{a|\alpha }^{3*}=\left[\begin{array}{ccc}0.3333& 0.3333& 0.3333\\ 0.3292& 0.3292& 0.3416\\ 0.2126& 0.2126& 0.5748\end{array}\right]\end{array}$$

The distribution vectors outline the probabilistic allocation of resources or outcomes within the game. These vectors represent the distribution of strategic variables or states among players, providing insights into the distributional dynamics and resource allocation strategies employed within the game environment.

$${\mu }_{\theta }^{1*}=\left[\begin{array}{c}0.4537\\ 0.1800\\ 0.1805\\ 0.1858\end{array}\right]{\mu }_{\theta }^{2*}=\left[\begin{array}{c}0.4032\\ 0.1974\\ 0.2024\\ 0.1971\end{array}\right]{\mu }_{\theta }^{3*}=\left[\begin{array}{c}0.4468\\ 0.1884\\ 0.1848\\ 0.1800\end{array}\right]$$

The convergence of strategies is illustrated in Figure 1, Figure 2 and Figure 3. These figures track the evolution of strategies over time, demonstrating their convergence towards stable equilibria. They offer valuable insights into the dynamics of strategic interactions within the game, highlighting how players’ behaviors evolve and stabilize over iterations.

Designing a mechanism that aligns the equilibrium state with the market state, as prescribed by the welfare theorem, offers a promising approach to addressing the complexities of oligopolies with homogeneous products. The literature on this topic extensively investigates the presence of Nash equilibrium in dominant tactics, employing various mechanism designs such as strategic games. In strategic games, players may lack the motivation to provide accurate information, and they are incentivized to reveal private information to manipulate outcomes. This dynamic underscores the intricate interplay between strategic behavior and information asymmetry in oligopolistic markets. Using RL methods, this example provides a game-theoretic study of economic systems and observers. RL offers a powerful tool for analyzing complex interactions among agents in economic systems, allowing for the exploration of emergent behaviors and equilibrium states. By leveraging RL techniques, researchers can gain insights into the strategic decision-making processes of agents in oligopolistic markets, shedding light on optimal strategies and equilibrium outcomes. This approach enriches our understanding of economic dynamics and informs the design of more effective mechanisms for managing oligopolistic competition.

6. Conclusions and Future Work

The paper introduced a pioneering mechanism design tailored for uncertain environments marked by the inability to observe players’ actions. Within the framework of a Markov game, players observe outcomes influenced by both the payoff state and their actions while constrained by either moral hazard or adverse selection. The proposed mechanism incentivizes players to disclose truthful information about their states, thus maximizing expected payoffs. This perspective offers valuable insights into the Bayesian literature by addressing challenges posed by hidden actions and information asymmetry in dynamic environments. Integrating concepts from game theory and Bayesian analysis, the model provides a robust framework for designing effective incentive schemes aligning players’ incentives with desired outcomes. Emphasizing truthful state revelation underscores the importance of aligning incentives to mitigate adverse effects of moral hazard and adverse selection. This approach not only enhances decision-making processes but also contributes to developing more efficient and equitable systems in uncertain environments.

A numerical example centered on oligopolies underscores the effectiveness of our proposed method. Through this example, we showcase how our approach can adeptly tackle challenges within oligopolistic markets, highlighting its practical relevance and applicability in real-world scenarios. Crafting a mechanism that synchronizes the equilibrium state with the market state, as advocated by the welfare theorem, presents a promising avenue for navigating the intricacies of oligopolies with homogeneous products. Extensive literature on this subject delves into probing the existence of Nash equilibrium under dominant tactics, employing diverse mechanism designs like strategic games. In strategic games, players might lack the motivation to furnish accurate information, as they are incentivized to disclose private information to influence outcomes. This exploration underscores the challenge of aligning incentives to induce truthful information revelation, which is vital for devising effective mechanisms to address the complexities inherent in oligopolistic market dynamics.

The model’s relevance extends beyond theoretical realms, offering practical implications for various real-world scenarios characterized by prevalent information asymmetry. By presenting a systematic approach to mechanism design in uncertain environments, the paper provides valuable insights for policymakers, practitioners, and researchers grappling with complex decision-making challenges in dynamic settings. The proposed mechanism design represents a significant contribution to the Bayesian literature, introducing a fresh perspective on addressing the hurdles of hidden actions and information asymmetry. Its focus on promoting truthful state revelation and maximizing expected payoffs underscores its significance and versatility across diverse contexts, rendering it a valuable tool for advancing our understanding of decision-making under uncertainty.