Manipulation Game Considering No-Regret Strategies

Abstract

This paper examines manipulation games through the lens of Machiavellianism, a psychological theory. It analyzes manipulation dynamics using principles like hierarchical perspectives, exploitation tactics, and the absence of conventional morals to interpret interpersonal interactions. Manipulators intersperse unethical behavior within their typical conduct, deploying deceptive tactics before resuming a baseline demeanor. The proposed solution leverages Lyapunov theory to establish and maintain Stackelberg equilibria. A Lyapunov-like function supports each asymptotically stable equilibrium, ensuring convergence to a Nash/Lyapunov equilibrium if it exists, inherently favoring no-regret strategies. The existence of an optimal solution is demonstrated via the Weierstrass theorem. The game is modeled as a three-level Stackelberg framework based on Markov chains. At the highest level, manipulators devise strategies that may not sway middle-level manipulated players, who counter with best-reply strategies mirroring the manipulators’ moves. Lower-level manipulators adjust their strategies in response to the manipulated players to sustain the manipulation process. This integration of stability analysis and strategic decision-making provides a robust framework for understanding and addressing manipulation in interpersonal contexts. A numerical example focusing on the oil market and its regulations highlights the findings of this work.

1. Introduction

1.1. Brief Review

The inclusion of believable behavioral assumptions into traditional economic models has sparked extensive debate. Manipulation emerges as a prominent concern, with game-theoretic studies revealing significant individual disparities in scenarios allowing such behavior. This work focuses on Machiavellianism—a personality trait associated with manipulation, callousness, and moral indifference—as studied in personality psychology. While enriching economic models with believable behavioral assumptions enhances their descriptive accuracy, it also introduces complexities and challenges in predicting human behavior and its economic outcomes. Thus, understanding the implications of incorporating these assumptions is crucial for advancing economic theory and policy [1].

Christie and Geis [2] utilized adapted Machiavellian remarks to investigate variations in human behavior, drawing from Machiavelli’s dichotomy of individuals as either manipulators or the manipulated. This classification reflects a worldview steeped in power dynamics and strategic interactions. Machiavellian individuals are characterized by a propensity to exploit others, harbor a pessimistic view of human nature, and demonstrate a lack of moral accountability. The concept of Machiavellianism underscores the absence of coordination in self-serving, conflict-laden situations, particularly in games where interests diverge.

Christie and Geis further delineated Machiavellian traits into three key components:

• Worldview shaped by manipulators and manipulated (views component), lacking sympathy; manipulated individuals also engage in manipulation.
• Lack of regard for conventional morality (morality component), indifferent to actions like lying and cheating.
• Emphasis on pragmatic problem-solving over ideological commitments, utilizing power strategies (tactics component) for personal goals rather than ideological ones.

Understanding Machiavellianism offers valuable insights into the intricacies of social dynamics, decision-making, and interpersonal relationships. This psychological framework helps unravel how individuals navigate power structures, negotiate conflicting interests, and strategize to achieve personal goals. By dissecting Machiavellian traits—such as manipulation, pragmatic problem-solving, and moral flexibility—researchers can better understand the complexities of human behavior. It sheds light on how individuals employ strategic thinking in diverse contexts, from interpersonal interactions to organizational settings, revealing patterns of influence, persuasion, and control. Such analysis is crucial for comprehending motivations, fostering cooperation, and addressing conflicts in various social and professional environments [3,4].

1.2. Related Work

Allen and Gorton [5] delved into the concept of manipulation equilibrium, emphasizing the impact of stock price dynamics and their implications for welfare. Their work highlights how manipulation strategies can disrupt market equilibrium, leading to unintended consequences for various stakeholders. Building on this, Bagnoli and Lipman [6] demonstrated that efforts to manipulate stock prices often result in a decline in pre-bid stock prices, which can inflate takeover bids. This inflation can create barriers to successful acquisitions, particularly in times when the market experiences limited takeover activity, effectively creating an environment where genuine value is obscured by artificial price alterations.

Furthering the discussion on manipulation, Clempner [7] introduced a comprehensive game-theoretical framework designed to simulate such manipulation, integrating reinforcement learning techniques to explore moral considerations surrounding these strategies. This innovative approach sheds light on the ethical dimensions of decision-making in financial markets, illustrating how self-interest can clash with broader market integrity.

In a broader empirical study, Cumming et al. [8] examined a dataset comprising suspected instances of stock price manipulation across nine countries over an eight-year period. Their findings revealed significant adverse effects of market manipulation on innovation, suggesting that such practices stifle creativity and investment in new ideas. Interestingly, their research pointed out that the detrimental impacts are particularly severe in jurisdictions characterized by weak intellectual property rights coupled with strong shareholder protections, indicating that the regulatory environment plays a critical role in shaping market behavior. Clempner’s contributions continued with the development of a manipulation model tailored for repeated Stackelberg security games [9], which explores strategic interactions over time among players with hierarchical roles. Additionally, Clempner [10] introduced a Machiavellian game model founded on ergodic Bayesian Markov processes, further enriching the discourse on strategic manipulation in competitive environments. This work addresses how players with Machiavellian traits might exploit informational asymmetries to gain an advantage. Finally, significant advancements in the study of Machiavellianism games were presented by [11], who explored the nuances of strategic deception and manipulation within competitive settings, emphasizing the psychological and behavioral aspects that drive such strategies. Collectively, these studies provide a comprehensive view of how manipulation in financial markets not only affects stock prices but also has far-reaching implications for overall market efficiency and ethical standards.

In this theoretical framework, the sender, armed with private information, formulates a signaling strategy designed to influence the decision-making process of the receiver. This strategic communication is crucial because it allows the sender to steer the receiver’s actions in a desired direction. To explore extensions of this model that incorporate multiple senders, one can refer to the research conducted by Milgrom [12] and Krishna [13], who delve into the dynamics and interactions that arise when several senders attempt to convey information to a common receiver.

The idea of private signal-based persuasion has been investigated in diverse scenarios, ranging from two-player, two-action games [14] to unanimity voting systems [15]. These contexts provide valuable insights into how information asymmetries can shape competitive interactions and collective decision-making. Focusing specifically on the interaction between a single sender and a receiver, Kamenica and Gentzkow [16] developed a Bayesian persuasion framework. This model outlines the conditions under which the signals sent by the sender can be beneficial and delineates optimal strategies for signaling that enhance the sender’s objectives. Bergemann and Morris [17] further explored the role of a mechanism designer within the Bayesian persuasion framework. Their research examines how such a designer selects an appropriate information structure to fulfill specific objectives, shedding light on the strategic considerations involved in information dissemination and persuasion. Expanding on their previous findings, Gentzkow and Kamenica [18] introduced a model with multiple senders, analyzing scenarios where a vast array of potential signals exists. They employed a lattice structure to facilitate intuitive comparisons and combinations of these signals, enriching the understanding of how senders can effectively convey their information in a competitive landscape.

Additionally, Brocas et al. [19] proposed a novel model where two competing agents allocate resources to acquire public information about an uncertain state of the world. Their analysis focuses on the equilibrium strategies for information acquisition, revealing how these strategies evolve when one agent faces greater costs in gathering information. This research highlights the complexities involved in information competition and the tactical decisions that arise from resource allocation. In the realm of political communication, Gul and Pesendorfer [20] developed a model in which two senders with opposing interests share asymmetric information regarding a binary state of the world. This study illustrates how differing objectives can influence the nature of information conveyed and the strategic behavior of the senders involved. Moreover, several authors, including [21,22,23,24], have explored frameworks in which the information revealed by the sender is exclusively accessible to the recipients. These works emphasize the significance of control over information flow and the implications it has for decision-making processes in various contexts.

Collectively, these studies contribute to a deeper understanding of the mechanisms of persuasion and information asymmetry, highlighting the strategic complexities involved in both competitive and cooperative environments.

1.3. Main Results

This paper concerns manipulation games based on the psychological theory of Machiavellianism, which analyzes the world from the Machiavellian perspective using three guiding principles: view, tactics, and immoral behavior. View is related to the existence of manipulating and manipulated individuals, corresponding to a hierarchical scheme represented by a Stackelberg game where all the participants are involved in some degree of manipulation. Tactics are concerned with employing interpersonal exploitation strategies as a method for control. Immoral refers to a behavior lacking conventional morals that would prohibit their actions, which is designed using a Bayesian–Markov game theory approach. A manipulator is someone who may temporarily stray from steady conduct by committing an unethical act and then reverting to their usual behavior.

We propose a solution to the manipulation game problem involving Lyapunov theory, which supports the existence and stability of Nash equilibria: each asymptotically stable equilibrium point admits the so-called Lyapunov-like function that converges to a Nash/Lyapunov equilibrium point if it exists. The asymptotic behavior of a Lyapunov-like function admits by definition no-regret strategies.

The idea behind the representation of the game is that it is a three-level Stackelberg game, where the manipulating players choose a manipulation strategy at the highest level that might not convince the manipulated players in the middle level. A best-reply strategy that equals the manipulating strategy can be selected by the player who is being manipulated. In order to ensure the manipulation process, the manipulating player at the lower level then chooses a strategy once more based on the strategy of the manipulated players.

2. Markov Games

In a discrete-time, finite-horizon setting, we define an environment characterized by private and independent valuations [25]. Machiavellian players, denoted as $\iota \in \mathcal{N}$, where $\mathcal{N}=\left\{1,2,...,n\right\}$, incur a cost at each period $t\in T,$ with $T\subseteq \mathbb{N}$. This cost is influenced by the current physical allocation $y_t^{\iota }\in Y_t^{\iota }$ and the player’s type ${\theta }_t^{\iota }\in {\Theta }_t^{\iota }$. The cost function ${\phi }^{\iota }$ for Machiavellian player $\iota$ is represented as ${\phi }^{\iota }(y_t^{\iota },{\theta }_t^{\iota })\ge 0$, where ${\phi }^{\iota }$ maps $Y_t^{\iota }\times {\Theta }_t^{\iota }$ to ${\mathbb{R}}^{+}$.

At time period t, the type vector is represented as ${\theta }_t=({\theta }_t^1,...,{\theta }_t^n)\in \Theta$, where ${\Theta }_t$ is defined as the product ${\times }_{\iota \in \mathcal{N}}{\Theta }_t^{\iota }$. The set of permissible allocations in period t can depend on the vector of prior allocations $y_t=(y_t^1,...,y_t^n)\in Y$, where $Y_t={\times }_{\iota \in \mathcal{N}}{Y}_t^{\iota }$. We denote by ${\Theta }_t^{-l}$ the product ${\times }_{\iota \in \mathcal{N},\iota \ne l}{\Theta }_t^{\iota }$, and by $\Delta \left(Y_t\right)$ the set of all probability distributions over $Y_t$. It is assumed that $Y_t^{\iota }$, ${\Theta }_t^{\iota }$, and T are finite sets.

For each Machiavellian player $\iota$, there exists a shared prior distribution ${\mu }^{\iota }\left({\theta }_0^{\iota }\right)$. The player’s current type, ${\theta }_t^{\iota }$, along with their current action, $y_t^{\iota }$, determines the probability distribution for the next period’s state variable ${\theta }_{t+1}^{\iota }$ on ${\Theta }_{t+1}^{\iota }$. We assume that this probability distribution is described by a transition function (or stochastic kernel) $F^{\iota }\left({\theta }_{t+1}^{\iota }\right|{\theta }_t^{\iota },y_t^{\iota })$.

We assume that the type ${\theta }_t^{\iota }$ of Machiavellian player $\iota$ follows a Markov process over the state space ${\Theta }_t^{\iota }$, and the dynamics adhere to the Markov condition, expressed as follows:

$$F^{\iota }\left({\theta }_{t+1}^{\iota }\right|{\theta }_t^{\iota },y_t^{\iota },{\theta }_{t-1}^{\iota },y_{t-1}^{\iota }\dots ,{\theta }_0^{\iota },y_0^{\iota })=F^{\iota }\left({\theta }_{t+1}^{\iota }|{\theta }_t^{\iota },y_t^{\iota }\right).$$

The sequence of states and actions for each player $\iota$ is represented by $\left\{({\Theta }_t,Y_t)|t\right\}$. In this case, the process described by a Markov chain is fully characterized by the transition function $F^{\iota }\left({\theta }_{t+1}^{\iota }\right|{\theta }_t^{\iota },y_t^{\iota })$ and the common prior distribution ${\mu }^{\iota }\left({\theta }_0^{\iota }\right)$, where ${\mu }^{\iota }\left({\theta }_t^{\iota }\right)$ belongs to $\Delta \left({\Theta }_t^{\iota }\right)$, the set of probability distributions over ${\Theta }_t^{\iota }$. It is further assumed that each chain $({\mu }^{\iota },F^{\iota }\left({\theta }_{t+1}^{\iota }\right|{\theta }_t^{\iota },y_t^{\iota }))$ is ergodic [9,25].

The cost functions ${\phi }^{\iota }(y_t^{\iota },{\theta }_t^{\iota })$ and the transition functions $F^{\iota }\left({\theta }_{t+1}^{\iota }\right|{\theta }_t^{\iota },y_t^{\iota })$ are considered common knowledge among all agents $\iota$. The shared prior distribution ${\mu }^{\iota }\left({\theta }_0^{\iota }\right)$ and the transition function $F^{\iota }\left({\theta }_{t+1}^{\iota }\right|{\theta }_t^{\iota },y_t^{\iota })$ are assumed to be independent for each agent. At the start of each period t, every agent $\iota$ privately observes their type ${\theta }_t^{\iota }$. By the end of the period, after choosing an action $y_t^{\iota }\in Y_t^{\iota }$, the cots for that period are realized. Asymmetric information arises from each player’s private observation of ${\theta }_t^{\iota }$ at every time t. Due to the independence of the prior distributions ${\mu }^{\iota }\left({\theta }_0^{\iota }\right)$ and the transition functions $F^{\iota }\left({\theta }_{t+1}^{\iota }\right|{\theta }_t^{\iota },y_t^{\iota })$ for each player $\iota$, the information available to player $\iota$, determined by ${\theta }_{t+1}^{\iota }$, does not depend on the type $\theta t^l$ of any other player $l\ne \iota$. After this, each participant simultaneously sends messages $m_t^{\iota }\in M_t^{\iota }$, and the entire message profile is made public. Additionally, we assume that $M_t^{\iota }={\Theta }_t^{\iota }$ from this point onward.

The relationship ${\varrho }^{\iota }\left(a_t^{\iota }\right|y_t^{\iota })$ for the Machiavellian players is given by ${\varrho }^{\iota }:Y^{\iota }\to \Delta \left(Y^{\iota }\right)$. The set of actions that are admissible is defined by

$${\mathcal{Y}}_{adm}^{\iota }=\left\{{\varrho }^{\iota }\left(a_t^{\iota }\right|y_t^{\iota })|\sum _{a_t^{\iota }\in Y^{\iota }}{\varrho }^{\iota }\left(a_t^{\iota }\right|y_t^{\iota })=1,y_t^{\iota }\in Y^{\iota }\right\},$$

The relationship ${\varrho }^{\iota }\left(a_t^{\iota }\right|y_t^{\iota })$ represents the likelihood that a Machiavellian player $\iota$ believes that $a_t^{\iota }$ is an action $y_t^{\iota }$.

A (behavioral) strategy ${\sigma }^{\iota }\left(m_t^{\iota }\right|{\theta }_t^{\iota })$ for player $\iota$ is a function ${\sigma }^{\iota }:{\Theta }_t^{\iota }\to \Delta \left(M_t^{\iota }\right)$, which indicates the probability that player $\iota$ associates a message $m_t^{\iota }$ with type ${\theta }_t^{\iota }$. The set of (behavioral) strategies is defined as follows:

$${\Sigma }_{adm}^{\iota }={\displaystyle \{}{\sigma }^{\iota }\left(m_t^{\iota }\right|{\theta }_t^{\iota })\ge 0\left|\sum _{m_t^{\iota }\in M_t^{\iota }}{\sigma }^{\iota }\left(m_t^{\iota }\right|{\theta }_t^{\iota })=1,{\theta }_t^{\iota }\in {\Theta }_t^{\iota }\right.{\displaystyle \}}.$$

A policy refers to a series $\left\{{\pi }^{\iota }\left(y_t^{\iota }\right|m_t^{\iota })\right\}$ where, for each time period t, ${\pi }^{\iota }\left(y_t^{\iota }\right|m_t^{\iota })$ represents a stochastic function on $Y^{\iota }$. The collection of all possible policies is represented by the symbol $\Pi$. We have that

$${\Pi }_{adm}^{\iota }=\left\{{\pi }^{\iota }\left(y_t^{\iota }\right|m_t^{\iota })\ge 0\left|\sum _{y_t^{\iota }\in Y_t^{\iota }}{\pi }^{\iota }\left(y_t^{\iota }\right|m_t^{\iota })=1,m_t^{\iota }\in M_t^{\iota }\right.\right\}$$

Considering the Markovian nature of the type processes, the utility at time t starts at the type vector ${\theta }_t^{\iota }$, accompanied by the (behavioral) strategy ${\sigma }^{\iota }\left({\theta }_t^{\iota }\right|m_t^{\iota })$, the policy ${\pi }^{\iota }\left(y_t^{\iota }\right|{\theta }_t^{\iota })$, and the ex ante probability ${\mu }^{\iota }\left({\theta }_t^{\iota }\right)$. These elements can be expressed as

$$\begin{array}{c}{\Phi }^{\iota }(\pi ,\sigma )=\\ {\displaystyle \sum _{t\in T}}{\displaystyle \sum _{{\theta }_t^{\iota }\in {\Theta }_t^{\iota }}}{\displaystyle \sum _{m_t^{\iota }\in M_t^{\iota }}}{\displaystyle \sum _{y_t^{\iota }\in Y_t^{\iota }}}{\displaystyle \sum _{{\theta }_{t+1}^{\iota }\in {\Theta }_{t+1}^{\iota }}}{\phi }^{\iota }(y_t^{\iota },{\theta }_t^{\iota })F^{\iota }\left({\theta }_{t+1}^{\iota }\right|{\theta }_t^{\iota },y_t^{\iota }){\displaystyle \prod _{\iota \in \mathcal{N}}}{\pi }^{\iota }\left(y_t^{\iota }\right|m_t^{\iota }){\sigma }^{\iota }\left(m_t^{\iota }\right|{\theta }_t^{\iota }){\mu }^{\iota }\left({\theta }_t^{\iota }\right)\\ ={\displaystyle \sum _{t\in T}}{\displaystyle \sum _{{\theta }_t^{\iota }\in {\Theta }_t^{\iota }}}{\displaystyle \sum _{m_t^{\iota }\in M_t^{\iota }}}{\displaystyle \sum _{y_t^{\iota }\in Y_t^{\iota }}}{r}^{\iota }(y_t^{\iota },{\theta }_t^{\iota }){\displaystyle {\displaystyle \prod }_{\iota \in \mathcal{N}}}{\pi }^{\iota }\left(y_t^{\iota }\right|m_t^{\iota }){\sigma }^{\iota }\left(m_t^{\iota }\right|{\theta }_t^{\iota }){\mu }^{\iota }\left({\theta }_t^{\iota }\right),\end{array}$$

where $r^{\iota }(y_t^{\iota },{\theta }_t^{\iota })={\displaystyle \sum _{{\theta }_{t+1}^{\iota }\in {\Theta }_{t+1}^{\iota }}}{\phi }^{\iota }(y_t^{\iota },{\theta }_t^{\iota })F^{\iota }\left({\theta }_{t+1}^{\iota }\right|{\theta }_t^{\iota },y_t^{\iota })$.

We consider that players are aware of their respective costs. A policy $\pi$ along with the strategy $\sigma$ work together to minimize the cost function ${\Phi }^{\iota }(\pi ,\sigma )$, which operates according to the following rule:

$$({\pi }^{\ast },{\sigma }^{\ast }):=Arg\underset{\pi \in {\Pi }_{adm}}{min}\underset{\sigma \in {\Sigma }_{adm}}{min}\sum _{\iota \in \mathcal{N}}\Phi (\pi ,\sigma ).$$

(1)

The policy ${\pi }^{\ast }$ and strategy ${\sigma }^{\ast }$ satisfy the conditions of a Bayesian–Nash equilibrium, holding true for all $\sigma$, such that

$${\Phi }^{\iota }({\pi }^{\ast },{\sigma }^{\ast })\le {\Phi }^{\iota }({\pi }^{\iota },{\pi }^{-\iota \ast },{\sigma }^{\iota },{\sigma }^{-\iota \ast }).$$

(2)

where the policy ${\pi }^{\ast }=\left({\pi }^{1\ast },...,{\pi }^{n\ast }\right)$ and the strategy ${\sigma }^{\ast }=\left({\sigma }^{1\ast },...,{\sigma }^{n\ast }\right)$ are called Bayesian–Nash equilibrium, where $\pi$ and $\sigma$ fulfill that ${\pi }^{-\iota \ast }=$$\left({\pi }^{1\ast },...,{\pi }^{\iota -1\ast },{\pi }^{\iota +1,\ast },\right.$$\left....,{\pi }^{n\ast }\right)$ and ${\sigma }^{-\iota \ast }=$$\left({\sigma }^{1\ast },...,{\sigma }^{\iota -1\ast },{\sigma }^{\iota +1,\ast },...,{\sigma }^{n\ast }\right)$ [26].

Here is an explanation in words. This mathematical expression describes a game-theoretic framework where a policy $\pi$ and a strategy $\sigma$ are optimized to minimize a cost function ${\Phi }^{\iota }(\pi ,\sigma )$. The goal for cost minimization is to find the optimal policy ${\pi }^{\ast }$ and strategy ${\sigma }^{\ast }$ that minimize the overall cost function $\Phi (\pi ,\sigma )$ across a set of entities $\mathcal{N}$. For the Bayesian–Nash equilibrium, the optimal policy ${\pi }^{\ast }$ and strategy ${\sigma }^{\ast }$ must satisfy the conditions of a Bayesian–Nash equilibrium. This means that no single entity $\iota$ can reduce its individual cost ${\Phi }^{\iota }(\pi ,\sigma )$ by unilaterally deviating from the optimal policy or strategy, while others maintain their optimal choices. This framework ensures that each entity $\iota$ adopts its optimal policy ${\pi }^{\iota \ast }$ and strategy ${\sigma }^{\iota \ast }$, given that all others are also acting optimally (${\pi }^{-\iota \ast },{\sigma }^{-\iota \ast }$). The result is a state where no entity has an incentive to change its actions unilaterally, which aligns with the concept of equilibrium in game theory. In summary, the expression defines a systematic approach to optimizing policies and strategies in a multi-agent system, ensuring equilibrium while minimizing the overall and individual costs.

Let us introduce the $\xi$-variable as follows

$${\xi }^{\iota }({\theta }_t^{\iota },m_t^{\iota },y_t^{\iota }):={\pi }^{\iota }\left(y_t^{\iota }\right|m_t^{\iota }){\sigma }^{\iota }\left(m_t^{\iota }\right|{\theta }_t^{\iota }){\mu }^{\iota }\left({\theta }_t^{\iota }\right).$$

(3)

such that

$$\begin{array}{c}{\Xi }_{adm}^{\iota }:={\displaystyle \{}{\xi }^{\iota }({\theta }_t^{\iota },m_t^{\iota },y_t^{\iota })\left|{\displaystyle \sum _{{\theta }_t^{\iota }\in {\Theta }_t^{\iota }}}{\displaystyle \sum _{m_t^{\iota }\in {\Theta }_t^{\iota }}}{\displaystyle \sum _{y_t^{\iota }\in Y_t^{\iota }}}{\xi }^{\iota }({\theta }_t^{\iota },m_t^{\iota },y_t^{\iota })=1,\right.\\ {\displaystyle \sum _{m_t^{\iota }\in {\Theta }_t^{\iota }}}{\displaystyle \sum _{y_t^{\iota }\in Y_t^{\iota }}}{\xi }^{\iota }({\theta }_t^{\iota },m_t^{\iota },y_t^{\iota })={\mu }^{\iota }\left({\theta }_t^{\iota }\right)>0,\\ {\displaystyle \sum _{m_t^{\iota }\in {\Theta }_t^{\iota }}}{\displaystyle \sum _{y_t^{\iota }\in Y_t^{\iota }}}{\displaystyle \sum _{{\theta }_t^{\iota }\in {\Theta }_t^{\iota }}}[{\delta }_{{\theta }_t^{\iota }{\theta }_{t+1}^{\iota }}-F^{\iota }\left({\theta }_{t+1}^{\iota }\right|{\theta }_t^{\iota },y_t^{\iota })]{\xi }^{\iota }({\theta }_t^{\iota },m_t^{\iota },y_t^{\iota })=0,\\ {\displaystyle \sum _{m_t^{\iota }\in {\Theta }_t^{\iota }}}{\displaystyle \sum _{y_t^{\iota }\in Y_t^{\iota }}}{\displaystyle \sum _{{\theta }_t^{\iota }\in {\Theta }_t^{\iota }}}{P}^{\iota }\left(m_t^{\iota }\right|{\vartheta }_t^{\iota }){\xi }^{\iota }({\theta }_t^{\iota },m_t^{\iota },y_t^{\iota })\ge 0,{\vartheta }_t^{\iota }\in {\Theta }_t^{\iota }{\displaystyle \}}\end{array}$$

such that $P^{\iota }\left(m_t^{\iota }\right|{\theta }_t^{\iota })={\left({\sigma }^{\iota }\left(m_t^{\iota }\right|{\theta }_t^{\iota })\right)}^{-1}$ is the inverse and it exists; then, the policy ${\pi }^{\iota }\left(y_t^{\iota }\right|m_t^{\iota })$ and the strategy ${\sigma }^{\iota }\left(m_t^{\iota }\right|{\theta }_t^{\iota })$ form a Bayesian–Nash equilibrium. In this equilibrium, each player minimizes their expected cost for every $\iota \in \mathcal{N}$

$${\Phi }^{\iota }({\pi }^{\iota \ast }\left(y_t^{\iota }\right|m_t^{\iota }),{\sigma }^{\ast }\left(m_t^{\iota }\right|{\theta }_t^{\iota }))\le {\Phi }^{\iota }({\pi }^{\iota }\left(y_t^{\iota }\right|m_t^{\iota }),\sigma \left(m_t^{\iota }\right|{\theta }_t^{\iota })).$$

The variable ${\xi }^{\iota }({\theta }_t^{\iota },m_t^{\iota },y_t^{\iota })$ defined in Equation (3) must satisfy the following individual nonlinear optimization problem if and only if

$$\begin{array}{c}\Phi (\pi ,\sigma )=\sum _{\iota \in \mathcal{N}}{\tilde{\Phi }}^{\iota }\left(\xi \right)\to {\displaystyle \underset{\xi \in {\Xi }_{adm}}{min}}\end{array}$$

(4)

where

$$\begin{array}{c}{\tilde{\Phi }}^{\iota }\left(\xi \right)={\displaystyle \sum _{{\theta }_t^{\iota }\in {\Theta }_t^{\iota }}}{\displaystyle \sum _{m_t^{\iota }\in {\Theta }_t^{\iota }}}{\displaystyle \sum _{y_t^{\iota }\in Y_t^{\iota }}}{\phi }^{\iota }(y_t^{\iota },{\theta }_t^{\iota }){\displaystyle \prod _{\iota \in \mathcal{N}}}{\xi }^{\iota }({\theta }_t^{\iota },m_t^{\iota },y_t^{\iota })\end{array}$$

and ${\Xi }_{adm}={\times }_{\iota \in \mathcal{N}}^{\iota }{\Xi }_{adm}$.

Remark 1.

We have that

$$\begin{array}{ccc}{\displaystyle \sum _{y_t^{\iota }\in Y_t^{\iota }}}{\pi }^{\iota }\left(y_t^{\iota }\right|m_t^{\iota })=1,& {\displaystyle \sum _{m_t^{\iota }\in {\Theta }_t}}\sigma \left(m_t^{\iota }\right|{\theta }_t^{\iota })=1,& {\displaystyle \sum _{{\theta }_t^{\iota }\in {\Theta }_t^{\iota }}}{\mu }^{\iota }\left({\theta }_t^{\iota }\right)=1.\end{array}$$

The solution of the problem (4) is defined as ${\xi }^{\ast }$.

The variables ${\pi }^{\iota \ast }\left(y_t^{\iota }\right|m_t^{\iota })$ can be derived from ${\xi }^{\iota }({\theta }_t^{\iota },m_t^{\iota },y_t^{\iota })$ in the following way:

$$\begin{array}{c}{\pi }^{\iota \ast }\left(y_t^{\iota }\right|m_t^{\iota })={\displaystyle \frac{1}{\left|{\Theta }_t^{\iota }\right|}}{\displaystyle \sum _{{\theta }_t^{\iota }\in {\Theta }_t^{\iota }}}{\displaystyle \frac{{\xi }^{\iota }({\theta }_t^{\iota },m_t^{\iota },y_t^{\iota })}{{\displaystyle \sum _{y_t^{\iota }\in Y_t^{\iota }}}{\xi }^{\iota }({\theta }_t^{\iota },m_t^{\iota },y_t^{\iota })}}.\end{array}$$

(5)

To retrieve ${\sigma }^{\iota \ast }\left(m_t^{\iota }\right|{\theta }_t^{\iota })$, for each player $\iota \in \mathcal{N}$, the relevant quantity is defined as follows:

$$\begin{array}{c}{\sigma }^{\iota \ast }\left(m_t^{\iota }\right|{\theta }_t^{\iota })={\displaystyle \frac{{\displaystyle \sum _{y_t^{\iota }\in Y_t^{\iota }}}{\xi }^{\iota }({\theta }_t^{\iota },m_t^{\iota },y_t^{\iota })}{{\displaystyle \sum _{m_t^{\iota }\in M_t^{\iota }}}{\displaystyle \sum _{y_t^{\iota }\in Y_t^{\iota }}}{\xi }^{\iota }({\theta }_t^{\iota },m_t^{\iota },y_t^{\iota })}}.\end{array}$$

(6)

As well as the distribution ${\mu }^{\iota \ast }\left({\theta }_t^{\iota }\right)$, we have

$$\begin{array}{c}{\mu }^{\iota \ast }\left({\theta }_t^{\iota }\right)={\displaystyle \sum _{m_t^{\iota }\in M_t^{\iota }}}{\displaystyle \sum _{y_t^{\iota }\in Y_t^{\iota }}}{\xi }^{\iota }({\theta }_t^{\iota },m_t^{\iota },y_t^{\iota })>0.\end{array}$$

(7)

The formulas that minimize Equation (4) with respect to the variables ${\xi }^{\iota }({\theta }_t^{\iota },m_t^{\iota },y_t^{\iota })$ have been determined, along with those needed to recover the strategy ${\sigma }^{\iota \ast }\left(m_t^{\iota }\right|{\theta }_t^{\iota })$, the policy ${\pi }^{\iota \ast }\left(y_t^{\iota }\right|m_t^{\iota })$, and the distribution ${\mu }^{\iota \ast }\left({\theta }_t^{\iota }\right)$. When players use a reporting strategy ${\sigma }^{\iota }\left(m_t^{\iota }\right|{\theta }_t^{\iota })$, they are minimizing the expected cost described by Equation (4). The calculated strategy profile ${\sigma }^{\iota \ast }\left(m_t^{\iota }\right|{\theta }_t^{\iota })$ and policy ${\pi }^{\iota \ast }\left(y_t^{\iota }\right|m_t^{\iota })$ constitute a Bayesian–Nash equilibrium.

3. No-Regret Strategy

To tackle this issue, we propose modeling the state-value function ${\Phi }^{\iota }$ using a linear framework, defined in terms of ${\pi }^{\iota }\left(y_t^{\iota }\right|m_t^{\iota }){\sigma }^{\iota }\left(m_t^{\iota }\right|{\theta }_t^{\iota })$. The aim is to develop a strategy that strictly minimizes the trajectory value. Accordingly, ${\Phi }^{\iota }$ is formulated in a recursive matrix structure, facilitating efficient computations and in-depth analysis across various states and control parameters. This formulation optimizes the policy by enhancing both performance and decision-making, providing a systematic approach for achieving optimal control in dynamic systems [25].

3.1. The Cost Function

The cost function ${\Phi }^{\iota }$ for any fixed strategies ${\pi }^{\iota }\left(y_t^{\iota }\right|m_t^{\iota }){\sigma }^{\iota }\left(m_t^{\iota }\right|{\theta }_t^{\iota })$ is defined over all possible state–action combinations. It represents the expected value of taking action $y_t^{\iota }$ and sending a message $m_t^{\iota }$ considering state ${\theta }_t^{\iota }$ and subsequently following ${\pi }^{\iota }\left(y_t^{\iota }\right|m_t^{\iota }){\sigma }^{\iota }\left(m_t^{\iota }\right|{\theta }_t^{\iota })$. This function enables the evaluation of the long-term cost associated with a particular strategy. The ${\Phi }^{\iota }$-values for all the states in open format can be expressed by

$$\begin{array}{c}{\Phi }^{\iota }(\pi ,\sigma )=\\ {\displaystyle \sum _{t\in T}}{\displaystyle \sum _{{\theta }_t^{\iota }\in {\Theta }_t^{\iota }}}{\displaystyle \sum _{m_t^{\iota }\in M_t^{\iota }}}{\displaystyle \sum _{y_t^{\iota }\in Y_t^{\iota }}}{\displaystyle \sum _{{\theta }_{t+1}^{\iota }\in {\Theta }_{t+1}^{\iota }}}{\phi }^{\iota }(y_t^{\iota },{\theta }_t^{\iota })F^{\iota }\left({\theta }_{t+1}^{\iota }\right|{\theta }_t^{\iota },y_t^{\iota }){\pi }^{\iota }\left(y_t^{\iota }\right|m_t^{\iota }){\sigma }^{\iota }\left(m_t^{\iota }\right|{\theta }_t^{\iota }){\mu }^{\iota }\left({\theta }_t^{\iota }\right)\\ ={\displaystyle \sum _{t\in T}}{\displaystyle \sum _{{\theta }_t^{\iota }\in {\Theta }_t^{\iota }}}{\displaystyle \sum _{m_t^{\iota }\in M_t^{\iota }}}{\displaystyle \sum _{y_t^{\iota }\in Y_t^{\iota }}}{r}^{\iota }(y_t^{\iota },{\theta }_t^{\iota }){\pi }^{\iota }\left(y_t^{\iota }\right|m_t^{\iota }){\sigma }^{\iota }\left(m_t^{\iota }\right|{\theta }_t^{\iota }){\mu }^{\iota }\left({\theta }_t^{\iota }\right),\end{array}$$

where $r^{\iota }(y_t^{\iota },{\theta }_t^{\iota })={\displaystyle \sum _{{\theta }_{t+1}^{\iota }\in {\Theta }_{t+1}^{\iota }}}{\phi }^{\iota }(y_t^{\iota },{\theta }_t^{\iota })F^{\iota }\left({\theta }_{t+1}^{\iota }\right|{\theta }_t^{\iota },y_t^{\iota })$.

$$\mathsf{E}\left({\Phi }_{t+1}^{\iota }(\pi ,\sigma )\right)={\displaystyle \sum _{t\in T}}{\displaystyle \sum _{{\theta }_t^{\iota }\in {\Theta }_t^{\iota }}}{\displaystyle \sum _{m_t^{\iota }\in M_t^{\iota }}}{\displaystyle \sum _{y_t^{\iota }\in Y_t^{\iota }}}{r}^{\iota }(y_t^{\iota },{\theta }_t^{\iota }){\pi }^{\iota }\left(y_t^{\iota }\right|m_t^{\iota }){\sigma }^{\iota }\left(m_t^{\iota }\right|{\theta }_t^{\iota }){\mu }^{\iota }\left({\theta }_t^{\iota }\right),$$

(8)

such that ${\phi }^{\iota }(y_t^{\iota },{\theta }_t^{\iota })$ is a loss value at type ${\theta }_t^{\iota }$ when the action $y_t^{\iota }$ is applied (without loss of generality it can be assumed to be positive) and ${\mu }^{\iota }\left({\theta }_t^{\iota }\right)$ for any given ${\mu }^{\iota }\left({\theta }_0^{\iota }\right)$ is defined as follows

$${\mu }^{\iota }\left({\theta }_{t+1}^{\iota }\right)={\displaystyle \sum _{t\in T}}{\displaystyle \sum _{{\theta }_t^{\iota }\in {\Theta }_t^{\iota }}}{\displaystyle \sum _{m_t^{\iota }\in M_t^{\iota }}}{\displaystyle \sum _{y_t^{\iota }\in Y_t^{\iota }}}{F}^{\iota }\left({\theta }_{t+1}^{\iota }\right|{\theta }_t^{\iota },y_t^{\iota }){\pi }^{\iota }\left(y_t^{\iota }\right|m_t^{\iota }){\sigma }^{\iota }\left(m_t^{\iota }\right|{\theta }_t^{\iota }){\mu }^{\iota }\left({\theta }_t^{\iota }\right)$$

or, in matrix format,

$$\begin{array}{c}{\mu }_{t+1}^{\iota }={\left({\mathbf{F}}_t^{\iota }\right)}_t^{\top }{\mu }_t^{\iota },\\ \\ {\mathbf{F}}_t^{\iota }:={\displaystyle \sum _{t\in T}}{\displaystyle \sum _{{\theta }_t^{\iota }\in {\Theta }_t^{\iota }}}{\displaystyle \sum _{m_t^{\iota }\in M_t^{\iota }}}{\displaystyle \sum _{y_t^{\iota }\in Y_t^{\iota }}}{F}^{\iota }\left({\theta }_{t+1}^{\iota }\right|{\theta }_t^{\iota },y_t^{\iota }){\pi }^{\iota }\left(y_t^{\iota }\right|m_t^{\iota }){\sigma }^{\iota }\left(m_t^{\iota }\right|{\theta }_t^{\iota }).\end{array}$$

Remark 2.

We will suppose that

$${\phi }^{\iota }(y_t^{\iota },{\theta }_t^{\iota })>0,\mathit{for}\mathit{all}\iota \in \mathcal{N}.$$

Considering

$$\begin{array}{c}\mathsf{E}\left({\Phi }_{t+1}^{\iota }(\pi ,\sigma )\right)=\\ \\ {\displaystyle \sum _{t\in T}}{\displaystyle \sum _{{\theta }_t^{\iota }\in {\Theta }_t^{\iota }}}{\displaystyle \sum _{m_t^{\iota }\in M_t^{\iota }}}{\displaystyle \sum _{y_t^{\iota }\in Y_t^{\iota }}}{\displaystyle \sum _{{\theta }_{t+1}^{\iota }\in {\Theta }_{t+1}^{\iota }}}{r}^{\iota }(y_t^{\iota },{\theta }_t^{\iota }){\pi }^{\iota }\left(y_t^{\iota }\right|m_t^{\iota }){\sigma }^{\iota }\left(m_t^{\iota }\right|{\theta }_t^{\iota }){\mu }^{\iota }\left({\theta }_t^{\iota }\right)=\\ \\ {\displaystyle \sum _{t\in T}}{\displaystyle \sum _{{\theta }_t^{\iota }\in {\Theta }_t^{\iota }}}{\displaystyle \sum _{m_t^{\iota }\in M_t^{\iota }}}{\displaystyle \sum _{y_t^{\iota }\in Y_t^{\iota }}}{\displaystyle \sum _{{\theta }_{t+1}^{\iota }\in {\Theta }_{t+1}^{\iota }}}\left[{\phi }^{\iota }(y_t^{\iota },{\theta }_t^{\iota })+c\right]·\left[F^{\iota }\left({\theta }_{t+1}^{\iota }\right|{\theta }_t^{\iota },y_t^{\iota }){\pi }^{\iota }\left(y_t^{\iota }\right|m_t^{\iota }){\sigma }^{\iota }\left(m_t^{\iota }\right|{\theta }_t^{\iota }){\mu }^{\iota }\left({\theta }_t^{\iota }\right)-c\right]\end{array}$$

the minimization of the state-value function $\mathsf{E}({\Phi }_{t+1}^{\iota }(\pi ,\sigma ))$ is equivalent to the minimization of the function $\mathsf{E}({\tilde{\Phi }}_{t+1}^{\iota }(\pi ,\sigma ))$ where

$${\tilde{\Phi }}_{t+1}^{\iota }(\pi ,\sigma )={\tilde{\Phi }}_{t+1}^{\iota }(\pi ,\sigma )+c,$$

which is strictly positive if we take

$$c>\underset{y_t^{\iota }\in Y_t^{\iota },{\theta }_t^{\iota }\in {\Theta }_t^{\iota }}{max}{\phi }^{\iota }(y_t^{\iota },{\theta }_t^{\iota }).$$

(9)

In a vector format, the Formula (8) can be expressed as

$$\begin{array}{c}\mathsf{E}\left({\Phi }_{t+1}^{\iota }(\pi ,\sigma )\right)=\\ {\displaystyle \sum _{t\in T}}{\displaystyle \sum _{{\theta }_t^{\iota }\in {\Theta }_t^{\iota }}}\left[{\displaystyle \sum _{m_t^{\iota }\in M_t^{\iota }}}{\displaystyle \sum _{y_t^{\iota }\in Y_t^{\iota }}}{\displaystyle \sum _{{\theta }_{t+1}^{\iota }\in {\Theta }_{t+1}^{\iota }}}{r}^{\iota }(y_t^{\iota },{\theta }_t^{\iota }){\pi }^{\iota }\left(y_t^{\iota }\right|m_t^{\iota }){\sigma }^{\iota }\left(m_t^{\iota }\right|{\theta }_t^{\iota })\right]{\mu }^{\iota }\left({\theta }_t^{\iota }\right)=〈{\mathbf{R}}_t^{\iota },{\mu }_t^{\iota }〉,\end{array}$$

where

$$\begin{array}{c}{\mathbf{R}}_t^{\iota }:={\displaystyle \sum _{m_t^{\iota }\in M_t^{\iota }}}{\displaystyle \sum _{y_t^{\iota }\in Y_t^{\iota }}}{\displaystyle \sum _{{\theta }_{t+1}^{\iota }\in {\Theta }_{t+1}^{\iota }}}{r}^{\iota }(y_t^{\iota },{\theta }_t^{\iota }){\pi }^{\iota }\left(y_t^{\iota }\right|m_t^{\iota }){\sigma }^{\iota }\left(m_t^{\iota }\right|{\theta }_t^{\iota }),\\ \\ r^{\iota }(y_t^{\iota },{\theta }_t^{\iota }):={\displaystyle \sum _{{\theta }_{t+1}^{\iota }\in {\Theta }_{t+1}^{\iota }}}{\phi }^{\iota }(y_t^{\iota },{\theta }_t^{\iota })F^{\iota }\left({\theta }_{t+1}^{\iota }\right|{\theta }_t^{\iota },y_t^{\iota }),\\ \\ {\mathbf{U}}_t^{\iota }:={\displaystyle \sum _{t\in T}}{\displaystyle \sum _{{\theta }_t^{\iota }\in {\Theta }_t^{\iota }}}{\displaystyle \sum _{m_t^{\iota }\in M_t^{\iota }}}{\displaystyle \sum _{y_t^{\iota }\in Y_t^{\iota }}}{F}^{\iota }\left({\theta }_{t+1}^{\iota }\right|{\theta }_t^{\iota },y_t^{\iota }){\pi }^{\iota }\left(y_t^{\iota }\right|m_t^{\iota }){\sigma }^{\iota }\left(m_t^{\iota }\right|{\theta }_t^{\iota }).\end{array}$$

Remark 3.

Lyapunov theory plays a critical role in ensuring the stability of dynamic systems by providing conditions under which a system remains in equilibrium. Beyond stability, Lyapunov theory also offers a pathway for these systems to converge toward a Nash equilibrium, where all participants optimize their strategies. This dual function—guaranteeing both stability and convergence to optimal outcomes—enhances the practical applicability of the framework. It ensures that not only will the system remain stable over time, but it will also naturally evolve toward a state of optimal strategic interactions. This makes the framework robust and suitable for real-world applications requiring both stability and optimality.

3.2. Optimal Stationary Strategy

Let us first introduce the following Lemma.

Lemma 1.

Any nonstationary strategy ${\xi }^{\iota }({\theta }_t^{\iota },m_t^{\iota },y_t^{\iota })$ cannot improve the average cost obtained by employing an optimal stationary strategy ${\xi }^{\iota \ast }({\theta }^{\iota \ast },m^{\iota \ast },y^{\iota \ast })$ defined on a compact set. An optimal solution exists.

Proof. 

According to the Weierstrass theorem, any nonstationary bounded strategy ${\xi }^{\iota }({\theta }_t^{\iota },m_t^{\iota },y_t^{\iota })$, defined on a compact set, necessarily contains a convergent subsequence that satisfies the following relationships:

$$\begin{array}{c}{\displaystyle \underset{n\to \infty }{lim\; inf}}{\displaystyle \frac{1}{t}}{\displaystyle \sum _{n=1}^t}{\tilde{\Phi }}^{\iota }\left({\xi }^{\iota }({\theta }_t^{\iota },m_t^{\iota },y_t^{\iota })\right)\ge {\displaystyle \underset{t\to \infty }{lim\; inf}}{\displaystyle \frac{1}{t}}{\displaystyle \sum _{n=1}^t}{\displaystyle \underset{k=1,...,n}{min}}{\tilde{\Phi }}^{\iota }\left({\xi }^{\iota }({\theta }_k^{\iota },m_k^{\iota },y_k^{\iota })\right)\ge \\ {\displaystyle \underset{n\to \infty }{lim\; inf}}{\displaystyle \frac{1}{t}}{\displaystyle \sum _{n=1}^t}{\displaystyle \underset{k\to \infty }{lim\; inf}}{\tilde{\Phi }}^{\iota }\left({\xi }^{\iota }({\theta }_{t_k}^{\iota },m_{t_k}^{\iota },y_{t_k}^{\iota })\right)={\tilde{\Phi }}^{\iota }\left({\xi }^{\iota \ast \ast }({\theta }^{\iota \ast \ast },m^{\iota \ast \ast },y^{\iota \ast \ast })\right),\end{array}$$

where ${\tilde{\Phi }}^{\iota }\left({\xi }^{\iota }({\theta }^{\iota },m^{\iota },y^{\iota })\right)$ is assumed to be a monotonically decreasing functional with respect to each component ${\xi }^{{\iota }^{\prime }}({\theta }^{{\iota }^{\prime }},m^{{\iota }^{\prime }},y^{{\iota }^{\prime }})$, when the others are held constant. Additionally,

$$\underset{k\to \infty }{lim\; inf}{\tilde{\Phi }}^{\iota }\left({\xi }^{\iota }({\theta }_{t_k}^{\iota },m_{t_k}^{\iota },y_{t_k}^{\iota })\right):={\tilde{\Phi }}^{\iota }({\xi }^{\iota \ast \ast }({\theta }^{\iota \ast \ast },m^{\iota \ast \ast },y^{\iota \ast \ast })$$

This lower bound is achieved by setting

$${\xi }^{\iota }({\theta }_t^{\iota },m_t^{\iota },y_t^{\iota })={\xi }^{\iota \ast }({\theta }^{\iota \ast },m^{\iota \ast },y^{\iota \ast })={\xi }^{\iota \ast \ast }({\theta }^{\iota \ast \ast },m^{\iota \ast \ast },y^{\iota \ast \ast })$$

and this is because

$${\displaystyle \underset{n\to \infty }{lim\; inf}}{\displaystyle \frac{1}{n}}{\displaystyle \sum _{h=1}^n}{\tilde{\Phi }}^{\iota }({\xi }^{\iota \ast }({\theta }^{\iota \ast },m^{\iota \ast },y^{\iota \ast }))={\tilde{\Phi }}^{\iota }({\xi }^{\iota \ast }({\theta }^{\iota \ast },m^{\iota \ast },y^{\iota \ast }))={\tilde{\Phi }}^{\iota }({\xi }^{\iota \ast \ast }({\theta }^{\iota \ast \ast },m^{\iota \ast \ast },y^{\iota \ast \ast })).$$

□

Remark 4.

It is essential to highlight that this property holds exclusively for average cost functionals defined on compact sets of strategies.

Proposition 1.

Let S be the unit simplex in ${\mathbf{R}}^M$, that is,

$$S=\{\kappa \in {\mathbf{R}}^{\left|Y\right|}|{\displaystyle \sum _{y_t^{\iota }\in Y_t^{\iota }}}\kappa \left(y_t^{\iota }\right)=1,\kappa \left(y_t^{\iota }\right)\ge 0\}.$$

Then,

$${\displaystyle \underset{\kappa \in S}{min}}\sum _{y_t^{\iota }\in Y_t^{\iota }}g\left(y_t^{\iota }\right)\kappa \left(y_t^{\iota }\right)=\underset{y_t^{\iota }\in Y_t^{\iota }}{min}g\left(y_t^{\iota }\right)=g\left(e\right),$$

such that the minimum is achieved at least for $\kappa =\left(\underset{e}{\underbrace{0,0,...,0,1}},0,...,0\right)$.

Proof. 

We have that

$${\displaystyle \sum _{y_t^{\iota }\in Y_t^{\iota }}}g\left(y_t^{\iota }\right)\kappa \left(y_t^{\iota }\right)\ge {\displaystyle \sum _{y_t^{\iota }\in Y_t^{\iota }}}(ming\left(y_t^{\iota }\right)\kappa \left(y_t^{\iota }\right))=ming\left(y_t^{\iota }\right){\displaystyle \sum _{y_t^{\iota }\in Y_t^{\iota }}}\kappa \left(y_t^{\iota }\right)=ming\left(y_t^{\iota }\right)=g\left(e\right),$$

and the equality is achieved at least for $\kappa =\left(\underset{e}{\underbrace{0,0,...,0,1}},0,...,0\right)$. □

As a result, it holds that

$$\begin{array}{c}{\Psi }_{t+1}^{\iota }=〈{\mathbf{R}}_t^{\iota },{\mu }_t^{\iota }〉={\displaystyle \sum _{{\theta }_t^{\iota }\in {\Theta }_t^{\iota }}}{\mathbf{R}}_t^{\iota }{\mu }_t^{\iota }\le \\ \\ {\displaystyle \sum _{{\theta }_t^{\iota }\in {\Theta }_t^{\iota }}}{\displaystyle \underset{{\pi }^{\iota }\left(y_t^{\iota }\right|m_t^{\iota }){\sigma }^{\iota }\left(m_t^{\iota }\right|{\theta }_t^{\iota })}{min}}{\mathbf{R}}_t^{\iota }{\mu }_t^{\iota }={\displaystyle \sum _{{\theta }_t^{\iota }\in {\Theta }_t^{\iota }}}{\mu }_t^{\iota }{\displaystyle \underset{y_t^{\iota }\in Y_t^{\iota }}{min}}\left[{\displaystyle \sum _{{\theta }_{t+1}^{\iota }\in {\Theta }_{t+1}^{\iota }}}{\phi }^{\iota }(y_t^{\iota },{\theta }_t^{\iota })F^{\iota }\left({\theta }_{t+1}^{\iota }\right|{\theta }_t^{\iota },y_t^{\iota })\right].\end{array}$$

At this point, let us introduce the following general definition of a Lyapunov-like function

Given fixed history of the process $\left({\mu }_0^{\iota },{\pi }^{\iota }\left(y_0^{\iota }\right|m_0^{\iota }),{\sigma }^{\iota }\left(m_0^{\iota }\right|{\theta }_0^{\iota }),...,{\pi }^{\iota }\left(y_t^{\iota }\right|m_t^{\iota }),{\sigma }^{\iota }\left(m_t^{\iota }\right|{\theta }_t^{\iota })\right)$

$$\underset{{\pi }^{\iota }\left(y_t^{\iota }\right|m_t^{\iota }){\sigma }^{\iota }\left(m_t^{\iota }\right|{\theta }_t^{\iota })}{min}{\mathbb{V}}_{t+1}^{\iota }=\sum _{{\theta }_t^{\iota }\in {\Theta }_t^{\iota }}{\left({\mu }_0^{\iota }\right)}_{{\theta }_t^{\iota }}\left[\sum _{{\theta }_{t+1}^{\iota }\in {\Theta }_{t+1}^{\iota }}{\phi }^{\iota }(y_t^{\iota \ast },{\theta }_t^{\iota })F^{\iota }\left({\theta }_{t+1}^{\iota }\right|{\theta }_t^{\iota },y_t^{\iota \ast })\right].$$

(10)

the identity in (10) is achieved for the pure and stationary local-optimal policy.

$${\pi }^{\iota \ast }\left(y_t^{\iota \ast }\right|m_t^{\iota \ast }){\sigma }^{\iota \ast }\left(m_t^{\iota }\right|{\theta }_t^{\iota })={\kappa }_{y_t^{\iota \ast }\left({\theta }_t^{\iota \ast }\right),{\theta }_t^{\iota \ast }}t=0,1,...$$

(11)

where ${\kappa }_{y_t^{\iota \ast }\left(m_t^{\iota \ast }\right),m_t^{\iota \ast }}$ is the Kronecker symbol and $y_t^{\iota \ast }\left(m_t^{\iota \ast }\right)$ is an index for which

$${\displaystyle \sum _{{\theta }_{t+1}^{\iota }\in {\Theta }_{t+1}^{\iota }}}{\phi }^{\iota }(y_t^{\iota \ast },{\theta }_t^{\iota })F^{\iota }\left({\theta }_{t+1}^{\iota }\right|{\theta }_t^{\iota },y_t^{\iota \ast })\le {\displaystyle \sum _{{\theta }_{t+1}^{\iota }\in {\Theta }_{t+1}^{\iota }}}{\phi }^{\iota }(y_t^{\iota },{\theta }_t^{\iota })F^{\iota }\left({\theta }_{t+1}^{\iota }\right|{\theta }_t^{\iota },y_t^{\iota }):=r^{\iota }(y_t^{\iota },{\theta }_t^{\iota }),\forall y_t^{\iota }\in Y_t^{\iota }$$

(12)

As a result, we can state the following lemma.

Lemma 2.

Given a fixed local-optimal policy, the $\mathbb{V}$-values for all state–action pairs from (8) in the recursive matrix format become

$${\mathbb{V}}_{t+1}^l=〈{\mathbf{R}}_t^{\iota \ast },{\mu }_t^{\iota \ast }〉$$

(13)

where

$${\mathbf{R}}_t^{\iota \ast }:={\displaystyle \sum _{{\theta }_{t+1}^{\iota }\in {\Theta }_{t+1}^{\iota }}}{\phi }^{\iota }(y_t^{\iota \ast },{\theta }_t^{\iota })F^{\iota }\left({\theta }_{t+1}^{\iota }\right|{\theta }_t^{\iota },y_t^{\iota \ast })={\displaystyle \underset{y_t^{\iota }\in Y_t^{\iota }}{min}}{r}^{\iota }(y_t^{\iota },{\theta }_t^{\iota }),$$

$$r^{\iota }(y_t^{\iota },{\theta }_t^{\iota })={\displaystyle \sum _{{\theta }_{t+1}^{\iota }\in {\Theta }_{t+1}^{\iota }}}{\phi }^{\iota }(y_t^{\iota },{\theta }_t^{\iota })F^{\iota }\left({\theta }_{t+1}^{\iota }\right|{\theta }_t^{\iota },y_t^{\iota }).$$

Remark 5.

Under the local-optimal strategy (11), the probability state vector ${\mu }_t^{\iota }$ satisfies the following relation

$${\mu }_{t+1}^{\iota }={\left({\mathbf{F}}_t^{\iota }\right)}_t^{\top }{\mu }_t^{\iota }={\left({\left({\mathbf{F}}^{\iota \ast }\right)}^{\top }\right)}^{t+1}{\mu }_0^{\iota },$$

where

$$\left({\mathbf{F}}^{\iota \ast }\right):=\sum _{y_t^{\iota }\in Y_t^{\iota }}{F}^{\iota }\left({\theta }_{t+1}^{\iota }\right|{\theta }_t^{\iota },y_t^{\iota }){\kappa }_{y_t^{\iota \ast }\left({\theta }_t^{\iota }\right),{\theta }_t^{\iota }}=F^{\iota }\left({\theta }_{t+1}^{\iota }\right|{\theta }_t^{\iota },y_t^{\iota \ast }).$$

Lemma 3.

The function ${\mathbb{V}}_t^{\iota }$ is a Lyapunov-like function (monotonically decreasing in time) that reaches an equilibrium point.

Proof. 

The function ${\mathbb{V}}_t^{\iota }$ satisfies the following properties:

• According to Remark 2, ${\mathbb{V}}_t^{\iota }>0$.
• According to Equations (10) and (11), $\Delta {\mathbb{V}}_t^{\iota }={\mathbb{V}}_{t+1}^{\iota }-{\mathbb{V}}_t^{\iota }<0$.
• ${\mathbb{V}}_t^{\iota }=0$ at the Lyapunov equilibrium point. It results straightforwardly from the following:

  $$\begin{array}{c}{\displaystyle \sum _{\iota \in \mathcal{N}}}{\displaystyle \underset{{\pi }^{\iota }\left(y_t^{\iota }\right|m_t^{\iota }){\sigma }^{\iota }\left(m_t^{\iota }\right|{\theta }_t^{\iota })}{min}}{\mathbb{V}}_{t+1}^{\iota }={\displaystyle \sum _{\iota \in \mathcal{N}}}{\displaystyle \sum _{{\theta }_t^{\iota }\in {\Theta }_t^{\iota }}}{\left({\mu }_0^{\iota }\right)}_{{\theta }_t^{\iota }}\left[{\displaystyle \sum _{{\theta }_{t+1}^{\iota }\in {\Theta }_{t+1}^{\iota }}}{\phi }^{\iota }(y_t^{\iota \ast },{\theta }_t^{\iota })F^{\iota }\left({\theta }_{t+1}^{\iota }\right|{\theta }_t^{\iota },y_t^{\iota \ast })\right]=\\ {\displaystyle \sum _{\iota \in \mathcal{N}}}{\displaystyle \sum _{t\in T}}{\displaystyle \sum _{{\theta }_t^{\iota }\in {\Theta }_t^{\iota }}}{\displaystyle \sum _{m_t^{\iota }\in M_t^{\iota }}}{\displaystyle \sum _{y_t^{\iota }\in Y_t^{\iota }}}{r}^{\iota }(y_t^{\iota },{\theta }_t^{\iota }){\pi }^{\iota \ast }\left(y_t^{\iota }\right|m_t^{\iota }){\sigma }^{\iota \ast }\left(m_t^{\iota }\right|{\theta }_t^{\iota }){\mu }^{\iota \ast }\left({\theta }_t^{\iota }\right)\times \\ {\displaystyle \prod _{i\ne \iota \in \mathcal{N}}}{\pi }^{i\ast }\left(x_t^i\right|m_t^i){\sigma }^{i\ast }\left(m_t^{i\ast }\right|{\theta }_t^{i\ast }){\mu }^{i\ast }\left({\theta }_t^{i\ast }\right)=\\ {\displaystyle \sum _{\iota \in \mathcal{N}}}\left({\displaystyle \sum _{t\in T}}{\displaystyle \underset{{\pi }^{\iota }\left(y_t^{\iota }\right|m_t^{\iota }){\sigma }^{\iota }\left(m_t^{\iota }\right|{\theta }_t^{\iota })}{min}}{\displaystyle \sum _{m_t^{\iota }\in M_t^{\iota }}}{\displaystyle \sum _{y_t^{\iota }\in Y_t^{\iota }}}{\pi }^{\iota }\left(y_t^{\iota }\right|m_t^{\iota }){\displaystyle \sum _{{\theta }_t^{\iota }\in {\Theta }_t^{\iota }}}{r}^{\iota }(y_t^{\iota },{\theta }_t^{\iota }){\sigma }^{\iota }\left(m_t^{\iota }\right|{\theta }_t^{\iota }){\mu }^{\iota }\left({\theta }_t^{\iota }\right)\times \right.\\ \left.{\displaystyle \prod _{i\ne \iota \in \mathcal{N}}}{\pi }^{i\ast }\left(x_t^i\right|m_t^i){\sigma }^{i\ast }\left(m_t^{i\ast }\right|{\theta }_t^{i\ast }){\mu }^{i\ast }\left({\theta }_t^{i\ast }\right)\right)<\\ {\displaystyle \sum _{\iota \in \mathcal{N}}}\left({\displaystyle \sum _{t\in T}}{\displaystyle \sum _{m_t^{\iota }\in M_t^{\iota }}}{\displaystyle \sum _{y_t^{\iota }\in Y_t^{\iota }}}{\pi }^{\iota }\left(y_t^{\iota }\right|m_t^{\iota }){\displaystyle \sum _{{\theta }_t^{\iota }\in {\Theta }_t^{\iota }}}{r}^{\iota }(y_t^{\iota },{\theta }_t^{\iota }){\sigma }^{\iota }\left(m_t^{\iota }\right|{\theta }_t^{\iota }){\mu }^{\iota }\left({\theta }_t^{\iota }\right)\times \right.\\ \left.{\displaystyle \prod _{i\ne \iota \in \mathcal{N}}}{\pi }^{i\ast }\left(x_t^i\right|m_t^i){\sigma }^{i\ast }\left(m_t^{i\ast }\right|{\theta }_t^{i\ast }){\mu }^{i\ast }\left({\theta }_t^{i\ast }\right)\right)=\\ {\displaystyle \sum _{\iota \in \mathcal{N}}}{\Phi }_t^{\iota }({\pi }^{\iota },{\pi }^{-\iota \ast },{\sigma }^{\iota },{\sigma }^{-\iota \ast })\end{array}$$

  Hence,

  $${\displaystyle \sum _{\iota \in \mathcal{N}}}\left({\Phi }_t^{\iota }({\pi }^{\ast },{\sigma }^{\ast })-{\Phi }_t^{\iota }({\pi }^{\iota },{\pi }^{-\iota \ast },{\sigma }^{\iota },{\sigma }^{-\iota \ast })\right)<0.$$

  (14)

  Given that the inequality in Equation (14) is valid for all admissible strategies $\pi$ and $\sigma$, it is valid when ${\pi }^i={\pi }^{i\ast }$ and ${\sigma }^i={\sigma }^{i\ast }$ for $i\ne \iota$, considering

  $${\Phi }_t^{\iota }({\pi }^{\ast },{\sigma }_{\pi }^{\ast })-{\Phi }_t^{\iota }({\pi }^{\ast },{\sigma }^i,{\sigma }^{-\iota \ast })<0.$$

  (15)

  Then, Equation (1) holds at $({\pi }^{\ast },{\sigma }_{\pi }^{\ast })$, and then ${\mathbb{V}}_t^{\iota }=0$.

□

4. The Three-Level Manipulation Model

The manipulation model is a game in which the manipulating players move first and then the manipulated players move sequentially [27].

4.1. The Stackelberg Game

To simplify the descriptions below, let us introduce the new variables

$$\left.\begin{array}{c}\alpha :=col\left[{\xi }^l\right],A_{adm}:={\Xi }^l,\left(l=\overline{1,n}\right).\\ \\ \beta :=col\left[{\xi }^f\right],B_{adm}:={\Xi }^f\left(f=\overline{1,m}\right).\end{array}\right\}$$

(16)

where $col\left[·\right]$ transforms a matrix in a vector format.

Consider a game involving manipulating players, where each manipulator’s strategy is denoted by ${\alpha }^l\in A^l$ and ${\alpha }^{-l}\in A^{-l}$ represents the strategies of the other manipulating players, $\alpha =({\alpha }^l,{\alpha }^{-l})$. Similarly, manipulated players have strategies ${\beta }^f\in B_{adm}^f$ and ${\beta }^{-f}\in B_{adm}^{-f}$ denotes the strategies of the other manipulated players, $\beta =({\beta }^f,{\beta }^{-f})$. The manipulating players anticipate the reactions of the manipulated players, choosing their strategies $\alpha \in A_{adm}$ before the manipulated players respond with their best-reply strategies $\beta \in B_{adm}.$

-

The individual aim for the manipulating players is to find ${\alpha }^{l\ast }\in A_{adm}^l$ for fixed $\beta ({\alpha }^{l\ast },{\alpha }^{-l})\in B_{adm}^f$ such that

$$v^l({\alpha }^{l\ast },{\alpha }^{-l}|\beta ({\alpha }^{l\ast },{\alpha }^{-l})):={\tilde{\Phi }}^l({\alpha }^{l\ast },{\alpha }^{-l}|\beta ({\alpha }^{l\ast },{\alpha }^{-l}))-{\tilde{\Phi }}^l({\alpha }^l,{\alpha }^{-l}|\beta ({\alpha }^{l\ast },{\alpha }^{-l}))\le \epsilon ,\epsilon \ge 0,$$

(17)

valid for all ${\alpha }^l\in A_{adm}^l$, ${\alpha }^{-l}\in A_{adm}^{-l}$ and all $l=\overline{1,n}$. Below ${\alpha }^{l\ast }$ will be associated with the $\epsilon$-Nash equilibrium in a Stackelberg–Nash game.

The individual objectives of the manipulating players can be expressed in a joint format by utilizing the regularized Tanaka function, as demonstrated below:

$$v_{\delta }({\alpha }^{l\ast },{\alpha }^{-l}|\lambda ,\beta ({\alpha }^{l\ast },{\alpha }^{-l})):={\displaystyle \sum _{l=1}^n}{v}_{\delta }^l\left({\alpha }^{l\ast },{\alpha }^{-l}|{\lambda }^{l,i},{\lambda }^{f,j},\beta ({\alpha }^{l\ast },{\alpha }^{-l})\right)\le \epsilon$$

(18)

for all $\alpha \in A$, $\lambda ={[{\lambda }^{j,i},{\lambda }^{f,j}]}_{l=\overline{1,n},f=\overline{1,m},i=\overline{1,N},j=\overline{1,N}}$ and

$$v_{\delta }^l\left({\alpha }^{l\ast },{\alpha }^{-l}|{\lambda }^{l,i},{\lambda }^{f,j},\beta ({\alpha }^{l\ast },{\alpha }^{-l})\right)={\mathbb{L}}_{\delta }^l\left({\alpha }^{\ast }\right|{\lambda }^{l,i},{\lambda }^{f,j},\beta ({\alpha }^{l\ast },{\alpha }^{-l}))-{\mathbb{L}}_{\delta }^l\left(\alpha \right|{\lambda }^{l,i},{\lambda }^{f,j},\beta ({\alpha }^l,{\alpha }^{-l}))$$

(19)

where ${\mathbb{L}}_{\delta }^l\left(\alpha \right|{\lambda }^{l,i},{\lambda }^{f,j},\beta ({\alpha }^l,{\alpha }^{-l}))$ corresponds with the Lagrange function.

The function $v_{\delta }({\alpha }^{l\ast },{\alpha }^{-l}|\lambda ,\beta ({\alpha }^{l\ast },{\alpha }^{-l}))$ is known as the regularized Tanaka function within the context of Stackelberg–Nash games. When both $\delta =0$ and $\lambda =0$, it corresponds to the original Tanaka function as outlined in [28].

-

The individual aim for the manipulated players is to find ${\beta }^{f\ast }\in B_{adm}^f$ for a fixed $\alpha ({\beta }^{f\ast },{\beta }^{-f})\in A_{adm}^l$ such that

$$u^f({\beta }^{f\ast },{\beta }^{-f}|\alpha ({\beta }^{f\ast },{\beta }^{-f})):=U^f({\beta }^{f\ast },{\beta }^{-f}|\alpha ({\beta }^{f\ast },{\beta }^{-f}))-U^f({\beta }^f,{\beta }^{-f}|\alpha ({\beta }^{f\ast },{\beta }^{-f}))\le \epsilon ,\epsilon \ge 0,$$

(20)

valid for all ${\beta }^f\in B_{adm}^f$, ${\beta }^{-f}\in B_{adm}^{-f}$ and all $f=\overline{1,m}$. Below ${\beta }^{f\ast }$ will be associated with the $\epsilon$-Nash equilibrium in a Stackelberg–Nash game.

The individual objectives of the manipulated players can be expressed jointly through the regularized Tanaka function, as outlined below:

$$u_{\delta }({\beta }^{f\ast },{\beta }^{-f}|\lambda ,\alpha ({\beta }^{f\ast },{\beta }^{-f})):={\displaystyle \sum _{f=1}^m}{u}_{\delta }^f\left({\beta }^{f\ast },{\beta }^{-f}|{\lambda }^{l,i},{\lambda }^{f,j},\alpha ({\beta }^{f\ast },{\beta }^{-f})\right)\le \epsilon$$

(21)

for all $\beta \in B^f$, $\lambda ={[{\lambda }^{j,i},{\lambda }^{f,j}]}_{l=\overline{1,n},f=\overline{1,m},i\in \left|{\Theta }_t\right|,j\in \left|{\Theta }_t\right|}$ and

$$u_{\delta }^f\left({\beta }^{f\ast },{\beta }^{-f}|{\lambda }^{l,i},{\lambda }^{f,j},\alpha ({\beta }^{f\ast },{\beta }^{-f})\right)={\mathbb{L}}_{\delta }^f\left({\beta }^{f\ast }\right|{\lambda }^{l,i},{\lambda }^{f,j},\alpha ({\beta }^{f\ast },{\beta }^{-f}))-{\mathbb{L}}_{\delta }^f\left({\beta }^f\right|{\lambda }^{l,i},{\lambda }^{f,j},\alpha ({\beta }^f,{\beta }^{-f}))$$

(22)

In this context, the equilibrium concept for games is defined as Nash equilibrium for simultaneous play scenarios and Stackelberg equilibrium for games with a hierarchical structure.

Definition 1.

(Stackelberg Equilibrium) In a game featuring l-leaders, a strategy ${\alpha }^{\ast }\in A_{adm}$ is considered a Stackelberg–Nash equilibrium strategy for the leaders if

$$\underset{\beta \in \rho \left({\alpha }^{\ast }\right)}{max}{v}_{\delta }\left({\alpha }^{\ast }|\beta \right)\le \underset{\beta \in \rho \left(\alpha \right)}{max}{v}_{\delta }\left(\alpha \xi |\beta \right)$$

such that

$$\zeta \left(\alpha \right)=\{\beta \in B_{adm}|u_{\delta }\left(\beta |\alpha \right)\le u_{\delta }\left({\beta }^{\prime }|\xi \right)\forall {\beta }^{\prime }\in B_{adm}\}$$

is the best-reply strategy set of the followers.

The previously defined Stackelberg equilibrium can be reformulated for the followers by replacing the set $\zeta \left(\alpha \right)$ with the set of Nash equilibria. In this context, if the leaders adopt the strategy $\alpha$; the optimal response of the followers will be a Nash equilibrium. In this scenario, the Nash equilibrium represents a state where, given the leaders’ strategy $\alpha$, the followers have optimized their own strategies. They cannot improve their outcomes by unilaterally changing their decisions, as doing so would lead to a less favorable result. This adjustment transforms the dynamics of the game, as the followers must consider not only their responses but also the broader implications of their strategies within the context of Nash equilibrium. To expand upon this, we can envision a multi-level decision-making environment where the leaders, acting as primary decision-makers, set the stage for the followers’ reactions. By defining their strategy $\alpha$, the leaders create a framework within which the followers must operate. The followers, recognizing the leaders’ strategic choices, analyze their own positions and responses. In essence, their responses are not isolated actions but part of a larger strategic equilibrium that includes the leaders’ moves.

Definition 2.

A strategy ${\alpha }^{\ast }\in A_{adm}$ of the manipulating player together with the collection ${\beta }^{\ast }\in B_{adm}$ of the manipulated players is said to be a Stackelberg–Nash equilibrium ($\epsilon =0$) if

$$\left({\alpha }^{\ast }{,\beta }^{\ast }\right)\in Arg\underset{\lambda \ge 0}{max}\underset{{\beta }^f,{\beta }^{-f}\in B_{adm}}{min}\underset{{\alpha }^{\prime }\in A_{adm}}{min}\left\{v_{\delta }({\alpha }^{\prime }|\beta )|u_{\delta }({\beta }^f,{\beta }^{-f}|\lambda ,\alpha ({\beta }^f,{\beta }^{-f}))\le 0\right\}.$$

(23)

Using the Lagrange multiplier approach, we can represent (23) as follows:

$${\mathbb{L}}_{\delta }({\alpha }^{\prime },{\beta }^f,{\beta }^{-f},\mu ,\lambda )\to \underset{\mu ,\lambda \ge 0}{max}\underset{{\beta }^f,{\beta }^{-f}\in B_{adm}}{min}\underset{{\alpha }^{\prime }\in A_{adm}}{min},$$

(24)

where

$${\mathbb{L}}_{\delta }({\alpha }^{\prime },{\beta }^f,{\beta }^{-f},\mu ,\lambda )=v_{\delta }({\alpha }^{\prime }|{\beta }^{\prime })+\mu u_{\delta }({\beta }^f,{\beta }^{-f}|\lambda ,\alpha ({\beta }^f,{\beta }^{-f})).$$

(25)

With $\delta >0$, the considered functions become strictly convex, providing the uniqueness of the considered conditional optimization problem (24). Notice also that the Lagrange function in (25) satisfies the saddle-point condition, namely, for all ${\alpha }^{\prime }\in A_{adm}$ and $\lambda \ge 0$ we have

$$\begin{array}{c}{\mathbb{L}}_{\delta }({\alpha }^{\prime \ast },{\beta }^{f\ast },{\beta }^{-f\ast },\mu ,\lambda )\le {\mathbb{L}}_{\delta }({\alpha }^{\prime \ast },{\beta }^{f\ast },{\beta }^{-f\ast },{\mu }^{\ast },{\lambda }^{\ast })\le {\mathbb{L}}_{\delta }({\alpha }^{\prime },{\beta }^f,{\beta }^{-f},{\mu }^{\ast },{\lambda }^{\ast }).\end{array}$$

(26)

4.2. Tri-Level Manipulation Game

The costs associated with different hierarchical roles are assumed to be distributed as follows: the outermost manipulators incur a cost $v_{\delta }$, the players being manipulated bear a cost $u_{\delta }$, and the innermost manipulators are assigned a cost $w_{\delta }$. These costs are thought to resemble Lyapunov-like functions.

Tri-level game problems have the following general form

$$\begin{array}{c}{\displaystyle {\displaystyle \underset{\alpha \in A_{adm}}{min}}}{v}_{\delta }\left(\alpha |\beta ,\gamma \right)\hfill \\ s.t\hfill \\ \begin{array}{c}\varphi (\alpha ,\beta ,\gamma )\le 0,\beta ,\gamma \in \zeta \left(\alpha \right)\hfill \\ {\displaystyle {\displaystyle \underset{\beta \in B_{adm}}{min}}}{u}_{\delta }\left(\beta \right|\alpha ,\gamma )\hfill \\ \begin{array}{c}s.t\hfill \\ \multicolumn{1}{c}{\varphi (\alpha ,\beta ,\gamma )\le 0,\alpha ,\gamma \in \zeta \left(\beta \right)}\\ {\displaystyle \underset{\gamma \in A_{adm}}{min}}{w}_{\delta }\left(\gamma \right|\alpha ,\beta )\hfill \\ s.t\hfill \\ \varphi (\alpha ,\beta ,\gamma )\le 0,\alpha ,\beta \in \zeta \left(\gamma \right)\hfill \end{array}\hfill \end{array}\hfill \end{array}$$

where $\zeta \left(\alpha \right),$$\zeta \left(\beta \right)$, and $\zeta \left(\gamma \right)$ are the solution set of the corresponding problem. The complement variables are fixed; for instance, for computing ${\displaystyle \underset{\alpha \in A_{adm}}{min}}{v}_{\delta }\left(\alpha |\beta ,\gamma \right)$ the variables $\beta$ and $\gamma$ are fixed.

At the initial stage of this framework, the manipulating players choose a manipulation strategy $\alpha \in A_{adm}$. In response, at the second stage, the manipulated players react by selecting a manipulation strategy $\beta \in B_{adm}$. In the final stage, the manipulating players, after considering both the manipulation strategy $\beta$ and the manipulation strategy $\alpha$, steer the system by selecting a strategy $\gamma \in A_{adm}$ that persuades the manipulated players.

At the highest level, the strategy $\alpha \in A_{adm}$ represents the approach taken by the manipulating players who initiate the game by choosing a manipulation tactic $\alpha$. This strategy may involve actively influencing the manipulated players. By controlling $\alpha \in A_{adm}$, the manipulating players aim to allocate resources in a way that builds an infrastructure system as resilient to manipulation as possible. The set $A_{adm}$ often encompasses one or more critical resource limitations that govern these actions.

The strategy $\beta \in B_{adm}$ represents the actions of the manipulated players at the middle level of the hierarchical structure. These players observe the manipulative tactics $\alpha \in A_{adm}$ and $\gamma \in A_{adm}$. Their goal is to resist the manipulation from the manipulating players to the greatest extent possible. Best-reply strategies at the middle level function as a defensive mechanism, effectively countering manipulation attempts from higher or lower levels. This counterbalancing dynamic enhances the system’s resilience by neutralizing external influences. As a result, it improves strategic robustness, ensuring stability and adaptability in complex, multi-level decision-making environments.

The strategy of the manipulating players, acting as operators, is reflected at the deepest level through the minimization of $w_{\delta }\left(\gamma \right|\alpha ,\beta )$, where they select $\gamma \in A_{adm}$ to reduce the system’s operational costs. It is important to note that the strategy $\gamma$ can be influenced by both the manipulation strategy $\alpha$ and the response strategy $\beta$, which are determined according to $u_{\delta }\left(\beta \right|\alpha ,\gamma )$. While the manipulating players may represent different real entities, they all pursue the same manipulation objectives.

Continuous refinement by lower-level players ensures their strategies remain aligned with those of higher-level players. This adaptability is vital for maintaining effectiveness, as it allows the system to respond to changing conditions and evolving strategies within the hierarchy, ensuring that decisions remain optimal and responsive to dynamic environments.

Decision-making at each level generates feedback loops that significantly influence the entire game’s dynamics. These loops create a continual exchange of information and adjustments between levels, ensuring that the system evolves toward stable outcomes. This interconnected process enhances the system’s predictability, allowing for more reliable forecasts and better strategic alignment over time.

The main point of this solution lies in its asymptotic stability, which means that even if players start with different strategies, as the game evolves, they will naturally gravitate toward a stable equilibrium, without needing to regret any past decisions. No matter how the manipulative players attempt to influence the middle level, or how the lower level responds, the structure of the game ensures that, in the long run, stability will prevail.

This concept of stability also ensures that players can adopt no-regret strategies, meaning they will not have to second-guess their moves, because over time, their actions will converge to a point that best suits the entire dynamic.

5. Numerical Example: Oil Market and Regulatory Influence

This example considers controllable Bayesian–Markov chains with Lyapunov-like cost functions to model manipulation in a hierarchical oil market system vs. the regulatory influence. The decision-making framework involves players that manipulate each other strategically in a three-level Stackelberg framework.

Consider a oil market system as follows. There are three type of players: (a) outermost manipulating players: a lobbying group representing oil companies, aiming to influence regulatory priorities; (b) middle-level manipulated players: national regulators, balancing energy production, environmental sustainability, and economic growth; (c) innermost manipulating players: oil producers, who adjust production and investment strategies based on regulatory policies that persuade the manipulated players.

Consider a market that evolves over discrete types $\{{\theta }_1,{\theta }_2,{\theta }_3,{\theta }_4\}$$\in {\Theta }_{adm}$, representing oil and environmental conditions: ${\theta }_1$—high demand, low environmental degradation; ${\theta }_2$—moderate demand, moderate degradation; ${\theta }_3$—steady demand, unaltered degradation; and ${\theta }_4$—low demand, high degradation.

Each player can take three possible actions $\{y_1,y_2,y_3\}\in {\mathcal{Y}}_{adm}$. The transition probabilities $F^{\iota }\left({\theta }_{t+1}\right|{\theta }_t,y_t)$ depend on the current type ${\theta }_t$ and actions $y_t$ by players at all levels: $y_1$—lobbying actions (e.g., media campaigns and policy framing); $y_2$—regulatory decisions (e.g., subsidies and emissions limits); and $y_3$—oil producers’ production and investment strategies.

The outermost manipulators that represent the lobbying group incur a cost $v_{\delta }$, which is the cost of lobbying activities, the players being manipulated, regulators, bear a cost $u_{\delta }$, which optimize policies based on their perception, penalizing deviations from environmental and energy demand equilibrium, and the innermost manipulators, energy producers, are assigned a cost $w_{\delta }$, which adjust their strategies to minimize profit fluctuations and operational costs. These costs are thought to resemble Lyapunov-like functions

$${\displaystyle \underset{{\pi }^{\iota }\left(y_t^{\iota }\right|m_t^{\iota }){\sigma }^{\iota }\left(m_t^{\iota }\right|{\theta }_t^{\iota })}{min}}{\mathbb{V}}_{t+1}^{\iota }={\displaystyle \sum _{{\theta }_t^{\iota }\in {\Theta }_t^{\iota }}}{\left({\mu }_0^{\iota }\right)}_{{\theta }_t^{\iota }}\left[{\displaystyle \sum _{{\theta }_{t+1}^{\iota }\in {\Theta }_{t+1}^{\iota }}}{\phi }^{\iota }(y_t^{\iota \ast },{\theta }_t^{\iota })F^{\iota }\left({\theta }_{t+1}^{\iota }\right|{\theta }_t^{\iota },y_t^{\iota \ast })\right].$$

Each level minimizes a cost function that encourages stability.

The strategic manipulation will be as follows:

• The outermost manipulating players manipulate transition probabilities $F^l({\theta }_{t+1}^l|{\theta }_t^l,y_t^l)$ into $F^l({\theta }_{t+1}^l|{\theta }_t^l,y_t^{l\ast })$, (i) sponsoring biased reports exaggerating oil demand and (ii) framing oil fuels as critical for economic growth. Their goal is to influence manipulated players (regulators’) cost function $u_{\delta }$.
• Manipulated players in the middle-level best reply to manipulated transition probabilities $F^l({\theta }_{t+1}^{\iota }|{\theta }_t^{\iota },y_t^{\iota \ast })$ by implementing policies by applying $min$${\pi }^f(y_t^{f\ast }|m_t^f){\sigma }^f(m_t^f|{\theta }_t^f)$ to $F^f({\theta }_{t+1}^f|{\theta }_t^f,y_t^{f\ast })$, such as (i) subsidies for oil fuel production and (ii) relaxed penalties for environmental violations. Their objective is to influence manipulating players’ cost functions $v_{\delta } w_{\delta }$.
• The innermost manipulating players again improve their strategies $F^l({\theta }_{t+1}^l|{\theta }_t^l,y_t^{l\ast })$ based on $F^f({\theta }_{t+1}^f|{\theta }_t^f,y_t^{f\ast })$, leading to the following: (i) increased production exploiting subsidies and (ii) reduced investment in renewable energy.

Considering

$$\begin{array}{cc}{F}^{\iota }(:|:,1)=\left[\begin{array}{cccc}0.8181& 0.6596& 0.8003& 0.0835\\ 0.8175& 0.5186& 0.4538& 0.1332\\ 0.7224& 0.9730& 0.4324& 0.1734\\ 0.1499& 0.6490& 0.8253& 0.3909\end{array}\right]& F^{\iota }(:|:,2)=\left[\begin{array}{cccc}0.8314& 0.5269& 0.2920& 0.1672\\ 0.8034& 0.4168& 0.4317& 0.1062\\ 0.0605& 0.6569& 0.0155& 0.3724\\ 0.3993& 0.6280& 0.9841& 0.1981\end{array}\right]\end{array}$$

$$\begin{array}{c}{F}^{\iota }(:|:,3)=\left[\begin{array}{cccc}0.4897& 0.0527& 0.5479& 0.3015\\ 0.3395& 0.7379& 0.9427& 0.7011\\ 0.9516& 0.2691& 0.4177& 0.6663\\ 0.9203& 0.4228& 0.9831& 0.5391\end{array}\right]\end{array}$$

and random cost matrices, the resulting optimal strategies are shown in Figure 1 of the oil market Stackelberg game.

$$\begin{array}{c}{\pi }^{\iota }\left(y_t^{\iota \ast }\right|m_t^{\iota })=\left[\begin{array}{ccc}0& 0.& 1\\ 0& 1& 0\\ 0& 0& 1\\ 0& 1& 0\end{array}\right]\end{array}$$

The manipulation occurs via transforming $F^{\iota }\left({\theta }_{t+1}^{\iota }\right|{\theta }_t^{\iota },y_t^{\iota })$ into $F^{\iota }\left({\theta }_{t+1}^{\iota }\right|{\theta }_t^{\iota },y_t^{\iota \ast })$, misleading regulators and energy producers into suboptimal but predictable decisions:

• Outermost manipulating players: $F^l\left({\theta }_{t+1}^l\right|{\theta }_t^l,y_t^{l\ast })$ maximizes influence and profit by ensuring system inertia favors oil fuels.
• Middle-level manipulated players: $F^f\left({\theta }_{t+1}^f\right|{\theta }_t^f,y_t^{f\ast })$ implements suboptimal policies based on manipulated dynamics.
• Innermost manipulating players: $F^l\left({\theta }_{t+1}^l\right|{\theta }_t^l,y_t^{l\ast })$ exploits short-term gains, worsening long-term environmental and economic sustainability.

6. Conclusions

We proposed a solution to the manipulation game problem by applying Lyapunov theory, demonstrating both the existence and stability of Nash equilibria within the system. Each equilibrium that is asymptotically stable is associated with a Lyapunov-like function, which converges towards a Nash/Lyapunov equilibrium when such an equilibrium is present. This convergence guarantees the system’s robustness, as the asymptotic properties of the function promote no-regret strategies, ensuring players have little incentive to deviate from their chosen course of action in the long run.

Adaptation is crucial for maintaining a strategic advantage in a dynamic environment. The ability to adjust based on observed responses allows the system to stay aligned with its evolving objectives. This ongoing adjustment process ensures that strategies remain relevant, effective, and responsive to changing conditions, fostering long-term success and competitiveness.

The manipulation game is modeled as a hierarchical, three-level Stackelberg framework. At the top level, the manipulating players select a strategy aimed at influencing the middle-level participants. However, the strategy chosen by the manipulators may not always succeed in persuading those on the middle tier. In response, the middle-level players “who are being manipulated” adopt a best-reply strategy that can effectively neutralize or match the manipulation, creating a counterbalancing dynamic. This interaction embodies the core of strategic manipulation, where each player’s decisions are interdependent and reactions must be carefully calibrated.

To ensure the success of the manipulation, the player operating at the lowest level of the hierarchy must refine and adjust their strategy in response to the behavior of the manipulated players above them. This bottom-up adjustment is crucial to aligning with the overall manipulation scheme, as it allows the lower-level manipulator to account for and anticipate the reactions of the middle-tier participants. In essence, the success of the manipulation depends not just on the initial strategy but also on continuous adaptation based on observed responses at higher levels of the hierarchy.

This structured approach underscores the complexity of manipulation in multi-tiered strategic games. Each layer of decision-making “whether it is at the top, middle, or lower level” creates feedback loops that influence the broader dynamics of the game. By using Lyapunov theory to map out stable equilibria, we provide a framework that ensures the system gravitates towards a stable and predictable outcome, even in the face of manipulation. The incorporation of no-regret strategies further reinforces the stability and long-term viability of these equilibria, suggesting that the players’ strategic choices will converge to an optimal solution over time. This comprehensive solution offers valuable insights into both the theoretical and practical applications of manipulation in game theory, with implications for economics, cybersecurity, and political strategy.