Novel Dynamic Defense Strategies in Networked Control Systems under Stochastic Jamming Attacks

Abstract

In contemporary networked control systems (NCSs), ensuring robust and adaptive security measures against dynamic threats like jamming attacks is crucial. These attacks can disrupt the control signals, leading to degraded performance or even catastrophic failures. This paper introduces a novel approach to enhance NCS security by applying stochastic game theory to model and resolve interactions between a defender and a jammer. We develop a two-player zero-sum game where the defender employs mixed strategies to minimize the expected cost of maintaining system stability and control effectiveness in the face of potential jamming. Our model discretizes the state space and employs backward induction to dynamically update the value functions associated with various system states, reflecting the ongoing adjustment of strategies in response to the adversary’s actions. Utilizing linear programming in MATLAB, we optimize the defender’s mixed strategies to systematically mitigate the impact of jamming. The results from extensive simulations demonstrate the efficacy of our proposed strategies in attack scenarios, indicating a substantial enhancement in the resilience and performance of NCSs against jamming attacks. Specifically, the proposed method improved network state stability by $75\%$, reducing the fluctuation range by over $50\%$ compared with systems without defense mechanisms. This study not only advances the theoretical framework for security in NCSs but also provides practical insights for the design of resilient control systems under uncertainty.

1. Introduction

Networked control systems (NCSs) are pivotal in modern infrastructure, managing everything from transportation networks to power grids [1,2]. The integration of computing and communication technologies with control systems has significantly enhanced their efficiency and flexibility. However, this integration also exposes NCSs to various cyber threats, among which jamming attacks represent a severe risk. Jamming attacks deliberately disrupt communication channels, thereby degrading the performance of control systems or even causing total system failure [3]. The dynamic and unpredictable nature of these attacks requires equally adaptive and robust defense mechanisms. Traditional security measures often focus on static defenses or reactive protocols that might not suffice in the face of sophisticated, evolving threats. Thus, there is a pressing need for strategies that anticipate and mitigate potential attacks in real time. Stochastic game theory [4,5] provides a promising framework to address these challenges.

By modeling the interactions between defenders and attackers as a game with probabilistic outcomes, we can develop strategies that adapt to changes in the environment and the attacker’s behavior.

This paper applies stochastic game theory to the specific case of jamming in NCS, proposing a two-player zero-sum game where the defender aims to minimize a cost function that represents the risk and impact of potential disruptions.

Our approach involves discretizing the state space of the system, allowing for a detailed analysis of possible states and transitions. Backward induction is used to iteratively calculate value functions, representing the minimum expected future costs from any given state, under the assumption of optimal play by both the defender and the attacker. This dynamic programming method [6,7] not only accommodates the stochastic nature of the threat but also provides a framework for updating defense strategies as the game progresses. The introduction of linear programming to optimize the defender’s mixed strategies further enhances the model’s practicality by providing a method to determine the optimal defensive actions quickly and efficiently. This integration of stochastic game theory with linear optimization offers new insights into developing proactive defense mechanisms in NCSs, aiming to keep the systems resilient against unpredictable adversarial behaviors [8].

In the literature, there are many algorithms to tackle the challenges of the security of NCSs. To better understand the strengths and weaknesses of various algorithms in the context of networked control system (NCS) security,

Table 1. Advantages and Disadvantages of Different Algorithms.

Algorithm Type	Advantages	Disadvantages
Static Defense Mechanism	Simplicity, low computational cost	Ineffectiveness against dynamic and adaptive attacks
Reactive Protocols	Quick response to detected threats	Delayed detection, high false positive rate
Stochastic Game theory	Adaptive to changing threats, strategic decision making	High computational complexity, requires accurate modeling
Linear Programming Optimization	Efficient calculation of optimal strategies, scalability	Sensitive to model inaccuracies, initial setup complexity
Machine Learning Approaches	Can learn from evolving threats, improved accuracy	Requires large datasets, high computational resources
Backward Induction	Provides optimal strategy updates, adapts to changing scenarios	Computationally intensive, requires precise state transition odeling

summarizes the advantages and disadvantages of static defenses, reactive protocols, stochastic game theory, linear programming optimization, machine learning approaches, and backward induction.

This article aims to propose a game-theoretical approach to address security issues in NCSs under intelligent jamming attacks. The major contributions of the proposed model are underlined as follows:

• This research introduces a refined model of stochastic game theory specifically adapted for handling security issues in NCSs under jamming attacks. It extends traditional game-theoretic approaches by incorporating dynamic and stochastic elements that more accurately reflect the complexities encountered in real-world scenarios.
• The use of backward induction to iteratively update value functions offers a robust framework for real-time decision making. This method allows for the strategic adjustments of defense based on evolving system states and attacker strategies, providing a continuous adaptation mechanism that is vital in dealing with dynamic threats.
• The application of linear programming to determine the defender’s mixed strategies is a practical contribution that enhances the computational feasibility of implementing game-theoretic solutions in real systems. This optimization technique allows for quick recalculations, which is essential for systems requiring immediate responses to security breaches.

2. Existing Literature

The security of networked control systems (NCSs) against cyber-physical attacks, particularly jamming, has garnered increasing attention due to the critical roles these systems play in modern infrastructure. This section reviews pertinent literature on the application of game theory to NCS security, the use of stochastic models in cybersecurity, and existing methods for real-time adaptive strategy development.

Early works, such as those by Zhu and Basar in [9], explored game theory as a tool to model the interactions between attackers and defenders in cyber-physical systems. They demonstrated how game theory could provide strategic insights into optimal defensive actions. However, many of these models assumed static game settings and did not account for the dynamic nature of attack vectors.

More recent studies have begun to incorporate dynamic game elements. For instance, Nguyen et al. in [10] extended zero-sum game models to include time-varying strategies and demonstrated their effectiveness in simulated environments. While these studies advanced the application of game theory in cybersecurity, they often neglected the stochastic variations in the state transitions and the real-time recalibration of strategies.

Stochastic models have been effectively used to describe uncertainties in cyber-attacks. The authors in [11] employed Markov decision processes (MDPs) to assess risk and devise countermeasures in real time. Their work provided foundational methodologies for understanding probabilistic attack impacts but did not directly address jamming in NCSs.

The adaptation of stochastic models specifically to jamming attacks can be seen in the work cited in [12], which used stochastic games to analyze the effectiveness of various jamming techniques against wireless communication systems. These contributions are crucial but often require further refinement to integrate real-time defense adaptations in NCSs.

The challenge of developing adaptive strategies that can respond in real time to evolving threats has led to innovative solutions, such as those proposed by Yu et al. in [13]. They introduced a learning-based algorithm that adjusts defensive tactics based on ongoing attack patterns, showcasing significant improvements in system resilience.

While the work of the authors in [14] considers that the attack signal depends on the results of the attack–defense game between the jammer and the transmitter, here, the state of the attack signal follows a random process to model the interaction between the players as close as possible to real-world scenarios.

Wang et al. [15] explored remote state estimation under DoS attacks in CPSs with arbitrary tree topology using a Bayesian Stackelberg game approach, highlighting the robustness of strategic defense mechanisms in complex network structures. Zhou et al., in [16], provided a comprehensive review of the cybersecurity landscape on remote state estimation, offering a broad perspective on the challenges and advancements in this field. The authors in [17] examined remote state estimation with a strategic sensor using a Stackelberg game framework, contributing to the understanding of strategic interactions in cybersecurity. In [18], Wu et al. presented a game-theoretic approach to remote state estimation in the presence of a DoS attacker, which emphasizes the relevance of game theory in designing resilient estimation strategies and finally, Yang et al., in [19], discussed cyber attacks on remote state estimation in cyber-physical systems using a game-theoretic approach, demonstrating the applicability of these models in real-world scenarios. In summary, while substantial research has explored various aspects of NCS security, the integration of game theory with stochastic modeling and linear programming for dynamic and real-time strategy optimization remains underexplored. Our work contributes to this niche by providing a comprehensive framework that not only anticipates potential attack scenarios but also adapts to them dynamically, ensuring sustained system performance and resilience.

3. Proposed Methodology

This section provides a comprehensive explanation of the mathematical frameworks and computational techniques used in our study to enhance the security of networked control systems against stochastic jamming attacks. The methodology is built around the formulation of a stochastic game model, the discretization of state spaces, the definition of cost functions reflecting system penalties under attack, and the employment of linear programming for optimizing the defender’s mixed strategies.

3.1. Modeling an NCS under Jamming Attacks

Networked control systems (NCSs) are particularly susceptible to physical attacks due to the way in which feedback measurements are transmitted [20]. In these systems, as shown in Figure 1, feedback data, crucial for the operation and control of physical processes, are often sent over wireless communication networks. This mode of transmission exposes the data to various security threats, including jamming attacks where a jammer sends noise signals that are greater than the transmitting signals of the transmitter, hence disrupting the system functionality and losing the feedback measurements [21]. Without these measurements, the NCS becomes an open-loop system, which leads to instability.

The dynamics of the NCS are modeled using a linear state transition equation influenced by potential jamming attacks, represented as stochastic disturbances in the control input. The state transition is described by

$$x_{k+1}=A{x}_k+{\gamma }_{k}BK{u}_k$$

(1)

where the following hold:

• $x_k$ is the state vector at time step k;
• A is the state transition matrix describing the system dynamics under normal operation;
• ${\gamma }_k$ is a binary variable that represents the presence (1) or absence (0) of a jamming attack at time step k;
• B is the control input matrix;
• K is the feedback gain matrix.

The matrix $A+{\gamma }_{k}BK$ thereby encapsulates the controlled system response, dynamically adjusting to reflect the impact of jamming on the control signals.

In this paper, we formulate a two-player zero-sum stochastic game for NCSs subjected to jamming attacks. The primary objective of our work is to design an optimal defense mechanism for the sensor transmitter, which operates within the constraints of limited network resources.

3.2. Formulation of the Zero-Sum Stochastic Game in NCSs

To this end, we formulate a zero-sum stochastic game [22] framework that captures the essence of this adversarial interaction, where the gains of one player are exactly balanced by the losses of the other. This section describes the foundation elements of the game’s formulation, including the game’s players, state space, actions, payoffs, and the stochastic nature of the game dynamics.

3.2.1. Game Players

The game involves two players with opposing objectives:

• Defender (D): The sensor transmitter, whose goal is to send feedback measurements to the plant.
• Attacker (A): The jammer, who aims to degrade the system’s performance by disrupting communications.

3.2.2. State Space

The state space of the game, denoted by S, comprises all possible conditions of the NCS that can be affected by both the defender’s actions and the presence of jamming. The state at any time k is represented as $x_k=[x_1^1,x_2^2,..,x_k^i..x_k^m]$, which represent system performance metrics constituting the state variables of the NCS and the operational status variable ${\gamma }_k$, which indicates the state of the wireless network. The set of ${\gamma }_k$ is $\{0,1\}$.

In dynamic programming, discretizing [23,24] the state space based on given ranges for state variables is a fundamental step. Dynamic programming algorithms typically require looping over all possible states and actions to compute the optimal policy. When state and action spaces are continuous, it becomes computationally infeasible to evaluate every possible state or action. Discretizing these spaces converts them into a finite set, making the computations manageable. Therefore, in this paper, each state variable $x_k^i$ is considered to be within the range $[\underline{l},\overline{l})$, and every range is divided into n number of discrete intervals, represented by discrete numbers.

3.2.3. Actions

Both players have a set of available actions depending on their objective. The defender can choose from a variety of control strategies or countermeasures designed to mitigate the impact of jamming and maintain the NCS’s stability. The attacker selects from strategies designed to maximize disruption and degrade the dynamic performance of the NCS.

In our case, the actions for D and A at time slot k are power levels to transmit signals denoted by $a_k^A$ and $a_k^D$. Each player has two actions, $a_k^A=\{NA,A\}$ and $a_k^D=\{ND,D\}$, which means either Attack (A), Defend (D), or do nothing (NA, ND) at any time slot k.

3.2.4. Transition Probabilities

The game’s dynamics are governed by stochastic elements, primarily the uncertainty in the attacker’s actions and the probabilistic effects of those actions on the state transitions. The transition probabilities are defined by the function $P\left(x_{k+1}\right|x_k,a^D,a^A)$. These probabilities encapsulate the likelihood of moving from one state to another, given the current state and the actions taken by both players. The Monte Carlo method [25,26] is very useful for estimating the probabilities of transitioning from one state to another in a stochastic game. Given the current state and the actions taken by both players, first ensure that the model correctly reflects the dynamics of the stochastic game, second, run multiple simulations of the game to estimate probabilities, and finally, refine the results by increasing the number of simulations.

The Monte Carlo method is very powerful, especially in game theory, where exact calculations may be cumbersome or require an impractical amount of computational resources.

3.2.5. Cost Function and Payoff

The payoffs in this zero-sum game are primarily defined through a cost function, $C(x_k,a^D,a^A)$ that quantifies the immediate cost to the defender based on the current state and the actions of both players. This cost function reflects not only the direct impact of jamming on system performance but also the operational costs associated with implementing various defensive strategies. For the attacker, the payoff is the negative of the defender’s cost, emphasizing the zero-sum nature of the game [27].

For each state and potential jamming scenario, the cost function is defined as

$$C(x_k,a^D,a^A)=c_1+c_{2}f\left(x_k\right)$$

(2)

$$C(x_k,a^D,a^A)=c_1+c_2\left(x_k^{T}Q{x}_k\right)$$

(3)

where the following hold:

• $c_1$ and $c_2$ are cost coefficients that weight the importance of different aspects of the state impact;
• $f\left(x_k\right)$ represents the quadratic state error that models the degraded dynamic performance of the NCS;
• Q is a weighting matrix that scales the quadratic cost associated with the state vector.

This cost function provides a basis for evaluating the efficacy of different defensive strategies in terms of minimizing expected costs arising from state transitions influenced by jamming.

3.3. Solving the Game

The strategic solution to this game involves finding a Nash equilibrium through minimax strategies [28]. The defender seeks to minimize the maximum possible expected loss, considering the worst-case actions of the attacker. Conversely, the attacker aims to maximize the minimum expected gain, exploiting the most vulnerable aspects of the system. Solving this game typically involves calculating value functions and employing strategies such as dynamic programming or linear programming to optimize the expected payoffs over a horizon. At each time step and for each state, a linear programming problem is formulated to minimize the expected cost, given the current state and the probability of attack scenarios [29].

In our analysis, the strategies deployed by both players are state-dependent and involve mixed strategies. Mixed strategies are critical in ensuring that each stage of the game reaches a saddle-point equilibrium, a concept crucial for stability in game theory.

Let $s\in S$ denote a specific state in the state space. Each player selects actions from the set of possible actions $\left\{a^D\right\}$ for the defender and $\left\{a^A\right\}$ for the attacker.

• The defender’s mixed strategy at state s, denoted as $p^D\left(s\right)$, is a probability distribution over the defender’s actions set. This strategy is represented by

  $$p^D\left(s\right):=[p_1^D\left(s\right),p_2^D\left(s\right)]$$

  (4)

  where $p_1^D\left(s\right)$ and $p_2^D\left(s\right)$ are the probabilities of choosing the first and second actions, respectively, at state s. These probabilities satisfy the following normalization condition:

  $$\sum _{i=1}^2{p}_i^D\left(s\right)=1.$$

  (5)
• Similarly, the attacker’s mixed strategy at state s, denoted as $q^A\left(s\right)$, is also a probability distribution over the attacker’s actions set. This strategy is represented by

  $$q^D\left(s\right):=[q_1^A\left(s\right),q_2^A\left(s\right)]$$

  (6)

  where $q{1}^A\left(s\right)$ and $q_2^A\left(s\right)$ are the probabilities of choosing the first and second actions, respectively, at state s. These probabilities must also adhere to the following normalization condition:

  $$\sum _{i=1}^2{q}_i^A\left(s\right)=1.$$

  (7)

Given the zero-sum nature of the game, the sum of outcomes for the defender and the attacker for any action pair at any state is zero. This means that the gains of one player are exactly offset by the losses of the other. Consequently, in zero-sum games, it suffices to consider only one set of game tables to understand the dynamics and outcomes, as the matrices for the opposing player can be directly inferred.

In this study, the defender aims to minimize his expected total cost, while the attacker wants to maximize his expected payoff. We give details on how to solve the game only from the defender’s point of view; thus, the defender game table has the structure presented in

Table 2. Defender’s Game Table.

	A	NA
D	$c_1+c_2{x}_k^{T}Q{x}_k$	$c_1+c_2{x}_k^{T}Q{x}_k$
ND	$-c_1+c_2{x}_k^{T}Q{x}_k$	0

.

In this paper, we are considering an N-stage stochastic game [30], where outcomes depend not only on the current actions of the players but also on the state of the system, which itself changes in response to these actions and random attack effects. The payoff for the defender at each stage depends on the current state, the actions chosen by both players and the resultant next state. The total payoff to the defender is typically the sum of the payoffs over all stages, discounted by a factor that accounts for the strategic preferences.

The one-stage value of the game is derived using the defender’s mixed strategy $p_D\left(s\right)$ and the game table in Table 2 as follows:

$$\begin{array}{c}\hfill c\left(s\right)=p_1^D\left(s\right)\ast q_1^A\left(s\right)\ast (c_1+c_2{x}_k^{T}Q{x}_k)+p_1^D\left(s\right)\ast q_2^A\left(s\right)\ast (c_1+c_2{x}_k^{T}Q{x}_k)\\ \hfill +p_2^D\left(s\right)\ast q_1^A\left(s\right)\ast (-c_1+c_2{x}_k^{T}Q{x}_k)+0.\end{array}$$

(8)

The expected total value of this game over all possible actions at time instant k, considering the one stage value $c^i\left(s\right)$, is

$$V_k\left(s\right)=E\left\{\sum _{i=k}^N\alpha c_i\left(s\right)\right\}$$

(9)

At any given stage k, we define the function $Q_k(a,d,s)$ represented by Equation (10) as the expected discounted cost when the attacker takes action a and the defender takes action d, when the current state is s.

The function $Q_k(a,d,s)$ represents a Bellman equation [31], which is frequently used in stochastic games. It involves determining the optimal policy by maximizing (or minimizing) the expected sum of current rewards (or costs) plus the discounted future rewards (or costs).

$$Q_k(a,d,s)=c_k(a,d,s)+\alpha \sum _{s^{\prime }\in S}p\left(s^{\prime }\right|s,a,d)V_{k+1}\left(s^{\prime }\right)$$

(10)

where $c_k(a,d,s)$ is the immediate reward at the kth stage, $V_{k+1}\left(s^{\prime }\right)$ is the total expected cost from the next state with states $s^{\prime }$, $p\left(s^{\prime }\right|s,a,d)$ are the transition probabilities described in Section 3.2.4, and $\alpha$ is the discount factor.

Since the defender’s objective is to minimize their expected total cost in opposition to the attacker’s objective, the minimax principle is used to determine optimal strategies for the defender; hence, the value of the stochastic game for the defender can be described as

$$V_k\left(s\right)=\underset{p^D\left(s\right)}{min}\underset{q^A\left(s\right)}{max}{q}^A{\left(s\right)}^T{Q}_k(a,d,s)p^D\left(s\right)$$

(11)

To solve the problem in Equation (11), we define a variable $z=ma{x}_i\left[Q_k(a,d,s)\right]$, and the problem comes to solve the linear programming problem described as follows:

$$\begin{array}{cc}\underset{p^D\left(s\right)}{min}z\hfill & \\ \mathrm{s}.\mathrm{t}.\hfill & {\left[Q_k(a,d,s)p^D\left(s\right)\right]}_i\le z,\hfill \\ \hfill & p^D\left(s\right)\ge \mathbf{0},\hfill \\ \hfill & {\mathbf{1}}^T{p}^D\left(s\right)=1.\hfill \end{array}$$

(12)

The linear programming problem for the attacker is as follows:

$$\begin{array}{cc}\underset{q^A\left(s\right)}{max}y\hfill & \\ \mathrm{s}.\mathrm{t}.\hfill & {\left[Q_k(a,d,s)q^A\left(s\right)\right]}_i\ge y,\hfill \\ \hfill & q^A\left(s\right)\ge \mathbf{0},\hfill \\ \hfill & {\mathbf{1}}^T{q}^A\left(s\right)=1.\hfill \end{array}$$

(13)

where $y=mi{n}_i\left[Q_k(a,d,s)\right]$

The following lines resume all the steps to model and solve the zero-sum game for an attack–defense approach in NCSs in order to find optimal defense strategies.

• Define the state space: Discretize the state space based on the given ranges for the state variables;
• Define the action space: Define actions for the defender and the jammer;
• Define the system dynamics: Use the provided state space model to compute the transition probabilities;
• Define the cost function: Use the provided cost function to calculate the immediate costs;
• Perform value function iteration: iterate to convergence to find the value function;
• Use linear programming: Formulate the LP problem to find the optimal policy for the Defender/Attacker;
• Solve the LP problem: Using MATLAB’s optimization toolbox to solve the LP problem [32].

4. Simulations and Discussion

This section provides a practical example to show the effectiveness of the proposed optimal scheme and evaluate the dynamic performance of the NCS under jamming attacks. We consider a linear system for a robotic arm as the plant of the NCS.

The system parameters and configurations for this NCS are presented as follows:

$$A=\left[\begin{array}{cc}-0.75& 0\\ 1& 0\end{array}\right], B=\left[\begin{array}{c}9.35\\ 0\end{array}\right], K=\left[\begin{array}{cc}-0.9777,& -0.9533\end{array}\right]$$

where the matrices A and B define the fundamental dynamics of the system.

The matrix A is designed to represent the natural dynamics of the robotic arm. The chosen values reflect the system’s physical properties described by the following model:

$$m\ddot{q}\left(t\right)+c\dot{q}\left(t\right)=\tau \left(t\right)$$

(14)

where $q\left(t\right)$ and $\dot{q}\left(t\right)$ are the actuator motor position and velocity, respectively, m is the inertia ($m=0.107\mathrm{kg}·{\mathrm{m}}^2$), and c is the Coriolis force parameter ($c=0.08\mathrm{Nms}$). The matrix B is selected to capture the influence of control inputs on the robotic arm’s states. The values in B ensure that the control signals appropriately affect the system, allowing for effective state regulation. The parameters are chosen based on standard modeling practices for robotic arms, which dictates how control efforts translate into state changes. The feedback gain matrix K is calculated using the Linear Quadratic Regulator (LQR) method. This method is widely used in control theory to design controllers that minimize a cost function representing the trade-off between state deviations and control effort. The specific values in K are obtained by solving the Riccati equation for the given system dynamics. This ensures that the controller is optimized for performance in the absence of jamming attacks, providing a baseline for evaluating the proposed scheme’s effectiveness under attack conditions.

The coefficients $c_1=0.85$ and $c_2=0.08$ are used to compute the cost associated with different states and actions within the game, and the discount factor $\alpha$ is set at $0.9$.

The Q matrix representing the quadratic costs associated with the state variables, emphasizing the penalization of deviations from desired states, is chosen as an identity matrix of size two.

The state space of the system is discretized through $x1range$ and $x2range$, which define the possible values of two state variables. $x1range$ progresses from 0.1 to 1 in increments of 0.1, while $x2range$ includes integer values from 1 to 10. This discretization creates a grid over which the game’s dynamics are computed, allowing for the approximation of continuous state spaces typically found in control problems. This setup is essential for applying numerical methods to solve the game, as it confines the potentially infinite state space to a manageable, finite set of states over which the value function and strategies can be optimized. Because of the discretization of states, a linear interpolation is necessary when system states are between values at the two nearest quantized states [32]. In this paper, we used linear interpolation using $interp2$, which is a two-dimensional interpolation of matrices in MATLAB. This function is used to estimate the values of the value function at nongrid points based on the known grid values.

In this study, two different matrices are used to calculate cost under different conditions to account for different scenarios related to the presence or absence of a jamming attack, indicated by the value of ${\gamma }_k$. These matrices represent different cost structures that the defender faces depending on whether or not a jamming attack is active at any given time step k.

• Matrix table when ${\gamma }_k=0$: Represented by Table 3 and its corresponding Transition probabilities represented by $P_{{\gamma }_0}$.
  $P_{{\gamma }_0}=\left[\begin{array}{cccccccc}0.2,& 0.8;& 0.3,& 0.7;& 0.9,& 0.1;& 0.5,& 0.5\end{array}\right]$
• Matrix table when ${\gamma }_k=1$: Represented by Table 4 and its corresponding Transition probabilities represented by $P_{{\gamma }_1}$.
  $P_{{\gamma }_1}=\left[\begin{array}{cccccccc}0.1,& 0.9;& 0.2,& 0.8;& 0.8,& 0.2;& 0.5,& 0.5\end{array}\right]$

Table 3. Defender’s Game Table when ${\gamma }_k=0$.

	A	NA
D	$5c_1+c_2{x}_k^{T}Q{x}_k$	$2c_1+c_2{x}_k^{T}Q{x}_k$
ND	$-c_1+c_2{x}_k^{T}Q{x}_k$	0

Table 3. Defender’s Game Table when ${\gamma }_k=0$.

	A	NA
D	$5c_1+c_2{x}_k^{T}Q{x}_k$	$2c_1+c_2{x}_k^{T}Q{x}_k$
ND	$-c_1+c_2{x}_k^{T}Q{x}_k$	0

Table 4. Defender’s Game Table when ${\gamma }_k=1$.

	A	NA
D	$3c_1+c_2{x}_k^{T}Q{x}_k$	$c_1+c_2{x}_k^{T}Q{x}_k$
ND	$-2c_1+c_2{x}_k^{T}Q{x}_k$	0

Table 4. Defender’s Game Table when ${\gamma }_k=1$.

	A	NA
D	$3c_1+c_2{x}_k^{T}Q{x}_k$	$c_1+c_2{x}_k^{T}Q{x}_k$
ND	$-2c_1+c_2{x}_k^{T}Q{x}_k$	0

According to the proposed method for solving the stochastic game in the previous section, we can obtain the optimal defense and attack strategies for all the possible states.

Figure 2 highlights the dynamic nature of the defender’s strategy in response to when ${\gamma }_k=1$, which means there is no attack. Initially, the defender committed to the strategy Defend, then the probability of defending gradually decreased from 1 to around 0.3, which suggests that over time steps, the defender starts to evaluate and shift their strategy in response to the absence of attack. Then, the probability of defense drops to zero, realizing that defending is no longer viable. ${\gamma }_k$ remains equal to 1.

Figure 3 highlights the opposite case when ${\gamma }_k=0$, which means there is an attack, and we can clearly see that the defender increases their defense probability until it converges to 1.

As explained previously in this paper, the attack signal represented by ${\gamma }_k$ is chosen to be random, as in real-time scenarios. The effectiveness of our approach relies on the response of the defender strategy to the attack signal depicted in Figure 4.

The fluctuations suggest that the defender’s strategy effectiveness varies significantly with the randomness of ${\gamma }_k$. These ups and downs indicate periods where the defense strategy successfully adapts to random changes in the value of the attack signal.

To better illustrate the effectiveness of our approach, Figure 5 shows the correlation between the defender’s strategy and the value of ${\gamma }_k$. Comparing both plots, we observe that when ${\gamma }_k$ is 1, the defender’s probability of defense tends to be lower (absence of attack). Conversely, when ${\gamma }_k$ is 0, the probability of defense increases (presence of attack). Finally, to evaluate the effect of the proposed defense mechanism on the communication channel state, in terms of packet loss rate, we simulate our program 100 times in Matlab R2021b environment.

The results of these simulations are shown in Figure 6; the first plot indicates that without any defense mechanism, the network is highly volatile and possibly vulnerable to external disruptions.

With defense mechanisms in place, the network state is significantly more stable, indicating effective mitigation of attacks or disruptions as shown in the second plot of Figure 6.

5. Limitations and Future Research Directions

While the proposed stochastic game-based defense mechanism significantly improves the security and stability of networked control systems (NCSs) against jamming attacks, several limitations need to be acknowledged.

• Computational Complexity: The implementation of the stochastic game framework requires significant computational resources. The iterative nature of the N-stage stochastic games and the need for real-time updates can lead to high computational costs, particularly for large-scale systems with numerous states and actions.
• Scalability Issues: The scalability of the proposed method is a concern, especially for very large NCSs with complex topologies. As the number of states and control actions increases, the complexity of solving the stochastic game also increases, potentially making it impractical for very large systems.
• Multiple-Attacker Scenarios: The current approach primarily focuses on single-attacker scenarios. In real-world applications, NCSs may face coordinated attacks from multiple adversaries. Extending the framework to handle multiple-attacker scenarios is a necessary step for comprehensive security.
• Impact on Network Performance: While the defense mechanism improves security, it may also impact the overall performance of the network. The balance between maintaining security and ensuring optimal network performance needs further exploration to minimize any adverse effects

The proposed stochastic game approach provides a robust defense mechanism against jamming attacks in NCSs, offering significant improvements in system stability and resilience. However, addressing the aforementioned limitations is crucial for enhancing the applicability and effectiveness of the method in real-world scenarios. Future research should focus on optimizing computational efficiency, improving modeling accuracy, and extending the framework to handle more complex attack scenarios.

6. Conclusions

This research has systematically explored the application of stochastic game theory to the security of networked control systems (NCSs) under conditions of potential jamming attacks. By modeling the interactions between defenders and attackers within a framework of N-stage stochastic games, we derived strategic insights that not only enhance the theoretical understanding of security dynamics but also provide practical guidance for the design and operation of more resilient control systems.

Our analysis demonstrates that the proposed defense mechanism significantly enhances the stability and security of NCS. The experimental results, illustrated in the figure, clearly show the following:

• State Stability Improvement: Without defense, the network state fluctuates significantly, indicating high instability. The state fluctuates between 0 and 1 frequently, demonstrating a lack of resilience against jamming attacks, while with defense, the network state remains consistently stable with minimal fluctuations. This indicates that the defense mechanism effectively mitigates the impact of jamming attacks.
• Percentage of Time Steps in Stable State: Without defense, the network state remains within a stable range (0.1 to 0.9) approximately $20\%$ of the time, while with defense, the network state remains within a stable range approximately $95\%$ of the time, showing a significant improvement in stability.

These results indicate that the proposed defense mechanism improves the network’s stability by 75 percentage points, making it substantially more resilient to jamming attacks.

The methodologies and strategies developed can be directly applied to the design of security protocols for real-world NCSs. By integrating these adaptive strategies into their operational frameworks, systems can better anticipate and react to cyber threats in a manner that minimizes disruption and damage.