An Electronic Jamming Method Based on a Distributed Information Sharing Mechanism

Abstract

In an electronic jamming system, the ability to adequately perceive information determines the effectiveness of an electronic countermeasures strategy. This paper proposes a new method based on the combination of a multi-agent electronic jammer and an information sharing mechanism. With the development of intelligent technology and deep learning, these technologies have been applied in electronic countermeasure game systems. Introducing intelligent technology into the electronic confrontation system can greatly improve decision-making efficiency. At the same time, a multi-agent electronic countermeasure cooperative system based on the information sharing method can break through the limited information perception capabilities of a single agent, thereby greatly improving the survivability of jamming systems in electronic warfare. Experimental results show that our method requires a lower jamming-to-signal ratio than the single jammer method to achieve effective electronic jamming. In addition, the electronic jamming parameters can be updated automatically as the external electromagnetic environment changes quickly, realizing a more intelligent electronic jamming system.

1. Introduction

An electronic jammer can jam targets by controlling the jamming signal parameters (e.g., signal center carrier frequency (Fc), signal bandwidth (BW), pulse repetition frequency (PRF), and pulse width (PW)) [1,2]. However, there is room for improvement in terms of perception ability and adaptive jamming signal parameters for electronic jammer systems.

Certain tasks in complicated environments are difficult to complete due to the limitations of a single jammer. Distributed jamming systems have been constantly evolving in recent years due to their vast application potential [3]. Electronic jammer swarms demonstrate group behavior by allowing individuals to interact locally in order to solve a global collaborative task [4]. Meanwhile, the present trend in electronic countermeasures (ECMs) is moving in the direction of intelligence and distribution. Distributed ECMs, as opposed to conventional centralized ECMs, take advantage of their number and area to achieve better jamming of targets [5]. The effect of distributed suppression and deception jamming on radar detection effectiveness is discussed in [6].

A number of experts have introduced artificial intelligence technology to electronic warfare. It was not until 2010 that DARPA [7,8] reported on adaptive electronic warfare learning, and the value of adaptive radar confrontation [9,10] was developed gradually. Wang Shafei [11] of the PLA Military Academy proposed a cognitive electronic warfare system architecture combining artificial intelligence with electronic warfare, greatly improving the ability of the electronic warfare system to perceive threat signals and make jamming decisions. Xing Qiang and Zhu Weigang [12] proposed an intelligent radar confrontation method based on Q-learning. By analyzing the convergence time and cycle times of the Q matrix, the jamming effect was realized along with independent perception and intelligent decision-making, and the adaptability of the radar confrontation system was improved as well.

This paper proposes an information sharing method based on a multi-agent electronic jamming system that realizes the real-time perception and efficient processing of three-dimensional electromagnetic situations. Simultaneously, multi-agent jamming parameters can be autonomously adjusted based on situational information sharing mechanisms between different jammers, ensuring that the electromagnetic situation of agents in the system is balanced. Meanwhile, this method can avoid jamming decision errors due to poor electromagnetic information.

2. Multi-Agent Cooperative Jamming Method

2.1. Range Gate Pull-Off Electronic Jamming Model

Figure 1 shows a schematic diagram of traditional static electronic jamming. In Figure 1, the active radar performs dynamic electromagnetic sensing of the external environment by transmitting signals with a specific modulation mode.

The electronic jammer intercepts the effective reconnaissance signal emitted by the radiation source to achieve rapid parameter estimation of the effective signal, then makes a specific electronic jamming decision for the radar source.

The signal $S(t)$ received by the radar can be expressed as

$$S(t)=A·\exp \left[j·2\pi ·\left(f_{0}t+\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$2$}\right.\mu t^2\right)\right]$$

(1)

where $A$ is the amplitude of the signal, $f_0$ is the carrier frequency, $\mu$ is the frequency modulation slope of the signal, and $t$ is the sampling time.

If the distance between the radar and the target is $R$, the receiver receives the radar echo signal after a delay of $\frac{2R}{c}$. Therefore, the echo signal $S_{rec}(t)$ received by the radar receiver can be expressed as

$$S_{rec}(t)=S(t-\frac{2R}{c})+n(t)$$

(2)

where $R$ is the distance between the radar and the target, $c$ is $3e^{8}m/s$, and $n(t)$ is modeled by white Gaussian noise with a distribution obeying the following formula:

$$n(t)~N(0,{\sigma }^2)$$

(3)

The receiver detects the effective linear frequency modulation (LFM) signal according to the energy accumulation method. The effective signal detection process is shown in Figure 2.

In the time-domain signal flow, the energy of the noise is much smaller than that of the effective signal. After the ADC sampling, the energy accumulation value of the 16-point discrete time-domain echo signal received by the jammer is compared with the threshold value. Here, we set the threshold amplitude to four times the amplitude energy of the echo signal. To ensure the detection of effective signals while reducing the probability of false alarms caused by noise, we adopt a 16-point energy accumulation method to detect LFM signals. When the cumulative value is continuously greater than the threshold, it represents the starting point of the effective signal, i.e., the time of arrival (TOA). Sliding detection is continuously carried out on the time-domain signal flow. When the cumulative value is continuously lower than the threshold value, it represents the time of end (TOE) of the effective signal. The time interval from the start point TOA to the end point TOE is the pulse width, $Pw$. The time interval between the first TOA and the second TOA is the pulse repetition period, $PRI$. The pulse repetition rate $PRF$ is the reciprocal of the pulse repetition period $PRI$ [13,14,15], i.e., $PRF=\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$PRI$}\right.$.

The carrier frequency $Fc$ of the LFM signal can be obtained by multiplying the signal delay conjugate $S_{\mathrm{rec}}{(T_1-t)}^{\ast }$ by the original signal $S_{\mathrm{rec}}(t)$, where $T_1$ is the time delay. The effective bandwidth $BW$ of a pulse signal can be calculated using the pulse width $Pw$ and pulse signal modulation slope $\mu$, that is, $BW=\mu ·Pw$. At this point, the pulse descriptor word (PDW) parameter estimation process for the target radar’s signal is completed.

The electronic reconnaissance process is shown in Figure 3, including the estimated parameters.

RGPO jamming adds a time delay to the received signal $S(t)$, resulting in a difference in distance between the jamming signal $S_j(t)$ and the real echo signal $S_{rec}(t)$ that interferes with the target’s recognition of the real echo signal. The jamming range gate can be either positive or negative. A positive distance means that the jamming signal is far from the radar receiver, while a negative distance means that the interference signal is close to the radar receiver.

Here, we apply a jamming range gate at a distance $R_j$ to the radar signal. Therefore, the electronic jamming signal after range gate pull-off (RGPO) jamming is as follows:

$$S_j(t)=S\left[t-\frac{2\left(R+R_j\right)}{c}\right]+n(t)$$

(4)

where $R_j$ is the range gate. The jamming range gate can be either positive or negative.

2.2. Balancing Parameters of the Multi-Agent Situation Information

Assuming the electronic jamming system contains N electronic jammer nodes, the situational information of a single electronic jammer agent can be expressed as follows:

$$\begin{array}{ll}{X}_i& =C_i\left({\left\{\mathrm{Information}\mathrm{of}\mathrm{Agent}i\right\}}_{i\in N}\right)\\ & =C_i:\left(Fc,Pw,PRF,BW\right)\end{array}$$

(5)

where $Fc$ is the center carrier frequency, $Pw$ is the signal pulse width, $PRF$ is the pulse repetition rate, and $BW$ is the effective bandwidth of the signal. Moreover, $C_i:\left(Fc,Pw,PRF,BW\right)$ represents the situation information of the parameter set $\left(Fc,Pw,PRF,BW\right)$ at the ith electronic jammer agent.

Then, the situation information that the multi-agent system can perceive is expressed as

$$C={\displaystyle \sum _{i=1}^N\left(X_i|i\in N_i\right)}$$

(6)

where ${\displaystyle \sum _{i=1}^N}$ is the sum situation information of N jammers.

Here, we assume that the weight of the parameter is $\gamma$. The parameter weight is represented as follows:

$$\{\begin{array}{l}4·\gamma =1\\ {\alpha }_{Fc}={\alpha }_{Pw}={\alpha }_{PRF}={\alpha }_{BW}=\gamma \end{array}$$

(7)

where $\left\{{\alpha }_{Fc},{\alpha }_{Pw},{\alpha }_{PRF},{\alpha }_{BW}\right\}$ corresponds to the weight of $Fc,Pw,PRF,BW$.

Because the situation parameter information between different jammer agents is not exactly the same at the same time, it is necessary to observe and balance the parameters between the different agents. A partial derivation of the reconnaissance situation parameters in the adjacent electronic jammer is carried out; the expression is as follows:

$$\begin{array}{ll}\frac{\partial \left(X_i-X_{i-1}\right)}{\partial Fc}=& \frac{\partial \left(C_i:Pa\left(Fc,Pw,PRF,BW\right)\right)}{\partial Fc}\\ & -\frac{\partial \left(C_{i-1}:Pa\left(Fc,Pw,PRF,BW\right)\right)}{\partial Fc}\\ \hfill =& \frac{\partial \left(C_i|Fc-C_{i-1}|Fc\right)}{\partial Fc}\end{array}$$

(8)

where $\frac{\partial F}{\partial Fc}$ is the derivative of $Fc$ with respect to the function F.

The expression of the situational information transmission of the jamming parameters $\left(Fc,Pw,PRF,BW\right)$ between different jammer agents can be expressed as

$$\{\begin{array}{l}\frac{\partial \left(X_i-X_{i-1}\right)}{\partial Pw}=\frac{\partial \left(C_i|Pw-C_{i-1}|Pw\right)}{\partial Pw}\\ \frac{\partial \left(X_i-X_{i-1}\right)}{\partial PRF}=\frac{\partial \left(C_i|PRF-C_{i-1}|PRF\right)}{\partial PRF}\\ \frac{\partial \left(X_i-X_{i-1}\right)}{\partial BW}=\frac{\partial \left(C_i|BW-C_{i-1}|BW\right)}{\partial BW}\end{array}$$

(9)

where $\frac{\partial F}{\partial (Pw,PRF,BW)}$ is the derivative of $(Pw,PRF,BW)$ with respect to the function F.

The purpose of having multiple sensing nodes in the electronic jamming system is to improve the accuracy of the system regarding the situation parameter information of the radiation source and to reduce the misjudgment of the situation information caused by environmental factors. If the reconnaissance parameter information of different agents is unbalanced, the parameters of each agent need to be adjusted accordingly. For example, if a node does not work or is interfered with, resulting in a large difference in the derivative of the parameter set $\left(Fc,Pw,PRF,BW\right)$ between adjacent agents, then the problematic situation information should be ignored or discarded.

Therefore, the effective inter-agent situation parameter information $C_i|\left(Fc,Pw,PRF,BW\right)$ should be the situation information after the problematic parameters are removed. The modified situation information expression is as follows:

$$\{\begin{array}{l}{C}_i|Fc=Mean\left(\underset{k\in N_i}{\Pi }\left(C_k|Fc\right)\right)\\ C_i|Pw=Mean\left(\underset{k\in N_i}{\Pi }\left(C_k|Pw\right)\right)\\ C_i|PRF=Mean\left(\underset{k\in N_i}{\Pi }\left(C_k|PRF\right)\right)\\ C_i|BW=Mean\left(\underset{k\in N_i}{\Pi }\left(C_k|BW\right)\right)\end{array}$$

(10)

where the function $\underset{k\in N_i}{\Pi }(\Xi )$ represents the average value of parameter set $\Xi$ in a certain range $\Vert Mean(\Xi )·(1-\ell ),Mean(\Xi )·(1+\ell )\Vert$. Here, $\ell$ is $0.1$.

Then, the multi-agent situation parameters after information sharing can be expressed as

$$\underset{i\in N_i}{X^{\prime }}=C_{Balance}:\left(F{c}^{\prime },P{w}^{\prime },PR{F}^{\prime },B{W}^{\prime }\right)$$

(11)

where $C_{Balance}$ represents the balanced agent situation information and $F{c}^{\prime },P{w}^{\prime },PR{F}^{\prime },B{W}^{\prime }$ represent the balanced jamming parameters’ values.

Based on the mechanism of multi-agent parameter information sharing, dynamic situation information perception of the external electromagnetic environment is efficiently achieved.

2.3. Autonomous Decision of Jammer Agent Parameters

Through information sharing mechanisms, multi-agent jammers realize the collaborative perception of electromagnetic situation information in the whole region through cooperative means. At the same time, in order to make the jamming signal of the electronic jammer adapt to the new situation, it is necessary to control the jamming parameters dynamically and independently in real time.

Suppose that the parameter of electronic jamming agent $i$ at time $t$ is

$${\underset{i}{Par}}^t{}_{\mathrm{jam}}=K_i{}^t:\left\{\gamma ·\left({\mu }_{Fc},{\mu }_{Pw},{\mu }_{PRF},{\mu }_{BW}\right)\right\}$$

(12)

where $K_i{}^t$ represents the total situation of the jamming parameters of agent $i$ at moment $t$, $\left({\mu }_{Fc},{\mu }_{Pw},{\mu }_{PRF},{\mu }_{BW}\right)$ represents the corresponding parameter values of $(Fc,Pw,PRF,BW)$, and $\gamma$ indicates the weight of each jamming parameter.

It is important to measure the rate of change between the current jamming parameters and the radar radiation source parameters. Understanding the current parameter variation trend is a key index for the subsequent jamming parameter set $(Fc,Pw,PRF,BW)$ decision at time $t+1$. The transformation rate of the parameter set $(Fc,Pw,PRF,BW)$ at a time between t and $t+1$ can be obtained by taking the partial derivative of parameter set $(Fc,Pw,PRF,BW)$ of the function $\left({\underset{i}{Par}}^t{}_{\mathrm{jam}}-{\underset{i}{X}}^{t+1}\right)$. Therefore, the transformation rate ${\nabla }^{t,t+1}{}_{(Fc,Pw,PRF,BW)}$ can be expressed as

$$\{\begin{array}{l}{\nabla }^{t,t+1}{}_{Fc}=\frac{\partial \left({\underset{i}{Par}}^t{}_{\mathrm{jam}}-{\underset{i}{X}}^{t+1}\right)}{\partial Fc},\\ {\nabla }^{t,t+1}{}_{Pw}=\frac{\partial \left({\underset{i}{Par}}^t{}_{\mathrm{jam}}-{\underset{i}{X}}^{t+1}\right)}{\partial Pw},\\ {\nabla }^{t,t+1}{}_{PRF}=\frac{\partial \left({\underset{i}{Par}}^t{}_{\mathrm{jam}}-{\underset{i}{X}}^{t+1}\right)}{\partial PRF},\\ {\nabla }^{t,t+1}{}_{BW}=\frac{\partial \left({\underset{i}{Par}}^t{}_{\mathrm{jam}}-{\underset{i}{X}}^{t+1}\right)}{\partial BW}.\end{array}$$

(13)

where $\frac{\partial \left({\underset{i}{Par}}^t{}_{\mathrm{jam}}-{\underset{i}{X}}^{t+1}\right)}{\partial (Fc,Pw,PRF,BW)}$ is the derivative of $(Fc,Pw,PRF,BW)$ with respect to the function $\left({\underset{i}{Par}}^t{}_{\mathrm{jam}}-{\underset{i}{X}}^{t+1}\right)$.

The trend of the situation parameters for the next moment is updated for the same agent at different times. Using the situational information sharing method, the next changing process of jamming parameter $K_i{}^{t+1}|(Fc,Pw,PRF,BW)$ is as follows:

$$\{\begin{array}{l}{K}_i{}^{t+1}|Fc={\mu }_{Fc}+\epsilon ·{\nabla }^{t,t+1}{}_{Fc},\\ K_i{}^{t+1}|Pw={\mu }_{Pw}+\epsilon ·{\nabla }^{t,t+1}{}_{Pw},\\ K_i{}^{t+1}|PRF={\mu }_{PRF}+\epsilon ·{\nabla }^{t,t+1}{}_{PRF},\\ K_i{}^{t+1}|BW={\mu }_{BW}+\epsilon ·{\nabla }^{t,t+1}{}_{BW}.\end{array}$$

(14)

where $\left({\mu }_{Fc},{\mu }_{Pw},{\mu }_{PRF},{\mu }_{BW}\right)$ represents the corresponding parameter values of $(Fc,Pw,PRF,BW)$ and $\epsilon$ is a constant with a value of 0.1.

The information about the electronic jamming parameters ${\underset{i}{Par}}^{t+1}{}_{\mathrm{jam}}$ updated at moment $t+1$ can be expressed as follows:

$$\begin{array}{l}{\underset{i}{Par}}^{t+1}{}_{\mathrm{jam}}=K_i{}^{t+1}\left({\left\{\begin{array}{c}\mathrm{Jamming}\mathrm{Paraments}\\ \mathrm{of}\mathrm{Agent}i\end{array}\right\}}_{i\in Ni}\right)\\ =K_i{}^{t+1}:\left({\mu }^{\prime }{}_{Fc},{\mu }^{\prime }{}_{Pw},{\mu }^{\prime }{}_{PRF},{\mu }^{\prime }{}_{BW}\right).\end{array}$$

(15)

where $\left({\mu }^{\prime }{}_{Fc},{\mu }^{\prime }{}_{Pw},{\mu }^{\prime }{}_{PRF},{\mu }^{\prime }{}_{BW}\right)$ represents the weight of $(Fc,Pw,PRF,BW)$ at moment $t+1$ for the ith electronic jammer agent.

3. Evaluation of Jamming Effectiveness

3.1. Jamming-to-Signal Ratio Definition

To effectively jam a radar signal, it is necessary to create a jamming signal at the target receiver for the radar signal. The jamming-to-signal ratio (JSR) is a good indicator for quantifying the jamming effect. The JSR is similar to the signal-to-noise ratio (SNR), where J represents the received jamming signal energy and S represents the received radar signal energy.

Consider a self-defense electronic jamming situation in which each target has a cognitive jammer. In the case of a radar and a jammer, the jammer focuses on optimizing the jamming efficacy by learning the radar’s strategy in order to protect the target from detection [16]. To make the study easier, we consider the target to be a point target with a radar cross-section (RCS) of $\rho$. We assume that the radar is jammed during each beam dwell time, which is referred to as a jamming round in this article. The number of radar pulses transmitted during a jamming round is determined by the beam dwell duration and the pulse repetition interval (PRI). For the $n$th radar pulse in a jamming round, the carrier frequency $Fc$ is $F{c}_r^{(n)}$, the bandwidth $BW$ is $B{w}_r^{(n)}$, the $PRI$ is $pr{i}_r^{(n)}$ which represents the time between the rising edge of the $(n-1)$th and $n$th radar pulses, the pulse width $PW$ is $P{w}_r^{(n)}$, and the transmission power is $P_r^{(n)}$. The jammer tries to align the jamming signal with the radar signal in both the time and frequency domains at each pulse. For the nth jammer pulse, the Fc is $f_{jam}^{(n)}$, the BW is $B{w}_r^{(n)}$, the pulse delay time is $d{t}_{jam}^{(n)}$, which indicates the time between receiving the $(n-1)$th radar pulse and sending the next jamming pulse, the $PW$ is $P{w}_{jam}^{(n)}$, and the transmission power is $P_r^{(n)}$. Furthermore, the distance between the radar and the target is $R$ and the wavelength of the radar signal is $\lambda$. The radar and jammer antenna gains are $G_r$ and $G_{jam}$, respectively. The radar and jamming signal propagation losses are $L_r$ and $L_{jam}$, respectively. The polarization matching loss coefficient between the jammer signal and the radar signal is $\psi$. The power of the echo at the radar receiver can be expressed as

$$P_{rs}^{(n)}=\frac{P_r^{(n)}{G}_r^2\rho {\lambda }^2}{{(4\pi )}^3{D}^4{L}_r}$$

(16)

The power of the nth jamming pulse at the radar receiver is

$$P_{rj}^{(n)}=\frac{P_{jam}^{(n)}{G}_{jam}{G}_r{\lambda }^2\psi }{{(4\pi )}^2{D}^2{L}_{jam}}$$

(17)

Using an effective jamming coefficient to modify the JSR calculation formula, the average JSR for the nth radar pulse is computed as follows:

$$JS{R}^{(n)}=\frac{P_{rj}^{(n)}·X_f^{(n)}·X_t^{(n)}}{P_{rs}^{(n)}}=\frac{P_j^{(n)}{G}_{jam}\mu 4\pi D^2{L}_r{X}_f^{(n)}{X}_t^{(n)}}{P_r^{(n)}{G}_r\rho L_{jam}}$$

(18)

where $X_f^{(n)}$ and $X_t^{(n)}$ are the effective jamming factors in the frequency and time domains, expressed as

$$\begin{gathered} X_f^{(n)}=\frac{\Delta f^{(n)}}{B_j^{(n)}}·\mathrm{sgn}\left(\Delta f^{(n)}\right) \\ {X}_t^{(n)}=\frac{\Delta t^{(n)}}{P{w}_j^{(n)}}·\mathrm{sgn}\left(\Delta t^{(n)}\right) \end{gathered}$$

(19)

where $\Delta f^{(n)}$ and $\Delta t^{(n)}$ are the overlapping rates in the frequency domain and the time domain, provided as follows:

$$\begin{gathered} \Delta f^{(n)}=\min \left(F{c}_j^{(n)}+B{w}_j^{(n)}/2,F{c}_r^{(n)}+B{w}_r^{(n)}/2\right)-\max \left(F{c}_j^{(n)}-B{w}_j^{(n)}/2,F{c}_r^{(n)}-B{w}_r^{(n)}/2\right) \\ \Delta t^{(n)}=\min \left(d{t}_j^{(n)}+P{w}_j^{(n)},PR{I}_r^{(n)}+P{w}_r^{(n)}\right)-\max \left(d{t}_j^{(n)},PR{I}_r^{(n)}\right) \end{gathered}$$

(20)

where $\mathrm{sgn}(x)$ can be written as

$$\mathrm{sgn}(x)=\{\begin{array}{ll}1,& x>0\\ -1,& \mathrm{otherwise}\end{array}$$

(21)

3.2. Q-Learning Method for Jamming Policy

The jamming process can be represented as a quaternion $\{\mathcal{S},\mathcal{A},\mathcal{P},ℛ\}$ through a finite Markov decision process (MDP) [17,18]. Here, $\mathcal{S}$ is a finite collection of radar states, with state $s\in \mathcal{S}$ determined by the radar pulse parameters $(Fc,Pw,PRF,BW)$, and $\mathcal{A}$ is a finite set of jammer actions, where action $a\in \mathcal{A}$ is defined by the jamming pulse parameters. When the jammer performs action $a^{(n)}$, the transition probability $\mathcal{P}\left(s^{(n+1)}\mid s^{(n)},a^{(n)}\right)$ describes how the present state $s^{(n)}$ transitions to the next state $s^{(n+1)}$. In addition, $ℛ$ is the reward after each action is taken.

Reinforcement learning is an effective method for solving MDP problems, with the key being the determination of the optimal policy $\pi :\mathcal{S}\to \mathcal{A}$ to determine which action should be performed in each state. The state-value function for policy $\pi$ is proposed to evaluate the effect of a policy, as follows:

$$v_{\pi }(s)=E_{\pi }\left[{\displaystyle \sum _{m=0}^{\infty }{\xi }^m}{ℛ}_m\mid s_m=s\right]$$

(22)

where $E_{\pi }[·]$ stands for the expected value with the given policy $\pi$ and $\xi \in [0,1)$ is the discount rate of the reward $R$, which means that a long-term reward is considered and its influence decreases with time.

The update strategy of the electronic jammer parameters based on Q-learning [19,20,21,22] is as follows:

$$Q^{*}(s,a)=Q(s,a)+\eta ·(r(s,a)+\xi ·\underset{a^{\prime }}{\max }Q(s^{\prime },a^{\prime })-Q(s,a))$$

(23)

where $s$ is the current jamming parameter status of the electronic jammer, $a$ is the current action taken, $r(s,a)$ indicates the reward after the operation of action $a$, $\underset{a^{\prime }}{\max }Q(s^{\prime },a^{\prime })$ is the best parameter adjustment action taken by the jammer agent at the next moment, $\eta$ is the learning rate parameter, and $\xi$ is the discount factor parameter. In continuous tasks, the discount factor $\xi$ is usually set in the range [0,1) in order to ensure that the reward $r(s,a)$ does not diverge to infinity.

In our work, the jammer applies jamming to the radar by changing the signal characteristic parameters. The more realistic the electronic jamming signal is compared to the real echo signal, the better the interference effect. Here, $a$ represents the action space of each of the jamming parameters $(Fc,Pw,PRF,BW)$ and $s$ represents the current state of each jamming parameters. At the beginning of moment $t$, when we perform action $a$ on the jamming parameter $(Fc,Pw,PRF,BW)$ the jammer obtains the current local state information as $s$ and the current Q table as $Q(s,a)$. At the same time, it receives a reward $r(s,a)$ for executing action $a$. Obviously, the reward function is mainly used to reward expected behaviors and punish undesirable actions. Then, we use the $r(s,a)+\xi ·\underset{a^{\prime }}{\max }Q(s^{\prime },a^{\prime })$ corresponding to action $a^{\prime }$ as the estimated value. Finally, $Q^{*}(s,a)$ is updated using Formula (23). The above operation is repeated until the jamming parameters $(Fc,Pw,PRF,BW)$ reach the optimal state.

The dynamic cognitive electronic jamming strategy based on Q-learning can realize real-time perception based on the environmental electromagnetic situation and update the electronic jamming parameters of the system in real time according to the perception results, ensuring that the jamming strategy of the electronic jamming system always has the best possible status. The system flow is shown in Figure 4.

In Figure 4, the distributed electronic jammer can obtain spatial electromagnetic situation information among multiple agents through collaborative perception. The agent situation after information sharing is expressed as $X^{\prime }$. The reconnaissance situation information at the current moment and the previous moment is input into the intelligent decision system. The optimal jamming strategy for the next epoch implemented by the electronic jamming machine is realized through the Q-learning decision algorithm.

4. Experiment and Analysis

A distributed multi-agent cooperative electronic jamming system uses a process of continuous correction during the process of interaction with the external electromagnetic environment, in which the jamming parameters change dynamically. In the process of electronic confrontation, electronic jammer agents constantly learn from and evolve with the electromagnetic environment to achieve the optimal jamming effect.

In order to verify the effectiveness of this method, the following experiments were designed.

Table 1. Electromagnetic parameter information at different times.

Stage	Fc/GHz	Pw/us	PRF/Hz	BW/MHz
T0	3.25	22.5	5000	10
T1	5	60	10,000	10
T2	4.375	40	5000	20
T3	2.6	50	2500	20
T4	5.45	20	10,000	15
T5	6.25	35	5000	15

shows the change trend of the electromagnetic situation information of the radiation sources in different stages. The period from T0 to T5 indicates the changes in the radiation source parameters at every 10-min interval. It can be seen from the table that the signal parameters emitted by the electromagnetic radiation source are changed in different stages, which can effectively avoid the problem of tracking brought about by the single parameters of the radiation source emission signal. The radiation source characteristics with parameter variations have a certain anti-jamming function. Electronic jammers need to perceive the change trend of the radiation source situation information quickly in a dynamic and complex electromagnetic environment. Moreover, the information is used as prior information to realize autonomous perception and a dynamic cognitive jamming strategy.

4.1. Situation Information Sharing

Based on the parameter situation information of the radiation source emission signal in Table 1, the electronic jammer is able to realize the electromagnetic parameter estimation and modulation type recognition of the radiation source through the passive electronic reconnaissance system.

In this experiment, six electronic jammer agents were distributed isometrically around the radiation source to realize the dynamic perception of the spatial global electromagnetic situation information. The signal-to-noise ratio of the environment was set to 0 dB, and the noise was assumed to be white Gaussian noise with a power density of one half. Because the situation parameter information between each agent is not completely consistent, it is necessary to balance the situation parameters obtained by the distributed electronic agents. The balanced jammer agent parameters can be expressed as $\underset{i\in N_i}{X^{\prime }}=C_{\mathrm{Balance}}:Pa\left(F{c}^{\prime },P{w}^{\prime },PR{F}^{\prime },B{W}^{\prime }\right)$. The results are shown in Figure 5.

The different characteristic parameters (including $Fc,Pw,PRF,BW$) of the radiation source perceived by the six electronic jammers at different sampling times are shown in Figure 5. In this experiment, the signal-to-noise ratio was 0db. Among them, the transverse coordinate is the agent number, the longitudinal coordinate is the value of the corresponding characteristic parameters, the virtual line represents the theoretical parameter value, and the real line represents the actual measurement value after the balanced treatment.

The radiation source signal parameter estimation results based on information sharing are shown in

Table 2. Measured values of electromagnetic parameters at different times.

Stage	Fc/GHz	Pw/us	PRI/us	$\mathit{\mu }$/Hz/s
T0	3.2539	22.367	198.3	4.492 × 1011
T1	4.9758	59.76	102.76	1.634 × 1011
T2	4.3863	40.432	200.843	4.9317 × 1011
T3	2.6036	50.372	399.346	3.9705 × 1011
T4	5.4620	19.874	100.264	7.5475 × 1011
T5	6.2510	35.006	201.149	4.285 × 1011

.

The relationships between $PRI$ and $PRF$ can be written as

$$PRI=\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$PRF$}\right.$$

(24)

The relationships between $\mu$, $Pw$, and $Bw$ are as follows:

$$\mu =\raisebox{1ex}{$Bw$}\!\left/ \!\raisebox{-1ex}{$Pw$}\right.$$

(25)

The multi-agent situation information sharing method can fully characterize the current electromagnetic situation. At the same time, the situation parameter imbalance caused by insufficient single jammer agent perception ability can be avoided.

4.2. Optimization of the Jamming Parameters in Real Time

The characteristic parameters ${X_i}^{\prime }$ of the balanced radiation source signal are input into the electronic jammer intelligence system as environmental perception information, and the real-time intelligent jamming strategy for the radiation source target is realized using the Q-learning method.

Hyperparameters are elements that must be set in Q-learning in order to adjust learning strategies. In this experiment, the learning rate $\eta$ was set to 0.01 and the discount factor $\xi$ was set to 0.9. The Q table of all electronic agents was initialized to 0 and the state $s$ of each agent was randomly initialized.

Based on the shared electromagnetic situation information obtained in Section 4.1, the jamming parameters of the electronic jammers can change independently with the change in the radiation source parameters and achieve accurate and effective jamming of the radiation source. The reward function and the loss function based on the Q-learning method in a certain time period are shown in Figure 6 and Figure 7.

Figure 6 and Figure 7 show the reward–return function curve and loss function curve for one period. They demonstrate that the electronic jamming parameters achieve convergence in a very short period of ten epochs. Moreover, the loss value reaches 0.0016 and the reward value reaches up to 0.9924, thereby realizing the rapid updating of the electronic jamming parameters.

Figure 8 shows a real-time jamming parameter response based on Q-learning with the radiation source situation parameters in Table 1 changed.

As can be seen in Figure 8, the electronic jammers can quickly estimate the current electromagnetic situation and adjust the jamming parameters in the next epoch to adapt to the current electromagnetic environment when the radiation source changes.

4.3. Effect of the Number of Jammers on Perception Accuracy

Considering the influence of environmental factors, we set the reconnaissance error probability of the jamming parameters to 5%. The ability of different numbers of electronic jammers to perceive the electromagnetic situation information was analyzed after 1000 independent Monte Carlo experiments.

The accuracy result for different jammers is shown in Figure 9.

In Figure 9, the accuracy increases as the number of electronic jamming agents grows. The accuracy of the situational information $P\left(X\right)$ perceived by the distributed jamming agents can be expressed as follows:

$$P\left(X\right)={\displaystyle \sum _{l=1}^N\frac{P\left(X_l|{}_{Fc}\right)+P\left(X_l|{}_{Pw}\right)+P\left(X_l|{}_{PRF}\right)+P\left(X_l|{}_{BW}\right)}{4}}$$

(26)

where $P\left(X_l|{}_{(Fc,Pw,PRF,BW)}\right)$ represents the accuracy of the parameters $(Fc,Pw,PRF,BW)$ in terms of the situation information $X_l$.

The estimation error $\Re \left({\underset{}{Par}}_i\right)$ of the parameter set $\left(Fc,Pw,PRF,BW\right)$ can be expressed as follows:

$$\Re \left({\underset{}{Par}}_i\right)=\frac{\raisebox{1ex}{${\displaystyle \sum _{n=1}^{1000}{\Vert {\underset{}{Par}}_{\mathrm{i}\_\mathrm{real}}-{\underset{}{Par}}_{\mathrm{i}\_\mathrm{estimate}}\Vert }_2}$}\!\left/ \!\raisebox{-1ex}{$1000$}\right.}{{\underset{}{Par}}_{\mathrm{i}\_\mathrm{real}}}\times 100\%$$

(27)

where ${\underset{}{Par}}_{\mathrm{i}\_\mathrm{real}}$ is the theoretical value and ${\underset{}{Par}}_{\mathrm{i}\_\mathrm{estimate}}$ is the actual measurement value of the parameter set $\left(Fc,Pw,PRF,BW\right)$.

Compared with the perception of radiation source information by a single agent, multiple agents are able to obtain a more accurate situational awareness of the radiation sources. When the number of electronic jamming agents reaches seven, the recognition rate can reach 98.43%.

4.4. Analysis of Jamming Effectiveness Evaluation

In order to verify the effectiveness of our proposed jamming method, range gate pull-off (RGPO) jamming was carried out on the target. The distance between the radar and the target was 100 km, and the range gate for the jamming target was 10 km. Other parameters in our simulation were as follows: $G_r$ = 20 dB, $G_{jam}$ = 5 dB, $L_r$ = 10 dB, $L_{jam}$ = 7 dB, $\rho$ = 1 m². Considering that radar antennas are typically linearly polarized, whereas jammer antennas are circularly or tangentially polarized, $\gamma$ was set to 0.25, $\kappa$ was set to 0.8, and $\epsilon$ was set to 0.1. The other radar parameters were set as shown in Table 1 in different stages. In particular, six electronic jamming agents were distributed in the system.

We compared the performance of our algorithm with six jammers based on a distributed information sharing method and jamming with a single jammer. Figure 10 shows the jamming effect of the two methods from T0 to T5.

In Figure 10, compared to the jamming method with a single jammer, the jamming signal spectrum of our proposed method is closer to the real radar signal. Additionally, the sidelobe power of the jamming signal taken by our proposed method is lower than that taken by the method with a single jammer, which reduces the threat of the jamming signal, thereby improving the survivability of the jammers.

According to the definition of the JSR in Section 3.1, we calculated the JSR value of the jamming signal of the two methods in different stages.

In

Table 3. Jamming signal JSR value of the two methods in different stages.

Method	JSR
	T0	T1	T2	T3	T4	T5
Our algorithm	1.0215	0.9987	1.0032	1.0386	1.0108	1.1250
Jamming with a single jammer	1.6732	1.7851	1.5738	1.6210	1.8327	1.5248

, it can be seen that the JSR value of the jamming signal generated by our method is smaller than that generated by the jamming method with a single jammer in each stage. This result further validates the effectiveness of our method.

Table 4. The feasibility verification of our method for the jamming parameter set $\left(Fc,Pw,PRF,BW\right)$.

NO.	Evaluation Indicator	Correlation with Evaluation Result
1	Fc	positive
2	Pw	positive
3	PRF	positive
4	BW	positive

shows the feasibility verification of our method for the jamming parameter set $\left(Fc,Pw,PRF,BW\right)$.

5. Conclusions

In this article, we have proposed a new electronic jamming method based on a distributed information sharing mechanism. This method, which is based on Q-learning, can realize the real-time perception of the external electromagnetic environment and the autonomous updating of the jamming parameters. At the same time, it can realize a fast and accurate response from the jamming system and improve the concealment of jamming signals. Compared with the jamming method with a single jammer, the proposed distributed multi-agent electronic jamming method based on information sharing can achieve effective jamming with a lower JSR. Therefore, the concealment of the jammer system is improved, and the survival probability of the jammer is enhanced. Moreover, the proposed method provides practical advantages for intelligent electronic jamming systems and can serves as inspiration for future cognitive electronic jammers.