Interactive Multiple-Model Learning Filter for Spacecraft Pursuit–Evasion Game Strategy Switch Based on Long Short-Term Memory Network

Abstract

Aiming to address the problem of pursuit and interception for spacecraft using multiple evasion strategies, a pursuit strategy involving the use of an interactive multiple-model filter (IMM) in a pursuit–evasion game is considered, where the Evader adopts a switchable evasion strategy based on a linear quadratic method and zero-effort miss method. In this case, an improved interactive multiple-model feedback-learning filter method based on a long short-term memory network (LSTM-IMML) is proposed to estimate the Evader’s strategy mode, with the resulting estimation allowing the Pursuer to then switch its own strategy to the appropriate pursuit strategy to intercept the Evader. Also, the improved interactive multiple-model feedback learning filter can feed back the fusion estimation of the last-time state to the next-time state to improve estimation accuracy. An LSTM-based probability estimation network is designed to accurately estimate the probability of different modes. The proposed LSTM-IMML method can be used in the pursuit–evasion game when the Evader is able to switch its evasion strategy. The simulation results show that the LSTM-IMML method has better state estimation accuracy, and the mode probability estimation of the Evader is more exact and stable.

1. Introduction

Recently, the pursuit–evasion problem between spacecraft remains an important area of study in aerospace engineering, where one spacecraft acts as the pursuer and the other as the evader. Various pursuit-and-evasion strategies have been proposed to solve the pursuit–evasion problem between two spacecraft. But these studies mostly focus on a single pursuit–evasion strategy; that is, the Evader only uses one evasion strategy. However, with the improvement of spacecrafts’ intelligence levels, the Evader may adopt a variety of strategies and will be able to switch to a more appropriate one according to the situation.

For the pursuit–evasion problem, the use of differential game theory, which had been widely used by other researchers in this field, was initially proposed as a solution by Issacs in 1965 [1]. Mauro et al. [2] studied the problem of a long-distance pursuit–evasion game by transforming it into a two-point boundary value problem and proposed a semi-direct method to obtain the saddle point solution. Ye et al. [3] also transformed the pursuit-and-evasion problem into a two-point boundary value problem and solved it through a heuristic search. This algorithm was applied in a close-range pursuit–evasion game under different thrust configurations. Li et al. [4] and Zhang et al. [5] studied the pursuit–evasion game considering the J2 perturbation and proposed new methods with which to quickly find the saddle point solution. Prince et al. [6] used the indirect heuristic method to study the differential game of proximity operations in elliptical orbits. Pang et al. [7] studied the pursuit–evasion game along an elliptical orbit by providing a precise gradient. Unfortunately, the methods mentioned above are all open-loop solutions and cannot be used for real-time feedback control.

Consequently, researchers studied the real-time feedback control of the pursuit–evasion problem. Li et al. [8] proposed an infinite-horizon nonlinear quadratic differential game considering the motion camouflage pursuit problem. Wang et al. [9] and Ye et al. [10] investigated pursuit–evasion control based on zero-effort miss and deduced a pursuit–evasion feedback control strategy. Zhang et al. [11] proposed a new adaptive weighted differential game guidance law to intercept maneuvering targets by combining two guidance laws derived from complete and incomplete information modes. Li et al. [12] designed a linear- quadratic duration-adaptive strategy to solve the orbital pursuit–evasion–defense game problem.

In addition, other methods have also been studied to solve the pursuit–evasion problem. Gong et al. [13] used the reachable region method to study the pursuit–evasion game under continuous thrust and derived the analytical form of the reachable region based on the Hill–Clohessy–Wiltshire (HCW) equation. Zhao et al. [14] proposed an impulsive pursuit–evasion algorithm based on a multi-agent deep deterministic policy gradient (MADDPG) that can yield a pursuit–evasion strategy under multiple constraints. In summary, in the continuous thrust pursuit–evasion game scenario, the real-time feedback control law based on game theory is more likely to be applied in space due to its simple structure and excellent pursuit ability.

In order to evade interception by the Pursuer more effectively, the Evader may employ a variety of strategies and switch from one to another in the process depending on the situation [15]. For example, in reference [16], Evaders have varying structural dynamics and will switch from one mode to another in the game, resulting in changes of evasion strategies. Therefore, for the Pursuer, it is necessary to adjust its pursuit mode to achieve efficient interception of the Evader. In the complete information pursuit–evasion game, the Pursuer can obtain all the information of the Evader and adjust its pursuit strategy in real time. However, the Pursuer generally cannot obtain all the information on the Evader in practice. Under the condition of incomplete information, the Pursuer has to estimate the strategy adopted by the Evader according to the states of the two players and adjust its strategy in real time. This motivated us to study the problem of intercepting the Evader with a switchable evasion strategy under conditions of incomplete information.

The interactive multiple-model filter (IMM) is a method for estimating the state of dynamic systems, and it can be used for Markov stochastic jump systems. It is widely used in state estimation [17], dynamic target tracking [18], fault detection [19,20], and so on. Considering the characteristics of multiple-model estimation, the IMM method is used to solve the problem of Evader strategy switching in a pursuit–evasion game. Zou et al. [21] proposed a cooperative estimation method that combines the IMM of the evader and the Kalman filter of the defender. The cooperative method can effectively estimate the state of the Pursuer and significantly improve the accuracy of active defense guidance. Tang et al. [22] used the IMM method to estimate the state information of the Evader by combining the smooth variable sliding filter with mode matching, achieving a good interception effect.

However, the difference in control strategies is very small in spacecraft pursuit–evasion game strategy switching due to the low thrust of satellite thrusters. The classical IMM cannot predict the evasion mode precisely, which will affect interception performance. In addition, the model probability estimated by an IMM is unstable and fluctuates greatly due to the influence of navigation information error. Motivated by the above problems, this paper aims to design a method that can accurately and stably estimate the strategy mode of the Evader so as to intercept the Evader quickly.

The main novelties and contributions of this paper are as follows: (1) A switchable escape strategy based on a linear quadratic game strategy and a zero-effort miss game strategy was designed. (2) An interactive multiple-model learning filter is proposed by introducing the idea of feedback. (3) An interactive multiple-model learning filter based on an LSTM network (LSTM-IMML) is proposed.

The remainder of this paper is organized as follows. The Pursuer and Evader dynamics in the pursuit–evasion game are introduced in Section 2. In Section 3, a switchable pursuit–evasion strategy based on linear quadratic and zero-effort miss distance is designed. Section 4 details the interactive multiple-model learning filtering method based on an LSTM network. In Section 5, the proposed LSTM-IMML method is compared with IMM to validate the performance of the proposed method. Finally, Section 6 concludes the paper.

2. Dynamics of Spacecraft Pursuit–Evasion

To describe the maneuver game of two players, a reference spacecraft which is very close to the Pursuer and Evader is selected as the origin of the reference coordinate system oxyz as shown in Figure 1. The coordinate systems OXYZ and oxyz represent the Earth inertial and the reference coordinate systems, respectively. The axis ox points from the Earth’s center to the center of mass of the reference spacecraft; the axis oy points to the velocity direction, and the oz axis completes the right-hand rule.

Assuming the orbit of the reference spacecraft is circular and the two satellites maneuver near the reference spacecraft, the dynamics of the Pursuer and Evader in the reference coordinate system oxyz can be simplified to the HCW equation.

$$\begin{array}{cc}\left\{\begin{array}{l}{\ddot{x}}_i=3{\omega }^2{x}_i+2\omega {\dot{y}}_i+u_{x_i}\\ {\ddot{y}}_i=-2\omega {\dot{x}}_i+u_{y_i}\\ {\ddot{z}}_i=-{\omega }^2{z}_i+u_{z_i}\end{array}\right.& i=P,E\end{array}$$

(1)

where, subscript P represents the Pursuer and subscript E represents the Evader, $x_i,y_i,z_i$ represent the positions of the Pursuer and the Evader in the reference coordinate system oxyz, ${\dot{x}}_i,{\dot{y}}_i,{\dot{z}}_i$ denote the velocities, $\omega$ is the orbital angular velocity of the reference spacecraft, $u_{x_i},u_{y_i},u_{z_i}$ represent the thrust acceleration in three axes.

Defining state vector $\mathit{X}={\left[\begin{array}{llllll}x\hfill & y\hfill & z\hfill & \dot{x}\hfill & \dot{y}\hfill & \dot{z}\hfill \end{array}\right]}^T$ and control vector $\mathit{U}={\left[\begin{array}{ccc}{u}_x& u_y& u_z\end{array}\right]}^T$, Equation (1) can be written

$${\dot{\mathit{X}}}_i=\mathit{A}{\mathit{X}}_i+\mathit{B}{\mathit{U}}_i$$

(2)

where

$$\mathit{A}=\left[\begin{array}{cccc}\begin{array}{ccc}\begin{array}{l}0\\ 0\\ 0\\ 3{\omega }^2\\ 0\\ 0\end{array}& \begin{array}{l}0\\ 0\\ 0\\ 0\\ 0\\ 0\end{array}& \begin{array}{l}0\\ 0\\ 0\\ 0\\ 0\\ {\omega }^2\end{array}\end{array}& \begin{array}{l}1\\ 0\\ 0\\ 0\\ -2\omega \\ 0\end{array}& \begin{array}{l}0\\ 1\\ 0\\ 2\omega \\ 0\\ 0\end{array}& \begin{array}{l}0\\ 0\\ 1\\ 0\\ 0\\ 0\end{array}\end{array}\right],\mathit{B}=\left[\begin{array}{ccc}\begin{array}{l}0\\ 0\\ 0\\ 1\\ 0\\ 0\end{array}& \begin{array}{l}0\\ 0\\ 0\\ 0\\ 1\\ 0\end{array}& \begin{array}{l}0\\ 0\\ 0\\ 0\\ 0\\ 1\end{array}\end{array}\right]$$

Then, the state space equations of the Pursuer and Evader can be obtained as

$$\begin{array}{l}{\dot{\mathit{X}}}_P=\mathit{A}{\mathit{X}}_P+\mathit{B}{\mathit{U}}_P\\ {\dot{\mathit{X}}}_E=\mathit{A}{\mathit{X}}_E+\mathit{B}{\mathit{U}}_E\end{array}$$

(3)

where subscript P represents the Pursuer and subscript E represents the Evader. In the reference coordinate system, the dynamics of the pursuit–evasion game are the difference between the dynamics of the Pursuer and the Evader. Defining ${\mathit{X}}_{PE}={\mathit{X}}_P-{\mathit{X}}_E$, the relative dynamics can be written as

$${\dot{\mathit{X}}}_{PE}=\mathit{A}{\mathit{X}}_{PE}+\mathit{B}{\mathit{U}}_P-\mathit{B}{\mathit{U}}_E$$

(4)

Considering the actual thruster limitation of the satellite, it is assumed that the constraints of thrust acceleration amplitude are

$$\left\{\begin{array}{l}‖{\mathit{U}}_P‖\le {\rho }_P\\ ‖{\mathit{U}}_E‖\le {\rho }_E\end{array}\right.$$

(5)

Using the control strategy that satisfies Equation (5), the Pursuer and Evader compete for the terminal distance. The goal of the Pursuer is to intercept the Evader in the shortest time, while the Evader expects to avoid the interception. The terminal interception set is defined as

$$\psi =\left\{{\mathit{X}}_{PE}:‖{\mathit{r}}_{PE}‖=‖{\mathit{r}}_P-{\mathit{r}}_E‖\le d_c\right\}$$

(6)

where ${\mathit{r}}_P$ and ${\mathit{r}}_E$ represent the position vectors of the Pursuer and the Evader, respectively. The Pursuer wants to use control ${\mathit{U}}_P$ to make the relative state enter the set $\psi$ as soon as possible, while the Evader expects to avoid this by applying control ${\mathit{U}}_E$.

3. Game Strategy Switch

In the pursuit–evasion game scenario, the Evader expects to increase the relative distance and quickly maneuver away from the Pursuer during the approach process. However, it will easily be intercepted by the Pursuer if the Evader adopts a fixed evasion strategy and this leads to the game strategy switch of the Evader. As the relative distance changes, the Evader will adjust evasion strategies in due time. This paper assumes that two game strategies are adopted, one is the linear quadratic game in consideration of fuel consumption, and the other is the zero-effort miss game strategy with maximum thrust.

3.1. Linear Quadratic Game Strategy

Linear quadratic differential game theory is widely used in pursuit–evasion problems. As described in [23], the objective function is first constructed, which is the quadratic function of the state difference and the control vector of the Pursuer and the Evader. The objective function of the Pursuer is

$$J_P={\displaystyle \frac{1}{2}}{\mathit{X}}_{PE}^T\left(t_f\right){\mathit{Q}}_f{\mathit{X}}_{PE}\left(t_f\right)+{\displaystyle \frac{1}{2}}{\displaystyle \underset{t_0}{\overset{t_f}{\int }}\left({\mathit{X}}_{PE}^T\left(t\right)\mathit{Q}{\mathit{X}}_{PE}\left(t\right)+{\mathit{U}}_P^T\left(t\right){\mathit{R}}_P{\mathit{U}}_P\left(t\right)-{\mathit{U}}_E^T\left(t\right){\mathit{R}}_E{\mathit{U}}_E\left(t\right)\right)dt}$$

(7)

Due to the opposite goals of the Pursuer and the Evader, the designed objective function is also opposite. Then the objective function of the Pursuer can be represented as

$$J_E=-J_P$$

(8)

where, $\mathit{Q}$ is a positive semi-definite matrix and ${\mathit{Q}}_f,{\mathit{R}}_P,{\mathit{R}}_E$ are positive definite matrices. The conditions for the control sets ${\mathit{U}}_P^{*}\left(t\right)$ and ${\mathit{U}}_E^{*}\left(t\right)$ of the Pursuer and the Evader to arrive at the saddle point solution of the game are

$$\begin{array}{l}{J}_{\mathrm{E}}\left({\mathbf{U}}_{\mathrm{P}}^{\ast },{\mathbf{U}}_{\mathrm{E}}^{\ast }\right)\le J_{\mathrm{E}}\left({\mathbf{U}}_{\mathrm{P}}^{\ast },{\mathbf{U}}_{\mathrm{E}}\right)\hfill \\ J_{\mathrm{P}}\left({\mathbf{U}}_{\mathrm{P}}^{\ast },{\mathbf{U}}_{\mathrm{E}}^{\ast }\right)\le J_{\mathrm{P}}\left({\mathbf{U}}_{\mathrm{P}},{\mathbf{U}}_{\mathrm{E}}^{\ast }\right)\hfill \end{array}$$

(9)

Based on linear quadratic differential game theory, the optimal feedback control law of both sides can be derived [23].

$${\mathit{U}}_P^{*}\left(t\right)=-{\mathit{R}}_P^{-1}{\mathit{B}}^T\mathit{P}\left(t\right){\mathit{X}}_{PE}\left(t\right)$$

(10)

$${\mathit{U}}_E^{*}\left(t\right)=-{\mathit{R}}_E^{-1}{\mathit{B}}^T\mathit{P}\left(t\right){\mathit{X}}_{PE}\left(t\right)$$

(11)

where, P is a symmetric matrix obtained by solving the algebraic Riccati equation reversely, which satisfies

$$\begin{array}{cc}\dot{\mathit{P}}=-{\mathit{A}}^T\mathit{P}-\mathit{P}\mathit{A}-\mathit{Q}+\mathit{P}\left(\mathit{B}{\mathit{R}}_P^{-1}{\mathit{B}}^T-\mathit{B}{\mathit{R}}_E^{-1}{\mathit{B}}^T\right)\mathit{P}& \mathit{P}\left(t_f\right)\end{array}={\mathit{Q}}_f$$

(12)

3.2. Zero-Effort Miss Game Strategy

When the pursuit–evasion game comes to the final stage, the Pursuer and Evader compete for the distance between them. In this case, only the relative distance of the two satellites is considered without considering fuel consumption, and the zero-effort miss ${\mathit{Z}}_{PE}$ is introduced.

$${\mathit{Z}}_{PE}=\mathit{D}\mathit{\Phi }\left(t_f,t\right){\mathit{X}}_{PE}=\left[\begin{array}{cc}{\mathit{I}}_3& 0_3\end{array}\right]\mathit{\Phi }\left(t_f,t\right){\mathit{X}}_{PE}$$

(13)

where, $\mathit{D}=\left[\begin{array}{cc}{\mathit{I}}_3& 0_3\end{array}\right]$ and $\mathit{\Phi }\left(t_f,t\right)=e^{\mathit{A}\left(t_f-t\right)}$ are the state transition matrix of system (2) from $t$ to $t_f$, as shown in [10], which satisfies

$$\dot{\mathit{\Phi }}\left(t_f,t\right)=-\mathit{\Phi }\left(t_f,t\right)\mathit{A}$$

(14)

By taking the derivative of zero-effort miss, we can obtain

$${\dot{\mathit{Z}}}_{PE}=\mathit{D}\left(\dot{\mathit{\Phi }}\left(t_f,t\right){\mathit{X}}_{PE}+\mathit{\Phi }\left(t_f,t\right){\dot{\mathit{X}}}_{PE}\right)={\mathit{B}}_P{\mathit{U}}_P-{\mathit{B}}_E{\mathit{U}}_E$$

(15)

where, ${\mathit{B}}_P={\mathit{B}}_E=\mathit{D}\mathit{\Phi }\left(t_f,t\right)\mathit{B}={\mathit{\Phi }}_{12}\left(t_f,t\right)$.

In the pursuit–evasion process, the Pursuer expects to reduce the zero-effort miss, while the Evader expects to increase the zero-effort miss in a way that is beneficial to itself as much as possible, so the zero-effort miss as the objective function is defined as

$$\underset{{\mathit{U}}_P}{\min } \underset{{\mathit{U}}_E}{\max } J=‖{\mathit{Z}}_{PE}\left(t_f\right)‖$$

(16)

According to the derivation process in reference [10], the control strategy of the Pursuer under maximum thrust acceleration can be obtained as

$${\mathit{U}}_P=-{\rho }_P{\displaystyle \frac{{\left(\frac{{\mathit{Z}}_{PE}^T\left(t\right)}{‖{\mathit{Z}}_{PE}\left(t\right)‖}{\mathit{B}}_P\right)}^T}{‖\frac{{\mathit{Z}}_{PE}^T\left(t\right)}{‖{\mathit{Z}}_{PE}\left(t\right)‖}{\mathit{B}}_P‖}}=-{\rho }_P{\displaystyle \frac{{\mathit{B}}_P^T{\mathit{Z}}_{PE}\left(t\right)}{‖{\mathit{B}}_P^T{\mathit{Z}}_{PE}\left(t\right)‖}}$$

(17)

where ${\rho }_P$ is the maximum thrust acceleration amplitude of the Pursuer.

In the same way, the control strategy of the Evader under maximum thrust acceleration is

$${\mathit{U}}_E=-{\rho }_E{\displaystyle \frac{{\left(\frac{{\mathit{Z}}_{PE}^T\left(t\right)}{‖{\mathit{Z}}_{PE}\left(t\right)‖}{\mathit{B}}_E\right)}^T}{‖\frac{{\mathit{Z}}_{PE}^T\left(t\right)}{‖{\mathit{Z}}_{PE}\left(t\right)‖}{\mathit{B}}_E‖}}=-{\rho }_E{\displaystyle \frac{{\mathit{B}}_E^T{\mathit{Z}}_{PE}\left(t\right)}{‖{\mathit{B}}_E^T{\mathit{Z}}_{PE}\left(t\right)‖}}$$

(18)

where ${\rho }_E$ is the maximum thrust acceleration amplitude of the Evader.

3.3. The Design of Switchable Pursuit-Evasion Strategy

The pursuit–evasion strategy based on the linear quadratic method takes into consideration the fuel consumption, so it is suitable for long-distance pursuit and evasion. In addition, the thrust acceleration according to the linear quadratic feedback control law is related to ${\mathit{R}}_E$ and the relative state ${\mathit{X}}_{PE}$, and the thrust acceleration of the Evader will decrease as the distance between the two satellites decreases. Hence, the Evader will increase its thrust acceleration by switching the parameters of linear quadratic strategy. That is, when the Pursuer approaches the Evader, the Evader increases the output thrust acceleration by decreasing ${\mathit{R}}_E$.

When the distance between the Pursuer and the Evader is reduced to the warning range of the Evader, the best option for the Evader is to maneuver away from the Pursuer with a maximum thrust amplitude. That is, the Evader will switch to the zero-effort miss evasion strategy regardless of fuel consumption.

Based on the above analysis, the evasion strategy of the Evader can be designed. Suppose that the Evader has $M$ evasion modes. Firstly, the Evader switches between different linear quadratic strategies by changing ${\mathit{R}}_E$ in the former $M-1$ strategies, and then, finally, the Evader switches to the zero-effort miss strategy of the M-th strategy. The Evader has $M$ evasion modes, the former $M-1$ strategies are linear quadratic evasion strategies, and the M-th is a zero-effort miss evasion strategy, which can be expressed as

$${\mathit{U}}_E=\left\{\begin{array}{ll}-{\mathit{R}}_{E_1}^{-1}{\mathit{B}}^T{\mathit{P}}_1{\mathit{X}}_{PE}\hfill & ‖{\mathit{r}}_{PE}‖>d_{P{E}_1}\hfill \\ ⋮\hfill & ⋮\hfill \\ -{\mathit{R}}_{E_{M-1}}^{-1}{\mathit{B}}^T{\mathit{P}}_{M-1}{\mathit{X}}_{PE}\hfill & ‖{\mathit{r}}_{PE}‖\ge d_{P{E}_{M-1}}\&‖{\mathit{r}}_{PE}‖<d_{P{E}_{M-2}}\hfill \\ -{\rho }_E{\displaystyle \frac{{\mathit{B}}_E^T{\mathit{Z}}_{PE}\left(t\right)}{‖{\mathit{B}}_E^T{\mathit{Z}}_{PE}\left(t\right)‖}}\hfill & ‖{\mathit{r}}_{PE}‖\le d_{P{E}_{M-1}}\hfill \end{array}\right.$$

(19)

where $‖{\mathit{r}}_{PE}‖$ represents the distance between the two spacecraft and $d_{P{E}_i},i=1,2,\dots ,M-1$ represents the strategy switch boundary of the Evader.

The Pursuer will also switch its strategy after the Evader performs a strategy switch. Therefore, the Pursuer’s strategy can be designed as

$${\mathit{U}}_P=\left\{\begin{array}{ll}-{\mathit{R}}_{P_1}^{-1}{\mathit{B}}^T{\mathit{P}}_1{\mathit{X}}_{PE}\hfill & {\mathit{U}}_E={\mathit{U}}_{E_1}\hfill \\ ⋮\hfill & ⋮\hfill \\ -{\mathit{R}}_{P_{M-1}}^{-1}{\mathit{B}}^T{\mathit{P}}_{M-1}{\mathit{X}}_{PE}\hfill & {\mathit{U}}_E={\mathit{U}}_{E_{M-1}}\hfill \\ -{\rho }_P{\displaystyle \frac{{\mathit{B}}_E^T{\mathit{Z}}_{PE}\left(t\right)}{‖{\mathit{B}}_E^T{\mathit{Z}}_{PE}\left(t\right)‖}}\hfill & {\mathit{U}}_E={\mathit{U}}_{E_M}\hfill \end{array}\right.$$

(20)

However, in the case of incomplete information, the Pursuer does not know the strategy adopted by the Evader. Hence, the difficulty of strategy switching for the Pursuer lies in the estimation of the evasion strategy used by the Evader.

4. Strategy Estimation Method

To maneuver away from the Pursuer more effectively, the Evader will actively switch evasion strategy, which requires the Pursuer to estimate the evasion strategy in real time, and then change to its appropriate pursuit strategy. In this section, an interactive multiple-model learning filtering combined with an LSTM neural network is proposed. Feedback learning filters are used to estimate the state of the Evader, and then the evasion strategy is estimated by the LSTM network. Afterwards, the Pursuer switches to an appropriate pursuit strategy based on the estimated evasion strategy to intercept the Evader.

4.1. IMM-Based Strategy Switch Method

The multiple-model idea of the strategy switch of the pursuit–evasion game is to map the possible evasion strategy of the Evader into a model set, where each model corresponds to an evasion strategy. At the same time, multiple filters are used, working in parallel to estimate the state of each model. Then, the evasion strategy of the Evader is obtained by calculating the effective probability of each model.

The strategy switch method based on IMM is mainly divided into the following four steps:

• Step 1: Reinitialization of Model Condition

The dynamics of the pursuit–evasion game between two spacecraft satisfy the Markov process, and the transition probability is

$${\pi }_{ij}=\mathrm{Pr}\left\{r_k=j|r_k=i\right\}$$

(21)

where $r_k$ represents the system mode at time k.

According to the estimation of each filter in the previous moment, the inputs of the filter corresponding to the j-th model at the current moment should be calculated first, these being mixed probability, mixed state estimation, and the corresponding error covariance matrix.

Mixed probability can be expressed as

$${\mu }_{k-1|k-1}^{i|j}=\mathrm{Pr}\{r_{k-1}=i|r_k=j,{\mathit{Z}}_{k-1}\}={\displaystyle \frac{1}{c_j}}{\pi }_{ij}{\mu }_{k-1}^i$$

(22)

where $c_j={\displaystyle \sum _{i=1}^M{\pi }_{ij}{\mu }_{k-1}^i}$ is constant, ${\mu }_{k-1}^i$ is the probability of matching model i at time $k-1$, ${\pi }_{ij}$ represents the transition probability from model i to model j.

Mixed state estimation and the corresponding error covariance matrix are given as

$${\widehat{\mathit{x}}}_{k-1|k-1}^{0j}=E\left\{{\mathit{x}}_{k-1}|r_k=j,{\mathit{Z}}_{k-1}\right\}={\displaystyle \sum _{i=1}^M{\mu }_{k-1|k-1}^{ij}{\widehat{\mathit{x}}}_{k-1|k-1}^i}$$

(23)

$$\begin{array}{l}{\mathit{P}}_{k-1|k-1}^{0j}=E\left\{({\mathit{x}}_{k-1}-{\widehat{\mathit{x}}}_{k-1|k-1}^{0j}){(x_{k-1}-{\widehat{\mathit{x}}}_{k-1|k-1}^{0j})}^T|r_k=j,{\mathit{Z}}_{k-1}\right\}\\ ={\displaystyle \sum _{i=1}^M{\mu }_{k-1|k-1}^{ij}\left[{\mathit{P}}_{k-1|k-1}^{ij}+({\widehat{\mathit{x}}}_{k-1|k-1}^i-{\widehat{\mathit{x}}}_{k-1|k-1}^{0j}){({\widehat{\mathit{x}}}_{k-1|k-1}^i-{\widehat{\mathit{x}}}_{k-1|k-1}^{0j})}^T\right]}\end{array}$$

(24)

where ${\widehat{\mathit{x}}}_{k-1|k-1}^i$ and ${\mathit{P}}_{k-1|k-1}^i$ are the state estimation and error covariance matrix of the i-th model filter at time k − 1, respectively.

• Step 2: Multiple-Model Filtering

According to different evasion strategies, the corresponding pursuit–evasion strategy model is constructed.

For the pursuit–evasion model with a linear quadratic evasion strategy, it can be expressed as

$${\dot{\mathit{X}}}_{PE}={\mathit{\Gamma }}_j{\mathit{X}}_{PE}+\mathit{B}{\mathit{U}}_P=\mathit{A}{\mathit{X}}_{PE}+\mathit{B}{\mathit{R}}_{E_j}^{-1}{\mathit{B}}^T\mathit{P}{\mathit{X}}_{PE}+\mathit{B}{\mathit{U}}_P$$

(25)

where ${\mathit{\Gamma }}_j=\mathit{A}+\mathit{B}{\mathit{R}}_{E_j}^{-1}{\mathit{B}}^T\mathit{P}$ and $j=1,\dots ,M-1$ denotes the $M-1$ linear quadratic pursuit strategy.

By discretizing the above equation, the state transition equation in discrete form can be obtained.

$${\mathit{X}}_{P{E}_k}={\mathit{F}}_{k|k-1}^j{\mathit{X}}_{P{E}_{k-1}}+{\mathit{B}}_{F_j}{\mathit{U}}_P+{\mathit{W}}_{k-1}$$

(26)

where ${\mathit{W}}_{k-1}$ is zero-mean Gaussian white noise and represents the process noise sequence.

For the pursuit–evasion model with a zero-effort miss evasion strategy, it can be expressed as

$$\begin{array}{l}{\dot{\mathit{X}}}_{PE}={\mathit{\Gamma }}_M{\mathit{X}}_{PE}+\mathit{B}{\mathit{U}}_P=\mathit{A}{\mathit{X}}_{PE}-{\rho }_E\mathit{B}{\displaystyle \frac{{\mathit{B}}_E^T{\mathit{Z}}_{PE}\left(t\right)}{‖{\mathit{B}}_E^T{\mathit{Z}}_{PE}\left(t\right)‖}}+\mathit{B}{\mathit{U}}_P\\ =\left(\mathit{A}+{\rho }_E\mathit{B}{\displaystyle \frac{{\mathit{B}}_E^T\mathit{D}\mathit{\Phi }\left(t_f,t\right)}{‖{\mathit{B}}_E^T{\mathit{Z}}_{PE}\left(t\right)‖}}\right){\mathit{X}}_{PE}+\mathit{B}{\mathit{U}}_P\end{array}$$

(27)

Similarly, the state transition equation can be obtained by discretizing the above equation

$${\mathit{X}}_{P{E}_k}={\mathit{F}}_{k|k-1}^M{\mathit{X}}_{P{E}_{k-1}}+{\mathit{B}}_{F_M}{\mathit{U}}_P+{\mathit{W}}_{k-1}$$

(28)

In this way, the state transition matrix under the two game strategies can be acquired after receiving the new measurement information. Then, a Kalman filter is used to update the state estimation of the corresponding matching model j based on the new measurement information, which includes

$$\begin{array}{l}{\widehat{\mathit{x}}}_{k|k-1}^j={\mathit{F}}_{k-1}^j{\widehat{\mathit{x}}}_{k-1|k-1}^{0j}+{\mathit{B}}_F^j{\mathit{U}}_{P_{k-1}}\\ {\mathit{P}}_{k|k-1}^j={\mathit{F}}_{k-1}^j{\mathit{P}}_{k-1|k-1}^{0j}{\left({\mathit{F}}_{k-1}^j\right)}^T+{\mathit{Q}}_{k-1}^j\\ {\widehat{\mathit{z}}}_{k|k-1}^j={\mathit{H}}_{k-1}^j{\widehat{\mathit{x}}}_{k|k-1}^j\\ {\mathit{S}}_k^j={\mathit{H}}_k^j{\mathit{P}}_{k|k-1}^j{\left({\mathit{H}}_k^j\right)}^T+{\mathit{R}}_k^j\\ {\mathit{K}}_k^j={\mathit{P}}_{k|k-1}^j{\left({\mathit{H}}_k^j\right)}^T{\left({\mathit{S}}_k^j\right)}^{-1}\\ {\widehat{\mathit{x}}}_{k|k-1}^j={\widehat{\mathit{x}}}_{k|k-1}^j+{\mathit{K}}_k^j({\mathit{z}}_k-{\widehat{\mathit{z}}}_{k|k-1}^j)\\ {\mathit{P}}_{k|k}^j={\mathit{P}}_{k|k-1}^j-{\mathit{K}}_k^j{\mathit{S}}_k^j{\left({\mathit{K}}_k^j\right)}^T\end{array}$$

(29)

where superscript $j=1,2,\dots ,M$ represents the j-th filter corresponding to the j-th pursuit–evasion model, ${\mathit{Q}}_{k-1}$ and ${\mathit{R}}_k$ are the process noise covariance and measurement noise covariance, ${\mathit{H}}_{k-1}$ is the measurement matrix, and ${\mathit{z}}_k$ is the measured value at time k.

• Step 3: Model probability update and pursuit strategy switch

For the model $j=1,2,\dots ,M$, the model posterior probability at time k is calculated

$${\mu }_k^j=\mathrm{Pr}\left\{r_k=j|{\mathit{Z}}_k\right\}={\displaystyle \frac{c_j{\mathrm{\Lambda }}_k^j}{{\displaystyle \sum _{i=1}^M{c}_j{\mathrm{\Lambda }}_k^j}}}$$

(30)

where ${\mathrm{\Lambda }}_k^j=N\left({\mathit{z}}_k;{\widehat{\mathit{z}}}_{k|k-1}^j,{\mathit{S}}_k^j\right)$ is the Gaussian likelihood function of the j-th model.

$$N\left({\mathit{z}}_k;{\widehat{\mathit{z}}}_{k|k-1}^j,{\mathit{S}}_k^j\right)\stackrel{def}{=}{\displaystyle \frac{1}{\sqrt{\det \left(2\pi {\mathit{S}}_k^j\right)}}}\exp \left[-{\displaystyle \frac{1}{2}}{\left({\mathit{z}}_k-{\widehat{\mathit{z}}}_{k|k-1}^j\right)}^T{\left({\mathit{S}}_k^j\right)}^T\left({\mathit{z}}_k-{\widehat{\mathit{z}}}_{k|k-1}^j\right)\right]$$

(31)

Obviously, the posterior probability at step k is satisfied

$${\displaystyle \sum _{j=1}^M{\mu }_k^j}=1$$

(32)

According to the posterior probability of each model, the evasion strategy corresponding to the model with the highest probability is the strategy adopted by the Evader.

$${\mathit{U}}_E={\mathit{U}}_E^j,j=\arg \max {\mu }_k^j$$

(33)

Then, as shown in Equation (20), the Pursuer chooses the corresponding pursuit strategy according to the evasion strategy.

• Step 4: Estimate fusion

Based on the output of each sub-filter, the overall estimate and estimation error covariance matrix at time $k$ can be calculated

$${\widehat{\mathit{x}}}_{k|k}=E\left\{{\mathit{x}}_k|{\mathit{Z}}_k\right\}={\displaystyle \sum _{j=1}^M{\mu }_k^j{\widehat{\mathit{x}}}_{k|k}^j}$$

(34)

$${\mathit{P}}_{k|k}=E\left\{\left({\mathit{x}}_k-{\widehat{\mathit{x}}}_{k|k}\right){\left({\mathit{x}}_k-{\widehat{\mathit{x}}}_{k|k}\right)}^T|{\mathit{Z}}_k\right\}={\displaystyle \sum _{j=1}^M{\mu }_k^j\left[{\mathit{P}}_{k|k}^j+\left({\widehat{\mathit{x}}}_{k|k}^j-{\widehat{\mathit{x}}}_{k|k}\right){\left({\widehat{\mathit{x}}}_{k|k}^j-{\widehat{\mathit{x}}}_{k|k}\right)}^T\right]}$$

(35)

4.2. Interactive Multiple-Model Feedback Learning Filter

In the IMM method, the final fusion estimate is only the output, with no feedback for the estimation of the next state. However, the final fusion tends to be more accurate than the model-dependent estimates, so the final fusion can be used as a reference for next state estimation. On this basis, an interactive multiple-model filter based on fusion estimation feedback learning is proposed.

First, the feedback learning term is defined based on the IMM method, namely

$$l_k^j={\widehat{\varphi }}_k^j-{\widehat{\mathit{x}}}_{k|k-1}^j={\mathit{F}}_{k-1}^j\left({\widehat{\mathit{x}}}_{k-1|k-1}-{\widehat{\mathit{x}}}_{k-1|k-1}^j\right)$$

(36)

where ${\widehat{\varphi }}_k^j$ is obtained from the overall estimate at time $k-1$ through the system state transition matrix, i.e.,

$${\widehat{\varphi }}_k^j={\mathit{F}}_{k-1}^j{\widehat{\mathit{x}}}_{k-1|k-1}$$

(37)

where ${\widehat{\mathit{x}}}_{k-1|k-1}$ is the total estimate at time $k-1$, calculated by Equation (34).

The feedback learning term is used for state estimation in the next moment; that is,

$${\widehat{\mathit{x}}}_{k|k}^j={\widehat{\mathit{x}}}_{k|k-1}^j+{\mathit{K}}_k^j\left({\mathit{z}}_k-{\mathit{H}}_k^j{\widehat{\mathit{x}}}_{k|k-1}^j\right)+{\epsilon }_k^j{\mathit{l}}_k^j$$

(38)

where ${\epsilon }_k^j$ is the gain constant of the feedback learning term.

Due to the introduction of the feedback learning term, the gain matrix ${\mathit{K}}_k^j$ needs to be redesigned. The error is defined as follows.

$$\begin{array}{l}{\mathit{e}}_k^j={\mathit{x}}_k-{\widehat{\mathit{x}}}_{k|k}^j\\ {\mathit{\eta }}_k^j={\mathit{x}}_k-{\widehat{\mathit{x}}}_{k|k-1}^j\\ {\mathit{\delta }}_k^j=x_k-{\widehat{\varphi }}_k^j\end{array}$$

(39)

According to the updated estimation equation, the relationship between the three errors can be represented as

$$\begin{array}{l}{\mathit{e}}_k^j={\mathit{\eta }}_k^j-{\mathit{K}}_k^j{\mathit{H}}_k^j{\mathit{\eta }}_k^j-{\mathit{K}}_k^j{\mathit{v}}_k^j-{\epsilon }_k^j\left({\mathit{\eta }}_k^j-{\mathit{\delta }}_k^j\right)\\ =\left(\mathit{I}-{\mathit{K}}_k^j{\mathit{H}}_k^j-{\epsilon }_k^j\right){\mathit{\eta }}_k^j-{\mathit{K}}_k^j{\mathit{v}}_k^j+{\epsilon }_k^j{\mathit{\delta }}_k^j\end{array}$$

(40)

The corresponding covariance matrix is

$$\begin{array}{l}{\mathit{P}}_{k|k}^j=E\left\{{\mathit{e}}_k^j{\left({\mathit{e}}_k^j\right)}^T\right\}\\ =\left(\mathit{I}-{\mathit{K}}_k^j{\mathit{H}}_k^j-{\epsilon }_k^j\mathit{I}\right){\mathit{P}}_{k|k-1}^j\left(\mathit{I}-{\mathit{K}}_k^j{\mathit{H}}_k^j-{\epsilon }_k^j\mathit{I}\right)+\\ {\mathit{K}}_k^j{\mathit{R}}_k^j{\left({\mathit{K}}_k^j\right)}^T+{\left({\epsilon }_k^j\right)}^2{\mathit{\Phi }}_k^j+{\epsilon }_k^j\left(\mathit{I}-{\mathit{K}}_k^j{\mathit{H}}_k^j-{\epsilon }_k^j\mathit{I}\right){\sum }_k^j+\\ {\epsilon }_k^j{\left({\sum }_k^j\right)}^T{\left(\mathit{I}-{\mathit{K}}_k^j{\mathit{H}}_k^j-{\epsilon }_k^j\mathit{I}\right)}^T\end{array}$$

(41)

where $\mathit{I}$ is the identity matrix and

$$\begin{array}{l}{\mathit{\Phi }}_k^j=E\left\{{\mathit{\delta }}_k^j{\left({\mathit{\delta }}_k^j\right)}^T\right\}={\mathit{F}}_{k-1}^j{\mathit{P}}_{k-1|k-1}{\left({\mathit{F}}_{k-1}^j\right)}^T+{\mathit{Q}}_{k-1}^j\\ {\sum }_k^j=E\left\{{\mathit{\eta }}_k^j{\left({\mathit{\delta }}_k^j\right)}^T\right\}={\mathit{F}}_{k-1}^j{\displaystyle \sum _{i=1}^M{\mu }_{k-1}^{i}E\left\{{\mathit{e}}_{k-1}^j{\left({\mathit{e}}_{k-1}^i\right)}^T\right\}}{\left({\mathit{F}}_{k-1}^j\right)}^T+{\mathit{Q}}_{k-1}^j\end{array}$$

(42)

The optimal gain matrix ${\mathit{K}}_k^j$ can be obtained by minimizing the covariance matrix trace, i.e.,

$${\displaystyle \frac{\mathsf{\partial }tr\left\{{\mathit{P}}_{k|k}^j\right\}}{\mathsf{\partial }{\mathit{K}}_k^j}}=0$$

(43)

Using the matrix calculation theory, any matrix satisfies

$$\begin{array}{l}{\displaystyle \frac{\mathsf{\partial }tr\left\{\mathit{X}\mathit{Y}\right\}}{\mathsf{\partial }\mathit{Y}}}={\mathit{X}}^T\\ {\displaystyle \frac{\mathsf{\partial }tr\left\{{\mathit{X}}^T\mathit{Y}\mathit{X}\right\}}{\mathsf{\partial }\mathit{X}}}=\left(\mathit{Y}+{\mathit{Y}}^T\right)\mathit{X}\end{array}$$

(44)

Hence, we can obtain

$${\mathit{K}}_k^j{\mathit{H}}_k^j{\mathit{P}}_{k|k-1}^j{\left({\mathit{H}}_k^j\right)}^T-\left(1-{\epsilon }_k^j\right){\mathit{P}}_{k|k-1}^j{\left({\mathit{H}}_k^j\right)}^T+{\mathit{K}}_k^j{\mathit{R}}_k^j-{\epsilon }_k^j{\sum }_k^j{\left({\mathit{H}}_k^j\right)}^T=0$$

(45)

The optimal gain matrix is

$${\mathit{K}}_k^j=\left[\left(1-{\epsilon }_k^j\right){\mathit{P}}_{k|k-1}^j{\left({\mathit{H}}_k^j\right)}^T+{\epsilon }_k^j{\sum }_k^j{\left({\mathit{H}}_k^j\right)}^T\right]{\left[{\mathit{H}}_k^j{\mathit{P}}_{k|k-1}^j{\left({\mathit{H}}_k^j\right)}^T+{\mathit{R}}_k^j\right]}^{-1}$$

(46)

Thus, the Kalman filter in Step 2 can be replaced by

$$\begin{array}{l}{\widehat{\mathit{x}}}_{k|k-1}^j={\mathit{F}}_{k-1}^j{\widehat{\mathit{x}}}_{k-1|k-1}^{0j}+{\mathit{B}}_F{\mathit{U}}_{P_{k-1}}\\ {\mathit{P}}_{k|k-1}^j={\mathit{F}}_{k-1}^j{\mathit{P}}_{k-1|k-1}^{0j}{\left({\mathit{F}}_{k-1}^j\right)}^T+{\mathit{Q}}_{k-1}^j\\ {\widehat{\varphi }}_k^j={\mathit{F}}_{k-1}^j{\widehat{\mathit{x}}}_{k-1|k-1}\\ {\mathrm{\Phi }}_k^j={\mathit{F}}_{k-1}^j{\mathit{P}}_{k-1|k-1}{\left({\mathit{F}}_{k-1}^j\right)}^T+{\mathit{Q}}_{k-1}^j\\ {\widehat{\mathit{x}}}_{k|k}^j={\widehat{\mathit{x}}}_{k|k-1}^j+{\mathit{K}}_k^j\left({\mathit{z}}_k-{\mathit{H}}_k^j{\widehat{\mathit{x}}}_{k|k-1}^j\right)+{\epsilon }_k^j\left({\widehat{\varphi }}_k^j-{\widehat{\mathit{x}}}_{k|k-1}^j\right)\\ {\mathit{S}}_k^j={\mathit{H}}_k^j{\mathit{P}}_{k|k-1}^j{\left({\mathit{H}}_k^j\right)}^T+{\mathit{R}}_k^j\\ {\mathit{P}}_{k|k}^j=\left(\mathit{I}-{\mathit{K}}_k^j{\mathit{H}}_k^j-{\epsilon }_k^j\mathit{I}\right){\mathit{P}}_{k|k-1}^j\left(\mathit{I}-{\mathit{K}}_k^j{\mathit{H}}_k^j-{\epsilon }_k^j\mathit{I}\right)+\\ {\mathit{K}}_k^j{\mathit{R}}_k^j{\left({\mathit{K}}_k^j\right)}^T+{\left({\epsilon }_k^j\right)}^2{\mathit{\Phi }}_k^j+{\epsilon }_k^j\left(\mathit{I}-{\mathit{K}}_k^j{\mathit{H}}_k^j-{\epsilon }_k^j\mathit{I}\right){\sum }_k^j+{\epsilon }_k^j{\left({\sum }_k^j\right)}^T{\left(\mathit{I}-{\mathit{K}}_k^j{\mathit{H}}_k^j-{\epsilon }_k^j\mathit{I}\right)}^T\\ {\mathit{K}}_k^j=\left[\left(1-{\epsilon }_k^j\right){\mathit{P}}_{k|k-1}^j{\left({\mathit{H}}_k^j\right)}^T+{\epsilon }_k^j{\sum }_k^j{\left({\mathit{H}}_k^j\right)}^T\right]{\left[{\mathit{H}}_k^j{\mathit{P}}_{k|k-1}^j{\left({\mathit{H}}_k^j\right)}^T+{\mathit{R}}_k^j\right]}^{-1}\end{array}$$

(47)

In this way, the interactive multiple-model feedback learning filter is obtained. In Step 2, the multiple-model feedback learning filter is used to estimate the state of the corresponding mode.

4.3. LSTM-IMML Method

RNN (Recurrent Neural Network) is a neural network used to deal with time series problems. It will memorize the previous information and apply it to the current output [24]. LSTM is a special RNN network, which is designed to solve the problem of long dependence. It can process sequence data efficiently and has been adopted in natural language processing, trajectory prediction [25], time series prediction [26], and other fields [27]. The LSTM unit can determine whether the current input is important, so long-term information is not affected by recursive operations and is stored in a more secure manner [24]. When estimating the evasion strategy, it is necessary to combine the current measurement information with the previous estimation state to improve accuracy and stability, which have a long-term dependence on information. Therefore, the LSTM network is embedded into the IMML method to estimate the mode probability of the evasion strategy in this section.

The LSTM cell structure is shown in Figure 2. $O_t$, $i_t$, and $C_t$ are the states of output, input, and the cell state in the t-th epoch, respectively. A standard LSTM unit includes a forget gate, an input gate, and an output gate [28]. The forget gate is used to determine which information will be forgotten from the cell state, the input gate determines whether new information can be kept in the cell state, and the output gate determines which information will be output. The mathematical expressions of the forget gate, input gate, and output gate can be seen in [27].

In this paper, the LSTM network is mainly used to estimate the possible mode probability of the Evader based on the state information output by the multiple-model feedback filter. As shown in Figure 3, a probability estimation neural network based on LSTM is established.

As illustrated in Figure 3, the LSTM-based probability estimation network includes an input layer, two LSTM layers, two fully connected layers, and an output layer. The number of neurons in each layer is shown in Figure 3. A dropout layer is added after each LSTM layer to prevent the LSTM network from overfitting and to enhance the generalization ability of the network [29]. The dropout probability of the dropout layer is set as 10%. During training, the dropout layer discards the activation value of the LSTM neurons with a probability of 10%, thereby improving the generalization ability of the network.

The framework of the proposed LSTM-IMML method for the Evader’s strategy estimation is shown in Figure 4. The method consists of two parts. The top block represents the offline training process and the bottom block represents the online estimation process. During training, the training data sets and the training test sets are first constructed according to the IMM method, and then the training is completed on the ground computer. After the training of the LSTM network is completed, the LSTM network can be applied in the actual space pursuit–evasion game scenario without online training.

The steps of using the LSTM-IMML algorithm to estimate the evasion strategy are detailed below.

(1).

Based on the last filter estimation, calculate the mixing probability ${\mathit{\mu }}_{k-1|k-1}$, the mixing state estimation ${\widehat{\mathit{x}}}_{k-1|k-1}^{0j}$, and the mixing error covariance matrix ${\mathit{P}}_{k-1|k-1}^{0j}$;

(2).

Use multiple feedback learning filters to estimate and update the state of each model based on the new measurement information ${\mathit{z}}_k$;

(3).

Take the measured residual ${\mathit{z}}_k-{\widehat{\mathit{z}}}_{k|k-1}^j$ of the filter as the input, adopt the trained LSTM network to calculate each mode probability ${\mathit{\mu }}_k$;

(4).

Estimate and fuse the output of each filter, then repeat step 1.

Compared with the classic IMM method, the proposed LSTM-IMML method mainly has the following improvements. (1) The IMML method was inspired by the feedback idea, and the final fusion estimation is introduced into the estimation of the next state in the pursuit–evasion game, which can effectively improve the precision of state estimation. (2) Combined with its memory function, the LSTM network is introduced to improve the estimation stability of the Evader’s evasion strategy mode and reduce fluctuations. (3) The introduction of the LSTM network avoids the error caused by singular value in model probability estimation using the IMM, which increases the robustness of the method.

5. Numerical Simulation

The pursuit–evasion game scenario is simulated to verify the effectiveness of the proposed strategy switch algorithm based on an LSTM network and multiple-model feedback learning filtering. The reference orbit is a geosynchronous orbit, the Pursuer and Evader maneuver around the reference spacecraft. The maximal thrust acceleration amplitude of the Pursuer and Evader are ${\rho }_P=0.1 {\mathrm{m}/\mathrm{s}}^2$ and ${\rho }_E=0.075 {\mathrm{m}/\mathrm{s}}^2$, respectively. The interception range is set as $d_c=20 \mathrm{m}$. The initial state of the Pursuer is ${\left[\begin{array}{llllll}500\hfill & -1500\hfill & 200\hfill & 0\hfill & 0\hfill & 0\hfill \end{array}\right]}^T$, and the initial state of the Evader is ${\left[\begin{array}{llllll}0\hfill & 0\hfill & 0\hfill & 0\hfill & 0\hfill & 0\hfill \end{array}\right]}^T$. The Evader adopts three evasion strategies, the first two are linear quadratic evasion strategies, and the third is a zero-effort miss evasion strategy.

$${\mathit{U}}_E=\left\{\begin{array}{ll}-{\mathit{R}}_{E_1}^{-1}{\mathit{B}}^T{\mathit{P}}_1{\mathit{X}}_{PE}\hfill & ‖{\mathit{r}}_{PE}‖>d_{P{E}_1}\hfill \\ -{\mathit{R}}_{E_2}^{-1}{\mathit{B}}^T{\mathit{P}}_2{\mathit{X}}_{PE}\hfill & ‖{\mathit{r}}_{PE}‖\ge d_{{PE}_2}\&‖{\mathit{r}}_{PE}‖<d_{{PE}_1}\hfill \\ -{\rho }_E{\displaystyle \frac{{\mathit{B}}_E^T{\mathit{Z}}_{PE}\left(t\right)}{‖{\mathit{B}}_E^T{\mathit{Z}}_{PE}\left(t\right)‖}}\hfill & ‖{\mathit{r}}_{PE}‖\le d_{{PE}_2}\hfill \end{array}\right.$$

(48)

where the mode switch boundary $d_{P{E}_1}=1000 \mathrm{m}, d_{P{E}_2}=500 \mathrm{m}$. In the linear quadratic pursuit–evasion strategy, ${\mathit{Q}}_f=diag\left(\left[\begin{array}{llllll}1\hfill & 1\hfill & 1\hfill & 0\hfill & 0\hfill & 0\hfill \end{array}\right]\right)$, ${\mathit{R}}_{E_1}=diag\left(\left[\begin{array}{ccc}3& 3& 3\end{array}\right]\right)\times {10}^{-7}$, ${\mathit{R}}_{E_2}=diag\left(\left[\begin{array}{ccc}2& 2& 2\end{array}\right]\right)\times {10}^{-7}$, and ${\mathit{R}}_P=diag\left(\left[\begin{array}{ccc}1.5& 1.5& 1.5\end{array}\right]\right)\times {10}^{-7}$. In the LSTM-IMML method, the Markov transition probability matrix is $\mathit{\pi }=\left[\begin{array}{ccc}0.9& 0.05& 0.05\\ 0.05& 0.9& 0.05\\ 0.05& 0.05& 0.9\end{array}\right]$, and the elements in this matrix correspond to ${\pi }_{ij}$ in Equation (21). The gain constant $\epsilon$ is 0.05.

In the feedback learning filter, the measurement noise covariance matrix is ${\mathit{R}}_k=diag\left(\left[\begin{array}{ccc}0.04& 0.04& 0.04\end{array}\right]\right)$ and the process noise covariance matrix is ${\mathit{Q}}_k=diag\left(\left[\begin{array}{llllll}1\hfill & 1\hfill & 1\hfill & 0.01\hfill & 0.01\hfill & 0.01\hfill \end{array}\right]\right)\times {10}^{-6}$. All prior model probabilities are set to the same value, i.e., the initial moment mode probability is ${\mu }_0={\left[\begin{array}{ccc}1/3& 1/3& 1/3\end{array}\right]}^T$.

Two cases are simulated in this section. One is that the Pursuer adopts a fixed pursuit strategy, and the other is that the Pursuer switches its own pursuit strategy according to estimation of the Evader’s strategy. The Evader performs a strategy switch in both cases.

5.1. The Case of a Pursuer with a Fixed Strategy

In this scenario, the Evader performs multiple strategy switches, as shown in Equation (46). The Pursuer adopts a fixed strategy, which is the linear quadratic pursuit strategy. The simulation results of pursuit–evasion are as follows.

Figure 5a shows the maneuvering trajectories of the Pursuer and the Evader, and the distance between the two satellites is depicted in Figure 5b. The simulation results indicate that the Pursuer does not intercept the Evader successfully, although the Pursuer gradually approaches the Evader at the beginning. After the distance reaches the switch boundary, the Evader switches its strategy to the zero-effort miss strategy. Then, the Evader maneuvers away from the Pursuer at maximum thrust acceleration, as shown in Figure 5d, which leads to an increase in the relative distance. Figure 5c,d show the velocity and control acceleration of the two satellites, respectively, and it is clear that the Evader adopts two strategy switches to increase the relative velocity between the Pursuer and the Evader.

From the above simulation results, we can conclude that the Pursuer with a fixed pursuit strategy may not be able to intercept the Evader with multiple switchable evasion strategies. Thus, the Pursuer also needs to switch its own strategy to intercept the Evader.

5.2. The Case of a Pursuer with Strategy Switch

In this scenario, the Pursuer performs a strategy switch to match the Evader’s strategy, and the two IMM and LSTM-IMML methods are compared.

Firstly, the constructed strategy estimation network needs to be trained offline. According to the IMM method, the strategy of the Evader is estimated, and the strategy of the Pursuer is switched to perform the pursuit–evasion game simulation. Based on this, the training data set is constructed. The maximum number of training sets is 250, the gradient threshold is set as 1, and the learning rate is 0.005. The loss value in the training process is shown in Figure 6. From the training results, the LSTM neural network tends to converge after 250 iterations, indicating that the LSTM neural network is well trained.

Under the same conditions, based on the trained LSTM neural network, the simulation results of the LSTM-IMML method are shown as follows.

Figure 7a,b depict the maneuvering trajectory and relative distance when the Pursuer adopts the LSTM-IMML method for the estimation of the evasion strategy. It is clear that the Pursuer approaches the interception range within 20 m and finally intercepts the Evader. Figure 7c shows the velocities of the Pursuer and Evader. The control acceleration of the two satellites is shown in Figure 7d, which indicates the Pursuer quickly switches strategy after the Evader switches its evasion strategy. The effectiveness of the proposed LSTM-IMML strategy switch method is verified.

Furthermore, the evasion mode probabilities of the Evader estimated by the IMM method and the LSTM-IMML method are further compared, as shown in the following.

Figure 8 shows the probability that the Pursuer uses the IMM method to estimate the evasion strategy adopted by the Evader. In general, the IMM method can estimate the evasion strategy effectively. However, when the difference between the different evasion strategies is small, the estimated probability difference is not obvious. Especially between 375 s and 420 s, the probability of mode 2 does not show an advantage, and the probability difference between mode 1 and mode 2 is weak. This is because, when the distance between the two satellites is reduced, the control outputs of the Evader’s strategy 1 and strategy 2 are similar, which leads to a weakening of the observability, and the filter cannot distinguish between the two strategy models. In addition, between 500 s and 600 s, although mode 3 is dominant, there is still a jump phenomenon.

The mode probability of the Evader estimated by the LSTM-IMML method is shown in Figure 9, which illustrates that the evasion strategy used by the Evader can be accurately estimated. Compared with the IMM method, the LSTM network is more accurate and stable in estimating mode probability. In the corresponding time interval, the mode probability corresponding to the evasion strategy estimated by the Pursuer is above 0.8. In particular, between 300 and 420 s, the output difference between the control strategies of mode 1 and mode 2 is very small, and the observability becomes weaker. However, the mode 2 probability estimated by the LSTM-IMML method still occupies an advantage, which also demonstrates that the LSTM-IMML method is better. In addition, compared with the IMM method, the proposed LSTM-IMML method is more stable, and the estimated probability does not appear prone to drastic fluctuation.

To further analyze the accuracy of state estimation by the IMM method and the LSTM-IMML method, 100 Monte Carlo simulations were performed to calculate the error between the estimated state and the real value. The root mean square error of the position and the root mean square error of the velocity obtained by the IMM method and the LSTM-IMML method are shown in Figure 10 and Figure 11, respectively. The position and velocity root mean square error of the LSTM-IMML method are smaller than those of the IMM method, indicating that the introduction of feedback items in the filter can improve state estimation accuracy.

In addition, the case of the Evader using four evasion strategies is considered to further compare the IMM and LSTM-IMML methods; that is, the first three are linear quadratic strategies, and the fourth is a zero-effort miss strategy. The boundaries of mode switch are $d_{P{E}_1}=1400 \mathrm{m}, d_{P{E}_2}=1000 \mathrm{m}, d_{P{E}_3}=500 \mathrm{m}$. In the linear quadratic pursuit–evasion strategy, ${\mathit{R}}_{E_1}=diag\left(\left[\begin{array}{ccc}3& 3& 3\end{array}\right]\right)\times {10}^{-7},{\mathit{R}}_{E_2}=diag\left(\left[\begin{array}{ccc}2& 2& 2\end{array}\right]\right)\times {10}^{-7},{\mathit{R}}_{E_3}=diag\left(\left[\begin{array}{ccc}1.4& 1.4& 1.4\end{array}\right]\right)\times {10}^{-7}$. In the LSTM-IMML method, the Markov transition probability matrix is $\mathit{\pi }=\left[\begin{array}{cccc}0.85& 0.05& 0.05& 0.05\\ 0.05& 0.85& 0.05& 0.05\\ 0.05& 0.05& 0.85& 0.05\\ 0.05& 0.05& 0.05& 0.85\end{array}\right]$. The IMM and LSTM-IMML methods are used for simulation, and the simulation results are shown in Figure 12 and Figure 13.

Figure 12a and Figure 13a show the relative distance between the two satellites when the Pursuer adopts the IMM and LSTM-IMML methods, respectively. When the Evader adopts a switchable evasion strategy with four modes, the Pursuer using the LSTM-IMML method approaches the interception range within 20 m at the end, while the Pursuer using the IMM method does not. This is because evasion mode 2 and mode 3 are very similar, which cannot be accurately estimated using the IMM method, but can be accurately estimated using the LSTM-IMML method, which can be seen from Figure 12b and Figure 13b. This simulation further proves the superiority of the proposed LSTM-IMML method in mode probability estimation.

6. Conclusions

In this paper, a new switchable pursuit strategy for the Pursuer is proposed when the Evader adopts multiple switchable evasion strategies. Firstly, the linear quadratic and zero-effort miss pursuit–evasion strategies are designed. Then, the IMM method is used to identify the evasion strategy of the Evader in parallel by using multiple filters. To overcome the problem that the IMM method has poor accuracy in estimating the Evader’s state, a feedback learning filter is proposed to improve the state estimation accuracy. The estimation accuracy is improved by introducing the feedback term. In addition, a mode probability estimation network based on LSTM is proposed to enhance the stability of the probability estimation, which is embedded in the interactive multiple-model learning filter. The simulation results verify the effectiveness of the proposed LSTM-IMML method. Compared with the IMM, the state estimation accuracy of the proposed LSTM-IMML method is improved, the estimated mode probability is more exact and stable, and the observability is stronger, which improves the interception effectiveness of the Pursuer. The proposed LSTM-IMML game switching strategy further enhances the recognition accuracy and stability of the Evader’s mode probability, which can be implemented in a space pursuit–evasion game mission with incomplete information. In subsequent studies, we will consider constraints such as navigation information loss and sensor operating distance.