Dynamic Behavior Analysis and Synchronization of Memristor-Coupled Heterogeneous Discrete Neural Networks

Abstract

Continuous memristors have been widely studied in recent years; however, there are few studies on discrete memristors in the field of neural networks. In this paper, a four-stable locally active discrete memristor (LADM) is proposed as a synapse, which is used to connect a two-dimensional Chialvo neuron and a three-dimensional KTZ neuron, and construct a simple heterogeneous discrete neural network (HDNN). Through a bifurcation diagram and Lyapunov exponents diagram, the period and chaotic regions of the discrete neural network model are shown. Through numerical analysis, it was found that the chaotic region and periodic region of the neural network based on DLAM are significantly improved. In addition, coexisting chaos and chaos attractors, coexisting periodic and chaotic attractors, and coexisting periodic and periodic attractors will appear when the initial value of the LADM is changed. Coupled by a LADM synapse, two heterogeneous discrete neurons are gradually synchronized by changing the coupling strength. This paper lays a good foundation for the future analysis of LADMs and the related research of discrete neural networks coupled by LADMs.

1. Introduction

Neurons are the main carriers of neural information coding, transmission, processing, and integration [1,2]. The biodynamic network formed by thousands of neurons through synapses is called a neural network [3,4]. Neural networks play a very important role in pattern recognition, signal processing, artificial intelligence, and other fields with its unique structure and information processing method [5,6,7]. To date, continuous neuron models and neural network models, proposed in the neurological discipline, are based on the four-dimensional Hodgkin–Huxley (HH) model [8] and its simplified derived models, for example, the FitzHugh–Nagumo (FHN) model [9], Hindmarsh–Rose (HR) model [10,11], Hopfield neural networks (HNN) [12,13,14,15,16], etc. These models are widely used in the study of dynamic behavior and application of neural networks [17,18,19,20,21]. Based on the research of continuous neuron models, a lot of discrete neuron models have been proposed. So far, Chialvo simplified the Hodgkin–Huxley (HH) model to develop the Chialvo model [22]. Inspired by Chialvo’s research, Rulkov and Kinouchi-Tragtenberg proposed the Rulkov [23] and KT [24] models, respectively. Based on the three-dimensional KT neuron model, the three-dimensional KTZ discrete neuron model was proposed in [25]. Discrete neuron models have been proven to be able to mimic real biological neurons [26], and have more advantages than continuous neuron models in terms of computing time, storage resources, etc. [27,28]. Therefore, discrete neuron models are more suitable for large-scale simulation and theoretical analysis [29,30,31], and have become most popular in recent years.

In 1971, Chua first put forward the concept of memristor as a device to describe the relationship between charge and flux [32]. Since HP Laboratory developed the entity of a memristor in 2008 [33], continuous memristors have been widely used in neural networks [34,35,36,37,38] and chaotic circuits [39,40,41,42,43], artificial intelligence [44,45], etc. Ding et al. [46] studied the hidden coexisting firing patterns of two HR neurons coupled by a memristor with fractional order and applied it to image encryption. Li et al. [47] studied the firing patterns of a memristor coupled heterogeneous continuous neural network and also analyzed the synchronization between neurons. Memristors are considered to be ideal devices to mimic neuron synapses [48]. Local activity is considered to be the source of complexity [49]. As far as we know, there are few discussions about discrete memristors in the field of neural networks. Most of the related researches focus on some discrete mappings or single-neuron dynamics. For example, Liu et al. [50] first applied the discrete memristor to a two-dimensional Rulkov neuron model and designed a new chaotic neuron model. Li et al. [51] used a discrete memristor to couple a single Rulkov neuron and successfully described the phenomenon of magnetic induction. Bao et al. proposed a discrete fractional order memristor and applied it to the Logistic map [52]. Li et al. coupled the discrete memristor with a logarithmic function to obtain a new discrete memristor map [53]. A new LADM was proposed in [54], and applied to hyperchaotic maps.

Inspired by the previous literature, a four-stable LADM was proposed and used to connect a two-dimensional Chialvo neuron and a three-dimensional KTZ neuron. In this way, a LADM-based HDNN model was constructed. The dynamic behavior of this HDNN was analyzed, and the firing patterns and synchronization of this HDNN were studied.

The layout of this paper is as follows. In the second section, a four-stable LADM is proposed, and its related characteristics are studied in detail. In the third section, a discrete neural network model is obtained by coupling heterogeneous discrete neurons with a LADM. The equilibrium point and its corresponding stability are analyzed theoretically. In the fourth section, the coexistence behavior of the neural network is studied. In the fifth section, the phase synchronization of the HDNN is discussed. Finally, conclusions are given.

2. LADM

2.1. LADM Model

In this work, a LADM was proposed and its mathematical model is as follows:

$$\{\begin{array}{l}i(n)=W[q(n)]v(n)=q(n)v(n),\\ q(n+1)=F(q,v)=\alpha f[q(n)]+\beta v(n),\\ f[q(n)]=\mathrm{sgn}[q(n)]+\mathrm{sgn}[q(n)+2]+\mathrm{sgn}[q(n)-2]+\epsilon q(n),\end{array}$$

(1)

where v(n), i(n), and W[q(n)] are the input voltage, output current, and memristance value of the memristor, respectively. α, β, and ε are the three parameters of the LADM. F(q,v) represents the internal state equation of the LADM.

2.2. Pinched Hysteresis Loops

In this paper, a sinusoidal voltage signal v(n) = Asin(2πwT(n)) with amplitude A and frequency w is added for numerical simulation. Consider the situation where α = 0.1, β = 0.1, ε = 9, and the initial value of the LADM $q\left(1\right)=1$. Changing amplitude A and frequency w, the volt–ampere curve of the LADM can be obtained and shown in Figure 1.

As shown in Figure 1, when the frequency w = 0.0001 is fixed, the area of the flap of the pinched hysteresis loop increases monotonically with the increase of the excitation amplitude A. If the amplitude A = 10 is fixed, the flap area of the pinched hysteresis loop decreases monotonically as the excitation frequency w increases, and the hysteresis loop gradually contracts to a single-valued function as the frequency tends to infinity. Obviously, the LADM has three fingerprint characteristics and satisfies the definition of a generalized memristor.

2.3. Nonvolatility and Local Activity

In addition, the non-volatile property of a memristor is always characterized by a power-off plot (POP), which is the solution curve of the internal state equation of the memristor when the voltage v(n) = 0. According to the non-volatile memristor theorem, a memristor is non-volatile if there are two or more negative slope intersections between the q-axis and the POP curve of the memristor. Here, let v(n) = 0 and substitute it into Equation (1) to obtain the simplified equation as follows.

$$q(n+1)-q(n)=F(q,0)=\alpha \{\mathrm{sgn}(q(n))+\mathrm{sgn}(q(n)+2)+\mathrm{sgn}(q(n)-2)+q(n)\}.$$

(2)

According to Equation (2), the POP curve of this LADM was obtained as shown in Figure 2. It can be observed that when $q(n+1)-q\left(n\right)=0,$ there are seven intersection points on the q-axis, which are Q₁(−3, 0), Q₂(−2, 0), Q₃(−1, 0), Q₄(0, 0), Q₅(1, 0), Q₆(2, 0), and Q₇(3, 0). It should be noted that each intersection point is an equilibrium point of the memristor, and those points with a negative slope are stable and the others are unstable. Therefore, Q₁, Q₃, Q₅, and Q₇ are stable equilibrium points and Q₂, Q₄, and Q₆ are unstable equilibrium points. According to the direction of the arrows shown in Figure 2, we can judge what state the internal state variable will eventually stabilize in different situations. The LADM has four stable equilibrium states at different initial states of $q\left(0\right)$, which are shown in Equation (3).

$$\{\begin{array}{ll}q=q(Q_1)=-3,& q(0)<-2.\\ q=q(Q_3)=-1,& -2<q(0)<0.\\ q=q(Q_5)=1,& 0<q(0)<2.\\ q=q(Q_7)=3,& q(0)>2.\end{array}$$

(3)

Therefore, the proposed LADM is a nonvolatile memristor.

In addition, it should be noted that not all nonvolatile memristors are locally active. Through observing DC V-I diagrams, the nature of local activities can be inferred. If there is a region with a negative slope in the DC V-I diagram, the memristor is a locally active memristor. To get the DC V-I diagram of a LADM, set q (n + 1) − q(n) = 0 to calculate the equilibrium equation, which is as follows:

$$\{\begin{array}{l}V=10\alpha \left[q-\mathrm{sgn}(q)-\mathrm{sgn}(q+2)-\mathrm{sgn}(q-2)\right],\\ I=Vq=10V\alpha \left[q-\mathrm{sgn}(q)-\mathrm{sgn}(q+2)-\mathrm{sgn}(q-2)\right],\end{array}$$

(4)

where V and I are the DC voltage and current, respectively. When DC voltages are applied to the LADM, the DC V-I diagram of the four-stable LADM can be obtained according to Equation (4), as shown in Figure 3a. The DC V-I diagram has four regions with negative slopes, which means that the LADM has four locally active regions. Therefore, the LADM is a four-stable four-locally active memristor. In addition, pinched hysteresis loops of the LADM with different initial values are shown in Figure 3b.

3. LADM Coupled HDNN

3.1. LADM-Coupled HDNN Model

The three-dimensional KTZ neuron model is as follows

$$\{\begin{array}{l}x(n+1)=\tanh \left(\frac{x(n)-ay(n)+z(n)+d}{t}\right),\\ y(n+1)=x(n),\\ z(n+1)=(1-g)z(n)-h(x(n)-u),\end{array}$$

(5)

where x (n) is the membrane potential of neurons; y (n) is a recovery variable; z (n) is the slow total ion current; and a, d, t, g, h, and u are parameters controlling the firing patterns of the neurons.

The two-dimensional Chialvo neuron model is as follows:

$$\{\begin{array}{l}x(n+1)={\left[x(n)\right]}^2\exp \left[y(n)-x(n)\right]+I(t),\\ y(n+1)=ay(n)-bx(n)+c,\end{array}$$

(6)

where x(n) is the membrane potential of the neuron; y(n) is a recovery variable; a, b, and c are parameters; and I(t) can be regarded as the external input current. Based on the three-dimensional KTZ discrete neuron model and the two-dimensional Chialvo discrete neuron model, a new LADM-based neural network model is proposed as follows:

$$\{\begin{array}{l}{x}_1(n+1)=\tanh \left(\frac{x_1(n)-a{y}_1(n)+z_1(n)+d}{t}\right)-k\left(x_1(n)-x_2(n)\right)q(n),\\ y_1(n+1)=x_1(n),\\ z_1(n+1)=(1-g)z_1(n)-h(x_1(n)-u),\\ x_2(n+1)={\left[x_2(n)\right]}^2\exp \left[y_2(n)-x_2(n)\right]+d_1+k\left(x_2(n)-x_1(n)\right)q(n),\\ y_2(n+1)=a_1{y}_2(n)-b_1{x}_2(n)+c_1,\\ q(n+1)=c\left(\mathrm{sgn}(q(n))+\mathrm{sgn}(q(n)+2)+\mathrm{sgn}(q(n)-2)-q(n)\right)+q(n)+0.1c\left(x_1(n)-x_2(n)\right),\end{array}$$

(7)

where k is the coupling coefficient. The parameters of this neural network model are set as a = 0.6, t = 0.35, c = 0.1, a₁ = 0.9, b₁ = 0.6, c₁ = 0.28, d = 0.04, d₁ = 0.03, g = 0.008, h = 0.004, and u = −0.6. This system is a six-dimensional discrete system. Compared with the continuous system, map-based neural network models have the advantages of requiring shorter computation time and fewer resources, using completely transparent iteration algorithms.

To better understand the new LADM-based neural network, the topology is shown in Figure 4.

3.2. Stability Analysis of Equilibrium Points

According to Equation (7), the equilibrium point E is

$$E\left({x_1}^{(e)},{x_1}^{(e)},-0.5({x_1}^{(e)}+0.6),{x_2}^{(e)},-6{x_2}^{(e)}+2.8,q^{(e)}\right)$$

(8)

Based on Equations (7) and (8), the following equations can be obtained:

$$\{\begin{array}{l}{x_1}^{(e)}-\tanh \left(\frac{-10{x_1}^{(e)}-26}{35}\right)+k{q}^{(e)}\left({x_1}^{(e)}-{x_2}^{(e)}\right)=0,\\ {x_2}^{(e)}-{x_2}^{(e)}{}^2\exp \left(-7{x_2}^{(e)}+2.8\right)-0.03-k{q}^{(e)}\left({x_2}^{(e)}-{x_1}^{(e)}\right)=0,\end{array}$$

(9)

and x₂^(e)is

$${x_2}^{(e)}={x_1}^{(e)}+10\left(\mathrm{sgn}(q^{(e)})+\mathrm{sgn}(q^{(e)}+2)+\mathrm{sgn}(q^{(e)}-2)-q^{(e)}\right).$$

(10)

It can be seen from Equation (9) that the equilibrium points of the system are only related to the coupling strength k. To get the Jacobian matrix, tanh(10⁶*x) was used to replace sign (x). The Jacobian matrix at the equilibrium point E of the system is expressed as follows

$$Je=\left[\begin{array}{cccccc}a& b& a& k{q}^{(e)}& 0& -k\left({x_1}^{(e)}-{x_2}^{(e)}\right)\\ 1& 0& 0& 0& 0& 0\\ -0.004& 0& 0.002& 0& 0& 0\\ -k{q}^{(e)}& 0& 0& c& d& k\left({x_2}^{(e)}-{x_1}^{(e)}\right)\\ 0& 0& 0& -0.6& 0.9& 0\\ 0.01& 0& 0& -0.01& 0& f\end{array}\right],$$

(11)

where

$$a=\frac{20}{7}{\left(\mathrm{sech}\frac{{x_1}^{(e)}-0.6{y_1}^{(e)}+{z_1}^{(e)}+0.04}{0.35}\right)}^2-k{q}^{(e)},$$

$$b=-\frac{12}{7}{\left(\mathrm{sech}\frac{{x_1}^{(e)}-0.6{y_1}^{(e)}+{z_1}^{(e)}+0.04}{0.35}\right)}^2,$$

$$c=2{x_2}^{(e)}\exp \left({y_2}^{(e)}-{x_2}^{(e)}\right)-{\left[{x_2}^{(e)}\right]}^2\exp \left({y_2}^{(e)}-{x_2}^{(e)}\right)+k{q}^{(e)},$$

$$d={\left[{x_2}^{(e)}\right]}^2\exp ({y_2}^{(e)}-{x_2}^{(e)}),$$

$$\begin{array}{l}f={10}^5\left(1-\tanh {({10}^6{q}^{(e)})}^2\right)+{10}^5\left(1-\tanh {({10}^6(q^{(e)}+2))}^2\right)\\ +{10}^5\left(1-\tanh {({10}^6(q^{(e)}-2))}^2\right)+0.9.\end{array}$$

The eigenvalue can be obtained by Equation (12):

$$P(\lambda )=\det (\lambda I-J_e).$$

(12)

When k = 0.25 or k = 0.3, and keeping other initial values unchanged, Equations (9) and (10) can be graphically shown in Figure 5. As shown in Figure 5, there are six intersections of the two curves. Obviously, $q^{(e)}(-2.001,0.016,2.026)$ in Figure 5a and $q^{(e)}(-1.996,0.016,2.025)$ in Figure 5b are not the solution of Equation (10). Therefore, the system has three equilibrium points (E₁, E₄, and E₆).

To discuss the stability of the equilibrium points with different coupling strength k, two cases, with k = 0.25 and k = 0.3, were considered. The relationship between the eigenvalues and stability at different equilibrium points is shown in

Table 1. Equilibrium points, corresponding eigenvalues, and stability, when k = 0.25 and k = 0.3.

k	NO.	(x1(e), q(e))	Eigenvalues	Stabilities
	E1	(−0.0337, 2.9189)	0.6698 + 1.1494i, 0.6698 − 1.1494i 0.5151, 0.9885, 0.8554, 0.9021.	Stable
0.25	E4	(−0.1708, 0.7795)	0, 0.9920, −0.0024, 0.0024, 0.9004, 0.8996.	Stable
	E6	(2.2301, −2.5028)	−2.7047, −2.1365, 1.2382, 0, 1.5248, 1.6534.	Unstable
	E1	(−0.0331, 2.9327)	0.6830 + 1.2400i, 0.6830 − 1.2400i 0.6666, 0.7710, 0.9894, 0.9018.	Stable
0.3	E4	(−0.1429, 0.8094)	0.7902 + 0.7365i, 0.7902 − 0.7365i, 0.2326, 0.9770, 0.9113, 0.8998.	Stable
	E6	(1.5934, −2.7005)	−1.4298, −1.6982, 2.1698, 1.9782, 0, 2.3712.	Unstable

. According to Equation (12), six eigenvalues were obtained. In addition, it is worth noting that an equilibrium point is stable when the eigenvalue is in the unit circle, otherwise the point is unstable. It can be seen from Table 1 that two equilibrium points (E₁ and E₄) are stable, and one equilibrium point (E₆) is unstable.

4. Dynamic Behavior Analysis of the LADM-Coupled HDNN

In this section, consider the situation where a = 0.6, t = 0.35, c = 0.1, a₁ = 0.9, b₁ = 0.6, c₁ = 0.28, d = 0.04, d₁ = 0.03, g = 0.008, h = 0.004, and u = −0.6.

In

Table 2. Comparison between our work and previous works.

Work	[54]	[55]	[56]	[57]	[58]	[59]	This Work
DLAM	✗	✓	✗	✓	✓	✓	✓
Balance point analysis	✓	✗	✓	✗	✗	✓	✓
Heterogeneous neuron	✗	✗	✗	✗	✗	✗	✓
Multistability	✓	✓	✓	✓	✓	✓	✓
Phase synchronization	✓	✓	✓	✗	✓	✓	✓

, we compare previous works with this one. This work combines some advantages of previous works, and LADM-coupled HDNN is studied for the first time.

4.1. Bifurcation and Lyapunov Exponent

The Lyapunov exponent diagram and bifurcation diagram under two different initial values of the LADM are shown in Figure 6 and Figure 7. According to the value of the maximum Lyapunov exponents, the state of the system can be judged [60].

The bifurcation diagrams in Figure 6a and Figure 7a show that complex dynamic behaviors appear in the LADM-coupled HDNN, which includes chaotic firing and periodic firing.

4.2. Coexisting Attractors

The phenomenon that the dynamic behaviors are different under different initial conditions is called attractor coexistence. Next, the coexisting attractors of the LADM-coupled HDNN are revealed.

The neural network exhibited different coexisting firing patterns under different k. To make it easier to distinguish, different color lines were used to describe different coexistence attractors. Figure 8 shows the phase diagrams and time series with different initial values. The blue line corresponds to the initial value of (x₁(1), y₁(1), z₁(1), x₂(1), y₂(1), q (1)) = (−1, 1, −1, 1, 1, 1). The red line corresponds to the initial value of (−1, 1, −1, 1, 1, −1). Figure 8 shows that this LADM-coupled HDNN had four kinds of coexistence patterns including the coexisting chaotic and chaotic firing patterns, coexisting chaotic and periodic firing patterns, coexisting periodic and chaotic firing patterns, coexisting periodic and periodic firing patterns. The system states shown in Figure 8 are consistent with those shown in Figure 6 and Figure 7. For example, when (x₁(1), y₁(1), z₁(1), x₂(1), y₂(1), q (1)) = (−1, 1, −1, 1, 1, 1) and k = −0.68, the corresponding Lyapunov exponents (LE1, LE2, LE3, LE4, LE5, LE6) = (−0.007, −0.095, −0.151, −0.812, −0.998, −13.632). When (x₁(1), y₁(1), z₁(1), x₂(1), y₂(1), q (1)) = (−1, 1, −1, 1, 1, −1) and k = −0.68, the corresponding Lyapunov exponents (LE1, LE2, LE3, LE4, LE5, LE6) = (0.098, −0.007, −0.071, −0.109, −0.191, −2.212). This means that the system exhibits periodic and chaotic states, respectively, in these two cases and the system has the coexistence pattern of coexisting periodic and chaotic firing patterns which is shown in Figure 8c,d.

5. Phase Synchronization and Synchronization Transition

To study the phase synchronization and synchronization transition of the LADM coupled HDNN, the definition of phase θ is given, which is expressed as

$$\mathsf{\theta }(n)=2\pi \frac{n-n_k}{n_{k+1}-n_k},(n_k<n<n_{k+1}).$$

(13)

where n_k is the time when the kth cluster occurs, and n_k+1-n_k is the time interval. The phase difference can be expressed as follows:

$$\left|\mathsf{\Delta }\mathsf{\theta }(n)\right|=\left|{\mathsf{\theta }}_1(n)-{\mathsf{\theta }}_2(n)\right|,$$

(14)

The synchronization transition behavior is shown in Figure 9. When k increased from 0 to 2, the transition from un-synchronization to complete synchronization occurred. When coupled strength k = 0, one neuron exhibited a bursting firing pattern and the other one was in the spike firing pattern as shown in Figure 9a,b. Figure 9c,d presents the intermediate transition process. When coupled strength k = 0.5, two neurons exhibited a bursting firing pattern and are synchronized with a little phase difference. When coupled strength k = 1, two neurons exhibited a spike firing pattern and were synchronized exactly as shown in Figure 9e,f.

6. Conclusions

In this paper, a four-stable LADM was proposed, and its characteristics were studied. The simulation results showed that the LADM satisfies the definition of a generalized memristor. Based on the two-dimensional Chialvo discrete neuron model and the three-dimensional KTZ discrete neuron model, a new LADM-coupled HDNN was constructed by connecting these two kinds of neurons with a LADM. Through numerical simulations, four kinds of coexisting firing patterns were found. They were the coexisting chaotic and chaotic firing patterns, the coexisting chaotic and periodic firing patterns, the coexisting periodic and chaotic firing patterns, and the coexisting periodic and periodic firing patterns. In addition, the phase synchronization and synchronization transitions were also revealed. Analyzing the mechanism of the dynamic behavior shown in this paper is our next work.