*2.2. LSTM-RNN*

The fading of previously learned patterns is a challenge experienced in standard RNN architectures. The LSTM-RNN has a memory cell to overcome this shortcoming. The memory cell is managed by non-linear gating units. The gated units of an LSTM-RNN unit can be seen in Figure 2. These gated units, the forget gate (*fn*), input gate (*in*), and output gate (*on*), are presented by Equations (5)–(7), respectively. Equations (8)–(10), respectively, present the input node (*gn*), the state (*sn*), and the cell state (*hn*). Here, *n* is the time step, ∅ is the tanh function, *σ* is the sigmoid function, and the *W* matrices are the respective network activation functions' corresponding input weights. The LSTM-RNN cells are stacked after each other to achieve a deep layered LSTM-RNN. The memory cells give the models the ability to sustain memory.

$$f\_n = \sigma \left(\mathcal{W}\_{fz} z\_n + \mathcal{W}\_{fh} h\_{n-1} + b\_f\right) \tag{5}$$

$$\dot{a}\_n = \sigma(\mathcal{W}\_{iz} z\_n + \mathcal{W}\_{il} h\_{n-1} + b\_i) \tag{6}$$

$$
\sigma\_n = \sigma(\mathcal{W}\_{oz} z\_n + \mathcal{W}\_{\text{ol}} h\_{n-1} + b\_o) \tag{7}
$$

$$\mathbf{g}\_{n} = \mathcal{Q}\left(\mathcal{W}\_{\mathbb{S}^{\mathsf{Z}}}\boldsymbol{z}\_{n} + \mathcal{W}\_{\mathbb{S}^{h}}\boldsymbol{h}\_{n-1} + \boldsymbol{b}\_{\mathbb{S}}\right) \tag{8}$$

$$s\_n = \mathcal{g}\_n \odot i\_n + s\_{n-1} \odot f\_n \tag{9}$$

$$h\_{\mathbb{N}} = \bigotimes(s\_{\mathbb{N}}) \odot o\_{\mathbb{N}} \tag{10}$$

**Figure 2.** An LSTM−RNN cell with gated units.
