*2.3. DBN*

The deep belief network (DBN) is built by stacking restricted Boltzmann machines (RBM). The technique was introduced in the mid-2000s by Geoffrey Hinton. There are no connections between the neurons on the same layer. There is a symmetrical and bidirectional connection between the layers. The model determines the hidden state, visible state, initial weight, and biases in the first step using unsupervised learning. Supervised learning, using back-propagation, is used to append the unsupervised learning pre-trained model. The joint distribution over the visible and hidden units is given by (11) [39].

$$P(m,h) = \frac{e^{-E(m,h)}}{\sum\_{\text{ltr}} e^{-E(m,h)}}\tag{11}$$

where *E*(*m*, *h*) is the energy function. The conditionally independent conditional probabilities are given by (12) and (13). If the values of the hidden and visible units are from 0 to 1, (12) and (13), respectively, become (14) and (15), with *i* = 1, 2 . . . *kh* and *j* = 1, 2 . . . *km*.

$$p(m|h) = \prod\_{j} p\left(m\_{j}|h\right) \tag{12}$$

$$p(h|m) = \prod\_{i} p(h\_i|m) \tag{13}$$

$$p\left(m\_{j} = 1 \middle| h\right) = \operatorname{sigmoid}\left(a\_{j} + \sum\_{i=1}^{k\_{h}} \mathcal{W}\_{ij} h\_{i}\right) \tag{14}$$

$$p\left(h\_{\bar{\jmath}} = 1 \, | \, m \right) = \operatorname{sigmoid}\left(\beta\_i + \sum\_{j=1}^{k\_m} \mathcal{W}\_{ij} m\_{\bar{\jmath}}\right) \tag{15}$$
