2.2.5. Deep Autoencoder (DAE)

An autoencoder also consist of a visible input layer (called an encoder) and a hidden layer (called a decoder). It first transforms (encodes) an input vector **x** to a hidden representation **z** through affine mapping

$$\mathbf{z} = s(\mathbf{w}\mathbf{x} + \mathbf{b})\tag{11}$$

and then maps (decodes) **z** back to a reconstructed vector **x** in the input space:

$$\mathbf{x}' = s(\mathbf{w}'\mathbf{z} + \mathbf{b}') \tag{12}$$

The aim of autoencoder training is to minimize the average reconstruction error over the training set:

$$\underset{\mathbf{w},\mathbf{b},\mathbf{w}',\mathbf{b}'}{\arg\min} \mathcal{L} = \frac{1}{N} \sum\_{i=1}^{N} ||\mathbf{x}\_i \mathbf{x}'||^2 \tag{13}$$

A DAE [23] consists of a stack of autoencoders. Its training consists of two stages. The first stage is to train each autoencoder layer by layer, and the second stage is to train the whole DAE to minimize the RMSE over the training set.

For the morbidity-prediction problem, we used a DAE with four hidden layers, and tuned the numbers of neurons in the hidden layers to 4500, 640, 80, and 12, respectively. It also employed a Gaussian mixture model to produce output morbidity.
