4.1.1. Artificial Neural Networks' Architecture

ANNs usually consist of three parts: one input layer; one or more hidden layers, and one output layer. All layers include neurons, and each neuron in a given layer is linked to the neurons of previous as well as successive layers. Each link between two neurons is characterized by an adaptable synaptic weight and bias. Three main different functional operations occur in ANNs:


ANNs are trained by means of a suitable learning method in order to obtain a specific target output from a particular input by regulating the weights and biases. The training process is stopped only when the error between the desired target and the corresponding network output is lower than a given tolerance value or when the maximum number of epochs (given number of iterations) is achieved. A transfer function is a mathematical representation of the relation between inputs and outputs. Transfer functions generally have a sigmoid shape, but they may also assume the form of piecewise linear functions, nonlinear functions, or step functions. One of the most commonly adopted transfer functions for multilayer networks is the hyperbolic tangent sigmoid transfer function (tansig) [32] generating outputs between −1 and 1.

Performance of artificial neural networks is sensitive to both the number of hidden layers as well as the number of neurons in their hidden layers [32]. In particular, networks with more hidden layers require a larger computation time, but their use gives the network more flexibility and could result in resolving challenging tasks more efficiently [32]. Larger numbers of neurons allow the network to figure out more difficult issues; however, they require more computation and they can play a part to "overfitting" (in that case the fitting curve fluctuates wildly among training points, even if these points are well fitted); on the other hand, few neurons can reduce the computation time, but they could also lead to "underfitting".

The MATLAB (The MathWorks Inc., Natick, Massachusetts, USA) Neural Network Toolbox [32] has been used in this work in order to develop and analyze 22 artificial neural network-based simulation models (ANN1-ANN22) of the HVAC system. All the artificial neural networks have been configured with 10 inputs and 5 outputs, varying the number of hidden layers and neurons in each hidden layer. One of the most common issues to be addressed in configuring the architecture of ANNs is connected to the ANNs topology allowing to achieve the requested accuracy and/or minimize the computation time. Several studies [41–46] have determined the number of hidden layers and the number of neurons in the hidden layers by trial and error, employing a grid search technique to find them. A sensitivity analysis has been performed in this study in order to find out the optimal number of hidden layers and neurons in each hidden layer according to the information and approaches reported in the current literature. In particular, several scientific papers investigated the application of ANNs for HVAC systems' modeling [41–43], adopting a number of hidden layers varying from a minimum of 1 [41] up to a maximum of 5 [41–43]. In addition, several formulas are available in the scientific literature [44–46] in order to provide a starting point for determining the optimal number of neurons in each hidden layer of ANNs as a function of (i) number of inputs [44–46], (ii) number of outputs [44], (iii) number of hidden layers [45], and (iv) number of training examples [45,46]; these formulas suggest a number of neurons per hidden layer in the range of 7 to 83 when applied to the ANNs investigated in this paper.
