*2.2. Model Reduction through LASSO Shrinkage*

The most common method used for regression is the least squares formulation, which aims to minimize the residual sum of squares (RSS):

$$RSS = \sum\_{z=1}^{N} \left( y\_z - \mathfrak{H}\_z \right)^2 \tag{3}$$

which is the sum of the differences between measured outputs and predicted outputs squared for *N* data points. Shrinkage methods attempt to reduce the magnitude of the predicted *β* values (shrinking them). This is performed by modifying Equation (3), adding an additional term, and in the case of LASSO shrinkage, this is given in Equation (4) [19]:

$$RSS = \sum\_{z=1}^{N} \left( y\_z - \hat{y}\_z \right)^2 + \lambda \sum\_{i=1}^{n} |\beta\_i| \tag{4}$$

where *n* is the number of input variables and *λ* is a tuning parameter. This is related to the linear model in Equations (1) and (2) but can also be applied to quadratic expressions as follows:

$$RSS = \sum\_{z=1}^{N} (y\_z - \hat{y}\_z)^2 + \lambda \sum\_{i=1}^{n} |\beta\_i| + \lambda \sum\_{i=1}^{n} \sum\_{j=i}^{n} |\beta\_{ij}| \tag{5}$$

such that all the parameters in the linear and quadratic terms are included together. In either case, Equations (4) or (5) are minimized during fitting, which simultaneously reduces the error between model and measured values and reduces the magnitude the *β* values. This is controlled by tuning the value of *λ*, and increasing this value should decrease the values of fitted parameters. In this case, using the LASSO formulation with absolute values of the parameters, it can be shown that this leads to increasing numbers of parameters set to zero [19]. This in turn allows parameters set to zero to be neglected together with the associated inputs producing a simplified or reduced model [19].
