• **Ridge Regression**

Since we added two more factors in the model, we applied ridge regression instead of OLS regression. On one hand, ridge regression can prevent an over-fitting result from extra factors. On the other, it can prevent multi-collinearity and increase the significance of the factors. Ridge regression is developed based on OLS regression by adding regularization term λI. The regularization term can discover the factor with collinearity and force the coe fficient to approach zero to ensure that the e ffect of multi-collinearity is minimized. To calculate the beta (coe fficient) of each independent variable, the matrix operation combines X, y, λ, and I. X and y are the same as the matrix in the OLS model. λ is an optimization hyperparameter and I is a 7 × 7 identity matrix.

$$\boldsymbol{\beta} = (\boldsymbol{\lambda}^{\mathrm{T}} \boldsymbol{\lambda} + \boldsymbol{\lambda} \boldsymbol{I})^{-1} \boldsymbol{X}^{\mathrm{T}} \mathbf{y} \tag{8}$$


When we applied the ridge regression, we need to find a λ that can provide the largest sum of R-square which are the number of positive factors and percentage of positive coefficient. We set the summary of these three measures as our target function to find out the optimal λ with iteration.

$$
\lambda = \operatorname\*{argmax}(\mathbf{Q}) \tag{10}
$$

Q = P average value + number of positive coefficients + R square mean + number of variables whose *p* value over 0.9 (9).
