Relationship between Mental Health and Socio-Economic, Demographic and Environmental Factors in the COVID-19 Lockdown Period—A Multivariate Regression Analysis

Abstract

Amongst the several consequences of the COVID-19 pandemic, we should include psychological effects on the population. The mental health consequences of lockdown are affected by several factors. The most important are: the duration of the social isolation period, the characteristics of the living space, the number of online (virtual) and offline (physical) contacts and perceived contacts’ closeness, individual characteristics, and the spread of infection in the geographical area of residence. In this paper, we investigate the possible effects of environmental, social and individual characteristics (predictors) on mental health (response) during the COVID-19 lockdown period. The relationship between mental health and predictors can be studied with a multivariate linear regression model, because “mental health” is a multidimensional concept. This work provides a contribution to the debate about the factors affecting mental health in the period of the COVID-19 lockdown, with the application of an innovative approach based on a multivariate regression analysis and a combined permutation test on data collected in a survey conducted in Italy in 2020.

1. Introduction

The psychological effects on the population are amongst the numerous (and important) consequences of the COVID-19 pandemic [1]. In fact, the pandemic is causing uncertainty about the present and future, insecurity about our health and the health of our relatives and friends, socio-economic problems, and other situations leading to stress and fears. Indeed, the pandemic, on a global scale, is not only a medical event, but also a psychological, economic and social event. In particular, the COVID-19 pandemic has generated anxiety, panic, and psycho-emotional mass effects. In most cases, governments have adopted containment policies to tackle the problem. These measures have been useful to limiting the spread of the contagion, but have had consequences in terms of people’s mental health [2]. Lockdown and the consequent social and physical distancing represent important sources of mental health disorders. Some studies report the effects of the COVID-19 lockdown on the mental health of the Spanish population. In particular, the crisis situation caused by COVID-19, and especially the confinement measures, have significantly affected people’s lives, with those who had previously been experiencing family and mental health difficulties being at greater risk. Likewise, the local spread of the virus could have further affected the impact of the imposed social isolation on mental health [3,4].

One of the main consequences of the lockdown for people exposed to long-lasting social and physical distancing has been psychological resignation, with feelings of depression, alienation, unworthiness, and helplessness [5]. The risk of developing anxiety and depression has grown exponentially, as physical distancing measures are themselves a key risk for the mental health of individuals who have developed a condition of loneliness over the months in question. However, according to the literature, there are several psychological problems attributed to social isolation.

The COVID-19 pandemic has caused numerous episodes of the worsening of mental health, especially in subjects who already presented symptoms of anxiety, depression, and sleep disturbances. Furthermore, it has affected the majority of the world population psychologically. Mental health worsening due to the pandemic is not a surprising event, as it is known that, the pandemic being an unpredictable and traumatic event, it leads to a change in individuals’ habits, thus causing uncertainty and confusion. According to several scientific contributions, in the lockdown period there was an increase in the consumption of alcohol and nicotine, but mainly of sleeping pills, anxiolytics and antidepressants to cope with panic attacks [6].

With regard to the emotional impact, however, some studies have revealed that people felt emotional repercussions due to the pandemic, in particular symptoms of stress, anxiety and panic. Specifically, those with past psychological problems, such as gamblers, were most affected by the lockdown restrictions [7]. Another category of vulnerable subjects in the lockdown period comprises frontline doctors and nurses. Indeed, they were very susceptible to developing mental health problems because of their psychological weakness and fragility due to restrictions that forced them to be away from their family and friends for a long time, fighting against the virus and the disease, in situations of severe stress and physical fatigue. In severe cases, this situation caused suicide episodes or attempts. The category of teachers and professors was also severely tested by the pandemic and the social distancing. In the lockdown phase, there was an increase in symptoms of anxiety and depression [8], a decrease in interpersonal relationships [9] and a worsening of sleep quality [6,10].

Several factors can affect the mental health consequences of lockdown. The most important are the duration of the social isolation period, the characteristics of the living space, the number of online (virtual) and offline (physical) contacts and perceived contacts’ closeness, individual characteristics, and the spread of infection in the geographical area of residence [11]. The main goal of this work is to investigate the possible effects of environmental, social and individual characteristics (predictors) on mental health (response) during the COVID-19 lockdown period. The relationship between mental health and predictors can be represented with a linear regression model and studied through a suitable regression analysis. We consider and analyze data collected in a survey conducted in Italy in 2020, and designed by a research group within a project focusing on a psychological study.

According to what we said above, the concept of “mental health” is multidimensional because it includes several partial aspects. Hence, the dependent variable of the regression model mentioned above is not univariate. As a consequence, we take into account a multivariate regression model. To test the effects of the explanatory variables on the dependent variables, a combined permutation test is carried out [12]. Indeed, this methodology is suitable because it is based on the idea that the general test of the validity of the model (Multivariate Regression Analysis of Variance) is a multiple test that combines all the tests on the significance of the single regression coefficients. It is a non-parametric solution because it follows the permutation approach. Hence, it is more flexible and robust with respect to the underlying distribution than Pillai’s test [13] or other typical parametric methods. In its general version, Pillai’s test statistic is a trace of the product of the extra sum of squares and cross product (SSCP) matrix and the inverse of the error SSCP matrix of the reduced model. The extra SSCP matrix is the difference between the error SSCP of the reduced model (the model under the null hypothesis) and the error SSCP of the full model. Equivalently, Pillai’s test statistic can be obtained as a function of the non-zero eigenvalues of the product between the extra SSCP matrix and the inverse of the error SSCP matrix of the full model. Under the null hypothesis, this test statistics approximately follows an F distribution, and the reliability of the inferential results depends on the level of approximation of such null distribution [14]. The methodology applied in this paper is suitable for both normal and non-normal (multivariate) errors and, above all, it does not make assumptions about the dependence structure of the error terms.

The application of permutation tests to linear regression models dates back to the 1930s [15,16,17,18]. To test the significance of one regression coefficient in the presence of nuisance explanatory variables (with the need to control for the effects of such variables), several authors proposed solutions based on the idea that a suitable test statistic is the square of the correlation between the residuals of the regression of the response on the nuisance predictors and the residuals of the regression of the predictor of interest on the nuisance variables [19,20,21,22,23]. In other words, the partial test is based on the correlation between the parts of the dependent and independent variables under test that are not explained by the remaining independent variables. The approach based on the so-called permutation of regression residuals (PRR) consists of replacing in the model the explanatory variable of interest with the residuals of the linear regression of this explanatory variable on the other predictors. It was proposed by Potter (2005) for logistic regression [24], and then extended to a General Linear Model [25]. In order to obtain the null distribution of the test statistic and compute the p-value, an alternative approach to the permutation of residuals is the so-called conditional approach, based on the permutation of the dependent variable [26]. Permutation tests have been successfully applied to mixed models [27,28] and, as said before, to Generalized Linear Models [29,30]. For goodness-of-fit tests in linear regression models, some permutation solutions are based on partial sums or on cumulative sums of residuals [31,32,33]. The use of the non-parametric combination of dependent permutation tests to make inferences of the effects of covariates is proposed in [28]. Another work wherein the overall test of the multivariate linear model is conceived as a multiple test is [34], even if the solution proposed in this paper is based on the interesting theory of rotation tests, rather than on the permutation approach.

The rest of the paper is organized as follows. Section 2 is dedicated to the description of the dataset, the definition of the testing problem and the presentation of the applied non-parametric method. Section 3 contains results of the empirical study, and comments on the output. Section 4 provides a discussion of the main findings of the work and the final conclusions.

2. Materials and Methods

2.1. Population and Data

The study population is represented by adult Italians during the period of the COVID-19 lockdown. The dataset refers to a survey carried out by the University of Milano Bicocca in Italy in March 2020 during the period of the first lockdown due to the COVID-19 pandemic (for details, see [11]). Specifically, the analyzed sample consists of 1177 Italians. The questionnaire was administered with the CAWI (Computer-Assisted Web Interview) method. A convenience sample of 2470 persons accessed the online questionnaire, but many of them just opened the link without answering any questions, only answered some preliminary questions, or did not give their consent to data treatment. These subjects could not be included in the sample. The original number of subjects in the sample was 1252, but in 75 cases (6% of the original sample) there were missing values. The missing data concerned both independent and explanatory variables. Since there is no reason to think that the probability of missingness depends on the values of some variables, we can assume a “Missing Completely At Random” (MCAR) mechanism. Hence, given that, under the MCAR assumption, excluding from the sample statistical units with missing data does not lead to inferential bias [35] and that the number of individuals with missing data in this sample is quite low, we decided to remove these 75 subjects from the sample.

The dependent variables are:

• Depression;
• Unworthiness;
• Alienation;
• Helplessness.

Each of these consists of the average of the values of five items on a Likert scale (1 = not at all, 7 = extremely) (see Appendix A for details). According to the specialized literature, a suitable composite index for each of the four components of mental health can be obtained as a sum of the scores of the five items [11]. To facilitate the interpretation of the variables with respect to the original scale from 1 to 7, for each component, instead of the sum, we adopted the arithmetic mean of the five values. Actually, this mean differs from the composite index only by a scale factor 1/5, and the inferential results are not affected by this choice. Therefore, for each response, the greater the value, the worse the mental health. The independent variables are:

• Space Adequacy—Size. This predictor refers to the adequacy of the living space, the environment in which the person spent the period of restrictions, in terms of size. It is measured by an integer value ranging from 1 to 7;
• Space Adequacy—Brightness. This explanatory variable relates to the adequacy of brightness of the living space. The domain contains the integer values from 1 to 7;
• Space Adequacy—Privacy. This is the third variable concerning the suitability of the living place, and it concerns the level of privacy protection in the living space. It takes integer values from 1 to 7;
• Gender—This takes value 1 for males and 0 for females;
• Age—Years of life of the person at the moment of the interview;
• Occupation—Dichotomous variable that takes value 1 if the subject is employed and 0 otherwise;
• Higher Education—This denotes an educational level corresponding to secondary or academic education. It takes value 1 in case the person completed high school or achieved a university degree, and 0 otherwise;
• Social Isolation Length—Integer number denoting the number of days since the beginning of the lockdown at the moment of the interview. It takes values from 4 to 20;
• Infected Ratio Province-Region—Ratio between the number of infected persons in the province where the individual lives and the number of infected persons in the region where the individual lives. The proportion of those infected in the region of residence who come from the province of the subject measures the level of local risk of the area of residence, which is presumed to negatively affect the psychological well-being of the subject;
• Infected Ratio Region-Country—Ratio between the number of infected persons in the region where the individual lives and the number of infected persons at the national level. It has a similar meaning and role to the previous variable, but it considers a wider geographical area;
• Provincial Incidence—Incidence of COVID-19 in the population of the province of residence. Unlike the two previous variables, it measures the spread of the virus locally with respect to the population, and not relative to other areas. This variable is calculated as the ratio between the number of individuals infected in the province on the day of survey completion and the overall province population multiplied by 100,000;
• Number of Contacts—Integer value that denotes the number of subjects (friends or relatives) with whom the interviewed was in contact (virtually or in real) in the lockdown period;
• Proportion Offline Contacts—Proportion of real (not virtual) contacts to the overall total of contacts in the lockdown period;
• Offline Contacts’ Closeness—Mean perceived closeness of offline contacts. For each contact, the closeness is expressed on a Likert scale from 1 (not close at all) to 5 (extremely close);
• Online Contacts’ Closeness—Mean perceived closeness of online contacts. For each contact, the closeness is expressed on a Likert scale from 1 (not close at all) to 5 (extremely close).

2.2. Model and Testing Problem

Given that the dependent variable “mental health” is multidimensional and, in our study, is represented by four variables, a suitable model to investigate the relationship between mental health and the explanatory variables listed above must be composed of four equations, one for each response. In other words, a multivariate regression analysis must be carried out.

In general, let us assume that the multivariate dependent variable is q-dimensional and that k predictors are considered in the study. The multivariate (specifically q-variate) linear model is:

$$Y_{ij}={\beta }_{0j}+ {{\displaystyle \sum }}_{v=1}^k{\beta }_{vj}{x}_{iv}+{\epsilon }_{ij},$$

(1)

with $i=1,\dots ,n$ (subjects) and $j=1,\dots ,q$ (dependent variables). In the classic regression analysis, the error terms are supposed to be normally distributed with a null mean and constant variance within each equation, and uncorrelated (consequently independent because Gaussian) with respect to units. Formally, for the random variable ${\epsilon }_{ij}$, the conditions $E\left[{\epsilon }_{ij}\right]=0, Var\left[{\epsilon }_{ij}\right]=Var\left[Y_{ij}\right]=E\left[{\epsilon }_{ij}^2\right]={\sigma }_j^2$ and $Cov\left[{\epsilon }_{ij}{\epsilon }_{ur}\right]=E\left[{\epsilon }_{ij}{\epsilon }_{ur}\right]=0$ with $i\ne u=1,\dots ,n \mathrm{and} j,r=1,\dots ,q$. It is worth noting that $Cov\left[{\epsilon }_{ij}{\epsilon }_{ir}\right]=E\left[{\epsilon }_{ij}{\epsilon }_{ir}\right]$ could be not equal to 0 because, for a given statistical unit, the errors referring to different responses could be correlated, thus the equations could be related.

The matrix representation of the model is:

$$\begin{array}{c}Y\\ n\times q\end{array}=\begin{array}{c}X\\ n\times \left(k+1\right)\end{array}\begin{array}{c}B\\ \left(k+1\right)\times q\end{array}+\begin{array}{c}E\\ n\times q\end{array}$$

(2)

where the first column of $X$ is the vector of ones corresponding to the q constants of the model. If we consider the generic $i$-th row of the random matrix $E$, denoted by ${\epsilon }_{\left(i\right)}$, the probabilistic assumptions of the model error can be simply represented by the equation

$${\mathit{\epsilon }}_{\left(\mathit{i}\right)}=\left({\epsilon }_{i1},\dots ,{\epsilon }_{iq}\right)~{\mathcal{N}}_{\mathit{q}}\left({\mathbf{0}}_{\mathit{q}},\Sigma \right)$$

where the q-variate errors, in the classic multivariate linear regression model, are independent and identically (normally) distributed random variables, with null vector of means and a constant $q\times q$ covariance matrix $\Sigma$. In general, for inferential purposes, the assumptions of normality and independence can be relaxed. The application of a nonparametric approach, without assuming a specific family of distributions for the multivariate errors, makes the inference of the model robust with respect to the departure from normality. The possibility of deriving reliable results with a valid approach, without assuming independence in the error terms, implies a higher flexibility. For these main reasons, for testing the significance of the regression coefficients, we adopt the permutation method. In this case, the assumption of independence is replaced by the milder condition of the exchangeability of the errors with respect to units. This assumption is satisfied because, under the null hypothesis, all the regression coefficients are equal to zero, and the model in each of the $q$ equations includes only the intercept. Consequently, when the null hypothesis is true, the conditional mean of $Y$ given $X$ does not depend on the predictors, and the null distribution of the test statistic can be obtained by permuting the residuals or, equivalently, reassigning the q-dimensional rows of $Y$ to the $\left(k+1\right)$-dimensional rows of $X$ (or vice-versa). In fact, we are interested in testing the significance of all the regression coefficients, jointly considered; i.e., we test the null hypothesis that no explanatory variable affects any dependent variable, versus the alternative hypothesis that at least one explanatory variable affects at least one dependent variable (negation of the null hypothesis). In terms of regression coefficients, the hypothesis under test can be represented as follows:

$$H_0:{\beta }_{11}={\beta }_{12}=\cdots ={\beta }_{kq}=0 \mathrm{vs}. H_1:\overline{H_0}.$$

(3)

This problem is the classic Multivariate Analysis of Variance (MANOVA) of the linear regression model.

2.3. Permutation Test

The testing problem defined above can be considered as a multiple test composed of all the tests on the single regression coefficients (partial tests). The null and alternative hypotheses of the partial test concerning the coefficient related to the $v$-th independent variable and the $j$-th dependent variable can be represented as $H_0^{vj}:{\beta }_{vj}=0_{}$ and $H_1^{vj}:{\beta }_{vj}\ne 0$ respectively. Hence, the null hypothesis of the overall problem is

$$H_0:{{\displaystyle \cap }}_{v=1}^k{{\displaystyle \cap }}_{j=1}^q{H}_0^{vj}$$

(4)

and the alternative hypothesis of the overall problem is

$$H_1:{{\displaystyle \cup }}_{v=1}^k{{\displaystyle \cup }}_{j=1}^q{H}_1^{vj}$$

(5)

where the symbol of “intersection” in Equation (4) implies that, under the overall null hypothesis, all the partial null hypotheses are true, and the symbol of “union” in Equation (5) means that under the overall alternative hypothesis, at least one partial alternative hypothesis is true.

The methodological solution applied to this problem consists of a combined permutation test. This method is very useful for solving complex problems, e.g., multivariate problems or problems wherein a multivariate statistical test might be suitable. The main advantage with respect to other standard parametric methods is that the multivariate distribution of the test statistic does not need to be known or estimated, and in particular, the dependence structure between variables does not need to be explicitly known or specified [36]. Permutation tests are distribution-free, hence they are flexible and robust with respect to the departure from normality [37].

The main idea of a combined permutation test is to find a suitable statistic to solve each partial test, and to combine the permutation p-values of the partial tests in order to solve the overall problem [36]. The absolute values of the least square estimators of the regression coefficients represent reasonable and appropriate test statistics for the partial tests of our problem. The procedure consists of the following steps:

• Computation of the vector of observed values of the test statistics
  ${\mathit{t}}_{\mathbf{0}}=\left(\left|b_{11}\right|,\left|b_{12}\right|,\dots ,\left|b_{kq}\right|\right)=\mathit{t}\left(X\right)$;
• B independent random permutations of the rows of the X matrix: $X_1^{*},X_2^{*},\cdots ,X_B^{*}$;
• Computation of the values of the test statistic vector for the B dataset permutations ${\mathit{t}}_{\mathit{b}}^{*}$ and the corresponding vector of p-values ${\mathit{l}}_{\mathit{b}}^{*}$, with $b=1,2,\dots ,B$;
• Computation of the value of the combined test statistic for each permutation and for the observed dataset through the combination of the partial p-values with a suitable function $\psi :{\left[0,1\right]}^{kq}\to ℝ$, $t_{comb,b}^{*}=\psi \left({\mathit{l}}_{\mathit{b}}^{*}\right)$;
• Computation of the p-value of the combined test according to the null permutation distribution.

The dependence of the partial tests is implicitly taken into account through the permutation of the rows of the dataset. Assuming, without loss of generality, that the null (partial and overall) hypotheses are rejected for large values of the test statistics, a suitable combining function $\psi$ should satisfy the following reasonable and mild conditions: (1) it is a monotonic non-increasing function of the p-values; (2) when one p-value tends to zero, it tends to the supremum, and when one p-value tends to one it tends to the infimum; (3) the acceptance region is limited. One of the most commonly used combining functions is that of Tippett:

$$t_{comb,b}^{*}=\underset{v,j}{\max }\left(1-l_{vj,b}^{*}\right)$$

In particular, Tippett’s combination function provides powerful tests when one or a few, but not all, of the partial alternative hypotheses are true [36]. The parametric methods used for the solution of univariate and multivariate testing problems require very restrictive assumptions, which are often unrealistic and not justified by the empirical evidence or by asymptotic theories. In particular, the typical assumptions of the classic linear regression model, such as the normality and uncorrelation of the error terms, are rarely satisfied or approximately satisfied. The permutation tests are preferable to the parametric tests when the parametric assumptions do not hold. However, they are performant also when these assumptions are valid, and in general, they represent a robust and flexible solution for complex testing problems.

Since the permutation MANOVA solution for the linear regression model is defined as a multiple test, in the case of a rejection of the null hypothesis in favor of the alternative and in order to attribute the overall significance to specific partial tests (i.e., to specific coefficient estimates), the control of the family wise error (FWE) is necessary [38]. In other words, to avoid the inflation of the type I error of the overall test, we must adjust the partial p-values. A suitable method, less conservative than the Bonferroni rule, is the minP presented and discussed in [39,40].

The typical parametric approach to test the goodness-of-fit of a model is based either on a general test on the whole model, or on a stepwise procedure based on the sequential application of t tests on the significance of the single regression coefficients. The typical general test on the whole model is the F test in the univariate model, the Pillai’s trace, the Wilk’s lambda, or the Roy’s largest root in the multivariate case [41]. In our opinion, the use of the described nonparametric method, based on the combined permutation test and on the adjustment of partial p-values, is not only appropriate in cases of possible violations of the assumption of normality or other typical assumptions of the parametric tests, but also consistent with the definition of the MANOVA problem as the multiple test composed of $q\times k$ partial tests, represented in (3)–(5).

Another important limit of the parametric approach concerns the degrees of freedom. When the number of partial tests is very large with respect to the sample size, there is a loss of degrees of freedom and a decrease in power. In the regression analysis, when the number of explanatory variables is larger than the sample size, the parametric approach is not applicable. On the other hand, when the number of partial tests under the alternative hypothesis increases (e.g., when new predictors whose regression coefficients are not null are added in the model), the power of the combined permutation tests increases. Hence the nonparametric procedure described above, when the number of independent variables is greater than the sample size, is not only feasible, but also very powerful [42]. Obviously, in the regression analysis, the problem of multicollinearity must not be ignored. Hence, when the number of predictors is very large, before the application of the permutation approach, multicollinearity should be avoided by computing the Variance Inflation Factors (VIFs) and eliminating from the model the predictors with high VIF (usually VIF > 5).

We considered the possibility of developing the analysis in the context of Generalized Linear Models. Indeed, this approach is very flexible and compatible with the use of permutation tests. However, we discarded this idea because the use of GLM is mainly adopted for non-normal errors or peculiar responses such as categorical, binary, mixed variables, etc. [43]. In our opinion, none of these reasons apply to our problem, given the responses under study and the flexibility and robustness of the permutation approach with respect to the departure from the assumption of errors normally distributed.

The analysis was carried out with original R scripts created by the authors, while also using some packages available in the CRAN network and a source file available online. The regression and diagnostic analysis for linear models was carried out with the basic commands lm(), qqnorm(), qqline(), hist() and plot() (packages stats and graphics). The Variance Inflation Factor (VIF) for the detection of multicollinearity problems in the matrix of regressors was performed with the command vif() available in the package usdm. For the implementation of the combined permutation tests, the following commands created by the authors were used (source file cptlm.R):

• perm() for creating the null permutation distribution of the multivariate test statistic by permuting the rows of the matrix of regressors;
• slf(): for the application of the significance level function for the computation of the p-values;
• psi(): for the nonparametric combination.

The function FEW.minP (source file FWEminP) was used for the adjustment of p-values (see [38]).

3. Results

3.1. The Dependent Variables

As said above, the responses of the model consist of four outcomes that represent partial aspects of the mental health under investigation. In

Table 1. Descriptive statistics of the dependent variables.

Dependent Variables	Mean	Median	Std. Deviation	Std. Error
Depression	2.96	2.80	1.45	0.042
Unworthiness	3.43	3.40	1.20	0.035
Alienation	3.11	3.00	1.44	0.042
Helplessness	3.91	3.80	1.45	0.042

, the mean, median and standard deviation of each dependent variable are reported. According to the values of the standard deviations, the variabilities of the data for the different outcomes seem to be very similar, in particular those of Depression, Alienation and Helplessness. An exception is represented by Unworthiness, whose distribution presents a lower dispersion around the central tendency. In other words, the unworthiness self-perceived by the respondents is more homogeneous among the individuals than the other three psychological conditions.

If we consider that the scale of the response variables is from 1 (not at all) to 7 (extremely), that mental health is worse for larger values and that the central value corresponding to the neutral evaluation is 4, according to sample mean and median, we can say that the mental health state tends not to be so bad. In fact, all means and medians are less than 4. In particular, the health issue seem to be less serious in terms of depression. This is the only variable with a mean and median that are both less than 3. In all the other cases, both sample location indices are greater than or equal to 3. If Alienation presents location measures close to those of Depression, this is not the case of Unworthiness and Helplessness—especially the latter, which has values not far from 4. Definitely, the state of mental health of the respondents in general seems not too worrying, but it is worse in terms of Helplessness, while being less relevant in terms of Depression.

3.2. Multivariate Regression Analysis

The regression coefficients of the model represented in (1) and (2) were estimated with the OLS method. Before the parameters’ estimation, we computed the VIFs of all the independent variables to investigate the possible collinearity. All the VIFs are less than 5, indicating that there is no multicollinearity issue for the data matrix of the explanatory variables. The permutation MANOVA described in Section 2.3 was applied to the data to test the hypotheses defined in (3) at the significance level $\alpha =0.10$. The p-value of the combined permutation test based on the Tippett combination is equal to 0.005. Since it is less than $\alpha$, the null hypothesis that all the regression coefficients are equal to zero must be rejected in favor of the alternative hypothesis that at least one coefficient is not equal to zero. Hence, we have empirical evidence of the effect produced by one or more predictors on the mental health of Italians during the lockdown due to the COVID-19 pandemic. In other words, some of the independent variables considered in this study affect one or more responses, because we have identified a significant effect of some explanatory variables on some outcomes (the estimates of some coefficients in the four equations are significant).

From a descriptive point of view, the goodness-of-fit is not high for any of the four equations of the multivariate model. The adjusted $R^2$ are 0.115 (Depression), 0.178 (Unworthiness), 0.173 (Alienation) and 0.109 (Helplessness). Maybe this is due to the lack of some important predictors; it may also depend on the model specification. A possible remedy to improve the goodness-of-fit with a different model specification could be the inclusion among the regressors of transformations and/or interactions of the original predictors. We think that this approach is not appropriate in this case for two main reasons: (1) it should be, at least partially, justified by the theory about factors affecting mental health and their functional relationships; (2) given the number of variables involved in the study, the possible different model specifications, even remaining in the context of linearizable models, would be too many. Regarding the former point, we did not find any useful references about non-linear functional relationships or interaction effects in the specialized literature. On the latter point, we think that the data-driven approach to model specification would risk becoming a computationally demanding exercise, that would not necessarily be successful, and that would risk becoming an excessive persistence in the search of a better goodness-of-fit, which does not actually contribute to explaining the relationships between variables from a psychological point of view. Furthermore, since our study employs an inferential and not a descriptive perspective, the results of the tests are much more important than the values of the R-squared. Regarding the low R-squared values, we do not believe that we should give too much importance to this aspect, given the significance of the global test and of the specific estimates of the regression coefficients. Low R-squared values could be due to the high variability of responses. The values of the coefficients of determination do not indicate by themselves whether the model is good or bad, nor do they indicate whether estimators and predictions are biased or not [44,45]. Hence, in our opinion, the empirical evidence derived from the application of the presented inferential approach to the data of the mentioned survey is interesting, and deserves attention in the debate on the factors affecting the mental health of people during the COVID-19 pandemic. In

Table 2. Estimates and adjusted p-values of the partial permutation tests on the regression coefficients of the multivariate regression model (significant estimates in bold).

	Depression		Unworthiness		Alienation		Helplessness
	Coeff	Adjusted p-Value	Coeff	Adjusted p-Value	Coeff	Adjusted p-Value	Coeff	Adjusted p-Value
Intercept	5.736		6.370		6.746		7.261
Space adequacy: Size	0.025	N.S.	0.027	N.S.	0.030	N.S.	-0.007	N.S.
Space adeq.: Brightness	−0.123	0.017 **	−0.104	0.009 ***	−0.064	N.S.	−0.097	N.S.
Space adeq.: Privacy	−0.125	0.005 ***	−0.101	0.005 ***	−0.117	0.005 ***	−0.105	0.005 ***
Gender	−0.066	N.S.	−0.061	N.S.	0.138	N.S.	−0.135	N.S.
Age	−0.009	N.S.	−0.021	0.005 ***	−0.026	0.005 ***	−0.011	N.S.
Occupation	−0.176	N.S.	−0.230	0.099 *	−0.128	N.S.	−0.054	N.S.
Social Isolation	0.024	N.S.	0.013	N.S.	0.004	N.S.	0.017	N.S.
Infected ratio Prov–Reg	−0.404	N.S.	−0.151	N.S.	−0.021	N.S.	−0.456	N.S.
Infec. ratio Reg–Country	−0.226	N.S.	−0.146	N.S.	−0.093	N.S.	−0.284	N.S.
Provincial Incidence	0.000	N.S.	0.000	N.S.	0.001	N.S.	0.000	N.S.
High Education	0.033	N.S.	0.053	N.S.	−0.047	N.S.	−0.229	N.S.
Number of contacts	−0.044	0.009 ***	−0.031	0.029 **	−0.051	0.005 ***	−0.034	0.081 *
Proportion Offline	−0.173	N.S.	−0.129	N.S.	−0.105	N.S.	−0.729	N.S.
Off. Contacts Closeness	−0.134	N.S.	−0.151	0.013 **	−0.134	0.099 *	−0.137	N.S.
On. Contacts Closeness	−0.099	N.S.	−0.118	N.S.	−0.297	0.005 ***	−0.196	0.009 ***

*: weak significance (p < 0.10); **: moderate significance (p < 0.05); ***: strong significance (p < 0.01).

, estimates and p-values of the partial tests on the regression coefficients are shown.

According to the adjusted p-values, the significance of the overall test can be attributed only to some of the partial tests on the single coefficients. We speak of weak significance when the p-value is less than 0.10 but greater than 0.05, moderate significance when the p-value is between 0.01 and 0.05, and strong significance when the p-value is less than 0.01. In particular, Depression is strongly affected by the number of contacts and the space adequacy in terms of privacy. A moderate effect on Depression of the space adequacy in terms of brightness is also present. Space adequacy with respect to brightness and privacy also strongly affect Unworthiness, together with age. A moderate significant effect on Unworthiness can be attributed also to the number of contacts and the perceived closeness of the offline contacts. Being employed is also an important condition that affects Unworthiness (even if with weak significance). Alienation is affected by a lack of privacy, age, number of contacts, and the perceived closeness of the online contacts (strong significance). A (weak) significant effect on Alienation is also provided by the perceived closeness of the offline contacts. Finally, Helplessness is only affected by place adequacy in terms of privacy and the perceived closeness of the online contacts (strong significance), as well as by the number of contacts (weak significance). All the coefficient estimates are negative. This is consistent with the expected sign in almost all the cases. An exception to this general rule is represented by age. This factor produces a strong significant effect on Unworthiness and Alienation. A priori, we could not say whether mental health was negatively or positively correlated with age, but the empirical evidence suggests that the younger the person is, the worse the mental health state with respect to the two mentioned outcomes.

3.3. Analysis of Residuals

According to the histograms of the residuals, the marginal distributions of the errors could be asymmetric. This is more evident for the regression equations concerning Depression and Alienation (see Figure 1).

Hence the condition of normality for the multivariate errors of the model seems to not be satisfied. The implausibility of this assumption is confirmed by the normal probability plots of the marginal distributions shown in Figure 2. These findings reinforce the conviction of the opportunity to adopt a nonparametric approach such as that based on the combined permutation test. In fact, with this method, as said before, the assumption of normality is relaxed, and the family of multivariate distributions for the error terms of the model does not need to be known.

Another important aspect to be considered concerns the dependence structure between the four equations of the multivariate model. Unless the independence (or at least the uncorrelation) of the errors corresponding to different equations is assumed, the dependence should be specified, and the parameters representing such dependence (e.g., the correlations) should be estimated, adding uncertainty in the results and reducing the degrees of freedom. The methodology of combined permutation tests does not require such a strong assumption, but the between-equation dependence is contemplated without the need for modeling it or estimating parameters.

Table 3. Sample correlations between the dependent variables.

	Depression	Unworthiness	Alienation	Helplessness
Depression	1.000	0.672	0.579	0.623
Unworthiness		1.000	0.457	0.532
Alienation			1.000	0.449
Helplessness				1.00

shows the sample correlations between the residuals of the four equations. All these correlations are positive and take values between 0.4 and 0.7, denoting a relevant linear dependence between the four variables. We limit the output at the correlations not because we assume that the type of dependence is linear (as said, the assumption about the type of dependence is not necessary), but to show that the equations are related, and a suitable multivariate approach that takes into consideration such dependence is needed. Indeed, correlation implies dependence.

4. Concluding Remarks

This work explores the effects of several demographic, socio-economic and environmental factors on the mental health of Italians during the COVID-19 lockdown. A multivariate regression analysis with the application of a permutation multiple test on the data of a sample survey carried out in Italy in 2020 is proposed.

The nonparametric test, based on a nonparametric combination of permutation tests on the significance of the single regression coefficients’ estimates, represents a valid solution to the MANOVA problem for the multivariate regression model. This method, used jointly with the adjustment of the p-values in the partial tests on the coefficients for controlling the FWER for the multiplicity, is preferable to the classic parametric approach based on the separate application of the overall test on the whole model (Wilk’s lambda, Pillai’s trace, Roy’s largest root or others), and the single t tests on the regression coefficients. First of all, the method is based on a procedure that takes into account the nature of multiple tests of the overall problem. Second, it is distribution-free, and consequently robust with respect to the departure from normality, or other eventual assumed probability laws of the error in multivariate distribution. Finally, it does not require that a specific dependence structure between the different model equations is assumed. Such dependence is implicitly taken into account in the testing procedure. In other words, the procedure used in the paper not only has the advantage of being more flexible and robust than methods based on the parametric approach, but it also considers the MANOVA for multivariate regression models as a multiple test. It incorporates in a single procedure the test on the overall model and the test on the single regression coefficients, which in practice is not often done, and it appears the most appropriate approach when the interest is both on the goodness-of-fit of the overall model and on the single regression coefficients.

Regarding the output of the multivariate regression analysis and the permutation MANOVA, it is evident that some factors do not affect any mental health aspect. For instance, this is the case of the adequacy of the living space in terms of size. Additionally, gender and education are not relevant predictors, nor are any of the variables that represent the severity of the pandemic at the local level in terms of spread, such as the ratio of infected province/region, the ratio of infected region/country, and the provincial incidence. Another factor that seems to have no effect on mental health is the social isolation length, that is, the number of days since the beginning of the lockdown at the moment of the interview. Finally, there is no empirical evidence of the importance of the proportion of offline contacts, i.e., the proportion of real contacts who are not online (virtual).

In conclusion, the adequacy of the living place matters, not in terms of size but mainly in terms of lack of privacy and, to a lesser extent, also in relation to brightness, which affects depression and unworthiness. This is consistent with some studies that suggest that an inadequate living space can compromise psychological well-being. In particular, space adequacy plays a key role in both low- and high-contagion areas. Indeed, the more adequate the space in which participants were confined, the fewer the mental health issues [46]. Age is the only relevant socio-demographic factor with a significant effect on mental health, because unworthiness and alienation are significantly higher for younger people. Additionally, the occupational condition plays a role among the considered predictors, because being employed reduces unworthiness. Changes in social life due to the lockdown, as expected, affect mental health. What matters is mainly the number of contacts rather than the type of contacts (Proportion Offline). The perceived closeness of contacts is also important: the closeness of offline contacts affects unworthiness and alienation, whereas the closeness of online contacts affects alienation and helplessness. In this case, several authors have also highlighted the risks of screen time for mental health, given that the time spent online reduces the commitment to offline activities and interactions [47]. These findings can be considered a good starting point in the study of the relationship between environmental, demographic and socio-economic factors and mental health, as a multivariate outcome composed of different dimensions, such as those considered in our study. Without claiming to have fully explained the phenomenon, we believe that this model and these results can represent a step forward in the study of this subject, and can represent a reference work that can be followed by other studies based on better models and a wider set of variables.