Next Article in Journal
Effects of Motivational Climate on Knowledge Hiding: The Mediating Role of Work Alienation
Next Article in Special Issue
Gender Differences in Attachment Anxiety and Avoidance and Their Association with Psychotherapy Use—Examining Students from a German University
Previous Article in Journal
The Nexus between Study Burnout Profiles and Social Support —The Differences between Domestic (Finnish) and International Master’s Degree Students
Previous Article in Special Issue
Job Satisfaction and Psychological Distress among Help-Seeking Men: Does Meaning in Life Play a Role?
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Behavioral Sciences
Volume 12
Issue 3
10.3390/bs12030080
behavsci-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Examining the Role of Traditional Masculinity and Depression in Men’s Risk for Contracting COVID-19

by
Andreas Walther
1,*,
Lukas Eggenberger
1,
Jessica Grub
1,
John S. Ogrodniczuk
2,
Zac E. Seidler
3,4,
Simon M. Rice
3,4,
David Kealy
2,
John L. Oliffe
5,6 and
Ulrike Ehlert
1
1
Department of Clinical Psychology and Psychotherapy, University of Zurich, 8050 Zurich, Switzerland
2
Department of Psychiatry, University of British Columbia, Vancouver, BC V6T 1Z3, Canada
3
Centre for Youth Mental Health, The University of Melbourne, Parkville, VIC 3010, Australia
4
Orygen, Melbourne, VIC 3052, Australia
5
School of Nursing, University of British Columbia, Vancouver, BC V6T 2B5, Canada
6
Department of Nursing, University of Melbourne, Parkville, VIC 3010, Australia
*
Author to whom correspondence should be addressed.
Behav. Sci. 2022, 12(3), 80; https://doi.org/10.3390/bs12030080
Submission received: 29 November 2021 / Revised: 24 February 2022 / Accepted: 10 March 2022 / Published: 16 March 2022
(This article belongs to the Special Issue Male Depression and Therapy)
Browse Figures
Review Reports Versions Notes

Abstract

:
In the light of the COVID-19 pandemic and claims that traditional masculinity may put some men at increased risk for infection, research reporting men’s health behaviors is critically important. Traditional masculine norms such as self-reliance and toughness are associated with a lower likelihood to vaccinate or follow safety restrictions. Furthermore, infection risk and traditional masculinity should be investigated in a differentiated manner including gender role orientation, underlying traditional masculine ideologies and male gender role conflict. In this pre-registered online survey conducted during March/April 2021 in German-speaking countries in Europe, 490 men completed questionnaires regarding contracting COVID-19 as confirmed by a validated test, fear of COVID-19 (FCV-19S), and experience of psychological burden due to COVID-19. In addition, depression symptomatology was assessed by using prototypical internalizing and male-typical externalizing depression symptoms. Furthermore, self-identified masculine gender orientation, endorsement of traditional masculinity ideologies, and gender role conflict were measured. A total of 6.9% of men (n = 34) reported having contracted COVID-19 since the beginning of the pandemic. Group comparisons revealed that men who had contracted COVID-19 exhibited higher overall traditional masculine ideology and gender role conflict. Logistic regression controlling for confounders (age, income, education, and sexual orientation) indicated that only depression symptoms are independently associated with the risk of having contracted COVID-19. While prototypical depression symptoms were negatively associated with the risk of having contracted COVID-19, male-typical externalizing depression symptoms were positively associated with the risk of contracting COVID-19. For traditional masculinity, no robust association for an increased risk of contracting COVID-19 could be established, while higher male-typical externalizing depression symptoms were associated with an increased risk of contracting COVID-19.

1. Introduction

The COVID-19 pandemic poses major health and economic ramifications worldwide and effectively responding to the pandemic remains an ongoing challenge. Investigation of the pandemic is critical to ending it and better controlling future pandemics [1]. As one such central concept that might underlie the increased spread of the COVID-19 virus, traditional masculinity is increasingly coming into focus [2].
Although men and women were infected with COVID-19 to a similar extent at the beginning of the pandemic, men were much more likely to die from COVID-19 [3]. Research indicates that the cause of the increased mortality rate in infected men was not rooted in male biology but in men’s health risk behaviors such as smoking and drinking, which can complicate COVID-19 infections and may lead to a more severe disease course [3,4]. Traditional masculinity in connection with health risk behavior has become increasingly important in explaining why specific sub-populations adhere less to state-imposed protective measures such as mask wearing or show increased mortality [2].
Traditional masculinity has long been related to men’s health risk behaviors [5,6]. This is reflected by consistent findings of gender differences emerging from the current pandemic showing that men were found to be more likely than women to downplay the risks associated with COVID-19 [7]. Men were also less likely than women to report fearing “very serious” consequences if they became infected [8,9]. Men in general were also less likely to follow public health protocols [10,11], including refusing to wear masks, demonstrating greater negative attitudes towards wearing a mask, and less reported handwashing and social distancing compared to women [9,11,12]. Relative to women, men are more likely to believe that wearing a face mask is embarrassing or a sign of weakness [9] and Palmer and Peterson [13] specifically linked stronger endorsement of the traditional male role norm of toughness with greater levels of negativity towards mask wearing. Taken together, men in general, show poorer adherence to protection measures and health behaviors and, therefore, are at increased risk for contracting COVID-19 in comparison to women [9,13,14,15].
These hesitant attitudes and behaviors to follow state-imposed protection measures for ending the COVID-19 pandemic by many men appear to be based on socially constructed and idealized male gender norms of strength, toughness, and self-reliance adopted during men’s gender role socialization [16,17]. Thus, studies comparing men with low and high adherence or conformity to traditional male role norms should provide further insight into the relation between traditional masculinity, following protection measures and the risk for contracting COVID-19. However, to date there are few available studies on this topic. A non-peer reviewed but publicly available report highlighted those men who identified as “completely masculine” were nearly three times more likely to report having been diagnosed with COVID-19 than men self-identifying as “mostly masculine” or “slightly masculine” [18]. Furthermore, the same authors reported that men self-identifying as “completely masculine” expressed less intention to get vaccinated for COVID-19 as men self-identifying as “mostly masculine” or “slightly masculine” [14]. Mahalik and colleagues [15] provided preliminary evidence that higher conformity to traditional masculine norms is associated with negative attitudes toward mask wearing during the ongoing pandemic and that conformity to traditional masculine norms interacts with more conservative political ideologies for the prediction of mask wearing attitudes.
The relationship between traditional masculine role norms and health behaviors or health outcomes in general is, however, complex. Greater adherence to traditional masculinity was initially associated with worse health behaviors or increased mortality [19]. A meta-analytic investigation revealed that conformity to traditional masculine norms is associated with overall worse mental health outcomes and reduced help-seeking [20]. Moreover, studies indicate traditional masculinity to be associated with delayed therapy initiation [21] and increased risk for suicide [22,23]. However, more recently, positive and health promoting aspects of traditional masculinity are being recognized [24]. The conceptualization and operationalization of masculinity has evolved over time, beginning in the gender role identity paradigm attributing gender-typical traits to men and women [25,26], followed by the gender role strain paradigm focusing on the adherence and conformity to traditional masculine role norms [27,28,29]. Based on the social constructionist understanding of masculinity, Levant and Wimer [30], for example, revealed that conforming to certain masculine norms acts as a protective buffer for some health behaviors, while others are consistently identified as risk factors. This was corroborated in a subsequent study showing that the traditional masculine role norm of primacy of work was mostly related with positive health outcomes, while for four other role norms (winning, risk-taking, pursuit of status, and disdain for homosexuals) a balance of positive and negative health outcomes was observed [24]. Yet, six subscales (emotional control, violence, power over women, dominance, playboy, and self-reliance) were predominantly related to negative health outcomes [24]. Therefore, besides reliance on total scores of commonly used masculinity measures, which often obscure the complexity of associations, subscales highlighting particular male role norms should be investigated and reported.
In addition, it is increasingly being discussed whether individuals with mental disorders might be at increased risk for contracting COVID-19 due to potential cognitive deficits including executive dysfunction, negative health behaviors (e.g., smoking), or structural barriers hindering the ability to successfully quarantine at home [31,32]. While there are reports suggesting depression is associated with an increased risk for contracting COVID-19 [33,34,35,36], a similar number of studies found no association or a reduced risk for contracting COVID-19 in these individuals [37]. There is also evidence that some individuals intentionally expose themselves to the virus with the intent of self-harm or to suicide, further suggesting increased risk for contracting COVID-19 among individuals with mental health problems [38]. However, a meta-analysis examining the relationship between mood disorders and the risk of contracting COVID-19 in over 91 million individuals could not identify significant associations, suggesting the need to examine more specific subgroups regarding this question [39].
As no association was identified in the above-mentioned meta-analysis on the relation between mood disorders and the risk of contracting COVID-19 [39], it emerges as relevant to investigate a more fine-grained research question regarding how certain disorders, and indeed their unique symptomatology, are associated with a risk of contracting COVID-19. Considering that individuals with prototypical depression symptoms such as depressive mood, anhedonia, or fatigue show increased social withdrawal, higher rates of social isolation and unemployment, one would assume a reduced risk of contracting COVID-19 due to the reduced social interactions [39]. In contrast, men with high traditional masculinity exhibit more male-typical externalizing depression symptomatology [21,40], which is characterized by anger and aggression, risk-taking, or substance misuse [40,41,42,43,44,45]. Such a symptom pattern might be assumed to be associated with an increased risk of contracting COVID-19. Risk-taking, for example, is characterized by not caring about the consequences of one’s actions, so men experiencing externalizing depression symptomatology may care less about their own well-being or the well-being of those around them.
It is, therefore, important to emphasize that the constructs of traditional masculinity and depression are not independent, as the endorsement of traditional masculinity ideologies directly influences the presentation of depressive symptoms [40,41,42,43,44,45]. While men with low endorsement of traditional masculinity ideologies and mental distress are more likely to exhibit prototypical depression symptoms, men with high endorsement of traditional masculinity ideologies and mental distress are more likely to show male-typical externalizing depression symptoms [40,41,42,43,44,45]. Thus, the association between the risk of contracting COVID-19 and traditional masculinity is not independent of depression symptomatology presentation and could possibly be represented by it rather than by a direct link between the endorsement of traditional masculinity ideologies and COVID-19 infection risk.
The aim of the present study was to investigate the relationship between traditional masculinity measured by different conceptual constructs (gender role orientation, traditional masculine ideology, gender role conflict), depressive symptomatology (prototypical and male-typical externalizing depression symptoms), and self-reported infection with COVID-19. Based on the outlined literature, we hypothesize traditional masculinity to be positively associated with having contracted COVID-19. We hypothesize this relationship uniformly for all masculinity constructs and their respective subscales. The a priori formulated directed hypotheses are retrievable under OSF (https://osf.io/q4pw3, accessed on 9 March 2022). To better understand the relationship between specific depression symptomatology and the risk of contracting COVID-19, we further investigated, in an explorative manner, whether men with higher prototypical depression symptoms are less likely to contract COVID-19 and whether men with higher male-typical externalizing depression symptoms are more likely to contract COVID-19.

2. Materials and Methods

2.1. Sample and Procedure

This anonymous, cross-sectional online survey entitled ‘Men’s Mental Health in Times of COVID-19′ was pre-registered and approved by the local ethics committee of the Faculty of Arts and Social Sciences of the University of Zurich (Authorization No. 21.2.4). Following the Open Science standards, a priori defined study hypotheses, statistical analyses, and the study specific data set can be retrieved from OSF (https://osf.io/q4pw3, accessed on 9 March 2022). For this study, male participants were recruited via advertisements distributed on social media platforms such as Facebook and the study’s webpage. Advertisements on social media platforms were restricted to men of 18 years or older in the countries of Germany, Switzerland, Austria, Lichtenstein, Luxembourg, and Belgium. Aiming to recruit a large sample of men older than 18 years with sufficient German language skills to read and respond to the questionnaire in German language, all men irrespective of mental health status were eligible to participate. During the recruitment period from 15 March 2021 to 28 April 2021, a total number of 1087 people expressed interest in this study by visiting the starting page of the online questionnaire. A little more than half of the initially interested participants (n = 597, 54.92%) were not included in the final analyses for one of the following reasons: data privacy agreement not provided, declaration of consent not provided, self-reported insufficient German language skills, gender requirements not fulfilled, age of minority and/or incomplete data in the questionnaires. This resulted in a total number of 490 participants included in the analyses. Figure 1 presents participant flow. After providing written informed consent at the beginning of the survey, participants agreed to the data privacy statement and then went on to answer sociodemographic questions, COVID-19-related questions, and a set of psychometric instruments. For the subsequently described study, the average completion time was 20 min. However, several participants went on to participate in further online experiments following the completion of initial questionnaires, which are beyond the scope of this report. Further information on the study is provided elsewhere [22] or on the preregistration in OSF: “osf-link”.

2.2. Instruments

2.2.1. Sociodemographics and COVID-19 Related Questions

The online survey started with sociodemographic questions assessing the sufficiency of German skills, gender, dimensional gender (positioning between the two poles 1 = masculine, 10 = feminine), age, height, weight, nationality, relationship status, sexual orientation, education, and the household’s yearly gross income. Participants were further asked whether a validated test confirmed that they were currently infected with COVID-19 or whether a validated test has previously confirmed that they had contracted COVID-19 since the beginning of the pandemic (yes/no). If participants answered “yes” to this question, they were asked which symptoms from a given list they currently or previously suffered from due to COVID-19 infection (e.g., respiratory symptoms, cardiovascular symptoms, fatigue, fever, pain symptoms, post-exertional malaise, cognitive symptoms, nausea, etc.). The list of symptoms was based on previous research investigating COVID-19 and long COVID symptoms [46]. Further descriptives are presented in Table 1.

2.2.2. Male Role Norm Inventory–Short Form (MRNI–SF)

The Male Role Norms Inventory–Short Form (MRNI–SF) consists of 21 items and measures of traditional masculine ideology with seven subscales [47]. The participant indicates the degree to which he agrees with traditional masculinity ideology statements on a seven-point Likert scale (1 = strongly disagree, 7 = strongly agree). The seven subscales of the MRNI–SF represent the following dimensions: restrictive emotionality, self-reliance through mechanical skills, negativity towards sexual minorities, avoidance of femininity, importance of sex, dominance, toughness. A Cronbach’s alpha of α = 0.92 for men has been reported for the original English version of the MRNI–SF [47]. Cronbach’s α in the current study: MRNI–SF total score = 0.94, MRNI–SF negativity towards sexual minorities = 0.87, MRNI–SF restrictive emotionality = 0.74, MRNI–SF self-reliance through mechanical skills = 0.88, MRNI–SF avoidance of femininity = 0.88, MRNI–SF importance of sex = 0.88, MRNI–SF dominance = 0.88, MRNI–SF toughness = 0.81.

2.2.3. Gender Role Conflict Scale–Short Form (GRCS–SF)

The Gender Role Conflict Scale [29] measures patterns of gender role conflict (GRC). According to O’Neil, GRC describes a psychological state in which socialized gender roles have negative consequences for the person or others and occurs when rigid, sexist, or restrictive gender roles cause restriction, devaluation, or violation of others or self. The Gender Role Conflict Scale–Short Form [48] measures the following four patterns of gender role conflict: (1) success, power, and competition (SPC), (2) restrictive emotionality (RE), (3) restrictive affectionate behavior between men (RABBM), and (4) conflict between work and family relations (CBWFR). Consisting of a total number of 16 items, four items are dedicated to each pattern of GRC. The participants indicate the degree of experienced conflict on a six-point Likert scale (0 = strongly disagree, 5 = strongly agree). Wester and colleagues [48] reported internal consistencies of α = 0.77–0.80. The German version of the Gender Role Conflict Scale–Short Form was used in the present study [49]. Cronbach’s α in the current study was at 0.79.

2.2.4. Patient Health Questionnaire-9 (PHQ-9)

The Patient Health Questionnaire (PHQ-9) [50] assesses nine symptoms of major depressive disorder specified by the Diagnostic and Statistical Manual of Mental Disorders 5 (DSM-5) [51]. For each of the nine symptoms, the participants rate how often they appeared within the preceding two-week period on a four-point Likert scale ranging from 0 (not at all) to 3 (almost every day). The PHQ-9 is applied in criteria-based diagnoses of depressive disorders with a cut-off ≥10 within research and clinical practice [52] and has been shown to provide a reliable and valid measure of depression severity [50]. In this study, a German version of the PHQ-9 was used, which has been previously validated on a representative German-speaking sample (Cronbach’s α = 0.89) [53]. Cronbach’s α in the current study was at 0.90.

2.2.5. Male Depression Risk Scale-22 (MDRS-22)

The Male Depression Risk Scale (MDRS-22) consists of 22 items assessing externalized depressive symptoms [40]. For each item, the participant rates how often the symptom appeared within the preceding month on an eight-point Likert scale ranging from 0 (not at all) to 7 (almost always). The scale consists of six domains and considers gender-specific externalizing symptoms of depression and assesses them within the context of cultural norms related to masculinity. The six domains of the MDRS-22 include emotion suppression (i.e., “I bottled up my negative feelings”), drug use (i.e., “I used drugs to cope”), alcohol use (i.e., “I needed to have easy access to alcohol”), anger and aggression (i.e., “I overreacted to situations with aggressive behaviors”), somatic symptoms (i.e., “I had regular headaches”). In the present study, the validated German version of MDRS-22 was used (Guttman’s λ2 = 0.62–0.91) [43]. Cronbach’s α in the current study was at 0.88.

2.2.6. Fear of COVID-19 Scale (FCV-19S) and COVID-19 Pandemic Stress Scale (CPSS)

As a measure for the evaluation of discriminate validity, with regard to depression measures, we included the German version of the fear of COVID-19 scale [54], consisting of 7 items where participants have to indicate the extent to which they do agree with the statements on fear of COVID-19 on a five-point Likert scale (1 = do not agree at all; 5 = fully agree). Cronbach’s α in the current study was at 0.85. We further included the German version of the COVID-19 Pandemic Stress Scale, consisting of 10 items on aspects and consequences of COVID-19, such as hygienic behavior rules, contact restrictions, or actual fear of COVID-19 infection [55]. The participants had to rate how stressed or anxious they felt about these topics during the previous two weeks on a four-point Likert scale (0 = not burdened at all, 4 = very burdened). Cronbach’s α in the current study was at 0.74.

2.3. Statistical Analysis

The statistical analyses and computations were performed with the software R [56] and the supplementary packages “psych” [57] for calculating internal consistencies, effect sizes, and correlations, “car” [58] for estimating variance inflation factors, “rcompanion” [59] for MLE estimation of goodness-of-fit indices for logistic regression models, and “ggplot2” [60] for data visualization. The subsamples used for the analyses described in the following were obtained by dividing the participants into two groups consisting of men who (previously or currently) tested positive for COVID-19 and men who never tested positive for COVID-19. While for the initial analyses, a significance level of α = 0.05 was used, a sequential Holm–Bonferroni correction for multiple testing was applied post-hoc to control the family-wise error rate in each separate step of the analysis [61]. Additionally, due to the a priori formulated hypothesis, one-sided hypothesis tests were used in all the analyses involving traditional masculinity constructs (BSRI, MRNI–SF, GRCS–SF, including all their subscales). Lastly, statistical assumptions were assessed using the Levene’s test for the homogeneity of variance [62] for the t-tests and the generalized variance inflation factor [63] as well as Cook’s distance [64] for the logistic regression models. Because the distribution of the MDRS-22 and MRNI–SF scores was quite skewed in some intervals (Figure S1), the p-values of the t-tests from these two variables, including the subscales of the MRNI–SF, were obtained by bootstrapping with 5000 repetitions.
Initially, sample characteristics were obtained by calculating mean scores and frequency distributions for the total sample as well as for the two subgroups separately. Additional t- and χ2-tests were then used to test for statistically significant group differences between the two subgroups. Secondly, correlational analyses were conducted by calculating the Pearson’s correlation coefficient for the relevant variables and subsequently testing their significance with two-sided t-tests. Thirdly, logistic regression analyses were performed to assess the predictive value of depressive symptoms, traditional masculinity, fear of COVID-19, and COVID-19 pandemic-related stress. These regression models included the participant’s age, income, education level, and sexual orientation as covariates. In a last part, a more explorative approach was used to assess a possible association between traditional masculinity and COVID-19 symptoms. For this purpose, men who have contracted COVID-19 were divided into two subgroups using median division of the variables assessing the traditional masculinity constructs so that men with high and low traditional masculinity could be identified and compared with regard to displaying specific COVID-19 symptoms. Wald-tests were then used to determine significant differences in the frequency of COVID-19 symptoms and t-tests were used to determine significant differences in the mean scores of the remaining questionnaires used in the study.

3. Results

3.1. Descriptive Statistics and Group Differences in Men with and without COVID-19 Infection

Out of the 490 men participating in the survey, 34 (6.9%) previously or currently tested positive for COVID-19, which corresponded to the COVID-19 positive rates at that time for the surveyed countries Germany (4.1%), Austria (6.9%), and Switzerland (1.5%). Their age ranged from 18 to 68 years old, with a mean age of 25.7 years. The majority of the participants originated from Germany (73.1%) and Switzerland (14.5%), self-identified as heterosexual (73.7%), were single (63.5%), and had completed secondary education (71.0%). More detailed sample demographics can be found in Table 1.
Regarding group differences (Figure 2, Table S1), men who had contracted COVID-19 had significantly higher MRNI–SF scores on the total scale (Cohen’s d = 0.44) as well as on the MRNI–SF subscales assessing Restrictive Emotionality (RE; d = 0.38), Importance of Sex (IS; d = 0.31), Dominance (D; d = 0.55), and Toughness (T; d = 0.30). Similarly, men who had contracted COVID-19 also had significantly higher GRCS scores on the total scale (d = 0.40) as well as on the subscale measuring Success, Power, Competition (SPC; d = 0.38). However, none of these effects remained significant after applying the Holm–Bonferroni correction for multiple testing to all inferences made in this step.

3.2. Correlational Analysis

As presented in Table 2, the PHQ-9 was significantly positively correlated with the MDRS-22 (r = 0.64), the GRCS (r = 0.41), the FCV-19S (r = 0.29), and the CPSS (r = 0.31), while also being significantly negatively correlated with the BSRI (r = −0.36). The MDRS-22 was significantly positively correlated with the MRNI–SF (r = 0.18), the GRCS (r = 0.38), the FCV-19 (r = 0.28), and the CPSS (r = 0.28). The BSRI was only significantly positively correlated with the MRNI–SF (r = 0.33) while also being significantly negatively correlated with the FCV-19S (r = −0.16). The MRNI–SF was again significantly positively correlated with the GRCS (r = 0.39) and significantly negatively correlated with the FCV-19S (r = −0.15). The GRCS was significantly positively correlated with the FCV-19S (r = 0.18) and the CPSS (r = 0.24). Lastly, the FCV-19S was significantly positively correlated with the CPSS (r = 0.58). All effects in this part of the analysis remained significant after applying the Holm–Bonferroni correction. A more detailed overview, including the various sub-scales, is provided in the supplementary materials (Table S2).

3.3. Logistic Regression Predicting COVID-19 Infection

For the binary logistic regression analysis, two models were fitted due to singularities in the covariance matrices caused by perfect linear combination among the MRNI–SF total score and its subscales, as well as the GRCS–SF and its subscales. The first model therefore included the total scores of the MRNI–SF and the GRCS–SF as predictors for a COVID-19 infection (Figure S2, Table S3) and revealed the PHQ-9 (OR = 0.91 (0.83–0.99), standardized OR = 0.53 (0.30–0.93)) and the MDRS-22 (OR = 1.03 (1.00–1.05), ORstd. = 1.66 (1.04–2.65)) to be the only two significant predictors for a COVID-19 infection. The second model (Figure 3) included only the subscales of the MRNI–SF and the GRCS–SF instead of the total scores, but the same two predictors still emerged as significant (PHQ-9: OR = 0.89 (0.81–0.98), ORstd. = 0.47 (0.25–0.85); MDRS-22: OR = 1.02 (1.00–1.05), ORstd. = 1.64 (1.01–2.66)). Both models, therefore, indicate higher PHQ-9 scores to be associated with a lower likelihood of contracting COVID-19 and higher MDRS-22 scores to be associated with a higher likelihood of contracting COVID-19. However, these results also became non-significant after applying the Holm-Bonferroni correction for multiple testing to all inferences made in this step simultaneously. Nonetheless, further post-hoc analyses favored the first model with the total scores of the MRNI–SF and the GRCS–SF (BIC = 297.7, χ2(11) = 23.7, p = 0.015) as a better overall fit in predicting a COVID-19 infection as compared to the second model including only the subscales of the MRNI–SF and the GRCS–SF (BIC = 349.3, χ2(20) = 27.8, p = 0.114).

3.4. Group Differences in Men with Low and High Traditional Masculinity

An explorative analysis was taken to further examine possible associations between high traditional masculinity, COVID-19 symptoms, and depressive symptomatology in men who contracted COVID-19. Here, only the subsample consisting of 34 men was used. As can be seen in Figure 4A, men with high traditional masculinity operationalized as high BSRI scores exhibited less general (i.e., pain symptoms, headache, joint pain, muscle pain, peripheral neuropathy) pain symptomatology (47.4%) as compared to men with low traditional masculinity (80.0%) (χ2(1) = 4.22, p = 0.040, ORstd. = 0.17 (0.03–0.84)) and lower PHQ-9 scores (M = 9.2, SD = 5.7) as compared to men with low traditional masculinity (M = 13.9, SD = 6.3) (t(32) = −2.26, p = 0.031, d = 0.37). Furthermore, as can be seen in Figure 4C, men who experienced higher gender role conflict exhibited higher MDRS-22 scores (M = 44.5, SD = 21.5) as compared to men who experienced lower gender role conflict (M = 23.8, SD = 23.1) (t(32) = 2.70, p = 0.011, d = 0.43). However, these results also faded after applying the correction for multiple testing to all inferences made in this step simultaneously.

4. Discussion

4.1. Summary of Results

The present study examined the relationships between mental health outcomes, traditional masculinity, and COVID-19 infection. The first set of hypotheses postulated that higher expression of traditional masculinity constructs (gender role orientation, traditional masculine ideologies, gender role conflict) would be associated with a higher likelihood of having contracted COVID-19. Although group comparisons between men infected with COVID-19 and those without infection showed that infected men exhibit significantly elevated scores of traditional masculine ideologies and gender role conflict (but not gender role orientation), this could not be confirmed in the logistic regression models controlling for the confounders of age, income, education, and sexual orientation. Notably, when applying correction for multiple testing, the significant group differences did not hold.
In a second step, the hypotheses that men with elevated prototypical depression symptoms would have a lower risk of being infected with COVID-19, whereas men with elevated male-typical externalized depression symptoms would have an increased likelihood of being infected with COVID-19 were tested. Although this tendency was observable in the group comparisons, no significant group differences emerged. However, these hypotheses were subsequently confirmed in the logistic regression analyses controlling for confounders. It is important to note that when the Holm–Bonferroni correction for multiple testing was applied, the significant effects did not hold.

4.2. Integration of Findings

The results partly support our assumption that higher expression of traditional masculinity (MRNI–SF and GRCS–SF but not BSRI-M) is associated with a higher likelihood of having contracted COVID-19. Two out of the three masculinity measures were significantly elevated in the group of men reporting having contracted COVID-19 since the beginning of the pandemic as compared to men without known COVID-19 infection. Specifically, for the endorsement of traditional masculinity ideologies (MRNI–SF), the subscales of restrictive emotionality, importance of sex, dominance, and toughness were elevated in the men with a positive COVID-19 test, while for gender role conflict (GRCS–SF), the subscale of power, competition, and success was elevated. Though, only reaching statistical significance in the univariate analysis, an increase of the MRNI–SF total score by one standard deviation suggests an increase in the likelihood for a COVID-19 infection by 1.12 (OR). As shown in Table S4, the MRNI–SF subscale Dominance (OR = 1.46) shows one of the strongest associations with COVID-19 infection risk of all traditional masculinity scores. A man having a higher score in the dimension of Dominance by one standard deviation would, therefore, be 1.46-times more likely to have contracted COVID-19 than a man with a lower score. These findings are consistent with previous studies showing that adherence to traditional masculine norms is associated with engagement in risky health behaviors in men [30,65]. The present study thereby partly corroborates the previous literature on traditional masculinity and health behaviors in general, but also with regard to mask wearing during the current pandemic [15], extending it with regard to the most critical outcome to fight the current Corona pandemic, namely contracting COVID-19.
Importantly, a non-peer reviewed report examining over 6000 adults suggested that men self-identifying as “completely masculine” were almost three times more likely to contract COVID-19 than men self-identifying as “mostly masculine” or “slightly masculine” [18]. Data stemming from the present study could not confirm such a direct association for any of the three validated masculinity measures, highlighting three potential explanations for the observed lack of direct association. Firstly, it could be that included confounders (age, education, income, sexual orientation) better explain COVID-19 infection risk than traditional masculinity. Secondly, the study might have been unable to detect a signal due to the number of positive COVID-19 infection cases. Thirdly, it is imaginable that using validated masculinity scales reveals a more complex picture than reported by Cassino [18] using a single item to measure masculinity. This is in line with a previous report suggesting that total scores obscure the complex relationship between several traditional masculine norms and health behaviors or outcomes [24]. This study further highlighted, that in using the Conformity to Masculine Norms Inventory, with its subscales as a measure for conformity to traditional masculinity, 30% of the findings reflected beneficial associations with specific masculine role norms and health or well-being outcomes, especially for health promotion. Therefore, the observed findings of uniformly increased traditional masculinity in the group having contracted COVID-19 (see Figure 2) are pointing towards a relationship between traditional masculinity ideology and COVID-19 infection risk, although the Holm–Bonferroni correction renders the effects null. A further study from our workgroup is currently underway with the goal of replicating the observed findings with the advantage of accessing a larger population of contracted individuals with less time restrictions for recruitment due to the largely stabilized pandemic situation.
It is interesting that when predictors of traditional masculinity (MRNI–SF, GRCS–SF) are included separately in a univariate model along with covariates, they result as significant predictors of the risk of contracting COVID-19 as shown in Table S5, whereas when they are combined with other relevant predictors such as fear of COVID-19 or depressive symptomatology in multivariate analyses, they no longer become significant (see Tables S3 and S4). On the one hand, it seems that traditional masculinity measures explain the risk of contracting COVID-19 through the dimensions of “dominance”, “restrictive emotionality”, or “toughness”, which stand opposed to fear of COVID-19. In line with this, men with higher traditional masculinity also report less fear of COVID-19 and its consequences [8,9]. However, since the variance of contracting COVID-19 is co-explained in multivariate analyses by the Fear of COVID Scale, this arguably leads to these constructs subtracting mutually explained variance from each other resulting in null associations. Similarly, depression symptoms may co-explain the risk of contracting COVID-19 in multivariate analyses. Since, in this study, the measure of endorsement of traditional masculinity ideologies (MRNI–SF) is negatively associated with prototypical depression symptomatology (PHQ-9) on the one hand, but positively associated with male-typical externalized depression symptomatology (MDRS-22) on the other (see Table 2 and Table S2 for subscale relations), a partitioning of the variance explanation between the endorsement of traditional masculinity and depression symptoms is likely to occur as well. Here, it appears that both forms of depression symptomatology combined with the constructs of traditional masculinity and COVID-19-related anxiety and burden are the strongest predictors of the risk of becoming infected with COVID-19 in the multivariate analyses. Nevertheless, this explanation must be considered tentative and tested in larger samples with more statistical power. In such larger samples, there may be sufficient statistical power to identify traditional masculinity measures as robust significant predictors.
The fact that men with high traditional masculinity exhibit worse health behaviors including more risk-taking and less mask-wearing [15] and, thus, are exposed to a higher risk of infection, also further explains why men are more likely to die from COVID-19 in comparison to women [3,4]. The present study sheds light on why men, and particularly men with high traditional masculinity, may be at increased risk for contracting COVID-19 and, thus, potentially increased mortality as shown in previous longitudinal studies [19,23].
Furthermore, results supported the exploratory hypotheses that higher prototypical depression symptoms (PHQ-9) are associated with a reduced risk of contracting COVID-19, while higher male-typical externalized depression symptoms (MDRS-22) are associated with an increased risk of contracting COVID-19. Thus, an increase in the PHQ-9 total score by one standard deviation lowers the likelihood for a COVID-19 infection by 0.53 (OR), while an increase in the MDRS-22 total score by one standard deviation increases the likelihood for a COVID-19 infection by 1.66 (OR). This can be interpreted as a man with a total PHQ-9 score 6.6 points higher than another man is only nearly half as likely to contract COVID-19, while a man with a total MDRS-22 score 20.6 points higher than another man is 1.66-times more likely to contract COVID-19. This finding is new insofar that no previous study has examined prototypical and male-typical externalized depression symptoms in parallel with regard to the risk of contracting COVID-19. The observation that prototypical depression symptoms, likely due to their relationship with a general social withdrawal, are associated with a reduced infection risk, has previously been reported by independent studies [39]. However, the contrary or a null association has also been reported several times [33,34,35,36], suggesting that this relationship is either non-existent or depends on examined subgroups and the specific depression symptomatology exhibited. A previous study from our workgroup examining adult men’s psychotherapy use found that only higher male-typical externalized depression symptoms, but not prototypical depression symptoms, predicted lower psychotherapy use [21]. Similarly, male-typical externalizing and prototypical depression symptoms were shown to differ in predicting a past-month suicide attempt [22,66]. These findings support the notion that internalizing and externalizing psychopathology are differentially related to health behavior risk and specifically to the risk of contracting COVID-19. However, it is also relevant that COVID-19 increased the risk of suffering from depression measured with the PHQ-9 more than 3-fold [67], which additionally highlights the importance to understand which individuals suffer from which depression symptoms and whether they are at particular risk of contracting COVID-19.
In summary, the present data support the hypothesis that traditional masculinity and the presence of male-typical externalized depression symptoms may be associated with an increased risk of contracting COVID-19. Therefore, health promotion campaigns should specifically target these men, as they might be relatively poorly informed due to a more indifferent attitude based in traditional masculinity ideologies of dominance, toughness, and self-reliance, or in extreme cases these men may also deliberately infect themselves with COVID-19 to self-harm or suicide [38]. One way to address this problem would be to appeal to the roles of protector and provider and ask men with high traditional masculinity to protect others and not put them in danger, which also means protecting themselves so as not to be carriers of the virus.
Mental health care specialists should pay particular attention to men with high traditional masculinity and male-typical externalizing depression symptoms and inform them about the detrimental personal and societal consequences a careless spreading of the virus has. The effect men with high traditional masculinity and male-typical externalizing depression symptoms have in spreading the virus, and thereby prolonging the current pandemic situation, is difficult to identify. Nevertheless, informing men and the entire health care system about the crucial role men with high traditional masculinity may play in this pandemic is of vital importance. However, the use of a strength-based approach by appealing to the roles of protector and provider or good leader may be all the more promising in limiting the risk of COVID-19 infection by men with high traditional masculinity as a means of protecting significant others.
It is important to note, however, that the present findings can only be considered preliminary results since limited power caused significant results to fade after correction for multiple testing. Due to a time-sensitive measurement period during the COVID-19 pandemic, it was not possible to recruit participants for a sufficiently long period and, therefore, only a small sample of 490 men in total, and 34 currently or previously COVID-19-infected men, could be obtained. Therefore, the present findings based on the a priori formulated hypotheses are to be considered tentative. The observation that findings uniformly fade when applying correction for multiple testing is, however, also to be interpreted with caution since no prior results on this specific research question is available and potentially a substantially larger sample size would be required to uphold the small to moderate findings in the range of statistical significance due to the small number of positive COVID-19 cases. Due to the time restrictions set by a dynamic pandemic situation, this poses a major challenge for research and, thus, corrections for multiple testing must be placed in perspective.
Of further interest are the observations regarding traditional masculinity and specific COVID-19-related symptoms (see Figure 4). A detailed discussion of these findings is provided in the Supplementary Materials.

4.3. Limitations

When interpreting the study results, some limitations should be taken into account. First, the cross-sectional design only provides information about associations and no causal conclusions can be drawn from the results. Furthermore, the sample with a total of 490 men and 34 men with a reported COVID-19 infection is not sufficiently powered to identify small effects. Therefore, all significant results faded after the Holm–Bonferroni correction. Furthermore, the main outcome measure of a positive corona infection by a validated test has been self-reported, and because of insufficient data, little can be said about its reliability despite its high face validity. It also becomes evident from the descriptive data showing a high percentage of men suffering from a psychological disorder or suicidal thoughts, that this is a sample of adult men with high psychological distress, which makes it difficult to generalize to the male population as a whole. Further, it should be noted that the PHQ-9 and MDRS-22 depression scales, while very well-validated measures of depression, are not capable of identifying depressive disorders and, thus, interpretations regarding depression status and risk for contracting COVID-19 in men need to be made on a syndrome level, while a clinical diagnostic interview enabling differential diagnosis would be preferable. Another limitation of the study may be the analysis of the relationships with logistic regressions as it was pre-specified in the preregistration process. In future investigations, structural equation modeling approaches could be used to better represent the complex network of constructs involved. Thus, it could be directly examined whether associations are mediated by third variables. Furthermore, an investigation in a longitudinal design could examine the covariation of depression symptoms and traditional masculinity with respect to their influence on relevant health outcomes, such as COVID-19 infection. Lastly, the fact that there is partial overlap between the subscales of the instruments complicates the analytic strategy and interpretation of the results. However, the aim of the study was to compare the different masculinity instruments, which are based on three different conceptual derivations, with respect to the risk of contracting COVID-19. Thus, the BSRI-M is derived from the Gender Role Identity Paradigm and describes self-attributed masculine traits, whereas the MRNI–SF is derived from the Gender Role Strain Paradigm and describes the endorsement of traditional masculinity ideologies. The GRCS–SF, on the other hand, measures the rigid adherence to or deviation from traditional male role norms and its negative psychological consequences for the man or his environment. Therefore, these constructs are each distinct from one another and can independently elucidate variance for the pre-specified research questions. Due to the fact that there was no previous research in this area, it emerged as important to test different relevant instruments associated with traditional masculinity with regard to the risk of contracting COVID-19. This was in the pursuit of a comprehensive investigation without previous research findings on the specific topic, while at the same time leading to a co-explanation of the primary outcome by these constructs. Therefore, we provide the interested reader with multivariate and univariate analyses at the total score and subscale levels in Tables S3, S4 and S6, as well as the variance inflation factors testing for multi-collinearity showing no multi-collinearity for the used constructs in Table S6.

5. Conclusions

Taken together, men exhibiting higher traditional masculinity and higher male-typical externalized depression symptoms may be at increased risk for contracting COVID-19 and should be informed by health care campaigns and mental health care specialists. Appealing to these men’s provider and protector roles might partially counter the overall detrimental effects of high traditional masculinity without challenging the endorsement of traditional masculinity ideology itself. The potential effect men with high traditional masculinity and increased male-typical externalized depression symptoms have in prolonging the current pandemic is difficult to identify. More research using validated traditional masculinity measures, combined with larger samples including more men with current or past COVID-19 infections, is needed.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/bs12030080/s1, Figure S1: Empirical and Theoretical Normal Cumulative Distribution Functions (CDF) of the Study Variables, Figure S2: Standardized Odds Ratios for COVID-19 Infection and their two- and one-sided 95% Confidence Intervals, Table S1: Results of Mean Score Comparisons between Men with and without COVID-19 Infection, Table S2: Correlation Matrix for Study Variables Including Subscales, Table S3: Results of Binary Logistic Regression Analyses predicting COVID-19 Infection using Total Scores, Table S4: Results of Binary Logistic Regression Analyses predicting COVID-19 Infection using Subscales, Table S5: Results of Univariate, Binary Logistic Regression Analyses predicting COVID-19 Infection, Table S6: Variance Inflation Factors (VIF) for the Logistic Regression Models, S-Discussion: Discussion part on traditional masculinity and specific COVID-19-related symptoms.

Author Contributions

Design and conceptualization of the study, A.W., J.G. and U.E.; Participant recruitment and testing, A.W. and J.G.; Data analysis and visualization, L.E. and A.W.; Data interpretation, A.W. and L.E.; Writing of the first draft, A.W. and L.E.; Editing of subsequent versions, A.W., U.E., Z.E.S., S.M.R., D.K., J.S.O. and J.L.O. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki and was approved by the Ethics Committee of the Faculty of Arts and Social Sciences of the University of Zurich (Nr. 21.2.4).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

A priori defined study hypotheses, statistical analyses, and the study specific data set can be retrieved from OSF (https://osf.io/q4pw3, accessed on 9 March 2022).

Acknowledgments

We thank all the participants for completing the survey and thereby supporting men’s mental health research.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Li, M.; Zhang, Z.; Cao, W.; Liu, Y.; Du, B.; Chen, C.; Liu, Q.; Uddin, M.N.; Jiang, S.; Chen, C.; et al. Identifying Novel Factors Associated with COVID-19 Transmission and Fatality Using the Machine Learning Approach. Sci. Total Environ. 2021, 764, 142810. [Google Scholar] [CrossRef] [PubMed]
  2. Bridges, T.; Barber, K.; Nelson, J.D.; Chatillon, A. Masculinity and COVID-19: Symposium Introduction. Men Masc. 2021, 24, 163–167. [Google Scholar] [CrossRef]
  3. Peckham, H.; de Gruijter, N.M.; Raine, C.; Radziszewska, A.; Ciurtin, C.; Wedderburn, L.R.; Rosser, E.C.; Webb, K.; Deakin, C.T. Male Sex Identified by Global COVID-19 Meta-Analysis as a Risk Factor for Death and ITU Admission. Nat. Commun. 2020, 11, 6317. [Google Scholar] [CrossRef] [PubMed]
  4. Hamer, M.; Kivimäki, M.; Gale, C.R.; Batty, G.D. Lifestyle Risk Factors, Inflammatory Mechanisms, and COVID-19 Hospitalization: A Community-Based Cohort Study of 387,109 Adults in UK. Brain Behav. Immun. 2020, 87, 184–187. [Google Scholar] [CrossRef] [PubMed]
  5. Courtenay, W.H. Engendering Health: A Social Constructionist Examination of Men’s Health Beliefs and Behaviors. Psychol. Men Masc. 2000, 1, 4–15. [Google Scholar] [CrossRef]
  6. Harrison, J. Warning: The Male Sex Role May Be Dangerous to Your Health. J. Soc. Issues 1978, 34, 65–86. [Google Scholar] [CrossRef]
  7. Umamaheswar, J.; Tan, C. “Dad, Wash Your Hands”: Gender, Care Work, and Attitudes toward Risk during the COVID-19 Pandemic. Socius 2020, 6, 2378023120964376. [Google Scholar] [CrossRef]
  8. Babalola, S.; Krenn, S.; Rimal, R.; Serlemitsos, E.; Shaivitz, M.; Shattuck, D.; Storey, D. KAP COVID Dashboard. In USA: Johns Hopkins Center for Communication Programs, Massachusetts Institute of Technology, Global Outbreak Alert and Response Network, Facebook Data for Good; Published September 2020. Available online: https://coronavirus.jhu.edu/map.html (accessed on 10 March 2022).
  9. Capraro, V.; Barcelo, H. The Effect of Messaging and Gender on Intentions to Wear a Face Covering to Slow down COVID-19 Transmission. PsyArXiv 2020. preprint. [Google Scholar] [CrossRef]
  10. Clark, C.; Davila, A.; Regis, M.; Kraus, S. Predictors of COVID-19 Voluntary Compliance Behaviors: An International Investigation. Glob. Transit. 2020, 2, 76–82. [Google Scholar] [CrossRef] [PubMed]
  11. Haischer, M.H.; Beilfuss, R.; Hart, M.R.; Opielinski, L.; Wrucke, D.; Zirgaitis, G.; Uhrich, T.D.; Hunter, S.K. Who Is Wearing a Mask? Gender-, Age-, and Location-Related Differences during the COVID-19 Pandemic. PLoS ONE 2020, 15, e0240785. [Google Scholar] [CrossRef]
  12. Okten, I.O.; Gollwitzer, A.; Oettingen, G. Gender Differences in Preventing the Spread of Coronavirus. Behav. Sci. Policy 2020, 6, 109–122. [Google Scholar] [CrossRef]
  13. Palmer, C.L.; Peterson, R.D. Toxic Mask-Ulinity: The Link between Masculine Toughness and Affective Reactions to Mask Wearing in the COVID-19 Era. Polit. Gend. 2020, 16, 1044–1051. [Google Scholar] [CrossRef]
  14. Cassino, D.; Besen-Cassino, Y. Of Masks and Men? Gender, Sex, and Protective Measures during COVID-19. Polit. Gend. 2020, 16, 1052–1062. [Google Scholar] [CrossRef]
  15. Mahalik, J.R.; Di Bianca, M.; Harris, M.P. Men’s Attitudes toward Mask-Wearing during COVID-19: Understanding the Complexities of Mask-Ulinity. J. Health Psychol. 2021. [Google Scholar] [CrossRef] [PubMed]
  16. Levant, R.F.; Richmond, K. The Gender Role Strain Paradigm and Masculinity Ideologies. In APA Handbook of Men and Masculinities; American Psychological Association: Washington, DC, USA, 2015; pp. 23–49. [Google Scholar] [CrossRef]
  17. O’Neil, J.M. Gender Role Conflict Research 30 Years Later: An Evidence-Based Diagnostic Schema to Assess Boys and Men in Counseling. J. Couns. Dev. 2013, 91, 490–498. [Google Scholar] [CrossRef]
  18. Cassino, D. Macho, Macho Man: Masculinity is a Major Risk Factor for COVID-19. Available online: https://www.fdu.edu/news/fdu-poll-finds-masculinity-is-a-major-risk-factor-for-covid-19/ (accessed on 10 March 2022).
  19. Lippa, R.A.; Martin, L.R.; Friedman, H.S. Gender-Related Individual Differences and Mortality in the Terman Longitudinal Study: Is Masculinity Hazardous to Your Health? Personal. Soc. Psychol. Bull. 2000, 26, 1560–1570. [Google Scholar] [CrossRef]
  20. Wong, Y.J.; Ho, M.H.R.; Wang, S.Y.; Miller, I.S.K. Meta-Analyses of the Relationship between Conformity to Masculine Norms and Mental Health-Related Outcomes. J. Couns. Psychol. 2017, 64, 80–93. [Google Scholar] [CrossRef] [PubMed]
  21. Eggenberger, L.; Fordschmid, C.; Ludwig, C.; Weber, S.; Grub, J.; Komlenac, N.; Walther, A. Men’s Psychotherapy Use, Male Role Norms, and Male-Typical Depression Symptoms: Examining 716 Men and Women Experiencing Psychological Distress. Behav. Sci. 2021, 11, 83. [Google Scholar] [CrossRef] [PubMed]
  22. Walther, A.; Grub, J.; Tsar, S.; Ehlert, U.; Heald, A.; Perrin, R.; Ogrodniczuk, J.S.; Seidler, Z.E.; Rice, S.M.; Kealy, D.; et al. Status Loss Due to COVID-19, Traditional Masculinity, and the Prediction of Suicidal Ideation and Recent Suicide Attempts. PsyArXiv 2021. preprint. [Google Scholar] [CrossRef]
  23. Coleman, D.; Feigelman, W.; Rosen, Z. Association of High Traditional Masculinity and Risk of Suicide Death: Secondary Analysis of the Add Health Study. JAMA Psychiatry 2020, 77, 435–437. [Google Scholar] [CrossRef] [PubMed]
  24. Gerdes, Z.T.; Levant, R.F. Complex Relationships among Masculine Norms and Health/Well-Being Outcomes: Correlation Patterns of the Conformity to Masculine Norms Inventory Subscales. Am. J. Mens. Health 2018, 12, 229–240. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  25. Bem, S.L. The Measurement of Psychological Androgyny. J. Consult. Clin. Psychol. 1974, 42, 155–162. [Google Scholar] [CrossRef] [PubMed]
  26. Pleck, J.H. The Myth of Masculinity; MIT Press (MA): Combridge, MA, USA, 1981. [Google Scholar]
  27. Levant, R.F.; Hirsch, L.S.; Celentano, E.; Cozza, T.M. The Male Role: An Investigation of Contemporary Norms. J. Ment. Health Couns. 1992, 14, 325–337. [Google Scholar]
  28. Mahalik, J.R.; Locke, B.D.; Ludlow, L.H.; Diemer, M.A.; Scott, R.P.J.; Gottfried, M.; Freitas, G. Development of the Conformity to Masculine Norms Inventory. Psychol. Men Masc. 2003, 4, 3–25. [Google Scholar] [CrossRef]
  29. O’Neil, J.M.; Helms, B.J.; Gable, R.K.; David, L.; Wrightsman, L.S. Gender-Role Conflict Scale: College Men’s Fear of Femininity. Sex Roles 1986, 14, 335–350. [Google Scholar] [CrossRef]
  30. Levant, R.F.; Wimer, D.J. Masculinity Constructs as Protective Buffers and Risk Factors for Men’s Health. Am. J. Mens. Health 2014, 8, 110–120. [Google Scholar] [CrossRef] [PubMed]
  31. Shinn, A.K.; Viron, M. Perspectives on the COVID-19 Pandemic and Individuals with Serious Mental Illness. J. Clin. Psychiatry 2020, 81, 14205. [Google Scholar] [CrossRef] [PubMed]
  32. Yao, H.; Chen, J.H.; Xu, Y.F. Patients with Mental Health Disorders in the COVID-19 Epidemic. Lancet Psychiatry 2020, 7, e21. [Google Scholar] [CrossRef]
  33. Chang, T.S.; Ding, Y.; Freund, M.K.; Johnson, R.; Schwarz, T.; Yabu, J.M.; Hazlett, C.; Chiang, J.N.; Wulf, A.; Geschwind, D.H. Prior Diagnoses and Medications as Risk Factors for COVID-19 in a Los Angeles Health System. medRxiv 2020. preprint. [Google Scholar] [CrossRef]
  34. Tang, O.; Bigelow, B.F.; Sheikh, F.; Peters, M.; Zenilman, J.M.; Bennett, R.; Katz, M.J. Outcomes of Nursing Home COVID-19 Patients by Initial Symptoms and Comorbidity: Results of Universal Testing of 1970 Residents. J. Am. Med. Dir. Assoc. 2020, 21, 1767–1773. [Google Scholar] [CrossRef]
  35. Taquet, M.; Luciano, S.; Geddes, J.R.; Harrison, P.J. Bidirectional Associations between COVID-19 and Psychiatric Disorder: Retrospective Cohort Studies of 62 354 COVID-19 Cases in the USA. Lancet Psychiatry 2021, 8, 130–140. [Google Scholar] [CrossRef]
  36. Wang, Q.; Xu, R.; Volkow, N.D. Increased Risk of COVID-19 Infection and Mortality in People with Mental Disorders: Analysis from Electronic Health Records in the United States. World Psychiatry 2021, 20, 124–130. [Google Scholar] [CrossRef]
  37. Izurieta, H.S.; Graham, D.J.; Jiao, Y.; Hu, M.; Lu, Y.; Wu, Y.; Chillarige, Y.; Wernecke, M.; Menis, M.; Pratt, D.; et al. Natural History of Coronavirus Disease 2019: Risk Factors for Hospitalizations and Deaths among >26 Million US Medicare Beneficiaries. J. Infect. Dis. 2021, 223, 945–956. [Google Scholar] [CrossRef] [PubMed]
  38. Ammerman, B.A.; Burke, T.A.; Jacobucci, R.; McClure, K. Preliminary Investigation of the Association between COVID-19 and Suicidal Thoughts and Behaviors in the U.S. J. Psychiatr. Res. 2021, 134, 32–38. [Google Scholar] [CrossRef] [PubMed]
  39. Ceban, F.; Nogo, D.; Carvalho, I.P.; Lee, Y.; Nasri, F.; Xiong, J.; Lui, L.M.W.; Subramaniapillai, M.; Gill, H.; Liu, R.N. Association Between Mood Disorders and Risk of COVID-19 Infection, Hospitalization, and Death: A Systematic Review and Meta-Analysis. JAMA Psychiatry 2021, 78, 1079–1091. [Google Scholar] [CrossRef] [PubMed]
  40. Rice, S.M.; Fallon, B.J.; Aucote, H.M.; Möller-Leimkühler, A.M. Development and Preliminary Validation of the Male Depression Risk Scale: Furthering the Assessment of Depression in Men. J. Affect. Disord. 2013, 151, 950–958. [Google Scholar] [CrossRef] [PubMed]
  41. Cavanagh, A.; Wilson, C.J.; Kavanagh, D.J.; Caputi, P. Differences in the Expression of Symptoms in Men Versus Women with Depression: A Systematic Review and Meta-Analysis. Harv. Rev. Psychiatry 2017, 25, 29–38. [Google Scholar] [CrossRef] [PubMed]
  42. Martin, L.A.; Neighbors, H.W.; Griffith, D.M. The Experience of Symptoms of Depression in Men vs Women: Analysis of the National Comorbidity Survey Replication. JAMA Psychiatry 2013, 70, 1100–1106. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  43. Walther, A.; Grub, J.; Ehlert, U.; Wehrli, S.; Rice, S.M.; Seidler, Z.E.; Debelak, R. Male Depression Risk, Psychological Distress, and Psychotherapy Uptake: Validation of the German Version of the Male Depression Risk Scale. J. Affect. Disord. Rep. 2021, 4, 100107. [Google Scholar] [CrossRef]
  44. Winkler, D.; Pjrek, E.; Kasper, S. Anger Attacks in Depression–Evidence for a Male Depressive Syndrome. Psychother. Psychosom. 2005, 74, 303–307. [Google Scholar] [CrossRef]
  45. Walther, A.; Seidler, Z.E. Männliche Formen der Depression und deren Behandlung. PiD-Psychother. Im Dialog 2020, 21, 40–45. [Google Scholar] [CrossRef]
  46. Sudre, C.H.; Murray, B.; Varsavsky, T.; Graham, M.S.; Penfold, R.S.; Bowyer, R.C.; Pujol, J.C.; Klaser, K.; Antonelli, M.; Canas, L.S.; et al. Attributes and Predictors of Long COVID. Nat. Med. 2021, 27, 626–631. [Google Scholar] [CrossRef] [PubMed]
  47. Levant, R.F.; Hall, R.J.; Rankin, T.J. Male Role Norms Inventory-Short Form (Mrni-Sf): Development, Confirmatory Factor Analytic Investigation of Structure, and Measurement Invariance across Gender. J. Couns. Psychol. 2013, 60, 228–238. [Google Scholar] [CrossRef] [PubMed]
  48. Wester, S.R.; Vogel, D.L.; O’Neil, J.M.; Danforth, L. Development and Evaluation of the Gender Role Conflict Scale Short Form (GRCS-SF). Psychol. Men Masc. 2012, 13, 199–210. [Google Scholar] [CrossRef]
  49. Komlenac, N.; Siller, H.; Bliem, H.R.; Hochleitner, M. Validation of the Internal Structure of a German-Language Version of the Gender Role Conflict Scale—Short Form. Front. Psychol. 2018, 9, 1161. [Google Scholar] [CrossRef]
  50. Kroenke, K.; Spitzer, R.L.; Williams, J.B.W. The Phq-9. J. Gen. Intern. Med. 2001, 16, 606–613. [Google Scholar] [CrossRef] [PubMed]
  51. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (DSM-5®), 5th ed.; American Psychiatric Pub: Arlington, VA, USA, 2013. [Google Scholar]
  52. Levis, B.; Sun, Y.; He, C.; Wu, Y.; Krishnan, A.; Bhandari, P.M.; Neupane, D.; Imran, M.; Brehaut, E.; Negeri, Z.; et al. Accuracy of the PHQ-2 Alone and in Combination with the PHQ-9 for Screening to Detect Major Depression: Systematic Review and Meta-Analysis. JAMA J. Am. Med. Assoc. 2020, 323, 2290–2300. [Google Scholar] [CrossRef]
  53. Martin, A.; Rief, W.; Klaiberg, A.; Braehler, E. Validity of the Brief Patient Health Questionnaire Mood Scale (PHQ-9) in the General Population. Gen. Hosp. Psychiatry 2006, 28, 71–77. [Google Scholar] [CrossRef]
  54. Ahorsu, D.K.; Lin, C.-Y.; Imani, V.; Saffari, M.; Griffiths, M.D.; Pakpour, A.H. The Fear of COVID-19 Scale: Development and Initial Validation. Int. J. Ment. Health Addict. 2020. [Google Scholar] [CrossRef] [Green Version]
  55. Werner, A.; Kater, M.-J.; Schlarb, A.; Lohaus, A. COVID-19-Pandemie-Stress-Skala (CPSS). PsyArXiv 2021. preprints. [Google Scholar] [CrossRef]
  56. R Core Team. R: A Language and Environment for Statistical Computing; R Foundation for Statistical Computing: Vienna, Austria, 2020. [Google Scholar]
  57. Revelle, W. Psych: Procedures for Personality and Psychological Research; Northwestern University: Evanston, IL, USA, 2020. [Google Scholar]
  58. Fox, J.; Sanford, W. An {R} Companion to Applied Regression, 3rd ed.; Sage: Thousand Oaks, CA, USA, 2019. [Google Scholar]
  59. Mangiafico, S. Rcompanion: Functions to Support Extension Education Program Evaluation [R Package Rcompanion Version 2.4. 0]; 2021. Available online: https://cran.r-project.org/web/packages/rcompanion/index.html (accessed on 10 March 2022).
  60. Wickham, H. Ggplot2: Elegant Graphics for Data Analysis; Springer: New York, NY, USA, 2016. [Google Scholar]
  61. Holm, S. A Simple Sequentially Rejective Multiple Test Procedure. Scand. J. Stat. 1979, 6, 65–70. [Google Scholar]
  62. Levene, H. Robust Tests for Equality of Variances. In Contributions to Probability and Statistics; Essays in Honor of Harold Hotelling; Stanford University Press: Palo Alto, CA, USA, 1961; pp. 279–292. [Google Scholar]
  63. Fox, J.; Monette, G. Generalized Collinearity Diagnostics. J. Am. Stat. Assoc. 1992, 87, 178–183. [Google Scholar] [CrossRef]
  64. Cook, R.D. Detection of Influential Observation in Linear Regression. Technometrics 1977, 19, 15–18. [Google Scholar] [CrossRef]
  65. Iwamoto, D.K.; Cheng, A.; Lee, C.S.; Takamatsu, S.; Gordon, D. “Man-Ing” up and Getting Drunk: The Role of Masculine Norms, Alcohol Intoxication and Alcohol-Related Problems among College Men. Addict. Behav. 2011, 36, 906–911. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  66. Rice, S.M.; Ogrodniczuk, J.S.; Kealy, D.; Seidler, Z.E.; Dhillon, H.M.; Oliffe, J.L. Validity of the Male Depression Risk Scale in a Representative Canadian Sample: Sensitivity and Specificity in Identifying Men with Recent Suicide Attempt. J. Ment. Health 2019, 28, 132–140. [Google Scholar] [CrossRef] [PubMed]
  67. Ettman, C.K.; Abdalla, S.M.; Cohen, G.H.; Sampson, L.; Vivier, P.M.; Galea, S. Prevalence of Depression Symptoms in US Adults Before and During the COVID-19 Pandemic. JAMA Netw. Open 2020, 3, e2019686. [Google Scholar] [CrossRef] [PubMed]
Behavsci 12 00080 g001 550
Figure 1. Flow diagram of the inclusion and exclusion process. Note: N = number of participants.
Figure 1. Flow diagram of the inclusion and exclusion process. Note: N = number of participants.
Behavsci 12 00080 g001
Behavsci 12 00080 g002 550
Figure 2. Mean score comparisons between men with and without COVID-19 infection and their two- and one-sided 95% confidence intervals. Note: a PHQ-9 = Patient Health Questionnaire-9; b MDRS-22 = Male Depression Risk Scale-22; c BSRI (m) = Bem Sex-Role Inventory (masculinity subscale); d MRNI = Male Role Norms Inventory (subscales: NT = Negativity toward Sexual Minorities; RE = Restrictive Emotionality; SR = Self-reliance through Mechanical Skills; AF = Avoidance of Femininity; IS = Importance of Sex; D = Dominance; T = Toughness); e GRCS = Gender Role Conflict Scale (subscales: SPC = Success, Power, Competition; RE = Restrictive Emotionality; RABBM = Restrictive Affectionate Behavior Between Men; CBWFR = Conflicts Between Work and Leisure–Family Relations); f FCV-19S = Fear of COVID-19 Scale; g CPSS = COVID-19 Pandemic Stress Scale; * = p < 0.05.
Figure 2. Mean score comparisons between men with and without COVID-19 infection and their two- and one-sided 95% confidence intervals. Note: a PHQ-9 = Patient Health Questionnaire-9; b MDRS-22 = Male Depression Risk Scale-22; c BSRI (m) = Bem Sex-Role Inventory (masculinity subscale); d MRNI = Male Role Norms Inventory (subscales: NT = Negativity toward Sexual Minorities; RE = Restrictive Emotionality; SR = Self-reliance through Mechanical Skills; AF = Avoidance of Femininity; IS = Importance of Sex; D = Dominance; T = Toughness); e GRCS = Gender Role Conflict Scale (subscales: SPC = Success, Power, Competition; RE = Restrictive Emotionality; RABBM = Restrictive Affectionate Behavior Between Men; CBWFR = Conflicts Between Work and Leisure–Family Relations); f FCV-19S = Fear of COVID-19 Scale; g CPSS = COVID-19 Pandemic Stress Scale; * = p < 0.05.
Behavsci 12 00080 g002
Behavsci 12 00080 g003 550
Figure 3. Standardized odds ratios for COVID-19 infection and their two- and one-sided 95% confidence intervals. Note: OR = Odds ratio; a PHQ-9 = Patient health Questionnaire-9; b MDRS-22 = Male Depression Risk Scale-22; c BSRI (m) = Bem Sex-Role Inventory (masculinity subscale); d MRNI = Male Role Norms Inventory (subscales: NT = Negativity toward Sexual Minorities; RE = Restrictive Emotionality; SR = Self-reliance through Mechanical Skills; AF = Avoidance of Femininity; IS = Importance of Sex; D = Dominance; T = Toughness; e GRCS = Gender Role Conflict Scale (subscales: SPC = Success, Power, Competition; RE = Restrictive Emotionality; RABBM = Restrictive Affectionate Behavior Between Men; CBWFR = Conflicts Between Work and Leisure–Family Relations); f FCV-19S = Fear of COVID-19 Scale; g CPSS = COVID-19 Pandemic Stress Scale. * = p < 0.05.
Figure 3. Standardized odds ratios for COVID-19 infection and their two- and one-sided 95% confidence intervals. Note: OR = Odds ratio; a PHQ-9 = Patient health Questionnaire-9; b MDRS-22 = Male Depression Risk Scale-22; c BSRI (m) = Bem Sex-Role Inventory (masculinity subscale); d MRNI = Male Role Norms Inventory (subscales: NT = Negativity toward Sexual Minorities; RE = Restrictive Emotionality; SR = Self-reliance through Mechanical Skills; AF = Avoidance of Femininity; IS = Importance of Sex; D = Dominance; T = Toughness; e GRCS = Gender Role Conflict Scale (subscales: SPC = Success, Power, Competition; RE = Restrictive Emotionality; RABBM = Restrictive Affectionate Behavior Between Men; CBWFR = Conflicts Between Work and Leisure–Family Relations); f FCV-19S = Fear of COVID-19 Scale; g CPSS = COVID-19 Pandemic Stress Scale. * = p < 0.05.
Behavsci 12 00080 g003
Behavsci 12 00080 g004 550
Figure 4. (A–C) depict differences in COVID-19 and depression symptoms, fear of COVID-19, and COVID-19 pandemic-related stress between men with low and high traditional masculinity. (A) compares the frequency of reported symptoms by high versus low masculine role orientation. (B) compares the frequency of reported symptoms by high versus low endorsement of traditional masculinity ideologies. (C) compares the frequency of reported symptoms by high versus low gender role conflict. Note: S. = symptoms, mal. = malaise, low/high = below/above the median. a PHQ-9 = Patient health Questionnaire-9; b MDRS-22 = Male Depression Risk Scale-22; (masculinity subscale); d MRNI = Male Role Norms Inventory; e GRCS = Gender Role Conflict Scale; f FCV-19S = Fear of COVID-19 Scale; g CPSS = COVID-19 Pandemic Stress Scale. * = p < 0.05.
Figure 4. (A–C) depict differences in COVID-19 and depression symptoms, fear of COVID-19, and COVID-19 pandemic-related stress between men with low and high traditional masculinity. (A) compares the frequency of reported symptoms by high versus low masculine role orientation. (B) compares the frequency of reported symptoms by high versus low endorsement of traditional masculinity ideologies. (C) compares the frequency of reported symptoms by high versus low gender role conflict. Note: S. = symptoms, mal. = malaise, low/high = below/above the median. a PHQ-9 = Patient health Questionnaire-9; b MDRS-22 = Male Depression Risk Scale-22; (masculinity subscale); d MRNI = Male Role Norms Inventory; e GRCS = Gender Role Conflict Scale; f FCV-19S = Fear of COVID-19 Scale; g CPSS = COVID-19 Pandemic Stress Scale. * = p < 0.05.
Behavsci 12 00080 g004
Table
Table 1. Descriptive statistics for the sample.
Table 1. Descriptive statistics for the sample.
Total (N = 490)No CV19 (N = 456)CV19 (N = 34)
N (%)M (SD)N (%)M (SD)N (%)M (SD)Test-Statistic (df)p
Age 25.7 (9.8) 25.7 (9.8) 25.9 (9.5)−0.10 (448)0.924
Nationality 26.63 (6)<0.001 ***
 Swiss71 (14.5) 62 (13.6) 9 (26.5)
 German358 (73.1) 342 (75.0) 16 (47.1)
 Austrian43 (8.8) 36 (7.9) 7 (20.6)
 Luxembourger4 (0.8) 4 (0.9) 0 (0)
 Liechtensteiner1 (0.2) 1 (0.2) 0 (0)
 Belgian1 (0.2) 0 (0) 1 (2.9)
 Other12 (2.4) 11 (2.4) 1 (2.9)
Sexual orientation 0.88 (4)0.927
 Heterosexual-identified361 (73.7) 334 (73.2) 27 (79.4)
 Gay/Lesbian-identified39 (8.0) 37 (8.1) 2 (5.9)
 Bisexual-identified67 (13.7) 63 (13.8) 4 (11.8)
 Asexual-identified5 (1.0) 5 (1.1) 0 (0)
 Other18 (3.7) 17 (3.7) 1 (2.9)
Marital status 1.45 (2)0.484
 Single311 (63.5) 291 (63.8) 20 (58.8)
 In a relationship168 (34.3) 154 (33.8) 14 (41.2)
 Separated after permanent relationship11 (2.2) 11 (2.4) 0 (0)
Education 3.69 (3)0.296
 None completed10 (2.0) 10 (2.2) 0 (0)
 Secondary education348 (71.0) 325 (71.3) 23 (67.6)
 Tertiary education106 (21.6) 99 (21.7) 7 (20.6)
 Other26 (5.3) 22 (4.8) 4 (11.8)
Yearly household income (in CHF) 1.88 (2)0.392
 <25,000233 (47.6) 220 (48.2) 13 (38.2)
 25,000–50,00092 (18.8) 86 (18.9) 6 (17.6)
 >50,000165 (33.7) 150 (32.9) 15 (44.1)
Due to CV19 Pandemic †
 Status Loss72 (14.7) 67 (14.7) 5 (14.7) 0 (1)1
 Financial problems100 (20.4) 92 (20.2) 8 (23.5) 0.06 (1)0.804
 Job insecurity112 (22.9) 102 (22.4) 10 (29.4) 0.54 (1)0.464
 Job loss41 (8.4) 37 (8.1) 4 (11.8) 0.18 (1)0.674
 Registration with the
 employment center (RAV)		34 (6.9) 30 (6.6) 4 (11.8) 0.64 (1)0.425
 Existential threat95 (19.4) 86 (18.9) 9 (26.5) 0.74 (1)0.391
Psychological Disorder †117 (23.9) 112 (24.6) 5 (14.7) 1.19 (1)0.275
Psychotherapy †95 (19.4) 92 (20.2) 3 (8.8) 1.93 (1)0.164
Psychiatric Medication †64 (13.1) 62 (13.6) 2 (5.9) 1.05 (1)0.306
Depression Cutoff
 PHQ-9 (≥10)322 (65.7) 305 (66.9) 17 (50.0) 3.29 (1)0.070
 MDRS-22 (≥51)67 (13.7) 58 (12.7) 9 (26.5) 3.07 (1)0.046 *
PHQ-9 a 12.6 (6.6) 12.7 (6.6) 11.3 (6.3)1.21 (488)0.225
MDRS-22 b 29.2 (20.6) 28.8 (20.3) 34.1 (24.4)−1.45 (488)0.169
BSRI (m) c 62.5 (14.6) 62.2 (14.6) 65.4 (15.2)−1.20 (488)0.116
MRNI-SF d 45.6 (21.4) 44.9 (20.7) 54.4 (28.9)−1.87 (35.6)0.025 *
GRCS-SF e 47.0 (12.6) 46.7 (12.2) 51.7 (16.0)−1.78 (35.9)0.042 *
FCV-19S f 12.8 (4.9) 12.9 (4.90) 11.8 (4.60)1.28 (488)0.200
CPSS g 23.9 (5.2) 24.0 (5.15) 22.6 (5.38)1.52 (488)0.130
Note: N = number of participants, M = mean, SD = standard deviation, test-statistic = t-value for continuous, χ2-value for categorical variables, df = degrees of freedom, p = p-value, CV19 = COVID-19. Subjective social status was dichotomized using a median-split. † Assessed in self-report a PHQ-9 = Patient Health Questionnaire-9; b MDRS-22 = Male Depression Risk Scale-22; c BSRI (m) = Bem Sex-Role Inventory (m = masculinity subscale); d MRNI-SF = Male Role Norms Inventory–Short Form; e GRCS-SF = Gender Role Conflict Scale–Short Form; f FCV-19S = Fear of COVID-19 Scale; g CPSS = COVID-19 Pandemic Stress Scale. Significant results are displayed in bold. * = p < 0.05, *** = p < 0.001.
Table
Table 2. Correlation matrix for study variables.
Table 2. Correlation matrix for study variables.
MSD123456
1. PHQ-9 a12.66.6̶
2. MDRS-22 b29.220.60.64 ***̶
3. BSRI (m) c62.514.6−0.36 ***−0.10̶
4. MRNI-SF d45.621.4−0.060.17 **0.33 ***̶
5. GRCS-SF e47.012.60.41 ***0.48 ***0.010.39 ***̶
6. FCV-19S f12.84.90.29 ***0.28 ***−0.16 **−0.15 **0.18 ***̶
7. CPSS g23.95.20.31 ***0.28 ***0.03−0.040.24 ***0.58 ***
Note: M = mean, SD = standard deviation. p-values were adjusted for multiple testing using the Holm–Bonferroni method. a PHQ-9 = Patient Health Questionnaire-9; b MDRS-22 = Male Depression Risk Scale-22; c BSRI (m) = Bem Sex-Role Inventory (masculinity subscale); d MRNI–SF = Male Role Norms Inventory–Short Form; e GRCS–SF = Gender Role Conflict Scale–Short Form; f FCV-19S = Fear of COVID-19 Scale; g CPSS = COVID-19 Pandemic Stress Scale; ** = p < 0.01, *** = p < 0.001. Significant results are displayed in bold.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Walther, A.; Eggenberger, L.; Grub, J.; Ogrodniczuk, J.S.; Seidler, Z.E.; Rice, S.M.; Kealy, D.; Oliffe, J.L.; Ehlert, U. Examining the Role of Traditional Masculinity and Depression in Men’s Risk for Contracting COVID-19. Behav. Sci. 2022, 12, 80. https://doi.org/10.3390/bs12030080

AMA Style

Walther A, Eggenberger L, Grub J, Ogrodniczuk JS, Seidler ZE, Rice SM, Kealy D, Oliffe JL, Ehlert U. Examining the Role of Traditional Masculinity and Depression in Men’s Risk for Contracting COVID-19. Behavioral Sciences. 2022; 12(3):80. https://doi.org/10.3390/bs12030080

Chicago/Turabian Style

Walther, Andreas, Lukas Eggenberger, Jessica Grub, John S. Ogrodniczuk, Zac E. Seidler, Simon M. Rice, David Kealy, John L. Oliffe, and Ulrike Ehlert. 2022. "Examining the Role of Traditional Masculinity and Depression in Men’s Risk for Contracting COVID-19" Behavioral Sciences 12, no. 3: 80. https://doi.org/10.3390/bs12030080

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop