Next Article in Journal
Prevalence of Smoking Various Tobacco Types in the Kazakhstani Adult Population in 2021: A Cross-Sectional Study
Previous Article in Journal
Musculoskeletal-Modeling-Based, Full-Body Load-Assessment Tool for Ergonomists (MATE): Method Development and Proof of Concept Case Studies
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Structural Relationships between Environmental Factors, Psychological Health, and Academic Performance in Medical Students Engaged in Online Learning during the COVID-19 Pandemic

by
Ola K. Taleb
1,
Abdullah Sarimah
1,*,
Ab Hamid Siti-Azrin
1,
Kamarul Aryffin Baharuddin
2,3 and
Ali H. Abusafia
4
1
Biostatistics and Research Methodology Unit, School of Medical Sciences, Universiti Sains Malaysia, Kubang Kerian 16150, Kelantan, Malaysia
2
Department of Emergency Medicine, School of Medical Sciences, Universiti Sains Malaysia, Kubang Kerian 16150, Kelantan, Malaysia
3
Hospital Universiti Sains Malaysia, Kubang Kerian 16150, Kelantan, Malaysia
4
School of Health Sciences, Universiti Sains Malaysia, Kubang Kerian 16150, Kelantan, Malaysia
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2023, 20(2), 1494; https://doi.org/10.3390/ijerph20021494
Submission received: 16 October 2022 / Revised: 8 January 2023 / Accepted: 9 January 2023 / Published: 13 January 2023

Abstract

:
As a result of the COVID-19 outbreak and the enforced quarantine, universities in Malaysia were required to switch to an online class format. The resulting changes in the environmental factors of students may have had an impact on their psychological health and academic performance. This study aimed to determine the effects of environmental factors and the psychological health of students and examine their structural relationship with academic performance. A cross-sectional design with an online self-reported questionnaire was adopted, and the study was conducted among 207 undergraduate medical students at the Health Campus of Universiti Sains Malaysia. The environmental factors were measured using the lighting–noise–temperature scale and technology scale, while psychological health was assessed using the short version of the General Health Questionnaire and academic performance was determined based on Grade Point Average. Descriptive statistics and structural equation modeling were used for analysis of the data. No significant relationship was found between environmental factors and academic performance, or between environmental factors and psychological health. Nonetheless, the hypothesized structural model provided scientific evidence of an inverse relationship between psychological health and academic performance. These findings could be helpful for academics, health policymakers, and health educators in terms of understanding and promoting psychological wellbeing among university students, as well as improving their academic performance.

1. Introduction

The global disruption caused by COVID-19 poses a massive challenge to global health and continues to affect people as well as students worldwide. Borders and all academic institutions were closed, and economic activities came to a standstill due to the implementation of the Movement Control Order (MCO) in Malaysia on 18 March 2020, to keep the spread of COVID-19 and associated mortality under control [1]. The campus closure and movement restriction order had an impact on formal learning. In April 2020, the Malaysian government ordered students to return to their home countries and continue their studies through online learning [2].
Medical students were one of the most severely affected groups, as their lessons at hospital and other health care centers where they learn essential clinical skills had been suspended until further notice. They were also required to adapt quickly to the online learning format, which required much adjustment and was unlike their face-to-face and patient-based learning format in the health care setting. These unfavorable circumstances probably led to an increase in stress levels among medical students, especially those who experienced coping and adjustment difficulties. All these factors may have contributed to depression, anxiety, and stress among medical students.
Mahdy (2020) demonstrated that the lockdown imposed as a result of the COVID-19 pandemic had varying degrees of impact on most students’ academic performance [3], as they were required to adapt themselves to new environmental conditions, such as lighting, noise, temperature (LNT) and technologies, that are different from those at classrooms in their university. These new environmental conditions could be uncomfortable and have an impact on their academic performance [4]. Adequate lighting is important for the learning process. During the day, natural light through windows is available, and it may be augmented with artificial light sources, with the artificial illumination continuing throughout the night [5]. In addition, excessive noise is also detrimental to the teaching and learning process because it is distracting and limits attention and cognition. Specifically, excessive noise can make it difficult for students to hear and understand the teacher’s speech [5,6]. In addition, thermal discomfort caused by elevated classroom temperatures has been demonstrated to affect students’ ability to accomplish standard academic tasks [7,8]. Further, heat stress caused by high outside temperatures has been found to be associated with an increase in the number of students failing tests [9]. Inadequate online learning infrastructure and limited accessibility to the internet also make the online learning process harder for students [10]. Students need to have access to reliable equipment [11], and students with limited access are at a significant disadvantage when compared to students with unlimited access [12].
During the COVID-19 pandemic, psychological health problems, especially depression, anxiety, and stress, have become common among students, especially medical students [13]. In students at higher-education institutions, psychological factors have been found to affect academic achievements as a result of test anxiety, poor performance, low self-confidence, unrealistic worry, and fear or uneasiness that interfere with their ability to function normally [14,15,16]. Several recent studies have suggested that environmental factors may also contribute to the onset or aggravation of psychological health [17]. Lighting is one of the most important factors for a positive learning environment [18], and it has an important role in improving the psychological health of students [19]. There is evidence for the significance of lighting for psychological wellbeing, as well as personal health and wellbeing [20]. For example, Mirrahimi et al. (2013) demonstrated the effects of lighting on the health, psychology, and cognitive abilities of students [19]. In addition, noise has been found to have negative effects on psychological health [21] by causing emotional distress and discomfort [22]. For example, Beutel et al. (2016) demonstrated that depression and generalized anxiety disorder increased dose-dependently with the degree of total noise annoyance [23]. Further, Bourion-Bédès et al. (2021) demonstrated that one of the main risk factors for anxiety among students during lockdown was noise, both inside and outside the house [24], and a significant association between noise exposure and stress levels has also been reported (p = 0.023) [25]. Rising temperatures can directly and indirectly cause human pathologies that affect both physical and mental health [26]. The tropical climate in Malaysia is hot and humid, and this environment may have an impact on air temperature, causing dissatisfaction, unhappiness, and discomfort among students and affecting their productivity [27].
The switch to the online learning mode has led to severe anxiety among medical students, who have also reported inadequate internet access, which is required for online learning [28]. Happiness does have a statistically significant wealth gradient, with students with internet access being more likely to be happy [29]. According to Haris & Al-Maadeed (2021), the pandemic and resulting transition to the online learning environment has led to mental stress among students due to the absence of a campus environment and connectivity, as well as technical problems [30].
However, no studies in Malaysia so far have focused on the relationship between psychological health and environmental factors (LNT and technology) and how they affect academic performance. Therefore, this research aimed to study the relationship between environmental factors (LNT and technology), psychological health, and academic performance among medical students at the Health Campus of USM during the COVID-19 period in Malaysia. It is important to examine the factors that influence students’ academic performance during the pandemic period, as this can provide insight into how the Malaysian university and government can improve the online learning environment in Malaysia. The hypothesized relationship between environmental factors, psychological health, and academic performance is depicted in Figure 1.

2. Materials and Methods

2.1. Study Design and Ethical Approval

A cross-sectional study was conducted on 207 undergraduate medical students from the School of Medical Sciences, USM. Second- and third-year undergraduate medical students in academic year 2021–2022 who participated in online learning during the second semester of academic year 2020–2021 were included. Re-sit students and students with a history of underlying psychological issues were excluded.
The study was approved by the Human Ethics Committee of USM (approval number USM/JEPeM/21080583) and the academic office of the university. This ethical approval allowed the researcher to conduct the research on the target population. Implied consent was obtained from each of the participants after they had received the link to the Google form of the questionnaire. The information provided about the research included the purpose of the study, study procedures, potential risks of participation, and benefits of participation in the study. All participants voluntarily participated in this research study. The researcher guaranteed that the data collected were kept confidential and only used for research purposes. After the survey was completed, the participants were thanked for their involvement in the research study.
Table 1 lists the participants’ demographic information. As shown in the table, 52% of the participants were in their second year, and 73% were women. Most participants (96.1%) were of Malay descent, and these participants continued to live on campus while learning online. Laptops were the primary digital device for 62.8%, who learned individually. Further, 88.4% of the participants used Wi-Fi to attend the online courses.
With regard to the classification of household income, there are three classes of household income in Malaysia: B40, M40, and T20. B40 represents the bottom 40% of the average Malaysian household income and corresponds to less than RM4850 per month. M40 represents the middle 40% and corresponds to RM4851–RM10970 per month. T20 represents the top 20% of Malaysian household income and corresponds to RM10971 per month. In this study, 42% of participants were classified under M40, that is, they had a family income of RM4851 to RM10970.

2.2. Sample Size

The sample size was calculated based on the method of Kline (2011) [31]. He suggested, using structural equation modeling (SEM), that the acceptable sample size for studies is about 200 cases. Considering a 20% dropout rate, the minimum required sample size for the study was 250 students. A convenience sampling method was applied to select participants based on the inclusion and exclusion criteria.

2.3. Data Collection

Data were collected using an online self-administered questionnaire. The link to the Google Form of the questionnaire was sent to the class leader of each year through WhatsApp and email. They shared the online survey link with other students. Participants completed the online survey in 20–30 min. They were informed that it was compulsory to answer all the questions. The GPA reported by the students in the survey was verified with the results provided by the academic office.
This questionnaire consists of three parts: (i) 9-item proforma form (about sociodemographic profiles and GPA for the second semester of the academic year 2020/2021 examination), (ii) 15-item questionnaire on environmental factors (LNT and technology questionnaires) and (iii) 12-item psychological health questionnaire (GHQ-12).

2.4. Instruments

2.4.1. The Environmental Questionnaire

LNT Scale

The LNT scale was developed by Realyvásquez-Vargas et al. (2020) [4] and consists of nine items that are used to assess the environmental factors that impact students’ online learning in three domains, namely, lighting, noise, and temperature. The English version of LNT was found to have a good fit based on the following values: comparative fit index (CFI) = 0.99, Tucker–Lewis index (TLI) = 0.98, standardized root mean square residual (SRMR) = 0.03, and root mean square error of approximation (RMSEA) = 0.03 (90% confidence interval [CI] = 0.00–0.07). For lighting, noise, and temperature, the composite reliability (CR) values were 0.81, 0.81, and 0.84, respectively, and the average variance extracted (AVE) values were 0.61, 0.59, and 0.63, respectively [11].
The present study shows that the measurement model had a good fit based on recommended fit indices without modification indices. The results of robust fit indices were as follows: CFI = 0.99, TLI = 0.98, SRMR = 0.03, and RMSEA = 0.03. The factor loadings ranged from 0.74 to 0.82 (p-value < 0.001). The CR for lighting, noise, and temperature was 0.82, 0.81, and 0.84, respectively. The AVE for lighting, noise, and temperature was 0.61, 0.59, and 0.63, respectively.
The tool uses a five-point Likert scale, where 1 = never, 2 = hardly ever, 3 = sometimes, 4 = usually, and 5 = always. The first domain is lighting (three items), which measures the effects of indoor lighting on students’ academic performance during online classes. The second domain is noise (three items), which measures the effects of noise pollution on students’ academic performance during online classes. The third domain is temperature (three items), which measures the effects of thermal comfort on students’ academic performance during online classes.

Technology Scale

The technology scale was developed by Abou Naaj et al. (2012) [11] and consists of six items that measure the adequacy of technology that impacts students’ academic performance during online classes. It uses a five-point Likert scale, where 1 = strongly disagree, 2 = disagree, 3 = neither agree nor disagree, 4 = agree, and 5 = strongly agree. The English version of the technology scale showed a good fit to the data: CFI = 0.99, TLI = 0.97, SRMR = 0.02, RMSEA = 0.06 (95% CI = 0.00–0.16). The CR was 0.84 and AVE was 0.51 [11].
The present study showed that the model had a good fit based on the values of all indices, except for the upper 90% CI of robust RMSEA, which was 0.16. These were the values for the remaining robust fit indices: CFI = 0.99, TLI = 0.97, SRMR = 0.02, and RMSEA = 0.06. The factor loadings ranged from 0.62 to 0.79 (p-value < 0.001). The CR for technology was 0.84, and the AVE for technology was 0.51. Figure 2 shows error covariances among the indicators for technology.

GHQ-12 Scale

The 12-item GHQ-12 scale provides a quantitative measure of psychological health. The GHQ is a screening instrument that was developed to determine the severity of psychological distress that an individual has experienced during the previous 6 months using a four-point scale ranging from 0 (more than usual) to 3 (much less than usual). It consists of 12 items that describe mood states, with six of them positively phrased (1, 2, 4, 7, 8, and 12) and six of them negatively phrased (2, 5, 6, 9, 10, and 11) [32]. According to the standard GHQ scoring method (GHQ–0011), GHQ items are scored dichotomously. Score 0 and 1 was replaced to Score 0, whereas Score 2 and 3 was replaced to 1. The scores range from 0 to 12, with a total score of 4 and above indicating psychological problems. The English version of GHQ-12 has good validity [33].

2.5. Statistical Analysis

IBM SPSS Statistics version 27.0 (IBM Corp., Armonk, NY, USA) and the Lavaan package of R Studio were used for data analysis. Descriptive statistics of the numerical variables are presented as mean and standard deviation (SD), while categorical variables are presented as frequency and percentage. Further, inferential statistics were analyzed with SEM, which was used to examine the structural relationship between environmental factors, psychological health, and academic performance. Due to the violation of normality, the maximum likelihood estimator (MLR) was used in the analysis. The items in the measurement model for psychological health were parceled into one domain (psychological health), while the measurement models LNT and Technology were maintained.
Fit indices were used to evaluate the model’s fitness. RMSEA < 0.08, SRMR < 0.08, TLI > 0.95, and CFI > 0.95 [31,34]. Significant paths (β) with the corresponding 95% CI, critical ratio, and statistical significance values (<0.05) are presented in this study.

3. Results

3.1. Descriptive Statistics of Constructs

The items in the measurement model for psychological health were parceled into one domain, named psychological health, and were treated as observed items. The descriptive statistics of the constructs are shown in Table 2.

3.2. LNT Scale

Nine items in Table 3 were applied to assess environmental factors (LNT) related to the online classes that affect academic performance. More than 80% of the students reported that they were satisfied with the lighting, and 70% were satisfied with the temperature. However, only half of the students stated that they were able to tolerate noise.

3.3. Technology Scale

The six items presented in Table 4 were applied to assess the technical quality of online classes that affect their academic performance. Only 57.8% of students were satisfied, while 15% encountered technical problems.

3.4. Psychological Health Scale

The twelve items presented in Table 5 were applied to assess the psychological health of participants and how it affected their academic performance. According to the results, 52.7% of students had a total score of less than 4, while 47.3% had a total score more than 4.

3.5. Initial Structural Model

The environmental factors (LNT and technology), psychological health, and academic performance were imputed into the proposed structural model using the estimator MLR, in order to examine their relationship. Table 6 shows the results from the initial model (model 1) which showed that the data had a good fit (CFI = 0.96, TLI = 0.95, SRMR = 0.05, RMSEA (90% CI) = 0.04 (0.02–0.06). The value of chi-square (degree of freedom) was 137 (102). The results met the recommended value. Therefore, no modification was required, and model 1 was considered as the final model (Figure 2). Model 1 was tested, and the results showed that only one out of nine of the hypothesized relationships was significant. The path relationships in the final model are summarized in Table 7, which differentiates between significant and non-significant paths based on the SEM results. The results from Table 7 show that there was an inverse relationship between psychological health scores and academic performance based on GPA (β = −0.21, p = 0.004).

4. Discussion

4.1. Online Learning

A virtual learning environment (VLE) is a set of teaching and learning tools designed to enhance a student’s learning experience by including computers and the internet in the learning process. The principal components of a VLE package include curriculum mapping (breaking curriculum into sections that can be assigned and assessed), student tracking, online support for both teacher and student, electronic communication (e-mail, threaded discussions, chat, and web publishing), and internet links to outside curriculum resources. In general, VLE users are assigned either a teacher ID or a student ID. The teacher sees what a student sees, but the teacher has additional user rights to create or modify curriculum content and track student performance. There are a number of commercial VLE software packages available, including Blackboard, WebCT, Lotus LearningSpace, and COSE.
In view of the COVID-19 pandemic and the MCO imposed by the government, a blended learning substitution model was adopted. With this model, students learn through online classes as well as traditional face-to-face lessons. Online class learning is composed of asynchronous and synchronous methods.
Asynchronous methods include online teaching and learning strategies that do not involve real-time interaction, for example, recorded videos/lectures and self-guided learning materials. The Elearn@USM platform is the university’s official e-learning platform used for asynchronous online teaching and learning. Both lecturer and student can access this learning platform.
Synchronous online teaching and learning methods involve online teaching and learning in real time. Lecturers use online meeting platforms, such as Webex, Microsoft Team, Zoom, Google Meet, and others, to conduct teaching sessions with students. The students can interact or ask questions, and lecturers can respond in real time.
The COVID-19 pandemic and lockdown had varying degrees of effects on most students [3]. Some students experienced isolation and loss of interest after participating in online learning for a prolonged period. They also experienced fatigue as a result of sleeping problems, technological problems, and rapid system and environmental changes. Several recent studies have suggested that environmental factors may contribute to the onset or aggravation of psychological health [27].

4.2. Environmental Factors (LNT and Technology), Psychological Health, and Academic Performance

In the current study, the overall mean (SD) scores for the lighting and noise scales were 12.78 (2.13) and 10.19 (2.76), respectively. A study by Realyvásquez-Vargas et al. (2020) on Mexican students which used the same scales reported that the overall mean score of lighting and noise were 11.88 and 9.95, respectively [4]. These results indicate that the USM students in this cohort had better lighting and noise conditions than the Mexican students from the previous study. This is probably because most students from USM lived within the campus, and the university provides good conditions for student learning. In contrast, the overall mean (SD) for the temperature scale was 11.79 (2.41) in this study and 11.48 in the Realyvásquez-Vargas et al. (2020) study [4]. This implies that the Mexican students had better temperature conditions than the USM students. This is also probably because most students from USM lived within the campus in rooms that did not have air conditioning. In addition, the tropical hot and humid climate of Malaysia may also have contributed to this score. This environment may have caused discomfort and, therefore, dissatisfaction unhappiness among students, and affected their productivity [27].
The overall mean score (SD) for the technology scale was 23.88 (3.47). Compared to study by Selvanathan et al. (2020), which was also conducted among students in Malaysia’s universities and used the same scale, the overall mean score for the technology scale was 3.66 and the overall mean score was 20.22 [35]. Although both studies were conducted in Malaysia, the current study has a higher mean score than the Selvanathan et al. (2020) study. This difference is probably because data collection in the previous study was done when the students were living at home. Therefore, the scores may be related to the socio-economic status of their families and the quality of the technical issues. In contrast, in the current study, most of the students stayed within the campus, where they have similar access to technology.
In the current study, the mean score (SD) for psychological health was 4.16 (3.92). A score of more than 4 indicates that the participant has psychological problems. Therefore, the mean score implies that most undergraduate medical students at USM have psychological problems. This result was somewhat similar to that for medical students from Suez Canal University, Egypt, who had a mean (SD) score of 4.7 (3.1). However, the score in the present cohort was higher than that reported in previous studies on undergraduate students (3.12 (3.32)) from the faculties of medicine, dentistry, and health sciences on the Health Campus of USM [36].
The interrelationship between environmental factors, psychological health, and academic performance were examined with SEM. The structural framework was built on a conceptual framework based on previous theoretical and empirical studies. The findings of the structural model analysis revealed that only one of the nine hypothesized relationships was significant. That is, in this study, psychological health scores were found to have an inverse relationship with academic performance (indicated by GPA). The findings showed that students with higher scores for psychological health were likely to have lower GPAs. This supports the findings of previous studies which showed that higher scores for psychological health were associated with lower GPAs [14,15,37,38]. Similarly, in a longitudinal study conducted among Emirati students to determine the relationship between mental health and students’ academic performance, higher Patient Health Questionnaire (PHQ-9) scores were associated with lower current GPAs (rs  =  −0.171, n  =  397, p  <  0.001) [37]. In addition, a cross-sectional study conducted among Malaysian medical colleges reported that stress, anxiety, and depression have a negative impact on the learning and academic performance of university students [38].
Further, Bas (2021) demonstrated a positive relationship between mental health and academic performance [39], and Chattu et al. (2002) also showed a significant association between psychological health and the academic performance of students [14]. According to Duffy et al. (2020), depressive symptoms cause a decrease in GPA [16]. Further, Beharu (2018) showed that psychological factors, such as stress, anxiety, depression, lack of motivation, loneliness, helplessness, and phobias, in students in higher-education institutions can affect their academic achievements by causing test anxiety, poor performance, low self-confidence, unrealistic worry, and fear or uneasiness, which interfere with their ability to function normally [15]. Along the same line, in Bas’s (2021) meta-analysis, a positive relationship was found between mental health and academic performance [39]. According to Mahdy (2020), the lockdown had variable degrees of impact on most students’ academic performance [3]. For example, loneliness, loss of interest, and fatigue are commonly reported after online learning for a prolonged period. Psychological health may influence academic performance in a variety of ways, as those with psychological problems may find interactions difficult and may fail to engage in their classes [40].
In the present study, environmental factors (LNT and technology) did not have a significant relationship with academic performance. This is because most of medical students had good score in academic performance. The students learned through Elearn@USM platform as an asynchronous online learning. Asynchronous learning includes recorded videos/lectures, self-guided learning materials, etc. If the students had any problems regarding their subjects, they can look back at the recorded videos, uploaded by their lecturers.
However, this contradicts the findings of previous studies [3,41,42,43,44,45,46]. For example, Mogas-Recalde & Palau (2021) confirmed the direct impact of classroom lighting on student performance (concentration, attentiveness, achievement, etc.) [41], and Brink et al. (2021) showed that the indoor environmental quality, which includes indoor air, as well as thermal, acoustic, and lighting conditions, can contribute positively to the quality of learning and short-term academic performance of students [42].
In Mexico, a study was conducted by Realyvásquez-Vargas et al. (2020) to explore whether environmental factors (LNT) affected students’ academic performance during COVID-19 [4]. They found that students’ academic performance was indeed affected by environmental factors during the COVID-19 pandemic. That is, students with better environmental conditions were found to have better academic performance according to the QEOC questionnaire. Similarly, Norazman et al. (2018) discovered that the quality of lighting in the classroom, whether natural or artificial, has an impact on students’ achievement, motivation, attendance, skills, concentration, and focus throughout the day [43], and Samani & Samani (2012) showed that there was a significant relationship between lighting quality and student’s learning achievement [44]. Along the same line, Gonzales et al. (2018) illustrated how intermittent internet disconnection problems negatively affected the GPA of students [45]. Thus, adequate internet services enhance access to information required by the students, and this has a positive effect on their academic achievement [46].
The present findings indicated that environmental factors (LNT and technology) do not have a significant relationship with psychological health, but these findings are also contrary to those of previous studies [21,24,25,26,28,29,30]. This difference in results is probably attributable to the living conditions of the present cohort, as most of them lived within the campus and were exposed to the same environmental factors. Thus, the incorrect conclusion that the independent variable has no significance could have been the result of a ceiling effect [46]. The ceiling effect occurs when a variable’s scores approach their maximum [47]. Pilot testing is the best solution to resolve this issue, since it allows for the problem to be discovered early [48]. Another method is to collapse categories that distribute data more evenly in order to decrease the floor or ceiling impact. This strategy, however, may result in loss of information and power, and this could change the implication of many health-related outcomes with a well-defined range of possible responses [49]. In addition, adjusted weight least square mean and variances or MLR with numerical integration can be applied to analyze ordered categorical data with the ceiling effect [50]. The researchers in the current chose MLR as the estimator. As only one of the hypothesized relationships was significant, the indirect effect and interaction were not analyzed.
To the best of the researcher’s knowledge, this is the first study to use SEM to examine the relationship between environmental factors (LNT and technology), psychological health, and academic performance among Malaysian students. SEM was used to investigate the multiple interrelationships between variables in the hypothesized structural model. As a result, the type 1 error for the study is assumed to be below. In addition, the questionnaires used were valid and reliable, and the GPA was verified by the academic office. However, there are a few limitations that need to be addressed. The data were collected only from a single college and university, and this may have led to a ceiling effect as most students were exposed to the same environmental factors. In addition, it limits the generalizability of the results to other university students. Due to time constraints, data re-collection to anticipate the problem arising from the ceiling effect was not conducted. The convenience sampling method was applied, instead of random sampling, because of the online format of the questionnaires, the large sample size needed for this study, and the time constraint. As online self-administered questionnaires were used for data collection, there is a possibility of bias arising from insincere or dishonest responses. Furthermore, the study had a cross-sectional design. As a result, the exposure and the outcome were evaluated concurrently, and this made it impossible to distinguish between the cause-and-effect factors. Thus, it was not possible to determine the direction of effect due to a lack of evidence to establish a temporal association between the exposure and the outcome. For example, psychological health scores were found to be significantly and negatively associated with GPA. However, the data do not shed light on whether psychological health problems lead to low GPA, or vice versa. Moreover, the researcher did not include internet speed issue as a technology-related item. It is recommended that this factor be included as a technology-related item in future studies on this topic.

5. Conclusions

The present study demonstrated an inverse relationship between psychological health scores and academic performance. In contrast, environmental factors were not found to be associated with psychological health or academic performance. Therefore, the environmental factors light, noise, temperature, and technology were not found to have a direct or indirect effect on academic performance.

Author Contributions

O.K.T., A.S. and A.H.S.-A. designed the study, O.K.T., A.S., A.H.S.-A., K.A.B. and A.H.A. carried out the literature searches, O.K.T. and A.S. extracted the data, O.K.T., A.S., A.H.S.-A., K.A.B. and A.S. performed statistical analysis, O.K.T. wrote the manuscript, O.K.T., A.S., A.H.S.-A., K.A.B. and A.H.A. revised the manuscript. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki, and approved by the Human Ethics Committee of USM (USM/JEPeM/21080583) and the academic office.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Not applicable.

Acknowledgments

The authors would like to thank all the medical students of USM who participated in this study.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Seetan, K.; Al-Zubi, M.; Rubbai, Y.; Athamneh, M.; Khamees, A.; Radaideh, T. Impact of COVID-19 on medical students’ mental wellbeing in Jordan. PLoS ONE 2021, 16, e0253295. [Google Scholar] [CrossRef]
  2. QS Quacquarelli Symonds (Firm). The Impact of the Coronavirus on Global Higher Education; QS Quacquarelli Symonds: London, UK, 2020. [Google Scholar]
  3. Mahdy, M.A.A. The Impact of COVID-19 Pandemic on the Academic Performance of Veterinary Medical Students. Front. Vet. Sci. 2020, 7, 594261. [Google Scholar] [CrossRef] [PubMed]
  4. Realyvásquez-Vargas, A.; Maldonado-Macías, A.A.; Arredondo-Soto, K.C.; Baez-Lopez, Y.; Carrillo-Gutiérrez, T.; Hernández-Escobedo, G. The impact of environmental factors on academic performance of university students taking online classes during the COVID-19 Pandemic in Mexico. Sustainability 2020, 12, 9194. [Google Scholar] [CrossRef]
  5. Kooijman, P.G.; Thomas, G.; Graamans, K.; De Jong FI CR, S. Psychosocial impact of the teacher’s voice throughout the career. J. Voice 2007, 21, 316–324. [Google Scholar] [CrossRef]
  6. De Almeida Filho, N.; Filletti, F.; Guillaumon, H.R.; Serafini, F. Intensity of noise in the classroom and analysis of acoustic emissions in schoolchildren. Int. Arch. Otorinolaringol. 2012, 16, 91–95. [Google Scholar]
  7. Bakó-Biró, Z.; Clements-Croome, D.; Kochhar, N.; Awbi, H.; Williams, M. Ventilation rates in schools and pupils’ performance. Build. Environ. 2012, 48, 215–223. [Google Scholar] [CrossRef]
  8. Porras-Salazar, J.A.; Wyon, D.P.; Piderit-Moreno, B.; Contreras-Espinoza, S.; Wargocki, P. Reducing classroom temperature in a tropical climate improved the thermal comfort and the performance of elementary school pupils. Indoor Air 2018, 28, 892–904. [Google Scholar] [CrossRef]
  9. Park, R.J.; Goodman, J.; Hurwitz, M.; Smith, J. Heat and learning. Am. Econ. J. Econ. Policy 2020, 12, 306–339. [Google Scholar] [CrossRef]
  10. Lee, S. Sabah Student Stays Overnight in Tree to Get Better Internet Connection for Online University Exams; The Star: Petaling Jaya, Malaysia, 2020. [Google Scholar]
  11. Naaj, M.A.; Nachouki, M.; Ankit, A. Evaluating Student Satisfaction with Blended Learning in a Gender-Segregated Environment. J. Inf. Technol. Educ. Res. 2012, 11, 185–200. [Google Scholar]
  12. Wegerif, R. The social dimension of asynchronous learning networks. J. Asynchronous Learn. Netw. 1998, 2, 34–49. [Google Scholar] [CrossRef]
  13. Zhou, S.-J.; Zhang, L.-G.; Wang, L.-L.; Guo, Z.-C.; Wang, J.-Q.; Chen, J.-C.; Liu, M.; Chen, X.; Chen, J.-X. Prevalence and socio-demographic correlates of psychological health problems in Chinese adolescents during the outbreak of COVID-19. Eur. Child Adolesc. Psychiatry 2020, 29, 749–758. [Google Scholar] [CrossRef]
  14. Chattu, V.K.; Sahu, P.K.; Seedial, N.; Seecharan, G.; Seepersad, A.; Seunarine, M.; Sieunarine, S.; Seymour, K.; Simboo, S.; Singh, A. An Exploratory Study of Quality of Life and Its Relationship with Academic Performance among Students in Medical and other Health Professions. Med. Sci. 2020, 8, 23. [Google Scholar] [CrossRef] [PubMed]
  15. Beharu, W.T. Psychological factors affecting students’ academic performance among freshman psychology students in Dire Dawa University. J. Educ. Prac. 2018, 9, 59–65. [Google Scholar]
  16. Duffy, A.; Keown-Stoneman, C.; Goodday, S.; Horrocks, J.; Lowe, M.; King, N.; Pickett, W.; McNevin, S.H.; Cunningham, S.; Rivera, D.; et al. Predictors of mental health and academic outcomes in first-year university students: Identifying prevention and early-intervention targets. BJPsych Open 2020, 6, e46. [Google Scholar] [CrossRef] [PubMed]
  17. Peng, Z.; Wang, Q.; Kan, H.; Chen, R.; Wang, W. Effects of ambient temperature on daily hospital admissions for mental disorders in Shanghai, China: A time-series analysis. Sci. Total Environ. 2017, 590, 281–286. [Google Scholar] [CrossRef] [PubMed]
  18. Brittin, J.; Sorensen, D.; Trowbridge, M.; Lee, K.K.; Breithecker, D.; Frerichs, L.; Huang, T. Physical Activity Design Guidelines for School Architecture. PLoS ONE 2015, 10, e0132597. [Google Scholar] [CrossRef] [Green Version]
  19. Mirrahimi, S.; Ibrahim, N.N.; Surat, M. Effect of daylighting on student health and performance. In Proceedings of the 15th International Conference on Mathematical and Computational Methods in Science and Engineering, Kuala Lumpur, Malaysia, 2–4 April 2013; pp. 2–4. [Google Scholar]
  20. Atre, U.V. Effect of Daylighting on Energy Consumption and Daylight Quality in an Existing Elementary School; Texas A&M University: College Station, TX, USA, 2003. [Google Scholar]
  21. Lim, J.; Kweon, K.; Kim, H.W.; Cho, S.W.; Park, J.; Sim, C.S. Negative impact of noise and noise sensitivity on mental health in childhood. Noise Health 2018, 20, 199. [Google Scholar] [PubMed]
  22. Matondang, N. The impact of noise level on students’ learning performance at state elementary school in Medan. AIP Conf. Proc. 2017, 1855, 040002. [Google Scholar]
  23. Beutel, M.E.; Jünger, C.; Klein, E.M.; Wild, P.; Lackner, K.; Blettner, M.; Binder, H.; Michal, M.; Wiltink, J.; Brähler, E.; et al. Noise Annoyance Is Associated with Depression and Anxiety in the General Population- The Contribution of Aircraft Noise. PLoS ONE 2016, 11, e0155357. [Google Scholar] [CrossRef]
  24. Bourion-Bédès, S.; Tarquinio, C.; Batt, M.; Tarquinio, P.; Lebreuilly, R.; Sorsana, C.; Baumann, C. Psychological impact of the COVID-19 outbreak on students in a French region severely affected by the disease: Results of the PIMS-CoV 19 study. Psychiatry Res. 2021, 295, 113559. [Google Scholar] [CrossRef]
  25. Jahangeer, S.M.A.; Hasnain, N.; Tariq, M.T.; Jamil, A.; Zia, S.Y.; Amir, W. Frequency and Association of Stress Levels with Modes of Commuting Among Medical Students of a Developing Country. Malays. J. Med. Sci. 2021, 28, 113–122. [Google Scholar] [CrossRef]
  26. Cianconi, P.; Betrò, S.; Janiri, L. The Impact of Climate Change on Mental Health: A Systematic Descriptive Review. Front. Psychiatry 2020, 11, 74. [Google Scholar] [CrossRef] [PubMed]
  27. Kamaruzzaman, K.; Tazilan, A.S.B.M. Thermal comfort assessment of a classroom in tropical climate conditions. Recent Adv. Energy Environ. Dev. 2013, 87–91. [Google Scholar]
  28. Irawan, A.W.; Dwisona, D.; Lestari, M. Psychological Impacts of Students on Online Learning During the Pandemic COVID-19. KONSELI: J. Bimbing. dan Konseling (E-Journal) 2020, 7, 53–60. [Google Scholar] [CrossRef]
  29. Asanov, I.; Flores, F.; McKenzie, D.; Mensmann, M.; Schulte, M. Remote-learning, time-use, and mental health of Ecuadorian high-school students during the COVID-19 quarantine. World Dev. 2020, 138, 105225. [Google Scholar] [CrossRef]
  30. Haris, R.M.; Al-Maadeed, S. COVID-19 lockdown-challenges to higher education in qatar. In Proceedings of the 2021 IEEE 11th IEEE Symposium on Computer Applications & Industrial Electronics (ISCAIE), Penang, Malaysia, 3–4 April 2021; pp. 267–273. [Google Scholar]
  31. Kline, R.B. Principles and Practice of Structural Equation Modeling, 3rd ed.; The Guilford Press: New York, NY, USA, 2011. [Google Scholar]
  32. Melaku, L.; Mossie, A.; Negash, A. Stress among medical students and its association with substance use and academic performance. J. Biomed Educ. 2015, 4, 1–9. [Google Scholar] [CrossRef] [Green Version]
  33. Goldberg, D.P. User’s Guide to the General Health Questionnaire; NFER-NELSON: Windsor, UK, 1988. [Google Scholar]
  34. Hair, J.F.; Black, W.C.; Babin, B.J.; Anderson, R.E. Multirative Data Analysis: A Global Perspective; Pearson: Harlow, UK, 2010. [Google Scholar]
  35. Selvanathan, M.; Hussin, N.A.M.; Azazi, N.A.N. Students learning experiences during COVID-19: Work from home period in Malaysian Higher Learning Institutions. Teach. Public Adm. 2020. [Google Scholar] [CrossRef]
  36. Che Rahimi, A.; Bakar, R.S.; Mohd Yasin, M.A. Psychological Well-Being of Malaysian University Students during COVID-19 Pandemic: Do Religiosity and Religious Coping Matter? Healthcare 2021, 9, 1535. [Google Scholar] [CrossRef]
  37. Awadalla, S.; Davies, E.B.; Glazebrook, C. A longitudinal cohort study to explore the relationship between depression, anxiety and academic performance among Emirati university students. BMC Psychiatry 2020, 20, 1–10. [Google Scholar] [CrossRef]
  38. Fauzi, M.; Anuar, T.; Teh, L.; Lim, W.; James, R.; Ahmad, R.; Mohamed, M.; Abu Bakar, S.; Yusof, F.M.; Salleh, M. Stress, Anxiety and Depression among a Cohort of Health Sciences Undergraduate Students: The Prevalence and Risk Factors. Int. J. Environ. Res. Public Health 2021, 18, 3269. [Google Scholar] [CrossRef]
  39. Bas, G. Relation between Student Mental Health and Academic Achievement Revisited: A Meta-Analysis. In Health and Academic Achievement-New Findings; IntechOpen: Rijeka, Croatia, 2021. [Google Scholar]
  40. Abu Ruz, M.E.; Al-Akash, H.Y.; Jarrah, S. Persistent (Anxiety and Depression) Affected Academic Achievement and Absenteeism in Nursing Students. Open Nurs. J. 2018, 12, 171–179. [Google Scholar] [CrossRef] [PubMed]
  41. Mogas-Recalde, J.; Palau, R. Classroom Lighting and Its Effect on Student Learning and Performance: Towards Smarter Conditions. In Ludic, Co-design and Tools Supporting Smart Learning Ecosystems and Smart Education; Springer: Singapore, 2021; pp. 3–12. [Google Scholar]
  42. Brink, H.W.; Loomans, M.G.; Mobach, M.P.; Kort, H.S. Classrooms’ indoor environmental conditions affecting the academic achievement of students and teachers in higher education: A systematic literature review. Indoor Air 2021, 31, 405–425. [Google Scholar] [CrossRef] [PubMed]
  43. Norazman, N.; Ani AI, C.; Jaâ, N.H.; Khoiry, M.A. Indoor Lighting in Classroom Environment Influences on Students’ Learning Performance. J. Soc. Sci. Res. 2018, 6, 986–990. [Google Scholar]
  44. Samani, S.A.; Samani, S.A. The impact of indoor lighting on students’ learning performance in learning environments: A knowledge internalization perspective. Int. J. Bus. Soc. Sci. 2012, 3, 127–136. [Google Scholar]
  45. Gonzales, A.L.; McCrory Calarco, J.; Lynch, T. Technology problems and student achievement gaps: A validation and extension of the technology maintenance construct. Com. Res. 2020, 47, 750–770. [Google Scholar] [CrossRef] [Green Version]
  46. Jibrin, M.A.; Musa, M.N.; Shittu, T. Effects of internet on the academic performance of tertiary institutions’ students in Niger State, Nigeria. Int. J. Educ. Learn Train. 2017, 2, 57–69. [Google Scholar]
  47. Cramer, D.; Howitt, D.L. The Sage Dictionary of Statistics: A Practical Resource for Students in the Social Sciences; Sage: Thousand Oaks, CA, USA, 2004. [Google Scholar]
  48. Šimkovic, M.; Träuble, B. Robustness of statistical methods when measure is affected by ceiling and/or floor effect. PLoS ONE 2019, 14, e0220889. [Google Scholar] [CrossRef]
  49. Lewis-Beck, M.; Bryman, A.E.; Liao, T.F. The Sage Encyclopedia of Social Science Research Methods; Sage Publications: Thousand Oaks, CA, USA, 2003. [Google Scholar]
  50. Gunzler, D.D.; Perzynski, A.T.; Carle, A.C. Structural Equation Modeling for Health and Medicine; Chapman and Hall/CRC: Boca Raton, FL, USA, 2021. [Google Scholar]
Figure 1. Hypothesized model of constructs.
Figure 1. Hypothesized model of constructs.
Ijerph 20 01494 g001
Figure 2. Final SEM Model (model 1). Tch = Technology, T = Temperature, N = Noise, L = Lighting, P_H = Psychological health, GPA = Grade Point Average (academic performance).
Figure 2. Final SEM Model (model 1). Tch = Technology, T = Temperature, N = Noise, L = Lighting, P_H = Psychological health, GPA = Grade Point Average (academic performance).
Ijerph 20 01494 g002
Table 1. Demographic characteristics of the participants (n = 207).
Table 1. Demographic characteristics of the participants (n = 207).
Characteristics Frequency (%)
GenderWoman
Man
152 (73.4)
55 (26.6)
EthnicityMalay
Chinese
Indian
Arab
Others
162 (78.3)
22 (10.6)
17 (8.2)
3 (1.4)
3 (1.4)
Academic years2nd year
3rd year
108 (52.2)
99 (42.8)
Family income (RM)<4850
4851–10,970
>10,971
73 (35.3)
87 (42.0)
47 (22.7)
Mode to access online classes (most of use)Wi-Fi
Mobile data
Internet cafe
183 (88.4)
24 (11.6)
0 (0.0)
Current accommodationInside campus
Urban
Rural
199 (96.1)
4 (1.9)
4 (1.9)
Digital toolsLaptop
Mobile phone
Ipad/Tablet
Desktop
130 (62.8)
21 (10.1)
53 (25.6)
3 (1.4)
Table 2. The descriptive statistics of constructs (n = 207).
Table 2. The descriptive statistics of constructs (n = 207).
ConstructRangeOverall Mean (SD) ItemsOverall Mean (SD) Scale
Lighting3–154.26 (0.71)12.78 (2.13)
Noise3–153.52 (0.91)10.19 (2.73)
Temperature3–153.93 (0.80)11.79 (2.41)
Technology6–303.98 (0.58)23.88 (3.47)
Psychological health0–12-4.16 (3.92)
GPA0–4-3.51 (0.32)
Table 3. Summary of item characteristics for LNT (n = 207).
Table 3. Summary of item characteristics for LNT (n = 207).
ItemsMean (SD)Median (IQR)Score, n (%)
12345
Q1: The level of lighting in my study area allows me to see clearly what is around.4.28 (0.87)4.0
(1.0)
1
(0.5)
2
(1.0)
31
(15.0)
78 (37.7)95
(45.9)
Q2: I can control the level of lighting in my study area when taking online classes4.26 (0.98)5.0
(1.0)
4
(1.9)
10
4.8)
24
(11.6)
59 (28.5)110 (53.1)
Q3: The level of lighting (from lamps, computer screen) in my study area allows me to have visual comfort4.25 (0.83)4.0
(1.0)
1
(0.5)
4
(1.9)
34
(16.4)
72 (34.8)96
(46.4)
Q4: I have privacy in my study area when taking classes online3.41 (1.15)4.0
(2.0)
10
(4.8)
27 (13.0)62
(30.0)
52 (25.1)56
(27.1)
Q5: The noise level (coming from devices, people’s talks, external sources) in my study area allows me to concentrate3.46 (1.01)3.0
(1.0)
7
(3.4)
22 (10.6)82
(39.6)
60 (29.0)36
(17.4)
Q6: I can control the noise level in my study area3.53 (1.12)4.0
(1.0)
6
(2.9)
35 (16.9)60
(29.0)
55 (26.6)51
(24.6)
Q7: The temperature in my study area allows me to be comfortable and concentrate3.93 (0.95)4.0
(2.0)
3
(1.4)
13
(6.3)
44
(21.3)
83 (40.1)64
(30.9)
Q8: I can control the temperature in my study area3.79 (1.08)4.0
(2.0)
5
(2.4)
25 (12.1)42
(20.3)
71 (34.3)64
(30.9)
Q9: The air quality in my study area is appropriate4.07 (0.87)4.0
(1.0)
2
(1.0)
6
(2.9)
41
(19.8)
84 (40.6)74
(35.7)
1 = never, 2 = hardly ever, 3 = sometimes, 4 = usually, 5 = always.
Table 4. Summary of item characteristics for technology (n = 207).
Table 4. Summary of item characteristics for technology (n = 207).
ItemsMean (SD)Median (IQR)Score, n (%)
12345
Q10: The instructor’s voice is audible4.02 (0.70)4.0
(1.0)
0.0
(0.0)
4
(1.9)
36
(17.4)
119 (57.5)48
23.2)
Q11: Course content shown or displayed on the smart board is clear4.17 (0.65)4.0
(1.0)
0.0
(0.0)
2
(1.0)
23
(11.1)
119 (57.5)63
(30.4)
Q12: The microphone is in good working condition4.06 (0.76)4.0
(1.0)
0.0
(0.0)
6
(2.9)
3.6
(17.4)
105 (50.7)60
(29.0)
Q13: The video image is clear and comprehensive4.02 (0.71)4.0
(1.0)
0.0
(0.0)
5
(2.4)
35
(16.9)
117 (56.5)50
(24.2)
Q14: Technical problems are not frequent, and they do not adversely affect my understanding of the course3.57 (0.96)4
(1.0)
5
(2.4)
26
(12.6)
51
(24.6)
96 (46.4)29
(14.0)
Q15: The technology used for online teaching is reliable4.04 (0.68)4.0
(1.0)
0.0
(0.0)
4
(1.9)
32
(15.5)
123 (59.4)48
(23.2)
1 = strongly disagree, 2 = disagree, 3 = neither agree nor disagree, 4 = agree, 5 = strongly agree.
Table 5. Summary of items descriptive of psychological health (n = 207).
Table 5. Summary of items descriptive of psychological health (n = 207).
Items0
n (%)
1
n (%)
Q16: Been able to concentrate on what you’re doing?125 (60.4)82 (39.6)
Q17: Lost much sleep over worry?126 (60.9)81 (39.1)
Q18: Felt you were playing a useful part in things?145 (70.0)62 (30.0)
Q19: Felt capable of making decisions about things?163 (78.7)44 (21.3)
Q20: Felt constantly under strain?110 (53.1)97(46.9)
Q21: Felt you couldn’t overcome your difficulties?115 (55.6)92 (44.4)
Q22: Been able to enjoy your normal day-to-day activities?138 (66.7)69 (33.3)
Q23: Been able to face up to your problems?151 (72.9)56 (27.1)
Q24: Been feeling unhappy and depressed?130 (62.8)77 (37.2)
Q25: Been losing confidence in yourself?117 (56.5)90 (43.5)
Q26: Been thinking of yourself as a worthless person?141 (68.1)66 (31.9)
Q27: Been feeling reasonably happy, all things considered161 (77.8)46 (22.2)
0 = Don’t have a problem, 1 = Have a problem.
Table 6. Model fit indices of the initial structural model.
Table 6. Model fit indices of the initial structural model.
Model CFI TLI SRMR RMSEA (90% CI)
Model 10.960.950.050.04 (0.02, 0.06)
CFI = comparative fit index, TLI = Tucker–Lewis index, SRMR = Standardized Root Mean square Residual, RMSEA = Root Mean Square Error of Approximation, CI = Confidence Interval.
Table 7. Hypothesized path relationships in the final model (model-1).
Table 7. Hypothesized path relationships in the final model (model-1).
Hypothesis PathwaysΒ (95% CI)CRp-Value
H1 GPA ß L0.23 (−0.02, 0.49)1.630.076
H2GPA ß N−0.06 (−0.29, 0.17)0.520.599
H3GPA ß T−0.01 (−0.26, 0.25)−0.050.960
H4GPA ß Tch−0.06 (−0.23, 0.12)−0.660.513
H5GPA ß P_H−0.21 (−0.35, −0.06)−2.830.004
H6P_H ß L−0.14 (−0.35, 0.06)−1.340.177
H7P_H ß N−0.18 (−0.41, 0.05)−1.470.137
H8P_H ß T0.04 (−0.19, 0.28)0.390.695
H9P_H ß Tch−0.09 (−0.28, 0.08)−1.020.307
β = regression weights of pathways, CR = Critical Ratio, L = Lighting, N = Noise, T = Temperature, Tch = Technology, P_H = Psychological health.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Taleb, O.K.; Sarimah, A.; Siti-Azrin, A.H.; Baharuddin, K.A.; Abusafia, A.H. Structural Relationships between Environmental Factors, Psychological Health, and Academic Performance in Medical Students Engaged in Online Learning during the COVID-19 Pandemic. Int. J. Environ. Res. Public Health 2023, 20, 1494. https://doi.org/10.3390/ijerph20021494

AMA Style

Taleb OK, Sarimah A, Siti-Azrin AH, Baharuddin KA, Abusafia AH. Structural Relationships between Environmental Factors, Psychological Health, and Academic Performance in Medical Students Engaged in Online Learning during the COVID-19 Pandemic. International Journal of Environmental Research and Public Health. 2023; 20(2):1494. https://doi.org/10.3390/ijerph20021494

Chicago/Turabian Style

Taleb, Ola K., Abdullah Sarimah, Ab Hamid Siti-Azrin, Kamarul Aryffin Baharuddin, and Ali H. Abusafia. 2023. "Structural Relationships between Environmental Factors, Psychological Health, and Academic Performance in Medical Students Engaged in Online Learning during the COVID-19 Pandemic" International Journal of Environmental Research and Public Health 20, no. 2: 1494. https://doi.org/10.3390/ijerph20021494

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop