Effects of COVID-19 on First-Year Undergraduate Research in Physical Geography

Abstract

Having confirmed that including research in first-year undergraduate teaching can actually help students understand the research process, link research with concepts, and improve both their academic and professional skills, we intended to evaluate how this experiential learning component fared during the COVID-19 challenge. For a first-year three-credit physical geography class, we have included a First Year Research Experience (FYRE) project for six iterations. A cluster analysis grouped students’ perceptions obtained from survey questions into five categories, from high to low. The results showed an overall improvement in perception of the FYRE during the pandemic, driven primarily by soft-skill development related to time management and self-motivation. Students were also able to better connect the research project with the theoretical content of the course. Components of the FYRE that suffered during the pandemic include engaging with course instructors and completing the oral presentation phase of the research. Soft-skill development continued through the second year of the pandemic, although students’ dissatisfaction with continued restrictions on in-person contact was evident.

1. Introduction

From previous work, it was confirmed that including research in first-year undergraduate teaching can help students understand the research process, link research with concepts, and improve both their academic and professional skills [1]. However, the efficacy of this teaching approach in the context of a pandemic is not clear. COVID-19 changed all teaching in an unexpected way, and higher education is one of the many fields affected. Normal in-person teaching converted to a remote and/or online modality. Many studies investigated the impacts of the COVID-19 pandemic on higher education [2,3,4]. Others quickly suggested new approaches for adapting to the circumstances to reduce the impacts on student learning [5,6]. These studies describe a wide range of impacts on undergraduate teaching globally and identify challenges and successes in the variety of approaches to teaching during the pandemic. For this work, we aimed to evaluate how students experienced the taught research component during the COVID-19 pandemic of 2020–2022 to understand the robustness of the inclusion of the first-year research experience (FYRE) in undergraduate teaching.

Course-based undergraduate learning experiences (CUREs) embed a research component into the typical post-secondary teaching and learning environment [7,8]. In one approach, students may complete a research project relevant to the course content alongside conventional lecture and laboratory/tutorial assignment course components. CUREs often focus on the research process (e.g., question, investigate, and share) via learning activities that focus on individual aspects of the process. The integration of research and conceptual understanding provided concurrently in a course benefits student learning and the overall course experience [1,9,10], as well as providing longer-lasting benefits as students move beyond their undergraduate education [7,11]. Course-based undergraduate research has also been shown to be more inclusive as all students are encouraged to participate, not just those few who may be selected for a research laboratory position [10,12].

A primary drawback of CUREs is the additional resourcing needed to guide students through the research process [10,13]. This is particularly notable when applied in high-enrollment (>50 students) courses, with each student being guided through a research project by a course instructor, teaching assistant, and/or research coach. Furthermore, developing and maintaining the course from year to year, as well as mentoring students during the course, present additional challenges to this form of pedagogy. The latter is noted as a key variable in student success [13,14].

In March 2020, the academic term was interrupted by the COVID-19 pandemic, resulting in post-secondary institutions transitioning from a traditional in-person classroom to a remote self-directed learning environment [15,16]. The differences between in-person and remote, self-directed learning modes are diverse, and each has its own advantages [17]. The transition from in-person to remote learning imposed by the COVID-19 pandemic highlighted these differences for students and instructors alike, accentuating the gap in technology awareness between the two groups while also jumpstarting the evolution of post-secondary education toward digital learning [18]. Course-based undergraduate research components and their high demand for individual mentorship were likely impacted greatly by such a transition. Therefore, the questions that this study aimed to answer were: (1) How did students perceive the effectiveness of the FYRE before and after the onset of COVID-19? (2) What were the effects of COVID-19 restrictions on student perceptions of learning during the FYRE? (3) Were there any changes in the student perspectives between the 2 years of the COVID-19 pandemic? Based on the challenges in transitioning to and teaching research skills in a virtual/remote way, we hypothesized that COVID-19 had negative impacts on student learning and skill development in the research component of the first-year geography course because we assumed that the FYRE requires intense interactions between students and instructors. Without the in-person guidance, the outcome of the class with the FYRE would be much worse.

2. Materials and Methods

For a first-year one-semester physical geography class (GEOG 120) in the Department of Geography and Planning at the University of Saskatchewan, we have included a First-Year Research Experience (FYRE) project since 2015. In general, FYREs are offered in a wide variety of academic programs at the University, with more than 3000 students enrolled in FYRE courses in 2020–21 [19,20]. Students are guided through a “question–investigate–share” research process (Figure 1). Each course offers the FYRE in a unique way, providing students with different experiences depending on the particular course content. At the conclusion of each course, students are invited to complete a survey questionnaire to evaluate their learning experiences in the FYRE. The survey instrument is administered by the Office of the Vice-President Research at the University, and is deemed exempt from ethics approval by the University Research Ethics Board due to the anonymity of the responses.

The FYRE for GEOG 120 requires each student to identify a specific location (e.g., a city, historic/vacation site, or landform) and develop a physical geography-based research question about that location. As a narrow, mandatory data collection component, students then collect daily data from their chosen location for 6 weeks; the data include maximum/minimum temperatures, sunrise/sunset times, normal maximum/minimum temperatures, and precipitation. In addition, students are encouraged to collect additional data to answer their research question; these data can be longer historical climate archives, local soil classifications, and/or records in the social or economic domains, such as population, crop yield, insurance, etc. Therefore, students would seek diverse data sources with the help of the instructor or research coaches. Research questions are open to anything in the context of physical geography, provided they can be answered with their acquired datasets (e.g., How does sea surface temperature affect precipitation in Guayaquil, Ecuador?) [1]. Each project requires that basic data analysis (e.g., descriptive statistics or correlation/regression analysis) is performed on the acquired data. Lastly, students design a poster to present their project to their classmates, instructors, and other faculty in a public setting. Overall, the FYRE is a required component of the course and accounts for 25% of the final course mark.

To understand the research process, groups of students meet in seminar sessions focused on the formation of a research question/hypothesis/objective, data analysis methods, data visualization, and results sharing/knowledge translation. Although each student is responsible for their own project, they are encouraged to work together to understand the research process. These sessions are led by a “research coach”, often a graduate student familiar with the research process from their own studies. The role of the research coach, with mentorship from the course instructor, is to guide and support students through their projects. During the COVID-19 pandemic, the course was moved to a full distance-learning mode, primarily using Zoom. Although the FYRE model remained unchanged, individual students worked in isolation, communicated with the course instructor and research coaches in a virtual way, and presented their research posters in a virtual forum.

In this study, we included FYRE survey results from six iterations of the course (2015–2016, 2016–2017, 2017–2018, 2018–2019, 2020–2021, and 2021–2022). Among these 6 years, two different survey questionnaires were used, one for the first 3 years (before the pandemic) (

Table 1. Survey questions distributed to students at the conclusion of the FYRE (2015–18).

#	Question
1	I thought the research experience was adequately organized.
2	I understood the purpose of the research experience when it was introduced.
3	I think I have an understanding of research: what it is and how it works.
4	To my knowledge, we investigated using resources relevant to geography.
5	In the research we conducted, I contributed to creating a research question.
6	We shared or disseminated our results with someone other than our instructor.
7	Because of the research experience, I have a better idea about what courses I would like to take (or not to take) in the future.
8	Because of the research experience, I have a better idea about what subject I want to major in (or what subject I know I do not want to major in).
9	I think I have an increased understanding of geography because of the research experience.
10	I think I have an increased understanding of how research works because of the research experience.
11	I believe my research skills have improved as a result of the research experience.
12	I believe my professional skills (i.e., critical thinking, problem-solving, and getting along in groups) have improved as a result of the research experience.
13	I believe my academic skills (i.e., finding and reading journal articles, summarizing research findings) have improved as a result of the research experience.
14	I found the research experience to be engaging (interesting and captivating).
15	The research experience made me more engaged (interested) in class.
16	The research experience made me more engaged in learning about geography.
17	I have a better idea about what I would like to pursue (or not pursue) as a career because of the research experience.
18	Because of the research experience, I have a better idea about my interest (or lack thereof) in graduate studies (i.e., pursuing a master’s degree or PhD).
19	I think I have a better idea about how researchers think and work as a result of the research experience.
20	I feel like I think and work more like a researcher since the research experience.
21	I believe participating in the research experience will contribute to my academic success.
22	I think I have an understanding of the purpose of research on campus.
23	I have discussed the good things about my research experience with other people on or off campus.

) and a different version was used for the second 3 years (1 year before the pandemic and 2 years during the pandemic) (

Table 2. Survey questions distributed to students at the conclusion of the FYRE (2019–22). Questions not related to the specific course FYRE activities are not shown.

#	Question
1	I have a clearer understanding of how a research question is constructed because of the FYRE project.
2	I am better equipped to identify and find good research/scholarly sources because of the FYRE project.
3	I am better equipped to summarize findings from multiple sources to construct a scholarly review because of the FYRE project.
4	I am better equipped to analyze data because of the FYRE project.
5	I am better equipped to interpret results and present evidence because of the FYRE project.
6	I better understand the importance of being a critical consumer of information because of the FYRE project.
7	I have an increased tolerance for obstacles and setbacks (resilience) because of the FYRE project.
8	I had opportunity to give constructive feedback to others during the FYRE project.
9	I asked for and received guidance and feedback (i.e., from my peers, research coach, or professor) during the FYRE project.
10	I practiced and further developed self-management and time management skills because of the FYRE project.
11	I practiced and further developed project management skills (i.e., task breakdown, critical thinking, decision-making, quality management, risk, and reward decisions) because of the FYRE project.
12	Sharing FYRE project findings with people beyond the professor helped me think and speak more clearly about my research.
13	I practiced and further developed writing skills to analyze, understand and explain the FYRE project findings.
14	I practiced and further developed presentation skills (oral, poster, or online) during the FYRE project.
15	I felt more connected to others (such as other students, the research coach, or the professor) in this class because of the FYRE project.
16	I was self-motivated and interested in working on the FYRE project.
17	Because of the FYRE project, I will go into my next course or research project with more confidence.
18	The FYRE project made me more engaged in learning about geography.
19	I have an increased understanding of a university’s role in generating and assessing knowledge because of the FYRE project.
20	I am proud of the work I contributed to this FYRE project.
21	The amount of time devoted to the FYRE project was of value to my overall education goals and skill development.

). In the survey, questions are designed to obtain students’ views about their in-class research experience with a score of 1 (strongly disagree), 2 (disagree), 3 (neither disagree nor agree), 4 (agree), and 5 (strongly agree). The questions cover aspects of how well the research component was organized, how well the research component related to the class concepts, how students’ thinking/research skills might have improved as a result of the research, and how the research might have an effect on students’ future majors and career selection. Non-parametric Wilcoxon rank-sum tests were used to evaluate differences in survey scores between years.

To understand how students viewed the FYRE component of the class, cluster analyses were performed for the two surveys separately. Hierarchical analysis was used first to determine the number of relevant clusters, followed by k-means analysis to establish cluster membership. To evaluate the change in project delivery during the COVID-19 pandemic, survey results for the 2 years (2020–21 and 2021–22) were averaged and then subtracted from the values of the year before COVID-19 (2018–19) to determine any difference (pre-2018 results were omitted from this analysis due to the change in the survey instrument). Hierarchical and k-means cluster analyses were conducted on the difference values to group the results based on gains or losses in the survey scores. A similar approach was used to determine any changes in the student perception of the FYRE during the pandemic (i.e., 2020–21 vs. 2021–22). The within-cluster sum of squares (WCSS) was used to evaluate the robustness of the cluster analysis results. All analyses were completed in R [21,22].

3. Results

3.1. Survey Responses

The survey responses showed a substantial decrease in response rate concurrent with the introduction of the new survey questionnaire (

Table 3. Student enrollment and survey responses per year.

Survey Version	Academic Year	Total Enrollment	Survey Response	Response Rate (%)
2015–18	2015–16	124	77	62.1
	2016–17	121	70	57.9
	2017–18	122	67	54.9
	Average	122	71	58.3
2018–22	2018–19	111	22	19.8
	2020–21	114	38	33.3
	2021–22	120	29	24.2
	Average	115	30	25.8

). The cause of this change is likely the move from a handwritten survey provided in class to an online survey completed outside class time. Response rates increased during the COVID-19 pandemic. Figure 2a shows that the student perception of the FYRE was consistent through the years 2015–16, 2016–17, and 2017–18. Wilcoxon tests indicate that there are no significant differences among these years (2015–16 vs. 2016–17: p = 0.231; 2015–16 vs. 2017–18: p = 0.077; 2016–17 vs. 2017–18: p = 0.380). During this period, the FYRE process was carried out in a consistent manner with the same instructor, although with different research coaches.

In the subsequent 3-year period with the new survey questionnaire (Figure 2b), significant differences exist between the years. The first full COVID-19 academic year (2020–21) resulted in improved student perception of the FYRE over the previous non-COVID-19 offering of the research project (i.e., 2018–19) (p = 0.006). Likewise, year 2 of COVID-19 restrictions on teaching resulted in even more improvement over the pre-COVID-19 offering (p < 0.001). A moderate improvement in student perception of the FYRE from 2020–21 to 2021–22 (i.e., during the COVID-19 pandemic) was also observed (p = 0.030).

3.2. Cluster Analysis

Cluster analysis of the 2015–18 survey data (Figure 3a) yielded five groups of student responses based on the survey score (WCSS = 96.5%). The highest-ranking perceptions of FYRE related to questions 3, 4, and 5, broadly related to understanding the purpose of research in general. The next best ranking related to questions 1, 2, 6, and 22, and indicates that the FYRE was well organized and integrated into the course, as well as providing students with a better picture of the role of research at the University. The third-ranked perceptions were related to questions 9, 10, 11, 14, 19, 21, and 23. These questions evaluate how students gauge their learning of the components of the research process. The fourth-ranked cluster contained questions 12, 13, 15, and 16, all related to transferable skill development and integration with the course subject matter. Lastly, cluster five contained questions 7, 8, 17, 18, and 20. These questions relate to whether students see the FYRE as guiding their future decisions regarding course selection, academic programs, and career aspirations. Summary statistics of all cluster analyses are presented in

Table 4. Summary statistics of the cluster analyses.

Cluster	Survey Questions	$\mathbf{M}\mathbf{e}\mathbf{a}\mathbf{n} \mathbf{S}\mathbf{c}\mathbf{o}\mathbf{r}\mathbf{e}$	Standard Deviation	Median	Range
Old surveys
1	3 (3, 4, 5)	4.21	0.08	4.18	0.15
2	4 (1, 2, 6, 22)	3.87	0.06	3.87	0.12
3	7 (6, 10, 11, 14, 19, 21, 23)	3.57	0.07	3.54	0.21
4	4 (12, 13, 15, 16)	3.30	0.07	3.30	0.15
5	5 (7, 8, 17, 18, 20)	2.96	0.13	2.97	0.29
New surveys
1	12 (1, 2, 3, 4, 5, 6, 11, 14, 17, 19, 20, 21)	3.88	0.07	3.88	0.26
2	7 (7, 9, 10, 12, 13, 16, 18)	3.61	0.08	3.60	0.22
3	2 (8, 15)	2.96	0.12	2.96	0.17
2019–2020 vs. 2020–2022 surveys
1	3 (10, 16, 20)	0.80	0.11	0.76	0.21
2	2 (8, 18)	0.53	0.06	0.53	0.08
3	10 (1, 2, 3, 4, 5, 12, 13, 15, 17, 21)	0.32	0.07	0.34	0.20
4	6 (6, 7, 9, 11, 14, 19)	0.06	0.10	0.05	0.28
2020–2021 vs. 2021–2022 surveys
1	2 (18, 21)	0.46	0.04	0.46	0.05
2	13 (2, 3, 4, 6, 7, 8, 9, 10, 11, 13, 15, 16, 20)	0.21	0.07	0.21	0.20
3	6 (1, 5, 12, 14, 17, 19)	-0.05	0.05	-0.06	0.13

.

Using the new FYRE questionnaire, cluster analysis (Figure 3b) yielded three groups of student responses based on the survey score (WCSS = 93.1%). The highest-ranked perceptions of FYRE related to questions 1–6, 11, 14, 17, and 19–21, which relate to skill acquisition through participation in a guided research project. The second-ranked cluster contained questions 7, 9, 10, 12, 13, 16, and 18. These questions relate to soft-skill development, such as time management, resilience to obstacles, and writing skills. The lowest-ranked cluster contained questions 8 and 15, both of which relate to engaging with other members of the course (instructors and students).

3.3. Pre-COVID-19 vs. COVID-19 Responses

A cluster analysis of the second set of survey scores divided into pre-COVID-19 and during COVID-19 (Figure 4) yielded four clusters of responses (WCSS = 91.2%). The highest-ranked cluster shows increased student perception regarding self-motivation and time-management skills, as well as pride in the completed work (questions 10, 16, and 20). The second highest-ranked cluster relates to student engagement with the subject matter, as well as opportunities to provide feedback to others in the course (questions 8 and 18). The third highest-ranked cluster includes student responses related to understanding the research process, developing soft and transferable skills (e.g., writing and data analysis), and connecting to students and instructors in the course (questions 1–5, 12, 13, 15, 17, and 21). The lowest-rated cluster was composed of questions related to project management, presentation skills, receiving feedback from instructors, and understanding the role that research plays at the University (questions 6, 7, 9, 11, 14, and 19).

Changes in student perception of the FYRE were also observed during the two years of pandemic restrictions on teaching mode (Figure 5). Differences in the per-question survey score from 2020–21 to 2021–22 were divided into three clusters (WCSS = 86.3%). The highest-ranked cluster related to an increased engagement in the subject matter and student valuation of their participation in the FYRE to their overall education (questions 18 and 21). The second-ranked cluster included learning about the research process, time- and self-management skills, as well as self-motivation and connections to others in the course (questions 2–4, 6–11 13, 15, 16, and 20). An average decrease in survey score was observed for survey questions related to understanding the role of research at the University, some specific components of the research process, and presentation skills (questions 1, 5, 12, 14, 17, and 19).

4. Discussion

Although the survey questions themselves changed midway through this study, we have evaluated the annual sequence as a continuum; in general, the essence of the questions is similar across the two survey questionnaires, and both use the same rank scoring system. The most significant change in student perception of the FYRE occurred between 2019 and 2020, and it reflects the impacts of the COVID-19 pandemic and indicates a positive impact.

4.1. Effects of FYRE in the Three Years before COVID-19 (2015–18)

Before changes to the FYRE imposed by the COVID-19 pandemic, students viewed the integration of an individual research project into the course favorably. The most positive responses suggest students gained an appreciation for the role that research plays in understanding the subject of geography. This is an important bridge across the gap between theory and practice in the discipline. Similarly, students gained an understanding of the University as a research institution, something first-generation and/or first-year students often miss. These positive effects might become more pronounced through tracking students’ future academic and career paths and corresponding views over a longer time period.

However, students did not see value in the FYRE as a guide toward future planning (cf. [2]). For example, few students responded that the research experience triggered an interest in post-graduate work or a career as a researcher. While students from all academic years are able to take the GEOG 120 course, enrollment is typically dominated by first-year students. Perhaps it is too much to ask of such young students to use a single research experience to motivate their future goals. Students also struggled to connect the theory presented in course lectures with what was being performed in the research process lessons. Again, for first-year students without a wide breadth of knowledge, it may be difficult to connect a singular research question with the broader concepts presented in the course. This may change as students complete more FYRE and FYRE-like opportunities over time.

4.2. Effects of FYRE on Difference Aspects with Different Survey Questions

It is difficult to compare the students’ views when the survey questions are different. Based on the second set of survey results, cluster analysis revealed that students valued the FYRE experience and the detailed components highly and valued the interactions with others the lowest (Figure 2b, Table 4). This is not surprising because of the COVID-19 transition to remote learning that eliminated in-person contact between students and the instructor. One area of improvement over the initial set of surveys was that students viewed the FYRE as valuable for their future classes and projects, supporting the findings of [2] and [13]. Despite the differences in FYRE delivery before and during the COVID-19 pandemic, students still benefitted from participating in the FYRE as a component of their overall undergraduate education.

4.3. Effects of COVID-19 on the FYRE

By comparing the survey results before and during COVID-19, nearly all aspects indicated improved responses (Table 4). As GEOG 120 is an introductory course aimed at new students without prior university-level learning, the impacts appear less severe than perhaps expected. The greatest improvements were in self- and time-management skills (10), self-motivation (16), and pride in completing their projects (20). The areas that exhibited minimal improvement include resilience (7), guidance and feedback (9), project management skills (11), and presentation skills (14) [23]. Clearly, FYRE components that required guidance and interaction with the instructor or research coaches were strongly negatively affected by COVID-19 because of the online teaching. Conversely, students benefitted from COVID-19-imposed isolation to develop time- and self-management/guidance skills. These are valuable skills in the development of students for their future studies and careers. Though not an intended outcome of the transition to distance learning, students displayed great resilience to the challenges, although perhaps not from their own perspective.

The COVID-19 imposed isolation forced students to rely more heavily on self-directed learning (using resources provided by course instructors, e.g., textbooks and lecture notes) in order to connect geographic concepts with their research topic, resulting in an improved perception of connecting FYRE with the other course materials. Also, and not surprisingly, is the low gain in asking for and receiving feedback related to the development of project management and presentation skills. For students new to university learning, these active learning skills are something many students are hesitant or shy to initiate. A lack of in-person contact with the course instructor or teaching assistants hinders the development of these skills.

The research presentation of the FYRE projects is of central importance as it encompasses the “share” component in the research cycle (Figure 1). Prior to COVID-19, this was conducted in a public venue, and students were given the opportunity to share their research with peers and evaluators, purposefully comparable to the experience of presenting at a research conference. The conversion of this approach to a brief online presentation to a subset of peers and evaluators diminishes the experience of the research-sharing component. Unsurprisingly, the students’ responses to this component of the FYRE project showed no improvement compared to pre-COVID-19 FYRE project presentations.

4.4. Changes during COVID-19

One question we sought to investigate was whether any change occurred between the pandemic years of 2020–21 and 2021–22. Of course, the transition to remote learning impacted instructors as much as students [24], and it is inevitable that lessons would be learned after the initial remote FYRE offering. The results revealed that students were more engaged in the class (18) and valued the FYRE in their education and skill development (21). However, students were frustrated with the specific research question formation (1), interpreting results (5), sharing results with others (12), presentation skills (14), and effects on future course selection and research interests (17) because of the reduced interactions with others. These responses suggest a growing lassitude or dissatisfaction with respect to the continuation of remote delivery of GEOG 120; during the 2021–22 academic year, some University courses were delivered in person, and some students may have had a combination of in-person and remotely delivered courses. A subset of students may have preferred an in-person delivery of the FYRE. Clearly, the instructors and research coaches improved the connection between the FYRE and the overall course content but struggled to engage with students on an individual task level. Again, the presentation component of the FYRE projects illustrates the negative impacts of the COVID-19 pandemic on teaching the research process.

These results show a shift in student experience during the initial transition to distance learning and also during the continued distance learning in year 2 of the pandemic. Unsurprisingly, the practice of self-management and self-motivation were ranked highly as students were largely isolated at home during the first year of the pandemic.

5. Conclusions

The First Year Research Experience (FYRE) provides students with an opportunity to engage in the research process from the beginning of their undergraduate education. Overall, the student response to this opportunity is positive. The FYRE requires more contact between course instructors and research coaches in order to engage students and to guide them through their individual research projects. The transition to distance learning imposed by restrictions put in place during the COVID-19 pandemic proved to be a significant challenge to teaching and learning in the FYRE; however, our hypothesis was rejected because the student responses suggested that the efforts made by students and instructors enhanced their learning experience. The student responses suggested that they gained greatly in their self-motivation and time-management skills, valuable soft skills that will benefit students throughout their careers. Unsurprisingly, the FYRE suffered with respect to contact between students and instructors and the final presentation component during the period of pandemic-related isolation. Our results reveal that including a research component in our first-year geography course is not only improving students’ understanding of concepts related to geographic topics, but it is also enhancing the robustness of the class design for facing challenges, with the COVID-19 pandemic as an example. Challenges remain with respect to communication between instructors/research coaches as well as the presentation component of the FYRE, both of which will be addressed in future iterations of the course. It remains unclear whether it was the integration of the FYRE component into regular course instruction that benefitted students most, particularly during the period of isolation. For example, searching for data/information to support the learning of concepts provides a different learning experience when compared to watching recorded lectures or reading texts. However, this needs to be further investigated to reach a solid conclusion.