Molecular Orbital Theory—Teaching a Difficult Chemistry Topic Using a CSCL Approach in a First-Year University Course

Abstract

Collaboration is regarded as one of the core competences of the 21st century when it comes to complex problem solving. In response to high dropout rates among STEM students, we developed a digital-collaborative intervention on a difficult topic, MO theory, for first-year chemistry students. First, students work independently in a Digital Learning Environment (DLE). Afterwards, they collaborate in small groups and create Concept Maps on MO theory. We evaluate this intervention through knowledge tests, tests of attractiveness, cognitive load, and usability during the DLE and concept mapping process, as well as audio and screen recordings during the collaborative group processes. This paper presents the detailed study design together with results from a first study in January 2021, focusing on the practicability of the intervention and students’ feedback. Overall, each small group succeeded in creating a Concept Map. Students rated all phases of the intervention as attractive, with high usability and low cognitive load, although the interactive videos scored better for attractiveness and usability than the concept mapping process. On that basis, first adjustments for a second cycle of the intervention, which will be conducted in January 2022, were derived.

1. Introduction

Over recent years, collaboration has been widely recognised as one of the core competences within 21st century education at a political [1,2], institutional [3] and scientific [4] level. It is often described as an inherent necessity to solve complex problems (ibid.).

This claim is supported by recent research: The implementation of digital-collaborative learning environments has shown positive impacts on building and improving knowledge, especially regarding highly interconnected scientific topics [5] as students can collaborate and work on the subject actively [6].

At university level, these scientific topics become especially complex, which can lead to complications: Recent reports on tertiary education show large dropout rates among STEM (Science, Technology, Engineering, Mathematics) students at university level [7,8,9,10]. Along with background factors such as living conditions or the institutional environment, the authors of these studies identify high study requirements and performance problems as decisive reasons for dropping out.

With regard to chemistry, the central barriers in the first semester are the chemical basics, e.g., stoichiometric calculations or electrochemistry [11,12], as students enter university with a widely diverse range of prior knowledge [13,14,15]. Conversely, quantum chemistry models of the atom and the atomic bond, such as valence bond or molecular orbital (MO) theory, are particularly challenging for first-year students due to their abstract nature and their demanding mathematical requirements [16,17,18,19,20]. However, these topics are very fundamental to understanding advanced chemistry. Consequently, there is a need for support measures for chemistry students, especially in the first semesters.

We aim to analyse how entry-level chemistry students can be supported in learning about quantum chemical bonding theories through a digital-collaborative approach using the (mobile) Computer-Supported Collaborative Learning ((m)CSCL) framework [21].

Few studies have already integrated digital-collaborative elements into learning processes about quantum theory: In the ReleQuant project, Bungum et al. developed and tested learning resources for an introductory course on quantum physics in Norwegian upper secondary schools, reporting positive effects on students’ understanding of the subject when they were engaged in small group discussions [22]. With regard to the topic of quantum chemistry, Partanen conducted a study on a quantum chemistry and spectroscopy course for Bachelor’s students at a Finnish university [17]. He evaluated an active and student-centred course structure and reported positive learning results when comparing the intervention to regular lectures.

Aside from these studies, research on student understanding and effective teaching methods about quantum chemistry is scarce, especially regarding the MO theory.

In this paper, we present a digital-collaborative intervention as well as selected results from a first implementation cycle. Here, we will investigate the potential of digital devices to optimise group processes and effective collaboration, a subject where recent meta studies indicate research gaps [23,24].

2. Materials and Methods

The intervention is aimed at first-year chemistry students at our university. During their first semester, they visit a lecture about general and inorganic chemistry, which is complemented by recitations.

In this lecture, students are first introduced to central concepts of quantum chemistry in general and MO theory, in particular. Since these concepts are not represented in German school curricula [25,26], it can be assumed that the majority of students entering these lectures do so without much prior knowledge.

At the beginning of the semester, each student was assigned to a recitation group. In these groups, they deepen their understanding of the topics presented in the lecture by working on tasks with a student tutor.

We integrated the intervention into four of these regular two-hour recitation groups. Figure 1 shows a general overview of these four sessions.

Due to the COVID-19 pandemic, courses at our university were held digitally via Zoom in the winter semester 2020/21. Thus, the entire intervention was designed in a way that every student could participate from home with their own technical devices.

2.1. Sessions 1 and 2: Pre-Testing, Team Building, and Interactive Videos (DLE Phase)

The first two sessions, i.e., the first half of the intervention, are visualised in Figure 1. Initially, the students were pre-tested. In addition to the knowledge test, we measured the students’ self-efficacy [27], see also Ref. [28] and the academic self-concept in Ref. [29], as well as Ref. [30] which also play an important role on individual learning success [13,14].

In order to evaluate the increase in knowledge, we developed a test containing a total of 40 questions, of which 7 were open and 33 were closed. The test was applied in a pre-post design at the beginning of the first and the end of the fourth session.

After filling out the test at the beginning of the intervention, students were randomly divided into small groups of three to five (see Figure 2) within their recitation groups, depending on course size. Each small group was then assigned a group supervisor who helped them with technical or methodological problems.

As most students participating in the intervention were first-year students at our university, we assumed that they did not know their fellows, especially against the background of a digital semester. Furthermore, students had previously not collaborated in the recitation groups they were separated into. Given this background, the intervention included multiple team building exercises at the end of the first seminar to improve students’ teamwork and group identity [31].

During the second seminar, students worked on five learning videos in a Digital Learning Environment (DLE). Each of these videos was supplemented by interactive video elements using H5P [32]. The interactive elements consisted of optional text boxes with additional informational resources and mandatory single- or multiple-choice questions. In doing so, active learning was promoted as students could not simply ‘consume’ videos in a passive way. The integration of these interactive video elements has shown positive impacts on students’ motivation and memory in other studies [33].

The interactive videos focused on qualitative quantum physical basics (F1) of the theory as well as practical applications (F2) i.e., the construction and interpretation of so-called MO diagrams to describe bond states between two or more atoms in a chemical compound.

Each recitation group was assigned to one of the intervention groups G1, G2 and G3 (see Figure 3). These three groups were parallelised based on the results of the first knowledge test.

• All students in G1 worked with the same videos containing the same interactive elements, covering both the quantum basics (F1) and the application (F2) of MO theory.
• Students in G2 watched the same videos as students in G1. However, the interactions cover either the quantum basics (F1) or the application (F2) of MO theory.
• Students in G3 worked with two different sets of videos, each with interactive elements. One half focused only on the quantum basics (F1); the other half dealt only with the application (F2) of MO theory.

To increase accessibility, we included technical and methodological assistance in the form of pre-recorded presentations at the beginning of each seminar session and video tutorials for each software the students had to work with. Furthermore, we assigned supervisors to each small group who helped with technical and methodological problems during all phases in all seminar sessions.

The students were required to work on the learning videos individually. This measure was taken to give every student a ‘safe thinking space’ to prepare for the following group phase. After watching the videos individually, the students returned to their respective small groups to prepare for the second half of the intervention. At the end of the second seminar session, they worked together in their groups of three to five to compose, clarify, and discuss a list of about 20 concepts they find essential for MO theory.

2.2. Sessions 3 and 4: Collaborative Concept Mapping (CMP Phase) and Post-Test

In the third and fourth seminar session (see Figure 1), the students worked together in their small groups of three to five to create collaborative Concept Maps using the software CmapTools [34]. Past studies have shown that the method is well suited for processing, structuring, and networking large amounts of information in a CSCL environment. Additionally, it allows students to develop metacognitive competences for their later studies [35,36]. Finally, Concept Maps can be used as an effective diagnostic tool which allows to support students through formative feedback [37].

After an introduction to the method and software, each small group first created a training map under the guidance of their supervisor. For that purpose, every student contributed three distinct items on their desk to a list of concepts which the group then had to map. This served multiple purposes; first, the students got used to the method and software without having to worry about potential content-related barriers; furthermore, they got to talk about themselves (or their room) so that this phase served further team building purposes.

After finishing their training map, each small group created a map about MO theory using the array of concepts they put together in seminar two. At the end of the third seminar, each group had created a first version of their own MO Concept Map.

In between sessions 3 and 4, we wrote each group an individual formative feedback on their training map and the MO map, where we highlighted errors regarding content, terminology, and map design.

In the fourth and final session, students reworked their annotated maps and prepared a presentation for the other students in the recitation group. For this, students were selected at random to present their group’s Concept Map. Afterwards, the intervention was concluded with the final knowledge post-test, where we also measured students’ self-concept for a second time. After the fourth session, students also received summative feedback in form of their test results.

Once per seminar session, the students also assessed the collaborative phases through a questionnaire in which they evaluated both themselves and their group members through a standardised questionnaire.

2.3. Evaluation

In addition to the knowledge test, we evaluated the DLE phase through tests of attractiveness [38], system usability [39], and students’ perceived cognitive load [40] (Mid-Test I in Figure 1).

To evaluate the CMP phase described in Section 2.2, students assessed both themselves and other members of their small groups using an adapted questionnaire [41] (see Figure 1, Mid-Test II). The Concept Mapping process was also evaluated through attractiveness and usability tests (see Figure 1, Mid-Test III).

Furthermore, we asked students what they liked or disliked about the DLE or the CMP phase and where they encountered difficulties. Since CmapTools does not allow more than one student per group to edit the map at the same time, we did not measure the students’ cognitive load in this phase.

However, we recorded the screens and audios of each small group of students during each collaborative phase (see Figure 1, phases CMP I to CMP IV) in the seminar sessions two to four and took screenshots of each Concept Map before and after the rework in session 4.

The final set of variables is represented in

Table 1. General overview of all variables collected throughout the intervention.

Data Type	Variable
Quantitative	Knowledge (closed and open questions); Academic self-concept; Assessment of the DLE (Attractiveness/AT, Cognitive Load/CL, Usability/US, open questions); Assessment of the CMP (AT, US, open questions); Assessment of the collaborative phase (self and other students in small groups).
Qualitative	Concept Maps before and after the rework in session 4 (screenshots); Collaborative processes (audio and screen video).
Control variable	Sociobiographic data; Self-efficacy;
Meta	Time spent on each questionnaire; Views of the DLE videos over time.

;

Table 2. Exemplary test items.

Test	Example Items	Likert Scale from … to …
Academic self-concept	“I consider myself to be…	1: …less intelligent than other students.” 5: … more intelligent than other students.”
Self-efficacy	“I don’t know whether or not I possess the skills required to study chemistry.”	1: Strongly disagree 5: Strongly agree
Attractiveness	“In the last work phase, I understood most of the…”	1: Strongly disagree 6: Strongly agree
Usability	“I felt very confident using the system.”	1: Strongly disagree 5: Strongly agree

shows example items from the questionnaires.

3. Results

The first study cycle was conducted in January 2021 with 139 students in total, N = 124 of which participated in all four sessions and are thus considered for the following evaluations. The students were separated into 10 recitation groups at the beginning of the semester. After the first session, each recitation group was assigned to one of the 3 intervention groups G1 to G3 (cf. Figure 3). As a result, n₁ = 40 students from 3 recitation groups worked on the G1 version of the DLE; n₂ = 44 students from 4 recitation groups worked on version G2, and n₃ = 40 students from 3 recitation groups on version G3.

In this paper, we focus on student feedback on the DLE phase and the CMP phase. We take into account both quantitative data from the questionnaires (attractiveness, AT; system usability, US; and cognitive load, CL) as well as answers to open feedback questions.

Aside from minor technical difficulties experienced by some students, such as bandwidth issues or hardware malfunctions (camera, microphone) both in the individual DLE phase as well as the collaborative CMP phase, each group managed to create a Concept Map following the instructions provided. The majority of maps ranged from 20 to 30 concepts.

3.1. Evaluation of the DLE Phase

As

Table 3. Combined quantitative student feedback on the DLE phase.

Variable	Likert Scale	N	M	SD	α
AT	1 (not attractive) to 6 (attractive)	124	4.90	0.67	0.828
US	1 (low usability) to 5 (high usability)	124	4.48	0.42	0.753
CL	1 (high cognitive load) to 6 (low cognitive load)	124	5.23	0.52	0.852

shows, The DLE was rated overall as attractive with high usability and a low cognitive load. Cronbach’s α was calculated as an estimate of internal test consistency.

Considering the open feedback questions, students found the environment to be well-structured with short and easily understandable explanations, and liked working on the interactive videos for themselves and at their own pace. Many students highlighted the interactive elements which enabled them to control their progress and most importantly kept them involved: Other than watching a regular, non-interactive lecture, there was no risk of them zoning out, as the videos were regularly interrupted by short tasks or questions. Without completing these, students could not continue watching.

On the negative side, some students felt pressured by the limited time frame given for the DLE (45 min combined video length plus 15 min for the interactive elements). Some wished for additional feedback on wrong answers and more explanations regarding technical terms or more difficult concepts, although other students reported that the questions in the DLE were too easy for them.

Altogether, our data on video views also indicate that many students worked on the interactive videos for a second time when they prepared for the exams at the end of the semester.

3.2. Evaluation of the Concept Mapping Process (CMP phase)

Table 4. Combined quantitative student feedback on the CMP phase.

Variable	Likert Scale	N	M	SD	α
AT	1 (not attractive) to 6 (attractive)	124	4.18	1.00	0.887
US	1 (low usability) to 5 (high usability)	124	3.66	0.69	0.861

shows that the students also rated the CMP phase as attractive with good usability, although the ratings are not as positive as in the preceding DLE phase. The internal consistency was again measured with Cronbach’s α.

In comparison, students rated both attractiveness and usability of the DLE phase to be higher than that of the CMP phase, as can be seen in

Table 5. Comparison of quantitative student feedback on DLE and CMP phase.

Variable/Group	N	M (DLE Phase)	M (CMP Phase)	d	p
AT/all	124	4.90	4.18	0.847	<0.001
AT/G1	40	4.84	3.90	1.018	<0.001
AT/G2	44	4.94	4.38	0.803	<0.001
AT/G3	40	4.92	4.23	0.748	<0.001
US/all	124	4.48	3.66	1.441	<0.001
US/G1	40	4.39	3.51	1.462	<0.001
US/G2	44	4.55	3.76	1.537	<0.001
US/G3	40	4.50	3.71	1.355	<0.001

. We observed highly significant effects (p < 0.001) with large effect sizes (Cohen’s d > 0.800) across all three intervention groups G1 to G3 and both dimensions (AT and US) in favour of the DLE phase. There is one single exception: students in group G3 also rated attractiveness of the DLE phase higher than of the CMP phase, but only with a medium effect size.

Possible reasons for the above can be found in the open student feedbacks: many of them reported a shortage of time both when creating and reworking the Concept Maps, as they had to get used to the unknown software, CmapTools.

Some wrote that they found it difficult to find a time balance between completing their map by covering all essential subjects and the discussion required to guarantee content correctness regarding the difficult topic. Some groups also disliked constraints the software put on them, e.g., that only one of them could edit the map at a time.

On the positive side, only few students reported cases of social loafing in their groups during the CMP phase. Most of them enjoyed working together with fellow students, listening to others’ ideas, and discussing concepts. They emphasised the high amount of own activity during the intervention in contrast to the rather passive ‘consumption’ in other seminars and lectures.

Regarding content, many students reported that the method, and especially the feedback, helped them to identify and fill their knowledge gaps. Throughout the collaborative phase, some also felt a sense of security concerning the topics they already understood well.

Finally, our data also indicate small differences between the three intervention groups G1 to G3. Overall, students in G1 who had to watch the longest videos with most interactions gave both the DLE and the CMP phase the lowest ratings across all variables. Students in G2 and G3 scored all variables similarly.

4. Discussion and Outlook

Reflecting the results from the first study from January 2021 described above, we are currently preparing for a next intervention cycle in January 2022. There, we will implement the following three changes:

• We are going to spread the intervention over five instead of four seminar sessions to reduce timing pressure for students. This will also affect the DLE, but primarily the CMP phases, where most students reported that they did not have enough time.
• We are going to modify the two phases based on the students’ feedback:
  • Regarding the DLE phase, we are going to add optional subtitles to all videos to improve accessibility. We are also adding a glossary and further explanations for technical terms as well as a few harder questions to the current set of interactive elements.
  • Concerning the CMP phase, we are going to clear methodological and technological difficulties in advance by expanding the provided explanations for the software.
  • We are also going to add an example map to provide additional clarity and guidance to students in the CMP phase.
• We will employ a short manual with relevant technological soft skills such as turning on the camera, changing the name or sharing the screen in Zoom to reduce technical difficulties on the students’ side.

We are currently evaluating the knowledge tests and developing manuals for the assessment of Concept Maps and the screen and audio recordings from the concept mapping process. As soon as these analyses are finished, we can also compare the effects of the intervention on the three different intervention groups G1 to G3.

Nonetheless, based on our observations and students’ feedback, we can already conclude that the current study design essentially has proven to be practical.