Next Article in Journal
AI-Supported Academic Advising: Exploring ChatGPT’s Current State and Future Potential toward Student Empowerment
Next Article in Special Issue
Exploring Students’ Learning Experience and Engagement in Asynchronous Learning Using the Community of Inquiry Framework through Educational Design Research
Previous Article in Journal
The Fascists Are Coming! Teacher Education for When Right-Wing Activism Micro-Governs Classroom Practice
Previous Article in Special Issue
Citizenship Outcomes and Place-Based Learning Environments in an Integrated Environmental Studies Program
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Integrating PhET Simulations into Elementary Science Education: A Qualitative Analysis

1
Teaching and Learning Program, Faculty of Advanced Studies, Al-Qasemi Academic College, Baka EL-Garbiah 30100, Israel
2
Abdo Salim School, Ibilin (North District) 3001200, Israel
3
Mathematics Department, Al-Qasemi Academic College of Education, Baqa 30100, Israel
4
Science Department, Al-Qasemi Academic College of Education, Baqa 30100, Israel
*
Authors to whom correspondence should be addressed.
Educ. Sci. 2023, 13(9), 884; https://doi.org/10.3390/educsci13090884
Submission received: 4 August 2023 / Revised: 28 August 2023 / Accepted: 28 August 2023 / Published: 31 August 2023
(This article belongs to the Special Issue Effects of Learning Environments on Student Outcomes)

Abstract

:
This research delved into the integration of PhET simulations in elementary science education, specifically aimed at Grade 3 students. The primary objective was to evaluate how the use of these digital simulations influenced students’ conceiving of scientific concepts, focusing on “States of M1atter and Phase Changes” and “Solubility and Saturation”. Employing a qualitative research approach, the study observed 19 students who worked in pairs and trios as they engaged with PhET simulations to explore assigned science topics and address related questions. The observations centered on tracking students’ interactions with simulations and their progression through different knowledge phases. We used deductive and inductive content analysis to analyze the transcripts of the observation. The findings reveal that in the “Remembering” phase, students demonstrated a tendency to relate personal experiences to simulations, underscoring real-life context’s role in learning. The “Understanding” phase highlighted how PhET simulations facilitated deeper comprehension, with students making insightful observations. Additionally, the “Application” phase showcased the effective translation of simulation-derived knowledge into practical scenarios, bridging theoretical and real-world understanding. Students’ use of high-order thinking skills, at the analysis, evaluation, and creative phases, showed that simulations supported Grade 3 students in their learning processes of scientific concepts. The research underscores the efficacy of integrating PhET simulations into elementary science education, enhancing students’ knowledge by promoting active engagement and problem-solving skills. Integrating simulations into teaching methodologies emerges as a promising avenue to nurture scientific expertise and holistic understanding among elementary school students.

1. Introduction

The present research addresses the use of simulations in science learning by elementary school students. Educational researchers call us to use simulations in science teaching and learning in elementary school as this tool would enable students to generate and test their own conjectures of the relations in a particular scientific phenomenon [1]. In addition, educational researchers report that simulation can support elementary school students in their science learning [2,3]. In the present study, we used Physics Education Technology (PhET) in the elementary school science classroom as a tool that Grade 3 students can use to learn scientific topics. The elementary school is located in a small city in Northern Israel. This context (an elementary school in a small Arab city in Northern Israel) continues the few studies reported on the use of simulations in elementary school students’ learning of science concepts and relations [2,3].
The use of simulations to enhance learning is a popular pedagogical technique. First of all, students can experiment with interactive simulations at any time and from any place on Earth where there is an Internet connection. Moreover, with the innovation of personal and classroom computing beginning in the 1990s, technology-enhanced simulations have seen increased use [4,5], and the further development of the web has expanded its global reach [6].
A simulation has two components: (1) a model that corresponds to some version of reality—a subject, system, process, or phenomenon, and (2) an interactive interface through which the user, adopting a real-world role [7], can manipulate the model at will within defined parameters, strategizing, making decisions, observing the consequences of actions, and adjusting knowledge and skills in response [7,8,9]. Designed for educational settings, simulations are typically used to prepare learners to think and act competently in real-world situations [10,11]. It is generally recognized, however, that like educational gaming, they require instructor contextualization and guidance to be successful [12,13,14].
Simulations have many advantages for enhancing learning [15]. For instance, they work well with inquiry-based and problem-solving projects [16,17] and open-ended learning environments [18], and they can simplify difficult, abstract concepts by presenting them in concrete ways [19] that are challenging and engaging and make use of perceptual and spatial abilities and memory in ways that are not possible with text or verbal instruction [16,20]. Learners are also able to manipulate situations, processes, and phenomena that would otherwise be too difficult, too dangerous (e.g., caustic chemicals), or impossible to work with (e.g., electrical currents, cell structure, varying accumulations of greenhouse gases) [12,16,21], and the cost of errors is minimal or nonexistent [8]. Simulations put students in charge of their own learning, allow for the fast manipulation of variables with immediate feedback [16], and lead students to adjust what they know in light of experimental data (i.e., manipulations of the simulation) [12]. Moreover, they are cost-effective, and a number of virtual experiments can be conducted by students in a short amount of time [12]. They can also be adapted for students with special needs, enabling them to participate in mainstream classrooms [16].
Many positive outcomes have been reported for simulations. They are an effective way to enhance cognitive, affective, and behavioral learning and to teach real-world skills [22], and they increase student interest and involvement in learning [13]. After a meta-analysis of 91 studies, Talan [19] concluded that, although the effect is not universal, the use of simulations has a strong, significant effect on students’ academic achievement. In the context of science education, Cayvaz et al. [15] studied the effects of simulation- versus textbook-based instruction on middle school students on the topic of work and energy, focusing on learning achievement, inquiry skills, and attitudes toward science. They reported better scores on the achievement test for lesson content and (the classes’ admittedly overall low levels of) inquiry skills (but not for attitudes toward science). Overall, they concluded that simulation-based instruction was significantly more effective than conventional methods. A 2011 report by the National Academies of Sciences, Engineering, and Medicine in the U.S. stated that gaming and simulations “have potential to advance multiple science learning goals, including motivation to learn science, conceptual understanding of science topics, science process skills, understanding of the nature of science, scientific discourse, and identification with science and science learning”. Thisgaard and Makransky [23] reported that, in keeping with social cognitive career theory, virtual simulations may work to boost students’ inclination to enter a STEM-related career.
There is currently a significant gap in the literature on simulation-based learning. A science mapping of 2812 studies between 1965 and 2018 by Hallinger and Wang [5] showed that, although there is global interest in this pedagogy, the predominance of publications come from authors located in a small number of Anglo-American countries, predominately the U.S., with Africa, Latin America, and Asia contributing only 17% of the entire corpus. Given that context influences the responses of teachers and students to teaching methods [11,24,25], there is therefore a need for investigation in different contexts. Moreover, most of the scholarly activity on simulation is concentrated in the fields of medical and management education. Talan [19] notes that studies of simulation techniques in education consistently target the sciences as well as the health sciences, but only a very limited number address primary (especially preschool) education; therefore, there should be more investigation of the impact of simulations on academic achievement at all levels of education.
The PhET simulations used in this study consist of small modules designed for scientific exploration, each narrowly focused on a single activity/lesson, which can be incorporated into classroom instruction or made available on students’ digital devices. Each simulation features a virtual representation of physical objects, often from the everyday life of students, that can be manipulated by the user (e.g., a seesaw or skateboard on which common objects or people can be loaded) and “disciplinary representations” that link the activity with the discourse of science [26]. For instance, in the simple simulation Build an Atom, the learner can select protons, neutrons, and electrons from dishes, drag them into the nucleus or shells of an atom, and receive feedback (disciplinary representations) in the form of a meter that registers the net charge and readouts that show the name of the element, its position in the periodic table, the mass number, and whether the atom is an ion or not. The PhET project offers over 160 simulations in physics, chemistry, mathematics, earth science, and biology; they are free and open-licensed, can be used online or downloaded for offline use with a minimal amount of preparation by teachers and students, and are designed for all levels of education, primary to postsecondary. The project, housed at the University of Colorado, is widely supported by foundations and commercial, government, and educational organizations, and the simulations are used all over the world [27,28].
The objective of this study is to evaluate the impact of integrating PhET simulations into science lessons at the elementary school level on students’ comprehension.
The research question being investigated is:
In the context of PhET simulations, what is the influence of scientific simulations on students’ understanding?

2. Materials and Methods

2.1. Research Context and Participants

The primary objective of this study was to assess the effectiveness of integrating PhET simulations into elementary school science education. To achieve this goal, we adopted a qualitative analysis approach and collected data through observations of Grade 3 students. These students were engaged in learning two specific science topics with the aid of digital simulations. The first topic pertained to “States of Matter and Phase Changes”, while the second topic focused on “Solubility and Saturation”.
The experiment involved the participation of 19 students in Grade 3 in an elementary school in a small city in Northern Israel. The students were organized into seven groups. These groups consisted of pairs or trios, depending on the students’ preferences. There were two groups comprising pairs of students and five groups with three students each. In the present research, we will report the learning of one group of three students as this group’s learning reflects the other groups’ learning. The experiment simulation lasted for three weeks, during which the students took three science lessons each week.
Utilizing PhET simulations, these student groups immersed themselves in learning the designated science topics and in addressing science-related questions relevant to the two areas of knowledge. For clarity, we will outline the individual objectives of each question, provide a screenshot showcasing the simulation employed for resolving the question, and elucidate the role of the student in interacting with and manipulating the simulation. All the simulations belong to the PhET simulation site (https://phet.colorado.edu/).

2.1.1. Remembering

Goals:
-
The student will develop the capability to articulate the sequence of events occurring during the transformation of a specific substance from one state to another.
-
The student will acquire the knowledge that the application of heat to a particular substance can instigate a change in its state.
-
The student will possess the competence to establish connections between alterations in the state of a substance and observable occurrences in everyday life.
Simulation:
Figure 1 shows how the simulation worked in the remembering phase.
Student Engagement: The student initiated the experiment by subjecting ice, existing in a solid state, to heat. The consequence was the transformation of ice into liquid water, representing the liquid state. The student then persisted in the heating process, resulting in the conversion of liquid water into water vapor, manifesting the gaseous state. Throughout this progression, the student keenly observed alterations in the substance’s physical form and the distribution pattern of its constituent particles within each of the three distinct states.

2.1.2. Understanding

Goals:
-
The student will attain an understanding of the mechanism involved in the dissolution of salt within water.
-
The student will grasp the concept of saturation as it pertains to the point at which salt can no longer dissolve effectively in water.
Simulation:
Figure 2 shows how the simulation worked in the understanding phase.
Student Involvement: The student systematically poured salt into a beaker already holding water, initiating a dissolution process. As the student continued this addition, the salt seamlessly dissolved within the water. However, a pivotal moment arrived when the salt’s dissolution ceased, and it commenced sinking to the beaker’s base. Through this engagement, the student attentively observed the unfolding dynamics of salt dissolution, astutely identifying the point where an oversaturation of salt prompted its descent, thereby defining the concept of “saturation”.

2.1.3. Applying

Goals:
The student should be capable of employing the comprehension they gained regarding solutions to effectively carry out the separation of both the solute and the solvent in a given scenario.
Simulation:
Figure 3 shows how the simulation worked in the applying phase.
Student Engagement: The student undertook the task of crafting a salt solution through the introduction of salt into a beaker already containing water. Subsequently, the student initiated the process of heating the solution. As heat was applied, the water within the solution underwent evaporation, resulting in the salt solidifying and settling at the beaker’s base. Throughout this sequence, the student astutely noted the gradual reduction in the solution’s volume as the heating continued. Ultimately, as the heating procedure concluded, all the water evaporated, leaving behind the salt in solid form at the bottom of the beaker.

2.1.4. Analyzing

Goals:
The student should have the capacity to scrutinize the microscopic composition of a given substance in its three distinct states. This involves conducting a thorough comparison among these states, aiming to ascertain whether they accurately depict the three distinct phases of the substance.
Simulation:
Figure 4 shows how the simulation worked in the analysis phase.
Student Involvement: The student initiated the process by selecting a solid state of water (ice) and then proceeded to apply heat, causing the ice to transform into liquid water. The student persisted in heating the liquid water until it reached a point of evaporation, transitioning into gaseous vapor. Throughout this sequence, the student keenly perceived the altering arrangement of water particles during the progression from a solid state to a liquid state, and subsequently to a gaseous state, brought about by the influence of heat.

2.1.5. Evaluation

Goals:
-
By the end of this process, the student should have the capability to design and suggest an experimental procedure aimed at demonstrating the correlation between the quantity of solvent and solute within a specified solution.
-
Furthermore, the student should possess the competence to assess the effectiveness of the experiment they devised and executed, determining whether it can yield a precise and reliable outcome regarding the interplay between the amounts of solvent and solute.
Simulation:
Figure 5 shows how the simulation worked in the evaluation phase.
Student’s Involvement: Students added salt gradually into the water, witnessing its dissolution. Eventually, there came a juncture when the salt particles began descending within the water. In response, the student introduced additional water, causing the sunken salt to re-dissolve. During this process, the student astutely noted that the added salt initially dissolved, followed by a phase of sinking. This precipitated salt later reverted to a dissolved state upon water addition. Furthermore, the student discerned a correlation between the concentration of dissolved salt and the quantity of salt initially dissolved, observing that the concentration increased progressively until reaching a saturation point.

2.1.6. Creation

Goal:
-
The student will be able to illustrate the distribution of the salt particles within the water particles in solutions with a variety of salt concentrations.
-
The student role: The student drew two illustrations showing the distribution of salt particles within water particles in two solutions, one diluted and the other saturated.

2.2. Data Collecting and Analysis Tools

Data collection was carried out through observational methods, wherein we recorded the learning activities of each group using video recordings. Our aim was to capture the interactions among students, as these exchanges provided insights into the specific stage of knowledge engagement within each group.
Subsequently, a combination of deductive and inductive content analysis was employed to scrutinize the transcripts of the students’ learning videos. In the context of deductive content analysis, we employed the newly developed Bloom’s knowledge taxonomy, encompassing distinct phases: remembering, understanding, application, analysis, evaluation, and creation. Table 1 outlines the thematic elements that facilitated the linkage between the unit of analysis (which was the sentence) and each corresponding phase of the taxonomy.
Table 2 describes an example of the analysis conducted in the frame of the analysis of the transcripts at the coding stage.
Taking advantage of the deductive and inductive content analysis methodologies, we followed different studies in the literature, such as Daher [25].
In the previous example on data analysis, we depended on the deductive content analysis when we linked the sentence “it reminded me” with the remembering process of the learner and thus with the category of remembering in Bloom’s taxonomy. We depended on inductive content analysis to verify the conditions or properties of the remembering, here the remembering of a real-life incident.

2.3. Validity and Reliability of the Analysis

The validity of the qualitative research analysis processes stemmed from the analysis method that ensured theoretical saturation [29]. This theoretical saturation stemmed from the existence of no new category type that emerged from the given data. In addition, it stemmed from the coder’s agreement regarding the categories and themes emerging from the units of analysis of the same data. According to Lincoln and Guba [30], validity guarantees reliability, so satisfying validity also ensures reliability, which means that theoretical saturation confers validity and reliability to the research procedure. Two experienced coders (two of the authors) coded the transcripts, where the agreement between them was 0.947, which is accepted for ensuring validity of the analysis, and thus its reliability.

3. Results

The primary objective of this research was to authenticate the learning processes of elementary school students when exposed to digital simulations, with a particular focus on the PhET platform. Herein, we outline and expound upon these processes according to Bloom’s taxonomy framework.

3.1. Remembering Phase

3.1.1. Description and Transcript

During this stage, the students responded to the inquiry: “How do you transition a substance from one state to another?” Prompted by discussions on the transformation of matter, the students drew upon their past experiments and relevant knowledge to elucidate their observations in the simulation regarding the alteration of matter from one state to another. Transcript 1 provides an account of the students’ learning during this phase.
5
Teacher: What will we do to convert a substance from the solid state to the liquid state?
6
Student 1: We are in the solid state, and I remember when we conducted the melting experiment in the laboratory. We lit a candle under the beaker containing ice. Now, I will do the same thing; I will ignite the fire under the beaker. Look how the particles move away from each other, and the substance takes the shape of the beaker. This was not visible in the previous experiment; it’s wonderful. I want to heat the substance a little, not too much, because if the heating process continues for a long time, the particles will move far apart and scatter. So, for now, this is sufficient; I will reduce the fire, as the substance has taken the shape of the container (See Figure 1a above).
7
Student 2: And now we have the substance in the liquid state. I will ignite the fire again to convert the substance into the gaseous state (See Figure 1b above). It reminded me of an incident with my mother where she forgot the water on the stove, and when I returned, there was no water left. Hahaha, I will go and tell my mother that she should not leave the water on the stove for a long time because it will evaporate if it continues to boil for a long time, and there will be no water left for tea.
8
Student 3: I remember when we learned about this topic in class, and you gave us an example of making an ice cream and how we can convert it from the liquid state to the solid state. Now, I want to do the same thing; I will place the ice under the container for it to solidify, and the particles come closer to each other (See Figure 1c above).
9
Student 2: We can also create any shape we want using liquid dough. We made ice cream in different shapes like squares and circles when we learned about the topic, using suitable molds.
Transcript 1: Students’ learning at the remembering phase.

3.1.2. Analysis of Students’ Learning at the Remembering Phase

During their interaction, the students established correlations between their observations in the simulation and real-life scenarios where they encountered outcomes without delving into the underlying process of scientific transformation. Student 2 remembered an incident involving her mother, specifically recalling the consequences of boiling water on the stove until it evaporated. This memory prompted her to caution her mother against leaving water unattended on the stove for an extended period [R7]. Moreover, Student 3 harked back to the experience of creating ice cream through an experiment, wherein the particles underwent solidification and drew nearer to each other [R8]. Regarding this, Student 2 also introduced the notion of manipulating the substance into various shapes by utilizing appropriate molds during the freezing of the liquid mixture [R9]. We can say that the PhET simulation enabled the students to connect with their real-life experiences and thus be able to move forward toward understanding the scientific phenomenon.

3.2. Understanding Phase

3.2.1. Description and Transcript

The next question was: “Place a quantity of salt in a water container. What happened?” Based on the students’ responses, as in Transcript 2, this question required an understanding of the process of dissolution and saturation of solutions.
12
Teacher: What happens when you place a quantity of salt in a water container?
13
Student 1: When I added some salt, it dissolved in the water, but when I added a lot of salt, the word “saturation” appeared, and some salt settled at the bottom of the container (See Figure 2 above).
14
Students: (The other students started pressing the saltshaker to release the salt, and after adding a large amount of salt, they observed the word “saturation”).
15
Student 2: This is indeed what happens because we added a large amount of salt that the water could no longer dissolve, so the salt precipitated at the bottom of the container.
Transcript 2: Students’ learning at the understanding phase.

3.2.2. Analysis of Students’ Learning at the Understanding Phase

Addressing the inquiry, students comprehended various facets associated with the dissolution of salt in water. Initially, they deduced that when a small amount of salt is introduced, it dissolves within the water [R13]. This signifies their grasp of the primary dissolution process wherein salt particles vanish within the aqueous solution. Furthermore, the students made an observation that, contrary to dissolution, the salt began to precipitate and accumulate at the container’s base [R13]. Additionally, the students recognized that upon introducing a substantial quantity of salt, the water loses its capacity to accommodate and dissolve the salt, resulting in salt deposition [R15]. All the students’ deductions during the understanding phase were due to their manipulation of the PhET simulation and observing the resulting changes in the scientific phenomenon.

3.3. Application Phase

3.3.1. Description and Transcript

The next question posed was: “How do we separate salt from water?” This question necessitated an explanation of the process for separating salt from water, where this explanation depended on the application of the students’ new knowledge. Transcript 3 describes students’ learning at the application phase.
16
Teacher: Can you describe how to separate salt from water?
17
Student 1: If we ignite the fire, the water will evaporate and only the salt will remain.
18
Student 2: We evaporate the water to separate it from the salt.
19
Student 3: During the experiment that we performed, we evaporated the water, causing it to turn into a gaseous state and disperse. As a result, only the salt remained. It became evident that the water evaporated while the salt was left behind (See Figure 3 above).
Transcript 3: Students’ learning at the application phase.

3.3.2. Analysis of Students’ Learning at the Application Phase

The students displayed their grasp of the water evaporation process and its impact on salt, revealing an understanding that heating leads to the evaporation and dispersion of water while salt remains behind. Student 1 emphasized the role of ignition in the evaporation process (R17), Student 2 highlighted the concept of water evaporation (R18), and Student 3 remembered the gaseous dispersion of water and the retention of salt while experimenting with PhET simulation in the understanding phase (R19). Student 3 further expounded on the results of heating the solution, applying the recently acquired knowledge. Here, the students applied the knowledge conceived using the PhET simulation in the understanding phase.

3.4. Analysis Phase

3.4.1. Description and Transcript

The subsequent query presented was: “Is it possible for a single substance to exist in three distinct states?” Transcript 4 provides insight into how the students responded to this question.
19
Teacher: Can the same substance exist in three states? Please provide an example.
20
Student 3: Yes, a substance that can exist in three states is water. It takes the form of solid as ice, then transforms into a liquid as water, and changes into a gas as vapor.
21
Student 1: (Ignited the fire under the ice till it melted into water. She kept heating the water until it evaporated (See Figure 4 above)).
22
Student 3: If we examine the three microscopic levels of water and compare them, we can categorize them as three states of the same substance.
Transcript 4: Students’ learning at the analysis phase.

3.4.2. Analysis of Students’ Learning at the Analysis Phase

The students showcased their ability to scrutinize the substance’s behavior and discern its various states (R19–R22). PhET played a crucial role in facilitating their analytical process, allowing them to observe the three states of water at the microscopic level. This analytical approach is reflected in how the students employed terms such as “examine”, “compare” and “categorize” to describe the processes that occurred through the PhET simulation (R22).

3.5. Evaluation Phase

3.5.1. Description and Transcript

The students’ assignment involved crafting an experiment aimed at elucidating the connection between the quantity of solvent and solute. Episode 5 captures the students’ progress during the evaluation phase of their learning journey.
27
Teacher: Now, I would like you to plan an experiment that demonstrates the relationship between the volume of the solvent and the quantity of the solute. (The students restarted the simulation by pressing the reset button).
28
Student 1: Currently, we only have water as the solvent. We will add a small amount of salt and observe its dissolution. (The students observed that the salt particles separated from each other).
29
Student 3: Let’s continue adding salt until the word “saturation” appears on the screen. Look, the word “saturation” has appeared, indicating that the water is saturated. If we add more salt now, it will precipitate to the bottom. (The students observed that the salt no longer dissolves in water due to reaching the saturation point (see Figure 5a above)).
30
Student 2: What happens when we add water to the precipitated salt? (The students added water to the container).
31
Student 1: Oh, the salt particles have separated again because we added water, and the salt dissolved in the added water (see Figure 5b above).
32
Teacher: So, what is the relationship between the solvent and the solute?
33
Student 3: Whenever the quantity of the solute (salt) increases, we must add more solvent (water). We also observed that the ion concentration of the salt increased when it dissolved, reaching a value of 271 (see Figure 5b above). On the other hand, when a portion of the salt precipitated, the ion concentration was 180. This indicates that the quantity of dissolved salt increases when water is added.
34
Teacher: Based on the experiment you performed in the simulation system, is it possible to accurately determine the relationship between the amount of solvent and the amount of solute?
35
Student 1: In the experiments we have carried out so far in the laboratory, we have come to the point where it is impossible to make a conclusion based on the results of one experiment. Therefore, we must have more measurements related to the volume of the solvent and the amount of the solute.
36
Teacher: What additional measurements are we required to carry out in order to accurately determine the relationship?
37
Student 2: We need to perform a number of experiments in which we maintain a constant volume of solvent (water) and add a different amount of salt each time and see if the salt dissolves or not. We then can observe when the salt begins to sink to the bottom of the container.
Transcript 5: Students’ learning at the evaluation phase.

3.5.2. Analysis of Students’ Learning at the Evaluation Phase

The query presented to the students entailed devising an experiment aimed at showcasing the correlation between the solvent’s volume and the amount of the solute (R27). The students proceeded to chart out and execute an experiment that effectively depicted the interplay between the volume of the solvent and the quantity of the solute (R28–R31). Upon conducting the experiment, the students arrived at a conclusive understanding of the relationship (R31). Subsequently, the instructor prompted the students to assess whether the outcomes of their experiment alone could ensure a precise relationship (R32). In response, the students proposed a method that, once enacted, would yield a reliable assessment of the relationship (R28–R37). Here, the PhET simulation served the students only in part of their evaluation, but it was behind their design and evaluation processes.

3.6. Creation Phase

3.6.1. Description and Transcript

The inquiry posed was as follows: Salt was introduced into a pair of glass containers filled with water. A single spoon of salt found its way into the initial container, while the second container received three spoons of salt. In the former container, all the salt underwent dissolution within the water, whereas in the latter container, a portion of the salt settled at the container’s base. Envisioning the presence of enchanted spectacles that grant you the ability to visualize the dissolved salt particles, create drawings depicting the arrangement of dissolved salt particles within each of the containers. Subsequently, elucidate the details and significance behind each of the illustrations you have rendered. Transcript 6 describes students’ learning at this phase.
38
Teacher: Please discuss the question and draw the required distribution.
39
Students: (The students in the group discussed the questions and each student drew a drawing depicting the containers).
40
Student 3: In the first container, the salt particles will be homogeneously distributed throughout the solvent particles, while in the second container, more salt particles will be in the cold lower part.
Education 13 00884 i001
41
Student 2: In both containers, the salt particles will be distributed fairly in the water, except that in the second container, there will be more salt particles in the water since we added more salt to the container.
Education 13 00884 i002
Transcript 6: Students’ learning at the creation phase.

3.6.2. Analysis of Students’ Learning at the Creation Phase

Within this task, the student’s objective was to integrate various components and synthesize them into a novel creation. Student 3 fashioned an illustration featuring concentrated salt in the lower section, whereas Student 2 crafted an illustration depicting salt particles uniformly dispersed throughout the entire solvent. Through her artwork, this student effectively contrasted the two containers, enabling her to aptly depict the precise scientific scenario. Here, the students utilized all that they learned using the PhET simulation to perform the creation processes of the scientific phenomenon.

4. Discussion

Educational researchers are interested in the different aspects of students’ learning in specific educational contexts [31,32,33,34,35], especially in digital contexts [36,37]. The present research intended to study students’ knowledge phases of Grade 3 students’ scientific concepts in the context of PhET simulations. Below, we will delve into students’ acquisition of scientific concepts and relationships through digital simulations.

4.1. Remembering Knowledge

Remembering occurred in two distinct contexts. The initial context is the domestic context, where Student 2 remembered an incident at his home and connected it with the experiment performed by the students. This act of remembering underscores the significance of home experiences in students’ learning. Educational researchers emphasized the positive and negative roles that home experiences play in classroom learning [38]. The second context pertains to prior experiences with the PhET simulations. This remembering of previous experiences with technological tools underscores the positive impact that engaging with technology-driven experiments can have on future learning experiences [39]. In both of these contexts, the work with simulations was behind students’ remembering of previous events and processes, which supported students’ understanding of the new scientific concepts and relations.

4.2. Understanding Knowledge

Through the utilization of a lifelike simulation illustrating salt dissolution in water, the students meticulously observed the complete dissolution of salt within the water, with dispersion occurring throughout the entire volume rather than accumulating solely at the bottom. This insightful observation was facilitated by the students’ engagement with digital simulations. Mengistu and Kahsay [40] reported that computer simulation used as teaching aids enhanced students’ understanding of electric field and electric force concepts. Additionally, Nafidi et al. [41] found the proper integration of a simulation of relative chronology into students’ training can augment their learning. Therefore, research, especially the present study, corroborates the assertion that simulations, particularly digital ones, enhance students’ grasp of scientific concepts and relationships. All the previous indicates that digital simulations influence positively elementary students’ learning of science.

4.3. Application Knowledge

The application followed students’ comprehension of scientific concepts and relationships. An understanding was achieved with the aid of simulations, while the application did not necessarily demand simulation usage; however, students relied on the knowledge they gained through simulations. Previous research has indicated that the application phase’s benefits through technology can vary. According to the study by Daher and Sleem [42], the conventional group outperformed the technology-based (360-degree video) group in terms of application knowledge. The present research results indicate that digital simulations support students’ performance and knowledge during the application phase, which indicates that digital simulations could positively influence students’ learning during the application phase.

4.4. Analysis Knowledge

During the analysis phase, PhET enabled students to scrutinize and compare the three states of water at the microscopic level, enabling them to analyze the substance’s behavior and distinguish its various forms. Much like in the understanding phase, the PhET simulation played a pivotal role in their analysis processes, involving examination, comparison, and categorization. This encouragement of analytical processes was attributed to PhET’s visualization capabilities, which allowed students to compare the observed water states, particularly at the microscopic level. This facilitated their usage of precise scientific terminology related to the water’s three states. This aligns with researchers who have highlighted simulations’ role in facilitating students’ analytical processes. Daher and Baya’a [43] elucidated how mobile simulations empowered students to analyze mathematical phenomena through authentic activities both inside and outside the classroom. Thus, the current research, in addition to previous research, indicates the positive influence of digital simulation on students’ learning at the analysis phase.

4.5. Evaluation Knowledge

In the evaluation phase, students were tasked with designing an experiment that illustrates the connection between solvent volume and solute quantity. When the students proposed such a plan, the teacher prompted them to assess whether their experiment alone could ensure an accurate relationship, leading the students to devise a procedure for this purpose. In this phase, the students employed the simulation in the initial part but not the latter part. Thus, technology was only employed in a portion of the evaluation phase, where we addressed the student’s ownership of the evaluation process rather than the teacher’s use of a tool to gauge the student’s comprehension. Here, it can be argued that the teacher asked the students to conduct a qualitative evaluation, in which the “quality” of the analysis is determined [44]. In this context, the students improved the quality of their earlier analysis pertaining to the relationship between solvent volume and solute quantity. The previous argument indicates that digital influences, even though used only in part of the learning process, can positively influence students’ learning at the evaluation phase.

4.6. The Creation Phase

In this stage, students were challenged to synthesize various elements and employ them to depict a scientific phenomenon visually. The students harnessed all the knowledge accumulated in prior phases to deliberate on the phenomenon and subsequently create a relevant image. Despite the collaborative discourse, the students produced distinct images, influenced by the knowledge garnered in previous phases. Here, the student’s individual attributes impacted the outcome of their learning, including their past aptitude in science. Researchers have asserted that personal traits influence current student learning. Daher and Shahbari [45] reported that students’ characteristics shape their learning experiences in the virtual classroom. Thus, the present research underscores the impact of these personal attributes. In addition to the above, here, the participating students made effective use of all that they conceived using the PhET simulation to carry out the creation processes of the scientific phenomenon. Thus, though not utilized directly, the digital simulation positively influenced students’ learning at the creation phase.

5. Conclusions and Future Practice

The outcomes of the research suggest that students have effectively grasped the process of dissolution and the concept of saturation point. This understanding is evident from their remarks, dialogues, and observations.
The students’ enthusiasm for utilizing simulations to illustrate and clarify real-world material transformation instances highlights their capacity to establish connections between the scientific phenomena they observe in the simulations and their prior experiences with scientific experiments. Furthermore, they draw upon instances from everyday life, which enriches their comprehension of scientific concepts pertaining to changes in matter states.
By incorporating simulations into teaching methodology, students obtained direct and interactive opportunities to witness phenomena and alterations at both macroscopic and microscopic scales. This approach effectively nurtures scientific thinking and inference through hands-on experiments. It empowers students to visualize and delve deeper into transformation processes, identifying underlying causes, and comprehending the associated scientific explanations. As a result, the incorporation of simulations in teaching promotes active learning and facilitates a profound grasp of scientific concepts, empowering students to apply their knowledge in real-life scenarios. Additionally, it cultivates problem-solving skills and the development of scientific expertise. In light of the previous evidence of the benefits of simulations for elementary school students’ science learning, it is logical to conclude that science teachers and learners in elementary school would benefit from using simulations in the classroom.

6. Limitations and Recommendations for Future Research

The present research examined the knowledge processes of Grade 3 students when learning science in the context of PhET simulations. The simulations influenced positively the students’ knowledge processes in terms of Bloom’s knowledge processes. One limitation of the present research is the small number of students who participated in the study, so future research is requested to affirm the results of the present research, especially among Grade 3 and elementary school students. The use of PhET simulations in the science learning of elementary school students is still new, which emphasizes that the current research should be furthered with future studies in all elementary grades, as recommended previously.
In addition to the above, the present research followed qualitative methodology to study the knowledge processes of elementary school students. Future research that follows the quantitative methodology is needed to triangulate the present results. We are aware of the difficulties of carrying out quantitative research among Grade 3 students, but this could be conducted with students in the late elementary school grades [46]. Wigfield [46] gives recommendations for researchers who come to use questionnaires in elementary school. Moreover, questionnaires are suggested for the early grades of elementary school students [47].
Students’ knowledge has little been studied using Bloom’s processes based on observation, where previously some attempts were made to do so through interviews [42]. This study attempts to accomplish that through observations. To advance our current effort to analyze knowledge processes based on observation data by using Bloom’s taxonomy as a theoretical framework, further research is needed.

Author Contributions

Conceptualization, B.R. and N.I.; methodology, B.R. and W.D.; software, B.R.; formal analysis, B.R., W.D., H.D. and N.I.; data curation, B.R.; writing—original draft preparation, B.R., W.D., H.D. and N.I. All authors have read and agreed to the published version of the manuscript.

Funding

This research was conducted without external funding.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Review Board of Al-Qasemi Academic College of Education (protocol code QSM 12-2023 and date of approval: 24 May 2023).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Not applicable.

Acknowledgments

This research received partial support from Al-Qasemi Research Authority. We wholeheartedly express our gratitude to Anwar Rayan for his invaluable insights provided throughout the course of this research, as well as for his thoughtful comments on the manuscript.

Conflicts of Interest

We declare no conflicts of interest.

References

  1. Brigas, C.J. Modeling and simulation in an educational context: Teaching and learning sciences. Res. Soc. Sci. Technol. 2019, 4, 1–12. [Google Scholar] [CrossRef]
  2. Lee, I.; Martin, F.; Apone, K. Integrating computational thinking across the K--8 curriculum. ACM Inroads 2014, 5, 64–71. [Google Scholar] [CrossRef]
  3. Shamir, G.; Tsybulsky, D.; Levin, I. Introducing Computational Thinking Practices in Learning Science of Elementary Schools. In Proceedings of the InSITE 2019: Informing Science+ IT Education Conferences, Jerusalem, Israel, 30 June–4 July 2019; pp. 187–205. [Google Scholar]
  4. Leutner, D. Guided discovery learning with computer-based simulation games: Effects of adaptive and non-adaptive instructional support. Learn. Instr. 1993, 3, 113–132. [Google Scholar] [CrossRef]
  5. Hallinger, P.; Wang, R. The Evolution of Simulation-Based Learning Across the Disciplines, 1965–2018: A Science Map of the Literature. Simul. Gaming 2019, 51, 32–39. [Google Scholar] [CrossRef]
  6. Jong, T.D.; Lazonder, A.W.; Pedaste, M.; Zacharia, Z.C. Simulations, Games, and Modeling Tools for Learning; Routledge: London, UK, 2018. [Google Scholar]
  7. de Jong, T.; van Joolingen, W. Model-facilitated learning. In Handbook of Research on Educational Communication and Technology, 3rd ed.; Spector, J.M., Merrill, M.D., Elen, J., Bishop, M.J., Eds.; Lawrence Erlbaum: Mahwah, NJ, USA, 2008; pp. 457–468. [Google Scholar]
  8. Crookall, D.; Saunders, D. Towards an integration of communication and simulation. In Communication and Simulation: From Two fields to One Theme; Crookall, D., Saunders, D., Eds.; Avon: London, UK; Multilingual Matters: Bristol, UK, 1989; pp. 3–19. [Google Scholar]
  9. Maran, N.J.; Glavin, R.J. Low- to high-fidelity simulation—A continuum of medical education? Med. Educ. 2003, 37 (Suppl. S1), 22–28. [Google Scholar] [CrossRef] [PubMed]
  10. Issenberg, S.B.; McGaghie, W.C.; Hart, I.R.; Mayer, J.W.; Felner, J.M.; Petrusa, E.R.; Waugh, R.A.; Brown, D.D.; Safford, R.R.; Gessner, I.H.; et al. Simulation technology for health care professional skills training and assessment. JAMA 1999, 282, 861–866. [Google Scholar] [CrossRef]
  11. Lu, J.-F.; Hallinger, P.; Showanasai, P. Simulation-based learning in management education: A longitudinal quasi-experimental evaluation of instructional effectiveness. J. Manag. Dev. 2014, 33, 218–244. [Google Scholar] [CrossRef]
  12. Jong, T.; Linn, M.; Zacharia, Z. Physical and Virtual Laboratories in Science and Engineering Education. Science 2013, 340, 305–308. [Google Scholar] [CrossRef]
  13. Keys, J.B.; Wolfe, J. The Role of Management Games and Simulation in Education and Research. J. Manag. 1990, 16, 307–336. [Google Scholar] [CrossRef]
  14. Garris, R.; Ahlers, R.; Driskell, J. Games, Motivation, and Learning: A Research and Practice Model; SAGE Publications: Singapore, 2017; pp. 475–501. [Google Scholar]
  15. Cayvaz, A.; Akcay, H.; Kapici, H.O. Comparison of simulation-based and textbook-based instructions on middle school students’ achievement, inquiry skills and attitude. Int. J. Educ. Math. Sci. Technol. 2020, 8, 34–43. [Google Scholar] [CrossRef]
  16. Learning Science through Computer Games and Simulations; The National Academies Press: Washington, DC, USA, 2011.
  17. Tsai, F.-H.; Hsu, I.Y. Exploring The Effects of Guidance in a Computer Detective Game for Science Education. J. Balt. Sci. Educ. 2020, 19, 647–658. [Google Scholar] [CrossRef]
  18. Land, S.; Hannafin, M. A conceptual framework for the development of theories-in-action with open learning environments. Educ. Technol. Res. Dev. 1996, 44, 37–53. [Google Scholar] [CrossRef]
  19. Talan, T. The effect of simulation technique on academic achievement: A meta-analysis study. Int. J. Technol. Educ. Sci. 2021, 5, 17–36. [Google Scholar] [CrossRef]
  20. Lindgren, R.; Schwartz, D. Spatial Learning and Computer Simulations in Science. Int. J. Sci. Educ. 2009, 31, 419–438. [Google Scholar] [CrossRef]
  21. Clark, D.; Nelson, B.; Sengupta, P.; D’Angelo, C. Rethinking Science Learning through Digital Games and 1 Simulations: Genres, Examples, and Evidence; ResearchGate: Berlin, Germany, 2009. [Google Scholar]
  22. Faria, A. The Changing Nature of Business Simulation/Gaming Research: A Brief History. Simul. Gaming 2001, 32, 97–110. [Google Scholar] [CrossRef]
  23. Thisgaard, M.W.; Makransky, G. Virtual Learning Simulations in High School: Effects on Cognitive and Non-cognitive Outcomes and Implications on the Development of STEM Academic and Career Choice. Front. Psychol. 2017, 8, 805. [Google Scholar] [CrossRef]
  24. Kember, D. Misconceptions about the Learning Approaches, Motivation and Study Practices of Asian Students. High. Educ. 2000, 40, 99–121. [Google Scholar] [CrossRef]
  25. Collins, A.; Duguid, P. Situated Cognition and Culture of Learning. Educ. Res. 1989, 18, 32–42. [Google Scholar]
  26. Moore, E.B.; Perkins, K.K. Advances in PhET interactive simulations: Interoperable and accessible. In Cyber-Physical Laboratories in Engineering and Science Education; Auer, M.E., Azad, A.K.M., Edwards, A., de Jong, T., Eds.; Springer International Publishing: Berlin/Heidelberg, Germany, 2018; pp. 141–162. [Google Scholar]
  27. Anderson, L.W.; Krathwohl, D.R. A Taxonomy for Learning, Teaching, and Assessing, Abridged Edition; Allyn and Bacon: Boston, MA, USA, 2001; Available online: https://www.astate.edu/dotAsset/11ca93f7-da45-4fe3-821b-b82a20cbc017.pdf (accessed on 9 August 2023).
  28. Daher, W. Discursive positionings and emotions in modelling activities. Int. J. Math. Educ. Sci. Technol. 2015, 46, 1149–1164. [Google Scholar] [CrossRef]
  29. Daher, W. Saturation in Qualitative Educational Technology Research. Educ. Sci. 2023, 13, 98. [Google Scholar] [CrossRef]
  30. Lincoln, Y.S.; Guba, E.G. Naturalistic Inquiry; Sage Publications: Newbury Park, CA, USA, 1985. [Google Scholar]
  31. Hernández-Ramos, J.; Cáceres-Jensen, L.; Rodríguez-Becerra, J. Educational Computational Chemistry for In-Service Chemistry Teachers: A Data Mining Approach to E-Learning Environment Redesign. Educ. Sci. 2023, 13, 796. [Google Scholar] [CrossRef]
  32. Yusa, N.; Hamada, R. Board Game Design to Understand the National Power Mix. Educ. Sci. 2023, 13, 793. [Google Scholar] [CrossRef]
  33. Janeš, A.; Madsen, S.S.; Saure, H.I.; Lie, M.H.; Gjesdal, B.; Thorvaldsen, S.; Brito, R.; Krasin, S.; Jwaifell, M.; Konca, A.S.; et al. Preliminary Results from Norway, Slovenia, Portugal, Turkey, Ukraine, and Jordan: Investigating Pre-Service Teachers’ Expected Use of Digital Technology When Becoming Teachers. Educ. Sci. 2023, 13, 783. [Google Scholar] [CrossRef]
  34. Thyssen, C.; Huwer, J.; Irion, T.; Schaal, S. From TPACK to DPACK: The “Digitality-Related Pedagogical and Content Knowledge”-Model in STEM-Education. Educ. Sci. 2023, 13, 769. [Google Scholar] [CrossRef]
  35. Rayment, S.; Evans, J.R.; Coffey, M.; Kirk, S.; Sivasubramaniam, S.D.; Moss, K. The Role of Technology in Undergraduate Bioscience Laboratory Learning: Bridging the Gap between Theory and Practice. Educ. Sci. 2023, 13, 766. [Google Scholar] [CrossRef]
  36. Daher, W.; Swidan, O. Positioning–Emotions Association of Young Students Using Digital Technology. Mathematics 2021, 9, 1617. [Google Scholar] [CrossRef]
  37. Daher, W.; Shayeb, S.; Jaber, R.; Dawood, I.; Abo Mokh, A.; Saqer, K.; Bsharat, M.; Rabbaa, M. Task design for online learning: The case of middle school mathematics and science teachers. Front. Educ. 2023, 8, 1161112. [Google Scholar] [CrossRef]
  38. Luo, R.; Song, L. The unique and compensatory effects of home and classroom learning activities on Migrant and Seasonal Head Start children’s Spanish and English emergent literacy skills. Front. Psychol. 2022, 13, 1016492. [Google Scholar] [CrossRef] [PubMed]
  39. Daher, W. Students’ adoption of social networks as environments for learning and teaching: The case of the Facebook. Int. J. Emerg. Technol. Learn. 2014, 9, 16–24. [Google Scholar] [CrossRef]
  40. Mengistu, A.; Kahsay, G. The effect of computer simulation used as a teaching aid in students’ understanding in learning the concepts of electric fields and electric forces. Lat. Am. J. Phys. Educ. 2015, 9, 3. [Google Scholar]
  41. Nafidi, Y.; Alami, A.; Moncef, Z.A.K.I.; El Batri, B.; Afkar, H. Impacts of the use of a digital simulation in learning earth sciences (the case of relative dating in high school). J. Turk. Sci. Educ. 2018, 15, 89–108. [Google Scholar]
  42. Daher, W.; Sleem, H. Middle school students’ learning of social studies in the video and 360-degree videos contexts. IEEE Access 2021, 9, 78774–78783. [Google Scholar] [CrossRef]
  43. Daher, W.; Baya’a, N. Characteristics of middle school students learning actions in outdoor mathematical activities with the cellular phone. Teach. Math. Its Appl. Int. J. IMA 2012, 31, 133–152. [Google Scholar] [CrossRef]
  44. Jing, D. The Study on Educational Technology Abilities Evaluation Method. Phys. Procedia 2012, 24, 2111–2116. [Google Scholar] [CrossRef]
  45. Daher, W.; Awawdeh Shahbari, J. Secondary students’ identities in the virtual classroom. Sustainability 2020, 12, 4407. [Google Scholar] [CrossRef]
  46. Wigfield, A. A Questionnaire Measure of Children’s Motivations for Reading; National Reading Research Center: College Park, MD, USA, 1996. [Google Scholar]
  47. Kohake, K.; Heemsoth, T. Need support, need satisfaction and types of motivation in Physical Education for children aged 8 to 13. Development and preliminary validation of the German SMoPE-instrument. Curr. Issues Sport Sci. 2021, 6, 5. [Google Scholar] [CrossRef]
Figure 1. PhET simulation for the application of heat to instigate a change in matter states. (a) The substance in its solid state, (b) the substance in its liquid state, (c) the substance in its gaseous state.
Figure 1. PhET simulation for the application of heat to instigate a change in matter states. (a) The substance in its solid state, (b) the substance in its liquid state, (c) the substance in its gaseous state.
Education 13 00884 g001
Figure 2. PhET simulation for dissolution of salt within water.
Figure 2. PhET simulation for dissolution of salt within water.
Education 13 00884 g002
Figure 3. PhET simulation for separating salt from water.
Figure 3. PhET simulation for separating salt from water.
Education 13 00884 g003
Figure 4. PhET simulation for the microscopic composition of a given substance in its three states. (a) The microscopic level of the substance in its solid state, (b) microscopic level of the substance in its liquid state, and (c) microscopic level of the substance in its gaseous state.
Figure 4. PhET simulation for the microscopic composition of a given substance in its three states. (a) The microscopic level of the substance in its solid state, (b) microscopic level of the substance in its liquid state, and (c) microscopic level of the substance in its gaseous state.
Education 13 00884 g004
Figure 5. PhET simulation for the interplay between the volume of the solvent and the quantity of the solute. (a) Precipitation of solute as a result of oversaturation, (b) dissolving of the solute as a result of adding a solvent.
Figure 5. PhET simulation for the interplay between the volume of the solvent and the quantity of the solute. (a) Precipitation of solute as a result of oversaturation, (b) dissolving of the solute as a result of adding a solvent.
Education 13 00884 g005
Table 1. Thematic elements of each phase of Bloom’s taxonomy—based on Anderson and Krathwohl [25].
Table 1. Thematic elements of each phase of Bloom’s taxonomy—based on Anderson and Krathwohl [25].
CategoryThemes
RememberingRecall, tell, what, when, list, find
UnderstandingExplain, extend, classify, relate, rephrase
ApplicationApply, choose, select, utilize, use
AnalysisCompare, classify, categorize, contrast, infer
EvaluationAgree, assess, appraise, criticize, estimate
CreatingBuild, change, combine, create, elaborate
Table 2. An example of data analysis.
Table 2. An example of data analysis.
RawParticipantAction/InteractionAnalysis
5Teacher:What will we do to convert a substance from the solid state to the liquid state
6Student 1We are in the solid state, and I remember when we conducted the melting experiment in the laboratory. We lit a candle under the beaker containing ice. Now, I will do the same thing; I will ignite the fire under the beaker. Look how the particles move away from each other, and the substance takes the shape of the beaker. This was not visible in the previous experiment; it’s wonderful. Remembering: The students remembered what they did in the previous experiment.
7Student 2Now we have the substance in the liquid state. I will ignite the fire again to convert the substance into the gaseous state. It reminded me of an incident with my mother where she forgot the water on the stove, and when I returned, there was no water left. Remembering: The student remembered a real-life incident related to the topic of the lesson that happened at home.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Rayan, B.; Daher, W.; Diab, H.; Issa, N. Integrating PhET Simulations into Elementary Science Education: A Qualitative Analysis. Educ. Sci. 2023, 13, 884. https://doi.org/10.3390/educsci13090884

AMA Style

Rayan B, Daher W, Diab H, Issa N. Integrating PhET Simulations into Elementary Science Education: A Qualitative Analysis. Education Sciences. 2023; 13(9):884. https://doi.org/10.3390/educsci13090884

Chicago/Turabian Style

Rayan, Baraa, Wajeeh Daher, Hussam Diab, and Nael Issa. 2023. "Integrating PhET Simulations into Elementary Science Education: A Qualitative Analysis" Education Sciences 13, no. 9: 884. https://doi.org/10.3390/educsci13090884

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop