**Mathematical Modeling Projects Oriented towards Social Impact as Generators of Learning Opportunities: A Case Study**

#### **Lluís Albarracín \* and Núria Gorgorió**

Departament de Didàctica de les Ciències Experimentals i la Matemàtica, Universitat Autònoma de Barcelona, 08193 Bellaterra (Barcelona), Spain; nuria.gorgorio@uab.cat

**\*** Correspondence: lluis.albarracin@uab.cat

Received: 23 October 2020; Accepted: 12 November 2020; Published: 15 November 2020

**Abstract:** This paper presents a case study carried out at an elementary school that led to a characterization of mathematical modeling projects aimed at generating social impact. It shows their potential as generators of mathematical learning opportunities. In the school project, upper-grade students (sixth grade, 11-year-olds) studied the way in which the rest of the students at the institution traveled from their homes to school. Its purpose was to identify risk points from the standpoint of road safety and to develop a set of recommendations so that all the children could walk safely to school. In our study, we identified, on the one hand, the mathematical learning opportunities that emerged during the development of the project and, on the other, the mathematical models created by the students. We discuss the impact of the project on the different groups in the school community (other students, parents, and teachers). We conclude with a characterization of the mathematical modeling projects oriented towards social impact and affirm that they can be generators of mathematical learning opportunities.

**Keywords:** mathematical modeling; modeling projects; elementary school; learning opportunities

#### **1. Introduction**

Mathematical modeling in the classroom encourages students to develop mathematical knowledge through the study of real-life situations, taking advantage of the link between reality and mathematical concepts and procedures (Blum [1]; Doerr and English [2]). Our participation, as researchers in various school projects in which the mathematical analysis of reality is a central part of the work, inspired us to propose the concept of the mathematical modeling project oriented towards social impact (MMPOSI). By way of an initial approach to the concept, we established that the elements defining an MMPOSI are structured around two key ideas: the role of mathematical modeling and its social component. In this article, we exemplify the concept of MMPOSI through a case study developed from a naturalistic perspective at an elementary school. Thus, the case studied allowed us to characterize MMPOSIs and exemplify their potential by showing how they can generate mathematics learning opportunities in elementary school (Cai et al. [3]; Cobb and Whitenack [4]). Given that this research was designed as a case study with an instrumental descriptive character (Merriam [5]; Stake [6–8]), we leave the characterization and discussion of the concept of MMPOSI to the end of the article.

The school where this case study was developed is in the center of Sabadell (Spain), a city with 200,000 inhabitants. The streets around the school are narrow and were laid out prior to motorized vehicles. They are now used by both road traffic and pedestrians. In 2017, an accident occurred in which one of the students was run over by a vehicle while returning home from school. This accident had a huge impact on the school community. New needs emerged both for students and their families, as well as for teachers. Among the various actions taken, the school management team, in coordination

with the parents' association, requested the assistance of the first author of this paper to carry out a project in which the pupils would study the difficulties they encountered in their everyday journeys on foot to and from the school. This request provided the basis of the school project "Let's get to school safely", in which upper-grade students (11-year-olds) were given the task of documenting the routes taken by the rest of the students to get to school, then analyzing the findings to identify the pedestrian danger points. The final goal of the project was to generate a set of indications that would ensure that the students could safely walk to the school through the city streets.

The text of this article is organized according to the structure that often appears in reports of instrumental descriptive case studies. In Sections 2 and 3, the conceptual framework of the study is introduced. In the second section, we explain the interpretation of the mathematical modeling that we apply when establishing the definition of MMPOSI and, in the third, the meaning that we give mathematical learning opportunities to justify our methodology. The empirical study is presented in a block consisting of Sections 4 and 5. The fourth section justifies and details the choice of a case study as the research design, and the fifth explains the chronological progress of the project Let's get to school safely. The results, which are essentially descriptive given the nature of the study, are the themes that emerge from the study and are presented in Sections 6 and 7. The sixth section describes the mathematical learning opportunities that emerged during the course of the project and, in the seventh, the models generated by the students. Sections 8 and 9 bring the article to a close. In the eighth section, we discuss the learning opportunities and models identified, and in the ninth, we conclude the article by setting out the basic characteristics of MMPOSIs.

#### **2. Modeling in Mathematics Education**

#### *2.1. Background*

Mathematical modeling as a research topic in the field of mathematics education began with the work of Pollak [9], who discussed the relationship between the applications of mathematics and the teaching/learning processes. Subsequently, the same Pollak [10] presented a first theoretical framework, which interpreted modeling processes by differentiating between the mathematical domain and the rest of the world. This separation necessarily leads us to the process of mathematizing a phenomenon, moving from reality to the mathematical domain, and to the interpretation within the real context of the models generated in the mathematical domain, as a form of validation.

Following Blum [1], mathematical modeling was established as a research topic of interest in mathematics education, aimed at setting up classroom activities that bring to light the close relationship between mathematics and the world around us. A great deal of research on educational mathematical modeling has been carried out since then, and it has diversified remarkably, in terms of both the goals and the approaches (Abasian, Safi, Bush, and Bostic [11]; Blomhøj [12]; Kaiser and Sriraman [13]). Thus, from the perspective of the mathematization of the environment and mathematical modeling, a clear need has been identified at an international level to link up students' mathematical knowledge with reality (Vorhölter, Kaiser, and Borromeo Ferri [14]).

Recently, Blum [15] reaffirmed that the teaching of applications and modeling has a twofold function: on the one hand, the knowledge of mathematics and its applications is vital to the real world and its advancement, principally with regard to solving real problems and developing complex projects; and on the other hand, the real world and the way it integrates mathematical knowledge are extremely important as a vehicle for giving meaning to the learning of mathematical concepts and, in general, mathematics as a discipline.

The theoretical advances made in the didactic use of modeling are well known. However, their presence has not yet been felt in the majority of classrooms. There are various reasons why the use of mathematical modeling has not increased in elementary and secondary education classrooms. For example, institutional constraints have been identified that make it difficult to fit modeling activities into the normal functioning of educational centers (Barquero, Bosch, and Romo [16]). It has also been

observed that teachers' attitudes and their training are key to the regular use of modeling activities (Blum and Leiß [17]; Schmidt [18]). Indeed, many mathematics teachers do not think modeling is an essential component of learning mathematics, and they also question their own mathematical modeling skills. Given that students may come up with many different solutions and it is not easy to identify the focus of the activity in these tasks, teachers regard the implementation of mathematical modeling tasks as rather complex (Ng [19]; Winter and Venkat [20]). These difficulties are possibly more marked among elementary school teachers, since their training is less grounded in mathematics.

#### *2.2. Models and Modeling Activity*

Problem solving is a key part of mathematics education (Lester [21]; Schoenfeld [22]). In this paper, we argue that contextualized problem solving activities can be interpreted as mathematical modeling activities, given the type of mathematical constructions that students create to solve them. Our approach is situated in that area of research that explores how students solve mathematical problems, situated in real contexts, when there are no defined heuristics. These problems promote metacognition and help to familiarize students with the methods of applied mathematics (Verschaffel [23]). From this perspective, the fundamentals of modeling are aligned with the principles of project-based learning, while promoting active learning (Krajcik and Blumenfeld [24]) in meaningful contexts that students can relate to their prior knowledge (Blumenfeld et al. [25]), with the focus on the development of mathematical activities and concepts. Thus, we consider a class task to be a modeling activity when students generate or use mathematical models to describe or analyze real phenomena. In this paper, we adopt the definition of a mathematical model proposed by Lesh and Harel [26]:

"Models are conceptual systems that generally tend to be expressed using a variety of interacting representational media, which may involve written symbols, spoken language, computer-based graphics, paper-based diagrams or graphs, or experience-based metaphors. Their purposes are to construct, describe or explain other system(s).

Models include both: (a) a conceptual system for describing or explaining the relevant mathematical objects, relations, actions, patterns, and regularities that are attributed to the problem-solving situation; and (b) accompanying procedures for generating useful constructions, manipulations, or predictions for achieving clearly recognized goals." (p. 159)

This definition makes it clear that some of the concepts and procedures that make up a model are mathematical. However, the constructed models may also contain non-formal aspects that allow an intuitive description of the reality under study, such as graphic representations or the use of metaphors. Modeling is a process of solving a real problem in which mathematical concepts, methods, and results are involved (Blum and Niss [27]). To this end, the objects, data, and relationships occurring in reality are transferred to the world of mathematics (horizontal mathematization), thereby obtaining a mathematical model. Mathematical methods are then applied to this model to reach a mathematical solution (vertical mathematization), which must be interpreted and validated in the real world where the problem is framed, resulting in a real solution (again, horizontal mathematization).

It is generally agreed in the world of mathematics education that modeling processes are of a cyclical nature (Blum and Leiß [17]; Carreira, Amado, and Lecoq [28]; Doerr and English [2]; Galbraith and Stillman [29]; Greefrath [30]; Kaiser and Stender [31]). During a modeling process, students try to solve a problem by going through different stages in which they move from reality to the mathematical domain, each time re-evaluating the phenomenon under study. The entire process is repeated in different cycles, with the students improving the models and solutions found for the problem they are working on, adapting the models to the requirements of the problem statement (Blum and Borromeo Ferri [32]). Finally, they have to communicate the result of this process. To successfully find their way through these stages, students have to draw on a series of competences that include aspects related to metacognition, motivation, and their own ideas about the nature of mathematics (Maaß [33]).

From the theoretical perspective of models and modeling (M&M) and from a model-eliciting point of view, students are understood to perform multiple cycles of interpretation, descriptions, conjectures, explanations, and justifications that are redefined and reconstructed iteratively as they interact with other students (Doerr and English [2]). Model building involves quantifying, dimensioning, coordinating, categorizing, algebratizing, and systematizing relevant objects, relationships, actions, patterns, and regularities (Mousoulides, Sriraman, and Christou [34]). The M&M perspective also considers data-modeling problems that focus on organizing and representing data, building patterns, and searching for relationships (Lesh, Amit, and Schorr [35]), as well as involving students in statistical reasoning such as decision-making, inference, and prediction.

The problems posed to students in the framework of M&M, contextualized in the real world and with characteristics and demands that make them modeling activities, are called "model-eliciting activities" (MEAs) (Doerr and English [2]). The M&M approach is integrated into problem solving because it considers that an MEA in itself constitutes the process of modeling and obtaining a model. Thus, the problem statement must allow the students to establish adequate criteria that help to decide which solution is the most appropriate among a set of different alternatives. It should also enable the students to judge for themselves whether the answers need improving, refining, or amplifying for a specific purpose. During an MEA, students are asked to work in small groups (Clohessy and Johnson [36]; Zawojewski, Lesh, and English [37]) and are confronted with a problematic situation that is significant and relevant to them, for which they must create, expand, and perfect their own mathematical constructions.

In MEAs, students are encouraged to generate products that go beyond providing brief answers to artificially restricted questions about pre-mathematized situations. It is a question of enabling the creation of models by the students on the basis of their previous knowledge, both mathematical and about the real world. Students' work during a modeling activity should result in productions that are shareable and re-usable in similar situations (Lesh and Lehrer [38]). Depending on the project needs, students may generate models to provide decision-making tools (Mousoulides, Sriraman, and Christou [34]). Students develop these tools from models that fulfill a functional or operational role. This includes drawing up specific action plans to deal with problematic situations and designing the assessment instruments needed to distinguish different scenarios in complex situations where it is necessary to use specific mathematical models.

#### **3. Mathematics Learning Opportunities**

There is a wide and continuous spectrum of situations in which learning opportunities have been studied. On the one hand, there are large-scale studies that measure the acquisition of learning. At the opposite extreme, there are micro-studies that address the achievement of specific learning goals, usually in classroom activities. In large-scale studies in general, the goal is to arrive at an interpretation of learning outcomes measured globally through performance tests, either in international comparative studies or accountability studies, or to explain why certain groups (minorities, students with specific needs (Kurz [39]), etc.) do not perform at the same level as the population taken overall. From this perspective, learning opportunities are defined in relation to the the measured contents of the curriculum, the educational level, and the learning conditions. In any case, in this field, the interest is in learning as a product, as a result of certain conditions that include learning opportunities. It is important to note that, under this interpretation, the opportunity is not necessarily thought to imply learning. In fact, Törnroos [40] pointed out that having the opportunity to learn is a necessary prerequisite for learning, but a learning opportunity does not guarantee that students will actually learn. In general, the studies related to learning opportunities try to explain the lack of learning as caused by a lack of opportunities. However, Floden [41] pointed out that other factors influence learning outcomes, including the quality of the teaching and the students' abilities.

At the level of what happens in the classroom, when the term mathematical learning opportunity is discussed, it is initially linked to the analysis of classroom interactions where mathematical knowledge is constructed. Cobb and Whitenack [4] argued that mathematics learning is a process of conceptual self-organization and enculturation. From this perspective, a mathematical learning opportunity is a situation in which students have the opportunity to reorganize their conceptual structures and approaches when solving problems or, in general, when dealing with a new mathematical activity. Therefore, it is a concept closely linked to the content being learned, the learning process, and the characteristics of the learning activity.

Cai et al. [3] stated that any definition of classroom-based learning opportunities must necessarily consider the interactions between three elements: the mathematics tasks, the teaching, and the students. They considered it impossible to separate out the influence of any of these components, since the nature of their interactions will determine whether an activity, or a classroom experience, becomes a learning opportunity for a given group of students in relation to a specific goal. Accordingly, Cai et al. [3] stated that the interactions between the three elements create complexities that can probably only be understood by means of multiple iterations of studies based on successive conclusions. In order to make progress, they suggest that multiple studies, often small-scale ones, must be carried out to move gradually towards more complete and accurate answers. Among the research methods they proposed is the study of the mathematical task set for students.

In recent years, a good number of studies of mathematics education related to mathematical learning opportunities have focused on teaching quality and the resources used, analyzing how the pedagogical and/or curricular characteristics of the teaching facilitate or limit students' opportunities to learn. In this respect, the study carried out by Wijaya, van den Heuvel-Panhuizen, and Doorman [42], who concluded that the lack of tasks in context-based textbooks limits students' learning opportunities, is particularly interesting in our opinion. These authors developed a framework with four perspectives for analyzing the role of context in mathematical activities: the types of context, the purpose of context-based tasks, the information used in tasks, and the type of cognition required by the tasks.

Wijaya, van den Heuvel-Panhuizen, and Doorman [42] concluded that when the procedure to be applied is made more or less explicit, students do not need to determine what the appropriate mathematical procedures might be to solve the task, which means they will not develop their ability to transform a context-based task into specific learning. These authors recommended including more tasks based on real-life contexts in classroom practice, and they set out how these tasks should be introduced. First, they pointed out that they should not only appear immediately after the explanations of concepts or procedures, since then the strategies to be followed seem clear. The quality of the tasks is also important: they should be presented in essential, relevant contexts, which can generate opportunities to mathematize situations or organize them mathematically. In addition, the assignments should, according to these authors, incorporate superfluous information or require a search for new information so that students have the opportunity not only to select what is relevant, but also to identify appropriate procedures. They should be tasks with a high cognitive demand so that students have the opportunity to develop complex reasoning, which requires reflection in relation to real-life contexts.

Research has shown that tasks posing a greater cognitive challenge intensify students' involvement in mathematical ideas (Boaler and Staples [43]; Tarr et al. [44]). Tasks with a high cognitive demand require the connection of procedures to their underlying concepts, or the completion of complex, non-algorithmic tasks; tasks with a low cognitive demand involve the memorization or performance of procedures without connecting them to the underlying concept. The best learning opportunities arise when the task meets two conditions. On the one hand, it should require the use of two or more forms of representation (Lesh, Cramer, Doerr, Post, and Zawojewski [45]) and the translation between them, and on the other, it should oblige students to explain their strategies and thinking (Walkowiak, Pinter, and Berry [46]). Ultimately, mathematical tasks play a central role in the type of interactions that are possible and in the nature of the learning opportunities that emerge (Cai et al. [3]).

Our study is among those that analyze factors related to the quality of teaching, and more specifically, the activities proposed to students. Therefore, we analyze the project as an activity that can generate learning opportunities, based on the study of what the students show they learned during its development. The study of learning opportunities in conjunction with classroom activities has been based traditionally on three basic methods: direct observation in the classroom, teacher's reports, and documentary analysis of different elements, including the students' products (Kurz [39]). Kurz and Elliott [47] suggested that the learning opportunities generated by an activity can be studied according to what students show they are capable of doing when coping with the activity.

#### **4. A Case Study Research Design**

Below, we present the case study carried out with the goal of supporting the concept of MMPOSIs as generators of learning opportunities. Whereas this research was a case study (Merriam [5]; Stake [6,7]), the object of study was a bounded system (Stake [8]) where we worked essentially with qualitative data and with the intention of providing a detailed account of the case. Our case was an inclusive one (Cresswell [48]; Merriam [5]; Yin [49]) since we not only focused on an object with a clear entity (a group of people), but we were also interested in studying an activity under development. Case study research can be used to address exploratory, descriptive, and explanatory research questions (Stake [7,50]; Merriam [5]; Yin [49]). Our research was an instrumental descriptive case study since it provided initial insights into an issue (Stake [7]). Then again, our interest went beyond understanding the particular case because we hoped that the project Let's get to school safely would illustrate how MMPOSIs operate as generators of learning opportunities.

All case studies have one thing in common: they focus on a case as a complete unit, just as it exists in its real-life context. When determining the case to be studied, we used purposive sampling, taking advantage of the opportunity to participate in the orientation of the project offered by the center. In this case study, we adopted a naturalistic perspective and took the role of research observers during an activity developed naturally at a school (Hatch [51]). We had no control over the environment or the variables that influenced the students' work, but we were able to obtain a close-up view of how things happened in reality.

In research designed as a case study, the researcher must provide a detailed account of the case. We present a detailed comprehensive description of the development of the project Let's get to school safely in the following section, organizing it around what we call episodes. In each of them, the students worked on a core activity based on the four specific goals of the project. The development of the project was documented with photos, videos of specific moments, and notes from classroom observation. We also collected all the work material produced by the sixth graders, which included maps that showed the routes taken by students to get to school, reports of measurements of danger points in the surroundings of the school, guidelines for safe walking, and presentations created to inform other students at the center.

The research issues that focus this study (Stake [8]) and the findings presented in this article revolve essentially around two ideas: the mathematics learning opportunities promoted by the project, and the mathematical models developed by the students during the project. Consequently, various methods of data collection were used: the primary data comprised the field notes together with the documents produced by the students, while the informal interviews with the teacher and the parents provided the secondary data. We drew on these data to develop what was essentially a content analysis that followed the steps proposed by Miles and Huberman [52], consisting of data reduction, data visualization, and conclusion/verification. Specific codes were used to group the data into the emerging categories that became obvious, and the categories were organized into two themes around the research issues.

In this way, we start with a description of the development of the project Let's get to school safely, and then, we analyze it through different levels of abstraction and provide our interpretation of the development of the project, relating it to our conceptual framework, i.e., the learning opportunities and construction of mathematical models. The discussion of the findings and the themes that emerged

from the categories led to the interpretation of the case studied. We conclude by proposing a detailed characterization of MMPOSIs that goes beyond the initial definition.

#### **5. Development of the Project in the Classroom**

The project that was the object of our case study was implemented in a school that has one class group per grade and welcomes students from three to eleven years old. As explained in the Introduction, the school is located in the center of a city with a high population density and surrounded by narrow streets shared by cars and pedestrians. Since most students go to and from school on foot, difficulties and potential dangers are generated by the coexistence of cars and pedestrians in a small area. Both the parents' association and the teaching staff had expressed their concern in this respect and proposed various activities to deal with these potential dangers.

Among other activities, the project Let's get to school safely was launched. The older students (sixth grade in elementary education, a group of twenty-six 11-year-olds) would study and analyze the routes to school taken by each of the rest of the students at the school, the goal being to establish appropriate guidelines and recommendations so that the children could get to and from school safely. The project took place over two weeks in six sessions lasting 90 min each. In the first session, the project and its role as a generator of recommendations for the rest of the students at the school were discussed. The teacher and the first author helped the students define the core activities of the project. These were: (i) identify the most common walking routes to the school; (ii) identify potentially dangerous places and situations for students on these routes, (iii) analyze these places to determine safe practices; and (iv) communicate the results of the project to the rest of the school.

Below, we describe how each of the core activities was carried out during the different episodes of project development in order to provide the reader with the details of the case under study.

#### *5.1. Episode I. Identification of Routes*

Eight work groups were set up in the sixth grade class. Each of these groups focused on studying one of the classes in the lower grades, which was their target class. Students were encouraged to choose a class where they had connections (siblings, neighbors, friends) so as to make communication easier and encourage greater involvement. As a first step, students asked students in their target class about the routes they use to get to school. This step was particularly challenging for the students who worked with the youngest children. Figure 1 shows a sixth grade student asking a four-year-old girl about the route she follows. Given her age, the younger student still had difficulty describing the route she takes to school. For this reason, the sixth grade student asked questions about well-known buildings, squares, and shops that she might recognize on the route in order to obtain the required information. This was then displayed on a map of the school surroundings.

**Figure 1.** A sixth grade student asking a four-year-old girl about the route she takes to go to school.

Once all the routes followed by all the students in each target class had been ascertained, the data had to be organized to make decisions. At this point, various interesting findings emerged from the data, such as the distribution of the students' homes or the age at which they begin to learn their route to school in detail. The sixth grade students observed that, below the third grade of primary education (8- to 9-year-olds), there is no guarantee that a child will know the details of the route that he/she usually takes. This prompted other questions that were discarded because they strayed from the main goal of the project. The sixth grade students decided that, for each target class, they would represent the routes followed by the students on a single map in order to structure the subsequent project activities (Figure 2).

**Figure 2.** A map showing the routes taken by the students in a target class.

#### *5.2. Episode II. Identification of Danger Points*

The next phase focused on identifying danger points in the surroundings of the school and examining them to identify the factors that could provoke accidents. To do this, a map of the area around the school was projected onto the classroom screen, and the students discussed the characteristics of the streets and nearby intersections and their own dangerous experiences in these places. When students considered it necessary, they used the Google Maps tool to visualize the streets from an immersive perspective and explore them virtually. The final product of this discussion was a list of danger points requiring further investigation (Figure 3). The danger points were then distributed among the groups for study, bearing in mind in each case that the point had to be part of a route frequented by the target class corresponding to the work group.

**Figure 3.** Identifying danger points in the surroundings of the school.

#### *5.3. Episode III. Analysis of Danger Points and Preparation of Recommendations*

Episode III consisted of fieldwork. Most of the risk points were street intersections with crosswalks presenting specific characteristics. These characteristics were discussed beforehand in the classroom during Episode II, since upon separating to do the fieldwork in the surroundings of the school, the students were unable to consult the teacher directly. This prior knowledge helped the students to work more efficiently in the field and ensured they obtained the necessary data.

For this activity outside the school, we counted on the assistance of 16 parents from the center who volunteered to make sure the activity went smoothly and looked after the students' safety while they were working in the streets. Each of the groups went to the place to be studied with the intention of drawing up their own map of the area, measuring the relevant features from the standpoint of the passage of vehicles and pedestrians, and indicating the way the latter group moves around. They also took photographs to illustrate their work and recorded videos simulating everyday situations to illustrate the difficulties encountered by pedestrians.

Once they had identified the potential dangers of each one of the points of interest and analyzed the causes, the sixth graders set about identifying safe behaviors that would enable students to move around securely. This search for safe behaviors began in the street and ended later, in the classroom, while the students were preparing to communicate the results of their project.

#### *5.4. Episode IV. Communication of Project Results*

During this episode, each group shared its proposals with the other class groups in order to reach a consensus. The information obtained was distributed to help each work group prepare the informative talk that they would give their target class. This collaborative approach made it possible to validate each group's proposals and also optimize the impact of their efforts with the preparation of experts on each risk point. The groups adapted the message content to the age of the target students in order to properly communicate the results of the project. Furthermore, thanks to their personal relationships, they were able to exemplify good practices based on specific cases. The presentations began with an introduction that insisted on the right to be able to move safely around the city. After that, each work group explained the procedure followed during the project, the difficulties encountered at the risk points, and their recommendations for safe passage through them. The talks with the different target classes were organized in parallel on a Friday afternoon, and the parents were also invited to learn about the project and its conclusions.

#### **6. Theme 1: Mathematics Learning Opportunities**

In this and the following section, we report the findings of this study, presenting the two emerging themes through a robust description (Merriam [5]) that incorporates the constructed categories and exemplifies them with units of meaning drawn from the data. This enabled us to show that there were learning opportunities and that models were generated.

The first emerging theme comprised the mathematics learning opportunities that appeared during the development of the Let's get to school safely project. Cobb and Whitenack [4] held that mathematical learning is a process of conceptual self-organization and enculturation. From this perspective, a mathematics learning opportunity is a situation in which students have the chance to reorganize their conceptual structures and approaches when solving problems or, in general, when they have to cope with a new mathematical activity. For each episode in the project, we identified mathematics learning opportunities by analyzing the actions taken by the students—understood as mathematical processes—in response to the proposed activity. These are indicated in italicsbelow.

During Episode I—route identification—the sixth grade students prepared the data collection and established a specific way of recording the information they would collect. The students discussed the various options and selected the most relevant ones. Thus, they realized that writing down the home address of each student in the survey was not relevant since it did not determine the route taken. Conversely, they noted that other options, such as recording the information on a printed map, generated useful data that clearly reflected the students' journeys. This discussion represented an opportunity *to reflect on the complexity of the phenomenon under study, the nature of the data to be collected, the data collection procedures that they were familiar with, and how they would need to use these data during the subsequent development of the project*.

When the sixth grade students asked the students in their target classes about their routes, this led to an interaction that contained various types of mathematical content. In all cases, the first step was for the sixth graders *to explain the information on the map and present the way to interpret it*, working on *orientation on a map and obliging the younger students to identify specific landmarks* in the city (parks, buildings, and shops) with points on the map, and *relating the directions of movements on real routes with movements on the map*. The younger students had to visualize their daily route to school and adapt their explanations to the needs of the sixth graders. This process of visualization included *interpreting graphic information on the map and the visual processing of the route they use to go to school*, with these two procedures understood as in Bishop [53] and Gorgorió [54].

Once the sixth grade students had identified all the routes, the mathematical activity focused on understanding the information they had collected. To do this, the students were asked to *organize the maps* and then *classify the identified routes into sets*, where the routes had to have a large section in common. We observed that the students hesitated when grouping routes with slight variations. The *need to reduce the complexity of the situation* was discussed in order to be able to study it with the methods at hand. After creating the sets of routes, each group had to *draw up a map showing the frequency of use* of all the streets around the school (Figure 4). This process of representing information included *visual coding*, since students developed their own graph format. Previously, they had only worked with pie charts and bar graphs in statistics as resources to represent frequency.

**Figure 4.** A map showing the intensity of use of the different routes taken to go to school.

In Episode II—danger points—the first mathematical activity consisted of drawing up a proposal for a list of dangerous places for pedestrians in the surroundings of the school. The sixth graders were asked to identify them on their own journeys to and from school and to try to *visualize possible risk situations* for younger students. This obliged the sixth grade students to reinterpret the use of the streets from the perspective of people with more limited mobility and who see the world from a lower height. This activity involved a *visual processing* procedure because it required anticipating movements and lines of sight that would be verified and complemented in the later field study. Before accepting a proposed risk point, students were required to verbally describe a potentially dangerous situation that could occur in that spot. This activity demanded *spatial reasoning*, since students had to describe the movement of different objects (vehicles and people) and their form of interaction. The various validated risk points were situated on a map, thereby repeating the activity of *location on the map*.

In Episode III—analysis of danger points and generation of recommendations—various mathematical activities related to the *use of measurement in context* and the representation of reality on a map were carried out, both as regards urban architecture and the use of the street by vehicles and pedestrians. Students recreated possible risk situations and took various types of measurement that were relevant to understanding how cars could interact negatively with pedestrians. They recorded them on a map of the risk point under study.

The corner in Figure 5 is shown by way of an example. As you can see in the picture, the curb has been lowered to allow cars to turn more easily. The students measured by how much the cars mount the sidewalk.

**Figure 5.** Fieldwork measuring the width of the sidewalk mounted by cars when turning.

In other cases, students *measured how high a pedestrian needs to be for a driver to see him/her* when waiting at a zebra crosswalk where the adjacent parked cars limit the drivers' field of view. Students also *measured the angle necessary for a pedestrian at that same crosswalk to see a car approaching* and have enough time to stop. In all these cases, the measurements they took were not of objects in the street, but rather measurements of distances that the students determined by *visualizing the interactions between vehicles and pedestrians*, because they offered information relevant to the decision-making in the following part of the project.

Lastly, in Episode IV—communication of the project results—the mathematical activity consisted of the interpretation of the collected data to prepare guidelines and recommendations for the students in the other grades. They used videos, photographs, screenshots of Google Maps, and annotations with measurements to distinguish safe behaviors from risky ones. This implied *using this information in a contextualized manner, interpreting the mathematical model generated in relation to the real world*; they had to consider traffic regulations and also the usual behavior of vehicles and pedestrians.

Figure 6 shows the categories that summarize the mathematics learning opportunities identified during the development of the project.

**Figure 6.** Mathematical learning opportunities identified during the project.

#### **7. Theme 2: Models Generated**

The second emerging theme consisted of the mathematical models created by the sixth graders during the development of the Let's get to school safely project. Mathematical models are conceptual systems that describe real phenomena (Lesh and Harel [26]), but from the research standpoint, the identification of the conceptual systems generated by students turns out to be rather complex. However, models are implemented according to specific procedures associated with these concepts. In previous works, we developed a tool for the characterization of mathematical models (Albarracín [55]; Albarracín and Gorgorió [56]; Gallart, Ferrando, García-Raffi, Albarracín, and Gorgorió [57]), which was based on identifying the chains of procedures that students implement and how these procedures are represented. In this study, we observed that sixth grade students created two types of mathematical models to tackle two aspects that were crucial to the development of the project: (i) maps of route use intensity and (ii) safe travel recommendations for other students.

#### *7.1. Maps of Route Use Intensity*

The first mathematical model they created consisted of the *maps of route use intensity* in the streets around the school. These maps were obtained from the data collected from the other students at the school, and they describe the phenomenon of student travel to school. The generation of these maps was crucial to the project because they were what enabled the students to identify the risk points to be studied. Table 1 describes the mathematical procedures that shaped the model.

**Table 1.** Description of the modeling process used to generate a map of route use intensity in terms of identified procedures.



5. Change the form of presentation to establish a code that makes the information displayed easy to read

A sample of the work involved in making these maps of route use intensity is shown in Figure 7. On the left, Figure 7a shows two students grouping the maps with the individual routes to make the map of route use intensity. On the right side, Figure 7b shows two different steps in the preparation of the same map. On the left, a count of the number of students passing through each street is shown.

#### *Mathematics* **2020**, *8*, 2034

On the right, the finished map is shown, with the numerical information simplified into a color code with the busiest streets marked in red.

**Figure 7.** (**a**) Organizing the collected routes, (**b**) representing them on a single map.

From the perspective of M&M, these maps are clearly the result of modeling work. They display the use of the streets for going to school and are a product that supported subsequent decision-making on how to prioritize the risk points that students were going to study. The sixth grade students used these maps as a starting point to establish a criterion of selection based on determining those risk points most frequented by students in the target classes.

#### *7.2. Recommendations for Safe Travel*

The second type of mathematical model generated was the *set of recommendations for safe travel* in the streets. These recommendations emerged from the analysis of the layout of the areas under study and the way cars and pedestrians move through them, and from the predictions made about what they considered safe behavior. Table 2 describes the mathematical procedures that shaped the model.

**Table 2.** Description of the modeling process used to generate recommendations for safe travel in terms of identified procedures.



3. Represent the interactions of risk on a map, and take the measurements (distances and angles) that characterize them

4. Propose alternatives for pedestrian movement to avoid risk situations

5. Represent recommendations for safe action on a map, using indicators of movement

Given the diversity of black spots in the area around the school, we show two examples of the recommendations drawn up by the students in their work. In the first example, the students decided that the corner shown in Figure 8 represents a potential risk area for pedestrians heading towards it from the north. Therefore, they indicated this movement with a red arrow in Figure 8. This becomes a danger point if a car mounts the sidewalk at the same time as when a pedestrian coming in the opposite direction turns the corner, since the two paths cross. The students suggested that pedestrians should avoid that corner by using two crosswalks, i.e., following the green arrow. This movement is not really natural as it forces pedestrians to walk a greater distance than the red option, but the field of view of both pedestrians and drivers is better at each crossing.

**Figure 8.** Scheme with recommendations for use at a corner where cars may mount the sidewalk.

The second example concerns a T-intersection where cars that turn right have to give way to cars coming from their left (Figure 9a). There is a mirror at the intersection so that drivers can see whether it is safe to turn without having to pull halfway out into the other street (Figure 9b). The need to pay attention to the left means that drivers do not pay much attention to the right when turning. As in the previous case, a car could easily mount the sidewalk on the right-hand corner and hit someone walking there. For this reason, the students in their recommendations suggested walking on the left-hand sidewalk (Figure 9c), marking it in green on the map, so that they would always be in the drivers' field of view and be able to use the crosswalks when the vehicles stop.

**Figure 9.** (**a**) Aerial view of the corner under study; (**b**) drivers' focus of attention; (**c**) recommendations for use represented by arrows.

The students' set of recommendations was the product of a modeling process that enabled us to distinguish between good and bad practices when moving through the streets. The recommendations were specific to each situation studied, but the procedure for creating them is shareable and reusable in other analogous situations. Beyond this possibility of direct transfer, the generated model could be used for the design of safe behaviors for other users in other situations, such as drivers of cars and other vehicles. Moreover, the sixth grade students communicated the recommendations resulting from their study to the rest of the students, highlighting that their construction was based on mathematical arguments and procedures. In this way, mathematics constituted a validating component of the work carried out.

#### **8. Discussion**

The results of this case study showed that the Let's get to school safely project generated a large number of mathematical learning opportunities for the sixth graders. During the project, students had to cope with activities that required the coordinated use of different content and mathematical skills. Statistical data collection, information processing, data coding, visualization, mapping, and measurements, among other processes, were highly relevant to shaping the mathematical models. Each of the work groups was responsible for studying the routes of one of the school classes and for providing age-appropriate recommendations for these students.

During Episodes I and II, the work groups focused on collecting and organizing the data obtained from their target classes, thereby raising opportunities for mathematics learning that had to be consolidated by means of concrete products in order to continue the project. During Episode III, each of the groups was responsible for analyzing a risk point and preparing specific recommendations for safe passage through that point. The sense of responsibility developed during the project was what compelled the various work groups to address the learning opportunities that emerged during the project and transform them into concrete learning that manifested itself in the form of tangible products. In other words, the responsibility of each group in the project was the medium that promoted the construction and reconstruction of mathematical models, which is the main objective of modeling activities that take the M&M approach (Doerr and English [2]; Lesh, Amit, and Schorr [35]; Mousoulides, Sriraman, and Christou [34]).

Some of the ideas and proposals examined to generate the different models that supported the products of the project began with interaction among a small number of students, who were the ones who contributed the ideas in the first place. However, following group discussions involving the entire class, each of the work groups had to consider, interpret, and implement these ideas according to their own needs. Thus, the project encouraged the students to explain their strategies and reasoning (Walkowiak, Pinter, and Berry [46]) relative to the mathematical activities for which the students had decided what information they needed and the methods needed to analyze it (Wijaya, van den Heuvel-Panhuizen, and Doorman [42]). We concur with Törnroos [40] when he stated that the existence of learning opportunities does not imply that learning is consolidated. However, during the project, we saw how the students developed a form of collaboration where the mathematical contents and methods constituted the core of the activity. Therefore, what we observed leads us to think that the students learned to work mathematically as a team. Not only that, they also developed a sense of belonging to the school community.

The project had an impact on other school groups apart from the sixth grade students. The younger students saw how their schoolmates in the last year were interested in a problematic aspect of their daily lives. The sixth grade students asked them about their experiences and later gave them explanations—tailored to their level of comprehension and their practical needs—about factors that they had to take into account to guarantee their own safety when walking the streets. The explanations about the analysis and decision-making process that the sixth graders gave the younger students were an essential aspect of this informative procedure. Thus, mathematics played a clear role as a validator of the project results for the rest of the students. Besides that, some of the children's parents were able to participate actively in one of the activities, and others attended the final informative talks. In all cases, their children received information that made it possible to address safe behavior on the streets from within the family unit, taking the results of the project as a starting point.

From the point of view of the teaching staff, the project brought to light a new way of teaching students to use mathematics in context. Furthermore, the project format, based on the analysis of a social reality in order to generate safe behavior guidelines, provides a guide for teachers. This format could allow them to overcome some of the difficulties identified in the literature, because when students contribute a variety of ideas that are difficult to manage (Ng [19]; Winter and Venkat [20]), the teacher can refer directly to the needs of each episode of the project to decide if the students' proposals point in the right direction and prioritize them appropriately. We understand that teachers must have an active mathematics disposition if they are to adequately guide their students. In other words, they must be open to asking themselves questions and setting themselves problems, have a knowledge of specific cases where mathematics helps to understand real situations, and be able

and willing to document themselves or seek specific help when they are not acquainted with the mathematics necessary to interpret or describe a situation. It is also necessary that teachers be trained to work in the classroom using open projects or have the experience needed to do so, even if they are not strictly mathematical projects. Thus, a key aspect of the role played by teachers is to identify the mathematical contents that students generate in order to properly organize and institutionalize them.

#### **9. Conclusions**

The case study developed around the project Let's get to school safely permits the characterization of a new type of school project that we labeled mathematical modeling projects oriented towards social impact and informs us of its possibilities, showing us that such projects can be generators of learning opportunities, as explained above. We conclude this article with a description of the basic characteristics of MMPOSIs.

First of all, a modeling project geared towards social impact must tackle the needs of people as a community (for example, road safety, pollution, immigration, etc.) and must be structured with the intention of having a direct impact on the educational community, either on the students themselves, their families, or other close groups of people with similar needs. In this way, the issue tackled in the project arises from the students themselves or the school itself, and the product resulting from the modeling process has a real impact on the community, even beyond the group of students who develop it. Furthermore, it is the fact of developing a sense of responsibility towards the community that leads to the search for solutions, thus favoring the emergence of mathematics learning opportunities and, in particular, stimulating students to take advantage of them. We call this characteristic the principle of social impact.

Then again, for an intervention to be an MMPOSI, it must be structured in such a way that it demands an analysis of the phenomenon from the perspective of mathematical modeling. It should be possible to organize the project into various coordinated activities that act as MEAs (Doerr and English [2]), which encourage the students to generate products based on a mathematical model, which can be communicated to and used by the project target group. Thus, the development of this type of project obliges the students to decide what information is needed to tackle it and what methods are required to analyze it, as well as to explain their strategies, procedures, and reasoning to their peers. We call this second characteristic the principle of mathematical modeling.

The mathematical knowledge gained during the project should support actions that provide a return to the educational community. Mathematical knowledge provides a basis for student decision-making and, at the same time, plays a key role in validating the results of a project when it is presented to other students. We call this third characteristic the principle of mathematical justification. In MMPOSI, mathematical modeling plays a twofold role. On the one hand, it allows students to develop their mathematical competence in complex situations that are familiar and relevant to them, and on the other, mathematics serves as a tool for validating the results. This mathematical validation also offers students a way to defend their products, since it is a procedure that guarantees the suitability of their recommendations for solving the problems under study. In previous studies, we observed that when students produce their own mathematical explanations of social phenomena, they tend to trust their own methods and conclusions to such a point that they generate a framework to support their decisions and freely criticize the results provided by external sources, such as information that appears in the media (Albarracín and Gorgorió [58]).

Producing safe proposals to move around the city demands complex, relevant decision-making, which entails developing a high level of responsibility and social awareness. The process leads students to question the way in which their environment is constructed, and given the doubts raised, they come up with their own ideas about how a city should be organized. Thus, group work, the connection of mathematical knowledge with reality, and the use of different technologies to collect, organize, and interpret data come together so that students can take a reasoned position on relevant social aspects that affect them. This requires the coordination of knowledge and procedures typical of different

disciplines, with which MMPOSIs can provide a way of promoting interdisciplinary work without reducing the role of mathematics to a minimum expression. They thereby play an active role as members of the community who begin to make their own decisions and develop their own ideas of what the world they live in should be like.

Finally, we would like to make it clear that because this case study was instrumental, i.e., developed with the intention of illustrating a concept, its findings should not be regarded as all-inclusive or generalizable. Given that we started from naturalistic observation to exemplify and characterize a new theoretical construct, it is obvious that this study needs to be followed up by others to continue exploring MMPOSIs at different educational levels and with different teachers, both in terms of their mathematical knowledge and their approach to mathematics. In particular, we think that it would be interesting to explore how the students' social commitment and sense of belonging to the community impact the emergence of learning opportunities during MMPOSIs and makes these learning opportunities materialize in concrete learning throughout the development of a project of this type. In this research, we saw that the students' survey of the routes led to the study of movements on a map. In this respect, we also think it would be interesting to explore how certain topics could be connected to develop projects with specific mathematical content. For this reason, we think that it would be worth studying teachers' modeling competencies, to see in particular if they can identify a priori these connections between reality and mathematical content.

**Author Contributions:** Conceptualization, L.A. and N.G.; methodology, L.A. and N.G.; formal analysis, L.A. and N.G.; investigation, L.A.; resources, L.A.; data curation, L.A. and N.G.; writing, original draft preparation, L.A. and N.G.; writing, review and editing, L.A. and N.G.; supervision, L.A. and N.G. All authors read and agreed to the published version of the manuscript.

**Funding:** This research was funded by Ministerio de Economía, Industria y Competitividad (Spain) Grant Number EDU2017-82427-R and Agència de Gestió d'Ajuts Universitaris i de Recerca (Generalitat de Catalunya) Grant Number 2017 SGR 497. The work of Lluís Albarracín is supported by Serra-Húnter program.

**Acknowledgments:** In loving memory of Roger Canudas Torralba.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## *Article* **Enhancing Computational Thinking through Interdisciplinary STEAM Activities Using Tablets**

### **L'ubomíra Valoviˇcová 1,\*, Ján Ondruška 1, L'ubomír Zelenický 1, Vlastimil Chytrý <sup>2</sup> and Janka Medová 3,\***


Received: 31 October 2020; Accepted: 24 November 2020; Published: 27 November 2020

**Abstract:** Computational thinking is a highly appreciated skill by mathematicians. It was forecasted that, in the next few years, half of the jobs in science, mathematics, technology and engineering (abbreviated as STEM, including arts as STEAM) will use some kind of computation. It is therefore necessary to enhance the learning of mathematics by collaborative problem-solving activities focused on both learning mathematics and developing computational thinking. The problems in science offer a reasonable context in which to investigate the common overarching concepts (e.g., measuring the length). An interdisciplinary STEAM collaborative problem-solving activity was designed and piloted with 27 lower secondary students aged 13.07 ± 1.21 years. Different levels of willingness to use the technology were observed and the factors influencing it were identified. We found that strong background knowledge implies high demands when controlling the used device. On the other hand, when a nice and user-friendly application was used, students did not need to perceive any control over it. After the intervention, the students' views on the tablet changed and they reported more STEAM-related functions of the device.

**Keywords:** computational thinking; STEAM education; leisure-time education

#### **1. Introduction**

To learn mathematics in the 21st century not only means obtaining mathematical proficiency, but also critical thinking, creativity and technology literacy [1]. Collaborative problem solving is one of the recommended pedagogies to promote the active learning of mathematics. Furthermore, collaborative problem solving led to better performance in standardized tests in mathematics than a traditional transmissive approach [2–5], particularly when the problems were related to the real life of the students [6] and used technology [7]. It was also reported that students educated using collaborative problem solving appreciate their knowledge of mathematics and science even in their future workplace. This influenced their academic performance and career choice [8].

Furthermore, it is predicted that, in the next few years, half of STEAM-related jobs will be in computing [9,10]. Children and young people use smartphones and tablets on a daily basis, but their use is mainly for entertainment, not for learning purposes. On the contrary, mathematicians consider the effective use of technological tools as a "valuable component of the practice of doing mathematics" [11] (p. 9). A similar perception can be given of the scientists [10,12]. This has led to the large-scale development and piloting of materials aimed at fostering computational thinking [13], but not all of them are suitable for problem-based learning. Cápay and Magdin [14] used black boxes

as the main concept for tasks developing computational thinking and they provoked very intensive reaction. Burbaite et al. [15] designed an activity where the students could learn about the physical principles of functioning an ultrasonic sensor, connecting knowledge of physics with knowledge from computer science. Students were able to gain conceptual knowledge in physics and design the algorithm at the same time. Another example of an interdisciplinary approach can be found in the work of Lytle et al. [16], aimed at an agent-based simulation with a special focus on student-perceived ownership of developed programs. Students using the use–modify–create approach felt more confident and perceived the code developed in the guided part, with their slight changes, as more familiar compared to the transmissive approach in the control group. Several studies [17,18] have shown that the design-based approach can improve the computational thinking of participating students and enhance the students' awareness of the different tasks that can be performed using the computer and their self-efficacy in using computers.

#### *1.1. Interdisciplinary Teaching*

A lot of current scientific problems can be addressed only if experts from several scientific fields collaborate together. New scientific fields (e.g., physical chemistry, biostatistics, and theoretical physics) have even been established. However, the school curriculum is divided into separate subjects. In Slovakia, even science subjects are separated to physics, chemistry and biology in secondary education [19]. Both mathematics and science education aim to enable students to understand the wonder of the world around us. They share strategies for solving problems and for scientific inquiry. These approaches include logical thinking, hypothesizing, observations, analysis and experimentation. Even university students are not used to solving practical problems and, therefore, they are not able to interpret the obtained results [20].

St. Clair and Hough [21] grouped arguments supporting an interdisciplinary approach to education into six groups. An interdisciplinary approach (i) is in agreement with the current body of knowledge about the needs of the secondary students; (ii) offers a substantial learning environment and, therefore, has a positive impact on the learning process as well as on achievement; (iii) provides students with a more holistic approach to problems; (iv) is global in content and better prepares participating students for critical citizenship; (v) improves the skills necessary for problem-solving by demonstrating different views and perspectives; (vi) encourages collaboration among teachers.

#### *1.2. Computational Thinking*

Weintrop et al. [9] stress the ability of mathematics and science to develop key skills in computation. Departing from the poor-technological view, we can focus on four main categories: "data practices, modelling and simulation practices, modelling and simulation practices, computational problem-solving practices and systems thinking practices" (p. 127). It is important to realize that computational thinking is more than using technology: it is a way of thinking while solving complex problems [22]. Computational thinking was defined by Aho as "the thought processes involved in formulating problems so their solutions can be represented as computational steps and algorithms" [23] (p. 832).

Similar to proficiency in mathematics [24], computational thinking may be also developed via problem-solving activities [25]. Gretter and Yadav [26] see "collecting, analysing and representing data, decomposing problems, using algorithms and procedures, making simulation" as the key component of computational thinking (p. 511). Bocconi et al. [27], based on studies [28,29], define six components constituting computational thinking: (1) abstraction, (2) algorithmic thinking, (3) automation, (4) decomposition, (5) debugging, and (6) generalization. Abstraction is understood as reducing the details in order to make the artefact more understandable. The essence of abstraction is competence in choosing the proper feature to hide and proper representation, so that the hiding results in easier problems with a suitable solution. Algorithmic thinking is a systematic way of thinking applied to splitting a complex problem into a series of (not necessarily ordered) steps utilising the

different tools available in the moment. Automation can be defined as a procedure aimed at saving the labour which the computer is uses to perform a (ordered) set of repetitive tasks instead of the slow and inefficient work of humans. Decomposition is a way of breaking down the artefacts into smaller parts that can be "understood, solved, developed and evaluated separately" [28] (p. 8), which makes complex systems simpler to design. Debugging is a looking-back ability when the outcomes are analysed and evaluated. Generalization, as a part of computer thinking, is connected to "identifying patterns, similarities and connections, and exploiting those features" [28] (p. 8), relating to previous experience with similar problems and adopting developed algorithms to solve the comprehensive class of similar problems. Although the positive effect of the use of technology on students' performance in mathematics and science was confirmed, very few studies investigated the use of these applications for mathematics. The results of Kosko et al. [30] suggest that integration of the applications over a three-week period significantly increased the mathematics achievements of participating students.

The ability to use the technology is not self-developing. The fact that students are able to use tablets or smartphones or any other technology for communication or browsing the internet does not imply an ability to use it for more sophisticated purposes, such as measuring the distance, temperature or size of an angle, calculating repetitive tasks or processing the measured data. It is necessary to provide students with the opportunity to experience this kind of use of the technology. The invention of mobile technologies allowed students to unplug the computers, leave the classroom and move outdoors [31].

The instrumental approach [32] seems to be a reliable framework to understand what is going on during the activities, supporting both mathematical learning and computational thinking. The technology introduced in the classroom can be considered as an artefact. Only when students learn to use it, when they develop the utilization scheme, does the artefact become a tool, an instrument [33]. The development of the utilization scheme can be described as having three levels: (1) usage schemes, (2) instrumental action schemes including gestures and operative invariants, and (3) instrumented collective activity schemes [34]. Usage schemes are directly related to the artefacts themselves. They are developed through manipulations and examinations by the artefact. Instrumented action schemes or instrument-mediated action schemes are higher-order, coherent and meaningful mental schemes, acquired from existing elementary usage schemes when a student manipulates an instrument with the aim of solving the problem. The developed schemes are specific to each activity. When an application is introduced in the classroom, students first have to become familiar with its basic features, developing the usage scheme. Only then they can use it for solving the problem and fostering the instrumental action schemes. Instrumented collective activity schemes or collective instrument-mediated activity schemes are the schemes developed in the context of collective, particularly collaborative, activity. The students are both influenced by artefacts' potentialities and constraints (instrumentation) and influencing the artefact via their preconceptions, knowledge, beliefs and usual ways of work (instrumentalisation) [34]. The two described dual processes are united in the instrumental genesis when the instrument arises as the result of the interactions between the student (subject of the activity) and the artefact [35–38].

The main aim of this study is to demonstrate the potential of interdisciplinary problem-solving activities, including several STEAM disciplines, to develop both the mathematical proficiency and computational thinking of involved students. Various activities were designed to develop computational thinking [13,14], but only a few of them were focused on the students' tendency to use technology to solve problems. In this article, we looked for the answer to the research question formulated as follows: What components of computational thinking may be developed by involving students in interdisciplinary STEAM activities using technology? How is this development manifested?

#### **2. Methodology**

The presented research was conducted with a more than 15-year longitudinal study about summer camps for lower-secondary students focused on STEAM, particularly physics and mathematics (mainly as the language of physics). The study based on the design research principles involved the following

phases: (i) the preparation and design of instructional materials, (ii) implementation of the materials, and (iii) retrospective analyses, cyclically repeating from 2006 to 2020, described in more detail in the work of Cobb [39,40]. The outputs from each year informed preparation and design in the following year. Development of the design and the overarching topic of each year are summarised in Supplementary Materials (Table S1). The camp leader had an input and primary responsibility for implementing the activity with the group of 3–4 students. In this article, we focus on one particular activity implemented in the year 2019. As the activity was held in the 14th year of the study, the design was informed by each activity implemented in previous years of the camp.

#### *2.1. Participants*

The camp in 2019 was attended by 27 students of grade 6–10 (age 13.07 ± 1.21 years), nine of whom were girls. The leaders in the group were a graduate student of Physics Education and a bachelor student of physics. The 27 participating students were divided into three big groups, and each group was further divided into three sub-groups to enable students to try the hands-on activities on their own. For some of the activities, the groups were divided into two smaller subgroups, one with younger students (YS) aged 11–12 and the second with older students (OS) aged 13–14. Written informed consent was collected from the parents of all participating students.

#### *2.2. Preparation Phase: Design of the Activity*

During all years, the camp was organized to last from Monday till Friday. Usually, there is an overarching topic for all the activities in the camp. On top of this, the overall design of the 2019 camp was focused on the development of students' skills in the area of project management including the development of related soft skills, e.g., leadership, organization and communication. The main topic of the camp was the construction of a rocket. Students were supposed to design a rocket in an environment simulating a real funding programme. They had to write a project proposal to get funding, prepare the budget and milestones, and defend their project at the ending of the camp-week. One of the required tasks in the final project was to assess their developed rocket based on various indicators, including the maximum height their rocket could achieve.

The design of the activity was led by the categorization of the strategies of interdisciplinary teaching into three groups, as proposed by Nikitina [27]: (1) contextualizing, providing a reasonable context for the interdisciplinary activity (preparation of estimating the height of the rocket), (2) conceptualizing, meaning that the activity is based on an overarching concept that is central for two or more disciplines (i.e., measuring and length) and (3) problem-centring, particularly using real-life or realistic problems in which concepts, processes and ideas of different disciplines have to be used in the solution. The main aim of the activities rooted in this strategy was to create a tangible outcome or product, in our case, a procedure to estimate the height.

In order to prepare students to solve the problem, e.g., to estimate the rocket range height, a series of three smaller activities was designed. The primary designers were two of the authors of this article, who were involved in the longitudinal study since its very beginning and therefore were informed in detail about the outcomes of the previously implemented activities. The camp leaders adjusted the ideas and prepared the worksheets for the students.

#### *2.3. Implementation and Reflection*

The designed activity was performed three times during camp-week. After each implementation, the leaders reflected on their experience with the guidance of the designers of the activity. After the first piloting, the activity was slightly changed. The second pilot did not lead to any significant change in the worksheet.

#### *2.4. Data Analysis*

The implementation of the activity was audio-recorded and transcribed. The students' actions connected to the use of technology (i.e., tablets) were analysed using the components of computational thinking, as described by Bocconi et al. [27], and the episodes where computational thinking was observable were chosen.

In order to compare the responses of the participating students in the items related to computational thinking in the beginning and at the end of camp-week, a McNemar test of symmetry [41] with Yates correction [42] was performed using the calculations, carried out in the programme STATISTICA 13.3 (StatSoft Inc., TIBCO Software, Palo Alto, CA, USA).

#### **3. Results and Discussion**

The activity was designed to teach children the procedure for measuring or estimating the height of the object. The overarching concept involved was measuring the distance. Length was seen as both the physical quantity and also the geometrical characteristics. On the other hand, the main aim of the activity was to estimate the height of the statue, so problem-centring was used as a strategy to create interdisciplinary teaching. The whole activity took approximately 90 min for each group. It was divided into the three phases: motivation and introduction, problem-solving, and concluding. Each of the three strategies took approximately 20 min.

#### *3.1. Strategy 1: Shadow*

The first strategy offered to students in order to solve the problem used measuring tape and a pole (140 cm). First, pupils measured the length of the pole when the pole was laid down on the flat surface. After that, the pole was placed to be orthogonal with the floor, and the length of the shadow of the pole was measured. The next step was to measure the shadow of the statue.

The first phase of the strategy was the introduction into similarity. Teacher explained the concept of similarity, and when she found that some of the pupils knew and understood, she left him/her to explain it to the other members of the group. The teacher facilitated the discussion and posed questions to lead the students to come up with the expression *sh* : *ss* = *ph* : *ps*, where *sh* means height of the statue, *ss* height of its shadow, *ph* means height of the pole and *ps* height of the pole's shadow. Students later expressed the height of the statue as *sh* <sup>=</sup> *ps pt* ·*ss*. The intention was to use the spreadsheet to calculate the height of the statue (*sh*). After expressing *sh*, the teacher asked the students how to calculate the height of the statue using the measured data.


The two students remaining at the stage with the Statue problem started to run the different applications in the tablet. The students had two minutes, so the remaining students did not get a chance to check more than three apps.


The students resisted using the spreadsheet prepared by their groupmate without understanding it. The teacher made them insert the data into the table and then showed them the formula written in the last column. Even though the students could see it, they were not familiar with the syntax and they were not able to see the formula they came up with before (see Figure 1). Their previous experience and usual ways of work were not satisfactory to understand the potentialities of the application.

**Figure 1.** Low student willingness to use the artefact as influenced by the low perceived control.

One of the older students gained this ability and did not expect that the younger members of his group would not understand his product. Therefore, the teacher asked the younger students to create a new file and introduced them to the basics of typing formulae in a spreadsheet. Then, the students were able to see their formula in the prepared spreadsheet and they were willing to use it for calculations (see Figure 2). The knowledge of basic syntax allowed them to use the spreadsheet while fully understanding how it works. As the students were fully aware of the method used to estimate the height of the state, they felt the need to influence the artefact in the process of instrumentalisation. In this case, students understood the formula and how it was developed, they had full background knowledge and resisted using the spreadsheet without controlling the process of computing.

**Figure 2.** High student willingness to use the artefact as influenced by the high perceived control.

This episode gives us two interesting moments. In contrast with the rest of the group, the young student YS2 did not perceive the tablet as an artefact; she intended to use it as a tool, but did not have the appropriate knowledge. She showed an unexpected level of computational thinking in the area of automation (1), understood as a labour-saving process when the computer executes repetitive tasks instead of humans [29], but she was able to come up with either a concrete application or the kind of software needed. Conversely, the older student OS1 had the utilization scheme at their disposal and was able to develop a needed spreadsheet, so he acted using the tablet as with an instrument.

We consider it a very positive sign that the younger students complained that they did not understand what the spreadsheet developed by their group-mate computed [2], and pushed the teacher to show them how to create it. The process of instrumental genesis was present in both ways: students' knowledge of mathematics led to their own development of the artefact and the unknown calculations provoked the students' perception that they need to understand how the artefact works. This need may indicate some level of evaluation [43], or at least a tendency toward the evaluation of results provided by the software. After a detailed explanation by the teacher, they could understand the process of how the formula was transformed into the spreadsheet. Even though we did not test whether they could create the spreadsheet on their own, they were able to use it, and developed an instrumented action scheme capable of solving the problem.

#### *3.2. Strategy 2: Protractor*

The second implemented strategy followed the successful solution by strategy 1, described above. The main objective of the activity was to give the students the opportunity of learning how to estimate the height of the object based on the angle between the horizon and the line connecting the observer's eyes and the object. The intention of this activity was to prepare students to use the application for measuring angles by giving them experience measuring them with a homemade tool.

This strategy did not depend on the weather, as it did not use the length of the shadow. It also needed less information compared to the shadow strategy. Instead, it used other manipulatives: a homemade protractor from cardboard and a rope. With the first of the six groups, we tried to create the protractor, but it took too long, as the students did not have satisfactory skills. As they used corrugated cardboard, the nib of the pen pierced the upper layer of the paper. Therefore, the other groups got the pre-prepared cardboard and had to finish the protractor. The students first fastened the rope to the paper. Then, they had to decide what kind of plummet they would use.


The teacher provided students with the box of different things, including the 20 g weight. All the groups intended to find an object that was easy to fasten and heavy enough to tighten the rope. One of the three groups wanted to find some object with a spike to point to the exact line. All the groups used the weight, but one of them tried to use the pen first (the nip should serve as the pointer).

After finishing the cardboard protractor, the students could finally measure the angle under which they see the top of the statue. The students were already familiar with the similarity of triangles. All the groups came up with the idea that if they knew the ratio and distance from the statue, they could calculate the height of it. All the groups started to calculate, and they came up with results different than those obtained using strategy 1. The students were confused, but the teacher asked them where they held the protractor. Then, they realized that they should subtract the height of the person doing the measuring. The results were quite different again. In each group, the teacher tried to question the students to lead them to realise that they should subtract the height of the eyes (see Figure 3), not the top of the head of the measuring person. With this correction, all the groups obtained results not very different to the ones estimated by strategy 1.

**Figure 3.** The sketch used to calculate the height of the statue based on the view angle.

In the final discussion after this strategy, the teacher provided the students with the definition of a tangent as the ratio of the length of the opposite side to the length of the adjacent side of the right triangle. The teacher put the formula into the tablet and developed a spreadsheet calculating the height of the statue based on the distance between the statue and observer and the viewing angle. Contrasting with the previous strategy, the children did not ask for an explanation of the formula. This may indicate that they did not understand the idea of the tangent in depth, but accepted the existence of it and believed that the teacher could teach it to them through a worksheet. Their background knowledge was lower and they did not perceive the need to fully control the method (see Figure 4). Furthermore, the worksheet gave the same result as the one they understood. Their knowledge was not deep enough to shape the instrument. The students were able to deal with the spreadsheet, as with the black box providing the same result. We assume that the same instrumented action scheme is needed to use the

spreadsheet as in the previous case. However, none of the students tested whether the results would be the same when measuring the height of the known object. This fact may indicate that the students did not have advanced computational thinking in the area of debugging, defined by Csizmadia et al. [28] as the evaluation of developed application by testing, tracing or critical thinking.

**Figure 4.** High student willingness to use the artefact as influenced by a similar level of background knowledge and perceived control.

#### *3.3. Strategy 3: Application*

The third strategy used to solve the statue problem involved directly measuring the height of the statue with the SmartMeasure application on a tablet. The input for the application is the height of the eyes of the observer. Then, two other measurements are carried out: measuring the distance between the observer and the bottom of the statue and measuring the viewing angle of the observer. The application then gives the height of the statue as the output. We expected that with direct use of the application, the students may be confused by the reason for inputting this particular information. The students were already familiar with the application, as they used it for measuring the length of the corridor. The students were pleased when they found out that they would use the tablets. The teacher instructed them to use the application.


After the confirmation of the teacher, the students launched the application and tried to use it without any previous instruction, but with the guidance of the teacher.


The students very smoothly used the application to measure the distance between the ground and the observer's eyes and use it as the input for the estimation of the height. We assume that the students were able to use the table so naturally because the utilization scheme for the cardboard protractor was very similar to the utilization scheme for the tablets. The relation with the previous experience was signalled by the icon of the software (1). The student YS3 said this explicitly.

The students had the same background knowledge in mathematics as in strategy 2, and the application uses the same principle to estimate the height of the object. On the other hand, their constructed protractor was fully controlled by them but, when using the tablet, they did not know how the technology measures the angle or the source calculating the height. Their background knowledge was even lower than in the previous case (Figure 5). The work was automated by the table, and the tool used was more complex.

**Figure 5.** High student willingness to use the artefact as influenced by higher complexity of the tool.

The students demonstrated a certain amount of abstraction when they were able to abstract from the concrete artefact and transfer the utilization scheme from a paper tool to an electronic one. We believe that the previous experience with the cardboard protractor enabled students to adopt the instrumented action scheme with a satisfactory depth for specific transfer. Furthermore, the simplicity of the homemade tool provided students with the opportunity to learn the process behind the estimation, and to concentrate on the mathematical basis used by the application without the disturbance caused by the more sophisticated affordances of the tablet.

The above analysis shows that there are more factors affecting the proper use of technology than the elements of computational thinking. Although students were not able to do this, they wanted to use the tablets for calculations. On the other hand, they refused to use the technology they did not understand. The need to understand the technology was influenced by both (1) the characteristics of subject (students), namely the background knowledge and perceived control over the artefact, and (2) the constraints and potentialities of the tool, its complexity and the complexity of its utilisation schemes. We believe that this is an important factor to be considered when assessing the computational thinking of students, particularly in the area of automation.

#### *3.4. Remarks about the Impacts of the Activity*

Besides the identification of the process of instrumental genesis, we would like to show how participation in the activity further influenced students' work and perception of the technologies.

Just after solving the statue problem, the students had to answer the question of which methods they would use to estimate the height of an object. All of them chose using tablets and the SmartMeasure app as the most appropriate approach, so we might assume that the artefact became an instrument. The students' reasoning varied. Students stressed the different potentialities of the application: (1) its simplicity of usage "we did not need to do anything, the tablet did the measure instead of us", "tablet needed just one number", "we can stand at one place", "we did not need any other equipment"; (2) independence of the weather "we can measure using the tablet if there is a shadow or not" or (3) the universal usage "I can install the same app to my smartphone that I always have on disposal". In the feedback for the activity, the students complained that they had to estimate the height of the statue using the shadow or protractor when the tablet offered a very convenient way to measure it. Even though they were not happy with the demanding process, we believe that the procedure gave students the opportunity to learn some new geometry, as well as stimulating their critical thinking in the further use of the technology. They were actors in the situation when they misused the technology by inputting the height of the person instead of the height of his/her eyes. When they inputted the wrong information in the application, the result was wrong.

Their experience with measuring the height of the statue was also observable in other activities performed during the week, mainly during estimating the height of their constructed rockets. When measuring the rocket range height, all the groups chose the tablet. This fact might be considered as partial evidence that the automation and abstraction components of computational thinking were developed. It is worth mentioning that younger students (age 10–11 years) referred to the measuring of the height as "statue", while older students (age 12–13 years) referred to it as "estimating the height by a protractor". The different terminology used may reflect the different levels of cognitive development of the students. While the younger students named the action according to the surface characteristic, the particular object measured in the problem, the older students focused more on the structure of the solution when the angle was used to calculate the height, so, surprisingly for them, the height was estimated by a protractor.

Besides this qualitative evidence, we may support our findings about the changed perception of the tablets from the participating students by the change that occurred in their responses to specific items in the questionnaires administered at the very beginning and at the end of the camp week. The responses are summarized in Table 1. In the first question, students were asked to list different means of measuring distance. In the pre-test, none of the participating students mentioned any electronic device, while in the post-test 17 of the 27 participating students mentioned the tablet or the SmartMeasure application (*chi*<sup>2</sup> = 15.059, *p* = 0.0001). When students listed what tablets can be useful for, they mainly listed calling and messaging in the pre-test, while in the post-test they listed different features useful for solving STEAM problems, e.g., calculator, measuring, GPS or compass - *chi*<sup>2</sup> = 5.882, *p* = 0.015 . The measurement of different characteristics was listed by 14 of 27 participating students (*chi*<sup>2</sup> = 7.118, *p* = 0.008) in the post-test. The change in students' responses supports the hypothesis that involving students in problem-solving activities when technology (i.e., tablet) is used as a tool to simplify their work can enhance their computational thinking, mainly in the automation component.


**Table 1.** Students' responses to the chosen items in pre- and post-questionnaire.

The episodes reported in the qualitative analysis indicate that the abstraction can be enhanced too. The process of the instrumental genesis of the students was not studied in great detail, but we assume that using the physical aid before the virtual one enhanced students' understanding of the process of how height is obtained based on the viewing angle. Similar results are reported by Vagova et al. [44], who used 3D-printed manipulatives before the virtual manipulation with cubes in the computer environment. The findings also support Lieban's [45] conjecture that the results obtained using a virtual environment should be confronted by results using physical aids. The implementation of both physical manipulatives and computers enhances the mathematical modelling skills of secondary students [46].

#### *3.5. Limitations of the Study*

We are aware of the limitations of the study, mainly the fact that the participants of the study were not regular students, but students choosing a summer camp focused on physics, so their enthusiasm and willingness to solve the problem is much higher than in the common population. Their interest in STEAM subjects allowed us to analyse the processes appearing when introducing technology without the need of demanding classroom management. When generalising the results, one should keep in mind that the sample comprised only 27 students. The arrangement of the activities in the summer camp, where a small group of students work under the supervision of one or two camp leaders, allowed us to see what is going on during the process of instrumentation genesis in detail. On the other hand, neither the teachers' (camp-leaders) work nor the teachers' influence on the process were analysed. Furthermore, the activity in which they were involved provided a plausible context to solve problems in mathematics and/or physics, and we therefore assume that similar processes would happen in usual classrooms.

#### **4. Conclusions**

The main aim of the presented study was to demonstrate the effect of incorporating ICT in solving STEM interdisciplinary problems on the computational thinking of the students. The concept of measuring connects mathematics and physics, and the problem-based orientation of the analysed activity supported the interdisciplinary learning of participating students. The mathematical apparatus was used in the procedural way or as a toolbox, but enhanced the mathematical repertoire of models of right-angled triangles by a nontrivial separated model. On the other hand, the students could develop skills in measuring the distance and length by concrete tools, as is usual in physics. The progressive involvement of technology in the problem-solving process allowed students to understand the way the application works and enhanced their computational thinking in a very meaningful way. They were able to progressively develop their utilisation schemes.

The detailed process of instrumental genesis during a problem-solving activity involving mathematics and physics should be studied further. The influence of different orchestrations should be researched in order to maximize the students' gain from the educational activity.

**Supplementary Materials:** The following are available online at http://www.mdpi.com/2227-7390/8/12/2128/s1, Table S1: Development of the design of the camp in years 2006–2020.

**Author Contributions:** Conceptualization, L'.V. and J.M.; methodology, L'.V. and J.O.; software, J.O.; validation, L'.Z.; formal analysis, J.M. and V.C.; investigation, L'.V.; resources, L'.V.; data curation, V.C.; writing—original draft preparation, J.M.; writing—review and editing, J.M.; supervision, L'.Z.; project administration, L'.V. and J.M.; funding acquisition, L'.V. All authors have read and agreed to the published version of the manuscript.

**Funding:** This work was supported by the Slovak Research and Development Agency under the contract No. APVV-15-0368, by the Scientific Grant Agency of the Ministry of Education, Science, Research and Sport of the Slovak Republic and the Slovak Academy of Sciences under the contract No. VEGA 1/0815/18, and by European Commission within the ERASMUS+ programme under the project no 2020-1-DE03-KA201-077363.

**Acknowledgments:** The tablets used in the study were loaned by the civic association VAU (More than Learning), Slovakia.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## *Article* **Factors Influencing Mathematics Achievement of University Students of Social Sciences**

#### **Alenka Brezavšˇcek 1, Janja Jerebic 1,2, Gregor Rus 1,3 and Anja Žnidaršiˇc 1,\***


Received: 29 October 2020; Accepted: 25 November 2020; Published: 1 December 2020

**Abstract:** The paper aims to investigate the main factors influencing the mathematics achievement of social sciences university students in Slovenia. A conceptual model was derived where three categories of variables were taken into account: attitude towards mathematics and math anxiety, engagement in learning activities, and attitude towards involving technology in learning mathematics. Data were collected for seven consecutive academic years and analysed using Structural Equation Modelling (SEM). The results showed a very high coefficient of determination for mathematics achievement (0.801), indicating that variables "Perceived Level of Math Anxiety", "Self-Engagement in Mathematics Course at University", and "Perceived Usefulness of Technology in Learning Mathematics", together, explain 80.1% of the total variance. Based on our findings, we can conclude that teaching in secondary school is a crucial determinant for success in mathematics at university. It is essential to identify the best methods for secondary school math teachers which will help them give future students better entry-level knowledge for universities. These methods will, hopefully, also improve the level of mathematics self-confidence, as well as lower the level of math anxiety, which all considerably affect the performance of students in university mathematics.

**Keywords:** mathematical education; good practices in mathematics education; mathematics achievement; influencing factors; university; social sciences; structural equation modelling (SEM)

#### **1. Introduction**

Mathematical skills have long been recognised as essential not only for academic success but also for efficient functioning in everyday life [1]. By studying mathematics, we train accuracy, consistency, and mental discipline, which are essential skills needed for effective and responsible problem solving and decision making in everyday life. Due to the global awareness of the importance of mathematical knowledge on the one hand, and the concern expressed for many years at various levels of education about underachievement in mathematics [2], the performance of students in mathematics from primary school to higher education is still a topic of concern [3].

After reviewing publicly available databases, we found that the majority of studies on mathematical performance and achievement are focused on either primary or secondary education or both (see, e.g., [4–19]). Studies focusing on higher education (i.e., tertiary or post-secondary education), which is the subject of our research, are less represented (see Section 2.2).

Knowledge of mathematics has often been cited as crucial for several disciplines in higher education, including technical fields, engineering, economics, and finance, as well as agriculture, pharmaceuticals, and health sciences [20–22]. Since mathematical knowledge offers widespread application, social sciences university programs around the world require their students to take at least one mathematics course. Their students gain essential mathematical knowledge and develop

the analytical and computational skills they need in their field of specialisation. Unfortunately, mathematics in university courses has often been identified as a significant obstacle for students and as one of the main reasons for dropping out of university [22]. This problem is particularly pronounced in non-scientific university programs, where the failure rate in mathematics can easily exceed 30 percent [23]. Since poor performance in mathematics indirectly affects the overall academic performance of students, there is an urgent need to investigate the factors that have contributed to poor performance in mathematics in higher education.

This study aims to develop a conceptual model to analyse the factors that influence the mathematical performance of university students of social sciences. The background knowledge of secondary mathematics, the attitude towards learning mathematics with technology, the perceived level of math anxiety, and the self-engagement and motivation during the mathematics course were taken into account. In our effort to investigate the relationships between the model components, we applied the Structural Equation Modelling (SEM). The results were then presented and discussed.

The rest of the paper is structured as follows. First, the results of a relevant literature review are outlined. The research model and the proposed hypotheses are developed. Furthermore, the methodology of our empirical study is explained. The results are presented and discussed. Finally, the conclusions are outlined based on research implications, the limitations of the study, and future research recommendations.

#### **2. Review of Related Literature**

#### *2.1. Factors that Influence Mathematics Performance*

To determine the predictors of mathematics achievement among various groups of individuals, a large body of studies have been conducted over the past several decades. Since education is a complex process with many variables interacting in a way that affects how much learning takes place [24], the authors express the diverse and complex nature of factors associated with mathematics performance. To provide a comprehensive and consistent insight, some authors try to classify the factors into various categories with related properties.

Papanastasiou [24] distinguishes between internal and external factors influencing mathematics performance. Internal factors are those related to the test (exam) material, while external factors refer to the environment which surrounds the individual as well as to his unique persona (e.g., socio-economic level and educational background of the family, the school climate, the language background, and students' attitudes toward mathematics).

Patterson et al. [4] express that factors associated with mathematics achievement range from the dynamics of individual cognitive processes to the social and environmental factors that affect a particular student.

Furthermore, Enu, Agyman, and Nkum [25] ascertained that the successfulness of learning mathematics is contingent on a myriad of factors: students' factors (entry behaviour, motivation, and attitude), socio-economic factors (education of parents and their economic status), and school-based factors (availability and usage of learning materials, school type, and teacher characteristics).

A comprehensive and systematic literature review on influential factors found to be responsible for success or failure in mathematics is provided by Kushwaha [26]. The author divided the factors under three general heads as follows:

• Psychological variables: attitude towards mathematics, intelligence, math anxiety, self-concept, study habits, mathematical aptitude, numerical ability, achievement motivation, cognitive style, self-esteem, interest in mathematics, test anxiety, reading ability, problem-solving ability, mathematical creativity, educational and occupational aspiration, personal adjustment, locus of control, emotional stability, and confidence in math.


In many studies on mathematics achievement, the psychological, social, biographical, and instructional variables were studied simultaneously, where the authors have focused on a limited number of factors or themes with the aim of demonstrating their role in the complex process of mathematics education. Kushwaha [26] found out that the most preferential factors of the investigation in the category of psychological variables are intelligence, attitude towards mathematics, self-concept, numerical ability, and math anxiety. Among social variables, the factors which were considered very widely are socio-economic status, parental involvement, and parents' education, while among biographical variables, the most frequently considered factor is gender.

#### *2.2. Investigation of Mathematics Performance at the Tertiary Level of Education*

Due to specific characteristics of the target population, we believe that research results relating to different levels of education (primary, secondary, and tertiary) cannot always be directly compared. Therefore, we have limited our attention to studies related to the tertiary level of education, as this is the subject of our research.

Concerns regarding the problem of unsatisfactory mathematics performance have been reported internationally. Most of these studies are related to developing countries, such as Malaysia [2,3,23,27–30], Iran [31], Nigeria [32], Ghana [25], and the Philippines [33,34]. Studies relating to the field of Australia are also quite common [20,21,35–38]. The territory of the USA and Canada seems to be somewhat less represented [39,40], while studies dealing with mathematics achievement in European higher education are very rare. In this regard, we found a few papers from the following countries: the UK [41], Finland [42], Spain [43], Ireland [44], Germany [22], and Sweden [45]. In our opinion, the origin of the study is an important factor that should be considered when comparing the results. Namely, specificities of a particular national education system, as well as cultural differences between different parts of the world, can lead to significant differences in conclusions.

The majority of the studies addressed mathematics performance in relation to certain selected factors. As one of the most influential sources and predictors of underachievement in mathematics at the tertiary level of education, the authors consider the insufficient level of mathematical background from secondary education (see, e.g., [2,20,21,36–40,42,44]).

Furthermore, many authors note that math anxiety also plays an important role in mathematics achievement (see, e.g., [3,23,27,32–34,43,45]).

Among other influencing factors, the following are also exposed: attitudes toward mathematics and/or self-confidence with regard to mathematics (see, e.g., [25,27,40,43]), mathematical self-efficacy and student engagement [41], academic self-beliefs [42], learning motivation (see, e.g., [22,25]), learning strategies and/or availability of teaching resources (see, e.g., [22,25]), importance of mathematics [23], teaching style ([31,33]), parent's profile [33], mathematics class size (see, e.g., [2]), gender (see, e.g., [2,28,32]), and age [32].

In our opinion, the study discipline is also a parameter that should not be neglected when selecting potential influencing factors of mathematical performance. Namely, factors that are relevant for technical, engineering, and other science-oriented studies are not necessarily relevant for students of social sciences and humanities or similar courses of study. In our experience, the first group of students expresses a much higher positive attitude towards mathematics than the second. This is also consistent with the results of Núñez Peña, Suárez-Pellicioni, and Cabré [43], which showed that the students who received good/excellent grades in mathematics were mainly from scientific and technological itineraries, while those who failed had mainly studied the humanistic and social syllabuses.

#### *2.3. Methodology Adopted for Studying the Phenomenon of Mathematics Achievement*

A detailed review of the related literature revealed that many researchers used descriptive research methodology. To collect the required data, they used suitable tools (standardised, well-known from the literature, or self-developed). Beside descriptive analyses, collected data have been mostly subjected to independent samples *t*-test (see, e.g., [25]), analysis of variance (see, e.g., [21,31,37,43]), correlation techniques (see, e.g., [22,27,28,33,45]), or to regression (linear, multiple) analyses (see e.g., [2,3,32,34]). Studies which applied factor analysis (see, e.g., [23,30]), principal component analysis (see, e.g., [36]), discriminant analysis (see, e.g., [2,44]), or mixed-effects models [40] are relatively rare.

Undoubtedly, the results of descriptive research would provide a solid basis for selecting the most effective variables and formulating a hypothesis accordingly [26]. However, the influencing factors are often interdependent variables. Therefore, more sophisticated techniques are needed to study the relationships between them. Definitively, one of them is Structural Equation Modelling (SEM), which enables analysis of relationships between latent and observed variables simultaneously [46]. In an in-depth review of the related literature on mathematics performance, we found only three applications of SEM. Two of them, [8,12], refer to secondary education level in a specific geographical area (Turkey, the city of Konya and its surroundings). The only application of SEM for the analysis of mathematical achievement in higher education was found in [42], where SEM was used to examine the relationships between prior knowledge, academic self-beliefs, and previous study success in predicting the achievement of university students participating in an obligatory mathematics course within a mathematics program.

The literature review allows us to conclude the following:


These statements provided the fundamental starting points for our research.

#### **3. Research Model Development**

It is well established in the literature that mathematical performance is influenced by numerous factors, including psychological, social, biographical, educational, and other factors, which are often not independent of each other and can influence each other. Furthermore, some of the influencing factors are very complex, so it is necessary to divide them into sub-variables and find out how each sub-variable is related to mathematical success [26].

Based on the findings in the literature and the results of our preliminary studies [47], we assume that students' performances in mathematics are influenced by at least the following dimensions: their attitude towards mathematics (including mathematics anxiety), their engagement in learning activities (including background knowledge), and their attitude towards integrating technology into mathematics education.

#### *3.1. Attitude towards Mathematics and Math Anxiety*

Much research has been conducted to examine students' attitudes towards mathematics, and most authors agree that it plays a vital role in the process of teaching and learning mathematics [30]. Results show that a positive attitude towards mathematics has a significant impact on effective student engagement and participation and will increase students' success in mathematics (Khoo and Ainley (2005), as cited in [30]). Furthermore, valuing the importance of mathematics was also claimed to have a positive effect on students' mathematics performances [23]. A positive attitude is also related to

students' self-confidence, which refers to their belief in their cognitive capacity to learn or perform actions to achieve intended results [42]. We believe that those who have confidence in their ability to perform well also expect success in a particular task.

On the contrary, it has been identified that fear of mathematics or mathematics anxiety, educational issues, and values and expectations towards mathematics can be treated as causes of low mathematics achievements among students [23]. Mathematics anxiety (also math anxiety) can be defined as a person's negative affective reaction to situations involving numbers and mathematical calculations in both academic and daily-life situations [48]. Math anxiety, being considered to have an attitudinal component, is also considered to be one dimension of attitude to mathematics and is considered as one of the severe problems that affects mathematics education (see, e.g., [48–54]).

The majority of the studies that examine the influence of math anxiety on mathematics performance in higher education report a significant relationship between math anxiety, mathematical thinking, and attitudes towards mathematics. Students with a higher level of math anxiety tended to score lower in their mathematical thinking, their attitudes to mathematics, and, consequently, their performance, and vice versa (see, e.g., [3,27,33,34,43]).

Similar to other types of anxiety, math anxiety is a complex set of multidimensional aspects in the form of cognition, affective, somatic, and behavioural reactions [55]. Due to its complexity, there is no unique and transparent measure of math anxiety. Several researchers have argued that a mathematics anxiety instrument should be bi-dimensional and concise, contrary to the unidimensional multiple-item instruments used in the past (Mahmood and Khatoon (2011), as cited in [56]). A systematic and chronological literature review of available instruments is provided by Zakariya [56]. One of the most extensively used mathematics anxiety instruments is the Mathematics Anxiety Rating Scale—MARS [57]—and its revised version, the Revised Mathematics Anxiety Rating Scale—RMARS [58].

#### *3.2. Engagement in Learning Activities*

Linnenbrink and Pintrich [59] divide student engagement in the classroom into three distinct components: behavioural engagement, cognitive engagement, and motivational engagement. Behavioural engagement is observable behaviour seen in the classroom that relates to the efforts students are putting into mathematical tasks and students' relations to each other and to the teacher in terms of their willingness to seek help, attendance at the classes, etc. Cognitive engagement recognises that a student appearing to work on a mathematics problem is not necessarily indicative of the student fully engaging mental faculties in trying to complete it. Motivational engagement is the personal interest that the student has in the subject, the utility that the student feels the subject brings, and, finally, the general importance of the subject to longer-term goals or desires. All three components of engagement are likely correlated. That is, if students are cognitively and motivationally engaged, they are likely to be behaviourally engaged. The literature suggests (and is supported by empirical evidence) that all three components of students' engagement are related to outcome measures of learning and achievement (Pajares and Miller (1994), as cited in [41,59]).

However, many authors emphasised the importance of an appropriate mathematical background from secondary school and its influence on success in mathematics at the tertiary level. Studies conducted in various parts of the world documented prior mathematical attainment to be a significant predictor of performance and progress in higher mathematics education [44]. Similarly, a weak mathematical background on entering higher education is reported as one of the fundamental reasons for and predictors of poor student performance [44].

The background knowledge of secondary mathematics is usually measured by achievement in the secondary school leaving qualification. A significant positive correlation has been revealed between students' grades of the secondary school leaving qualification and their performance in mathematics at the university level (see, e.g., [2,37,40]).

Furthermore, some studies report positive and facilitative effects of prior knowledge on learning (Dochy, Segers, and Buehl (1999), as cited in [42]). The authors revealed that students who were able to operate at a higher cognitive level at the beginning of the course, by applying their knowledge and by solving problems, were more likely to perform better than the students whose prior knowledge consisted mainly of facts and a surface-level understanding of the issue. Moreover, inaccurate prior knowledge and misconceptions within a specific domain can make it difficult for students to understand or learn new information.

#### *3.3. Attitudes towards Involving Technology in Learning Mathematics*

Computer-based technologies are now commonplace in the classroom, and the integration of these media into mathematics teaching and learning is supported by government policies in most developed countries [60]. The use of technology for learning mathematics is one of the main issues for leading professionals involved in mathematics education at different levels of education (e.g., ERME—European Society for Research in Mathematics Education; NCTM—National Council of Teachers of Mathematics). A review of recent CERME (Congress of the European Society for Research in Mathematics Education) research is presented in [61]. At the same time, Li and Ma [62] provide a systematic literature review and a comprehensive meta-analysis of the existing empirical evidence on the impact of computer technology on mathematics education.

The literature reports many positive effects of integrating technology into mathematics education. It enables educators to create powerful collaborative learning experiences that support problem solving and flexible thinking. Therefore, the use of technology is seen as a useful tool for promoting mathematics learning [62]. Furthermore, Attard and Holmes [63] recently noted that teachers use technological tools to enhance their awareness of students' individual learning needs and to promote student-centred pedagogies, leading to greater student engagement with mathematics. On the other hand, the results of [64] suggest that the use of educational technologies generally has a positive effect on mathematics achievement in comparison to traditional methods, where the most remarkable effect has been experienced with the application of computer-assisted instructions. Barkatsas, Kasimatis, and Gialamas [65] reported positive attitudes among students towards learning mathematics with computers, even if they are not confident in using computers or express negative attitudes towards mathematics. They experienced the benefits of technology in learning mathematics, and they aim to improve mathematics performance via the use of technology.

Additionally, Al-Qahtani and Higgins [66] examined the impact of e-learning, blended learning, and classroom learning on students' achievement to determine the optimal use of technology in teaching. They confirmed a statistically significant difference between the three methods in terms of students' performance, favouring the blended learning method. The use of new and different technologies in studying a subject can increase students' enthusiasm and provide them with additional skills. Similarly, the analyses of Lin, Tseng, and Chiang [67] showed that the blended learning experience benefited the students in the experimental group, as it had a positive effect not only on learning outcomes but also on their attitude towards studying mathematics in a blended environment. Moreover, web-based learning systems and electronic materials allow users to repeat exercises and to learn simultaneously. This learning model helps to overcome time and space constraints in the classroom [60]. However, today's use of ICT coupled with the global crisis being experienced of COVID-19 makes e-learning not only a possible but also a necessary teaching method [68].

#### *3.4. Conceptual Model and Hypotheses*

Arising from the discussion in the previous subsections, we present our conceptual model in Figure 1, and summarise the propose d hypotheses as follows:

**Figure 1.** Conceptual model of relationships among the factors influencing social sciences students' mathematics achievement.

### **H1a:** *Mathematics confidence negatively a*ff*ects perceived level of math anxiety.*

**H1b:** *Behavioural engagement negatively a*ff*ects perceived level of math anxiety.*

**H2:** *Perceived level of math anxiety negatively a*ff*ects achievement in mathematics.*

**H3:** *Background knowledge from secondary school positively a*ff*ects self-engagement and motivation to fulfil obligations during a mathematics course at university.*

**H4:** *Self-engagement in a mathematics course at university positively a*ff*ects achievement in mathematics.*

**H5:** *Confidence with technology positively a*ff*ects perceived usefulness of technology in learning mathematics.*

**H6:** *Perceived usefulness of technology in learning mathematics positively a*ff*ects achievement in mathematics.*

#### **4. Materials and Methods**

#### *4.1. Measurement Instruments and Data Collection*

For our research, a three-part questionnaire was prepared. The first part served to collect students' socio-demographic data (gender, age, year of study, and study course) and data on students' background knowledge in mathematics from secondary school. The second part was designed to measure the level of math anxiety as perceived by students. In the last part, a scale for monitoring students' attitudes towards mathematics, technology, and towards involving technology in learning mathematics was used. In addition, we provide data on students' engagement in the university mathematics course and mathematics achievements, which were added to the database.

#### 4.1.1. Measuring Students' Background Knowledge from Secondary School

Three variables were used to measure students' Background Knowledge from Secondary School (BKSS):


#### 4.1.2. Measuring Students' Level of Math Anxiety

To explore the estimate of the Perceived Level of Math Anxiety (PLMA) among students, we used the reduced version of the Math Anxiety Rating Scale—RMARS [58]—which has been demonstrated to be highly reliable [54]. Balo ˘glu and Zelhart [69] employed an exploratory factor analysis on a proposed 25-item questionnaire to explore the relationships between items and to identify the underlying factors. Three factors were identified, and five items were omitted from the scale. Therefore, a simplified version of the scale of 20 items was used in our research. This scale evaluates 20 situations which may cause math anxiety, divided into three dimensions:


Students were asked to indicate their level of anxiety associated with each item on a 5-point Likert-type scale from 1 ("no anxiety") to 5 ("high anxiety").

4.1.3. Measuring Students' Attitude towards Mathematics, Technology, and Involving Technology in Learning Mathematics

A scale for assessing students' attitudes towards mathematics, technology, and the learning of mathematics with technology was adjusted from [70]. We used four out of the five constructs identified by the factor analysis for the Mathematics and Technology Attitudes Scale—MTAS—in [70]:


#### *Mathematics* **2020**, *8*, 2134

Students were asked to indicate their level of agreement with each of 16 statements on a 5-point Likert-type scale from 1 ("I do not agree at all") to 5 ("I agree completely").

4.1.4. Measuring Students' Self-Engagement and Achievement in Mathematics Course at University

The Self-Engagement of students in a Mathematics Course at University (SEMCU) was measured by two variables:


Mathematics Achievement (MA) was measured with the percentage of points achieved in the final exam. There are two ways to obtain a final grade in the university mathematics course. Either the student collects at least 50% of the points in three written mid-term exams or the student passes a final written exam (by achieving at least 50% of the points). Additional points, as described in the previous paragraph, are added to the student's percentage of points obtained in the exam or mid-term exams if they have achieved more than half of the points. In this way, their final grade in the mathematics course can be increased by one grade or, in rare cases, by two grades. The dataset, therefore, contains the value of the variable MA as the percentage of achieved points, regardless of the form of the exam (final or mid-term).

#### *4.2. Data Collection*

Data were collected for seven consecutive academic years (from 2013–2014 to 2019–2020) at the beginning of the mathematics course at the Faculty of Organizational Sciences, University of Maribor, Slovenia. All 1st level students were invited to participate in the research. Participation in this research was voluntary. The online questionnaire was distributed to the students via the e-learning environment Moodle. At the end of each academic year, we supplemented the data with the assessment of students' self-engagement and final achievement at the mathematics course. After that, the data were anonymised.

All subjects gave their informed consent for inclusion before they participated in the study. The study was conducted in accordance with the Declaration of Helsinki, and the protocol was approved by the Ethics Committee for Research in Organizational Sciences (514-3/2020/3/902-DJ).

#### *4.3. Statistical Methods*

Data were analysed using the two-stage approach to the structural equation modelling (SEM) approach (see, e.g., [71–73]). Analyses were performed using R-package lavaan [74–76] and e1071 [77] for assessing univariate normality.

The standard estimation method in SEM, maximum likelihood, assumes multivariate normality. Tests designed to detect violations of multivariate normality, including Mardia's test, have limited usefulness, since small deviations from normality in large samples could be denoted as significant [73]. Therefore, multivariate normality was assessed by examining univariate frequency distributions,

including histograms, skewness, and kurtosis, and values for skewness and kurtosis between −2 and +2 are considered acceptable in order to prove normal univariate distribution [78].

The first step of SEM involves validation of the measurement model. A confirmatory factor analysis (CFA) was used to validate the measurement instrument in order to determine how well the measured items reflect the theoretical latent variables. A construct validity was investigated in order to determine how well a set of measured items actually reflects the corresponding theoretical latent variable. To assess construct validity, convergent validity and discriminant validity were examined. As suggested in [79,80], convergent validity was examined by:


In the second step of the data analysis, SEM was used to test the structural relationships among the latent variables. The results of SEM are presented with the values of the standardised path coefficient β together with its z-values and denoted the significance level. For each of the endogenous latent variables, a coefficient of determination (*R*2) was also calculated, which shows the percentage of the explained variance by the set of variable predictors.

#### **5. Results**

#### *5.1. Sample Characteristics*

In total, 347 students collaborated in the study. Among them, 45.8% were men, while 54.2% were women. The average age of participants was 21.2 years (with a standard deviation of 1.74 years), ranging from 18 to 32 years.

#### *5.2. Descriptive Statistics*

Background Knowledge from Secondary School.

We analysed the respondents' graduation grades from secondary school. The results showed that 9.5% of the respondents completed secondary school with a grade 2 (sufficient), more than half (50.7%) of them achieved a grade 3 (good), 32.9% a grade 4 (very good), and 6.9% a grade 5 (excellent).

In addition, we checked their grades in a mathematics course in the last (fourth) year of secondary school. It turned out that 26.2% of the respondents achieved a grade 2 (sufficient), 41.2% a grade 3 (good), 25.6% a grade 4 (very good), and 6.9% a grade 5 (excellent). The average grade in mathematics in the last year of secondary school was 3.1, with a standard deviation of 0.88.

Finally, we examined the mathematics achievement at the secondary school leaving exam Matura. Only 83.6% of the respondents took mathematics at the Matura or Vocational Matura. Among them, 26.2% received a grade 2 (sufficient), 35.5% a grade 3 (good), 30.7% a grade 4 (very good), and 7.5% a grade 5 or higher (excellent). The average grade in mathematics at the Matura examinations was 3.2, with a standard deviation of 0.92.

The highest absolute values of skewness for three variables describing background knowledge were 0.35 and 0.72 for skewness and kurtosis, respectively, indicating fairly normally distributed variables [78].

#### 5.2.1. Attitude towards Mathematics and Math Anxiety

Descriptive statistics for items related to students' attitude towards mathematics and math anxiety are presented in Table 1. First, eight statements related to students' attitude towards mathematics (represented by constructs MC and BE, which were adjusted from MTAS) are listed. On average, the students agreed most with the statement "I am confident that I can overcome difficulties in math problems" (M = 3.95) and least with the statement "I am confident in my skills at mathematics" (M = 3.19). The lowest anxiety was assessed for RMARS items from the NTA construct, where the

lowest average value belongs to the statement "Being given a set of subtraction problems to solve" (M = 1.54). Respondents perceived the highest level of anxiety when "Being given a 'pop' quiz in a math class" (M = 3.80). Values of skewness were in the range from −0.81 to 1.55 and values of kurtosis ranged from −0.91 to 1.70, indicating a normal univariate distributions [78].


**Table 1.** Descriptive statistics for items related to students' attitude towards mathematics and math anxiety.

#### 5.2.2. Attitude towards Involving Technology in Learning Mathematics

Table 2 presents descriptive statistics for eight statements related to students' attitude towards involving technology in learning mathematics (represented by the constructs CT and PUTLM from MTAS). On average, the lowest agreement was expressed with the statement "It is more fun to learn mathematics if we are using a computer" (M = 3.20), while the highest average value belongs to the statement "I can use DVDs, MP3s, and mobile phones well" (M = 4.27). The highest absolute values of skewness and kurtosis were 0.93 and 0.91, respectively, and were considered acceptable in order to demonstrate a normal univariate distributions [78].


**Table 2.** Descriptive statistics for the items related to students' attitude towards involving technology in learning mathematics.

#### 5.2.3. Self-Engagement and Achievement in Mathematics Course at University

More than a quarter of the respondents (28.5%) did not take the opportunity of additional points, while a quarter of the respondents received eight additional points or more. The overall average number of additional points earned was 4.6, with a standard deviation of 4.25. If we consider only those who solved at least one problem given additionally, the average value of additional points is 6.4 (SD = 3.70). The values of skewness and kurtosis were 0.55 and −0.95, respectively. In the left panel of Figure 2, the histogram and boxplot of additional points earned (without zeroes) are presented.

**Figure 2.** Boxplots (with an average denoted by an asterisk) and histograms for additional points (**left**), points from eActivities (**middle**), and mathematics achievement (**right**).

The respondents collected between 25 and 99.2 points from eActivities (e-lessons and quizzes in Moodle), with an average of 73.9 and a standard deviation of 11.83 points. Half of the respondents earned, on average, between 65.7 and 82.6 points (middle panel in Figure 2). The values of skewness and kurtosis were −0.94 and 0.92, respectively.

Mathematics achievement is measured as the percentage of points in the final examination and can range from 0 to 113 points (with additional points). Five respondents received zero points, while the highest score was 109.7 points. The average score was 66.4, with a standard deviation of 22.86 points. A quarter of the respondents received 58.0 points or less, while the quarter of the most successful respondents received at least 82.7 points (right panel in Figure 2). The values of skewness and kurtosis were 0.55 and −0.95, respectively.

#### *5.3. Construct Validity of the Measurement Model*

CFA was used to evaluate the measurement model. Construct validity was examined through evaluation of convergent validity and discriminant validity. First, the standardised factor loadings were examined. Five measured items were sequentially omitted from the model due to factor loadings bellow 0.5: MTA8: λ = 0.287; MCA1: λ = 0.428; BE1: λ = 0.469; BE2: λ = 0.432; and NTA1: λ = 0.494.

The unstandardised and standardised factor loadings of the final measurement model, together with corresponding z-values for each measured item, are presented in Table 3. It can be seen that all standardised factor loadings exceed a threshold of 0.5 for convergent validity, while 78% of values exceed even the threshold of 0.7.

**Table 3.** Parameter estimates, error terms, and z-values for the measurement model.


 <sup>a</sup> Indicates a parameter fixed at 1 in the original solution. Fit indices: χ<sup>2</sup> = 1294.9, *df* = 570, χ2/*df* = 2.27, comparative fit index (CFI) = 0.908, root mean square error of approximation (RMSEA) = 0.061, 90% confidence interval for RMSEA = (0.061, 0.065).

The values of CR and AVE for all latent variables of the final measurement model are presented in Table 4. The CR values of each latent variable easily fulfil the criterion CR > 0.7, except for SEMCU being equal to 0.601. AVE values for all nine latent variables are above the desired threshold of 0.5. According to the obtained results, the convergent validity for the set of latent variables and corresponding items in the measurement model can be confirmed. Therefore, all measured items included in the final model are significantly related to the corresponding latent variable.


**Table 4.** Composite reliability (CR), average variance extracted (AVE), square root of AVE (on the diagonal), and correlations among the latent variables.

a—the square root of AVE.

To assess the discriminant validity of the measurement model, the square root of the AVE of each latent variable is compared to the correlations between the latent variables. The correlations among the latent variables are given in the right panel of Table 4, while on the diagonal, the values of the square root of AVE are presented. The values of the square root of AVE for the corresponding latent variables are all greater than the inter-variable correlations. This indicates that the discriminant validity can be determined for all latent variables.

The overall fit of the measurement model was assessed based on a set of commonly used fit indices. Since χ<sup>2</sup> statistics itself is sensitive to the sample size, the ratio of χ<sup>2</sup> to the degrees of freedom (*df*) was used. An obtained ratio lower than 3 (χ2/*df* = 2.27, χ<sup>2</sup> = 1294.9, *df* = 570) indicates an acceptable fit [81]. The value of the comparative fit index (CFI) is above 0.9 (CFI = 0.908) and, hence, according to [80], indicates an adequate model fit. The root mean square error of approximation (RMSEA) of our measurement model is equal to 0.06, and the upper bound of RMSEA 90% confidence interval (0.061, 0.065) is below 0.08, as suggested by [82].

#### *5.4. Evaluation of the Structural Model and Hypotheses Testing*

SEM was used to test the predicted relationships (as shown in Figure 1) among the constructs of our model.

First, the goodness of fit of the structural equation model was evaluated. The results show that the model has a good fit according to the following indices: χ2/*df* = 2.37 (χ<sup>2</sup> = 1458.0, *df* = 614), CFI = 0.897, and RMSEA = 0.063 with its 90% confidence interval (0.059, 0.067).

Second, the structural paths were evaluated. The results are presented in Table 5 and Figure 3. The values of the standardised path coefficient β and corresponding z-values are listed. Each path coefficient β is interpreted in terms of magnitude and statistical significance. For each endogenous latent variable, the coefficient of determination (*R*2) was calculated. The results are shown in Figure 3. For the second-ordered factor PLMA, the loadings to the three first-ordered factors are written in grey in Figure 3.


**Table 5.** Summary of hypotheses testing for the structural model.

\*\* *p* < 0.01; \*\*\* *p* < 0.001.

**Figure 3.** Structural Equation Modelling (SEM) model of relationships among the factors influencing social sciences students' mathematics achievement.

Based on the values of the standardised path coefficients and corresponding z-values, each of the proposed research hypotheses in Section 3.4 is either supported or rejected. A summary of the hypotheses testing is given in Table 5, which shows that 5 out of 7 hypotheses were supported. The predictive capability of the proposed model is satisfactory because all values of *R*<sup>2</sup> are higher than 0.1 (suggested by Falk and Miller (1992), as cited in [83]), while the coefficient of determination *R*<sup>2</sup> for Mathematics Achievement is extremely high since it equals 0.801.

The results confirmed that hypothesis H1a could be supported at a 0.1% significance level (H1a: β = −0.669, z = −6.667), while hypothesis H1b could not be supported (H1b: β = −0.131, z = −1.824) at a 5% significance level. The hypotheses from H2 to H5 were all supported at a significance level of 0.1% (H2: β = −0.243, z = −5.307; H3: β = 0.328, z = 3.781; H4: β = −0.131, z = −1.824; H5: β = −0.131, z = −1.824). Furthermore, the hypothesis H6 could not be supported at a 5% significance level (H6: β = −0.047, z = −1.189). Finally, we found out that PLMA, SEMCU, and PUTLM, together, explain 80.1% of the total variance in MA.

#### **6. Discussion and Conclusions**

In this study, relationships among factors influencing social sciences students' mathematics achievements were examined using Structural Equation Modelling (SEM). The factors considered in the study were divided into three categories:


The first category included three main variables: mathematics confidence, behavioural engagement, and perceived level of math anxiety. Since negative attitudes toward mathematics and the negative influence of math anxiety are often identified in the literature as important predictors of underachievement in mathematics, we assumed that both mathematics confidence and behavioural engagement negatively affect the perceived level of math anxiety, which also negatively affects the mathematics achievement. Our assumptions have only been partially confirmed. The results showed a strong negative influence of mathematics confidence on the perceived level of math anxiety (H1a), while the influence of behavioural engagement does not seem to be significant (H1b). It was also confirmed that the perceived level of math anxiety has a negative effect on mathematics achievement (H2), meaning that a higher level of math anxiety leads to poorer performance in the mathematics exam. This result is consistent with the findings of many authors (see, e.g., [3,27,34,43]). Of the three dimensions of math anxiety considered in our study, the highest factor loading was determined for mathematics test anxiety and numerical task anxiety.

We can presume that finding ways to enhance mathematics confidence and to reduce math anxiety can significantly improve the students' performance in mathematics, leading to better mathematics exam score. It would, therefore, make sense to focus our further research to this area. The literature suggests several methods and best practices (see, e.g., [54,84]). In our opinion, upgrading the traditional teacher-centred teaching methods with newer, advanced teaching methods (e.g., problem-solving and discovery learning) can strengthen students' self-confidence in mathematics (see, e.g., [85,86]). According to our experience, E-lessons and quizzes in the online classroom are also well received among the students. Such activities can be used as a trigger to achieve more intensive self-engagement of students during the mathematics course and, consequently, lead to their better achievement in mathematics (proved with H4 in our study). We also think that it is necessary to provide creative learning environments (see, e.g., [87,88]) that will enable the students to experience success in mathematics, support their self-confidence, and develop positive attitudes towards mathematics. Moreover, [45] suggests taking more help from other students, group assignments, study groups, and buddy systems as very beneficial methods for students with high math anxiety. In our opinion, math anxiety can also be reduced by increasing the value of mathematics learning. Lecturers should try to introduce carefully designed activities into the learning process and prepare real-world problems. According to our experiences, such real-world problems are interesting for students and motivate them to understand better the results obtained. This approach is especially important when dealing with students from non-technical or science-oriented disciplines. Namely, some previous research confirmed that math anxiety is more pronounced for students of social sciences and humanities than for students of physical sciences, engineering, and math [49].

Regarding the second category, two variables were taken into account: background knowledge from secondary school and self-engagement in a mathematics course at university. It was confirmed that background knowledge from secondary school positively affects students' engagement in the university mathematics course (H3). Furthermore, a positive and high-level relationship was found between self-engagement in learning activities at the university and the final achievement in mathematics (H4). These results support previous studies which indicate that incoming skills measured by grades in high school mathematics are among the most significant predictors of students' success in mathematics and science courses [2,22,37,40,44]. Hence, in order to improve mathematics performance at a higher education level, more attention must be given to the students in secondary school, especially those with weak mathematics results [3].

The third category refers to the students' attitude toward involving technology in learning mathematics. Many authors claimed that educational technologies provide greater opportunities for creating new learning experiences that engage students and generally positively affect mathematics achievement [35,64]. For learning and doing mathematics, technology in the form of mathematics analysis tools can assist students' problem solving, support exploration of mathematical concepts, provide dynamically linked representations of ideas, and can encourage general metacognitive abilities, such as planning and checking [70]. However, a positive association between the perceived usefulness of technology in learning mathematics and mathematics achievement was not confirmed in this study (H6), although this variable was confirmed to be positively influenced by the confidence with technology (H5). In our case, the e-learning component accounted only for 25% of the subject's total scope. Hence, we estimate that this percentage was too low to influence the students' achievements in mathematics significantly. Since the value of the H6 path coefficient is very low, we assume that a more intensive integration of technology into the pedagogical process could lead to different results.

Summary results showed a very high coefficient of determination for mathematics achievement (0.801), indicating that the variables "Perceived Level of Math Anxiety", "Self-Engagement in Mathematics Course at University", and "Perceived Usefulness of Technology in Learning Mathematics", together, explain 80.1% of the total variance of "Mathematics Achievement". These results prove that the variables considered in the model are relevant for our study.

If we summarise our findings, we can conclude that mathematics achievements of university students of social sciences depend on the following factors: math anxiety, mathematics confidence, students' engagement in a mathematics course, and background knowledge from secondary school. This finding, therefore, opens up guidelines for our further research. In our opinion, a great responsibility for improvements lies on university teachers, who must strive to enable students to progress in these segments. However, many studies emphasise the role of secondary school mathematics (see, e.g., [20,37]), which also proved to be important in our study. We agree with the author of [40], who found that teaching in secondary school is a key determinant for the success of university students in mathematics. We believe that secondary school teachers can play an important role in building students' mathematics self-confidence. In addition, their role in preventing and reducing the level of math anxiety among their students is also essential [43]. Therefore, it is very important to identify the best ways in which secondary school math teachers can help students to achieve better incoming skills and, consequently, higher performance at university. One suggestion given in the literature is to train high school teachers to advocate skilfully for the achievement of students by employing practical mathematics learning activities and by developing an appropriate curriculum and educational programs that are focused on how to engage students in solving mathematics problems [40]. Studies in the future may be emphasised in terms of reducing mathematics anxiety, especially emotional factors, from the early stage at primary or secondary school as a possible preventive measure to reduce the level of the severe mathematical anxiety level. Thus, the finding is hoped to provide some useful information to those involved in improving the mathematics performance in higher-level institutions [3].

However, we agree with Awaludin et al. [23], who believe that all mathematics educators, irrespective of the education level, should play a role in raising students' awareness about the

importance of mathematics in everyday life, their majors, its applications for other courses in their field of expertise, and also future careers. Consequently, this can contribute to better mathematical literacy in the general population, which has received growing attention in the last few decades [89].

In conclusion, it is clear from both the literature review and the results of our study that the factors that contribute to students' mathematics performance are very complex. Therefore, mathematics performance continues to be an important area of research to support the planning of effective educational programs in mathematics that meet the needs of diverse students and a well-prepared workforce.

Finally, some limitations of our study have to be acknowledged. The first limitation is the population under consideration. The sample cohort was drawn from a single faculty of a single Slovenian university. Consequently, the findings may have limited generalisability to other contexts, nationally and internationally. Replication of the study with a different sample would enable examination of the generalisability of the findings.

The measurement instruments taken from the literature [58,70] were translated from English into Slovenian at the beginning of our research (see Appendix A). It is, therefore, possible that the meaning of a particular questionnaire item was somewhat "blurred" during translation. However, since all respondents answered the same questionnaire, we believe that this fact does not directly affect the results themselves. Nevertheless, a considerable amount of attention is required when we compare our results with the results of research conducted in English (or other) language areas.

Furthermore, the measurement instrument RMARS, which was used to examine math anxiety among the students, is mainly focused on mathematical activities based on numbers and calculations. Therefore, we have not included other areas of mathematics that are not directly related to numbers in our study. In future research, it would be worthwhile to investigate the extent to which such activities generate anxiety to the students.

We are also aware that we have excluded, from our model, some important variables (e.g., achievement motivation and teacher effectiveness) and potential relationships (e.g., background knowledge from secondary school to the perceived level of math anxiety) which may have influenced our results. Further research should address these issues.

**Author Contributions:** Conceptualization, A.B. and A.Ž.; methodology, A.Ž. and J.J.; data curation, G.R. and A.Ž.; formal analysis, A.Ž., J.J. and G.R.; writing—original draft preparation, A.B., J.J, and A.Ž.; writing—review and editing, A.B., J.J., G.R. and A.Ž. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Acknowledgments:** The authors acknowledge the financial support from the Slovenian Research Agency (research core funding No. P5-0018).

**Conflicts of Interest:** The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.


**Table A1.** Slovenian Translation of the Measurement Instruments.

**Appendix A**



#### **References**


**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## *Review* **Impact of the Flipped Classroom Method in the Mathematical Area: A Systematic Review**

### **Francisco-Domingo Fernández-Martín 1, José-María Romero-Rodríguez 2, Gerardo Gómez-García 3,\* and Magdalena Ramos Navas-Parejo <sup>3</sup>**


Received: 31 October 2020; Accepted: 1 December 2020; Published: 4 December 2020

**Abstract:** Currently, the use of technology has become one of the most popular educational trends in Higher Education. One of the most popular methods on the Higher Education stage is the Flipped Classroom, characterised by the use of both face-to-face and virtual teaching through videos and online material, promoting more autonomous, flexible and dynamic teaching for students. In this work, we started to compile the main articles that used Flipped Classroom within the mathematical area in Higher Education, with the aim of analysing their main characteristics, as well as the impact caused on students. To do so, the method of systematic review was used, focusing on those empirical experiences published in Web of Sciences and Scopus. The results indicated that, in most cases, the implementation of Flipped Classroom led to an improvement in students' knowledge and attitudes towards mathematical content and discipline. In addition, aspects such as collaborative work, autonomy, self-regulation towards learning or academic performance were benefited through this method.

**Keywords:** Flipped Classroom; flipped learning; mathematics; higher education; university

#### **1. Introduction**

Nowadays, traditional teaching–learning methods have become obsolete and do not respond to the demands of today's students, who behave passively and are unmotivated and do not encourage critical thinking. Likewise, traditional teaching methods do not adapt to the pace of society, which is advancing at a dizzying rate, experiencing important changes in all areas [1]. The work that teachers had done for decades now also needs to be different in order to provide an adequate response to their students [2]. They need to be kept up to date with methodological innovation, which goes far beyond the master class. These new methods prepare students to successfully face the real world, using their knowledge and enabling them to adapt autonomously to the changing pace of society [3]. Thus, we are looking at a revolution in the way we understand and operate the teaching–learning processes, where different skills are proposed and digital resources are incorporated, which also characterise today's society [4]. Information and Communications Technology (ICT) is essential for this methodological change, as they enable students to work independently and in a personalised way. [5].

These innovative strategies called active methodologies offer students a significant role, making them the principal participants in their own learning [6,7]. The Flipped Classroom method, also known as the hybrid model or blended learning, forms part of one of these types of active

methodologies [8]. It is a methodological proposal based on the theory of social learning and constructivism, so that students are the active actors in their learning [9].

If traditionally, during class time the teacher actively presented the theoretical contents and left the practical part for the students to work at home individually, from this new approach, the organisation and management of time is reversed [10], differentiating two parts: in the first, the students work on the theoretical contents individually, asynchronously and autonomously, before the classroom session. In this phase, ICT play a fundamental role, since these theoretical contents reach the students through videos, images, computer graphics or iconic materials [11]. The second part, coincides with the time in class, during which the questions are raised and the practical work is done; developing the competences and solving problems connected to the real world, in which the theoretical contents learned are used, in a collaborative and active way, under conditions of self-regulation and structuring of cognitive scaffolds [12–16].

The role of the teacher using this methodology is to guide learning, adapting teaching approaches to needs and preparing the different learning scenarios [17].

This method includes three lines of learning: (1) Individual learning, which is adapted to the different learning rhythms, since the contents of the first phase can be visualised as often as necessary, and it encourages responsible and autonomous work. (2) Collaborative learning, worked on during the second phase, where in groups we pursue objectives that are agreed upon until the final objective is reached. (3) Problem-based learning, which also takes place in the second part of this method, in which what is learned is put into practice in a contextualised way, enriched by the contributions of the group's colleagues, and it is checked whether the learning has been effective [18].

The advantages that the Flipped Classroom methodology brings to teaching are the following [19]: It respects learning rhythms, since theoretical explanations can be used at any time, promoting oblique learning [5]. Self-evaluation is made possible, providing constructive feedback on their progress and the quality of their work. It develops responsibility in one's own learning [17,20]. It is a methodology which is in accordance with the motivations and interests of the students [21], who tend to prefer virtual environments, to those which they are increasingly used to. It develops autonomy by also increasing their interest and motivation for learning [22], contributing to favour the "learning to learn" competence, so important to acquire in a society that is in continuous change [23]. This autonomy also favours creativity and critical thinking in the student [18]. It develops collaborative teamwork [24] and exposes students to problematic situations that encourage meaningful learning [25], which improves academic performance [26,27].

For this methodology to be effective and for all these educational advantages to be enjoyed, it is essential that teachers have acquired digital competence, which allows them to create audio–visual material and move around content management platforms, as well as having adequate methodological training [28]. Therefore, one of the drawbacks that make the use of this method difficult is the lack of training of the teaching staff in aspects related to innovative methodologies and ICTs, together with the necessary dedication to carry it out and the lack of habit of the students with the invested learning [29,30].

Numerous interventions, particularly in the area of mathematics, at all educational stages, show the benefits of this method for learning in this area [31–35]. Authors such as [36] argue that this form of learning mathematics allows higher levels of Bloom's taxonomy to be worked on in the classroom, such as analysis, which requires more discussion, making the face-to-face class more profitable. To this concept, [37] adds the impact on increasing performance and motivation in this subject, which is often difficult for students to assimilate [38] and together with [39], who also highlight the improvement in the working environment and the attitude of the students.

On the other hand, [40] implemented this method in the differential calculus classroom of higher education, appreciating, from the results obtained, its advantages of motivation and break with the classical routines, as well as the need to develop the methodological foundations of the Flipped Classroom. In this way, the personalization, meaning, idealization and representation of mathematics teaching was not impaired, besides the need to take into account the students' previous knowledge. Ref. [41] reiterates the importance of teacher training, which will lead them to change the traditional methodology, accepting the new active methodological strategies, to which they are not used to.

Ref. [42], after the application of this didactic strategy in the secondary education mathematics classroom, obtained as a result a substantial improvement in the evaluation and attitude of the students, verifying the increase in motivation and skills in the analysis and representation of graphics.

Based on these ideas, the main objective of this work was to locate the main educational experiences that would use the Flipped Classroom method for the promotion of mathematical knowledge within the Higher Education stage. With this purpose in mind, the next research questions were configured:

RQ1.What are the main experiences in which the Flipped Classroom method is being implemented to achieve an acquisition of mathematical knowledge?

RQ2.In which disciplines within mathematics are these experiences framed?

RQ3.What journals have published scientific articles on this field?

RQ4.What has been the impact of the Flipped Classroom method on students?

RQ5.What instruments were used to measure the effectiveness of the Flipped Classroom method?

#### **2. Materials and Methods**

Based on the ideas set out above, this work is part of the systematic literature review method, conceived as that which analyses information provided to generate an overview of a certain object of study, specifically information provided in databases or scientific reports [43]. This type of research allows the categorization of the results to date on the topic, as well as measuring the data based on different criteria regarding the relevant issues that need to be clarified [44,45].

For this purpose, the methodological process consists of a sequence of steps that go from the defining of the scope to the classification of the data obtained. For this purpose, the work phases proposed in the PRISMA declaration (Preferred Reporting Items for Systematic reviews and Meta-Analyses) were followed [46].

The examination process carried out was divided into two steps:


#### *2.1. Search Strategy*

The search of the scientific papers was carried out in the international databases Web of Science (WoS) and Scopus. These two databases were chosen for their potential and international reputation, as well as for the criteria they use to index their articles [47]. In the case of the Web of Sciences, the search was carried out in the Social Sciences Citation Index (SSCI), Science Citation Index Expanded (SCIE) and Arts and Humanities Citation Index (AHCI). The search equation was used, which is composed of the following descriptors: "Flipped Classroom" or "Flipped Learning", and "Mathematics".

The descriptors were applied in the searching engine of both databases in order to filter them further. In order to do so, a list of inclusion and exclusion criteria was set up to limit the study sample (Table 1).

To avoid any bias in the selection of the studies, after other works of systematic review [48], two researchers conducted the systematic review using the identical descriptors and criteria for inclusion and exclusion. The degree of consensus in the inclusion of the article was 95%. The disagreement was addressed by a third researcher who chose to include 100% of the extracted scientific literature.


**Table 1.** Inclusion and exclusion criteria.

#### *2.2. Procedure*

First, using the Prism Declaration [46,49] as a reference, the procedure was divided into four specific phases. The first, called "Identification", consisted of applying the database search equation, filtering the search for scientific articles (IC1, EX1) in English or Spanish (IC3, EX3), obtaining a total of 10 documents (WoS; Scopus). After that, in the review phase, most of the inclusion criteria (IC2, IC4, IC5) and exclusion criteria (EX2, EX4, EX5) were applied. Finally, duplicate articles were eliminated (EX7) in order to finally obtain a sample of articles to be analysed (*n* = 10).

In order to shorten this procedure, a flow chart is presented that shows the process described from the initial location of documents to the final scrutiny of the sample of articles that make up the systematic review study (Figure 1).

**Figure 1.** Flowchart of the phases that make up the systematic review.

#### **3. Results**

First, the studies were grouped according to the year in which they were published (Figure 2). In this case, it can be seen that the year 2019 was the one with the most contributions, followed by 2016, 2018 and 2017, respectively.

**Figure 2.** Number of articles per year.

On the other hand, looking at the journals in which these scientific papers have been published (Table 2), it can be seen that the articles have been published in different platforms. In particular, *Educational Technology, & Society* stands out, with a total of two works on this topic. According to the origin of the journals, they correspond to different countries, among which the United Kingdom stands out, with a total of four publications.


**Table 2.** Journals to which the works belong and country.

According to the mathematical contents that have been addressed during the Flipped experiences, it mainly corresponds to the treatment of the derivative and the limit of functions (Table 3). However, there are also works coming from the computer, algebraic and even didactic field.


**Table 3.** Mathematical content covered in each educational experience.

Finally, the objectives of each investigation were analysed in detail, as well as the methodology used and the impact on students after the application of the methodology (Table 4). In general terms, all the studies claim that the application of Flipped Classroom has improved students' attitudes towards the content taught, and in some cases towards the mathematical discipline [51,56]. In addition, parallel aspects of learning benefit, such as self-regulation [57], collaborative learning and the social climate of the classroom [52] and improved academic performance [55]. On the other hand, the methodological plurality found in the methodology employed by the authors is also noteworthy, with qualitative and quantitative works, as well as experimental, quasi-experimental designs, of a comparative nature between academic years.


**Table 4.** Mathematical content covered in each educational experience.


**Table 4.** *Cont.*

#### **4. Discussion and Conclusions**

The implementation of Flipped Classroom is considered as one of the latest and most relevant methodological innovations in recent years. Specifically, in the field of mathematics, the arrival of this teaching methodology has led various Higher Education teachers to incorporate it into their daily work in the classroom [34,35]. The aim of this study was to analyse the main experiences using this methodology when teaching mathematics, and to see what effect it had on students.

Thus, the main findings of the work indicated that in the majority of the investigations analysed, the implementation of Flipped Classroom meant a notable improvement in students' knowledge of the specific content covered, as was the case of algebraic calculation, the derivation and limits of functions, mathematical modelling and mathematical critical thinking, among others. Therefore, this finding adds to the ideas indicated by previous studies that determine the effectiveness of Flipped Classroom in this sense [34,40].

Likewise, the experimentation of the method promoted an improvement in the motivation rates of the students, as well as a more positive evaluation towards the contents and, in general, towards mathematics [51]. Therefore, we continue on a path, as corroborated by the scientific literature, of improvement in the motivation rates after the implementation of active methodologies that integrate technology in their teaching procedures [37,42].

Similarly, the improvement of aspects such as collaborative learning within the classroom activity, as well as autonomous learning, should also be highlighted. With regard to the former, dividing the teaching process into two phases encourages that time in the classroom to be dedicated to active and collaborative learning, in which students take on a greater role, giving rise to a better social climate and better group synchrony [24]. On the other hand, placing part of the activity of the methodology outside the classroom promotes the development of self-regulation and autonomy skills towards learning. However, this assertion cannot be fully corroborated, since in turn, the results of this review determined that although the majority of students fulfilled this purpose, a minority group was found not to follow the Flipped videos and that they presented greater difficulty in incorporating themselves into the new class dynamic [57,58]. In view of this situation, it is necessary to continue working on didactic strategies and the implementation of resources that allow the entire student body to be correctly incorporated into this method.

In short, the implementation of active methodologies such as Flipped Classroom is becoming an emerging practice that is gaining prominence within the mathematical landscape. The arrival of technology in the classroom has changed the concept of teaching, in favour of online education, student autonomy and the practical nature of classroom attendance.

Among the limitations of this work is that it was not possible to analyse those studies that were not in Open Access, which limited the sample of studies to a small number. In this way, there is a part of the scientific literature that cannot be analysed and interpreted. On the other hand, the future lines of research are to continue promoting the use of the Flipped Classroom method within the mathematical branch as one of the main lines of innovation at a didactic level. The proposal of good practices in this sense, will provide teachers of all educational stages in the area of mathematics with ideas to be able to undertake in their classroom and promote an improvement through technology.

In conclusion, the dizzying technological progress experienced by society, and therefore the education system, has led to a profound transformation in the teaching–learning process. Faced with a student body that is very different from that of a few years ago, it is necessary for teachers to explore their interests and motivations in order to plan their teaching. For this reason, the inclusion of technology and its application in teaching methods is seen as a solution that motivates students, increases interest in the subject and the content it covers and, as this research has shown, promotes better knowledge acquisition.

**Author Contributions:** Conceptualization, G.G.-G.; methodology, F.-D.F.-M.; validation, J.-M.R.-R.; formal analysis, M.R.N.-P.; investigation, F.-D.F.-M. and G.G.-G.; writing—original draft preparation, G.G.-G. and M.R.N.-P.; writing—review and editing, J.-M.R.-R. and G.G.-G.; visualization, F.-D.F.-M.; supervision, F.-D.F.-M. All authors have read and agreed to the published version of the manuscript.

**Funding:** Ministry of Education, Culture and Sport of the Government of Spain (Project reference: FPU17/05952).

**Acknowledgments:** To the researchers of the research group AREA (HUM-672). Research group by belonging to the Ministry of Education and Science of the Junta de Andalucía and based in the Department of Didactics and School Organization of the Faculty of Education Sciences of the University of Granada.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## *Article* **Using Robotics to Enhance Active Learning in Mathematics: A Multi-Scenario Study**

### **Edgar Lopez-Caudana 1,\*, Maria Soledad Ramirez-Montoya 2, Sandra Martínez-Pérez <sup>3</sup> and Guillermo Rodríguez-Abitia <sup>4</sup>**


Received: 31 October 2020; Accepted: 27 November 2020; Published: 4 December 2020

**Abstract:** The use of technology, which is linked to active learning strategies, can contribute to better outcomes in Mathematics education. We analyse the conditions that are necessary for achieving an effective learning of Mathematics, aided by a robotic platform. Within this framework, the question raised was "What are the conditions that promote effective active math learning with robotic support?" Interventions at different educational scenarios were carried in order to explore three educational levels: elementary, secondary, and high school. Qualitative and quantitative analyses were performed, comparing the control and treatment groups for all scenarios through examinations, direct observations, and testimonials. The findings point to three key conditions: level, motivation, and teacher training. The obtained results show a very favourable impact on the attention and motivation of the students, and they allow for establishing the conditions that need to be met for an effective relationship between the teacher and the technological tool, so that better learning outcomes in Mathematics are more likely to be obtained.

**Keywords:** educational robotics; active learning; educational innovation; mathematics learning; case studies

#### **1. Introduction**

For a long time, human beings have endeavoured to develop new methods to perform tasks in an easier way, with the aim of doing those activities that benefit him in any area of his life faster and more efficiently. It is then that technology comes into play. Despite the fact that technology has several paths, it can be assured that each and every one of them has a common purpose: to help humanity solve some problem that is inherent in it. Information and communications technologies (ICTs) currently occupy an extremely important place in society and the economy. Their importance has been increasing enormously. The concept of ICT has emerged as a technological convergence of electronics, software, and telecommunications infrastructure. Robotics is one of the expressions of technology whose application has extended to various contexts of life. In the educational field, it becomes a valuable resource to facilitate learning and develop general skills, such as socialisation, creativity, and initiative in students today [1].

Speaking of education, the results in the latest PISA test showed that Mexico's performance is below the OECD average in science (416 points), reading (423 points), and Mathematics (408 points) [2]. According to [3], one of the main causes of school failure in students is a lack of interest and boredom. This is mainly due to the fact that, in most cases, the current education is not interested in generating innovative activities that favour the participation of their students. For this reason, the use of ICTs has been promoted within the classrooms. The use of new technologies allows for an incentive towards learning in students of different educational levels, and the learning of Mathematics is a very specific area.

One of the most relevant challenges in Mexico is the attitude towards learning Mathematics. Currently, it is a necessity that Mathematics instructors find better ways of teaching; this would allow for students to be more empathic and make sense of this area of knowledge. Thus, in most classes nationwide, teachers do not improve the use of teaching materials, due to a lack of creativity, time, proper training, or planning. Hence, the importance of showing how technology allows for significant improvements in attention and motivation towards Mathematics, which, in turn, allows for an improvement in training programs and teaching practices; thus, achieving a positive impact on student learning. The principles of mathematical modelling include learning on our own; using technological devices and basic productivity tools to investigate and produce learning material in an ethical and efficient manner; and, applying numerical, algebraic, and geometric procedures for the understanding and analysis of real situations. The use of a technological platform would help to fulfil the expected objectives.

Although there are already ways to incorporate technology in the classroom, it is still not very common to use robots as a support tool for lesson delivery. Robotics in the classroom not only allows to study topics of automation and process the control in the area of technology and computer science, but it also serves as an aid in learning different areas of knowledge. The robots arouse interest in students, as they are concrete objects striking. An educational robotic proposal can be implemented under an approach that takes the learning environment, the planning of activities, resources, the time needed for the realisation of each of these, and the methodology to perform them into account. In this framework, active learning with strategies for doing, reviewing, learning and applying can be of support to contribute with the construction of mathematical knowledge. When joining these resources, one could ask oneself: What are the conditions that promote effective active Math learning with robotic support?

#### *1.1. Learning Mathematics in Mexico*

Mathematics is a universal language that contributes to the development of logical thinking, the ability to reason, and to face new challenges. Learning Mathematics is a subject that emphasises problem solving. If students think critically, they can solve problems effectively [4]. Mathematical competences, according to the Organisation for Economic Cooperation and Development [5] refer to students' abilities to analyse, reason and communicate effectively when they identify, formulate, and solve mathematical problems in different situations [6]. Mathematical competence "implies the ability and willingness to use mathematical modes of thinking (logical and spatial thinking) and their representation (formulas, models, graphs, and diagrams)" [7] (p. 164). Mathematical competences must be placed as part of the key elements within learning for life.

The Government of Mexico states that the general purposes of learning Mathematics are: (a) to conceive Mathematics as a social construction in which mathematical facts and procedures are formulated and argued; (b) acquire positive and critical attitudes towards Mathematics; and, (c) to develop skills that allow them to pose and solve problems while using mathematical tools, make decisions, and face non-routine situations (Government of Mexico, n.d.). Thus, the profile of the preschool graduate in mathematical thinking is [8]: "he or she counts at least to 20, reasons to solve problems of quantity, builds structures with figures and geometric bodies, and organises information in simple ways" (p. 68). For the primary education profile: "he or she includes concepts and procedures for solving various mathematical problems, and for applying them in other contexts" (p. 74). Finally, for secondary education: "he or she expands his knowledge of mathematical concepts and techniques

to pose and solve problems of varying degrees of complexity, as well as to model and analyse situations. He or she values the qualities of mathematical thinking favourable to Mathematics" (p. 80).

In Mexico, not all of the students have the same opportunities to learn. Those who cannot access mathematical content in school are at a lifelong social and economic disadvantage. According to [2], the lack of equal learning opportunities in the Mexican education system leads to the reinforcement of gaps and inequalities in society. Learning in Reading, Mathematics, and Science remains below the international average. According to the PISA report, only 1% of students performed at the highest levels of competence and 35% of students did not achieve a minimum level of competence. The obtained results continue to show a lack of competence in Mathematics [3].

Mexico seeks quality education that is innovative in its pedagogical practices, and that has the necessary means for the integral development and greater well-being of its society. In order to achieve this, socio-educational policies search for new opportunities in order to reduce inequalities between communities, promote training in values, minimise gaps, and promote equity [9]. In this regard, the Secretary of Public Education (SEP) and the National Institute for the Evaluation of Education (INEE), highlighted the importance of reviewing and redesigning existing educational programmes and modalities with the aim of: (a) facilitating access to all citizens, (b) encouraging the development and use of new technologies, and (c) promoting the skills of all students in order to ensure a full life. The integration of technologies and, in the case of educational robotics, helps to develop various skills and promote the construction and acquisition of knowledge in general, and Mathematics in particular. To this end, they designed the National Plan for the Evaluation of Learning (PLANEA), which classifies those evaluated into four levels of mastery: I (insufficient), II (basic), III (satisfactory), and IV (outstanding). This plan aimed to assess the teaching-learning processes and the learning achievements in Language-Communication and Mathematics [10].

Thus, the Secretary of Public Education [8] raises the need to ensure quality of learning in basic education, and educational inclusion and equity for the construction of a more just society. If we focus on the area of Mathematics, we seek to modify the actions with innovative practices, integrating the use of technologies in teaching methodologies for learning them, and promoting the development of mathematical skills.

#### *1.2. Educational Robotics*

Educational robotics (ER), which is also known as pedagogical robotics, is a discipline that aims at the conception, creation, and implementation of robotic prototypes and specialised programs for pedagogical purposes [11]. ER is not a new concept, but, rather, it has been growing exponentially in recent years. It has a major impact on learning [12], and it is associated with the STEAM disciplines (Science, Technology, Engineering, Art, and Mathematics) for the development, skills, and understanding of mathematical, physical, engineering, and related concepts [13–15]. Across the various faculties and universities, and in order to reduce the gender gap in STEAM careers, we have to train students in/with robotics related skills in all disciplines. Additionally, in this way, promote learning centred on the student, on his or her interests, and on the demands of society, using innovative methods; and, promoting critical training, in order to develop active and co-participating citizens in today's society [16,17].

It could be defined as "an interdisciplinary discipline, which requires the construction of a technological object with a specific purpose (some authors call it an educational robot, others call it a robotic prototype, others refer to automatisms ... ); it aims at the pedagogical field; and it develops key competences and skills for the students of the 21st century" [18] (p. 4).

The integration and use of educational robotics in the teaching-learning process in pre-school, primary, and secondary levels can become visible, and be a turning point, as a resource to address the diversity of the classroom, as a means to help the inclusion of all students, as well as keep them active and motivated [15,19–21]. Additionally, it can be utilised as a tool to promote the construction of knowledge and the achievement of results. Therefore, when working with ER, apart from working

on science and technology, the aim is to promote other cross-disciplinary skills, such as: creativity, communication, collaboration, critical thinking, teamwork, innovation, the development of solutions to problems, digital skills, and computational thinking [22,23]. In order to do this, teachers must have basic knowledge to be able to teach the contents with robots [24].

Introducing students to the areas of science and technology, in the case ofMathematics, through play and constructionist learning in order to generate new knowledge, is one of the goals of the usability of ER in the classroom. Learning with robots offers students intrinsic motivation, and invites them to investigate, foster their curiosity and imagination, to ask questions, to work in teams, to overcome challenges, to make decisions, and to be responsible of their own process [12,25].

There are many robots on sale, according to shape, size, function, working environment, and autonomy. Depending on the shape, we find: zoomorphic (imitation of a creature, e.g., bee), humanoid (reproduction of the shape of a human and its movements, in this case, the NAO robot), hybrid (combination of the above), and polymorphic (different shapes, adapting its structure according to the task). The size of the robots can be: robots, microrobots, nanorobots, or nanobots (the smallest, nanometric size). Function: domestic, medical, military, entertainment, space, educational, among others. In relation to the work environment: stationary robots (they are fixed and immovable, they are the majority of industrial robots), ground, underwater, air, and microgravity robots. Additionally, in terms of their autonomy: tele-operated (drones) and semi-aquatic [26].

The NAO robot, humanoid robot, was born in 2008 and developed by the company Softbank Robotics, we are currently facing the model evolution v5. The NAO robot measures 58 cm, weighs no more than 5 kilos, speaks, listens, sees, relates to the environment as programmed, and interacts naturally. It is capable of perceiving its environment from multiple sensors. It is composed of two cameras, four microphones, nine tactile sensors, two ultrasonic sensors, eight pressure sensors, an accelerometer, a gyroscope, a voice synthesiser, and two speakers [27]. The robot includes a graphic programming software, Choregraphe, which allows accessible communication with the NAO; this software is a graphic programming interface by means of blocks, which provide specific tasks for the NAO [28].

NAO has two key advantages: (1) its versatility (customising its functions and individualising its uses) and (2) its body language (freedom of movement, adapted to the environment, manageable and friendly, and it is designed for any age). In relation to Education, NAO is designed to be used from the age of five up to university. Its use in the classroom means that the students are more playful, motivated in the learning process, able to interact and communicate, and establish a link between theory and practice. On the part of the teaching staff, they have more engaged students, and more dynamic classes, promoting student interest, and obtaining results of the programming in real time. As far as researchers are concerned, NAO has been used in different universities, as it is considered to be ideal for practical experiments, as it is an intuitive software that offers endless possibilities with multi-language programming [27].

#### *1.3. Active Learning and Robotics in Mathematics*

Learning environments for Mathematics require the diversified use of methods, techniques, and strategies that support the acquisition of processes of analysis and construction. One technique that has proven to be effective in developing mathematical skills is active learning. In active learning, the teacher uses a methodology that seeks to promote the participation of the student as a prosumer of knowledge [29]. In this technique, the teacher must plan continuous stimulation activitie, so that the student individually or collectively performs procedures of higher order: analysis, synthesis, interpretation, inference, and evaluation [30]. According to [31], the family and community socio-productive activities in which students participate in on a daily basis constitute areas of experience that demand their incorporation into the didactics of Mathematics focused on problem solving.

It is important that the practical experience that students have had regarding mathematical reasoning be considered within the teaching of Mathematics in a formal school space, with the support of practical research. In the study [32], the impact of the use of an Adaptive Tutoring System (ATS) on the development of three mathematical competences was measured: the use of symbolic language, modelling of mathematical problems, and problem solving through mathematical reasoning. Research [33] suggested increasing the level of student mastery of mathematical concepts, while applying the concept of active learning that involves collaborative metacognitive activities among students during the learning process. Through this strategy, it was possible to improve the understanding and mastery of mathematical concepts.

The current educational practices postulate learning as a construction of knowledge, the attention has special relevance in the storage and integration only of the information that is relevant, becoming an element that the student must actively train. Thus, active, dynamic, participative learning, far from passivity, is done with the attention and concentration focused and directed towards the significant elements of it; only in this way, the acquired knowledge will be permanent and effective. In conclusion, attention and concentration are both basic elements of all learning. Project based learning has a key place in active learning. Project-based learning is an example of active learning and it is a current instructional strategy that is driven by students in an interdisciplinary, collaborative, and technology-based manner [34]. Students who implement project-based learning perform better after using this method in teaching and learning sessions [35].

The phases of active learning are differently defined, depending on the author who writes about them. Nonetheless, the main elements remain common among models. One version that is quite easy to understand is presented in [36,37], and described in detail in Table 1.


**Table 1.** Stages of active learning.

Source: [36] (p. 30).

Furthermore, in [38], it is explained how active learning is not a subject area competence (Table 2), but that there is a wide variety of ways in which it can be used in the life of the classroom: from an individual activity to the development of a group project. In all of them, the same processes and principles are applied: Do, Review, Learn, and Apply.



Source: [38] (p. 76).

The examples that are shown in Table 2 follow a similar process, a process that can be described as a cycle. This cycle models the process of learning from experience with four phases: Do, Review, Learn, and Apply [37,38]. To this cycle, it is also important to add a previous stage in doing, Planning, as shown in Figure 1, below.

**Figure 1.** The active learning process (adapted from [36]).

When planning the teaching-learning processes, teachers focus on the experience they want to show and carry out with the students. To this end, the model of the phases presented by [37] can help them to acquire the learning. Active learning has been integrated to support the construction of mathematical knowledge, both at the basic education level while using open educational resources and learning objects [39] and in higher education in virtual and remote laboratory environments [40]. Project-based learning enhances students' academic performance.

When planning the teaching-learning processes, teachers focus on the experience that they want to show and carry out with the students. To this end, the model of the phases that are presented by [36] can help them to acquire the learning.

Math teaching can find great support in the use of technology. Math learning should be done while using instructional media [41], particularly visual learning media, with the support of technology-based ecosystem environments) [42]. Ref. [43] confirmed, through their research, that students who access a multimedia environment provided by their teacher on a blended learning basis can perform significantly better than those that are taught in the classroom. In [44], the design principles that underlie the development and delivery of a blended learning professional development program for high school mathematics teachers were analysed and theoretical frameworks where face-to-face and computer-mediated instruction, teacher identity formation, and structural and basic characteristics of effective teacher professional development are coordinated were identified: the form, duration, and coherence of activities; the nature of teacher participation; focus on content knowledge

(mathematics); and, opportunities to engage in active learning. The implementation of visual and technological resources has had good results in the teaching-learning process of Mathematics.

Numerous studies have been conducted on technology applied to the teaching of Mathematics. Ref. [45] developed a research that allowed concluding that cognitive, affective, and metacognitive factors can be modelled and supported by intelligent tutoring systems. The system also helped to improve Math performance on standardised tests, as well as improve student engagement and affective outcomes. In [46], their study found that students who learned using Microsoft Mathematics performed higher on their assessments and a positive effect on student confidence in Mathematics was observed. The work [47] focuses on the description of the principles on which a hypermedia tool (Hipatia) is based and then analyses its impact in three key areas: the learning process of students in Mathematics, their self-management, and affective-motivational variables, such as perceived utility, perceived competence, intrinsic motivation, and anxiety towards Mathematics. The studies focus not only on the cognitive aspect, but they also analyse other elements, such as motivation and confidence.

#### **2. Materials and Methods**

In all scenarios, the robot was used as a support tool, which facilitated the teacher's action in the execution of explanations, interactions with students, and review of results. The design and execution of the robot intervention was different for each scenario. It is important to point out that decisions by the administration of each school limited the participation of the protocol in each one of them; therefore, the number of visits, duration of interventions, and application of tests was different, in accordance with the guidelines that each school determined and approved for the execution of the protocol.

#### *2.1. Plan*

To develop this project, several aspects were considered, and a selection of topics was carried out for the planning of robot support for the different scenarios. The main aspects to consider are the number of visits, measurement tools, exams, and interviews, as described below.

Visits: in the case of the primary school, two visits were carried out on consecutive days, having almost one and a half hours to carry out the planned routines in mathematical reasoning, through activities to work with distance measurement and fractions. Our project team carried out the visits. For the secondary school case, the robot assisted in four full 50-min. classes, one for each visit. In each session, the teacher used the robot at certain times that were previously defined in the design of the sessions. Finally, in the case of high school, the robot visited the class on a daily basis for a week. Three visits were done at different times of the semester. The first visit was carried out in an introductory way, so that the robot had already been presented with the students for creating a first impression by them. The next two were to support the teacher in the topics established from the beginning. It was the teachers who manipulated the robot when they needed it.

Measurements: in each scenario, a control group and an experimental group were established. Same interventions or classes were carried out with the intervention of the instructor or with the instructor using the robot. The intention was to compare and be able to measure the impact that the robot had among the students. Therefore, tests with numerical results were applied, as well as observation scales in order to measure motivation [48]. Both of the tools were applied in two types of groups. Only in high school were the tests were given before and after starting the robot interventions. In addition, interviews were carried out on the experience carried out with the participating teachers and, in the case of high school, with the students.

Tests: the tests were different for each scenario. At elementary school, a questionnaire was applied, and the students had to solve a mathematical exercise based on the explanation of the topic made by the robot. This exercise was chosen from the Mathematics book that was used by the course teacher, and the number of correct answers, in this questionnaire, was accounted for. For secondary school, an exam was conducted on the topics chosen by the teacher, those where there was support from the robot. Almost all of the exercises were about Analytical Geometry. The design of these exams was

done by the teachers in charge of the two groups. In the case of high school, the applied exams were designed and were considered in the planning of the course from the beginning. That is, they were the standard exams that are commonly applied in the subject of Trigonometry, coordinated by the Department of Mathematics of the institution in question.

Interviews: they were carried out in order to find out the opinion of the teachers involved, and the more formal implementation of the structure of the interview, as well as the testimonies of the students and teachers, was established in the high school. As can be appreciated, the protocol became more formal as experience was gained from the previous scenarios. In Section 3.1 of this document, more details of each scenario are given, and a more detailed quantitative analysis of the obtained results is carried out.

The planning for the project is summarised in the following three aspects:

(1) The robot: as mentioned above, a NAO robot was used to support the teacher. The robot delivered the explanation of the topic to be developed, provided that the appropriate environment was put in place. In addition, sometimes it guided the exercises that were established by the teacher. Furthermore, the robot confirmed to the students whether the exercises had correct results. Figure 2 shows the general way in which the robot interacted.

**Figure 2.** Structure of the sessions.

(2) The scenarios: primary school, two groups of 3rd and 5th grades were visited, with the completion of mathematical reasoning exercises in two consecutive visits. The teacher did not directly use the robot, which carried out demonstrations of previously chosen topics, but was autonomous in carrying out the activities. Before and after, questionnaires were made to the students in order to compare the performance obtained.

For secondary school, four visits were made to the school, with a control group and a treatment group (using the robot), before and after exams were compared, in addition to the observation scales of motivation and attention for the students. The teachers selected and designed Math exercises for their execution.

In high school, performance and motivation were observed in five groups, two experimental and three control groups, during an academic semester with daily visits in full intervention weeks. The subject was trigonometry and the teachers designed and planned the exercises, with greater control

over the way the robots intervened. Before and after, the results in the semester were compared as well as the measurement of the students' attention.

(3) The measurements: for quantitative analysis, according to the scenario, teachers applied tests and/or exercises, aimed at measuring the students' learning, and most of them were conventional exams.

In all of the scenarios, a qualitative analysis was carried out, based on previous works [47]. In these analyses, the applied scale was composed of the following indicators: concentration (precision and recall), habituation, withdrawal, distraction (neglect), and interest in the task (motivation and enthusiasm). Based on the definition of care and the indicators that compose it, operational definitions of these indicators were developed to be later translated into observable items of dimensions of care.

The analysis and taking of the scales were carried out by students of Psychology, taking care to observe the guidelines of the reference indicated in terms of human behaviour.

#### *2.2. Methodology*

We explored different approaches in three different scenarios in order to shed light on what constitutes a successful application of a robotic platform for enhancing active learning.

The scenarios were divided by educational level. That is, elementary, secondary, and high school settings. Each one of the interventions is described below.

Scenario 1: elementary school. There were two interventions undertaken to two sections of the third and fifth grades, with 28 students each. For these visits, activities were planned without teachers' involvement. They limited their participation to allowing the research group to enter the room and coordinate the activities, manipulate the robotic platform, and perform a brief interview at the end of the session, both for students and teachers. Evaluation was done by applying quizzes to sections where the robot was used, and to others where it was not, so that a results comparison could be made.

Scenario 2: secondary school. For this scenario, visits were performed every Friday for an entire month. This time, two sections were provided use of the robotic platform, and another one was observed as a control with no treatment. Each section had 25 students enrolled. Another variation was that, this time, teachers were involved in planning the activities, including the topic to cover, and the way that the robotic platform was to be used. A group of psychology students was active making observations throughout the process, and capturing both qualitative and quantitative information for later interpretation. Interviews with students were also performed. The emphasis was on assessing the levels of attention and motivation gained by the students translated into measurement scales, as well as performance on the subject matter.

Scenario 3: high school. For this scenario, the actions became more complex. The robotic platform was applied to two treatment groups, and observations were also made to three control groups. The robot was used in every class during an entire week at the beginning of the course, then it was done again in the middle, and at the end of the course. The involvement of teachers in this case was intensive, being trained in the use of the robot, and deciding together the moments and themes where the robot would intervene. Pre- and post-tests were performed for all sections, in order to compare performance results. Additionally, as in the previous scenario, Psychology students undertook behavioural observations and made interviews and testimonials.

For all scenarios, an assessment on how well the process met the requirements of the four cited phases of active learning was made, assigning a rubric-like success categorisation for each. This assessment was made qualitatively, and was based on panel expert discussions, drawing from the experiences that were obtained at each scenario. A comparison was then derived in order to identify the appropriate success factors.

Figure 3 summarises the overall methodological process.

**Figure 3.** The robotics & active learning exploratory methodological process.

#### **3. Results**

#### *3.1. Quantitative Analysis*

The use of the analysis instruments that are mentioned in Section 2.1, of the different scenarios grew gradually, starting in primary school, improving aspects of implementation in secondary school, until reaching a more developed scenario, with a longer participation time in high school.

#### 3.1.1. Primary School Tests Performed

Two different sessions were scheduled, taking into account that they were students from third and fifth grades, for a total count of 65. In these sessions, it was evaluated how much attention the students paid to the class, if they retained more information with the help of the robot, and results were compared with a class without robotic help. The sessions were designed in order to address the following topics: propagation of sound, the metric system, and whole number fractions. These topics helped to develop mathematical reasoning and were approved as examples of application by the teachers. All the academic procedures designed by the team followed the following structure: a personal presentation, a presentation of the robot, a brief explanation of the topic, a learning activity, an exam, and a questions session by the students.

For the propagation of sound activity, it was observed how far the sound could be detected, having a "receiver" and a "transmitter". In the session without the robot, the receivers would be the students and the transmitter would be the teacher. Instead, in the session with the robot, the transmitters would be the students and the receiver the robot. In both cases, the activity would consist of the transmitter speaking at an initial distance and going backwards until the receiver can no longer hear it, once this happens the transmitter will begin to speak through a foam cone, until the same happened, and later on by a paper one. In the case of the metric system topic, the activity consisted of the students having a piece of tape 2-m long having a different colour every 20 cm. An object would be placed at the beginning of this tape, either provided by the teacher or the Nao robot, and it would move forward or backward through each colour. The student should observe the distance from the beginning to the position of the object to attain a better perception of the measurements. Finally, the last activity was to explain the components of a fraction, particularly how to convert a fraction with whole numbers and equivalent fractions. To do this, some examples of fractions, equivalent fractions, and fractions with whole numbers would be seen visually to make the subject clearer. Once the topic was explained, an activity would be carried out reaffirming the knowledge acquired.

In the last session, Psychology students were present to observe the interaction of the teams. The session was applied in two groups, in order to observe differences, and it only occurred in one with the robot. In this session, the topic of fractions would be explained, and examples would be provided for the students to observe. Later, an exercise chosen by the teacher would be carried out, from the book of Mathematical Challenges for fifth grade students. To end the session, a test was applied in order to measure the knowledge acquired.

Some of the results obtained for two of the indicated activities are shown below. Figure 4 shows the percentages of students in a range of scores on the sound propagation test. It can be observed that there was a reduction of 17% in the number of students who obtained a grade that is smaller than seven, thus increasing the proportion of passing grades, which is six or higher.

**Figure 4.** Test result: sound propagation.

Figure 5 shows the results of the three parts of the session: initial prior knowledge test (three questions), final test for comparison with the initial test (three questions), and final question asked during the session.

**Figure 5.** Test result: fractions with whole numbers.

For the results of attention to the class with the robot, an observation scale made up of 34 questions was used, which are grouped and measured the effect in the following dimensions: (1) concentration (precision), (2) concentration (recall), (3) habituation, (4) de-habit, (5) distraction, (6) interest in the task (enthusiasm), and (7) interest in the task (motivation) [28].

In Figure 6, the results obtained on the observation scale are shown.

**Figure 6.** Results of the observation of the activity of the metric system.

When observing the set of all the graphs, it was determined that the favourable points for the NAO robot are: greater concentration, less habituation and dishabituation, and greater interest in the task (motivation). It has to be noted that lesser habituation and dishabituation are desirable, since they indicate that the students will not become bored with the robot with time, and they will not create a dependency on the robot either. There are no apparent differences for distraction and enthusiasm. This can be the result of the influence of observers in the classroom, so it should be more deeply looked into in future studies.

#### 3.1.2. Secondary School Tests Performed

In collaboration with the psychology group, the directors of the institution and the teachers, it was decided that the robot would interact with the students during four sessions of the Mathematics class, on Friday at 7:35 p.m. The group that did not interact with the robot was observed during their Math class on Friday at 2:30 p.m. Before each session, both of the teachers explained the objective of the class and the robot would be prepared for its interaction (analytical geometry). Among the forms of interaction were mainly the dictation of exercises, the response to those same exercises, answers to questions that may have arisen during the class, and even participating as a student during the class. Finally, during the sessions, the psychologists were in charge of observing the students and filling in an observation scale.

Through the tests carried out before and after the sessions, the results of the tests that were applied to the students were averaged, and that both of the observed groups obtained during the investigation.

The observation scales that were performed by the Psychology students turned out to be a more influential tool for hypothesis verification. Thanks to these questionnaires, significant differences could be found in the behaviour of both groups, as shown in Figure 7. This graph shows how behaviours were present and how they varied through the different sessions. It is indicated in the "yes" part of each session, and the frequency in which the behaviour was observed. Similarly, the "no" section indicates the absence of each behaviour at certain point in time for each session. Thus, it is important to note the increase in the "no" part for habituation (from 7 to 11) and distraction (from 4 to 5), from session 1 to session 4, with this being favourable since they are unwanted behaviours.

**Figure 7.** Comparison of the observed behaviour dimensions.

#### 3.1.3. High School Tests Performed

The robot was a support tool for the teacher in the teaching of a hybrid teacher-robot class where two topics of the subject were addressed. During the first visit, the topic would be the statistical analysis of graphs. The disciplinary competencies to be developed are graphical representation of statistical data and the generic competence was collaborative work. For the second visit, the collaborative activity would be dealing with the right triangle solution in the measurement of inaccessible distances within the campus facilities. The disciplinary competencies to be developed were right triangle solution and generic collaborative work.

At this educational level, the teachers were co-designers of the entire research strategy, even controlling the robot to a much greater extent than in the other scenarios. For this, experimental tasks were carried out, divided by periods of six months each. The first was the design and planning of the sessions to begin preparing the programming of the robots, following a structured script by the teachers in charge of the groups. This task focused on the detection of errors in the programming of the robot and dynamics of class teaching and their respective evaluation. The second was the teaching of class by a NAO robot and a Math teacher, following a script that was similarly structured by the high school teaching team. It was expected that the explanations and topics given by the robotic platform would follow the same thematic guide that was used in the high school session plan.

Together, the behaviour of the students was analysed, through the application of a behaviour observation protocol more appropriate for this scenario. The project was carried out in three control and two experimental groups, with the intention of observing more aspects in a population of around 140 students in total.

Figure 8 shows the results of the exams applied as pre-test and post-test at the end of the semester. In all cases, it is clear that the bar for the experimental groups is higher than the one for the control

groups. This means that students scored higher with the use of the NAO. This is more evident in the case of the second teacher, where the grades vary in a range of three to four points between exams.

**Figure 8.** Ratings obtained by the groups.

Figure 9 shows the relative percentage of occurrence of the different behaviours that were organised by the dimensions of the observation scale during a class session (concentration, habituation, withdrawal, distraction, and motivation). The percentage is used since the groups are not equivalent in number of students and in the total frequency of the behaviours, then it is weighted on a percentage, so that all are comparable.

**Figure 9.** Behavioural observations by dimensions and group.

It is observed that the most concurrent dimensions are concentration and motivation among all groups. However, there is also a certain tendency for increase in these dimensions from the control group to the experimental group in the respective groups of each teacher.

#### *3.2. Qualitative Analysis*

Based on the observations made, it was clear that attention and motivation were boosted for the students at all levels. Nonetheless, the differences in the planned activities played an important role in the level of success achieved for each one of the four phases of active learning amongst scenarios. Now, we will discuss all three scenarios for each phase at a time.

#### Phase 1: Do

At this stage, it is expected that some stimulating activities will trigger motivation and interest by the students. It is expected that students themselves will decide on their learning strategy, and they will reflect on the results along the way, while they are still performing the task.

For the elementary level case, the activities posed and the presence of the robot did, in fact, stimulate the interest of the students. Nonetheless, they had almost no saying in the learning strategy. Their autonomy was limited to trying to solve the problems in the best way possible, once the instructions were given and understood. At the end of the class, they were to reflect a little on how they felt during the activity, and an example was analysed for application in the real world. Hence, a medium level of success was obtained in strategy decisions and in reflecting on one's own learning process.

The secondary level case was equally motivating in terms of the presence of the robot. For this scenario, the teachers solved problems, which then needed to be addressed by the students with the aid of the robot. During the process, partial results were analysed with the teacher, and corrections were made accordingly. This represented an increase in learning awareness by the students. However, it can still be considered to be a medium level of success.

Finally, when considering the high level case, challenges and exercises were presented for the students to solve, which were designed by the teachers, and they used the robot as an aid for their solution. Most exercises were trigonometric problems with examples. In this occasion, students were responsible for establishing the route to solve the problem, using the robot as well as the information that were provided in the class. The robot served as a tool for verifying results. Students received feedback during the task, not only about results, but about the theme's objective, thus restating the results. This process was quite satisfactory for the independence of the students in deciding their own learning strategy, and medium success was achieved for reflecting on their knowledge while doing the task.

#### Phase 2: Review.

In this phase, it is expected that students will take breaks during the task to become aware of what has just happened, what was important, and how they felt. Subsequently, they monitor their own progress and review their plan. Finally, they document their observations and learning outcomes.

For the elementary school scenario, students commented on the routine that was shown to them and explained, in their own words, what the session consisted of, and how well they believed they understood the lesson. In the second session, they only mentioned how the robotic platform had helped. On the first visit, the students discussed the experience, and commented about what was about to happen in the second visit, but had no say in deciding what that would be, or relating it to their learning process. No documentation was made by the students. The general success score was medium for stopping to evaluate progress, and for monitoring it. Furthermore, it is low for documenting their knowledge acquisition process.

For the second scenario, there were time stops to review and share solutions to the rest of the class, discuss their efficacy, and reflect on the problems and results. They discussed their progress and their experience with the robot. These results were within the desired range. However, documentation focused more on their experience with the robot than in assessing their own learning process.

For the last scenario, the high school students discussed questions that are triggered by the robot or the teacher. The class planning included review and adjustment times, and the students received new exercises that were applied to alternative contexts in order to improve their comprehension of the themes. Constant documentation was included in the process, making this stage considered to be successful.

#### Phase 3. Learn

For this phase, it is expected that the new ideas that are generated by the activity be made explicit. Students are responsible for identifying on their own their learning outcomes. Additionally, they identify barriers for their progress and propose new strategies.

In the elementary case, the objectives of the exercise were made explicit by students with the authorisation of their teacher. Additionally, there was a conversation space provided for them to verbalise what the experiment consisted of, but suggestions were only given as commentaries by the students. These elements drive to give a score of medium, in terms of the first two elements, but deficient in the proposition of new strategies.

The secondary school scenario, on the other hand, counted with a planned strategy to develop via the robot and the teacher together to make the learning outcomes explicit. Even though it was positive overall, it was not as effective as expected, since it resulted in being somehow confusing for the students. Some exercises were assigned to externalise what was being learned, and students were interviewed at the end. The comments gathered had no influence in the planning of the next sessions. This phase was considered to be positive in making learning explicit, but medium for identifying barriers and deciding strategies.

For the high school level, a space was given to analyse the learning process with the students, and together define strategies for the next sessions. The identification of barriers fell, in turn, a little short.

Phase 4. Apply

This stage is certainly the most complex, and it requires the greatest maturity of the students in the use of active learning. Therefore, it is the most difficult to achieve. First, future actions are planned based on new discoveries and learnings, and the possibility of transferring the knowledge gained to other situations is examined. Students are in charge of reviewing their plans, building on their recent learning experiences, and they move forward to plan future observations and experimentation of their learning strategies.

For the elementary school scenario, this was not covered, since only some documentation of the experience was made by the research team and considered for application in other similar contexts. Nonetheless, this activity did not include teachers or students, making it deficient.

The results improved marginally for the secondary level students, where the experience lived allowed for preparing and improving the process for future situations. However, no significant participation by the students was observed for experimenting learning strategies, beyond some reflections on the effectiveness of the robotic platform as an aid for their learning process.

Finally, the outcomes are far better for the high school scenario, where time was given to reflect on lessons learned to assess the applicability of the process to other contexts and define pertinent improvements. This was explicit also as an activity requested to the students where they had to describe how to apply the knowledge obtained to other domains. The weak point, considered to be a medium success, is that related to the relatively low impact of students' recommendations to redefine the course strategies in future scenarios.

Table 3 presents the summary of the success scores assigned in each case. The first score is the predominant one, and the one after the hyphen indicates there is a small component in a higher or lower category.



In summary, Table 4 shows the most relevant aspects regarding the tests carried out in the three scenarios, not only qualitative and quantitative, but also a comment that we highlight about each experience obtained.


**Table 4.** Summary of observations and test results.

#### **4. Discussion**

These results lead to think that there are many factors at play that need to be considered. The robotic solution will never result in significant learning improvement unless accompanied by the right strategy. However, it is a great tool for attracting interest and motivating students to participate, regardless of level. The active learning process, on the other hand, needs to be carefully planned according to level, since the cognitive capabilities and styles of the students vary greatly depending on it. Invariably, the robotic platform and the active learning strategy have a great potential to generate synergy and be more effective when well harmonised.

Another issue that is worth mentioning is the level of involvement of the teachers. The more prepared and comfortable to use the robot, the better they can plan and adapt their strategy, based on the feedback and outcomes that were provided by the students. This allows for the flexibility needed to customise learning strategies to each student and makes them responsible for their own learning. In the long run, they may develop the right level of maturity to be real active learners. It is also important to note that digital skills acquisition has greatly increased its relevance, not only for students, but for teachers. Today's digital agendas introduce two essential axes to work: on one hand, the trend towards a digital education, and on the other, the acquisition of digital competences and skills for digital transformation [49]. Studies, such as [50,51], highlight the shortage of digital competences in initial training and the lack of knowledge and skills on educational technologies for teaching practice in pre-school, primary, and secondary classrooms.

The time of exposure also seems to be an important issue. Those who were exposed longer to the robot, and to well planned activities, obtained more significant results. One or two sessions might even be counterproductive, as they raise expectations that will no longer be met, causing discouragement of the students.

Finally, capturing behavioural data and observations may be greatly enriched when combining techniques, even including the students themselves as self-documenters. Quantitative and qualitative techniques and assessments will provide a better panorama of the situation. The documents obtained always need be shared and discussed, to gain collective intelligence, and provide more robust changes.

#### **5. Conclusions**

In previous years, different scenarios have been developed, in which a humanoid robotic platform has been used in order to increase motivation and interest in students towards Mathematics. The presented case studies show that the proper use of a robotic platform, together with an appropriate teacher participation, can mean giving high-quality hybrid classes, enhancing the student's attention to the topics that are exposed by changing the stimulus, and obtaining effective learning. The results presented above show a numerical improvement in the scales that are used to assess the presence of specific behaviours, and performance in all scenarios. Even though these results should be looked at with caution, they provide a good perspective of the potential usefulness of Robotics in Mathematics teaching. The ultimate goal is to make learning more meaningful, which should translate into better grades and better abilities for students overall.

The study highlights the motivation of teachers to learn more about the use of robots. Beyond the motivation that can be considered on the part of the students, it was the teachers themselves who expressed an attitude of learning in order to integrate them into the educational experiences. The robot is a mediating tool, but the teacher is the one who has the capacity of inventiveness to integrate it in the classes. This integration helps to promote hybrid systems in learning environments, with the teacher-robot binomial (this was observed, to a greater extent, in the upper-middle level).

This study was guided by the question: what are the conditions that promote effective active Math learning with robotic support? The findings point to three key conditions: level, motivation, and teacher training: (a) Encouraging active learning with emerging technologies (in this case, with robotics), involves considering the educational level where the learning environments are targeted, with the profile of students (levels of construction of learning) as a central focus. (b) The motivation, according to

the profile of the students of the formative experience, where robotics helps personalised learning. The introduction of robotics supports presenting the contents in a different way in order to change how they are taught in an ordinary situation. (c) The formation of teachers as a relevant aspect, where the pedagogical foundation goes beyond "programming the robot", the teachers are the ones who "should be programmed" for the didactic use that starts from the planning, the articulation of strategies, the strategic arrangement of the moments in which the use of the robots is integrated, and the evaluation of the effects.

Thus, robotics is one of many technologies that can support the processes for increasing mathematical learning, where strategies cannot be left aside, regardless of the technology that is being integrated. Critical thinking skills, digital skills, and teamwork skills are reinforced with the introduction of these technologies, linked to the stages of doing, reviewing, learning, and applying that are encouraged through active learning. Future studies could be geared towards deepening the exploration of the effects of the use of robotics on the stages of active learning, and their contribution to concrete mathematical knowledge acquisition. Similarly, it is worth analysing the teaching processes in order to demonstrate the results of theoretical and practical knowledge when applying emerging technologies, such as robots in their classrooms. It is required, in itself, to expand the studies of management, psycho-pedagogical, and socio-cultural issues, in the application of technologies in innovative learning environments.

**Author Contributions:** Conceptualisation, E.L.-C. and M.S.R.-M.; Formal analysis, G.R.-A., S.M.-P. and E.L.-C.; Funding acquisition, E.L.-C. and M.S.R.-M.; Investigation, E.L.-C., G.R.-A., S.M.-P. and M.S.R.-M.; Methodology, G.R.-A. and E.L.-C.; Project administration, E.L.-C., M.S.R.-M., G.R.-A. and S.M.-P.; Writing—original draft, E.L.-C., G.R.-A., S.M.-P. and M.S.R.-M.; Writing—review & editing, G.R.-A. All authors have read and agreed to the published version of the manuscript.

**Funding:** The authors would like to acknowledge the financial support of Novus and Writing Lab, TecLabs, Tecnologico de Monterrey, Mexico, in the production of this work.

**Acknowledgments:** To the teachers, students and institutions involved. To Itzel Hernández at UNAM for her original illustration of the active learning process.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **References**


**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## *Article* **The Possibilities of Gamifying the Mathematical Curriculum in the Early Childhood Education Stage**

**Verónica Marín-Díaz 1,\*, Begoña E. Sampedro-Requena 1, Juan M. Muñoz-Gonzalez <sup>1</sup> and Noelia N. Jiménez-Fanjul <sup>2</sup>**


Received: 30 October 2020; Accepted: 11 December 2020; Published: 14 December 2020

**Abstract:** The addition of gamification to the classroom as a methodological tool means that the teacher's opinion about this has become an inflection point that can affect its use or not in the classroom. In this sense, the main objective of the present article is to explore the opinion of future Early Childhood education teachers on the use of this resource for the development of the mathematics curriculum at this education stage and to obtain an explanatory model that explains it. The design of the study utilized a descriptive perspective and a cross-sectional quantitative focus through a quantitative exploratory study. For data collection, an ad hoc questionnaire was utilized, which was administered to a sample of 232 teachers-in-training. The main result obtained was that the future early childhood education professionals considered that gamification could be a resource for the learning of specific mathematics contents, and this was shaped around a model of two elements. On one hand, the development of mathematical thinking, and on the other, the establishment of relationships between mathematical concepts. Lastly, we can conclude that elements such as age or gender do not determine the perception of the use of gamification in the early childhood education classroom.

**Keywords:** gamification; videogame; early childhood education; mathematics

#### **1. Introduction**

The diverse social, cultural and technological changes currently experienced by society, as indicated by [1], are shaping students and having an effect on their learning processes from different points of view. In this sense, games, in the broadest sense, have evolved as a methodological resource at the same time that society has moved forward. As of today, their use in a digital format has become very important, as the students in the classroom, from diverse educational stages, have been cataloged as technological youth, adolescents, and children. This is why the education community has begun to think of gamification as a valid strategy for the development of teaching-learning processes.

From the start, it must be indicated that current literature recognizes two ways of adding gamification to the classrooms, one supported by traditional games [such as Monopoly, Risk, cards, Parcheesi, the game of the goose, dungeon master, etc.], and the other supported by digital ones (Minecraft, Mario Bros, Fortnite, Pokemon Go, etc.) [2,3], with the objective of this article focusing on the latter trend.

The use of digital gamification or videogames, per se, to the learning process of the students, begins with the search for a transformation in education that promotes the growth of immersive learning of the students, and a demand for the improvement of the teachers' methodology. In addition, other researchers consider that the use of videogames will allow the students to be initiated in curricular

competencies, which is the digital competence in this case [4–6], so that their learning process improves from another perspective.

At first, the digital gamification of the classrooms at any educational level implies accepting the use of digital games or videogames, a resource that has been demonized, hated, and loved by the education community and society at large [7–10]. It also implies that one must consider that the strategies utilized in gamification, such as the use of awards or badges, could be yet another element that motivates the students in their learning processes [3]. Thus, we must think beyond the thought that the final intent is to gamify the classroom. The use of gamification in the classroom, from a digital point of view, implies the use of videogames or strategies such as *Escape Room* or educational *Breakouts* to a level where the main objective is to introduce the curricular contents in a manner that is attractive to the student [1,3]. Thus, the contents that are considered to be difficult, either due to their nature, the manner of teaching them, or the manner of learning them, could be presented in a more motivational way, with their virtues shown to the students [11]. This will get the student's attention and promote internal strategies of assimilation and comprehension of this content.

In this sense, various successful experiments on the use of videogames in the classrooms exist, as a tool for propitiating effective learning, and at the same time, for favoring the cohesion and integration of the content to the social reality of the student. Authors such as [12,13] have pointed out that the gamified classroom promotes improvements in learning, metacognition, evaluation, and the process of conceptual support of the students. Along this line, [14] conducted a study with teachers from the area of special education and concluded that when they implemented gamification strategies into their classes, an improvement was achieved in the reasoning processes of the students. As a result, we could say that gamification, per se, tries to promote motivation for the content and the creativity of the individuals. On the other hand, it is interesting to note how the studies that have focused their attention on gender have underlined the importance of their use to foment the mathematical talents of girls [15].

Focusing our attention on the area of mathematics, we find the work by [4], who underlines how a game created more than two decades ago, The Lemmings, helps with the initiation of the basic contents of mathematics, just as the data presented by [11], who, after using various digital games, verified that significant learning of algebra content was achieved. On their part, [16] conducted a pretest-posttest study utilizing QR codes placed on cards, which allowed the students to perform different mathematics activities. The main result was that an improvement was observed in the acquisition of the rational number concept, improving the connections between its different representations such as fractions, decimals, and percentages. Along this line, [17], utilizing experimental and control groups, concluded that significant improvements had been achieved on the comprehension of basic concepts of mathematical logic of the students with whom digital games had been used as a digital resource, as compared to those who had followed a traditional methodology of learning. These results [18] show that the use of game-based interactive materials in the mathematics classroom promotes the improvement of the comprehension of mathematical concepts of the students. Therefore, the use of digital resources [19], defined as manipulative materials that allow us to visualize mathematical concepts more easily and in a more attractive manner, signifies a helping tool in the process of abstraction of mathematical concepts when coming into existence as virtual models of mathematical concepts [20].

Focusing our interest on the early childhood stage, it should first be indicated that this stage is characterized by being a point in time in which immersion into the curricular contents begins, which will be further developed in higher education stages. It is in this initial stage when the teachers begin to observe the first differences in the act of teaching and learning, meaning that different levels of learning and understanding of the contents taught to begin to appear [12,21,22]. As indicated by [23], learning is conducted due to curiosity, exploration, and immersion into the content; experimentation occurs, and initiation into research begins in a playful manner, as games are the main elements in learning processes [24]. For this reason, we can consider that gamification in the early childhood classroom

will provide a new learning scenario where fiction comes closer to the educational reality of the student, thereby promoting a creative learning process, which is vital in the first years of socialization of individuals. As for the area of mathematics, we are in agreement with [24] in that the process of logic-mathematics acquisition is conducted through a reflective process that is never forgotten so that the use of different types of resources could promote this reflection in a more effective manner.

Diverse research studies [4,25,26] have pointed out that the use of gamification for teaching the curricular content in the area of mathematics in the early childhood stage promotes experiencing the content, which results in a positive view of the students towards this subject matter. We are in agreement with [27,28] that the use of digital games in the area of mathematics implies that the student learns, in a playful manner, concepts such as probability while they play, so that learning is produced in a manner that is more motivating and personal to the student, helping with the overcoming of obstacles during this learning process.

The objectives of the present research study are:


The following hypothesis has been posited with the objectives described above:

Starting hypothesis: There are significant differences in mathematical thinking, depending on the establishment of relationships between concepts, without it being influenced by the age and gender of the study subjects.

#### **2. Materials and Methods**

The research study utilized a descriptive design and a cross-sectional quantitative focus, given that a survey was utilized as the data collection instrument.

#### *2.1. Sample*

The sample was selected by utilizing a nonprobabilistic, convenience sampling method [29], given that sample was accessed through the classroom where the virtual teaching was done.

The sample was composed of 232 students enrolled in the early childhood education degree at the University of Cordoba (Spain). If the sampling error calculation for finite populations is lower than 3%, as in our case, it is understood that the sample is significant, considering what is mentioned in [30]. In the sample, we found that 88.8% were women, 10.8% men, and 0.4% identified themselves as transgender. From the start, it can be indicated that there was a bias towards the female gender. Nevertheless, it should be pointed out that as indicated by [31,32], this university degree tends to have a higher female component than other university studies, just as with engineering degrees, where the presence of men tends to be higher [32]. As for the academic year, 50.4% were in their first year, and 49.6% in their second year. The mean age of the participants was 19.69 years old (SD = 2.408), which was distributed, as shown in Table 1.

Focusing our attention on the devices they possessed, and which could be used to play, we found that 24.1% had a laptop and a smartphone, while only 0.4% had a smartphone + videogame console or a laptop + smartphone, or a desktop + tablet + smartphone. It is interesting to note that 11.2% had access to all the devices shown (laptop, desktop, smartphone, videogame console, and tablet) (see Table 2).


<sup>1</sup> N total = 232.



<sup>1</sup> N = 232; f. = frequency.

As for their characterization as videogame players, in our sample, 44% occasionally played, 24.1% sometimes, and 4.3 and 1.3% often or very much, respectively. It is notable that 26.3% indicated that they had never played. As a function of these results, and when asked about the number of hours they played, we found that 88.4% and 90.5% played between 1 and 3 h throughout the week and throughout the weekend, respectively. It is interesting that 0.9% played less than 10 h during the week, and 1.3% during the weekend (see Figure 1).

Lastly, when asked about the type of videogames they tended to use, 72.8% utilized the one from the videogame platforms, 23.3% fighting games, and 3.9% strategy games.

**Figure 1.** Distribution of the hours of play during the week and the weekend.

#### *2.2. Instrument*

The instrument utilized for the collection of data was the survey created ad hoc under the parameters found in [33], which sets the minimum education levels in Early Childhood Education in Spain, summarized in [34], which regulates the curriculum of Early Childhood Education in Andalusia. This instrument was composed of 2 dimensions comprised of 25 items, written in an affirmative, closed and polythematic character. This was an anonymous questionnaire, which was administered online without the in situ assistance of the researcher. A Likert-type response scale was utilized, composed of five response options (where 1 indicated complete disagreement and 5 complete agreement). This type of response scale with this number of options will allow for coming close to the assumption of continuity. In addition, the instrument contained a set of independent variables which allowed us to describe the participating sample: gender, age, academic year, digital devices available for playing (laptop computer, desktop computer, Tablet, Smartphone, and Videogame console), hours spent playing during the week and the weekend, and type of games they like to play.

To determine the validity of the instrument created, an exploratory factor analysis (EFA) was performed through the use of polychoric matrices, along with an "optimal implementation of parallel analysis" [35] and "non-weighted least squares" with a "weighted Oblimin rotation" [36] to determine the number of factors, utilizing the statistical package SPSS 23 and the Factory Analysis (10.10.03) software. This analysis allowed for the verification of the viability of the construct through the correlation matrix 0.000; Bartlett's sphericity test, with a significance of 0.000; and KMO = 0.931, as well as the root mean square of residuals (RMSR) = 0.038, with extracted factors that explained 63.27% of the variance, and whose rotated factors had loads higher than 0.3, providing us with the following two-factor structure (see Table 3).

Therefore, through the EFA, a structure of the instrument was obtained, shaped by two factors, which are:


In addition, a confirmatory factorial analysis (CFA) was performed, which allowed us to compare the fit indices of the model obtained in the EFA, taking into account the following tests: χ2 test/degrees of freedom, comparative fit index (CFI), incremental fit index (IFI), normed fit index (NFI), the Tucker-Lewis index (TLI), the root mean square error of approximation (RMSEA), and the expected cross-validation index (ECVI). In the first analysis, the results found indicated that items 2, 3, 6, and 14 from the first dimension should be eliminated, as well as items 18 and 25 from the second dimension, as the modification indices indicated the existence of covariances between the errors associated with the items that belonged to different factors. Once the model was reformulated, with a total of 19 items, the following results were obtained: x<sup>2</sup> = 171.97; df = 131; *p* = 0.009; X2/df = 1.31; CFI = 0.975; IFI = 0.976; NFI = 0.905; TLI = 0.967; RMSEA = 0.053; and ECVI = 2.63. Taking into account the values found, adequate results were observed in that model, as a value lower than 0.060 was found for RMSEA, with values higher than 0.90 in CFI, IFI, NFI and NNFI [37,38].


**Table 3.** Matrix of rotated factors.

On the other hand, and to verify if the reliability of the instrument was appropriate, Cronbach's alpha was calculated to study the internal consistency, with a value of α = 0.962 obtained for the instrument as a whole and values of α = 0.962 for factor 1, and α = 0.932 for factor 2, indicating high-reliability values [39].

Lastly, alluding to the data collection procedure, it should be noted that 15 min were provided for completing the questionnaire, with the study researchers present through a videoconference call to resolve any possible doubts that could arise during this process.

#### *2.3. Analysis Performed*

Once the collection of data were complete, the following statistical analyses were performed to provide an answer to the objectives set in this study:


#### **3. Results**

#### *3.1. Descriptive Study*

In the first place, the results (mean, standard deviation, kurtosis, and asymmetry) of the descriptive study are shown, which was conducted with the final version of the instrument, composed of 19 items (see Appendix A, Table A1).

As observed in Appendix A (Table A1), only the items that referred to reader comprehension (items 2 and 3) were evaluated low by the teachers-in-training, with the rest accepted indifferently, in disagreement with the findings from [12], highlighting those that referred to the help provided by this resource to the early childhood students, for the establishment of relationships between different matters and sizes (items 16 and 17).

As for the dimensions established by the CFA, it was verified that the descriptive values showed high values for the mean, as shown in Table 4, with the results being:

**Table 4.** Descriptive study of the dimensions.


#### *3.2. Inferential Study*

After a Student's *t*-test was performed according to gender, and with the objective of verifying if hypothesis 1 is accepted or not, the results showed that no significant differences were found. Therefore, it can be inferred that the gender variable did not determine the use of the videogames as a curricular tool in the early childhood education stage.

With respect to the starting hypothesis, an analysis of variance (ANOVA) and an analysis of covariance (ANCOVA) were performed, which did not show differences when using either age or gender as the discriminatory variable. Thus, the hypothesis set forth must be rejected, and it must therefore be pointed out that age was not a determining variable between mathematical thinking and the use of video games as a resource in the early childhood classroom.

#### *3.3. Correlational Study*

The correlational study performed referred to Pearson's correlation test for bivariate relations between the factors. As observed, the relation between the dimensions is positive. As one increases, so do the other by a high [39] and a significant amount (r = 0.710 and *p* < *0*.001).

#### *3.4. Regression Analysis*

In the search for an explanatory model that possesses the best parsimony possible, the multivariate analysis of linear regressions shows that factor 1 (development of mathematical thinking) can be explained by factor 2 (establishment of relationships between concepts) and gender and age. This analysis obtained the following parameters: F(1, 230) = 337.63 and *p* < *0*.001; with a corrected coefficient of determination R2 = 0.593, and a Durbin–Watson value = 1.9, which indicates the interdependence of the residues [40]. The only variable that intervenes in the Factor 2 equation has the following statistical values: t = 18.37 and *p* < *0*.001.

Analyzing what is shown in Table 5, the equation explains the development of mathematical thinking, factor 1 = 0.59 + 0.81 factor 2, with a root mean square error of 0.22. Likewise, to generalize the explanatory model, the residues were studied, observing the non-multicollinearity with the VIF value = 1.000, and the independence of the residues. On the other hand, their linearity and the homoscedasticity of the residues, observed in figures, comply with these assumptions [41] just as their values of normality through the Kolmogorov–Smirnov test (Z = 0.078 and *p* = 0.200).


**Table 5.** Linear regression of the development of mathematical thinking <sup>a</sup> in early childhood education with the use of video games.

Note. a. Dependent variable: development of mathematical thinking. b. Predictors: (constant) factor 2 (establishment of relationships between concepts). \* Level of significance, *p* = 0.05.

Given the complexity of the relation between the dependent variable (F1) and the independent variable (F2), an analysis of covariance (ANCOVA) was performed. Initially, an ANOVA was performed through a univariate linear model with a full factorial design between the main effect F2 and the interaction between F1 and F2, whose results are shown in Table 6.


**Table 6.** Tests of the inter-subject effects between F1 and F2.

Note. a. R-squared 0.606 (R-squared corrected = 0.599). Dependent variable F1 \*. Level of significance *p* = 0.05.

Utilizing the development of mathematical thinking (F1) as the dependent variable and the establishment of relationships between concepts (F2) as the independent one, it can be observed that the main effect F(4227) = 87.346 and *p* < 0.005, as well as the interaction, are significant.

Afterward, the univariate linear model was repeated, with the ANCOVA performed with the covariables gender and age, utilizing the development of mathematical thinking (F1) as the dependent variable and the establishment of relationships between concepts (F2) as the independent one. The results obtained are shown in Table 7.



Note. a. R-squared 0.618 (R-squared corrected = 0.595). Dependent variable F1 \*. Level of significance *p* = 0.05.

In light of the data comparisons shown in Tables 6 and 7, it can be concluded that the effect between the dependent variable F1 and the independent variable F2 is altered by the covariables, given that significance does not exist in the model. Thus, the results indicate that a study must be made with the covariables separately to verify if any of the covariables has an influence on the dependent variable, with the results shown in Tables 8 and 9.


**Table 8.** Tests of the inter-subject effects between F1 and F2 with covariable gender.

Note. a. R-squared 0.610 (R-squared corrected = 0.598). Dependent variable F1 \*. Level of significance *p* = 0.05.


**Table 9.** Tests of the inter-subject effects between F1 and F2 with covariable age.

Note. a. R-squared 0.612 (R-squared corrected = 0.598). Dependent variable F1 \*. Level of significance *p* = 0.05.

Tables 8 and 9 show that none of the covariables has an influence on the dependent variable of the univariate linear model. In summary, the results show that factor 1 (development of mathematical thinking) can be explained by factor 2 (establishment of relationships between concepts), without gender or age having an influence.

On the other hand, the participating sample was divided as a function of the student's gender, for a more itemized study of the development of mathematical thinking in the use of the video games, obtaining a variation in this variable as a percentage which is explained with factor 2.

In the case of the men, the general model only explained 17.5% (R<sup>2</sup> = 0.175), with the parameters being F(1,23) = 6.092 and *p* = *0*.021; factor 2, which is the only variable that intervenes in the equation, obtained the following statistics: t = 2.46 and *p* = *0*.021. After analyzing Table 10, the equation that explains the development of mathematical thinking for the men is factor 1 = 1.63 + 0.60 factor 2, with a root mean square error of 0.27.


**Table 10.** Linear regression of the development of mathematical thinking a,c in early childhood education for men.

Note. a. Dependent variable: development of mathematical thinking. b. Predictors: (constant) factor 2 (establishment of relationships between concepts). c. Selection of cases for which the gender = men. \* Level of significance, *p* = 0.05.

Likewise, to try to generalize this explanatory model as a function of the men, the residues were studied, observing the non-multicollinearity with a VIF value = 1.000 and the independence of the residues in the Durbin–Watson values = 1.953. In addition, their linearity and the homoscedasticity of the residues observed in the graphics complied with these assumptions [41], just as their values of normality through the Kolmogorov–Smirnov test (Z = 0.076 and *p* = 0.200).

In the meantime, for women, factor 2 explains 62% (R2 = 0.620) of the general model of the development of mathematical thinking, where the parameters are F(1204) = 335.923 and *p* < *0*.001; factor 2, which is the only variable that intervenes in the equation, obtained the following statistics: t = 18.328 and *p* < *0*.001.

Table 11 shows the values that make up the equation that explains the development of mathematical thinking for women, which is factor1 = 0.55 + 0.82 factor 2.

**Table 11.** Linear regression of the development of mathematical thinking a,c in early childhood education with the use of video games for women.


Note. a. Dependent variable: development of mathematical thinking. b. Predictors: (constant) factor 2 (establishment of relationships between concepts). c. Selection of cases for which the gender = women. \*. Level of significance, *p* = 0.05.

Likewise, to try to generalize this explanatory model as a function of the women, the residues were studied, observing the non-multicollinearity with a VIF value = 1.000 and the independence of the residues in the Durbin–Watson values = 2.066. In addition, their linearity, and the homoscedasticity of the residues observed in the graphics complied with these assumptions [41], as do their values of normality through the Kolmogorov–Smirnov test (Z = 0.078 and *p* = 0.200).

#### **4. Discussion**

The initiation of learning of mathematical contents in the early ages is an important matter for the education community, given that the establishment of prior knowledge in this area will allow the teacher to detect and determine future hurdles in the acquisition of concepts that will become more complex as the students make progress in the curricular content [4,11].

On the other hand, the addition of the methodologies defined as active, based on the inclusion of diverse digital resources, will provide the teachers with a set of tools that will allow them to bring the social reality experienced by the students closer to the educational reality where they are immersed in during a considerable period of time in their lives [42,43]. However, the inclusion of a digital resource in the classroom methodology will be determined by the perception of the teachers [44]. Thus, knowing their opinion is of the utmost importance. In this sense, to discover their predisposition towards the addition of videogames to the curricular development of the subject of mathematics in the childhood education stage, the profile of the education professional must be understood. Our results showed that the participants in this study had a low level as videogame players, given that

they spent a small amount of time playing videogames, as opposed to works by [45,46], where the teachers-in-training spent a greater amount of time playing videogames and digital games.

Focusing our attention on the first objective of this work (Determine the perceptions of university students about the question of if the use of videogames makes possible the development of the curriculum in the early childhood stage in the area of mathematics), it can be verified that the teachers-in-training considered that the videogames would help students aged from 3 to 6 years old to understand and represent some logical and mathematical notions and relationships [47], which can be linked to their everyday lives [48].

On the other hand, they also considered that it would allow students aged from 3 to 6 years old to develop mathematical skill and knowledge [27] and reading comprehension and language, as opposed to the findings by [42], but in line with those found by [49]. In addition, in agreement with [50], the early childhood teachers-in-training believed that videogames would help children to acquire thinking schemes that will bring them closer to the basic notions of order, quantity, number series and functions [51], as well as problem-solving [52].

As for the second objective (To explore the existence of different dimensions about the use of videogames, to make progress feasible in the curriculum of the early childhood stage in the area of mathematics), the existence of two dimensions was corroborated, which brought together, on one hand, the items related to the skills linked with the comprehension of concepts where other curricular elements intervened, such as language, writing, and what was named "development of mathematical thinking" [52]. Moreover, a second dimension is linked to the "establishment of relationships between concepts" [27]. As for the grouping of the items into two dimensions, we verified that a line was followed as set by [6], given that this author also discusses experimentation and discovery on one hand and the relationships between concepts on the other.

With respect to the third objective of the work (To learn about the behavior of these factors, considering their relationships and the existence of an explanatory model for them), it was verified that the two-factor general explanatory model obtained pointed out that the women were closer to it than the men.

Lastly, focusing our attention to the starting hypothesis (There are significant differences in mathematical thinking depending on the establishment of relationships between concepts, without it being influenced by the age and gender of the study subjects), it is accepted and is also in line with the results obtained by [43].

In conclusion, gamification can increase both the cognitive load and the levels of performance, and generally, the students have positive beliefs with respect to the gamification strategies. This is a methodological strategy that allows creating work habits, fomenting participation and autonomy in problem-solving, promoting continuous learning, developing self-confidence and the ability to self-evaluate, promoting mathematical abilities and skills, and it could even motivate the Student's to perform activities that seemed boring to them before [11,12,25].

#### **5. Limits**

The field of Social Sciences has limitations in the development of research studies, which is related to the size of the sample. In relation to this, in the present study, a specific sample of teachers-in-training was utilized, which initially allowed us to validate the measurement instrument created, and whose results, although not able to be generalized, can serve as the basis of future research studies which utilize random samples of the population as the starting point. On the other hand, the data are unbalanced and have no element of randomization, so the results of the inferential study are very tenuous.

**Author Contributions:** Conceptualization, V.M.-D.; methodology, V.M.-D., J.M.M.-G. and B.E.S.-R.; validation, J.M.M.-G. and B.E.S.-R.; formal analysis, V.M.-D., J.M.M.-G. and B.E.S.-R.; investigation, V.M.-D. and N.N.J.-F.; resources, V.M.-D.; data curation, V.M.-D. and N.N.J.-F.; writing—original draft preparation, N.N.J.-F.; writing—review and editing, V.M.-D., B.E.S.-R.; project administration, V.M.-D., N.N.J.-F., J.M.M.-G. and B.E.S.-R.; funding acquisition, V.M.-D. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **Appendix A**



#### **References**


**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

*Article*
