Evaluation of Using Mathematics Educational Software for the Learning of First-Year Primary School Students

Abstract

The use of educational software offers many advantages but may also become a frustrating experience for teachers and students that lack experience of its use. We presented an evaluation of the use of Maths educational software to support the learning of first-year primary students in the city of Mazatlan, Mexico. One of our reasons was to determine whether the use of Information and Communication Technologies (ICTs) would motivate students and make learning easier, more interesting and even fun. Three surveys were conducted wherein the following people participated: teachers who teach Maths in the first-year of primary school; managers of the computing facilities used in this study; and first-year primary students. Due to time constraints, the three surveys were only applied in three schools, in two shifts in each school. Overall, our study suggests three key aspects for learning to be improved through ICTs, namely: (1) learning activities should be designed considering the use of educational software; (2) learning materials and activities should include guiding material to facilitate learning; and (3) promote learning through play. We propose a few design principles for educational software to support learning better and more effectively.

1. Introduction

Education, together with science and technology, are the main drivers of the progress and transformations of a country. Mexico must make them the support of its development. In this idea, the Digital Skills for All Programs (PHDT, in Spanish), a strategy adopted by the Ministry of Public Education of Mexico (SEP, in Spanish) to promote the development and use of information and communication technologies (ICT) in basic education schools to support the learning of the students, to extend their competencies for life and to favor their insertion in the knowledge society. It is important to note that the goal of the Digital Skills for All Program is to contribute to the development of digital skills and the use of ICT not only among students but also between managers and teachers. The above, based on differentiated educational models relevant and operable according to each educational level [1]. According to Pere Marques, “educational software includes computer programs created with the specific purpose of being used as teaching tools, that is to say, used to facilitate the teaching and learning processes” [2].

At present, students and teachers face new challenges that demand different attitudes towards mathematical knowledge and different ideas about what it means to teach and learn. It is not a question of the teacher looking for the simplest and most amiable explanations, but also of analyzing and proposing interesting and duly articulated problems so that students take advantage of what they already know and advance in the use of increasingly efficient techniques and reasoning.

The approach of helping students to study mathematics, based on study activities supported by carefully selected problem situations, may prove strange to many teachers who are sympathetic to the idea that their role is to teach in the sense of transmitting information. However, it is worth a try as it opens the way for a radical change in the classroom environment; it will be noticed that the students think, comment, discuss with interest and learn, while the teacher revalues his work.

There is a wide range of approaches to creating educational software when considering the different types of interaction that are possible between the actors in the teaching–learning process, i.e., educator, learner, knowledge, and computer.

Educational software according to Pere Marques is characterized by ([3,4]),

• having been developed with a didactic purpose;
• using computers as support;
• being interactive;
• adapting to the pace of work of the student;
• easy to use.

In 1966, Seymour Papert, Wallace Feurzeig, and Daniel Bobrow discuss new techniques for teaching children using computers. The result was Logo, an aspiration to revolutionize fundamental methods in education. In 1981, Seymour Papert founded Logo Computer Systems, Inc. (LCSI) with the primary objective to create software that helps “develop a child’s problem-solving and critical thinking skills [5].

Since the 1990’s, commonly, computers have been used as a tool in the classroom and in laboratories to support the teaching practice; however, such use has been poor overall. For instance, Vicente Ferreres, in his PhD thesis, “The Center as a Basic Unit of Change” (from the Spanish: El Centro como Unidad Básica de Cambio), notes that:

In some of the schools that were surveyed it was apparent the little knowledge of teachers on the use of the computer and how to use it to support their teaching in general, and to address certain areas in particular, as in the case of mathematics. On the other hand, the teachers in the classroom and laboratory used cooperative work to support their activities, which could help them to plan and develop tasks relevant to team work in order for students to reinforce the knowledge acquired in the classroom with lab activities.

[6]

There are many multimedia materials available in the market that can be classified as educational software and that address learning in various ways. Some support the teaching of mathematics efficiently, taking into account certain minimum requirements specific to the level and content to be taught ( [7,8,9,10,11]).

There also exist several studies about the effects of a mathematics software program in several levels of education as shown the authors in [12,13,14,15,16] in levels that analyze that mathematics teaching can be made much more interesting, inventive and exploratory.

However, it is necessary to implement new educational reforms in order to achieve good results in the area of mathematics, among other areas. Such reforms should foster a new paradigm, wherein the teacher is considered a mediator between knowledge and the learner, and learning is based both on theories of learning and on the aspiration of teaching for life for the purpose of improving the quality of both the teaching and learning in the educational process.

At all levels of education, the purpose of using computers is to improve learning, i.e., that students learn more, easier and better. According to surveys conducted by the Ministry of Public Education of Mexico (SEP: Secretaría de Educación Pública) [17], the use of computers in education tends to enrich learning, to highlight strengths, to eliminate or substantially reduce the sense of failure, to help identify talents and vocational interests, to reduce the antipathy against a subject, and to foster improvement in the quality of life of students. In the same surveys, students have also expressed their desire to learn through play in an interesting, colorful environment that catches their attention. With this purpose, the use of educational software is conceived to give priority to education. However, it is through the interaction with educational software that basic computing skills are promoted and developed for proper use of the computer in the teaching-learning process. This work is aimed to strengthen the skills acquired by first-year students, but it should also be useful for other people. Such as professors who work with these students and directors in charge of managing the study plans.

It also aims to help students’ integration into society, through providing them with another alternative to reaffirm and maintain their knowledge and to help them discover the skills they possess.

The aim of the investigation was to present results about the positive impact of the use of educational software in mathematical courses in the first level of primary. The purpose of this article is the evaluation of using mathematics educational software for the learning of first-year elementary school students.

2. Ease of Adoption/Use of Educational Software

There are several studies in current literature with regard to the adoption of educational software, some of them provide an analysis of the factors that influence its adoption. In [18], the author argues that the technology is mature enough, although care needs to be taken about the definitions of terms used, and remarks that the problem is that we know enough about the design of educational technology environments, but that this knowledge is not widely applied. In this sense, Buabeng-Andoh [19] highlights that the barriers that prevent teachers from the adoption of educational software use are the teachers’ lack of ICT skills, confidence, pedagogical training, and suitable educational software, as well as the limited access to ICT, the rigid structure of traditional education systems, restrictive curricula, etc. With regard to m-learning (mobile learning), besides its spreading rapidly in many regions of the world, research addressing the driving factors of m-learning adoption is in short supply, a solution for this issue is the one proposed in [20], the authors proposed a hypothesized model of m-learning adoption, the results of the application of this model report that offering high-quality contents complying with students’ future targets is key to the success of m-learning in China. Other factors that affect the software adoption and use according to [21] are the self-efficacy and the gender issues. Regarding the students’ response to the use of educational software, the response is usually judged based on a change in motivation and overall learning on the topic. This response is evaluated mainly by student interviews and scores obtained during the tasks. In this respect, studies, such as the one presented in [22], demonstrate that traditional lecture styles are not the best, valuing feedback, class standing, previous experience with lecture courses, anticipated course performance, and amount of clicker use in the classroom. These results underscore the importance of considering the social and the communicative elements of the classroom when adopting student response technology. One important fact highlighted by the students to improve their response to the use of educational software is named the persona effect, which is that the presence of a lifelike character in the educational software interface can have a strong positive effect on students’ perception and in their learning level [23].

There are actually a lot of different types of educational software, including: interactive software for preschool children who are not ready to use a mouse or keyboard quite yet; educational software for teaching academic skills such as mathematics; programs that will teach children basic keyboarding skills; software that teaches children how to think logically and to solve problems and artistic software that students can use to create and color 3D pictures.

Educational software can also enable students to gather information that otherwise would have been impossible, time-consuming, or costly. For instance, data from outer space can now be utilized. Students can also experiment with the changing aspects of a model like increasing or lowering interest rates in order to see how this affects the economy [24]. All of this can be very motivational for students and these are just some of the numerous advantages of educational software. With regard to disadvantages of the use of educational software, it needs to be carefully chosen in order to ensure that what is holding students’ attention is also educating them at the same time. Moreover, until now, it is rare to find a school that has enough computers for every student, thus scheduling can become problematic. Finally, one of the most discussed disadvantages is that it is difficult to get a student’s attention whenever they are on the computer [25].

3. Basic Education in Mexico

Basic education in Mexico spans nine years and includes both primary education and secondary education; primary education spans six years and starts at five or six years of age. Basic education is a right and is compulsory for any child in the country, and can be free. The illiteracy rate in Mexico is 10% according to the United Nations Educational, Scientific, and Cultural Organization (UNESCO) [26].

In the first grade in elementary school, the expected learning, according to the mathematics guide of the SEP [27] are: numerical sense and algebraic thinking. The students are expected to demonstrate satisfactory progress in the following areas: moving from everyday language to a mathematical language to explain procedures and results, expand and deepen knowledge, so as to promote the understanding and efficient use of mathematical tools and moving towards autonomous works. The curricular standards of this period correspond to two thematic axes: (a) numerical sense and algebraic thinking (numbers and numeral systems, additive problems, multiplicative problems) and (b) form, space and measure (figures and geometric bodies, measure). The attitudes towards the study of mathematics are:

• Develops a positive concept of self as a user of mathematics, taste and inclination to understand and use notation, vocabulary and mathematical processes.
• Applies mathematical reasoning to solving personal, social and natural problems, accepting the principle that there are various procedures to solve particular problems.
• It develops the habit of rational thinking and uses the rules of mathematical debate when formulating explanations or showing solutions.
• Share and exchange ideas about procedures and results when solving problems.

Finally, mathematical competences, whose development is important in elementary school: solve problems autonomously, communicate mathematical information, validate procedures and results and manage techniques efficiently. ICTs have changed and continue to change the way people in all walks of life communicate and interact with each other. In education and training, ICTs make it possible to reach more people (students and trainees) anywhere, anytime and in different ways according to the learning skills of each student.

As a result, the traditional educational model based on the blackboard, printed books and lengthy lectures by a teacher seems no longer adequate. Certainly, it should be complemented with the use of computers and the Internet, since, as Buckingham notes, “in all industrialized societies, and also in many developing countries, children today spend more time in the company of the media than in front of a book” [28].

Although the Program for International Student Assessment (PISA) of the Organization for Economic Cooperation and Development (OECD) aims to evaluate education systems worldwide by testing the skills and knowledge of 15-year-old students, it is important to mention that the results for Mexico in 2016 show that only 52% of Mexican students achieve minimum learning in science, 43% in mathematics and 59% in reading. In addition, in all three domains, less than 1% of students are considered to be high performers. In the three domains evaluated, Mexico is below the average of the OECD countries, Canada and the United States (trading partners), Spain, Portugal, Chile and Uruguay (Ibero-American countries); at the same time (with some exceptions) from Costa Rica and Colombia; and, above Brazil, Peru, Dominican Republic and the average of the Latin American countries. However, considering their socioeconomic levels, the scores of Mexican students are slightly higher than expected. However, Mexico is one of the Latin American countries that presents the smallest gap between the highest and lowest students in the PISA tests, which speaks of greater equity in educational outcomes.

Learning objectives using ICTs seek to develop abilities, which can be classified into four major areas: planning; using software; searching for information; and communication through different network technologies.

With the increasing use of technology (in education), we found that teaching and learning are becoming a collaborative process between adults and children. The schools will become a learning community that will be able to lead and manage learning with the support of their teachers and other children; they will feel more control over the learning and therefore more enthusiastic [29].

4. Educational Software

According to Màrques [3], the term “educational software” refers to all the software developed with a didactic purpose in mind, including: the traditional programs based on the behavioural models of teaching; Computer Assisted Teaching (CAT) programs; and the experimental Computer Assisted Intelligent Teaching (CAIT) or Intelligent Computer-Assisted Instruction (ICAI) programs. The latter use techniques of the field of expert systems and artificial intelligence in order to imitate the personalized (for-each-student) tutorial work made by teachers; they also use models of knowledge representation in line with the cognitive processes that students develop.

However, this definition, mostly based on a criterion of purpose rather than of functionality, excludes (as being educational software) all the programs of general use in the business world which, in educational centers are used for, and are the target of, training purposes, for example: word processors, database managers, spreadsheets, graphic editors, among others. There is nowadays a huge variety of educational software in most subjects including, of course, mathematics [30,31,32,33,34,35]. They have proved most useful in many ways, including a means to discover students’ abilities. In addition, several research works have been conducted regarding the perception of technology in learning and teaching mathematics [36,37].

Most mathematics educational software applications include many learning activities classified by learning objectives and age, the latter allowing for filtering access to activities adequate for children.

Educational software in general should be easy to use, intuitive, interesting, and, of course, effective in producing learning in students. This requires good instructional design, which, for young children, may be embodied as games in order for them to enjoy and to willingly spend time in sessions related to knowledge acquired in lessons taken at school.

Sacar 10 an Internet-based platform helps students enjoy mathematics and improve their results by motivating them through interactive games and exercises in a dynamic environment appropriate for any style and pace of learning. The achievement of learning is achieved by facilitating the student to draw mathematical abstractions from information, observe properties, establish relationships and solve concrete problems. It has a large number of exercises with which each concept is approached from different angles, with visual representations, text problems, interactive activities, etc, rewards the effort and encourages students to seek the information they need to learn from their own mistakes and move toward new challenges, increasing their self-esteem and building their own mathematics skills in a fun way.

5. Methodology

This study was carried out in three schools located in the urban area of the city of Mazatlan in Mexico. In each school, three sets of surveys were applied: (1) to teachers who teach Mathematics in the first-year of primary school (four professors per school, 12 professors in total); (2) to managers of the computing facilities used in the study (two managers per school, six managers in total); and (3) to the first-year primary students in two shifts in each school (115 students in school 1, 135 students in school 2 and 132 students in school 3—383 students in total).

Our surveys had the purpose of determining:

• To evaluate using mathematics educational software for first-year elementary school students from the perspective of students, teachers and school administrators.

The process of observing students while they used an educational software in the computing facilities spanned two months. During these two months, the educational software used was sacar10 (getting10) [38], a web-based graphical system designed to foster the learning and practice of mathematics skills at preschool, primary, and secondary schools in most subjects according to the current study plans by the Ministry of Public Education of Mexico. It has plenty of mathematics games and interactive exercises for each year level and it helps students by gradually increasing the complexity of the tasks.

Sacar10 includes several exercises depending on the subjects (i.e., counting, sums, subtractions, comparisons, relations, combinations, etc.) and the students learn through visual representations, interactive activities, and games. Motivating the students in a dynamic environment, and its suitability to each style and learning level (see Figure 1).

Teachers were trained in the use of the educational software so that they could help the managers of the computing facilities in assisting students carrying out their tasks with the software and in responding to students’ questions (see Figure 2 and Figure 3).

After the time of evaluation (two months), the surveys were applied to teachers, managers of the computing facilities, and to students in order to analyze the advantages or disadvantages of using this technological tool.

Table 1. Questions applied to teachers, managers of the computing facilities, and students.

SURVEY
MANAGERS OF THE COMPUTING FACILITIES
1.	What is the level of utility of the educational software when it is implemented in the learning of students of first level of primary?
2.	When the software is implemented, what kind of software offers better results in the learning process of the students of first level?
3.	Is the implementation of these activities in class useful for the professors?
4.	Did the use of educational software in class have a positive impact on students?
5.	Did the use of educational software in class have a negative impact on students’ data?
TEACHERS
1.	Do you know the concept Educational Software?
2.	Have you used some educational software as support in the teaching process in class?
3.	Which software have you used recently?
4.	Which subject has the higher rate of disapproval?
5.	Do you believe that the use of educational software improves the learning process in students, that is, the students learn in an easier way?
6.	What is the percentage of distraction in students when using the computer as support in learning?
7.	Do the students require extra assistance to use educational software?
STUDENTS
1.	Do you know what educational software is?
2.	Did you use some educational software?
3.	Do you believe that it is easier to learn mathematics using educational software?
4.	Do you believe that it is more fun to learn mathematics using educational software?
5.	Did you need extra help in order to use the educational software?
6.	Did you need extra help in order to use the educational software?

summarizes the questions that were used to analyze the benefits of the use of educational software. The survey consists of a few questions about the utilization of the software in the previous two months. To students, because of their age and, in some cases, lack of fluency in reading, the surveys were applied through an interview explaining the questions in an easy and familiar way. The surveys also were applied to teachers and managers of the computing facilities due to the importance of knowing the point of view of them in the learning–teaching role.

6. Results

The results of educational software sacar10 (getting10) [38] in the three schools of analysis are presented in this section after two months of use.

Working along with teachers to determine the impact of using educational software, we determined that Mathematics are the subject with the highest level of failure, according to teachers that teach in front of students. Seeing Figure 4 clearly shows that Mathematics is the subject with the highest level of failure. In addition to the question, “What is the reason for so much failure in this subject?”, 60% of the teachers answered that it was because of the difficulty of the subject.

Most teachers and managers of the computing facilities thought that the use of educational software would have a good impact on the learning of students as can be seen in Figure 5. Most of them thought that this was because the use of the computer involved an element novelty that attracted students’ interest.

However, for the question “What is the percentage of distraction that students show when using a computer?”, it was found that a greater percentage of students get distracted, as perceived by teachers, compared to students that do activities not supported by the computer. Furthermore, 67% of the students are distracted with another activity that has not been planned by teachers and only 33% are engaged in what the teacher asked them to do. Regarding the students’ perspective, the survey results show that a high percentage of students like to use an educational software program, and that they think it is easy and fun to use. Around 70% of the students consider that learning is easier when educational software is used.

Table 2. Percentage of distraction to the students using computer and learning skills using educational software.

Question	Percentage	N
Students are distracted with another activities	67%	256 students
Students are engaged in what the teacher asked them to do	33%	127 students
Students consider that learning is easier when educational software is used	70%	268 students

resumes this information.

On the other hand, although using technology helps achieving better results overall, students need assistance in order to use educational software properly. Figure 6 shows that 79% of the students (302 children) said that they need help from the teacher or someone else to understand properly the functioning of the educational software (perhaps because they were first-year primary students), while only 21% (81 students) said they understood clearly the task they had to do.

Even though students presented a higher level of distraction, whether trying to start other software or finding games in the computer, etc., the students obtained better results in class as seen in Figure 5. This behavior is present because, after having explained the use of the educational software, extra assistance is needed due to this being a new tool in the class. This is when the students start working on the topics of the class with higher interest and obtain better final results (Figure 5).

Finally, the perception of managers of computing facilities, teachers and first-year primary students regarding the positive impact that using educational software has on learning is compared. More than 50% of people in all these groups think and perceive that the use of educational software does help to get better results in learning.

7. Discussion and Suggestions for Improvement

We need a discussion on the reason for the three recommendations: (1) learning activities should be designed considering the use of educational software; (2) learning materials and activities should include guiding material to facilitate learning; and (3) learning through play.

The purpose of this study was to examine the software’s impact and to introduce educational software to a class of students of the first year (in this case, sacar10). By examining educational software in literature, we found that The Waterford Mental Math Games [39] program was one of the programs that teachers reported using for drill and practice in mathematics. When students used the program, they completed an electronic mathematics work sheet containing basic addition problems (e.g., $14+9=23$). Students worked as fast as they could because they knew that they were being scored on speed as well as accuracy. When they completed the worksheet, the computer gave them a score that was represented in two bar graphs—one for accuracy and another one for speed. Another successful educational software program is Exploring Measurement, Time and Money [40]. This program provides students with a variety of ways to interact with mathematical concepts.

The findings indicate that the views about effective computer-based pedagogy are related to the types of software teachers reported using with their students. We think that sacar10 can be easily adapted considering the experience of educational software’s such as Waterford Mental Math Games and Exploring Measurement, Time and Money. Specifically, the educational software provided an environment and experiences that allowed students to construct and develop their own understanding of a concept. This educational software described suggests that teachers’ selection of different types of educational software can lead to very different learning experiences for students.

Finally, it is important to mention that the main factors affecting the adoption and use of educational software are human, and these can only be addressed through effective leadership and change management.

Regarding the results obtained in this paper, the software used in this study, sacar10 [38], has been found to support and improve the learning of mathematics well overall. This conclusion was obtained from the survey answers, from the teacher’s perspective (see Figure 5) and the student’s perspective (see Table 2). However, we believe the design of educational software in general, and sacar10 in particular, should be improved as described next in order to facilitate learning even further. Particularly, according to the results of our study, it would be convenient to reduce the level of students’ distraction and the level of assistance to students in understanding the functioning of the educational software. The latter point is rather important because a single teacher will find it difficult to assist many students.

Although the educational software sacar10 is rather good overall, because of the positive perception of the use of educational software after two months of use (of this software specifically), it shows too much information on each page and this tends to distract students, as suggested by Colvin Clark [41], Applying the Coherence Principle: Adding Material Can Hurt Learning. That is, sacar10 or other educational software should display information in a simplified way. For instance, sacar10 shows a boy and girl saying supporting phrases in text that should only be shown at the start of a session but not in each page. In addition, on each page, there is also too much textual information that is not relevant to the exercises being solved by the student, e.g., information regarding material for other years, which is clearly not relevant for a first-year student once it has started a study session. Other principles discussed in [41] could be used to design better sacar10 interface.

Distraction could also be reduced through a more interactive interface based on audio. Given that first-year primary students still do not read well in general, it would help them to concentrate if the educational software gives students instruction through audio, and if students can answer either through audio or through clicking on displayed options. We also believe it convenient to personalise each session so that the educational software addresses each student by name, and interacts with each student according to the answers that each student gives and how long each student takes to respond. For instance, “Do you need more time Pedro?”, or “Would you like the exercise to be explained again”, etc.

Sound-based interactivity requires that each student uses earphones so that the interactivity of the educational software with each student does not interfere with that of other students. However, earphones are rather cheap and the benefit may be significant. Perhaps the most costly aspects are speech synthetizing and speech recognition, but some smartphones already do both.

Sound-based interactivity is also, we believe, the most promising way to reduce assistance to students. As mentioned already, the main reason for this problem is that first-year primary students still do not read well in general. Thus, having to read instructions to solve exercises is more likely to be felt like an exam in itself, thus adding to tension and reducing any ability to understand textual instructions. Sound-based friendly instructions, including a welcoming introduction to each student by name, could even help students relax and hence do better. See, for instance, [41], Applying the Personalization Principle: Use Conversational Style and Virtual Coaches.

8. Conclusions

We have presented an evaluation of the use of Mathematics educational software to support the learning of first-year primary students in the city of Mazatlan, Mexico. Overall, our study suggests, based on the surveys conducted, that the use of mathematics educational software has a positive impact on the learning of those students. However, there is still much room for improvement. Particularly, it was found that there is a need to work on the following issues: (1) learning activities should be designed considering the use of educational software; (2) learning materials and activities should include guiding material to facilitate learning; and (3) learning through play. We have also proposed a few design principles for educational software to support learning better and more effectively in the future. It is considered to extend this research and to include data of all the states of Mexico; at this moment, the results are representative only of the city of Mazatlan, Mexico. In addition, in the future, implementing multivariate data analysis techniques with the purpose of performing predictions of academic performance based on a set of student data is also considered.