Bayesian Approach to Analyze Reading Comprehension: A Case Study in Elementary School Children in Mexico

Abstract

In the educational field, reading comprehension is connected to learning achievement, and through it, one can interpret, retain, organize and value what has been read. It is an essential ability for the understanding and processing of information in learning. Furthermore, it is an essential skill to developing sustainable education. In this sense, sustainable development needs an advanced reading comprehension ability at elementary school in order to teach and learn future knowledge areas such as climate change, disaster risk reduction, biodiversity, poverty reduction, and sustainable consumption. Nevertheless, there have been few works focused on analyzing reading comprehension, particularly in Mexico, where the reference is the Programme for International Student Assessment (PISA) test on how well the Mexican students have developed this skill. Hence, this article shows the usefulness of employing Bayesian techniques in the analysis of reading comprehension at elementary school. The Bayesian network model allows for the determination of the language and communication level of achievement based on parameters such as learning style, learning pace, speed, and reading comprehension, obtaining an 85.36% precision. Moreover, the results confirm that teachers could determine changes in lesson planning and implement new pedagogical mechanisms to improve the level of learning and understanding contents.

1. Introduction

Reading comprehension is the process through which the reader builds, based on their prior knowledge, new meanings through written texts, with the aim of reaching personal goals, developing new knowledges and applying it; in addition to participating effectively in society as a progressive set of knowledge, abilities, and strategies that individuals develop throughout their lives in different contexts and interacting with their peers. Having total control over reading comprehension allows the student to use in a much more flexible way different comprehension strategies that they will apply to other particular areas and life in general. Therefore, if the student has a low level of reading comprehension, they will not be critical, reflexive, act independently, develop they creativity, be responsible, and obtain new knowledge for their everyday life [1].

In Mexico, many children and youngsters have reading problems because of their lack of skill when reading texts and low comprehension, which affects their learning and school performance in general. The Program for International Student Assessment (PISA) has confirmed several Mexican education problems. PISA is a test carried out the countries that are part of the Organization for Economic Co-operation and Development (OECD) and whose objective is to coordinate the states’ economic and social politics that integrate it [2,3]. The PISA test aims to evaluate to what extent students have acquired the knowledge and abilities in Reading, Sciences, and Mathematics. The test results obtained in 2018 place Mexico below the average among the countries that are part of the OECD. Mexico obtained 419 points in sciences, 420 points in reading, and 409 points in mathematics. The evaluation average in each of these areas was 489, 487, and 489 points, respectively. This information shows that less than 23% of the students reached excellence levels in reading competence. This is why more and more strategies are implemented in the educational system to improve the results obtained in the international tests, for example, the incorporation of technology in education.

Furthermore, reading comprehension is an essential skill to develop sustainable education. In this sense, sustainable development needs an advanced reading comprehension ability at the elementary school to teach and learn in areas such as climate change, disaster risk reduction, biodiversity, poverty reduction, and sustainable consumption [4].

Educational technology is the incorporation of Information and Communication Technologies (ICT) in the field of education to give support to teaching and learning processes in different contexts of formal and informal education. ICT pursuits improve teaching and learning processes through reaching the educational objectives, searching for the effectiveness and meaning of learning. The inclusion of technology in educational processes requires constant innovation and substantial resource investment to offer quality academic services. This is carried out by using international standards, methodologies, study plans, and last-generation laboratories, all in the process of continuous improvement for the students’ benefit, and thanks to the advances of Artificial Intelligence (AI). The field of education is starting to change slowly, but steadily [5].

Bayesian networks are probabilistic graphical models that consist of an acyclic directed graph that codifies the probabilistic relationships of dependent and independent samples through the Bayes theorem. These are tools destined to represent a set of related uncertainties, and are appropriate to model multivariate systems oriented to classification, diagnosis, and decision making [6]. As remarked, there are two procedures for generating Bayesian networks: 1) automatic routines where a certain number of algorithms can identify the underlying structure in a dataset, and 2) expert judgment, where the knowledge of a group of experts in a particular domain is used to generate the network’s architecture [7]. We followed the second one by using the knowledge of a group of experts in a particular domain to generate the network architecture (connections between nodes).

Therefore, in this paper, we show the usefulness of employing Bayesian techniques in the analysis of reading comprehension in elementary school children in Mexico to improve the teaching–learning process through educational technology. The contributions of this document are as follows:

• We identified and used parameters such as learning style, learning pace, speed, and reading comprehension that promote appropriate reading comprehension following the Mexican educational system’s standards [8,9,10]. The levels allow us to determine and evaluate the reading comprehension of the students.
• We also developed a Bayesian network model to evaluate reading comprehension in elementary school children in Mexico.

The rest of this paper is organized as follows: Section 2 shows the works related to our research. The applied methodology in the development of the study is shown in Section 3. The evaluation of the reading comprehension of the participants is shown in Section 4. Section 5 presents the obtained results in the Bayesian network model to evaluate the participants in their reading comprehension. Finally, the conclusions of this research are presented.

2. Literature Review

In the educational field, reading comprehension is connected to learning achievement, and through it, the students can interpret, retain, organize, and value what has been read. Reading is an essential process for understanding and processing information during learning activities. In the student, reading comprehension is critical as it stimulates their cognitive-linguistic development, strengthens their self-concept, and provides self-confidence. This will only be obtained if one motivation toward learning exists. Acquiring student motivation is a difficult task that entails preparing the reading process consciously [11]. Motivation is one of the obstacles that impede teaching the importance of reading comprehension in primary schools of Mexico. This cause implies that students show little interest and performance in reading exercises due to not being adequately motivated [11]. Nevertheless, the inclusion of technology in education positively affects the students’ attitudes [12].

In addition to this, the use of technologies has increased more and more in educational institutions. The need to introduce technological tools in education is favorable due to the many resources they offer. Furthermore, Refs. [13,14,15] affirm that the use of technology allows the development of autonomous skills in students, which allows demonstrating differences regarding the effectiveness of using (or not) technology tools in text comprehension [16]. The application of technologies in education, or educational technology, implies using ICT with a pedagogical purpose, integrating them as resources within the learning process’s planning. Technologies will not replace the teachers. Instead, technology is a tool with which the educational process can operate. The tasks of designing, planning, and determining the educational strategy used are in charge of the teachers’ academic expertise [17].

The number of investigations that work with a broad group of variables where there are complex relationships is increasing [18,19,20]. However, these tasks require a computer system design under artificial intelligence principles and techniques [21,22]. A Bayesian network is a type of intelligent system. It is known as a probabilistic model, a Bayesian model, or a belief network. Obtaining a Bayesian network based on data is a learning process divided into two stages: structural learning and parametric learning. The first consists of obtaining the Bayesian network structure, in other words, the dependent and independent relationships between the involved variables. The second stage aims to get probabilities a priori and conditionals required based on a given structure. Once causal relationships between each variable and its parent are established, there are established algorithms to make inferences. It is vital to observe the network structure that supplies information about the probabilistic dependencies between the variables and the conditional independencies. The insertion of independent relationships in the Bayesian network structure is an excellent tool to represent knowledge compactly based on the propagation of probabilities according to the probability theory laws.

Since Bayesian networks have appeared, researchers have integrated them in modeling in the education domain to measure the technical and cognitive performance of a student. There are different subjective perspectives in the interpretation of the context in the educational field such as the measurement of individual educational experience, emotional state, learning methods, financial background, student learning style, and specific academic abilities [23,24,25]. Although they have been widely used in the last 20 years in the artificial intelligence community, the knowledge models based on these techniques have only had limited acceptance in general psychometry. The Bayesian networks provide a convenient way of specifying complex relationships between latent cognitive variables. However, because of these models’ complexity, there are still many challenges and open research questions that have prevented achieving a more generalized use.

Bayesian networks can obtain more impulse in typical educational evaluation practice, providing enriched diagnostic information to support all students’ academic success [26]. In this way, it is possible to model the learning style, and pace, representing the way and speed students acquire and understand the dictated or read topics. Hence, it would be possible to involve students in the pedagogical process and significantly improve their academic performance [27,28,29].

3. Methodology

In this research, we designed an exploratory experiment to understand the reading comprehension in elementary school children in Mexico. This experiment describes the study units, standardized measuring instruments, and the procedure to gather the information.

3.1. Material Resources

The Secretariat of Public Education (SPE) is the institution in charge of defining regulations, plans, and study programs for education in Mexico. The SEP has defined national standards to improve reading ability in primary school students [8,9,10] including reading speed, reading comprehension, and reading fluency. Furthermore, research shows a significant relationship between learning styles and reading comprehension [30,31]. Hence, we selected the variables learning style, learning rhythm, reading speed, and reading comprehension to determine the language and communication level. In this study four standardized instruments related to reading comprehension in elementary school children were identified:

• Learning styles test. The objective of this test is to identify the way a student learns a topic (see Appendix A).
• Learning pace test. This test identifies the speed in which a student learns or acquires new knowledge (see Appendix B).
• Reading speed test. This test counts the number of words a student can learn in a minute.
• Reading comprehension test. This test determines the comprehensive ability of a student by means of readings and questions (see Appendix C).

3.2. Development Tools

Netica is a commercial program developed for the analysis of Bayesian networks. We chose Netica because it presents a friendly interface and easy comprehension for the design and analysis of the results. It has a graphic interface to model Bayesian networks and has exact and approximate reasoning algorithms for both discrete and continuous variables [32].

3.3. Participants

The sample comprised students from the elementary school General Manuel Gutiérrez Zamora in the city of Misantla, Veracruz, Mexico during the school year 2018–2019. The experimental study integrated fifth and sixth grade students (groups A, B, and C), making up a sample of a total of 107 participants, 42 female and 65 males, within the age range of 10 and 12 years old. It is important to point out that the three groups from both grades had different teachers. This reflects the diversity of participants in the experiment and different styles of teaching reading comprehension.

3.4. Procedure

The experiment was carried out in approximately six weeks, taking into account groups A, B, and C from the fifth grade and groups A, B, and C from the sixth grade. In this period, it was decided to work with one group per week during Spanish class hours with a maximum duration of 60 min per day. On the first day, the visual, auditory, and kinesthetic (VAK) learning styles test (20 min) was applied. On the second day, the test about learning pace was applied (40 min), whose objective was to analyze how a person has to learn content quickly or slowly. On the third day, the reading speed test took place, consisting of counting the number of words that a child could read during a minute. As a response to the teachers’ suggestion, the test was carried out ten times in approximately 40 min, and at the end, the number of words per student was averaged. The reading comprehension test was applied two times on the last days of the week. This test consisted of assigning a text that the students had to read (20 min) and a series of questions related to the text that they had to answer (40 min) to analyze the comprehension of the assigned text. Figure 1 shows the complete scheme of the methodology proposed in this research, which allows for the evaluation of reading comprehension in primary education in children from the application of tests of learning styles, learning pace, reading speed and reading comprehension. These variables are used as nodes in the design of a Bayesian network that classifies the language and communication level of achievement.

4. Case Description

4.1. Data Analysis

The first test evaluates that when something needs to be learned, each individual uses their own method or set of strategies. Although the concrete strategies used differ according to what needs to be learned, each individual tends to develop some tendencies more than others—these preferences or tendencies of using determined ways of learning more than others constitute our learning style [33,34]. Although the notion of learning styles is controversial, there is research that indicates the impact of learning styles on reading comprehension [35,36,37,38]. Furthermore, the results in [27,39,40] model the learning style using Bayesian networks. In this research, the participants’ learning style is divided into auditory, visual, and kinesthetic; the results are shown in Figure 2. The experts (teachers) first established the variables of interest to build the Bayesian network (tests related to reading comprehension). Second, the graph was designed with the variables and the relationships between them (causal dependencies). In this sense, eliminating some of the variables of interest, for example, learning style generates a different Bayesian network model, which possibly works in predicting the language and communication level of achievement but does not reflect the needs identified in the study sample.

In the issue of learning, the same point happens as in the other aspects of development; each student follows a personal learning pace that should be respected [41]. Each activity and learning experience must be designed, taking into account the different pace of the students to allow them to make progress and avoid labels that discriminate and cause problems in self-esteem and how they connect with themselves [42,43]. Consequently, without putting pressure by asking more than what they can do at that moment as the risk of mental block and frustration would hinder their progress. In this sense, for the second test, the results can be observed in Figure 3.

In addition, the reading speed is the number of words that a person reads per minute with the goal of understanding the context of a text. Due to that fact, practicing specific techniques and strategies to accelerate reading speed is one of the objectives of reading comprehension. An ideal reading speed is the one that suits the reader and text in each moment because not everyone reads at the same speed, nor do we read at the same pace. This test consists of a student reading 125 words in an average of a determined reading, either a fable, story, or informative text, as long as it is a reading established as the student’s corresponding grade. Once the ability to decode text in a lineal form is present, it is necessary to increase reading speed. However, the number of words read per minute is not equal to the reading comprehension per se. The number of words per minute was transformed to a discrete value under the following criteria:

• Requires support. A student reads less than 100 words.
• Almost standard. A student reads between 100 and 114 words.
• Standard. A student reads between 115 and 125 words.
• Advanced. A student reads more than 125 words.

Bayesian networks allow for a domain of knowledge to be represented with uncertainty. In many cases, the structure and the assignment of probabilities of a Bayesian network are given through the opinion of experts, manually or traditionally [44]. Therefore, in this experiment, the teachers of the elementary school General Manuel Gutiérrez Zamora decided to apply the reading speed test presented in this article based on their teaching experience. Furthermore, it was a request that they carried out this study because they are attached to their study program, and their technical teacher’s board validated the test. Figure 4 shows the results obtained in the reading speed test of the participants in this experiment.

In the reading comprehension test, we presented a text to the students that had to be read. After reading, questions related to the text were asked to identify the students’ comprehension ability in reading. Figure 5 shows the results of the reading comprehension test.

4.2. Pedagogical Evaluations

The data analysis was centered on the test results (learning style, learning pace, reading comprehension, and speed) applied to the participants in the experiments. These four tests were the main variables of the research and, based on them, the language and communication level of achievement. The professors assigned a language and communication level of achievement based on their experience and the four applied test results.

Table 1. Pedagogical evaluation descriptions.

Evaluation	Values
Learning style	{auditory, visual, kinesthetic}
Learning pace	{low, almost low, medium, almost superior, superior}
Reading speed	{requires support, almost standard, standard, advanced}
Reading comprehension	{very weak, weak, enough, ideal}
Language and communication level of achievement	{I, II, III, IV}

summarizes the research’s main variables, which are useful for the evaluation of reading comprehension in elementary school children in Mexico.

The aim of this research was to determine the language and communication level of achievement based on the tests applied to participants. These levels established by the Secretariat of Public Education [8,9,10] in Mexico and are described below:

• Level I. Students can identify, select and extract accurate and explicit information of a related text fragment, literally, the question with the content of brief literary narrative texts (story or journal).
• Level II. Students can identify, select and extract explicit information of a related text fragment through synonyms, the question with the content of narrative texts (literary and informative), and expository, both continuous and discontinuous.
• Level III. Students can identify, select, extract and relate explicit and implicit information in several related text fragments through paraphrase, the question with content in narrative texts (literary and informative, both continuous and discontinuous). Furthermore, they can understand global and specific content based on the integration of information present in several major semantic units in the sentence.
• Level IV. Students can identify, select, extract and relate explicit and implicit information in several related text fragments through paraphrase and inferences, question the content in narrative texts (literary and informative, expository, argumentative, and dialogic, both continuous and discontinuous). Additionally, they demonstrate through graphic representations (concept map) the understanding of global and specific content as well as the textual purpose through hierarchy and integration of information.

4.3. Data Integration

The test results were applied to 107 participants and used to integrate data in a knowledge database, considering: (1) learning style, (2) learning pace, (3) reading speed, and (4) reading comprehension. Additionally, experts (teachers) gave each of the students an assigned language and communication level of achievement based on the test results and their pedagogical experience. Figure 6 shows the knowledge base generated in .cvs format.

5. Proposed Bayesian Network Model

5.1. Variables, Parameters and Relationships

The Bayesian network allows for assigning the language and communication level of achievement in elementary school students based on their style and learning pace, speed, and reading comprehension. A Bayesian network is defined as a structure and the corresponding probability model. Bayesian networks use a probability distribution in each variable that comprises it, described in Equation (1).

$$P\left(x_1,\dots ,x_n\right)={\displaystyle \prod }_{i=1}^{n}p(x_{i }| parents\left(x_i\right))$$

(1)

The probability tables’ initial data were based on the analysis of data collected from the tests applied to the students. These data are essential because the empirical construction of each probability table can be a difficult task. There are two procedures for generating Bayesian networks. The first one is based on automatic routines where a certain number of algorithms can identify the underlying structure in a dataset. The second one focuses on expert judgment, where the knowledge of a group of experts in a particular domain is used to generate the network architecture [7]. Accordingly, we defined the network crucial elements, which include the variables (nodes), links between variables, and parameters.

One way to generate a Bayesian network is to use the knowledge of a group of experts in a particular domain to generate the network architecture (connections between nodes). In this way, the relationships between nodes were established by the elementary school teachers of Escuela Primaria General Manuel Gutiérrez Zamora located in Misantla, Veracruz, Mexico. The Bayesian network proposed to evaluate and assign the language and communication level of achievement based on the test results and the experience of the pedagogical teachers is shown in Figure 7.

The proposed model represents the tests applied to the students and the language and communication level of achievement. Specifically, Figure 7 shows the final node representing the language and communication levels of reading that a student can achieve from some parent nodes such as learning style, learning pace, reading speed, and reading comprehension, influencing this end node. The data matrix represents the probabilities associated with each of the values in the nodes. In this study, we obtained complete and sufficient data for all the model variables. The probabilities were estimated based on the data frequency (maximum likelihood estimator [7]). Furthermore, 31.5% of the students were classified as level I, 23.4% as level II, 23.9% as level III, and 21.2% as level IV. The proposed model can learn from past estimates and achieve increasingly precise results, making it useful in changing learning environments such as basic education.

In Bayesian networks, a node pointed by an arrow (bow) is a dependent variable on the one at the origin of the arrow. In the proposed model (Figure 7), the dependent variables were reading_comprehension_1, reading_comprehension_2, and LC_level; the independent variables present in the model were learning_style, learning_pace, and reading_speed. Furthermore, the level of reading comprehension (node labeled LC_level) was a latent variable present in the students, and in the proposed model, we inferred this variable from the Bayesian mathematical model. As a remark, latent variables were unobserved random entities that do not have a specific unit of measurement and, therefore, any estimate is subject to an associated error. Hence, in the case of having a valid instrument for measuring a variable, any value assignment would have an associated margin of error. Furthermore, its measurement unit could vary between studies because there is no consensus on its comparison parameter [45].

In this research, we inferred the value of the language and communication level of achievement node according to the results obtained in the first and second reading comprehension tests (both with a weight of 50%). Simultaneously, the reading comprehension test results depend on the tests learning style, learning pace, and reading speed (each weighing ⅓ on the incidence of reading comprehension).

Finally, the proposed Bayesian network simulates situations during a student’s reading activity in a school. A teacher inserts the learning test results into the network and analyzes the values of the conditional probabilities. In the last step, the probability of classification by levels is observed at the final node. The result gives the teacher the percentage that a student has to understand the reading, which will allow the development of communication and language skills. In relation to this, understanding the impact of reading comprehension could lead to the development and implementation of pedagogical strategies, educational policies in the management of teaching, and will even provide crucial information to teachers on how to act so that the deficiencies in this ability do not affect school performance.

5.2. Validation of the Proposed Model

We established a group control of 41 students, formed by the test results belonging to 5^th C and 6^th C. The analysis was carried out in Netica, and the command test with cases was executed, considering the evaluation of the behavior concerning the previously carried out training.

Table 2. Confusion matrix results.

Real	Predicted
	Level I	Level II	Level III	Level IV
Level I	8	0	0	0
Level II	1	9	0	1
Level III	0	1	4	3
Level IV	0	0	0	14

shows the validation results of the confusion matrix. The model obtained an 85.36% of precision; this means that those students with level I were classified as level I (8) as well as level II–level II (9), level III–level III (4), and level IV–level IV (14).

6. Discussion

The main contribution of this work is developing a Bayesian network that evaluates the reading comprehension in fifth and sixth-grade students at an elementary school level in Mexico based on the factors learning style, learning pace, and reading speed. Furthermore, the Bayesian network will make it possible to identify students at the lowest level of reading comprehension. From this, teachers will have the necessary educational tools to plan specialized and individualized lessons within the classroom. We tested the model in an elementary school and detected that most of the fifth and sixth-grade students had a low reading comprehension level. The students were only able to identify simple information that was found explicitly in descriptive texts. Furthermore, they obtained support from graphs such as drawings or images, distinguishing only basic elements of grammar. However, there was a better performance in the reading speed test because the development of this ability was distributed in several classes, which motivated students to start their like of reading earlier and, hence, achieved better reading comprehension.

These results allow teachers to determine lesson planning changes and implement new pedagogical mechanisms to improve learning and understanding of the lessons. On the other hand, based on the generated information, the model will be able to predict if a student is at the highest levels of reading comprehension (levels III and IV). The model also generates pertinent information for the teacher about each student’s situation, allowing for feedback through activities inside and outside the classroom.

7. Conclusions

Educational technology carries out a fundamental role in decision making, planning time, activity management and, therefore, will increase student motivation during their learning experience, which is supported by the teachers’ experience. The Bayesian network model obtained an 85.36% precision when determining the language and communication level of achievement based on their learning style, learning pace, speed, and reading comprehension. Therefore, the results showed that Bayesian network implementation allows for the language and communication level of achievement to be determined with great precision. In this sense, teachers can carry out changes in class planning. For example, encouraging students to read aloud even if they have reading difficulties enhances literary text analysis from the students’ perspective through forums, debates, and round tables [46,47,48,49,50]. Furthermore, the implementation of new pedagogical mechanisms will improve the level of learning and understanding of contents.

As future work, we will extend the study including other variables that can influence the interest values to have a clearer idea about why the evaluation of reading comprehension has a deficit in Mexico. For example, we will contemplate related variables to the socioeconomic and psychological field and even analyze the differences of comprehension and learning by gender.

On the other hand, we could integrate the Bayesian network model into a decision-making system to automate the process and let the student interact with technology. Generally, they seem to be more receptive to it, obtaining trustworthy results and avoiding boredom distractions that conventional methods can generate. This intelligent system could be easily implemented in different elementary school institutions of the 32 states in the Republic of Mexico and provide a tool that can help identify the reading comprehension capacity in fifth and sixth-grade students.