Next Article in Journal
Teacher Creativity: When Professional Coherence Supports Beautiful Risks
Previous Article in Journal
Association between Non-Verbal Intelligence and Academic Performance of Schoolchildren from Taza, Eastern Morocco
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
J. Intell.
Volume 10
Issue 3
10.3390/jintelligence10030061
jintelligence-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Contrasting Profiles of Low-Performing Mathematics Students in Public and Private Schools in the Philippines: Insights from Machine Learning

by
Allan B. I. Bernardo
1,*,
Macario O. Cordel II
2,
Minie Rose C. Lapinid
3,
Jude Michael M. Teves
2,
Sashmir A. Yap
2 and
Unisse C. Chua
2
1
Department of Psychology, De La Salle University, Manila 1004, Philippines
2
Dr. Andrew L. Tan Data Science Institute, De La Salle University, Manila 1004, Philippines
3
Department of Science Education, De La Salle University, Manila 1004, Philippines
*
Author to whom correspondence should be addressed.
J. Intell. 2022, 10(3), 61; https://doi.org/10.3390/jintelligence10030061
Submission received: 29 June 2022 / Revised: 11 August 2022 / Accepted: 23 August 2022 / Published: 30 August 2022
Browse Figures
Versions Notes

Abstract

:
Filipino students performed poorly in the 2018 Programme for International Student Assessment (PISA) mathematics assessment, with more than 50% obtaining scores below the lowest proficiency level. Students from public schools also performed worse compared to their private school counterparts. We used machine learning approaches, specifically binary classification methods, to model the variables that best identified the poor performing students (below Level 1) vs. better performing students (Levels 1 to 6) using the PISA data from a nationally representative sample of 15-year-old Filipino students. We analyzed data from students in private and public schools separately. Several binary classification methods were applied, and the best classification model for both private and public school groups was the Random Forest classifier. The ten variables with the highest impact on the model were identified for the private and public school groups. Five variables were similarly important in the private and public school models. However, there were other distinct variables that relate to students’ motivations, family and school experiences that were important in identifying the poor performing students in each school type. The results are discussed in relation to the social and social cognitive experiences of students that relate to socioeconomic contexts that differ between public and private schools.

1. Introduction

Filipino students were among the lowest performing groups of students among all the participating countries in the 2018 Programme for International Student Assessment (PISA). In mathematics, less than 20% of students demonstrated the minimum proficiency level (Level 2), while more than 50% showed very low proficiency (below Level 1). Scoring below the lowest level of proficiency in the PISA, these Filipino students have been clearly left behind in terms of mathematics education; more than half of this age group of Filipino students have inadequate mathematical skill compared to their peers in other parts of the world. The poor performance in mathematics also varied in degree between the students in public and private schools, where the means were 343 and 395, respectively (Department of Education 2019).
The study aims to identify the factors (personal and contextual) that differentiate the lowest-performing students from the other Filipino students in mathematics in public and private schools in the Philippines. Previous studies have shown that public and private schools in the Philippines have very different environments for learning resources (Trinidad 2020) and for supporting student motivation and engagement (Bernardo et al. 2015), and we explore whether different factors identify low-performing students in each type of school. We use a range of machine learning approaches to analyze the Philippines 2018 PISA data from the student questionnaire and the school questionnaire and analyze the data of students from public and private schools separately. Typical education research in the Philippines studies investigate predictors of achievement at one level of analysis, the machine learning approach allows researchers to consider factors at the student level, their family, the instructional experiences in school, and other school characteristics, and thus reveal a more complex set of factors that identify the students that are left behind in mathematics education in the two types of school in the Philippines.

1.1. Filipino Students’ Mathematics Proficiency in PISA 2018

Students’ mathematics proficiency in the PISA assessment relates to the students’ capacity to formulate, use, and interpret mathematics in different contexts, including familiar personal experiences and in broader and more abstract contexts of work, society, and science. Students who are assessed to have good mathematics proficiency are able “to reason mathematically and use mathematical concepts, procedures, facts and tools to describe, explain and predict phenomena” (OECD 2019a, p. 104). The test items were given in combinations of the different mathematical processes, mathematical content, and contexts. The mathematical processes included formulating situations mathematically, employing mathematical concepts, facts, procedures and reasoning, and interpreting, applying, and evaluating mathematical outcomes. Underlying these mathematical processes were fundamental mathematical capabilities such as understanding a problem situation, its tasks, and questions; being able to present, explain and justify a solution; translating and representing the problem and its quantities into a mathematical form; and utilizing mathematical content knowledge and tools to solve the problem and to communicate results (OECD 2019a).
Six proficiency levels were described to represent the range of mathematics skills, knowledge, and understanding in the 2018 PISA mathematics assessment; the same six levels have been used since mathematics became a focal area of assessment in 2002 (OECD 2019a). Level 2 is considered as the minimum proficiency standard, and less than 20% of Filipino students attained Level 2 proficiency or better. This means that an overwhelming majority of Filipino students score below standard; more specifically, 27% scored at Level 1 proficiency and 54% scored below Level 1 (OECD 2019a). According to the PISA mathematics proficiency guide, Level 1 means: “… students can answer questions involving familiar contexts where all relevant information is present and the questions are clearly defined. They are able to identify information and to carry out routine procedures according to direct instructions in explicit situations. They can perform actions that are almost always obvious and follow immediately from the given stimuli” (OECD 2019a, p. 105). So less than 3 of every 10 15-year-old Filipino students can do math only at that level, and more than half of these students cannot even do those actions.
While the results suggest that most Filipino high school students are not learning what they are supposed to in mathematics, the situation seems to be worse for the students in Philippine high schools. On average, private school students’ scores were at Level 1 proficiency, while those from public schools were below Level 1. While about 3 of every 10 private school students scored below Level 1 proficiency in mathematics, 6 out of every 10 public school students scored below Level 1.
In a sense, the results are not surprising as the Philippines had been consistently performing poorly in mathematics in the global assessments. It had not been able to improve from the bottom 5 ranks since it joined Trends in Mathematics and Science Study (TIMSS) in 1999 (Mullis et al. 2004). However, the Philippine government chose to participate in PISA 2018 with the aim of gaining knowledge from the international large-scale assessment to help improve the current educational system (National Economic Development Authority 2020). Indeed, the PISA provides data on a wide range of variables that can be studied as possible predictors of successful (or unsuccessful) learning in the different domains. These variables might be interacting in ways that predict either poor or good mathematics achievement. In the next section, we consider the possible types of variables known to be associated with students’ mathematics learning.

1.2. Predictors of Mathematics Learning and Achievement

Research has revealed many important predictors of mathematics learning and achievement, and most of the predictors can be classified under one of five broad categories: student factors, family factors, teacher factors, classroom and school factors, and policy factors (Maamin et al. 2021). We will not attempt a comprehensive review of such factors but refer to some that were measured in the PISA 2018 questionnaire and that were included in the analysis for the current study; these factors fall under the first four broad categories, as no policy related factors were included in the student questionnaire of PISA.
Beyond the typical student factors such as gender, cognitive abilities, and metacognitive (Desoete and De Craene 2019; Lindberg et al. 2010), research has confirmed the importance of a range of non-cognitive social psychological factors predictors of student academic success (Lindberg et al. 2010; Kim and Choi 2021). In mathematics achievement, these factors include motivation (Levpušček et al. 2013; Saw and Chang 2018), goal orientations (Dela Rosa and Bernardo 2013), attitudes (Gjicali and Lipnevich 2021), self-beliefs (Damrongpanit 2019; Szumski and Karwowski 2019) and academic emotions (Villavicencio and Bernardo 2013, 2016). There are more specific student factors that relate to these social psychological factors such as the students’ educational and career aspirations; students who have higher career aspirations that also require higher educational qualifications showing stronger motivations related to achieving in mathematics (Watt et al. 2019; Webster and Fisher 2000). On the other hand, poor motivation in learning is associated with students’ absenteeism and tardiness, which are also associated with lower mathematics achievement (Vesić et al. 2021; Gottfried and Ansari 2022).
Family factors that relate to students’ mathematics achievement include factors related to the family’s socioeconomic background, which relates to parents’ education and occupation, as well as the educational resources available in the home (Lam and Zhou 2021; Lombardi and Dearing 2021; Marks and Pokropek 2019). The types of parental support for the students’ learning are also important predictors of students’ achievement (Bernardo et al. 2015; Soenens et al. 2007); parental support also relates to the quality of parent–child relationship (Christenson and Havsy 2004), parental involvement in their children’s learning in mathematics (Hyde et al. 2006; Jay et al. 2018) and expectations of their children’s achievement (West et al. 1998).
Teachers’ expectations of students also play an important role in students’ achievement in mathematics (Szumski and Karwowski 2019), as do other social and interpersonal teacher factors. Teacher characteristics relate to instructional quality, and both predict higher student achievement in mathematics (Toropova et al. 2019; Wayne and Youngs 2003; Wedel 2021). Teacher characteristics such as teacher preparation (Boyd et al. 2009; Fung et al. 2017), continuing professional development (Desimone 2013; Harris and Sass 2011), mathematical knowledge (Baumert et al. 2010), and teachers’ self-efficacy (Fung et al. 2017; Zee et al. 2018) are some of the qualities that relate to their instructional performance.
Some teacher factors are also shaped by school-level factors such as school policies on class sizes (Woessmann and West 2006) and support for teachers’ continuing professional development (Desimone 2013). However, other important aspects of the school environment also play an important role in predicting student achievement in mathematics.
The school environment can influence teachers’ and students’ behavior in the teaching and learning process and eventually students’ achievement (Trinidad 2020). A school culture that promotes shared values and norms for learning, high academic standards (Jesse et al. 2004), strong personal bonds between teachers and students showing genuine concern to students for academic success (Mateos et al. 2021) are said to be important predictors of student achievement. Other important predictors include orderly and highly structured schools, classes where rules and procedures are consistently and reasonably enforced (Ilg and Massucci 2003; Pressley et al. 2004), and school environment that encourages student participation in after-class activities (Wigfield et al. 2006).
However, perhaps one of the most important school factors that predict student achievement relates to the schools’ learning resources (Levpušček et al. 2013; Caponera and Losito 2016). In the Philippines, for example, material constraints and lower teacher resources are associated with lower student attention, lower student respect, more concerns with attendance, bullying, other problematic student behaviors, and student achievement (Trinidad 2020). These resource constraints distinguish the school environments in public and private schools in the Philippines (Lockheed and Jimenez 1994) and also other countries (OECD 2019b), and more importantly, they are associated with achievement gaps (Braun et al. 2006; Carbonaro and Covay 2010). Interestingly, one study showed that school type differences were more pronounced in mathematics achievement compared to other subjects (Lubienski and Lubienski 2006). Other studies found that the achievement gap between private and public schools in the Philippines is also associated with different levels of student motivations and perceived support from parents and teachers (Bernardo et al. 2015) and the higher student selectivity in private schools (Yamauchi 2005).

1.3. The Current Study

The various student, family, teacher, and school factors are also assumed to be interconnected in predicting students’ achievement in mathematics. For example, individual students’ career aspirations are related to their motivational beliefs about math, which are also related to how they perceive their classroom environment (Lazarides et al. 2020). Students’ self-perceptions also interact with the school’s social context in influencing students’ engagement (Wang and Eccles 2013; Fall and Roberts 2012), and their self-beliefs also interact with their socioeconomic status in influencing their mathematics achievement (Bernardo 2021). Thus it is important to try to explore a range of predictors of students’ mathematics achievement to see how they might be working together.
In the current study, we wanted to study the factors that distinguish the Filipino students who perform poorly in the PISA 2018 mathematics assessment from those who met the minimum performance standards. The PISA 2018 obtained self-report data on a wide range of factors –the students, their families, teachers, classes, and schools—that are possible predictors of students’ proficiency in mathematics. Our objective was to identify the models that best identifies the Filipino students who performed poorly in mathematics using machine learning approaches, and we wanted to identify the model for public school students and for private school students. For this purpose, we trained and evaluated different machine learning models on the PH data to determine the best classifier for classifying poor and better performing students. Eighty percent of the data were used to iteratively adjust the model’s parameters during the training phase. Training iterations were terminated based on any of the following conditions: (i) the training performance converges and is less than a preset value, (ii) the validation performance worsens, or (iii) the validation performance does not improve. Each trained model was evaluated using Region of Convergence (RoC and ROC-AUC) scores to determine how well it separates the two categories, standard metrics, e.g., average F1-score to measure its prediction performance, and cross validation to demonstrate its performance on unseen data. By exploring models for identifying poor performing students in mathematics in public and private schools, we hope to identify variables that will point to poor learners’ vulnerabilities that could be the target of interventions.

2. Methods

2.1. The Dataset

The data used in the study were derived from the Philippine sample in the PISA 2018 database (publicly accessible at https://www.oecd.org/pisa/data/2018database/ (accessed on 25 November 2021)). The sample was obtained following a two-stage stratified random sampling system. First, 187 schools were randomly selected across the country’s 17 regions, and the students were randomly sampled for each school. The sample comprised 7233 15-year-old students, and of this sample, 18.5% meet the minimum standard defined in the PISA 2018 (i.e., Level 2 or higher) and 26.9% were assessed at Level 1 proficiency. The lowest proficiency group (below Level 1) comprised 54.6% of the sample.
From the dataset, 96 variables including the estimate for the mathematics achievement (i.e., plausible values 1 or PV1MATH), the school type (SCHLTYPE), and other relevant student-, family-, teacher-, and school-related variables were considered for this exploratory study. We removed three variables with 100% missing values (these were not included in the Philippine version of the survey: ICTSCH, ICTHOME and ST225Q03HA). We also excluded students with more than 50% missing values, decreasing the number of entries to 7091 students. Of this total, 1156 were from private schools (SCHLTYPE = 1 and 2) and 5935 were from public schools (SCHLTYPE = 3).
The remaining variables with missing values in the reduced data set wre imputed using k-nearest neighbor (kNN) algorithm, where k is empirically determined as being equal to 7. PVMATH1 variable was then transformed such that the lowest proficiency students (i.e., students with PVMATH1 < 357.7 or below Level 1), was set to 1 and all the remaining better performing students (i.e., students with PVMATH1 ≥ 357.7 or Levels 1 to 6) is set to 0.
Normalization per variable, except for SCHLTYPE, was then performed such that each variable range is from 0 to 1. We further reduce the number of variables by removing variables with strong positive or negative correlation, i.e., |rho| > 0.75, resulting in a more condensed dataset with 58 variables. More details on the data description can be found in the Supplementary File.

2.2. Machine Learning Modeling

Our objective was to discover the key variables that characterize the poor performing students, or more specifically that differentiate them from the better performing Filipino students in mathematics. Machine learning (henceforth, ML) algorithms are typically used to discover the intrinsic and highly complex relationship of the input data and output data. An exhaustive search approach on the hyperparameters of different ML models, namely Logistic Regression, Multilayer Perceptron (MLP), Support Vector Machine (SVM), Decision Tree and Random Forest, was performed to zero in on the most optimal model for the classification task. Table 1 summarizes the hyperparameters considered in the exhaustive search.
The first two ML models considered, i.e., Logistic Regression and MLP, are the perceptron-type models whose generic representation is shown in Figure 1, top left. Each node h of the hidden and output layers, l, compute for the activation, z h ( l ) , given the previous activations z ( l − 1 ) , such that
z h ( l ) = f h ( l ) ( z ( l − 1 ) )
f h ( l ) ( z ( l − 1 ) ) = a ( w ( l ) T   z ( l − 1 ) )      
z ( l − 1 ) = input   x   for   l − 1   =   − 1
where the superscript indicates the layer with l = 0 to L and L is the number of hidden layers. Note that L = 0 for logistic regression. Also, the subscript indicates the node in a particular layer with h = 0 to H and H is the number of nodes for a particular layer. The operator a is the activation function and T is the transpose operation. The parameters that define the model are the weight connection values, w. Equations (1) and (2) are computed from input to output using the weight connection values. To adjust these during training, a backward pass, i.e., from output to input is performed as guided by the prediction error every iteration and the learning hyperparameters.
Another ML model considered was the kernel-based type ML model, particularly SVM, illustrated in Figure 1, bottom left. SVM looks for the most optimal decision plane to optimally separate data into different categories. The SVM decision plane is defined by wTx + b = 0, where x is the input feature vector and w is the weight vector. The training objective is to look for representative data or samples, called the support vectors, that provide maximum margin between the decision boundary and these support vectors. For non-linearly separable data, the variable space is transformed to higher dimension through transformation kernels. The nonlinearity in kernels can be varied using the kernel parameters. See Table 1 for the evaluated hyperparameters.
Finally, the last ML models considered were tree-based models which are more powerful for data whose normality cannot be assumed. Tree-based models, e.g., a Random Forest (see Figure 1 right), split the data from the top down to its decision nodes, grouping the data into the most homogeneous “sub-nodes”, based on their attributes. A Decision Tree is very intuitive and applicable for explaining key variables in the prediction decision. However, these models are prone to overfitting. The Random Forest model addresses this issue by utilizing several Decision Tree estimators. The datasets in Random Forest are bootstrapped and features are randomly sampled per estimator to form its training data. The decisions of these trees are then combined using majority voting. The quality of the overall data split is monitored using Information Gain which measures the impurity reduction.
The classification task was performed for 7091 participants from private schools and public schools. For each group of participants, samples were randomly shuffled and split into 80-20 training sets, respectively. For private schools, there were 1238 training samples after data balancing using oversampling and undersampling (i.e., 619 for each class, 0 and 1) and 232 test samples (72 for class 1 and 160 for class 0). For public schools, 5316 training samples were used after data balancing using oversampling and undersampling (i.e., 2658 each for class 0 and 1), and 1419 testing samples were set aside. Finally, the exhaustive search for the best ML model and the corresponding set of hyperparameters was performed. Each training in the exhaustive search carried out five-fold cross validation, used 600 iterations and reported the average precision, recall, F1-score and accuracy. More details on the data description can be found in the Supplementary File.

3. Results

3.1. Machine Learning Modeling Results

The results suggest that the best classifier for the task for both private and public schools is the Random Forest classifier. Table 2 and Figure 2 summarize these results.

3.2. Most Important Variables

To identify the level of importance with which poor and better performers in mathematics were classified, we used Shapley Additive Explanations (or SHAP values) which tell how to fairly distribute the prediction outcome among the features (Lundberg and Lee 2017). The SHAP value is the mean marginal contribution of a feature value across all possible feature groups. It produces a ranked list of several features in descending order, indicating the degree of significance of the features.
Initial works used top 10 (Chen et al. 2021), top 15 (Dong and Hu 2019; Bernardo et al. 2021a), and top 20 (Chen et al. 2021; Dong and Hu 2019) variables in their feature importance analysis. In this work, to manage complexity in comparing the key variables for private and public student performance classification, the 10 most significant features for the public and private school groups are analyzed and illustrated in Figure 3.
Four variables were consistent significant features for both models for the private and public school students: ST012Q05NA, ST225Q05HA, ST166Q02HA, and HISEI. All four had inverse relationships with identifying poor performing students, which means that lower scores in the variables were associated with better identification of poor performing students in mathematics.
ST012Q05NA is the questionnaire item that inquired about how many mobile phones that have internet access there are in the student’s home. So having lower values on this item strongly identified poor performing students in both private and public schools; presumably, these are students with no internet access and/or no mobile phones at home. ST225Q05HA is the item that asked the students if they expect to complete a vocational degree after high school. So students in both private and public schools who do not expect to complete this postsecondary credential are more likely to be identified as poor performing in mathematics. ST166Q02HA, is a specific item in a set of that assess students’ view of appropriate responses to receiving a possible SPAM email message. This item refers to checking the email address of the email’s sender. Students who say that checking the email address is not an appropriate response are more likely to be identified as poor performing in mathematics, both for private and public school groups. The final important variable was HISEI or the parents’ occupational status, which were scored using the international socioeconomic index of occupations (Ganzeboom 2010). Students whose parents had lower status occupations were more likely identified as poor performing in mathematics in both public and private schools.
One variable—BSMJ—was important in both private and public school models but in different directions. BSMJ is the variable that asked the students to indicate their expected job when they are 30 years old, and these were also scored using the international socioeconomic index of occupations (Ganzeboom 2010). For the private school group, higher expected occupational status negatively indicated the poor performing students; that is, students who indicated lower expected occupations were more likely to be poor performing studies. However, for the public school, the result was reversed. Higher expected occupational status directly indicated the poor performing students; students who expected higher occupational status were more likely to be poor performing students in mathematics. This is an unexpected but interesting finding that might reflect on how high school students from the two school types think about how their education (and possibly how their mathematics education) relates to the jobs they are likely to have in the future. We discuss this result in more detail in the Discussion section but use this divergent result to begin presenting the other different important variables for private and public school students.
For public school students, three non-cognitive motivation-related variables were important in identifying the poor performing students in mathematics: WORKMAST, ATTLNACT, and ST188Q02HA. WORKMAST and ATTLNACT are both indexes computed based on responses to a set of items, and both inversely related identifying poor performing mathematics students. WORKMAST represents the motivation and persistence to master given learning tasks, whereas ATTLNACT represent the value of schooling, specifically, the importance of trying hard at school to get a good job or into a good college in the future. So students who had low scores in these two motivational variables are more likely to be identified as poor performers. ST188Q02HA is a specific self-efficacy item that states, “I feel proud that I have accomplished things” and students who had a high score on this item were more likely to be identified as poor performing. The result seems odd as feeling proud about one’s accomplishment is not an emotion that one would associate with poor performance, but that is what the results indicate. Two other important variables related to the students’ school record. REPEAT was a categorical variable that indicated whether the student had previously repeated a grade level, while ST062Q01TA referred to how often the student skipped a whole day in school during the past two weeks. Both variables positively identified poor performing students in public schools.
For private school students, a different set of non-cognitive engagement-related variables were important in identifying the poor performing mathematics students: PERCOOP, EMOSUPS, and ST184Q01HA. PERCOOP represents the students’ perception that cooperation is encouraged in their school and was inversely related to identifying poor performing students; students’ who reported the cooperation is not encouraged in their private school were identified as poor performing in mathematics. EMOSUPS was the index of emotional support from parents, which was directly related to identifying poor performing student; students who report having parents who were emotionally supportive were likely to be identified as low performing. ST184Q01HA is the single-item measure of fixed mindset for intelligence (i.e., the belief that one’s intelligence cannot be changed) and was inversely related to identifying poor performing students. Therefore, students who do not believe that intelligence is fixed are more likely to be identified as poor performing in mathematics. The other two important variables were both directly related to identifying poor performing students. ICTRES was an index of available ICT resources in the students’ home, and this is a broader set of resources compared to ST012Q05NA, which were mobile phones with internet. Interestingly, students with more ICT resources were identified as poor performing, which may indicate that the ICT devices may not necessarily be used to support learning in mathematics. Finally, ST05902TA which referred to the number of required class periods for mathematics per week; fewer required class periods identified poor performing students.

4. Discussion

The aim of the study was to use machine learning approaches to pinpoint important variables that can be used to identify the poor performing Filipino students in public and private schools, with the goal of possibly identifying those factors that make students more vulnerable to poor achievement in mathematics. We analyzed data from students in public and private schools separately, assuming that there might be different identifying factors given the different environments and contexts of the two types of schools in the Philippines. Random Forest classifiers generated the best performing models for both private and public school groups, and Shapley Additive Explanations (SHAP) analysis pointed to notable similarities and differences in the top ten variables that identified poor performing students in each school type.
For students in both private and public schools, variables that indicate resource constraints identify poor performing students, but the constraint goes beyond material disadvantage and relates to aspirational constraints as well. However, the poor achieving Filipino students are also identified as having lower expectations of completing a postsecondary vocational degree and lower expected occupations when they become adults and have parents who have low status occupations also. Previous studies have noted how the occupational status of parents also tends to be associated with students’ own educational and occupational aspirations (Al-Bahrani et al. 2020; Gutman and Schoon 2018), with parents’ occupational status typically associated with socioeconomic status, as well (Lee and Byun 2019; Ng and Choo 2021). The relationship of lower student educational and occupational aspirations with lower achievement is typically associated with less positive motivations and less engagement (Al-Bahrani et al. 2020; Watt et al. 2019).
In the case of the public school students, these less positive motivations were among the important variables in the model; poor performing public school students were identified by the low importance they ascribe to trying hard at school get a good future (ATTLNACT) and lower persistence to master given tasks (WORKMAST). In the context of the poor performing students’ low educational and occupational aspirations, it seems to make sense that feeling proud of their school achievement (ST188Q02HA) even if their performance is poor also identifies the poor performing students in public schools. That is, given their limited expectations, they may be quite satisfied by their limited achievement, as well.
The variables associated with low educational/occupational aspirations and poor performance in mathematics have a different dynamic among the private school students. As mentioned earlier, higher emotional support from parents identified the poor performing private school students. This might reflect a parenting style that provides unconditional emotional support to the children, which has been shown to be an important factor in academic success among disadvantaged students (Osman et al. 2021). Low fixed mindset (or higher growth mindset) is also said to be associated with higher achievement in mathematics (Hwang et al. 2019), but it also identified poor performing mathematics students in private schools. It is possible that unconditional emotional support and the growth mindset are constructed differently by these private school students, in ways that do not relate to being more motivated and engaged to achieve; these possibilities could be investigated in future research.
The SHAP analysis for both private and public schools show that fewer (or no) mobile phones with access to the internet had a very strong impact on identifying the poor performing student in both private and public schools. This result is consistent with studies showing how mobile phones can promote more positive motivations and higher achievement in mathematics (Güler et al. 2022; Yoon and Yun 2021). However, among private school students, having more ICT resources at home also identifies the poor performing students, which may be explained by how such higher access to ICT is used for non-educational purposes. There are some studies that do show how use of ICT for leisure is negatively associated with mathematics achievement (Petko et al. 2017; Skryabin et al. 2015). So while, lack of access to a specific form of ICT resources seems to adversely affect students, in the case of private students, having more ICT resources seems to also have adverse effects on achievement, or at least seems to identify some of the poor performing students in mathematics.
Among public school students, skipping class days was also associated with identifying poor performance in mathematics. A qualitative study of absenteeism in the Philippines found varied reasons why students skip their classes, including feeling helpless in their classes, having mixed priorities, and unappealing learning environments (Clores 2009), but among students from lower income families, one reason for skipping classes is not having money for transportation to go to school, or they may be too hungry to go to school on an empty stomach (Jabar 2021). In the case of public school students, these different reasons might be intersection, and having an adverse impact on the students’ learning of mathematics.
We also note that there is one variable related to the students’ metacognitive abilities related to potential misinformation on emails. This result that was found in both private and public school students suggests that there might be some specific metacognitive skills that are lacking or not well developed among the Filipino students who are poor in mathematics (Desoete and De Craene 2019).
Note that other than the last mentioned variable, most of the important variables that identify the poor-performing students in mathematics are variables that relate to the resource limitations (i.e., associated with more disadvantages socioeconomic conditions) and associated motivations and aspirations. Indeed, the poor performing students tend to have parents with low status occupations, who do not have mobile phones that have internet access and may not have as sharp metacognitive skills in dealing with possible false information online, and who have lower educational and occupational aspirations for themselves. In the case of the students from public schools, the relative deprivations seem to be also associated with skipping classes, weaker motivations to persist in task mastery, lower appreciation of the value of education to succeed in the future, and engagement.
In the case of private school students, there are identifying variables that do not relate to resource limitations, and are instead, known to be associated with higher achievement in the research literature (e.g., ICT resources at home, emotional support from parents, growth mindset, and classrooms that encourage cooperation). Thus, among private school students, there might be students who perform poorly in mathematics for other reasons. That is, these students are doing badly in mathematics even as there are aspects of their learning experiences and environments that, in theory, should be helping them do well. Earlier we suggested that these students might be giving different meanings to these positive aspects. For example, the emotional support from their parents might be constructed as being unconditional regardless of how well they do in school and a signal not to try to work harder to achieve. Their low fixed mindset about intelligence might suggest that they do not see their poor performance in mathematics as defining their intelligence and sense of worth. Furthermore, their perception that the school encourages cooperation among learners might be recognized as an opportunity to rely on others to get by in their mathematics courses. These interpretations are speculative and will need to be further probed in future research studies.
However, what the above discussion points to is that while there are common variables that identify poor performing students across schools, there are specificities in the experience and context of private school students that suggest different identifying variables and vulnerabilities. Indeed, it is possible that if we look at a longer list of important variables, we might also find such specificities in the vulnerabilities in different public school contexts, as well.
We applied machine learning techniques to try to identify the variables that identify the poor performing Filipino students in mathematics, with the assumption that such variables will point to possible vulnerabilities or risk factors associated with poor learning in mathematics. The foregoing discussion points to most factors that relate to the student, but there are also important predictors that relate to the student’s family, and the results also indicate that school type (i.e., public or private) is a factor that also comes into the picture. However, machine learning approaches cannot actually reveal how variables at the student level interact with the variables at the family and school levels. Indeed, machine learning approaches can find the most accurate predictive models but treat variables equally without any levels. As such machine learning approaches cannot be used to test explanatory models that specify theoretical propositions regarding relationships among variables at different levels (Shmueli 2010). To test such explanatory models, statistical analysis to test multilevel models that propose interactions among factors at different levels and also control variables at different levels. For PISA mathematics data, other researchers have used hierarchical linear modeling techniques (Hu et al. 2018; Osborne and Ma 2020) and multilevel structural equation modeling (Bernardo et al. 2021b) to study such multilevel models. Given the limitations of machine learning in this regard, other have suggested combining machine learning and statistical approaches to capitalize on the strengths of both approaches for studying large scale assessment data like the PISA (Lezhnina and Kismihók 2022).

5. Conclusions

Our study points to a cluster of resource constraint-related variables that include motivational and social cognitive elements, and also possible distinct convergences of factors for those in public and in private schools. We note that in the Philippines’ Department of Education’s report on the 2018 PISA results (Department of Education 2019), the country’s educational policy decision makers reiterated their four focal thrusts in the efforts to improve student learning: curriculum review and update, improving the learning environment, teacher “upskilling and reskilling,” and engaging stakeholders for support. These thrusts have been the focal points of improving mathematics education in the Philippines for many years now (Ogena et al. 2018). Our results call attention to the need to go beyond curricular and instructional factors, as there are elements of the students’ social and psychological experiences in school that are important identifiers of poor performing students. While improving the learning environment might be a good entry point to begin addressing these vulnerabilities, the specific ways of enhancing these learning environments might require a deeper understanding of the particular social and psychological factors that make school environments less effective.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/jintelligence10030061/s1, File: Additional Data Description.

Author Contributions

Conceptualization, A.B.I.B., M.O.C.II, M.R.C.L.; machine learning methodology, M.O.C.II; machine learning modelling and evaluation: J.M.M.T.; data preprocessing and feature engineering, S.A.Y.; data visualization, U.C.C.; writing—original draft preparation, review, and editing, A.B.I.B., M.O.C.II, M.R.C.L.; project administration, M.O.C.II, M.R.C.L.; funding acquisition, M.R.C.L., A.B.I.B., M.O.C.II, All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by research grant to the third author from the De La Salle University-Angelo King Institute for Economic and Business Studies (AKI Research Grants 2020–2021 Project No. 500-138, and a Research Fellowship to the first author from the National Academy of Science and Technology, Philippines.

Institutional Review Board Statement

(The study involved secondary analyses of the officially published PISA 2018 dataset. This dataset was downloaded as a public use file from the website of the Organisation for Economic Co-operation and Development at https://www.oecd.org/pisa/data/2018database/ (accessed 25 November 2021). Therefore, neither consent to participate nor ethics approval were required for the reported analyses.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data analyzed in this study are available in the PISA 2018 Database page on the website of the Organisation for Economic Co-operation and Development at https://www.oecd.org/pisa/data/2018database/ (accessed on 25 November 2021).

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

References

  1. Al-Bahrani, Muna A., Suad M. Allawati, Yousef A. Abu Shindi, and Bakkar S. Bakkar. 2020. Career aspiration and related contextual variables. International Journal of Adolescence and Youth 25: 703–11. [Google Scholar] [CrossRef]
  2. Baumert, Jürgen, Mareike Kunter, Werner Blum, Martin Brunner, Thamar Voss, Alexander Jordan, Uta Klusmann, Stefan Krauss, Michael Neubrand, and Yi-Miau Tsai. 2010. Teachers’ mathematical knowledge, cognitive activation in the classroom, and student progress. American Educational Research Journal 47: 133–80. [Google Scholar] [CrossRef]
  3. Bernardo, Allan B. I., Fraide A. Ganotice, and Ronnel B. King. 2015. Motivation gap and achievement gap between public and private high schools in the Philippines. The Asia-Pacific Education Researcher 24: 657–67. [Google Scholar] [CrossRef]
  4. Bernardo, Allan B. I. 2021. Socioeconomic status moderates the relationship between growth mindset and learning in mathematics and science: Evidence from PISA 2018 Philippine data. International Journal of School & Educational Psychology 9: 208–22. [Google Scholar]
  5. Bernardo, Allan B. I., Macario O. Cordel II, Rochelle I.G. Lucas, Jude M.M. Teves, Sashmir A. Yap, and Unisse C. Chua. 2021a. Using machine learning approaches to explore non-cognitive variables influencing reading proficiency in English among Filipino learners. Education Sciences 11: 628. [Google Scholar] [CrossRef]
  6. Bernardo, Allan B. I., Yuyang Cai, and Ronnel King. 2021b. Society-level social axiom moderates the association between growth mindset and achievement across cultures. British Journal of Educational Psychology 91: 1166–84. [Google Scholar] [CrossRef]
  7. Boyd, Donald J., Pamela L. Grossman, Hamilton Lankford, Susanna Loeb, and James Wyckoff. 2009. Teacher preparation and student achievement. Educational Evaluation and Policy Analysis 31: 416–40. [Google Scholar] [CrossRef]
  8. Braun, Harry, Frank Jenkins, and Wendy Grigg. 2006. Comparing Private Schools and Public Schools Using Hierarchical Linear Modeling; Washington: U.S. Department of Education, National Center of Statistics, Institute of Education Sciences, U.S. Government Printing Office. Available online: https://nces.ed.gov/nationsreportcard/pdf/studies/2006461.pdf (accessed on 1 January 2022).
  9. Caponera, Elisa, and Bruno Losito. 2016. Context factors and student achievement in the IEA studies: Evidence from TIMSS. Large-Scale Assessessments in Education 4: 12. [Google Scholar] [CrossRef]
  10. Carbonaro, William, and Elizabeth Covay. 2010. School sector and student achievement in the era of standards based reforms. Socioliogy of Education 83: 160–82. [Google Scholar] [CrossRef]
  11. Chen, Jiangping, Yang Zhang, Yueer Wei, and Jie Hu. 2021. Discrimination of the contextual features of top performers in scientific literacy using a machine learning approach. Research in Science Education 51: 129–58. [Google Scholar] [CrossRef]
  12. Christenson, Sandra L., and Lynne H. Havsy. 2004. Family–school–peer relationships: Significance for social, emotional, and academic learning. In Building Academic Success on Social and Emotional Learning. Edited by Joseph E. Zins, Roger P. Weissberg, Margaret C. Wang and Herbert J. Walberg. New York: Teachers College Press, pp. 59–75. [Google Scholar]
  13. Clores, Michael A. 2009. A qualitative research study on school absenteeism among college students. The Asia-Pacific Education Researcher 18: 151–65. [Google Scholar] [CrossRef]
  14. Damrongpanit, Suntonrapot. 2019. From modern teaching to mathematics achievement: The mediating role of mathematics attitude, achievement motivation, and self-efficacy. European Journal of Educational Research 8: 713–27. [Google Scholar] [CrossRef]
  15. Dela Rosa, Elmer D., and Allan B. I. Bernardo. 2013. Are two achievement goals better than one? Filipino students’ achievement goals, deep learning strategies and affect. Learning and Individual Differences 27: 97–101. [Google Scholar] [CrossRef]
  16. Department of Education. 2019. PISA 2018 Philippine National Report; Pasig City: Department of Education. Available online: https://www.deped.gov.ph/wp-content/uploads/2019/12/PISA-2018-Philippine-National-Report.pdf (accessed on 3 January 2022).
  17. Desimone, Laura. 2013. Teacher and administrator responses to standards-based reform. Teachers College Record 115: 1–53. [Google Scholar] [CrossRef]
  18. Desoete, Annemie, and Brigitte De Craene. 2019. Metacognition and mathematics education: An overview. ZDM—Mathematics Education 51: 565–75. [Google Scholar] [CrossRef]
  19. Dong, Xin, and Jie Hu. 2019. An exploration of impact factors influencing students’ reading literacy in Singapore with machine learning approaches. International Journal of English Linguistics 9: 52–65. [Google Scholar] [CrossRef]
  20. Fall, Anna M., and Greg Roberts. 2012. High school dropouts: Interactions between social context, self-perceptions, school engagement, and student dropout. Journal of Adolescence 35: 787–98. [Google Scholar] [CrossRef] [PubMed]
  21. Fung, Dennis, Peter Kutnick, Ida Mok, Frederick K.S. Leung, Betty P. Lee, Yee-Yan Mai, and Matthrew T. Tyler. 2017. Relationships between teachers’ background, their subject knowledge and pedagogic efficacy, and pupil achievement in primary school mathematics in Hong Kong: An indicative study. International Journal of Educational Research 81: 119–30. [Google Scholar] [CrossRef]
  22. Ganzeboom, Harry B.G. 2010. A new international socio-economic index (ISEI) of occupational status for the international standard classification of occupation 2008 (ISCO-08) constructed with data from the ISSP 2002–2007. Paper presented at the Annual Conference of International Social Survey Programme, Lisbon, Portugal, May 1. [Google Scholar]
  23. Gjicali, Kalina, and Anastasiya A. Lipnevich. 2021. Got math attitude? (In)direct effects of student mathematics attitudes on intentions, behavioral engagement, and mathematics performance in the US PISA. Contemporary Educational Psychology 67: 1–14. [Google Scholar] [CrossRef]
  24. Gottfried, Michael, and Arya Ansari. 2022. Classrooms with high rates of absenteeism and individual success: Exploring students’ achievement, executive function, and socio-behavioral outcomes. Early Childhood Research Quarterly 59: 215–27. [Google Scholar] [CrossRef]
  25. Güler, Mustafa, Suphi Ö. Bütüner, Şahin Danişman, and Kadir Gürsoy. 2022. A meta-analysis of the impact of mobile learning on mathematics achievement. Education and Information Technologies 27: 1725–45. [Google Scholar] [CrossRef]
  26. Gutman, Leslie M., and Ingrid Schoon. 2018. Emotional engagement, educational aspirations, and their association during secondary school. Journal of Adolescence 67: 109–19. [Google Scholar] [CrossRef]
  27. Harris, Douglas N., and Tim R. Sass. 2011. Teacher training, teacher quality, and student achievement. Journal of Public Economics 95: 798–812. [Google Scholar] [CrossRef]
  28. Hyde, Janet S., Nicole M. Else-Quest, Martha W. Alibali, Eric Knuth, and Thomas Romberg. 2006. Mathematics in the home: Homework practices and mother-child interactions doing mathematics. The Journal of Mathematical Behavior 25: 136–52. [Google Scholar] [CrossRef]
  29. Hu, Xiang, Yang Gong, Chun Lai, and Frederick K.S. Leung. 2018. The relationship between ICT and student literacy in mathematics, reading, and science across 44 countries: A multilevel analysis. Computers & Education 125: 1–13. [Google Scholar]
  30. Hwang, Nayoung, Marcela Reyes, and Jacquelynne S. Eccles. 2019. Who holds a fixed mindset and whom does it harm in mathematics? Youth & Society 51: 247–67. [Google Scholar]
  31. Ilg, Timothy J., and Joseph D. Massucci. 2003. Comprehensive urban high school: Are there better options for poor and minority children. Education and Urban Society 36: 63–78. [Google Scholar] [CrossRef]
  32. Jabar, Melvin A. 2021. Qualitative inquiry on parental involvement in children’s education: Perspectives of parents, children, and teachers in select elementary schools in the Philippines. Asia Pacific Journal of Education 41: 488–502. [Google Scholar] [CrossRef]
  33. Jay, Tim, Jo Rose, and Ben Simmons. 2018. Why is parental involvement in children’s mathematics learning hard? Parental perspectives on their role supporting children’s learning. Sage Open 8: 1–13. [Google Scholar] [CrossRef]
  34. Jesse, Dan, Alan Davis, and Nancy Pokorny. 2004. High achieving middle schools for Latino students in poverty. Journal of Education for Students Placed at Risk 9: 23–45. [Google Scholar] [CrossRef]
  35. Kim, Suehye, and Jungah Choi. 2021. A cross-cultural examination of socio-psychological resources in mathematics achievement between Korea and the US. Social Psychology of Education 24: 1043–64. [Google Scholar] [CrossRef]
  36. Lam, Si-Man, and Yisu Zhou. 2021. SES-Achievement Gaps in East Asia: Evidence from PISA 2003–2018. The Asia-Pacific Education Researcher 22: 1–20. [Google Scholar] [CrossRef]
  37. Lazarides, Rebecca, Anna L. Dicke, Charlott Rubach, and Jacquelynne S. Eccles. 2020. Profiles of motivational beliefs in math: Exploring their development, relations to student-perceived classroom characteristics, and impact on future career aspirations and choices. Journal of Educational Psychology 112: 70–92. [Google Scholar] [CrossRef]
  38. Lee, Bora, and Soo-yong Byun. 2019. Socioeconomic status, vocational aspirations, school tracks, and occupational attainment in South Korea. Journal of Youth and Adolescence 48: 1494–505. [Google Scholar] [CrossRef] [PubMed]
  39. Levpušček, Melita P., Maja Zupančič, and Gregor Sočan. 2013. Predicting achievement in mathematics in adolescent students: The role of individual and social factors. The Journal of Early Adolescence 33: 523–51. [Google Scholar] [CrossRef]
  40. Lezhnina, Olga, and Gábor Kismihók. 2022. Combining statistical and machine learning methods to explore German students’ attitudes towards ICT in PISA. International Journal of Research & Method in Education 45: 180–99. [Google Scholar]
  41. Lindberg, Sara M., Janet Shibley Hyde, Jennifer L. Petersen, and Marcia C. Linn. 2010. New trends in gender and mathematics performance: A meta-analysis. Psychological Bulletin 136: 1123–35. [Google Scholar] [CrossRef]
  42. Lockheed, Marlaine, and Emmanuel Jimenez. 1994. Public and Private Secondary Schools in Developing Countries: What Are the Differences and Why Do They Persist? ESP Discussion Paper Series; Washington: Education and Social Policy Department, The World Bank. Available online: https://documents1.worldbank.org/curated/en/712691468743991134/pdf/multi-page.pdf (accessed on 5 January 2022).
  43. Lombardi, Caitlin M., and Eric Dearing. 2021. Maternal support of children’s math learning in associations between family income and math school readiness. Child Development 92: 39–55. [Google Scholar] [CrossRef]
  44. Lubienski, Christopher, and Sarah T. Lubienski. 2006. Charter, Private, Public Schools and Academic Achievement: New Evidence from NAEP Mathematics Data. New York: National Center for the Study of Privatization in Education, Teachers College, Columbia University, Volume 16, Available online: https://nepc.colorado.edu/publication/charter-private-public-schools-and-academic-achievement (accessed on 3 January 2022).
  45. Lundberg, Scott M., and Su-In Lee. 2017. A unified approach to interpreting model predictions. Paper presented at the 31st Conference on Advanced Neural Information Processing Systems (NIPS), Long Beach, CA, USA, December 4–9; pp. 4766–75. [Google Scholar]
  46. Maamin, Martina, Siti M. Maat, and Zanaton H. Iksan. 2021. Analysis of the factors that influence mathematics achievement in the ASEAN countries. Cypriot Journal of Educational Sciences 16: 371–88. [Google Scholar] [CrossRef]
  47. Marks, Gary N., and Artur Pokropek. 2019. Family income effects on mathematics achievement: Their relative magnitude and causal pathways. Oxford Review of Education 45: 769–85. [Google Scholar] [CrossRef]
  48. Mateos, Naiara E., Arantza F. Zabala, Eider G. Palacios, and Iker I.D. Dias-de-Cerio. 2021. School climate and perceived academic performance: Direct or resilience-mediated relationship? Sustainability 13: 68. [Google Scholar] [CrossRef]
  49. Mullis, Ina V.S., Michael O. Martin, Eugenio J. Gonzalez, and Steven J. Chrostowski. 2004. TIMSS 2003 International Mathematics Report. Chestnut Hill: Boston College. Available online: https://timss.bc.edu/pdf/t03_download/t03intlmatrpt.pdf (accessed on 2 January 2022).
  50. National Economic Development Authority. 2020. Updated Philippine Development Plan 2017–2020. Pasig City: NEDA. [Google Scholar]
  51. Ng, Irene Y.H., and Hyekyung Choo. 2021. Parental education and youth educational aspiration in Singapore: A path analysis in institutional and psychological context. Asia Pacific Journal of Education 41: 55–69. [Google Scholar] [CrossRef]
  52. Ogena, Ester B., Marilyn Ubiña-Balagtas, and Rosemarievic V. Diaz. 2018. Philippines: Mathematics and its teaching in the Philippines. In Mathematics and Its Teaching in the Asia-Pacific Region. Edited by John Mack and Bruce Vogeli. Singapore: World Scientific Publishing Company Pte. Limited, pp. 157–90. [Google Scholar]
  53. Organisation for Economic Co-Operation and Development. 2019a. PISA 2018 Results (Vol I): What Students Know and Can Do. Paris: OECD Publishing. Available online: https://doi.org/10.1787/5f07c754-en (accessed on 17 February 2020).
  54. Organisation for Economic Co-Operation and Development. 2019b. PISA 2018 Results (Volume V): Effective Policies, Successful Schools. Paris: OECD Publishing. Available online: https://www.oecd-ilibrary.org/sites/14bbef20-en/index.html?itemId=/content/component/14bbef20-en (accessed on 16 December 2021).
  55. Osborne, Michael C., and Xin Ma. 2020. Effects of student help-seeking behaviors on student mathematics achievement. Journal of Mathematics Education at Teachers College 11: 21–31. [Google Scholar]
  56. Osman, Ali, Carina C. Ydhag, and Niclas Månsson. 2021. Recipe for educational success: A study of successful school performance of students from low social cultural background. International Studies in Sociology of Education 30: 422–39. [Google Scholar] [CrossRef]
  57. Petko, Dominik, Andrea Cantieni, and Doreen Prasse. 2017. Perceived quality of educational technology matters: A secondary analysis of students’ ICT use, ICT-related attitudes, and PISA 2012 test scores. Journal of Educational Computing Research 54: 1070–91. [Google Scholar] [CrossRef] [Green Version]
  58. Pressley, Michael, Lisa Raphael, David J. Gallagher, and Jeannette DiBella. 2004. Providence-St. Mel School: How a school that works for African American students works. Journal of Educational Psychology 96: 216–35. [Google Scholar] [CrossRef]
  59. Saw, Guan, and Chi-Ning Chang. 2018. Cross-lagged models of mathematics achievement and motivational factors among Hispanic and non-Hispanic high school students. Hispanic Journal of Behavioral Sciences 40: 240–56. [Google Scholar] [CrossRef]
  60. Shmueli, Galit. 2010. To explain or to predict. Statistical Science 25: 289–319. [Google Scholar] [CrossRef]
  61. Skryabin, Maxim, Jingjing Zhang, Luman Liu, and Danhui Zhang. 2015. How the ICT development level and usage influence student achievement in reading, mathematics, and science. Computers & Education 85: 49–58. [Google Scholar]
  62. Soenens, Bart, Maarten Vansteenkiste, Willy Lens, Koen Luyckx, Luc Goossens, Wim Beyers, and Richard M. Ryan. 2007. Conceptualizing parental autonomy support: Adolescent perceptions of promotion of independence versus promotion of volitional functioning. Developmental Psychology 43: 633–46. [Google Scholar] [CrossRef]
  63. Szumski, Grzegorz, and Maciej Karwowski. 2019. Exploring the Pygmalion effect: The role of teacher expectations, academic self-concept, and class context in students’ math achievement. Contemporary Educational Psychology 59: 1–10. [Google Scholar] [CrossRef]
  64. Toropova, Anna, Stefan Johansson, and Eva Myrberg. 2019. The role of teacher characteristics for student achievement in mathematics and student perceptions of instructional quality. Education Inquiry 10: 275–99. [Google Scholar] [CrossRef]
  65. Trinidad, Jose Eos. 2020. Material resources, school climate, and achievement variations in the Philippines: Insights from PISA 2018. International Journal of Educational Development 75: 102174. [Google Scholar] [CrossRef]
  66. Vesić, Dragan, Vladimir Džinović, and Snezana Mirkov. 2021. The role of absenteeism in the prediction of math achievement on the basis of self-concept and motivation: TIMMS 2015 in Serbia. Psihologija 54: 15–31. [Google Scholar] [CrossRef]
  67. Villavicencio, Felicidad T., and Allan B. I. Bernardo. 2013. Negative emotions moderate the relationship between self-efficacy and achievement of Filipino students. Psychological Studies 58: 225–32. [Google Scholar] [CrossRef]
  68. Villavicencio, Felicidad T., and Allan B. I. Bernardo. 2016. Beyond math anxiety: Positive emotions predict mathematics achievement, self-regulation and self-efficacy. The Asia-Pacific Education Researcher 25: 415–22. [Google Scholar] [CrossRef]
  69. Wang, Ming-Te, and Jacquelynne Eccles. 2013. School context, achievement motivation, and academic engagement: A longitudinal study of school engagement using a multidimensional perspective. Learning and Instruction 28: 12–23. [Google Scholar] [CrossRef]
  70. Watt, Helen M., Micaela Bucich, and Liam Dacosta. 2019. Adolescents’ motivational profiles in mathematics and science: Associations with achievement striving, career aspirations and psychological wellbeing. Frontiers in Psychology 10: 1–23. [Google Scholar] [CrossRef]
  71. Wayne, Andrew J., and Peter Youngs. 2003. Teacher characteristics and student achievement gains: A review. Review of Educational Research 73: 89–122. [Google Scholar] [CrossRef]
  72. Webster, Beverley J., and Darrell L. Fisher. 2000. Accounting for variation in science and mathematics achievement: A multilevel analysis of Australian data Third International Mathematics and Science Study (TIMSS). School Effectiveness and School Improvement 11: 339–60. [Google Scholar] [CrossRef]
  73. Wedel, Katharina. 2021. Instruction time and student achievement: The moderating role of teacher qualifications. Economics of Education Review 85: 1–16. [Google Scholar] [CrossRef]
  74. West, Anne, Philip Noden, Ann Edge, and Miriam David. 1998. Parental involvement in education in and out of school. British Educational Research Journal 24: 461–84. [Google Scholar] [CrossRef]
  75. Wigfield, Allan, James P. Byrnes, and Jacquelynne S. Eccles. 2006. Development during early and middle adolescence. In Handbook of Educational Psychology, 2nd ed. Edited by Patricia A. Alexander and Philip H. Winne. Mahwah: Lawrence Erlbaum Associates Publishers, pp. 87–114. [Google Scholar]
  76. Woessmann, Ludger, and Martin West. 2006. Class-Size Effects in School Systems around the World: Evidence from Between-Grade Variation in TIMSS. European Economic Review 50: 695–736. [Google Scholar] [CrossRef]
  77. Yamauchi, Futoshi. 2005. Why do schooling returns differ? Screening private schools and labor markets in the Philippines and Thailand. Economic Development and Cultulal Change 53: 959–81. [Google Scholar] [CrossRef]
  78. Yoon, Meehyun, and Heoncheol Yun. 2021. Relationships between adolescent smartphone usage patterns, achievement goals, and academic achievement. Asia Pacific Education Review. [Google Scholar] [CrossRef]
  79. Zee, Marjolein, Helma M.Y. Koomen, and Peter F. de Jong. 2018. How different levels of conceptualization and measurement affect the relationship between teacher self-efficacy and students’ academic achievement. Contemporary Educational Psychology 55: 189–200. [Google Scholar] [CrossRef]
Jintelligence 10 00061 g001 550
Figure 1. Illustration of a Multilayer perceptron (top left), Support Vector Machine with linearly separable data (bottom left), and a Random Forest (right) with four Decision Tree predictors.
Figure 1. Illustration of a Multilayer perceptron (top left), Support Vector Machine with linearly separable data (bottom left), and a Random Forest (right) with four Decision Tree predictors.
Jintelligence 10 00061 g001
Jintelligence 10 00061 g002 550
Figure 2. (a) Area under the ROC curve (AUC) indicators for the private (left) and public (right) school participants. AUC score indicates how well separated are the classes 0 and 1 in the Random Forest classifier. (b) Confusion matrix for the Random Forest Classifier model for the private (left) and public (right) school participants. (c) a cursory look at the accuracy of the different ML models in the exhaustive search for best hyperparameters for the private (left) and public (right) school participants. Note that RF performs better than other ML models in terms of performance consistency regardless of the hyperparameters.
Figure 2. (a) Area under the ROC curve (AUC) indicators for the private (left) and public (right) school participants. AUC score indicates how well separated are the classes 0 and 1 in the Random Forest classifier. (b) Confusion matrix for the Random Forest Classifier model for the private (left) and public (right) school participants. (c) a cursory look at the accuracy of the different ML models in the exhaustive search for best hyperparameters for the private (left) and public (right) school participants. Note that RF performs better than other ML models in terms of performance consistency regardless of the hyperparameters.
Jintelligence 10 00061 g002
Jintelligence 10 00061 g003 550
Figure 3. Top 10 most significant variables (in descending order) in the Random Forest model classifier for (a) private school participants and (b) public school participants. Red bars represent direct relationships with identifying the poor performing students while blue bars represent inverse relationships with identifying poor performing students. SHAP values represent the level of variable importance relative to other variables.
Figure 3. Top 10 most significant variables (in descending order) in the Random Forest model classifier for (a) private school participants and (b) public school participants. Red bars represent direct relationships with identifying the poor performing students while blue bars represent inverse relationships with identifying poor performing students. SHAP values represent the level of variable importance relative to other variables.
Jintelligence 10 00061 g003
Table
Table 1. List of the considered ML models and the different hyperparameters during the grid search. Hyperparameters define the complexity of the ML and each model’s learning performance during the training.
Table 1. List of the considered ML models and the different hyperparameters during the grid search. Hyperparameters define the complexity of the ML and each model’s learning performance during the training.
ML ModelsHyperparameters
Logistic Regressionsolver: newton-cg, lbfgs, liblinear
penalty: none, l1, l2, elasticnet
c: 1 × 10−5, 1 × 10−4, 1 × 10−3, 1 × 10−2, 1 × 10−1, 1, 10, 100
MLPhidden layer sizes: (10, 30, 10), (10, 30), (32, 32), (10, 10, 10, 10)
activation: tanh, relu, logistics
solver: stochastic gradient descent, adam
alpha: 1 × 10−4, 5 × 10−3, 5 × 10−2
learning rate: constant, adaptive
SVMkernel: radial basis function, polynomial
gamma: 1, 1 × 10−1, 1 × 10−2, 1 × 10−3, 1 × 10−4
c: 1 × 10−1, 1, 10, 100, 1000
Decision Treecriterion: gini, entropy
max depth: 4, 5, 6, 7, 8, 9, 10, 11, 12, 15, 20, 30, 40, 50, 70, 90, 120, 150
Random Forestcriterion: gini, entropy
number of estimators: 200, 500
max features: auto, sqrt, log2
max depth: 4, 5, 6, 7, 8, 9, 10, 11, 12, 15, 20, 30, 40, 50, 70, 90, 120, 150
Table
Table 2. Summary of best validation performance per ML model after grid search. Text in bold indicates best ML model performance for a specific metric and school type. For both participants from private and public schools, the best classifier is the Random Forest in terms of accuracy.
Table 2. Summary of best validation performance per ML model after grid search. Text in bold indicates best ML model performance for a specific metric and school type. For both participants from private and public schools, the best classifier is the Random Forest in terms of accuracy.
School TypeML ModelValidation PerformanceHyperparameters Optimal Values
PrecisionRecallF1-ScoreAcc
PrivateLogistic regression0.630.750.680.74C: 1; penalty: l2; solver: newton-cg
MLP0.670.560.610.73activation: ‘relu’; alpha: 0.005,
hidden_layer_sizes: (32, 32)
learning_rate: ‘constant’, solver: ‘adam’
SVM0.670.020.040.63C: 10; gamma: 1; kernel: rbf
Decision tree0.540.540.540.72criterion: gini; max_depth: 12
Random forest0.690.610.650.79criterion: ‘gini’; max_depth: 20 max_features: log2 n_estimators: 500
PublicLogistic regression0.810.750.780.75C: 1; penalty: l1; solver: liblinear
MLP0.800.750.780.74activation: ‘relu’; alpha: 0.05;
hidden_layer_sizes: (32, 32)
learning_rate: ‘constant’, solver: ‘sgd’
SVM0.750.760.750.70C: 100; gamma: 0.1; kernel: rbf
Decision tree0.760.760.760.71criterion: gini; max_depth: 6
Random forest0.810.780.790.79criterion: ‘gini’; max_depth: 15
max_features: auto n_estimators: 200
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Bernardo, A.B.I.; Cordel, M.O., II; Lapinid, M.R.C.; Teves, J.M.M.; Yap, S.A.; Chua, U.C. Contrasting Profiles of Low-Performing Mathematics Students in Public and Private Schools in the Philippines: Insights from Machine Learning. J. Intell. 2022, 10, 61. https://doi.org/10.3390/jintelligence10030061

AMA Style

Bernardo ABI, Cordel MO II, Lapinid MRC, Teves JMM, Yap SA, Chua UC. Contrasting Profiles of Low-Performing Mathematics Students in Public and Private Schools in the Philippines: Insights from Machine Learning. Journal of Intelligence. 2022; 10(3):61. https://doi.org/10.3390/jintelligence10030061

Chicago/Turabian Style

Bernardo, Allan B. I., Macario O. Cordel, II, Minie Rose C. Lapinid, Jude Michael M. Teves, Sashmir A. Yap, and Unisse C. Chua. 2022. "Contrasting Profiles of Low-Performing Mathematics Students in Public and Private Schools in the Philippines: Insights from Machine Learning" Journal of Intelligence 10, no. 3: 61. https://doi.org/10.3390/jintelligence10030061

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop