Sustainable e-Learning by Data Mining—Successful Results in a Chilean University

Abstract

People are increasingly open to using online education mainly to break the distance and time barriers of presential education. This type of education is sustainable at all levels, and its relevance has increased even more during the pandemic. Consequently, educational institutions are saving large volumes of data containing relevant information about their operations, but they do not know why students succeed or fail. The Knowledge Discovery in Databases (KDD) process could support this challenge by extracting innovative models to identify the main patterns and factors that could affect the success of their students in online education programs. This work uses the CRISP-DM (Cross-Industry Standard Process for Data Mining) methodology to analyze data from the Distance Education Center of the Universidad Católica del Norte (DEC-UCN) from 2000 to 2018. CRISP-DM was chosen because it represents a proven process that integrates multiple methodologies to provide an effective meta-process for data knowledge projects. DEC-UCN is one of the first centers to implement online learning in Chile, and this study analyses 18,610 records in this period. The study applies data mining, the most critical KDD phase, to find hidden data patterns to identify the variables associated with students’ success in online learning (e-learning) programs. This study found that the main variables explaining student success in e-learning programs are age, gender, degree study, educational level, and locality.

1. Introduction

Current advances in education and technology facilitate people to develop competencies in defined areas at home [1]. As [2] highlight, online learning is a model that has revolutionized education thanks to the inclusion of Information and Communication Technologies (ICTs) and the growing of Educational Data Mining (EDM). Educational institutions place attention on this revolution for the use of new methodologies in the educational process [3]. Multiple studies exist that evaluate the success of online learning technology platforms, mainly based on the success of DeLone and McLean information systems model to measure and assess the success and sustainability of electronic learning systems [4,5]. However, despite the rapid growth of online learning and EDM, there are many problems faced by institutions that offer online courses, and the variables that impact student success in distance education is yet unknown.

Tools for identifying behavioral data patterns and factors for the success of online learning already exist [6]. This study fills the gap in terms of the variables for the student success in e-learning programs by adapting the data mining methodology CRISP-DM (Cross-Industry Standard Process for Data Mining) to discover the variables of success [7]. E-learning could readily meet the needs, features, and requirements of potential students who select this modality of study [8,9,10], even more so during the pandemic [11]. No previous research exists in South American countries that identifies determinants for the student success in e-learning programs.

Knowledge Discovery in Databases (KDD), commonly known as data mining [12,13], is a process for the patterns discovery and predictive modeling in large databases [14]. KDD makes extensive use of data mining methods, automated techniques, and algorithms for pattern recognition and identifying hidden patterns in e-learning environments [15]. Characteristically, data mining uses machine learning methods developed in the domain of artificial intelligence [16]. Data mining uses statistical, mathematical, artificial intelligence, and machine learning techniques to extract and identify pertinent information and related knowledge hidden in large volumes of raw data [17]. Data mining is technically the process of finding correlations or patterns between thousands of fields in large databases [15]. Data mining finds these patterns and relationships using data analysis tools and techniques to build models and machine learning [18,19].

Data mining comprises various techniques for pre-processing, analyzing, and interpreting data. Most researchers in the area agree that we could organize them into pattern recognition and machine learning. Pattern recognition aims to identify implicit objects and relations, and machine learning techniques are mainly applied to extract generalized knowledge from data for use in prediction tasks. Researchers can use classification techniques in data mining to predict group membership for data occurrences. Consequently, data mining involves more than collecting and managing data because it includes analysis and prediction. Classification techniques allow the processing of a wider variety of data than regression, and they are growing in popularity. There is a great variety of algorithms for classification purposes. Scheuer and Mclaren [20] propose a model to identify the most influential factors that predict student academic performance. They predict students’ passing or failing status by considering and defining their academic performance (high, medium, or low).

Educational Data Mining (EDM) is concerned with developing, investigating, and applying machine learning, data mining, and statistical methods to detect patterns in extensive collections of data from educational institutions that would otherwise be impossible to analyze using traditional computing techniques [2,10,21]. In this sense, in recent years, the use of deep learning techniques has emerged in EDM. Hence, developing data mining competencies represents a research area. For example, the work of [22] presents positive experiences to enhance knowledge acquisition about data mining via the game-based approach. Regarding the implementation of EDM systems, the work of Almaiah et al. [23] discuss about traditional issues, and the success of using modern programming languages.

Problem Statement, Goal, and Contributions

In EDM, the data of interestis not limited to individual student interactions in an educational system. We can consider different administrative and demographic data such as gender, age, and grades for discovering patterns. As [2] discusses, EDM applications exist to find educational patterns such as cognitive skills, motivational effects, and social emotions. The work of [24,25,26] presents successful results of EDM regarding administrative issues of e-learning systems and the effects on students’ performance, factors that influence the use and success of mobile learning, and critical challenges and factors to determine students’ success in pandemic time. A report still needs to be made about EDM to discover patterns in students’ success in online education in South American countries. This work asks for the following research question: Can the use of data mining tools in education allow the identification of student success patterns in e-learning programs? Trying to answer it, the primary goal of this article is to determine variables associated with student success in e-learning programs by using the CRISP-DM methodology [7] with data from a university in the developing country Chile. This study analyses the causes of success or failure from the set of student variables since we consider their demographic and performance features.

The main contributions of this paper are the following:

• First, this article identifies potential variables for success or failure in e-learning programs, not only academic factors, through a systematic literature review.
• Second, this article defines a repeatable data mining application for identifying students’ success patterns in e-learning environments using a large set of data. This analysis was not feasible with other methods.
• Third, this article provides a utilization example of multi-year historical data starting when e-learning programs began being a phenomenon in Chile and other countries (the year 2000). Other institutions in the region could repeat this application.

The remainder of this paper is organized as follows. Section 2 defines the main concepts of e-learning, CRISP-DM, the data mining process, and previous data mining in education experiences. Section 3 describes the applied methodology and case study data: we define the main steps of the data mining process, the data source, concepts, and expected results. After that, Section 4 highlights obtained results to validate our hypothesis. Section 5 describes the usefulness of this work for a similar context, overall, for online educational institutions and programs concerning what variables are relevant to consider. The paper concludes with a discussion of future work in Section 6.

2. e-Learning and Data Mining

The e-learning process is characterizable by recording most of the traditional learning process variables, which range from student entry data to the efficiency and ease-of-use of the applied platforms [1]. The large volume of data in the e-learning process provides the opportunity to analyze that data using knowledge discovery tools. The KDD (Knowledge Discovery Database) process looks for hidden patterns in large volumes of data that information systems usually store. Those patterns can be high-value information for the decision-making process in organizations [27]. Figure 1 [14,28] illustrates the KDD. Data mining is an essential KDD phase for applying algorithms to find hidden behavior patterns in the data [29].

The application of data mining techniques has two primary purposes: building models and detecting patterns [30]. The model building seeks to produce a summary of the data set to identify and describe the main characteristics. Pattern detection seeks to identify small deviations from the norm to detect unusual behavior patterns by discovering patterns and rules and searches for content. When it is not possible to build models for the data set, you can look for behavior patterns. Pattern and rule discovery seeks frequent combinations and associations of attributes found in database transactions (for example, products purchased together). Techniques based on association rules usually address that issue.

2.1. CRISP-DM

CRISP-DM method is one of the most efficient methodologies for developing projects applying data mining [31,32]. The objective of CRISP-DM is to allow different using a common vocabulary, methodology, and tools in data mining activities. CRISP-DM organizes in six phases from general to specific tasks:

• Business Understanding Phase: The first phase analysis of the problem includes understanding the project’s objectives and requirements from a business or institutional perspective.
• Data Comprehension Phase: The second phase of data analysis includes the initial data collection, identifying the quality of the data.
• Data Preparation Phase: This phase includes general data selection tasks for applying modeling techniques (variables and samples), data cleaning, generation of additional variables, integration of different data sources, and format changes.
• Modeling Phase: In this phase, selecting the most appropriate modeling techniques takes place to generate and evaluate the model. The parameters used in the model generation depend on the characteristics of the data.
• Evaluation Phase: In the evaluation phase, the model is evaluated, not from the data point of view, but for fulfilling the problem’s success criteria. If the generated model is valid based on the success established in the first phase, the model is exploited.
• Implementation Phase: At this stage, in addition to the implementation of the model, the results must be presented and documented understandably, to achieve an increase in knowledge.

Figure 2 [33] illustrates CRISP-DM stages.

2.2. Data Mining Techniques

This research applied four techniques, naive Bayes, random forest, AdaBoost, decision trees (J48), and neural networks, which are recognized as successful algorithms for classification purposes [34,35]. Some of the main characteristics of those classifiers are:

• Naive Bayes: It is based on Bayes theorem with an assumption of independence among predictors. It assumes that the presence of a particular feature in a class is unrelated to the presence of any other feature. Naïve Bayes mainly targets the text classification industry. It is mainly used for clustering and classification purposes depending on the conditional probability of happening [36].
• Random Forest: This classifier combines prediction trees in which a hierarchical division of the underlying data space is sustained. In the hierarchical division of the data space, comment partitions are created that are more skewed in terms of their distribution of terms [37].
• AdaBoost: The name is an acronym for Adaptive Boosting and it is a meta-algorithm. This algorithm supports a distribution or set of weights over the training set. Initially, all weights equally set, but on each round, the weights of incorrectly classified samples are increased then the weak learner focuses on these samples. AdaBoost originally ability to minimize the error, and maximize the margin with respect to features [38].
• Decision Trees (J48): The J48 algorithm builds a decision tree that classifies the class attribute based on the input attributes. The algorithm is based on the C4.5 algorithm developed by Quinlan [39]. The algorithm uses a greedy search method to create decision trees and allows changing different parameters to obtain a better classification accuracy [40].
• Neural Networks: The development of the neural network uses a non-linear optimization model. Unlike other analyses, it is not easy to interpret clearly, unlike the results and parameters provided by other analyses. For the construction of the classifier used in this research, the multilayer perceptron neural network was used, which builds a neural network in the form of a waterfall, which has one or more hidden layers [41].

2.3. Data Mining in Education

E-learning is the result of the adaptation and use of information technologies in education [42,43,44]. The works of Wani et al. [45,46,47] highlight the importance of e-learning in higher education using a virtual training environment for the development of professional competencies. The success of e-learning systems is associated with different variables, such as information technology and the program’ modality applied. Multiple studies exist that evaluate e-learning technology platforms’ success, most of which used the information systems success model of DeLone and McLean [4,48,49,50,51,52]. Despite the rapid growth of e-learning, institutions face problems in teaching courses in that modality. The knowledge of the variables that impact student success in e-learning is still unknown in different countries such as Chile and many in developing countries. Tools such as data mining that permit identifying behavioral data patterns could identify factors for e-learning success.

Higher distance education permits developing competencies regarding the current market demands without geographical, economic, and time barriers [53,54]. In this way, quality e-learning programs would contribute to developing countries such as Chile. For example, Cidral et al. [55] reviewed the success of e-learning platforms in Brazil. That study concluded that users’ satisfaction with program content quality and easy-to-use interfaces are critical issues for success. Several authors have studied the success or failure of students in e-learning programs, such as [56], who analyzed the causes of online university dropouts in a systematic analysis of the literature; they defined a classification of the involved variables: student, institution, teachers, media, degree of social, and academic integration. Those authors indicate that knowledge of personal, social, and demographic characteristics can be essential to predict student success and failure in e-learning programs.

By applying the data mining process of [20], Figure 3 shows our proposed model to predict student academic performance. Decision tree and random forest have known classification data mining techniques, whereas CHAID ID3 represents a version of the CHi-square automatic interaction detection algorithm, the oldest decision tree algorithm in history, and RMS (Root Mean Square) [34].

3. Methodology

This study looks to determine the success of the online learning modality provided by the Distance Education Center of the Universidad Católica del Norte (DEC-UCN) by using data mining to support the case analysis methodology to know about the initial conditions of students in educative programs.

This study worked with data on the admission and final results of DEC-UCN students between 2000 and 2018. The total number of students was of 12,264. The study stages were developed from the CRISP-DM model to analyze the database information and apply the corresponding tools. We used particular data mining techniques and algorithms, such as decision trees, descriptive statistics, and neural networks. The computational tool used was SPSS Statistics 22 [57]. The benefit of this technique is that it provides an easy understanding of data mining decision making.

3.1. Institution Background

In the form of programs leading to a professional degree, the origin of DEC-UCN dates back to 1982. The DEC-CNU was instituted in 1996. Located in Antofagasta, the capital city of Chile’s second region, the DEC-UCN is under the office of the university’s academic vice-rector. The pedagogical model of the DEC-UCN is in harmony with the PE-UCN design (PEdagogical model at the Universidad Católica del Norte). In other words, a pedagogical model that is accountable to the institutional mission supported by education in human values, based on training by competencies and taking into account the constant changes in society. This academic unit develops continuous training spaces through 100% online programs, using an education model focused on technologies to support distance education. One of DEC-UCN’s biggest problems is the lack of knowledge of student success variables who choose to study at a distance program.

3.2. Selection and Understanding of Data

This study required the data of the students enrolled in the period between January 2000 and December 2018 stored on a local server (LICANCABUR) with access restricted. This server provides services to the Oracle Developer 2000 database system called ANTEC, the official database of the DEC-UCN. Other complementary data are stored in files and printed matter managed by the managers of the different areas. We then obtained records of 18,610 DEC-UCN, data exported from the ANTEC database system, through SQL (Structured Query Language) queries for the generation of an Excel file with the requested data. Regarding the data source, DEC-UCN offered online education mainly for technical majors in the areas of management, business, and computer programming with a record of 430, 558, and 296 students, respectively. Since records are from a set of years, some students appear in more than one record.

In the ANTEC database system, we used the ANTEC browser to perform the necessary SQL queries. We selected tables STUDENT, STUDENT_ADDRESS, PROGRAM, and STUDENT_PROGRAM to obtain the personal data and the prior academic status that a student reached in a given program. The excluded data are indifferent to the sample since they did not contain reliable information for the investigation. Figure 4 presents an extract of the relational model of the ANTEC system.

3.3. Data Preparation

We applied the necessary changes to the files with the SPSS Statistics 22 tool due to its presentation capabilities that make the result more understandable to the end-user.

• Data selection and cleaning: first, we selected the data attributes, considering the objective and data quality problems. As a result, we selected the following tables and attributes: (i) STUDENT (RUT of the student, sex, date of birth, profession, nationality, marital status, education); (ii) STUDENT_ADDRESS (RUT of the student, commune, city); (iii) PROGRAM (name of the program, date of registration, type of program); (iv) STUDENT_PROGRAM (RUT of the student, academic situation, final grade of the program). For the analysis, we followed the next filter: Students who enrolled in the programs between 2000 and 2018 (enrollment date $>=$ 2000) and (enrollment date $<=$ 2018)).
• Data Quality: a problem presented by the data is the amount of missing data in some of them, such as the program’s attribute name. We decided to keep the records with unknown values because their elimination results in excluding rows with valid values in the objective set. Moreover, we faced data categorization because the applied techniques for the analysis (classification) mainly use categorical data to facilitate their construction and interpretation.
• Construction of the Data: to fulfill the project’s objective, an attribute called region was created, which derives from the attribute commune and city. The region attribute takes the value corresponding to the region that the commune and city belong to. When the data selection and construction process was complete, the changes were saved to the files for their use later in the modeling stage. The new files are in SPSS format.

3.4. Modeling

We carried out the data modeling for the DEC-UCN at a global level. In this study, the classification model predicts the student profile’ associated with success in programs with an online learning modality. Considering the research and results of [58,59,60], we applied the decision tree AdaBoostM1 and tree J.48 along with the naive Bayes and random forest algorithms for classification. The classification model takes as a dependent variable “state”, which is a categorical variable, and the category “Graduated” as the highest level of success of a student in the model. The academic success in this perspective measures students in the category graduated from a started program.

For the formulation of the model, we applied neural networks to identify relationships between the variables and determine their importance concerning the target variable. For constructing the decision trees, we initially used two algorithms: (i) the C5.0 algorithm that presents rules that allow a clearer understanding of the generative partitioning; (ii) the CHAID algorithm that, from a statistical point of view (based on the significance of the chi-square test), constructs the trees by comparing the categories, contracting those that do not present differences in their results. Subsequently, a decision tree algorithm is selected based on the results obtained (case prediction) and the analysis of the construction of the tree itself.

In order to predict the accuracy and ensure precision, the study established a confusion matrix for each algorithm, which was necessary to calculate the metrics of Precision, Recall, F1, Accuracy, and the Matthews correlation coefficient.

Table 1. Data mining metrics.

Metric	Definition
Precision	It is used to measure the positive patterns that are correctly predicted from the total predicted patterns in a positive class [62].
Recall	It permits to measure the fraction of positive patterns that are correctly classified [62].
Accuracy	It measures the ratio of correct predictions over the total number of instances evaluated [63].
F1	Metric that represents the harmonic mean between recall and precision values [41].
Matthew’s correlation    coefficient (MCC)	Measure that is not affected by the dataset problem of being unbalanced. MCC is a correlation coefficient between observed and predicted binary rankings; returns a value between $-1$ and $+1$. A coefficient of $+1$ represents a perfect prediction, 0 is no better than a random prediction, and $-1$ indicates complete disagreement between prediction and observation [64].
ROC curve	It is a graphic representation of the relationship between the true-positive and false-positive ratios of the classifier. The area under the ROC curve provides an approach to evaluate which model is better on average. A model will be considered to discriminate better than chance if the curve lies above the diagonal of no discrimination, i.e., if the AUC is higher than [65].

defines the procedure and characteristic of those measures [61].

4. Results

The statistical results are the behavior patterns that influence students’ success in online learning modality and their failures. The programs with the largest number of students are Human Resources Administration, Environmental Management, Family Medication, Psychopedagogy, Total Quality Management, Integrated Management, and Educational Orientation (see

Table 2. List of educational programs with the largest number of students.

Program	Frequency	Percentage
Human Resources Management	1996	10.7
Environmental Management	1617	8,7
Family Mediation	1506	8.1
Psychopedagogy	1499	8.0
Total Quality Management Total	1169	6.3
Integrated Management: Quality. Environment. and Safety	1009	5.4
Educational Orientation	894	4.8
Higher Education	648	3.5
Education and Professional Technical High School Teacher	545	2.9
Primary Education Teacher with a Minor in NB1 and NB2	509	2.7
Family Counseling	507	2.7
Behavioral Management Techniques Applied to Children and Adolescents	499	2.7
Education and Primary Education Teacher	467	2.5
Criminal Procedural Law: “Accusatory System or Oral Trial”	446	2.4
Communication and Language Disorder	418	2.2
Educational Administration	402	2.2
Administration of Technical-Pedagogical Units	388	2.1
Preparation and Evaluation of Investment Projects	326	1.7
Minor in Language and Communication for Teachers of the Second Cycle of Language and Communication	292	1.6
Minor in Education in Mathematics for Teachers of the Second Cycle of Basic General Education	247	1.3
Degree in Education and Primary Education Teacher	237	1.3
Management in Corporate Communication	179	1.0
Continuous Improvement	173	0.9
Higher level in Executive Secretariat	154	0.8
Mathematics Education for Primary Education Teachers	145	0.8
Pedagogical Management for Higher Level Technical Training	129	0.7
Formulation and evaluation of projects	98	0.5
Others	2180	11.6

).

Initially, we present the analysis of the data using decision trees. This analysis shows that the first level of the tree identified the variable “type of programs” as the main predictor of student success at the DEC-UCN, from left to right, from nodes 20 to 22. The type of program with the highest percentage of graduates is Continuous Improvement. With a p-value of 0.001, a chi-square of 66.4 and a degree of freedom (df) of 8, we can observe that students belonging to the Metropolitan, Magallanes, Tarapacá, and Bio Bio areas obtained the highest percentage of graduated students with 76.8%, followed by the Aysén and Los Lagos areas with 67.9%. Figure 5 illustrates the mentioned results.

The second type of program with the highest percentage of graduate students is Training and Technical Courses with 53.8%. In this program, students with non-university professions obtained a larger portion of qualifications than students with university professions from the art and health science areas. Students prefer this program due to its high degree percentage compared to professional and technical courses (see Figure 6).

Students with the highest percentage of elimination belong to undergraduate degree programs. Figure 7 depicts a total of 853 students in undergraduate programs. With a p-value of 0.002, a chi-square of 22.251, and a degree of freedom (df) of 3, we classify students with complete primary and secondary schools but without completing professional or higher-level technician studies with the highest elimination trend: 362 of 454; that is, 79.7% of students of undergraduate programs. In the same context, we also distinguish students who completed higher education studies and who present an elimination trend of 66.2%, 264 of 399 students.

The analysis of the data using neural networks gave additional information about the main variables that could predict success of students in e-learning programs.

Table 3. Importance grade of variables in the global program type using neural networks.

Sample		Predicted
		Act	Rem	Abn	Trans	Grad	Cert	Correct %
Training	Active (Act)	0	2	0	0	0	34	0.0%
	Removed (Rem)	0	6102	0	0	0	1140	84.3%
	Abnegated (Abn)	0	426	0	0	0	20	0.0%
	Transferred (Trans)	0	17	0	0	0	3	0.0%
	Graduated (Grad)	0	302	0	0	0	201	0.0%
	Certified (Cert)	0	2906	0	0	0	1720	37.2%
	Overall %	0.0%	75.8%	0.0%	0.0%	0.0%	24.2%	60.8%
Testing	Active (Act)	0	2	0	0	0	16	0.0%
	Removed (Rem)	0	2565	0	0	0	479	84.3%
	Abnegated (Abn)	0	181	0	0	0	8	0.0%
	Transferred (Trans)	0	13	0	0	0	0	0.0%
	Graduated (Grad)	0	103	0	0	0	97	0.0%
	Certified (Cert)	0	1300	0	0	0	753	36.7%
	Overall %	0.0%	75.0%	0.0%	0.0%	0.0%	24.5%	60.2%

shows that when using neural networks to identify the variables, the analysis classifies 60.8% of correct predictions. The model in Figure 8 showed that the determinant factors for academic success for all programs, from the highest to the lowest, are age, program code, profession, scholarity, type of program, region, and finally sex, according to the student’s final academic situation in the most successful programs, which gives a reasonable first approximation regarding the topic. The study analyzed comparatively the performance of the classification algorithms used, as defined in the research model. These results indicated that AdaBoostM1 and naive Bayes were the algorithms with the lowest performance.

Table 4. Classification results of applied algorithms.

	TP Rate	FP Rate	Precision	Recall	F-Measure	MCC	ROC Area	PRC Area
AdaBoostM1	0.622	0.532	0.593	0.622	0.574	0.118	0.543	0.559
Naïve Bayes	0.617	0.616	0.548	0.617	0.475	0.007	0.535	0.554
Random Forest	0.645	0.435	0.634	0.645	0.636	0.222	0.652	0.658
TREE J.48	0.643	0.494	0.625	0.643	0.607	0.184	0.604	0.609

shows that precision, recall, and F-measure indicators were comparatively low. The AdaBoost M1 algorithm achieved a correct classification of 62.15% compared to naive Bayes, with 61.7%. The MCC values are also closer to zero (0.118 and 0.007), so their prediction is not much better than chance. The ROC values are also quite close to 0.5, which is not an indication of good prediction. The tree J.48 and random forest algorithms had the best results. The random forest algorithm stands out as the one with the best result, with 64.5% of the correctly classified instances, achieving the best prediction of graduate students. In addition, this algorithm obtains the best MCC value, indicating a better relationship between the observed data and the prediction. The ROC value is also a sign of its good performance with a value of 65.2%, well above the rest of the algorithms.

5. Discussion

The advances in technology permit the massive application of data mining nowadays. As Soria-Barreto et al. [66] remark, computing tools and technologies permit a more effective e-learning success. In this research, we aimed to identify variables for student success or failures in the e-learning programs at DEC-UCN by applying the CRISP-DM methodology, which is one of the most widely used tools in this research field. We identified factors that determined student success in studying online programs through the decision tree and neural network techniques. Those results contribute to a greater understanding of the factors with the contingent issue of distance education in Chile. Our study identified the types of programs with the greatest success in terms of the student’s final academic situation and the programs with the greatest failure. The greatest failure programs are undergraduate and bachelor degrees that require more time and dedication for their completion. The number of programs without a degree continues increasing due to its short-term characteristics.

This study is highly relevant for e-learning programs because of data from a database of the oldest online program in Chile. The database contained student records from 2000 to 2018 inclusive; that is, 18,610 records in nineteen years. We highlight that our results found variables that determine the success and failure of students. Our study established that student success and failure largely depend on age, sex, previous education, job, and region. Understanding each program’s academic success factors is decisive for the students’ selection and dissemination of the programs. These results support the organization’s know-how to establish policies for disseminating and maintaining students in online learning modalities. The found variables are relevant for online education in Chile and other neighboring countries because educational institutions can consider those variables to organize their programs.

6. Conclusions, Recommendation, and Future Work

This study showed that data mining techniques are essential for discovering educational data patterns. Educational institutions can apply the described data mining techniques to analyze their data. Because we used data from a university in a developing country, institutions in countries such as Chile could use results from the presented techniques. Regarding them, we draw the following main conclusions:

• The use of educational data mining, particularly the CRISP-DM methodology, greatly contributes to systematization and efficiency in identifying patterns in the data of distance education. The study allowed us to systematize the data in various sources and formats of the distance education platform in the institution under study (DEC-UCN) and provide valuable information for future analyses in this context.
• Data mining tools can present more significant advantages than purely statistical tools since they are exploratory, allowing working with different dimensions of the same problem. It is also essential to highlight the possibility and flexibility of these analysis tools to allow us to work with categorical and numerical variables in the same analysis.
• The performance analysis of the different classification algorithms indicated that the random forest and decision tree algorithms were the ones that allowed a better prediction of results and, therefore, identified the variables that could better explain the performance of students in e-learning programs. The decision tree proved to be a beneficial tool to find relationships between variables unidentified by previously used analysis tools, mainly because a decision tree uses techniques less restrictive than statistics. Those techniques do not require, for example, conditions of data normality and are tolerant of noise in the data.

The study results in the case of the DEC-UCN will allow the organization to focus the admission efforts on the retention of students potentially more exposed and prone to dropping out. We are currently working on applying big data techniques and data mining for pattern discovery to compare their results and know the best approach.