Dynamic Multi-Criteria Decision Making of Graduate Admission Recommender System: AHP and Fuzzy AHP Approaches

Abstract

The optimal management of personal resources impacts everyone’s quality of life. An investment in graduate education is a sustainable opportunity for improved outcomes in human life, including cognition, behavior, life opportunities, salary, and career. Advanced technology dramatically reduces the risk of personal resources in graduate program admission recommendations that depend on multiple individual needs and preferences. In the digital age, a dynamic recommender system enhances the suitably effective solution for students’ university selections. This study focused on designing, developing, and testing a recommender system for graduate admission using a dynamic multi-criteria AHP and fuzzy AHP approach. The explicit multi-criteria recommender system was a platform as a service (PaaS) web application created to aid in graduate admissions management and decision-making. The design proposed that the bit representation store a dynamic explicit multi-criteria data structure. The recommendations adopting dynamic multi-criteria were validated by comparing them to the programs to which the students were actually admitted and enrolled. They individually ranked the evaluation outcomes of dynamic explicit multi-criteria and alternative preferences to provide graduate admission recommendations. Eighty graduate students in information technology evaluated the recommender system. Using top-1, top-2, and F1-score accuracy, the effective system accuracy performance on the dynamic multi-criteria recommender system was evaluated using AHP and fuzzy AHP approaches. The fuzzy AHP demonstrated marginally greater practical accuracy than the AHP method.

1. Introduction

The universal demand for better education has resulted in a rapid increase in enrollment and higher tuition fees in many countries around the world [1]. Access to education solely is insufficient [2]. Students who do not graduate run the risk of losing opportunities, resources, time, and motivation. In the meantime, higher education can contribute to improved careers, attitudes, life opportunities, and salary outcomes. Consequently, long-term success in higher education becomes essential [3]. The achievement of sustainable success in higher education may involve two possible stakeholders: university staff and administrators who provide educational needs and services. Students must choose a suitable university for their studies. As the number of educational institutions and programs increases, the situation may become increasingly challenging for students to meet their informational needs by conducting online searches about educational products and services [4]. Without sufficient students graduating, educational institutions are under increasing pressure to produce graduates who are prepared to enter a labor market that is undergoing change. Developing a technology-enhanced relationship between students’ interests and universities’ preferences to define and pursue learning goals using a recommender system [5,6] is one of the critical challenges of achieving sustainable success in the recommendation of higher education. This allows each student to attend the university that best suits his or her individual interests and preferences.

Users are increasingly exposed to the entirety of the digital universe [7,8] through the prism of their information, knowledge, behaviors, activities, preferences, and interests using recommendation systems [9]. Due to the increase in online information and database systems, a recommender system employing a powerful information filtering technology that allows users to make their own decisions to increase user satisfaction [10] has attracted considerable interest. A recommender system contributes to the development of an original application and offers users subsequent actions based on a variety of factors and choices, such as user interests or preferences and user history [11,12]. The recommender system has become a valuable tool for resolving the problem of information overload, enabling users to make difficult decisions and meeting their needs in a variety of solutions, including digital commerce [13], finance [14], entertainment (film [15], music [16], and television shows [17]), travel [18], healthcare [19], energy [20], and education [5,21].

Recommender systems have been studied in various research areas, such as algorithms, processing techniques, and software developments [22,23]. The contents and information for recommendations and the recommendation algorithm must be tightly associated when creating and constructing a recommender system [24]. Several approaches to decision-making support employing intelligent information-processing technology have been proposed, such as collaborative filtering [22,25], content-based filtering [26,27], and clustering [28,29]. However, the conventional recommendation approaches also encounter several limitations because the traditional recommender system uses a single-criterion rating value. In the last two decades, the integration of algorithms known as hybrid systems have been combined to improve decision scenarios. The new approaches require a criterion to use more than a rating on the item to seek the appropriate recommendation items. Since the item selection for an individual user might depend entirely on more than one criterion, multiple factors in the decision-making model, which comprises innovative integration and analysis, can result in more accuracy in the recommendation [23].

In recent years, recommender systems have increasingly utilized the Analytic Hierarchy Process (AHP) and Fuzzy Analytic Hierarchy Process (fuzzy AHP) techniques. These methods permit ranking criteria and sub-criteria, resulting in more accurate recommendations. AHP is one of the algorithms used to prioritize public and private organizations’ diverse decision-making and planning activities [30]. AHP is a decision-making support technique that employs multi-criteria decision problems [31]. Multi-criteria decision-making (MCDM) can effectively increase user satisfaction by incorporating multi-criteria inventory classification [32] for decision-makers. The MCDM method is designed to support uncertain decision-making processes and includes numerous decision-making variables or criteria [33]. MCDM is utilized to solve problems in various domains, including performance evaluation, resource management, corporate and strategic policies, public policy, political planning, and strategy. The limitations of AHP capability mentioned earlier and several other features can impede AHP [34] from effectively handling large-scale solutions, complex problems, consideration of risk and uncertainty, conflicting criteria, lack of transparency and interpretability, and dynamic environments. These limitations can lead to computational difficulties and increase the time and effort required to perform the pairwise comparisons and derive the final priority rankings. The complexity of the decision model grows exponentially, including a number of criteria, conflicting criteria, alternatives, and stakeholders. AHP can struggle to handle large-scale and complex decision problems and does not explicitly incorporate measures of risk, uncertainty, trade-offs, and conflicting criteria in the decision-making process. In addition, AHP involves a series of calculations, matrix transformations, and pairwise comparisons, making the decision-making process complex, less transparent, and making it difficult to understand and interpret the rationale behind the priority rankings. Moreover, AHP assumes a static decision environment where the criteria and weights remain fixed throughout the decision-making process. In application scenarios, decision problems can be dynamic and subject to changes over time. AHP may struggle to adapt to such dynamic environments and require frequent updates and recalibrations. While AHP can handle quantitative data and pairwise comparisons to derive the priority rankings, the qualitative data using a conversion of numerical scales may lead to subjectivity and loss of information. A fuzzy hybrid approach is one of the most well-known effective MCDM techniques. The fuzzy logic encompasses a number of preference settings that are more uncertain, imprecise, and obscure than AHP and other MCDM strategies. Fuzzy pairwise comparison enables the inspection of inconsistencies to prevent bias in decision-making for various alternatives based on different criteria. Combining fuzzy strategies with AHP [18] is one of the techniques for advancing complex problems using AHP. Fuzzy AHP, also known as one of the MCDM tools [35], was developed to address the hierarchical and selection problem.

A considerable number of recommendation systems have been proposed in education, as well as in teaching and learning services and information management. A study [26] examines the operation of AHP and fuzzy AHP in various domains of decision support systems. However, it has not been determined whether the fuzzy AHP approach is superior in terms of accuracy and the ability to handle uncertain and ambiguous criteria and alternatives. Another study [36] proposed a hybrid method that combined AHP and fuzzy AHP for graduate course recommendation and found that the hybrid approach improved the recommendations’ accuracy compared to using either method alone. In conclusion, prior research has demonstrated that AHP and fuzzy AHP are multi-criteria decision-making methods that can be applied effectively to various higher education objectives [37,38]. Fuzzy AHP methods offer a methodical and objective approach to evaluating and comparing alternative options based on multiple criteria [39]. Several studies have found that fuzzy AHP is more adept than AHP at handling uncertain and ambiguous criteria and alternatives [40]. In addition, several studies have demonstrated that combining AHP and fuzzy AHP can enhance the precision of recommendations [41]. Another study [42] utilized fuzzy AHP to develop a system for the university selection process based on multiple criteria, including networking and knowledge-exchanging capability, general attractiveness, research capability, and commercialization capability. A content-based recommender system was used in a study [27] to develop a ranking system for graduate programs based on program characteristics and university reputation. AHP and fuzzy AHP have also been underutilized in admissions recommendation systems. AHP and fuzzy AHP techniques in a recommender system have not yet been investigated for graduate program recommendations. The comparison of the graduate admission recommender system with the evaluations of the AHP and fuzzy AHP algorithms has not been exhaustively studied, especially using a dynamic multi-criterion for decision-making in actual applications.

Moreover, digital data is readily accessible and can be utilized by recommender systems to collect various types of input data, which can indirectly or directly improve the system’s accuracy. Explicit multiple criteria on recommender systems refer to a methodology that employs multiple criteria to evaluate the suitability of various options to provide personalized recommendations to users [43]. Multiple explicit criteria on the methodology of the recommender system provide a robust and sophisticated approach to delivering personalized user recommendations, incorporating multiple decision-relevant criteria. The explicit consideration of multiple criteria in decision-making is essential because the recommender system enables decision-makers to make more informed and rational decisions by evaluating the trade-offs between different criteria [44,45]. Multiple explicit criteria in the recommender system involve designing and developing a set of explicit criteria relevant to users’ decision-making processes, such as academic performance, work experience, research interests, and desired outcomes. AHP and fuzzy AHP can evaluate the significance of multiple criteria and calculate the weights of each criterion using a sophisticated technique and explicit methodology. These techniques assist in capturing the inherent imprecision and uncertainty of human judgments in decision-making processes and provide a more precise and reliable evaluation of the alternatives. Moreover, a dynamic multi-criteria selection is a method that enables users to tailor their multi-criteria preferences and requirements. To the best of our knowledge, no dynamic multi-criteria recommender system utilizing AHP and fuzzy AHP is currently available. In this study, explicit multiple-criteria processing serves as a framework for decisions based on dynamic multiple criteria or attributes. Typically expressed as weights or significance values, the framework for decision-makers must explicitly specify student preferences for different attributes or criteria of the graduate admissions decision.

Numerous studies have been conducted on explicit user preferences in support of a multi-criteria recommender system. Previous studies [46,47,48] proposed a hybrid method for a recommendation based on enhanced fuzzy multi-criteria collaborative filtering. Integration was restricted to an explicit item-based ontological semantic filtering strategy. One study [49] developed an AHP method for multi-criteria decision-making that incorporates TOPSIS (order preference by similarity to the ideal solution) techniques. AHP computes the global weights of each criterion and sub-criterion, which are inputs for the TOPSIS method. The explicit weights of the selected criteria were considered, but there is no implicit formulation of uncertainty. While explicit queries of internet users and search engines filter out various pages and types of data, recommender systems may not achieve the users’ preferences [50], particularly when explicit user preferences are applied.

Therefore, this study proposes a novel approach for solving this research gap by employing bit representations on explicit criteria of graduate admission recommender systems utilizing AHP and fuzzy AHP. Despite the fact that AHP has many limitations, the selection of AHP as a decision-making method in this study will allow us to explore numerous gaps and potential solutions. Obviously, students’ preferences are factored into the examination of graduate admission criteria study procedures. The explicit criteria are broken down into a hierarchy of criteria, sub-criteria, and alternatives to facilitate a systematic decision-making process. The hierarchical structure, pairwise comparisons, and aggregation of preferences contribute to elucidating the decision-making rationale and facilitating discussions between decision-makers. Even though the original MCDM method, such as AHP, can be studied and evaluated, the preference-based prioritization of multi-criteria and alternative supports enables a more comprehensive evaluation of the decision alternatives. Bit representations, a form of encoding that transforms the original data into a binary representation, have been used to improve the efficiency of these methods. In the context of recommender systems, this study also introduces the use of bit representations to express a data structure for explicit criteria, which is an original method for dealing with binary data. Using AHP and fuzzy AHP, this study presents the design and development of an original graduate admission recommender system. For the AHP and fuzzy AHP systems, a comprehensive dynamic criteria list of recommended universities for students interested in applying to the information technology department is compiled by user needs. This set of recommendations is used for comparing the recommendations and actual admission outcomes of students’ study decisions. Our proposed system intends to develop a bit-representation–based recommender system for graduate admissions utilizing AHP and fuzzy AHP algorithms to evaluate explicit admission criteria, provide personalized program recommendations, and overcome the limitations of existing graduate admission recommender systems in terms of the accuracy, efficiency, and suitability of the admission recommendations for individual needs.

Based on software engineering principles for decision-support systems, this investigation measured the dynamic multi-criteria recommender system of graduate admission in the system design and evaluation of actual system testing. The study served as a guideline or framework for examining the success of software architecture design, system development, and the accuracy of recommendation outcomes that affect an individual’s decision-making regarding innovations and information systems. To comprehend the objectives, the following research queries are defined.

Research question 1: What can the software architecture and design do to support the AHP and fuzzy AHP recommender system, which promises up to eight dynamic multi-criteria with explicit user preference definitions?

Research question 2: What are the available criterion requirements for a multi-criteria graduate admission recommender system, and the top five most desirable criteria?

Research question 3: What are the differences in accuracy between AHP and fuzzy AHP recommender systems for graduate admission to master’s degree and doctoral programs based on explicit multi-criteria?

The research contribution improves the accuracy of the recommendation by incorporating multiple explicit criteria to better match students with universities that meet their preferences and needs. The application of AHP and fuzzy AHP enables a more objective and systematic evaluation of admission criteria, whereas the use of bit representations enables the incorporation of binary data into the decision-making process. By providing accurate and personalized recommendations for graduate admissions based on AHP and fuzzy AHP, the graduate admission recommender system allows them to contribute to multiple sustainable educational admission systems. First, the proposed recommender system can optimize the allocation of educational resources to ensure that students are matched with programs that align with their personal information, such as their interests, preferences, capabilities, skills, and career objectives. By minimizing human capital loss, students can achieve success and have a positive impact despite inefficiencies and resource waste. Studying demotivation and dropout rates can be decreased. Second, the system dynamically considers a variety of student multi-criteria and preferences, including factors such as university popularity, academic staff, and sustainability initiatives. Thirdly, the system promotes sustainable practices in the curriculum and campus operations because students are guided to programs that prioritize sustainability and provide quality education. Finally, the system can contribute to long-term career satisfaction and success by recommending admission programs that align with students’ career objectives. When students pursue careers that align with their interests and aspirations, they are more likely to make significant contributions and positively affect societal, economic, and personality challenges.

The remainder of this article is organized as follows: In Section 2, the data and methodology supporting the design, development, and evaluation of accuracy with explicated criteria integration are presented. Section 3 outlines the methodology for designing, developing, and evaluating AHP and fuzzy AHP recommender systems. The actual implication of the AHP and fuzzy AHP recommender system is summarized and discussed in Section 4. Section 5 summarizes the evaluation results of the actual user tests for the graduate admission recommendations for Thailand’s master’s degree programs.

2. Materials and Method

This section describes the research methodology used to demonstrate the design, development, and evaluation of the AHP and fuzzy AHP recommender system for graduate admission based on explicit multi-criteria decision-making, as depicted in Figure 1.

A practical comparison of AHP and fuzzy AHP accuracy results is proposed to validate the performance analysis for a sustainable method of software evaluation: reviewing current graduate admission problems and challenges, reviewing decision algorithms and techniques, designing explicit criteria of graduate admission preferences, proposing the bit representations of explicit criteria, developing the graduate admission recommender system, admitting AHP and fuzzy AHP for ranking criteria, evaluating the actual use of the system based on accuracy scales, and comparing the AHP and fuzzy AHP techniques, and summarizing the results.

2.1. Analytical Hierarchy Process and Fuzzy Analytical Hierarchy Process Studies

AHP is a method for making multi-criteria decisions that assist decision-makers in choosing between alternatives. The analysis processes of AHP consist of a definition of criteria, a matrix comparison of criteria, a weight calculation of criteria, and a summarization of the optimal weight. The method involves breaking down a complex problem into a hierarchy of criteria and alternatives and then evaluating the relative importance of each criterion and alternative [51]. AHP uses pairwise comparison matrices to evaluate the relative importance of the criteria and alternatives and then calculates priority vectors representing each element’s relative importance in the hierarchy. This method allows for comparing multiple options based on a combination of criteria and can provide recommendations based on the most important criteria [52,53].

The AHP, first proposed by [54,55], can evaluate effects qualitatively and quantitatively. Reflecting the influence of each factor, the AHP model consists of an object layer, a ruler layer, and a factor layer. Establishing an analytic hierarchy model based on relative factors, creating an assessment matrix using Equation (1), calculating the weight coefficients of each factor using Equation (2), validating the consistency ratio (CR < 0.1) of the judgment matrix using Equation (3), and determining the comprehensive weight coefficient for each factor are the primary steps of AHP.

$$A_u={\left(a_{ij}\right)}_{nxn}=\left(\begin{array}{ccc}1& \cdots & a_{1n}\\ ⋮& 1& ⋮\\ a_{1n}& \cdots & 1\end{array}\right)$$

(1)

where $a_{ij}$ is the relative value of the i factor to the j factor, which ranges from 1 to 9, and their reciprocals. On the basis of the evaluation matrix, the weight of each factor can be determined. The scale is divided into a series of intervals that correspond to different levels of importance. The intervals are defined as follows: 1 → Equal importance, 2 → A little more importance, 3 → Moderately more importance, 4 → Considerably more importance, 5 → Strong importance, 6 → Very strong importance, 7 → Extremely strong importance, 8 → Near absolute importance, 9 → Absolute importance.

$$w_i=\frac{M_i}{\sum _{i=1}^n{M}_i}$$

(2)

where $w_i$ is the weight coefficient of each factor; $M_i=\sqrt[n]{\coprod _{j=1}^n{a}_{ij}}$.

$$CR=\frac{CI}{RI}=\frac{\sum _{j=1}^n{a}_i{CI}_i}{\sum _{j=1}^n{a}_i{CR}_i}$$

(3)

where $CI=\frac{{\lambda }_{max}-n}{n-1};$ ${\lambda }_{max}$ is the largest eigenvalue of the judgment matrix; $n$ is the dimension of the judgment matrix. According to [53,54], the judgment matrix is reasonable when the value of the consistent ratio (CR) is less than 0.1; whenever the value of CR is greater than 0.1, the judgment matrix is unreasonable and must be recalculated.

Fuzzy logic is a method for processing uncertain and imprecise data and information. Decision-makers who must make a choice under uncertain conditions can utilize fuzzy AHP. Based on [36], the primary processes of an AHP model can be expanded to support a fuzzy AHP model. The primary algorithm of a fuzzy AHP model focuses on a Fuzzy pairwise comparison matrix. By synthesizing the criterion weight, defuzzification can be computed in the output of the fuzzy weight sets to facilitate accurate decision-making. The combination of fuzzy mathematics and the analytic hierarchy process aims to address issues of uncertainty and imprecision via the fuzzy analytic hierarchy process [56]. Fuzzy sets and AHP have been utilized to address a portion of AHP’s human opinion deficiencies.

Fuzzy AHP theory provides decisions through approximate information and uncertainty in human reasoning. In the decision-making process, ambiguity is always present. Good decision-making models can compensate for a factor’s vagueness. Equation (4) shows that an objected feature matrix characterizes a fuzzy set model. To consider the relative membership degree of judgment factors, the objected feature matrix must be transformed into a relative membership degree matrix, as illustrated by Equation (5).

$$B_p={\left(b_{ij}\right)}_{nxn}={\left(\begin{array}{cccc}{b}_{11}& b_{12}& \dots & b_{1n}\\ b_{21}& b_{22}& \dots & b_{2n}\\ ⋮& ⋮& \ddots & ⋮\\ b_{m1}& b_{m2}& \dots & b_{mn}\end{array}\right)}_p$$

(4)

where $B_p$ represents the objected feature matrix and $b_{ij}$ represents the score of $j$ plan to $i$ factor as determined by an expert’s score.

$$R_p={\left(r_{ij}\right)}_{nxn}={\left(\begin{array}{cccc}{r}_{11}& r_{12}& \dots & r_{1n}\\ r_{21}& r_{22}& \dots & r_{2n}\\ ⋮& ⋮& \ddots & ⋮\\ r_{m1}& r_{m2}& \dots & r_{mn}\end{array}\right)}_p$$

(5)

where $R_p$ represents the relative membership degree matrix, and $r_{ij}$ represents the relative membership degree of $j$ for the $i$ factor.

The value comparison matrices used for weight calculation in fuzzy AHP approaches are also used in ranking the available alternatives along with the scores obtained from the alternatives in each criterion at a later stage. The fuzzy AHP method, on the other hand, uses a scale that ranges from 0 to 1 to represent the degree of membership of an alternative or criterion to a particular level. This scale is divided into a series of intervals corresponding to different membership degrees. Thus, the primary process is derived from comparison matrices of the weights [32,57]. The nine scores represent the relative importance of the factor contributing to selecting a graduate program based on the importance criteria (from 1 = equal importance to 9 = extremely strong importance), as shown in

Table 1. Definitions of quantitative scales adapted from [58].

Intensity of Importance	Definitions
1	Equal Importance
3	Moderate Importance
5	Strong Importance
7	Very Strong Importance
9	Extremely Strong Importance
2, 4, 6, 8	Intermediate Value between Previous Levels

.

Hierarchy Process (AHP) and fuzzy AHP decision-making techniques depend strongly on quantitative scales. These scales facilitate the quantification of the relative significance or preference of various alternatives or criteria. In AHP and fuzzy AHP, scales assign numeric values to various levels of importance or preference.

A particular linguistic term has defined scales. Levels of linguistic terms also correspond to the scales described in relation to set series. These scales connect numerical and verbal expressions. Five-level, 7-level, and 9-level scales for scale selections are also being implemented [58,59]. Most of the research utilizes the 9-level scales shown in Figure 2, adapted from Zadeh [60]. The use of quantitative scales provides a structured and consistent framework for evaluating alternatives or criteria based on their relative importance or preference in both approaches. The numerical values assigned to various levels of significance or membership are used to calculate weighted scores for each alternative or criterion, which are then applied to make informed decisions.

AHP and fuzzy AHP are multi-criteria decision-making techniques that can be incorporated into a recommender system for choosing a postgraduate program. As shown in Figure 3, the hierarchical structure of AHP and fuzzy AHP is designed to evaluate alternatives and criteria systematically and objectively. The three levels of the AHP’s hierarchical structure are the objective, the criteria, and the alternatives. The objective is the overarching purpose of the decision-making procedure. The criteria are significant factors in achieving the objective, while the alternatives are the available options for achieving the objective. The criteria and alternatives are organized under the objective. Each hierarchy level is evaluated with a pairwise comparison matrix. The pairwise comparison matrix compares the relative significance of each element at the same level. The matrix is a square matrix with diagonal elements set to 1 because each element is equally significant. The off-diagonal elements represent the importance of one element in comparison to another. A scale of values is used to determine the relative importance of elements. The eigenvector method determines the relative weights of the criteria and alternatives following the completion of the pairwise comparison matrix.

Fuzzy AHP is an extension of AHP that can handle ambiguous and uncertain criteria and alternatives. In fuzzy AHP, the hierarchical structure is comparable to that regarding AHP, except for replacing the matrix of pairwise comparisons with a fuzzy pairwise comparison matrix. The fuzzy pairwise comparison matrix enables decision-makers to express their views more flexibly than in AHP. The values in the matrix are expressed using linguistic terms such as “very important”, “moderately important”, and “slightly important”. The fuzzy set theory represents values as fuzzy numbers to account for uncertainty in the pairwise comparison matrix. The fuzzy numbers are then utilized to determine the relative weights of the criteria and options. To aggregate criteria and alternatives, the weighted sum method is applied.

In conclusion, the hierarchical structure of AHP and fuzzy AHP provides a methodical and objective evaluation of alternatives and criteria. Fuzzy AHP employs a fuzzy pairwise comparison matrix to address uncertainty and ambiguity, whereas AHP employs a pairwise comparison matrix. Both approaches may be utilized in a recommender system for selecting a postgraduate program.

2.2. Recommender System in Education

A recommender system is an information system of user decision-making that can recommend information and services based on the preferences of each individual user [13,61]. The principles of recommender systems address issues of information overload and complex decision-making by using information filtering to support data prediction and to recommend user alternatives in numerous interest areas [62]: for instance, agriculture [36,63], logistics [64], product [13,65], and society [66,67].

The recommender systems in education have proposed particular approaches and methods with many objectives. A study [36] proposed a recommender system for course selection based on two statements: course taken (1) and course not taken (0). By evaluating the system’s accuracy, Top-N algorithms have been developed, but not enough to summarize each course’s weighting decision. Dahdouh et al. [68] applied association rules of data mining techniques for course recommendation, which did not prioritize the selection and preferences of criteria based on the interests of users. Numerous researchers have investigated e-learning recommender systems to support hybrid recommendation strategies. Combining weights, feature combinations, and filtering could improve the performance of the methods [69,70]. Camacho and Alves-Souza [71] compiled papers utilizing social network data to mitigate the cold-start issue resulting from insufficient items or learners to initiate a recommender system.

Moreover, some studies have focused on using AHP in a scholarship-recipient recommendation system to provide a systematic and objective approach for evaluating and comparing alternative options based on multiple criteria [72]. A study [73] proposed an AHP-based recommender system for the best graduate student that performed best on competence graduate studying. The study also found that the AHP-based approach provided accurate recommendations for applicants, and it was able to handle the multi-criteria problem effectively. Another study [74] proposed an AHP-based decision support system for graduate study programs focusing on the supervision of their parents and student interests, not concerning the accuracy of the recommender system. A study showed the undergraduate admission recommender system using the Top-N approach focusing on general university admission requirements, such as GRE, cumulative GPA, TOEFL scores, GRE scores, and more [75]. The study found that the fuzzy AHP-based approach provided accurate recommendations for the best graduation [76] and university major selection [77]. Fuzzy AHP has been shown handling the uncertain and ambiguous criteria and alternatives.

2.3. Proposed Design of Criteria for Graduate Admission Recommender System

2.3.1. Graduate Admission Criteria Design

In the field of higher education, AHP and fuzzy AHP have been applied to the evaluation and ranking of academic programs and universities. Numerous approaches have been proposed to account for the criterion preferences and recommendation utilization of individual requirements. To fill in data gaps and resolve inconsistencies and erroneous results of evaluation standards, criteria were explicitly centered on the objective system’s primary concern. The critiques of the recommender system for graduate admission were analyzed based on the conduct of the students. The research survey opinions were used to generate a set of behavioral patterns that determined university admission information. According to the relevant documents for implementing recommender systems in education [78], the essential elements of user recommendations were learning resources, courses, and educational programs. Elahi et al. [79] created a university recommender system based on the online international student survey of Quacquarelli Symonds [80] using user preferences as rating scales. A Singular Value Decomposition (SVD) algorithm performed well in accuracy and perceived personalization, whereas a K-Nearest Neighbor (KNN) algorithm performed better in novelty. The predictive models were developed to predict the item’s rating and recommendation for the chosen university. According to literature reviews, seventeen criteria were established, as shown in

Table 2. Proposed criteria of graduate admission recommender system.

Criteria Code	Criteria	Definition	References
C1	State/Private University	Types of higher education institutions will be based on considering the original affiliation, establishment objectives, and the target students’ group. It is divided into state universities and private universities.	[21]
C2	Popularity/University Reputation	The wide popularity and reputation of the university will be based on consideration of the university ranking from various criteria on both national and international levels. This includes statistics on the number of students that reflect students’ interest in further study.	[21,80,81,82,83,84,85]
C3	Location of University	The university location will be related to distance, transport routes, and the proximity to amenities such as trains and department stores, among others.	[21,81,84,85,86]
C4	Entry Requirements/Eligibility to Apply	This includes eligible candidates who will apply or who fit the eligibility criteria in each study plan as required by the curriculum. Thus, if one of these is missing, applicants will not be able to apply for further study; for example, completing the required degree, cumulative average grade, and other additional qualifications according to the course requirements.	[81,86]
C5	Length of Study/Course Duration	The total number of credits and the study period specified by the course as specified based on the academic year or semester in case the student studies beyond the course duration. Tuition fees may be charged, or they must maintain their educational status until graduation.	[81]
C6	Class Time/Flexibility of Timetable	Class schedule and flexibility in scheduling to accommodate the needs of students, such as weekdays, Saturday–Sunday, or schedules that can be agreed with the instructor later based on the majority of students’ demands. This includes studying thesis subjects where students can report directly to their advisors.	[84]
C7	Education Language	Languages used for the class include Thai and English, which may be related to the curriculum; for example, English is the primary language for international programs.	[21]
C8	Condition of Publications/Programs Offered/Academic Quality/Preparation for Graduation	Conditions for publishing academic works include the number of works and the level of publications both nationally and internationally, such as peer-reviewed journals or academic publications. It also includes the publication at an academic conference with an article evaluation system with external committees participating in screening, including a full research article as a continuation of the academic conferences (proceeding). This includes course support during study or research and the quality of academic work, following the curriculum’s requirements, and various preparations for graduation.	[80,81,86]
C9	English Scores	Criteria for submitting English language test scores from English language testing institutes according to announcements specified by the university, such as TOEFL, IELT, CEFR, or other conditions for enrollment in English courses. This is for graduate studies to measure English proficiency if the test scores cannot pass the specified criteria. The course may specify the period for submitting the score results before the study, the start of the thesis topic exam, or before graduation.	[81,87]
C10	Academic Staff	The expertise of academic personnel, including instructors, researchers, professors, and program personnel, reflects the readiness to advise on research to graduate according to the study plan. Moreover, this also includes the availability of support for students to apply for national or international research grants.	[21,81]
C11	Technological Facilities	Technology learning support includes research tools and equipment, a computer service room open 24 h, multimedia services, a journal database system for overseas, and practical training courses on software packages and support for IT Certificate exams and professional qualification certificates.	[21,81,85]
C12	Connection with Foreign Universities	Cooperation networks with overseas companies and educational institutions, teacher and student exchanges, joint research, and co-curricular arrangements based on a memorandum of understanding, as well as opportunities to participate in study-abroad programs or internships with partner networks.	[21]
C13	Cost of Program/Tuition Fee/Cost of Tuition	Tuition fee or estimated total tuition fees throughout the study until graduation, including payment terms and tuition fee installments which cover a group of ordinary and provisional students (if any).	[81,82,83,84]
C14	Scholarship Opportunities/ Financial Assistance	Opportunities for scholarships, grants, or financial support to students in various fields during studies, such as full scholarships, research assistant scholarships, teaching assistant scholarships, and scholarships to support research both in the country and abroad. This also includes scholarships for presenting works nationally and internationally, scholarships for journal publication, monthly living allowance, or scholarships from various agencies.	[21,81,86]
C15	Promotion or Discount	Promotional campaigns or discounts for those interested in further studies, such as credit reductions, alumni discounts, discounts when applying within the specified time (early bird), or other special discounts.	[81]
C16	Application Fee	Application fees include admission fees, entrance examination fees, and English proficiency test fees.	[80,84]
C17	Admission Processes	Admission process for those interested in further study and the application period for both early and late semesters, as well as recruitment channels and contact channels to see the order of consideration, such as written examination, interview for readiness and graduation opportunities, and testing knowledge related to the program. Some programs may have a procedure for submitting a letter of acceptance as a thesis advisor from a faculty member to be eligible to study.	[81,84]

.

Therefore, the recommender system in this study predetermined the criteria to sup-port the development of graduate admissions decisions. Users can dynamically select the multi-criteria decision-making (MCDM) of graduate admission recommendations using up to eight criteria and up to four alternative decisions. A dynamic multi-criteria selection is a method that enables users to customize their multi-criteria preferences and requirements for graduate admission circumstances. Using AHP and fuzzy AHP modules, the recommender system generates, displays, and processes the associated explicit multi-criteria on a dynamically generated selection screen.

2.3.2. Bit Representation of Explicit Criteria Decision-Making

Bit representation is a data structure used to represent information as a sequence of bits or binary digits. Each bit has two possible values, either 0 or 1, and can be used to represent Boolean values such as TRUE or FALSE or to represent binary numbers.

Table 3. Samples of criteria selections using bit representations.

Samples	Criteria								Samples	Criteria
	C1	C2	C3	C4	C5	C6	C7	C8		C1	C2	C3	C4	C5	C6	C7	C8
1	1	0	0	1	1	1	0	0	11	0	0	0	1	0	1	0	0
2	0	0	0	0	0	1	0	1	12	1	0	0	0	0	1	0	0
3	0	1	0	0	0	0	0	0	13	0	0	0	0	0	1	0	0
4	0	1	1	0	0	0	0	0	14	0	0	0	1	0	0	0	0
5	0	0	0	0	0	0	0	0	15	1	0	1	0	0	1	0	0
6	0	1	0	0	0	0	0	0	16	1	1	1	0	0	0	0	0
7	1	1	0	0	0	0	0	0	17	0	1	0	1	0	0	0	0
8	0	1	0	0	0	1	0	0	18	0	0	0	0	0	0	0	0
9	0	0	0	0	0	0	0	1	19	1	0	0	0	0	0	0	0
10	1	0	0	0	0	0	0	0	20	0	0	1	0	1	1	0	0

illustrates an example of a bit-representation data structure with 20 user-collected samples, each representing an 8-bit criteria series. The criteria bit is known as the consistency bit for criteria (CCB). The values in Table 3 are either 0 or 1, representing the absence or presence of a particular criterion. The first sample, for example, contains eight bits of 10011100. The first bit in each row represents criterion C1. The first column contains the value 1 for C1, indicating that it is TRUE or that a user selected the criteria, while the second column contains the value 0, indicating that C2 is also FALSE. The value of 0 in the third column indicates that C3 is also FALSE, whereas 1 in the fourth column indicates that C4 is TRUE. The fifth and sixth columns contain the value 1, indicating that C5 and C6 are TRUE. The seventh and eighth columns contain the value 0, indicating that C7 and C8 are also FALSE.

By integrating binary data in selection criteria, bit representation data structures are commonly used to store and manipulate large amounts of data efficiently, particularly in NoSQL (Not Only SQL (Structured Query Language)) databases. For instance, bit representation can be used to represent features or attributes of a dataset, which can then be converted from a 2-bit binary number to an n-bit binary number. The authors assume that each collection string in a binary representation contains 32 bits.

As shown in Figure 4, an example of the bit representation supporting up to eight criteria, the bit string consists of a criteria preference bit (CPB) for 24 bits and a criteria consistency bit (CCB) for eight bits. The CPB is comprised of eight criterion indexes. Each block must have three binary digits. Depending on the three binary codes, a block of criteria index can be converted into the number of criteria definitions and criteria preferences. For instance, the first block’s three binary strings can convert from binary code to decimal code from 010 to 2. Therefore, the conversion of the first criteria index reveals that the criteria definition is C1 and that C1’s criteria preferences are 2. The CCB section is used for error correction and verification of the user-selected explicit criteria to ensure data consistency of the explicit bit CPB section. For instance, the definition of criterion C1 is explicitly defined as the second most preferred criterion. The CCB has confirmed that the first bit has the value 1.

By combining the digital number of 32 criteria bits from the example in Figure 4, the binary number 01000000000101110000000010011100 represents the integer value 107,524,930,810₁₀. Conversion from binary to base-10 is:

01000000000101110000000010011100₂ = (0 × 2³¹) + (1 × 2³⁰) + (0 × 2²⁹) + (0 × 2²⁸) + (0 × 2²⁷) + (0 × 2²⁶) + (0 × 2²⁵) + (0 × 2²⁴) + (0 × 2²³) + (0 × 2²²) + (0 × 2²¹) + (1 × 2²⁰) + (0 × 2¹⁹) + (1 × 2¹⁸) + (1 × 2¹⁷) + (1 × 2¹⁶) + (0 × 2¹⁵) + (0 × 2¹⁴) + (0 × 2¹³) + (0 × 2¹²) + (0 × 2¹¹) + (0 × 2¹⁰) + (0 × 2⁹) + (0 × 2⁸) + (1 × 2⁷) + (0 × 2⁶) + (0 × 2⁵) + (1 × 2⁴) + (1 × 2³) + (1 × 2²) + (0 × 2¹) + (0 × 2⁰) = 1075249308₁₀.

The database system can then record the integer value instead of the entire binary number or the user-selected array values. Additionally, bit representation can be used in cryptography and security, where data encryption algorithms employ binary digits to represent encrypted data. In networking protocols, where data is transmitted as a series of bits over a network, a bit representation can also be used. The use of bit representation has the benefit of saving space and reducing storage needs while facilitating access. In addition, the bit representation automatically encrypts user data. To decrypt data, it is necessary to understand the meaning of bit representation. A disadvantage of bit representation is the difficulty for humans to interpret, process, and manipulate large amounts of data. In addition, since each bit has only two possible values, it may not be suitable for representing continuous or analog data, such as temperature or sound, because each bit can only take on one of two values. The actual data of this study was stored on MySQL database system using BIT datatype of 32-bit values, which is equivalent to one field of unsigned integer datatype.

2.3.3. Explicit Multi-Criteria Recommender System Architecture

The explicit multi-criteria recommender system is a responsive web application designed to aid graduate admissions management and decisions through a Platform as a Service (PaaS) [88]. As depicted in Figure 5, the system architecture of the graduate admission recommender system utilizing Vue.js, Node.js Express, and MongoDB with AHP and fuzzy AHP algorithms can be divided into two main compartments (front-end and back-end components). The front-end application is built on the Vue.js platform. The web application is responsible for the user interfaces of the system, which allow users to interact with the system via the design of the web pages and the graphical user interface. The front-end and back-end systems communicate using the RESTful APIs provided by the Node.js Express server. Node.js Express and MongoDB are used to develop the back-end system. The recommender system stores and retrieves information such as university names, graduate programs, criteria preferences, and criteria using MongoDB as its primary database. The user’s criteria data is encrypted and decrypted using a bit-representation approach. The binary numbers are identified to support user’s criteria selection. The bit conversion processing can transform into integer values. The web application handles the system’s business logic, including data storage, data processing, and algorithm implementation. The front-end system consists of a user interface module. The back-end system consists of three systems, including Vue.js, Node.js, and MongoDB. The vue.js consists of two parts, including administrator and user modules. The administrator’s modules have the graduate program data management and the criteria decision data management modules. The user’s modules consist of six modules, including the graduate program data selection, the criteria decision selection, and the AHP and the fuzzy AHP modules of multi-criteria decision-making, evaluation criteria, calculation criteria weights, and graduate program ranking. Figure 6 illustrates the overall system using a use case of a multi-criteria graduate admission recommender system.

The graduate program management module is responsible for managing the available graduate programs and universities. The system allows administrators to add, edit, or delete programs and universities, as well as define each program’s admission requirements and information. However, there were only eight universities and doctoral and master’s degree programs in this experiment.

The criteria management module handles the graduate recommender selection criteria. Administrators are able to add, modify, or delete criteria and define the default weights for each criterion.

The client management module is responsible for managing user accounts and authentication, which enables users to create accounts, sign in, sign out, and utilize the recommender system. A user interested in applying to graduate programs can interact with the recommender system to receive university suggestions based on explicit multi-criteria user preferences. In addition, the AHP and fuzzy AHP algorithms were utilized to determine the weighted rankings of the graduate admission programs that met the criteria and to generate program recommendations for the students. Finally, the system can generate recommendations for graduate programs based on the student’s profile and the explicit weights assigned to graduate admission criteria.

2.3.4. Information Flow Diagram of Explicit Criteria Graduate Recommender System

As depicted in Figure 7, the information flow diagram (IFD) of the explicit criteria graduate recommender system is designed to provide a seamless and user-friendly experience for identifying suitable graduate programs. The principles of IFD novelty deliver a unified visual representation of system architecture, data and information flow, processes, database systems, user interfaces, and user interaction with the system [89,90,91,92] that can be used to describe the entire related design and development in this study. The system design can perform up to any number of criteria selections. However, from user interviews, the number of user criteria selections concluded that the maximum selections of up to eight criteria were satisfied for all 80 users. The user can decide on up to eight specific requirements from a dynamic list of criteria based on the program’s significance. The system then generates a page where the user’s preferences for each selected criterion can be specified. After the user has selected the pertinent criteria, they can begin by selecting multiple criteria and explicit criteria preferences. The user then inputs the scores for each pairwise admissions criterion for graduate programs. Then, a user can select graduate programs of interest and specify unambiguously preferences. The system calculates a weighted score for each university based on the user’s preferences and displays the recommendation results by ranking and using a chart, providing the user with an easy-to-understand ranking of the universities. The user interface is designed to ensure that the user can navigate the system easily, searching for the most suitable program effortlessly. In the context of a recommender system, the dynamic criteria section refers to factors open to change over time and can affect the recommendation process. One way to describe the dynamic criteria section of the recommender system is to identify the types of possible changes and how they may impact the recommendation process. Changes in user preferences, item availability or popularity, and the significance of particular attributes could be examples of dynamic criteria. To accommodate these dynamic criteria, the recommender system can be designed to incorporate real-time data and adapt the recommendation procedure accordingly. For instance, if a user’s preferences change, the system can update the user’s profile and make appropriate adjustments to the recommendations. However, the design is required to assess the accuracy of the design’s effectiveness for the dynamic criteria of the recommender system, particularly for graduate admission.

2.4. Evaluation and Data Collection

Evaluation and data collection are important for developing and enhancing a recommender system. The primary focus of this study is the individual user-selected program recommender system based on multiple dynamic criteria explicitly defined for a graduation admission recommendation. The actual testing system is then accumulated to determine the recommendations’ accuracy rate. The eighty graduate applicants’ students were invited to test the recommender system. All students have studied Information Technology in master’s and doctoral programs. The recommendation results of the dynamic multi-criteria recommender system for graduate programs are validated against the graduate programs in which students are currently enrolled, and true positive and negative results are demonstrated. This study compared AHP and fuzzy AHP approaches, and the recommended system was evaluated. The authors demonstrate the probability correctness metrics of the dynamic multi-criteria recommendation for graduate admission using precision, recall, F1-score, and top-N accuracy. Each metric provides a unique perspective on the comprehensive accuracy combination assessment. This study’s accuracy scores did not account for false positive and false negative results. Therefore, it is essential to consider evaluation of metrics to understand the system’s accuracy comprehensively.

The recommender system forecasts a student’s list of universities. The accuracy of recommendations is determined by whether the student applies to and is admitted to the universities recommended. If the student is admitted to a university the system recommends, the recommendation is accurate. If the student decides not to apply to the university suggested by the system, the recommendation is considered inaccurate.

Multiple metrics [93,94,95], such as precision, recall, and F1-score, are considered when evaluating a recommender system for graduate program recommendations to provide a comprehensive assessment of the system’s accuracy. The system’s precision is measured by the number of recommended universities to the admitted student. Precision measures the accuracy of the system’s positive predictions. It is the proportion of true positive recommendations to the total number of system recommendations. A high precision score indicates that the recommender system provides accurate recommendations and minimizes irrelevant suggestions. Using Equation (6), the precision can be computed.

$$\mathrm{Precision}=\frac{Number of relevant recommended items}{Total number of recommended items}$$

(6)

The recall measures the proportion of recommended universities, and the student makes the admission decision. Recall indicates the system’s ability to identify each relevant item in the recommendation. Recall determines the proportion of true positive recommendations relative to the total number of relevant items in graduate admission programs. For graduate admission, the recall metric refers to the proportion of programs the user selects in the preferences and for which the system recommends. In this study, true positive always has a value of 1, representing only the recommendation with the highest weight ranking. A high recall score indicates that the recommender system identifies all relevant items and those that are irrelevant. A low recall score indicates that the system is missing relevant items and may not be appropriate for a recommender system that requires complete coverage of relevant items. Using Equation (7), the recall can be computed.

$$\mathrm{Recall}=\frac{True Positive}{True Positive+False Negative}$$

(7)

F1-score is the harmonic mean of precision and recall providing a balanced evaluation metric for the recommender system. F1-score is a practical aim to achieve a balance between the two. The F1-score can be calculated using Equation (8).

$$\text{F1-score}=2 \times \frac{Precision\times Recall}{Precision+Recall}$$

(8)

System accuracy can also be calculated using Equation (9).

$$\mathrm{System} \mathrm{accuracy}=\frac{Correct recommendations}{Total recommendations}\times 100\%$$

(9)

Moreover, top-N accuracy reflects the percentage of recommended items that appear in the list’s top-N positions. This metric is useful when the user is presented with a list of recommended items and is instructed to consider only the top N items. This study utilizes top-1 accuracy to determine the proportion of cases in which the top-ranked item has significance to the user. This metric is useful when only the highest-ranked item is considered. Using top-N and top-1 accuracy, user feedback can be used to evaluate the system’s accuracy performance. For instance, if the system’s top-1 accuracy is high, it recommends the most relevant graduate program as the top-ranked item. Top-1 accuracy quantifies the frequency with which the system recommends the most pertinent item. It is the most stringent metric for evaluating the recommender system, as it requires that the system recommend the best-suited item to the user. When the system is expected to recommend only one item, and it is crucial to recommend the most applicable item to the user, top-1 accuracy is practical.

To evaluate the system performance accuracy of a recommender system, these metrics were calculated to compare recommender systems to determine which one is performing better. Top-N and top-1 accuracy are appropriate metrics for a recommender system that requires a specific number of recommendations and demands high accuracy. The choice of evaluation metrics depends on the recommender system’s specific requirements and the expected evaluation results. This study reports multiple evaluation metrics to comprehensively assess the system’s performance for a dynamic multi-criteria graduate recommender system.

3. Results

3.1. Descriptive Statistic Results

The demographic data collection of the sample testing group includes various items such as gender, age, education level, institution, degree, and institution degree, as shown in

Table 4. Demographic’s testing users.

Items	Description	Sample	%
Gender	Male	59	73.75
	Female	21	26.25
Age	22–31	24	30.00
	32–41	34	42.50
	42–51	19	23.75
	>52	3	3.75
Users of Institutions	UA	20	25.00
	UB	16	20.00
	UC	3	3.75
	UD	5	6.25
	UE	19	23.75
	UF	4	5.00
	UG	3	3.75
	UH	10	12.50
Seeking Degree	PhD	29	36.25
	MS	51	63.75
Students Admitted Graduate Degrees of Institutions	PHD-UA	5	6.25
	PHD-UB	4	5.00
	PHD-UC	3	3.75
	PHD-UD	5	6.25
	PHD-UE	5	6.25
	PHD-UF	4	5.00
	PHD-UG	3	3.75
	PHD-UH	0	0
	MS-UA	15	18.75
	MS-UB	12	15.00
	MS-UE	14	17.50
	MS-UH	10	12.50

Note disclosure universities’ name, location, and user information. However, the universities are private and public universities in Thailand.

. In terms of gender, the sample consisted of 59 males (51.25%) and 21 females (26.25%). The age range of the participants was from 22 to 51, with the highest percentage of participants falling within the 32–41 age range (42.50%). The education level of all participants was undergraduate, with 51 (63.75%). The sample included users from eight universities: UA, UB, UC, UD, UE, UF, UG, and UH. The number of users from each institution was 20 (25%), 16 (25%), three (3.75%), five (6.25%), 19 (23.75%), four (5%), three (3.75%), and 10 (12.50%), respectively. All participants are studying for a doctoral degree, with 29 (36.25%), or for a master’s degree, with 51 (63.75%), from UA 15 (18.75%), UB 12 (15.0%), UE 14 (17.50%), and UE 10 (12.50%). Overall, the testing users were diverse in gender, age, and educational background, which could provide a more comprehensive representation of the sample group.

3.2. Dynamic Multi-Criteria AHP and Fuzzy AHP Results

During system usage and recommendation processing, the dynamic multi-criteria recommender system for graduate admission confirmed that users could select multi-criteria based on their explicit criteria preference justifications and graduate admission interest constraints. The system provides dynamic multi-criteria and multi-criteria ordering preferences.

Table 5. Binary representations on data collection of dynamic multi-criteria decisions from users.

Users	Criteria
	C1	C2	C3	C4	C5	C6	C7	C8	C9	C10	C11	C12	C13	C14	C15	C16	C17
1	0	1	0	1	0	0	1	0	0	0	0	0	0	0	1	0	0
2	0	0	0	0	0	1	0	1	0	0	0	0	1	0	0	0	0
3	0	1	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0
4	0	1	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0
5	0	0	0	0	0	0	0	0	0	1	0	0	0	1	0	0	0
6	0	1	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0
7	1	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
8	0	1	0	0	0	1	0	0	0	0	0	0	0	1	0	0	0
9	0	0	0	0	0	0	0	1	1	1	0	0	1	0	0	0	0
10	1	0	0	0	0	0	0	0	0	1	0	0	0	1	0	0	0
11	0	0	0	1	0	1	0	0	0	0	0	0	0	0	0	0	1
12	1	0	0	0	0	1	0	0	0	0	0	0	1	0	0	0	0
13	0	0	0	0	0	1	0	0	0	1	0	0	0	0	0	0	0
14	0	0	0	1	0	0	0	0	0	1	0	0	0	0	0	0	0
15	1	0	1	0	0	1	0	0	0	1	0	0	1	0	0	0	1
16	1	1	1	0	0	0	0	0	0	0	0	0	1	1	1	0	1
17	0	1	0	1	0	0	0	0	0	1	0	0	1	0	0	0	0
18	0	0	0	0	0	0	0	0	0	1	0	0	0	1	0	0	0
19	1	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0
20	0	0	1	0	1	1	0	0	0	1	1	0	0	0	0	0	0
…
38	1	0	1	1	1	1	1	1	1	0	0	0	0	0	0	0	0
…
80	0	1	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0

demonstrates the bit representations of eighty students’ dynamic multi-criteria preferences. Each row’s data represents a binary response of a student selecting multi-criteria in differences, where a value of 1 indicates that the criterion was established and a value of 0 indicates that the criterion was not chosen. By analyzing the criteria selection based on various user decisions, the authors discovered that the number of specified criteria ranged from two to eight, representing various dissimilar criteria selections. Even though a few students selected the same criteria, their preferences for those criteria were distinct. For instance, the third, fourth, sixth, and seventh users selected the same two criteria but selected different criteria aspects. The third user chose C6 and C8. The fourth user chose C2 and C3. The sixth user chose C2 and C10. The seventh user chose C1 and C2. Therefore, the data analysis can provide insight into the dynamic multi-criteria with explicit preferences that users consider to be the most important constraints when selecting an admission to a graduate program via the recommender system.

By analyzing the values of Table 5, the most and least important essential criteria were summarized and ranked for the findings of the most interesting criteria selected based on user needs, as shown in

Table 6. Dynamic multi-criteria of graduate admission preferences based on explicit selections.

Criteria Code	Criteria	Count	Preferred	Percent
C2	Popularity/University reputation	17	0.137097	13.7097
C10	Academic staff	16	0.129032	12.9032
C13	Cost of program/Tuition fees/Cost of tuition	14	0.112903	11.2903
C1	State/Private university	13	0.104839	10.4839
C6	Class time/Flexibility of timetable	12	0.096774	9.6774
C3	Location of university	9	0.072581	7.2581
C4	Entry requirements/Eligibility to apply	8	0.064516	6.4516
C8	Condition of publications/Programs offered/Academic quality/preparation for graduation	8	0.064516	6.4516
C14	Scholarship opportunities/Financial assistance	6	0.048387	4.8387
C9	English scores	6	0.048387	4.8387
C11	Technological facilities	4	0.032258	3.2258
C15	Promotion or discount	4	0.032258	3.2258
C17	Admission processes	3	0.024194	2.4194
C7	Education language	2	0.016129	1.6129
C5	Length of study/Course duration	2	0.016129	1.6129
C12	Connection with foreign universities	0	0	0
C16	Application fee	0	0	0
		Total	1.0	100

. Exact criteria evaluation based on graduate admission preferences was conducted to determine the preferences of eighty students on seventeen distinct criteria (C1 to C17), indicating varying degrees of user preference and significance for each criterion. Overall, the data analysis reveals a great deal of variation in students’ criterion preferences, with no clear consensus on the most important criterion. However, according to the table data, popularity/university reputation (C2) is the most frequently selected criterion and was chosen by 17 users, or approximately 13.71 percent. With 16 selections, the academic staff (C10) criterion is the second most popular among users, accounting for approximately 12.90 percent. The third criterion is the cost of the program/tuition/fees/cost of tuition (C13), which 14 users, or 11.29 percent, selected. The fourth criterion, State/Private university (C1), is favored by 13 users, or approximately 10.48 percent. Class time/timetable flexibility (C6) is the fifth criterion chosen by 12 users, accounting for roughly 9.68 percent.

Figure 8 displays the exhaustive user data and calculated values of the dynamic multi-criteria recommender system for graduate admission. The results display the first, second, and third weight-ranking scores of alternatives based on AHP and fuzzy AHP for eighty master’s and doctoral students who served as test users. The consistency ratio (CR) quantifies decision-making consistency. A CR value below 0.1 indicates an acceptable consistency level. The consistency ratio values are all less than 0.1, indicating a satisfactory degree of consistency in the decision-making process. The majority of individuals had three options for graduate admission to universities, while a few students had four options. The first weight-ranking alternative predicted the highest weight based on the user’s explicit preference for multiple criteria. The second and third alternatives for weight ranking were sequence-ordering weight-ranking scores. In comparison to the first weight-ranking alternative of AHP and fuzzy AHP, the results indicate that AHP calculates weight-ranking scores with a broader range or slightly higher values than fuzzy AHP. The trend lines of the first weight-ranking alternative reveal a slight rise for both AHP and fuzzy, indicating that the relative criteria have become significant in calculating the weight-ranking values for both approaches over time. Compared to the second weight-ranking alternative of AHP and fuzzy AHP, the calculated weight-ranking scores for fuzzy AHP have a wider range or are slightly higher. The trend lines of the second weight-ranking alternative exhibit a similar decline for both AHP and fuzzy, indicating that the relative criteria used to calculate the weight-ranking values of both approaches have become insignificant over time. In comparison to the third weight-ranking alternative of AHP and fuzzy AHP, the results indicate that fuzzy AHP calculates weight-ranking scores with greater variability or slightly higher values than AHP. The fuzzy AHP trend line for the third weight-ranking alternative demonstrates a slight increase, indicating that the relative criteria have become increasingly significant in calculating weight-ranking values for the approach over time. In contrast, the AHP trend line for the third weight-ranking alternative demonstrates a slight decrease, indicating that the relative criteria have become irrelevant in calculating the weight-ranking values of the approach over time.

3.3. System Accuracy Performance Results

The study’s findings reveal the system accuracy of the dynamic multi-criteria recommender system for graduate admissions with a maximum of up to eight criteria and up to four alternatives, based on the information responses of 51 test users for master’s degree students. The users established explicit multi-criteria preferences and ranked the master’s degree program alternatives. Their responses were analyzed utilizing AHP and fuzzy AHP methodologies to determine the weight rankings of alternatives. Figure 9 illustrates the AHP-calculated weighted rankings of the four alternatives. All weight-ranking alternatives are validated using the sum of their respective values, which is always 1. Only four incorrect predictions were made out of fifty-one test cases, indicating that the recommender system could provide reasonably accurate recommendations. The results of all first alternatives indicate that user #38 had the highest weight ranking, with a weighted ranking of 0.701%, followed by user #29, with a weighted ranking of 0.6824. User #13 had the lowest weight ranking, of 0.3706, one of the incorrect predictions, and was therefore ranked last. The finding provides valuable insights into the users’ preferences and can aid in decision-making. The accuracy of the predictions was determined by comparing the predicted weight rankings to the graduate programs to which the student was actually admitted. Six users out of 51 made incorrect predictions, resulting in an accuracy rate of 88.24%. Overall, the AHP method effectively determined the relative weights of the alternatives and predicted the recommendations with the most powerful preference.

Based on the master’s degree users of 51 students, the study’s results present the weighted rankings of four alternatives determined using fuzzy AHP. The test users were asked to rank the alternatives according to their conditional preferences, and their explicit multi-criteria needs were analyzed using fuzzy AHP to determine weight rankings. Figure 10 illustrates the calculated weight rankings for the four graduate admission programs. The sum of all weight alternatives indicates the total weight rankings for every user, which equals 1. The correct dot indicates whether or not the recommendation made by the recommender system was precise, based on the actual graduate program accepted. With only four incorrect predictions out of 51 users, the recommender system provided reasonably accurate recommendations. In calculating the weight rankings of the alternatives based on user responses, the fuzzy AHP methods proved effective. The initial weight-ranking results indicate that user #38 was ranked highest with a weighted ranking of 0.6975, followed by user #29 with a weighted ranking of 0.6808. User #13’s first weight ranking of 0.3663 was the lowest of all users. The accuracy of the predictions was determined by comparing the predicted rankings to the actual graduate program into which the student was accepted. Four incorrect predictions were identified, resulting in an accuracy rate of 92.16 percent.

The study’s findings reveal the weighted rankings of up to four alternatives calculated using AHP and fuzzy AHP for 29 test users based on explicit multi-criteria and graduate program preferences for doctoral degree programs. Users of the examination were required to rank graduate programs in order of preference. Using AHP and F-AHP, their explicit multi-criteria conditions and preferences were analyzed to determine the weight rankings of admission recommendations. The weighted sum of all alternatives displays the total weight rankings for every user, which is always 1. Figure 11 and Figure 12 depict the AHP and fuzzy AHP results of admissions recommendations for doctoral programs. Green circle dots represent the correct recommendations. In addition, inaccurate recommendations are represented by red circle dots. Whether or not the user’s score recommendations were accurate, based on the actual first weight rankings, they resulted in the same outcome as the students’ real doctoral program. The results indicate that the dynamic multi-criteria recommender system provides reasonably accurate graduate admission recommendations, with only four incorrect predictions using the AHP method and three inaccurate predictions using fuzzy AHP out of 29 test cases.

In calculating the first weighted rankings of graduate recommendations based on explicit multi-criteria of user preference requirements, the fuzzy AHP methods proved effective and accurate. Using the AHP method, the first weighted ranking results indicate that user #18 was ranked highest with a weighted ranking of 0.6989, followed by user #19 with a weighted ranking of 0.6647. User #26 had the lowest first weight ranking, with a value of 0.3269. Employing the fuzzy AHP method, the first weighted ranking results indicate that user #18 was also ranked highest with a weighted ranking of 0.6984, followed by user #19 with a weighted ranking of 0.6676. User #26 also had the lowest first-weight position, with a score of 0.3186. The accuracy of the predictions was determined by comparing the predicted rankings to the actual graduate program into which the student was accepted. The AHP results revealed four incorrect predictions, resulting in a system accuracy rate of 86.21 percent. The fuzzy results revealed three inaccurate forecasts, resulting in a system accuracy rate of 89.66 percent.

As depicted in Figure 13, Top-1, Top-2, and F1-score can be used to draw the following conclusions based on the accuracy performance analysis of the comparison between AHP and fuzzy AHP approaches on the dynamic multi-criteria recommender system of graduate admission, specifically in the master’s degree and doctoral degree admission. The accuracy scores for TOP-1 and TOP-2 are consistently high across all four findings. The overall top-1 system accuracy of the AHP recommender system for graduate admissions to master’s degree and doctoral programs is 88.24 and 86.21 percent, respectively. In contrast, the top-1 system accuracy performance of the fuzzy AHP recommender system for graduate admissions to master’s degree and doctoral programs is 90.20 and 89.66 percent, respectively. All top-2 system accuracy performances of the dynamic multi-criteria recommender system employing AHP and fuzzy AHP for graduate admission are 100%. The system accuracy scores for the AHP approach for graduate admission recommendations of both master’s and doctoral students are slightly lower than those for the fuzzy AHP approach, indicating that the recommender system with the fuzzy approach is more effective and accurate in evaluating the criteria and making decisions. The recommendation results are likely unquestionably appropriate for decision-making for graduate admission to the respective programs. The F1-Score obtained for each of the four methods reveals insignificant differences in system precision, indicating that precision and recall are nearly balanced. Overall, the dynamic multi-criteria recommender system employing the fuzzy AHP approach appears more effective and accurate in suggesting admission to master’s and doctoral programs. However, additional analysis and evaluation are required to validate these results and ensure the recommender system’s accuracy and dependability.

4. Discussion

This study’s software architecture is designed with a layered approach [96,97] that separates the presentation layer, application layer, and data layer, allowing for better modularization and scalability and ensuring the system can accommodate a large number of users and data points with a highly customizable and adaptable solution to meet individual user requirements. With a user-friendly interface and intuitive navigation, the system is intended to allow users to easily input their multi-criteria preferences and receive personalized recommendations based on their specific needs. The software architecture and design support the responsive web recommender system for graduate admission. The architecture represents numerous software components pertaining to users, internet connectivity, multi-criteria management, program selection, university selection, decision-making processes, consistency checking, weight calculation, graduate program ranking, server configuration, and database connection and storage. The system was designed and implemented using a software architecture incorporating both AHP and fuzzy AHP approaches to support explicit multi-criteria and provide a robust dynamic multi-criteria recommender system. Our system supports up to eight dynamic criteria with explicit user preference definitions, providing a highly configurable solution and capability to adapt to individual user requirements. This study utilized MongoDB as a NoSQL cloud database service. According to [98,99], NoSQL is a non-relational database that handles a large volume of data with high performance, schema flexibility, availability, data replication, and scalability. A bit string can be divided into a criteria preference bit (CPB) and a criteria consistency bit (CCB) to support binary conversion to n-bit numbers that can save storage space, reduce data misuse, prevent data modification, and support data encryption and decryption. The first set of binary strings consists of the user’s explicit selection of criteria, while the second set verifies the consistency of the explicitly preferred and selected criteria.

Several criterion requirements must be met in a multi-criteria graduate admission recommender system to ensure effectiveness and accuracy. Our research uncovered seventeen significant and relevant criteria for a graduate admission recommender system. By comparing the early research on explicit criteria of program selection, this study has found that the top five desirable ranking criteria are university popularity and reputation, academic staff, and tuition fees, state/private university, and class time/flexibility of timetable as well as the study [81,85]. According to the study, the university’s reputation has the highest statistical scores, followed by completion time and academic quality. In addition, study [21] found that the academic staff, university ranking scores, popularity, and facilities have become the most essential factors in deciding where to study. Similar to study [84], structural equation modeling (SEM) was used to examine how university reputation influenced students’ choice of university. The results of weight analysis in descending order indicate that university reputation is an essential factor in determining whether to pursue further education, regardless of the country in which the studies are conducted.

In this study, AHP and fuzzy AHP can be used to evaluate and compare the decision-making accuracy results stated in [37,56,58]. This study evaluated the accuracy of AHP and fuzzy AHP recommender systems for graduate admission to master’s degree and doctoral programs based on dynamic multi-criteria. The results revealed that both AHP and fuzzy AHP systems had a high degree of accuracy in predicting admission recommendations based on explicitly defined individual criteria. Comparing the recommendation results of 80 user cases between the two systems revealed that the recommender system utilizing the fuzzy AHP approach had a marginally higher system accuracy than the AHP approach. The results of the fuzzy AHP system allow for greater accuracy in recommendations and the justification of uncertainty and imprecision in the graduate admission criteria defined by individuals. Both AHP and fuzzy AHP systems identified the admission recommendation accurately based on the top-1 and top-2 accuracy rankings. The findings suggest that a dynamic multi-criteria recommender system utilizing AHP and fuzzy AHP techniques is an innovative and viable option for developing a graduate admission recommender technique. However, deciding to use one method over another will depend on the institution or organization implementing the system’s particular needs and preferences.

MCDM techniques are used to handle the complex decision-making process involving multiple criteria and alternatives. In this study, AHP and fuzzy AHP are compared using weight-ranking alternatives to support the system’s accuracy. While AHP is widely used in decision-making scenarios involving complex hierarchical structures and multiple criteria, fuzzy AHP extends AHP to handle imprecise and linguistic information, where decision data involves fuzzy sets or linguistic variables with uncertainty in the decision process [36,37,38]. However, other techniques and methodologies are available in the field of MCDM approaches, such as ordinal priority approach (OPA), robust OPA, data envelopment analysis–ordinal priority approach (DEA–OPA), and fuzzy and interval OPAs. OPA is used in multiple attributes that can calculate weights and rank experts, alternatives, and attributes simultaneously [100] by avoiding the usage of pairwise comparison matrices, decision-making matrices, and normalization procedures [101]; for example, when a complex recommender system for an educational system is developed. Many criteria should be identified to support various domains, such as admission program selection, interesting research projects, and adviser’s research fields. Robust OPA extended to OPA enables it to handle uncertainty and variability in criteria values and conditions [102], such as user preferences, criteria weights, data accuracy, and prediction accuracy. The graduate admission recommender system enables control of the university preferences depending on a user’s selection. The DEA–OPA model can consider the experts’ opinions, weights of experts, and quantitative data to calculate the efficiency of and handle efficient alternatives in case of a lack of information and knowledge by considering multiple inputs and outputs [103]. Fuzzy OPA enables the modeling of linguistic variables and preferences to handle imprecise and vague information, allowing decision-makers to express subjective assessments more flexibly and nuancedly. Subjective assessments in the field of fuzzy OPA for graduate admission refer to the process of incorporating the subjective judgments and preferences of decision-makers into the decision-making process [104]. In the context of graduate admission, decision-makers, such as admission committees or evaluators, often rely on their expertise, experience, and subjective assessments when evaluating and selecting candidates. The flexibility accommodates the inherent subjectivity involved in graduate admission decision-making. In addition, interval OPA extends OPA for considering interval-valued criteria. The internal OPA can be used for uncertain criteria, but ranges can be identified. In the context of graduate admission, subjective assessments using interval OPAs allow decision-makers to express their subjective judgments and preferences when evaluating and selecting candidates. Each method has its own strengths and applicability in different decision-making scenarios, depending on the nature of the problem, available data, and decision-makers preferences supported in [105].

5. Conclusions

5.1. Theoretical Contributions

This study provides a number of theoretical contributions to the success of software architecture design, system development, and the evaluation of the accuracy of recommendation outcomes that influence an individual’s decision-making using the AHP and fuzzy AHP approaches to a dynamic multi-criteria recommender system, particularly in graduate admission studies. The study sheds light on the explicit multi-criteria preferences of graduate admission programs chosen by users based on their dynamically explicit needs. The data analysis collected from 80 graduate students revealed the majority of most significant and minority of least significant criteria based on user preferences. The results analysis revealed the diversity of criterion preferences among students and emphasized the absence of a consensus regarding the most important criterion. In addition to evaluating the accuracy performance of the system, the study demonstrated the utility of dynamic multi-criteria AHP and fuzzy AHP methods for explicitly prioritizing multiple criteria in the graduate admission recommender system. In addition, the data structure utilized in this study was created using a bit-representation technique to save storage space, reduce data misuse, prevent data modification, and support encryption and decryption. In this study, the fuzzy AHP technique demonstrated slightly higher practical accuracy than the AHP method. Comparing the two techniques can assist decision-makers and developers in software engineering in selecting the appropriate technique based on the specific application’s requirements.

5.2. Practical Implications

This research has implications for designing and implementing graduate admission recommender systems. Universities and institutions can gain insight into the factors influencing students’ decision-making by investigating the most intriguing criteria. This information can be utilized to enhance program offerings, enhance academic quality, and optimize tuition fee structures to attract prospective students. In addition, applying dynamic multi-criteria AHP and fuzzy AHP methods enables universities to provide students with individualized recommendations based on their preferences and priorities for admission conditions. The implication of the recommender system is that the use of dynamic multi-criteria and explicit criteria preferences can improve the admission recommendation process’ precision and transparency. AHP and fuzzy AHP can be utilized by decision-makers and developers in software engineering to aid with the appropriate technique, depending on the specific application’s requirements.

5.3. Limitations and Future Work

Although the differences in system accuracy performance between AHP and fuzzy AHP approaches were relatively small, they may become more pronounced when dealing with larger datasets or more complex multi-criteria models. This study is essential for identifying current research gaps. The research was conducted with a specific sample group from private and public universities in Thailand, which may limit the findings’ applicability to other contexts. In addition, the research centered on predefined criteria and the implementation of additional criteria or customization options that could further improve the effectiveness and accuracy of the recommender system. Future research could investigate integrating machine learning and data mining techniques to improve the recommender system’s accuracy and efficacy. Nonetheless, conducting multiple studies and collecting information from a more diverse and extensive sample would provide a deeper understanding of the factors influencing graduate admission decisions.