**4. Discussion and Conclusions**

The main aim of the paper is to introduce a methodology suitable for e-learning course evaluation and selection. The emergence of e-courses as the modern way of learning has provoked the need for finding the methods ideal for their assessment. In this paper, the methodology based on the PIPRECIA and the interval-valued triangular fuzzy ARAS methods is proposed. The applicability of the proposed

methodology is presented through a numerical case study. When defining criteria, special attention was dedicated to the issues of organization and teaching. So, in this case, the attention was not directed on the technical and informational performance of an e-learning course, but towards the quality of the offered content and how the teaching process was implemented.

Based on the data obtained from the respondents, the PIPRECIA method was applied, and the weights of the criteria for each of the twenty-four respondents were obtained, as is shown in Table 1. The obtained results show that the weights of the criteria are approximate, which impose the fact that the given features are nearly equally important to all respondents. This is entirely understandable because the e-learning course should satisfy all the requirements and, in that way, offer the quality of "service" to the users.

The reason for applying the PIPRECIA method for the determination of the weights lies in its simplicity and suitability for use in cases where a large number of decision-makers are involved in the evaluation process. The advantage of the PIPRECIA method over the well-known and widely used AHP method is reflected in a more straightforward computational procedure that does not diminish the reliability and relevance of the results obtained. Also, when interviewing respondents who are not familiar with the MCDM methods, the process of evaluating weights by using the PIPRECIA method is far more understandable to respondents, than is the case with the AHP method. If the PIPRECIA method is compared with the SWARA method (on which the PIPRECIA method was developed), it can be concluded that the PIPRECIA method has certain advantages over it. Namely, the SWARA method requires that the evaluation criteria should be sorted according to their intended significance, which complicates its application in group decision cases. Many complex decision-making problems require the participation of a group of respondents. In such cases, the individual attitudes of the respondents have to be transformed into group attitudes, with an as small as possible loss of information. In order to take into account the uncertainty and imprecision of the data on which decision-making is very often based, the application of the interval-valued triangular fuzzy ARAS method is proposed. The approach in which individual ratings are transformed into interval-valued triangular fuzzy numbers can be very useful in this regard. The interval-valued triangular fuzzy ARAS method may use such information to rank alternatives and/or analyze different scenarios. Thus, by applying this method, decision-makers have been given the opportunity to express their optimistic, pessimistic, and realistic attitudes.

In this paper, the numerical case study of the e-learning course selection was examined. The reason for that relies on the increasing importance of this kind of learning. In order to create the high quality e-learning course, it is necessary to determine the pros and cons of the considered course and its position relative to the competition. In that way, the creators will know what aspects of the course should be improved and what are of satisfactory quality. The application of the proposed integrated approach has proven to be quite justified and appropriate in this case. The reason is that if the e-learning courses were evaluated based solely on the use of crisp numbers, the obtained results would not include uncertainty. This would result in a decision that would not be completely realistic and, ultimately, unreliable. The obtained results confirmed this point of view. To get the most reliable results and to make the best possible decisions, it is necessary to respect the risk and uncertainty to the maximum extent possible. So, based on the conducted numerical case study, the e-learning course designated as *A*<sup>2</sup> is the best in terms of evaluated criteria.

As the examination of the literature has shown, the authors used different approaches for e-course evaluations. Chao and Chen [56] examined which factors are crucial for the quality of the e-learning courses. They applied the consistent fuzzy preference relations (CFPR) with AHP methodology. They evaluated four groups of factors that are elaborated in a particular number of criteria. The final results showed that the most influential criteria are: the e-learning material, friendly user interface, using the web discussion zone, and distant learning without time and space. The main point of this paper is the quality of the content of the e-learning courses. The assessment of the evaluation criteria showed that e-learning material has the greatest influence together with the group work and interactivity, which is in line with the results obtained from the mentioned authors.

Garg and Jain [57] applied the combination of the methods for defining the best e-learning website. They divide the evaluation criteria into two groups called quality factors and e-learning specific factors. The second group of factors is pointed to the quality of e-learning content and their results showed that the most important criterion is the ease of learning community, which could be considered as a counterpart to the group work and interactivity presented here.

Besides the mentioned works, others present the utilization of different MCDM techniques for e-learning course evaluation and selection. For the resolving of the problem of evaluation of e-course quality, the authors have proposed the application of the proximity indexed value (PIV) model [58], fuzzy ANP [59], DANP and VIKOR [60], and so on. The conducted numerical case study presented in this paper, as well as the comparison with the results of the given authors, confirmed that the proposed approach is also very beneficial for e-learning course evaluation and selection because the obtained results clearly outlined the crucial characteristics and the position of the specific e-learning course comparing to the others.

Therefore, the proposed integrated PIPRECIA-IVTFN-ARAS model has proven to be useful and feasible, especially in circumstances where it is essential to make the most relevant and realistic decision possible. The proposed integrated model can be extended to other areas of the business as well.

As a direction for future research, a significantly larger sample could be used in order to obtain results from a macro point of view. Besides that, the learning outcomes of the learners that would study the ranked courses can be further investigated as well.

**Author Contributions:** Conceptualization, K.J.J., D.K., and G.P.; methodology, D.K., D.S., and E.K.Z.; validation, G.J.; investigation, G.J.; data curation, G.P.; writing—original draft preparation, D.S. and E.K.Z.; writing—review and editing, K.J.J. and P.T.N.; supervision, D.K.; funding acquisition, P.T.N. All authors have read and agreed to the published version of the manuscript.

**Funding:** The APC was funded by: Department of Project Management, Ho Chi Minh City Open University, Ho Chi Minh City, Vietnam; and Faculty of Applied Management, Economics and Finance, University Business Academy in Novi Sad, Belgrade, Serbia.

**Conflicts of Interest:** The authors declare no conflict of interest.
