*5.2. Sensitivity Analysis*

The sensitivity analysis was performed to study the consistency of the obtained ranking. Ranks of the two direct weighting techniques (point allocation and direct rating) and VASMA weighting were determined and compared (Figure 5). Two popular direct weighting techniques SMART and SWING were not included in the comparison because of the methodological differences in the data collection procedure [9]. *Symmetry* **2020**, *12*, 1641 16 of 20

**Figure 5.** Criteria rank comparison for the different preference elicitation methods. **Figure 5.** Criteria rank comparison for the different preference elicitation methods.

A comparison of the criteria ranks reveals differences between the point allocation (PA), direct rating (DR), and the VASMA weighting approaches (Figure 5). Due to the different direct weighting methodologies, PA and DR techniques give significantly different results. Greater stability can be observed between the criteria ranks determined by the direct rating and the VASMA weighting approaches. However, the weight values calculated by the DR and VASMA weighting techniques noticeably differ. Since a little variation of the weight values is repeatedly identified as the downside of the straightforward DR technique, VASMA weighting can be chosen as the solution to this issue. Results presented in the Figure 4 prove that the variance of the weighting values for the criterions C10 (Distance from home), C2 (Skills of kindergarten teachers), C8 (Outdoor safety and hygiene), and C7 (Indoor safety and hygiene) are considerably wider when the novel preference elicitation technique VASMA weighting is applied. A comparison of the criteria ranks reveals differences between the point allocation (PA), direct rating (DR), and the VASMA weighting approaches (Figure 5). Due to the different direct weighting methodologies, PA and DR techniques give significantly different results. Greater stability can be observed between the criteria ranks determined by the direct rating and the VASMA weighting approaches. However, the weight values calculated by the DR and VASMA weighting techniques noticeably differ. Since a little variation of the weight values is repeatedly identified as the downside of the straightforward DR technique, VASMA weighting can be chosen as the solution to this issue. Results presented in the Figure 4 prove that the variance of the weighting values for the criterions C10 (Distance from home), C2 (Skills of kindergarten teachers), C8 (Outdoor safety and hygiene), and C7 (Indoor safety and hygiene) are considerably wider when the novel preference elicitation technique VASMA weighting is applied.

#### **6. Conclusions 6. Conclusions**

the calculation of the objective weights.

variation of the criteria weights.

Criteria weighting is an integral part of the multicriteria decision-making process. When the opinions of the wider audience are needed, electronic surveys may be successfully employed to collect data for the preference elicitation procedure. Since both the psychologists and psychometricians agree that humans are much better at making comparative judgments than at making absolute judgments, visual analogue scales (VAS) have been proposed as the affective data collection tool for the assessment of the respondents' traits. However, survey-based criteria weighting processes are typically accompanied by the biases of the evaluators and the uncertainty of the experimental conditions. Besides, end-aversion bias and the positive skew are also the companions of the VAS based preference elicitation. The novel criteria weighting technique VASMA weighting respects the psychometric features of the VAS scales and analyzes the uncertainties caused by the survey-based criteria weighting. It is achieved by integrating the WASPAS-SVNS multicriteria Criteria weighting is an integral part of the multicriteria decision-making process. When the opinions of the wider audience are needed, electronic surveys may be successfully employed to collect data for the preference elicitation procedure. Since both the psychologists and psychometricians agree that humans are much better at making comparative judgments than at making absolute judgments, visual analogue scales (VAS) have been proposed as the affective data collection tool for the assessment of the respondents' traits. However, survey-based criteria weighting processes are typically accompanied by the biases of the evaluators and the uncertainty of the experimental conditions. Besides, end-aversion bias and the positive skew are also the companions of the VAS based preference elicitation. The novel criteria weighting technique VASMA weighting respects the psychometric features of the VAS scales and analyzes the uncertainties caused by the survey-based criteria weighting. It is achieved by integrating the WASPAS-SVNS multicriteria decision making

regarding the choice of the kindergarten institution was performed to reveal the practicalities of the proposed methodology. The experiment presented in this paper revealed that the data processing technique integrated into the VASMA weighting methodology is able to overcome the main disadvantages of the direct rating technique—the high biases of the collected data and the low

In the future, it would be interesting to analyze why the last three criterions presented in the VAS matrix got significantly lower weights than the rest of them. Is it an accidental situation, or is it associated with their position in the VAS Matrix? An optimal number of the criteria that can be weighted with the VASMA weighting methodology also should be analyzed in the future. approach for the determination of the subjective weights and Shannon entropy for the calculation of the objective weights.

A numerical example analyzing the importance of the criteria that affect parents' decisions regarding the choice of the kindergarten institution was performed to reveal the practicalities of the proposed methodology. The experiment presented in this paper revealed that the data processing technique integrated into the VASMA weighting methodology is able to overcome the main disadvantages of the direct rating technique—the high biases of the collected data and the low variation of the criteria weights.

In the future, it would be interesting to analyze why the last three criterions presented in the VAS matrix got significantly lower weights than the rest of them. Is it an accidental situation, or is it associated with their position in the VAS Matrix? An optimal number of the criteria that can be weighted with the VASMA weighting methodology also should be analyzed in the future. It would be also interesting to disclose how the number of respondents and their homogeneity affects VASMA weighting values.

**Author Contributions:** Conceptualization, I.L., E.K.Z., R.B. and B.J.; methodology, I.L., E.K.Z., R.B. and B.J.; software, I.L.; validation, I.L., E.K.Z., R.B. and B.J.; formal analysis, I.L. and R.B.; investigation, I.L. and B.J.; resources, I.L. and B.J.; data curation, I.L.; writing—original draft preparation, I.L. and B.J.; writing—review and editing, I.L., E.K.Z., R.B. and B.J.; visualization, I.L.; supervision, R.B.; project administration, I.L. and R.B. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Conflicts of Interest:** The authors declare no conflict of interest.
