*2.2. Direct Weighting Approaches*

Most of the currently used subjective approaches are based on the opinions of the specially trained experts [39]. Subjective weights calculation from the survey data is much rarer.

Theoretically, the VAS matrix might be exploited to collect data for the preference elicitation based on the pairwise comparisons. PIPRECIA-E [34] is an example of the pairwise comparison technique that might be used to obtain the attitudes of the respondents that were not specially trained for the criteria weighting. However, it should be mentioned that pairwise comparison is highly sensitive to the data loss caused by the respondent's unwillingness to assess all the criteria. Since a high level of the missing data is normally generated in the survey-based preference elicitation, application of the pairwise comparisons techniques might be especially challenging. Due to the nature of the pairwise comparison, responses, where at least one criterion is not weighed, should be omitted. Such a data cleaning procedure drastically reduces the number of responses; therefore, it might be an important disadvantage of its exploitation for the survey-based criteria weighting.

Direct weighting techniques are the most commonly used for online preference elicitation. In the direct methods, the decision-maker compares criteria by using a ratio scale, whereas, in indirect methods, criteria weights are calculated based on the preferences of the decision-maker [30]. Direct weighting approaches like the SWING [42], SMARTS [43], SMARTER [43], direct rating [44], and the point allocation [44] were recently used in a survey-based preference elicitation [9,45,46].

SWING method implies the construction of the extreme hypothetical scenarios, where initially a hypothetical worst-case scenario is presented, and then the criterion that might be enhanced to improve the overall situation the most is identified as the most important criterion which gets 100 points. All other criteria are weighted in a similar manner and get the point values less than 100 points.

In SMART (Simple Multi-Attribute Rating Technique) the order of the criteria importance is determined primarily and then, starting from the least important criterion, the relative importance of the criteria is assigned in the ascending order. SMARTS and SMARTER are elaborated versions of SMART [43]. SMARTS imply the procedure for determining criteria weights by comparing criteria with the best and the worst criterion from a defined set of criteria. SMARTER (SMART Exploiting Ranks) uses the centroid method to determine criteria weights [47].

Point allocation (PA) and direct rating (DR) are two relatively simple techniques that have lots in common but produce systematically different weighting results [44]. Decision-makers are asked to allocate 100 points among the analyzed criteria when the PA is applied. In the DR methodology, each object is separately assessed on a scale from 0 to 100. Since DR weights do not add up to 1 (100%), they should be normalized at the final stage of the preference elicitation. Direct rating is highly

recommended when the performance evaluation relies on a large number of the criteria and when a respondent does not feel comfortable using complex weighting methods. Moreover, the weights elicited by DR are more reliable than those elicited by PA [44]. However, a little variation of the averaged weights is repeatedly identified as the downside of the straightforward DR technique [48].
