Multi-Criteria Decision Analysis of Road Transportation Fuels and Vehicles: A Systematic Review and Classification of the Literature

Abstract

Multi-Criteria Decision Analysis (MCDA) methods help decision makers to consider and weigh diverse criteria that include economic, environmental, social and technological aspects. This characteristic makes them a popular tool to comparatively evaluate road transportation fuels and vehicles (RTFV). The aim of this paper is to systematically classify and analyse the literature applying MCDA methods on the evaluation of RTFV. To this end, 40 relevant papers are pinpointed and discussed. We identified a great number of evaluation criteria employed in the reviewed papers from which we have established a concluding list of 41 criteria, that can serve as a pool for future research. A further analysis of the evaluation criteria reveals that the process of criteria selection partly suffers from a lack of scientific foundation and standardization. We propose to standardize the criteria selection process by using the Life Cycle Sustainability Assessment (LCSA) methodology as a guiding reference. In addition, we compared the MCDA results obtained from studies with relatively similar setups and found that the evaluation results are also generally similar and seem not to be influenced by the particular MCDA method employed. Based on the results of the reviewed papers, one may say that electricity and ethanol appear to be good alternatives for light vehicles, whereas gaseous fuels seem more appropriate for heavy vehicles like buses. Striking deviations from these generally observed results are often caused by specific evaluation contexts, particular criteria taken into account and unusual weight sets applied.

1. Introduction

Road transportation is dominated by internal combustion (IC) vehicles fueled by conventional fuels like gasoline or diesel [1]. Nevertheless, a variety of alternatives for fossil fuels exists. These may comprise fuels replacing gasoline or diesel in IC engines, vehicles with electric engines in various forms and combinations with IC engines and as a more specific case hydrogen as a potential fuel for fuel cell vehicles. But how can decision makers decide which road transportation fuels and vehicles (RTFV) are worthwhile to support via policy measures and which are not?

Evaluating which RTFV are better suited to replace the prevailing road transportation fuels is complicated because manifold and interrelated economic, social, and environmental aspects need to be simultaneously considered [2]. For example, electric vehicles (EV) emit zero tailpipe emissions, which makes them preferable to IC vehicles especially in highly populated areas. Their global warming impact, however, depends on the greenhouse gas (GHG) emissions emitted during the production of the electric power [3]. Also the overall environmental impact is not solely related to the GHG emissions. For a thorough analysis, it is necessary to consider environmental impacts throughout the entire product life cycle [4]. In addition, the production of batteries for EV can lead to increases in terrestrial acidification levels or human toxicity potential and thus entails additional negative environmental and social impacts [5]. A transition from the current conventional to an electric refueling system would also require high additional investments into infrastructure [6].

Such complex considerations need to be taken into account when evaluating RTFV. Multi-Criteria Decision Analysis (MCDA) methods offer the possibility to evaluate and rank a number of alternatives on the basis of different evaluation criteria with varying scales or units [7], which makes MCDA methods a suitable tool for evaluating alternatives to the currently dominating road transportation fuels [8]. In this way, MCDA methods also assist in soliciting and resolving conflicts among decision makers.

The aim of this paper is to systematically classify and analyse the literature applying MCDA methods on the evaluation of RTFV. The classification of papers is done according (a) to the MCDA methods applied, (b) to the alternatives evaluated, and (c) to the criteria used in the MCDA process. By grouping and classifying the criteria the paper aims to provide a concluding list of relevant evaluation criteria as a pool for criteria selection and source of inspiration for future research. The particular focus of our analysis on RTFV allows us to compare the MCDA evaluation results obtained by studies with relative similar setups in order to identify generally valid ranking results.

Section 2 continues with a definition of the RTFV covered in this review and Section 3 shortly introduces the MCDA methods most often applied in the reviewed papers. Relevant literature is presented and discussed in Section 4. Section 4.1 presents the findings of similar literature review papers, and Section 4.2 summarizes the MCDA methods applied and the background of the reviewed papers. Subsequently, Section 5 categorizes the alternatives evaluated in the reviewed papers (Section 5.1) and groups the evaluation criteria considered for these analyses (Section 5.2). In Section 5.3 we additionally discuss how and why different authors select their specific evaluation alternatives and criteria. Section 6 compares the ranking results of similar analyses and detects generally preferable alternatives. Section 7 includes conclusions and outlook.

2. Definition of Road Transportation Fuels and Vehicles

The RTFV analyses in the reviewed papers are defined in this section. Conventional fuels like diesel and gasoline are refined from crude oil and are commonly used in IC engines. Alternative RTFV either replace these fuels or work with a different motor technology. Three categories of alternatives are considered in this article: (a) fuels that directly replace diesel and gasoline, (b) electric and electrified vehicles and (c) hydrogen as a special case, with a focus on its means of production.

The first category (a) consists of fuels that can be used in IC engines.

Table 1. Alternative road transportation fuels as defined in the reviewed papers (Source: [9]).

Alternative Road Transportation Fuels	Feedstock & Production	Used in
Compressed Natural Gas (CNG)	Processed from natural gas	IC engines, also in various combinations with gasoline or diesel
Propane/Liquid petroleum gas (LPG)	Propane and butane extraction from natural gas
Biogas (also called synthetic natural gas—SNG)	Anaerobic digestion/fermentation of biomass or landfill gases. Upgrading to higher purity
Ethanol (1st generation)	Fermentation of biomass (e.g., corn, wheat).	IC gasoline engines, blended with gasoline
Cellulosic ethanol (2nd generation)	Hydrolysis using enzymes and subsequent fermentation of cellulosic material (e.g., straw, wood)
Methanol	Steam-reforming of natural gas, catalytic synthesis
Biodiesel/fatty acid methyl ester (FAME)	Transesterification of vegetable oil (from e.g., canola, rapeseed, soybean)	IC diesel engines, blended with diesel
Dimethyl ether (DME)	Gasification and catalytic synthesis (e.g., methanol, synthesis gas, often derived from natural gas)	IC diesel engines (with slight modifications)
Biomass to liquid (BTL), synthetic diesel	Thermochemical production: Gasification & Fischer-Tropsch synthesis of biomass (e.g., wood, black liquor)	IC diesel engines as drop-in fuel (DIF)—blending at all levels possible, also full replacement, no modifications necessary
Hydrotreated vegetable oils (HVO), Hydroprocessed Esters and Fatty Acids (HEFA)	Hydroprocessing of vegetable oils (e.g., palm oil)

summarizes data on these alternative fuels that are discussed in the reviewed papers, including the relevant production processes and feedstocks.

The second category (b) is represented by electric and electrified vehicles including fuel cell vehicles. Different forms of combinations of electric and IC engines are possible, as shown in

Table 2. Electric and electrified vehicles.

Electric/Electrified Vehicles	Fuel(s)/Carrier(s)	Storage of Fuel(s)	Motor(s)
Battery Electric Vehicles (BEV)	Electric power	Batteries	Electric motor
Plug-in hybrid electric vehicles (PiHV)	Electric power and usually diesel or gasoline	Batteries and tank for diesel or gasoline	Electric motor and IC engine
Hybrid electric vehicles (HEV)
Fuel cell vehicles	Hydrogen, methanol	Tank for hydrogen or methanol	Electric motor, electric power generated through chemical process in fuel cell

.

Battery electric vehicles (BEV) are ones running solely on electric power. They derive power from battery packs through an electric motor and have no IC engine, fuel cell, or fuel tank. Hybrid electric vehicles (HEV) carry both an IC and an electric engine. Two options are discussed in the papers reviewed: HEV with recharging of batteries via regenerative brakes and through an electric generator powered by the IC engine, and plug-in hybrid electric vehicles (PiHV) with the extra capability to recharge the battery by external sources. Fuel cell cars power their electric engine with a combination of a battery and a fuel cell system, which converts the chemical energy from fuels with a relatively high energy density into electricity. Most fuel cell vehicle systems work with hydrogen, but also methanol, ethanol or methane, among others, are possible fuels [10].

The last category (c) is concerned with hydrogen in particular, as is a potential energy carrier in the aforementioned fuel cell vehicles. Three specific hydrogen production paths are evaluated, i.e., electrolysis, steam reforming and gasification, sourced from different feedstock [11,12,13,14,15].

3. Multi-Criteria Decision Analysis Methods

This Section shortly outlines the MCDA methods most often used in the reviewed papers. MCDA methods can be broadly divided in Multi-Attribute Decision Making (MADM) with limited solutions for a finite set of alternatives and Multi-Objective Decision Making (MODM) with indefinite solutions for an indefinite set of possible scenarios [16]. MODM methods usually comprise some form of optimization, for instance linear or nonlinear programming. These methods start with certain restrictions which determine the optimal solution [17] and find a solution by optimizing the resulting optimization function. Only few papers applying MODM techniques are discussed in this review, whereas MADM models are applied in most of the reviewed papers.

The most elementary MADM method is the ‘Weighted Summation Method’ (WSM). The score of an alternative is calculated by multiplying the normalized criteria scores with weightings attached to each criterion. Adding up the scores yields the final score. The higher the score, the better the alternative is placed in the ranking [18].

Another method that is widely used is the ‘Technique for Order Preference by Similarity to Ideal Solutions’ (TOPSIS), developed by Hwang and Yoon [19]. The best solution is found as the alternative whose weighted Euclidean distance has the shortest distance to the positive ideal solution and the greatest distance to the negative ideal solution.

Another popular method is the ‘Analytical Hierarchy Process’ (AHP), developed by Saaty [20]. It decomposes the evaluation problem in a hierarchical structure with several criteria and sub-criteria. The criteria are evaluated by pairwise comparisons. The eigenvectors for each of the evaluation matrices are calculated, weighted and aggregated to obtain the final score.

The ‘Preference Ranking Organization Method for Enrichment Evaluation’ (PROMETHEE) was originally developed by Brans [21] and is classified as a multi-criteria outranking method. The method attaches an own preference function with specific thresholds to each criterion. Then, pairwise comparisons of the alternatives in combination with the preference functions result in specific outranking relations. The positive or incoming flow—measuring the strengths of all alternatives—and the negative or outgoing flow—measuring the weaknesses of all alternatives—determine the PROMETHEE rankings. Both flows are used separately to establish the PROMETHEE I partial rankings, whereas the net result of the two flows determines the PROMETHEE II final ranking.

VIKOR (from the Serbian abbreviation for multi-criteria optimization and compromise solution), developed by Opricovic [22], is similar to TOPSIS. It determines a compromise solution as the closest to the ideal alternative (Tzeng et al. [23]).

Fuzzy set theory, developed by Zadeh [24], is not a MCDA method per se, but it can be used in such a context. It enables analysts to include subjectivity and uncertainty in human judgements as well as incomplete or vague information in an analysis. Instead of providing a crisp number, i.e., a fixed value, as evaluation, decision makers provide interval judgements. In the aggregation process, the fuzzy numbers or judgements are defuzzified in a prespecified way.

The analysis in most of the reviewed papers is structured in two evaluation steps. Firstly, weightings of the importance of each of the criteria are defined. Some authors apply predefined weightings, but also MCDA methods—in particular AHP—are used to calculate the weights. Secondly, the final ranking is obtained by applying a MCDA method. The weightings previously obtained are needed for this second step. The analysis in our review is merely concentrated on the second step, the final ranking procedure.

4. Literature Review

This Section discusses the findings of other similar to ours literature reviews in Section 4.1. Then, Section 4.2 shortly explains how we scrutinized the literature for relevant papers and subsequently summarizes the evaluation context and background of the reviewed papers.

4.1. Overview of Other Similar Literature Reviews

This section presents papers that review articles with research questions similar to the research questions guiding our analysis as outlined in Section 1. Pohekar and Ramachandran [16] review more than 90 papers applying MCDA techniques to sustainable energy planning. They find the most popular techniques to be the AHP, followed by PROMETHEE and ‘Elimination and Choice Expressing Reality’ (ELECTRE).

Wang et al. [25] assess papers applying MCDA methods on sustainable energy related problems. They identify AHP as the MCDA method most frequently used. The authors also aggregate the criteria used in the reviewed papers and classify them by technical, economic, environmental, and social aspects. Investment cost and CO₂ emissions are the criteria most often used, followed by efficiency, operation and maintenance cost, NO_x emissions, and land use.

Huang et al. [26] review papers applying MCDA in the environmental field. Similar to Wang et al. [25] they find AHP to be the method most often used, followed by Multi-Attribute Utility Theory (MAUT) and outranking methods like PROMETHEE and ELECTRE.

Scott et al. [27] analyse MCDA literature related with bioenergy systems, covering in total 57 papers. They discovered that most papers—25—use MCDA techniques that usually handle few alternatives, i.e., methods like AHP, ELECTRE and PROMETHEE, followed by 16 papers applying multi-objective optimisation via programming algorithms.

Arce et al. [28] provide a literature review on papers applying grey-based MCDA analysis to the evaluation of sustainable energy systems. Similar to Wang et al. [25], they count the use of different criteria and find technical efficiency and maturity, followed by operational and maintenance costs, GHG emissions and investment costs to be most often applied.

Mardani et al. [8] review 89 papers applying MCDA techniques to transportation system problems. Most of the papers are concerned with the airline industry and few with the specific evaluation of RTFV. They find hybrid MCDA and fuzzy MCDA as well as AHP and fuzzy AHP to be the dominating MCDA methods in the reviewed papers. The content and findings of the identified papers are summarized in

Table 3. Overview over similar MCDA-related literature reviews.

Review Papers	Focus on	MCDA Methods Most Often Used	Evaluation Criteria Most Often Used
Pohekar and Ramachandran [16]	Sustainable energy planning	AHP, ELECTRE, PROMETHEE	Not analysed
Wang et al. [25]	Sustainable energy decision-making	AHP, WSM, ELECTRE	Investment cost and CO2 emissions, efficiency, operation and maintenance costs, NOx emissions, land use
Huang et al. [26]	Environmental sciences	AHP, MAUT, PROMETHEE	Not analysed
Scott et al. [27]	Bioenergy systems	Optimisation with few alternatives	Not analysed
Arce et al. [28]	Sustainable energy systems	Focused only on Grey relational analysis. Often in combination with AHP	Technical efficiency and maturity, operational and maintenance costs, gas emissions and investment costs
Mardani et al. [8]	Transportation systems	Hybrid MCDA and fuzzy MCDA methods, AHP and fuzzy AHP	Not analysed

.

The focus of these literature reviews is wider than the focus of our review paper, which is specifically on scientific articles applying MCDA methods on RTFV. Thus, and due to this wider research focus, the review papers discussed above are neither able nor do they intend to provide a comparison of the results obtained. Accordingly, the list of evaluation criteria found in Wang et al. [25] and Arce et al. [28] are formed from papers with a much broader study range than ours. The particular scope of our analysis allows us to compare the study setups and results of similar papers. The novelty and contribution of our analysis is to (a) group relevant evaluation criteria for evaluations of RTFV as a pool for future research, (b) analyse the rationale for the selection of evaluation criteria and alternatives, and (c) to detect generally observed ranking results based on comparable analyses. In the next section, the identified papers are briefly discussed.

4.2. Summary of MCDA Methods and Background of Reviewed Papers

In this section, we summarize the MCDA methods and the background of the reviewed papers. A total of 40 papers applying MCDA methods on selecting RTFV have been identified and discussed in our systematic review. In order to be relevant for our analysis, papers have to be published in a peer-reviewed journal and apply MCDA methods on a comparison of RTFV related alternatives. This specific focus of our analysis limits the number of matches to 40 papers.

Table 4. Summary of reviewed papers and their evaluation context.

Reviewed Paper	Scope of Study	Region of Interest	MCDA Ranking Method
Maciol and Rebiasz [29]	RTFV for light vehicles	Portugal	AHP, TOPSIS, conventional crisp reasoning, Mamdani’s method of fuzzy inference
Büyükozkan et al. [30]	RTFV for buses	Istanbul, Turkey	Intuitionistic Fuzzy Choquet Integral, 2-additive Choquet integral, TOPSIS
Sehatpour et al. [31]	RTFV for light vehicles	Iran	PROMETHEE
Oztaysi et al. [32]	Light trucks and vans for utility company	USA	TOPSIS with interval-valued intuitionistic fuzzy sets
Osorio-Tejada et al. [33]	RTFV for road freight transport	Spain	AHP
Mukherjee [34]	RTFV for the road sector in general	In general	Intuitionistic fuzzy TOPSIS
Cai et al. [35]	RTFV for tax fleet in Beijing	China	TOPSIS, WSM, ELECTRE III
Onat et al. [36]	RTFV for light vehicles	USA	TOPSIS
Onat et al. [37]	RTFV for light vehicles	USA	Compromise Programming Model
Maimoun et al. [38]	RTFV for waste collection vehicles	USA	TOPSIS/WSM
Yavuz et al. [39]	RTFV for home health care service provider	USA	Hierarchical hesitant fuzzy linguistic model—WSM
Ren et al. [40]	Bioethanol production pathways	China	VIKOR
Lanjewar et al. [41]	RTFV for light vehicles, and buses in urban areas	China, USA, Taiwan	Graph theory and matrix approach (GTMA)
Ziolkowska [42]	Biodiesel, ethanol	USA	Linear programming, fuzzy PROMETHEE
Hayashi et al. [43]	Biodiesel fuel production	Japan, Kyoto	WSM
Aydın and Kahraman [44]	RTFV for buses	Turkey, Ankara	VIKOR/AHP
Ziolkowska [45]	Biodiesel, ethanol	USA	Fuzzy PROMETHEE
Tsita and Pilavachi [46]	Various fuel types	In general	AHP
Streimikiene et al. [47]	RTFV for light vehicles	Lithuania	Interval TOPSIS
Ren et al. [15]	Hydrogen production	In general	Fuzzy MAMCDM
Tsita and Pilavachi [48]	RTFV for the road sector in general	Greek road sector	AHP
Traut et al. [49]	RTFV for light vehicles	USA	Nonlinear Programming
Manzardo et al. [13]	Hydrogen production	In general	GRA
Heo et al. [12]	Hydrogen production	Korea	Benefits, Opportunities, Costs, and Risks approach (BOCR)
Vahdani et al. [50]	RTFV for buses in urban areas	Taiwan	Fuzzy TOPSIS, Preference Selection Index (PSI)
Turcksin et al. [51]	Policies to promote RTFV	Belgium	PROMETHEE
Turcksin et al. [52]	Various fuel types	Belgium	AHP & Multi-Actor Multi-Criteria Analysis (MAMCA)
Perimenis et al. [53]	Biofuel options	In general	Own scheme, point scale from 1 to 5 to obtain comparable results. Simple multiplication of weights and criteria.
Fazeli et al. [54]	RTFV for light vehicles	Portugal	MAUT, Pareto Optimal, Trade-off weights, Data Envelopment Analysis, WSM
Chang et al. [11]	Hydrogen production	Taiwan	WSM of fuzzy criteria
Safaei Mohamadabadi et al. [55]	RTFV for light vehicles	USA	PROMETHEE
Rogers and Seager [56]	Various fuel types	USA	PROMETHEE/SMA-LCIA
Pilavachi et al. [14]	Hydrogen production	In general	WSM
Dinh et al. [57]	Biodiesel production	USA	AHP
Quintero et al. [58]	Ethanol production	Colombia	WSM
Papalexandrou [59]	Biofuels, for 5% replacement of fossil fuel light vehicles	EU	WSM
Zhou et al. [60]	RTFV for light vehicles	China	WSM
Tzeng et al. [23]	RTFV for buses in urban areas	Taiwan	VIKOR, TOPSIS
Yedla and Shrestha [61]	2-stroke vs 4-stroke 2-wheelers, conventional fuel vs CNG buses, conventional fuel vs CNG cars	India, Delhi	AHP
Poh and Ang [62]	RTFV for land transportation	Singapore	AHP, WSM (simple multiple criteria screening model)

presents the MCDA methods used in each paper, the scope of each paper and the regional context, if any. Some of the methods are modified to allow for fuzzy evaluations. The scope of most studies is on the evaluation of alternative fuels for light vehicles, followed by the comparison of alternative fuels in general. The regional context for the evaluation is most often the USA.

As further summarized in

Table 5. MCDA methods of reviewed papers: frequency of appearance (more than one method possible per paper).

MCDA Methods	Frequency of Appearance
Weighted summation method (WSM)	12
of which fuzzy methods	2
TOPSIS	10
of which fuzzy TOPSIS	3
AHP	9
of which fuzzy AHP	1
PROMETHEE	6
of which fuzzy PROMETHEE	2
VIKOR	3
of which fuzzy VIKOR	1
MODM methods (objective optimization)	3
of which fuzzy LP	1
Other	11

, the MCDA method most often applied is the WSM, followed by TOPSIS, AHP, and PROMETHEE. Three papers apply MODM methods. Traut et al. [49] develop a mixed-integer nonlinear optimization model, Ziolkowska [42] applies a fuzzy linear programming (LP) model and Onat et al. [37] use ‘Compromise Programming’, a multi-objective optimization model, which was first introduced by Zeleny [63].

A number of MCDA methods are summarized under item ‘Other’ in Table 5. Maciol and Rebiasz [29] compare the results and applicability of four methods: Conventional crisp reasoning, Mamdani’s method of fuzzy inference, TOPSIS and AHP. Rogers and Seager [56] use PROMETHEE with a stochastic weight set, an approach called ‘Stochastic Multiattribute Life Cycle Impact Assessment’ (SMA-LCIA). Vahdani et al. [50] apply TOPSIS and additionally the ‘Preference Selection Index’ (PSI), developed by Maniya and Bhatt [64], which they extend for a fuzzy environment. Fazeli et al. [54] use a ‘Multi-Attribute Utility Theory’ (MAUT) model with a multi-stage screening process consisting of various stages applying the ‘Pareto Optimal’ (PO) approach, ‘Data Envelopment Analysis’ (DEA) and the ‘Trade-off Weights’ procedure. The final ranking is obtained with WSM. Manzardo et al. [13] develop an improved ‘Grey Relational Analysis’ (GRA) with interval numbers instead of a single crisp number. Heo et al. [12] use the ‘Benefits, Opportunities, Costs, and Risks’ (BOCR) concept as proposed by Saaty and Ozdemir [65]. Ren et al. [15] develop a novel fuzzy ‘Multi-Actor MCDA” (MAMCDA) method which consists of evaluations based on linguistic variables provided by stakeholders. Lanjewar et al. [41] combine the ‘Graph Theory and Matrix Approach’ (GTMA) and the AHP method. Cai et al. [35] apply ELECTRE additionally to WSM and TOPSIS. Büyüközkan et al. [30] compare the results of a newly proposed Intuitionistic Fuzzy Choquet Integral to the ones obtained with a 2-additiive Choquet integral and TOPSIS.

The following Section 5 analyses which alternatives and criteria were selected in this study and why. Section 6 compares the results of similar evaluation setups with different MCDA methods.

5. Selection of Alternatives and Criteria

This Section categorizes the reviewed papers by the analyzed alternatives (Section 5.1) and the criteria used in this process (Section 5.2), while Section 5.3 discusses how and why the authors select their evaluation alternatives and criteria.

5.1. Categorization of the Reviewed Papers by the Analysed Alternatives

The following detailed categorization of the reviewed papers by the alternatives analysed is essential to understand the comparison of results in Section 6 and facilitates researchers to detect papers relevant to their field. Two of the reviewed papers do not directly fit into this categorization and are thus left out: Turcksin et al. [51] are concerned with the analysis of policies promoting RTFV and Perimenis et al. [53] present a general MCDA framework to analyse biofuels without conducting a comparative evaluation of specific alternative fuels.

The first group of reviewed papers shown in

Table 6. Alternative fuels and vehicles as analysed in the reviewed papers.

Alternatives	Maciol and Rebiasz [29]	Büyüközkan et al. [30]	Sehatpour et al. [31]	Oztaysi et al. [32]	Osorio-Tejada et al. [33]	Mukherjee [34]	Cai et al. [35]	Onat et al. [36]	Maimoun et al. [38]	Yavuz et al. [39]	Ren et al. [40]	Hayashi et al. [43]	Aydin and Kahraman [44]	Ziolkowska [45]	Tsita and Pilavachi [46]	Streimikiene et al. [47]	Ren et al. [15]	Tsita and Pilavachi [48]	Turcksin et al. [52]	Fazeli et al. [54]	Safaei Mohamadabadi et al. [55]	Rogers and Seager [56]	Dinh et al. [57]	Quintero et al. [58]	Papalexandrou et al. [59]	Zhou et al. [60]	Tzeng et al. [23]	Yedla and Shrestha [61]	Poh and Ang [62]
Conventional fuels
Diesel	x	x	x		x	x			x			x	x		x	x	x		x		x	x				x	x	x
Gasoline	x						x	x								x	x	x			x	x				x		x	x
Other									x																				x
Gaseous fuels
CNG/LNG		x	x	x	x	x			x	x			x			x					x					x	x	x	x
Propane/LPG		x	x	x		x				x			x			x											x		x
Biogas (SNG)			x						x						x				x
Replacing diesel
Biodiesel/FAME				x					x	x		x	x	x	x	x	x		x	x	x	x	x		x				x
BTL (gasification)																	x		x						x
DME															x
HVO/HEFA (DIF)					x												x
Replacing gasoline
Ethanol 1st gen.			x	x							x		x	x	x	x	x		x	x	x	x		x	x	x
Cellulosic ethanol															x		x								x
Methanol		x	x			x									x											x	x		x
Electric and electrified vehicles
BEV	x	x		x		x	x	x		x			x		x	x	x	x		x		x					x
PiHV	x						x	x					x			x	x	x		x	x						x
Fuel cell vehicles		x	x			x							x		x		x			x	x						x		x
HEV	x	x				x	x	x					x				x	x		x									x

consists of papers that assess the first two categories of alternatives as defined in Section 2: (a) alternative fuels that can be used in IC engines and (b) electric drive vehicles The second group consists of papers evaluating alternatives that are concerned only with hydrogen production methods, as listed in

Table 7. Hydrogen production methods as analysed in the reviewed papers.

Hydrogen Production Methods	Ren et al. [15]	Manzardo et al. [13]	Heo et al. [12]	Chang et al. [11]	Pilavachi et al. [14]
via electrolysis		x	x	x	x
via steam reforming		x	x	x	x
via gasification	x	x	x	x	x
via fermentation	x			x
via pyrolysis of biomass	x	x
via nuclear thermochemical cycle technology		x
via supercritical water gasification of biomass	x
via partial oxidation of hydrocarbons					x
as by-product from steel production			x

.

5.2. Analysis and Grouping of Relevant Evaluation Criteria

Although the analyses in the reviewed papers are relatively similar, the criteria used in the assessment process are often differently defined. Simply adding up all the individually labelled criteria used in the papers would lead to more than 100 criteria. To make the analysis of the criteria meaningful and understandable, similar criteria are aggregated, as far as it is possible, without losing valuable information. This grouping of criteria is done in a way that considers the system boundaries of the analysis. Our boundaries distinguish the feedstock production or drilling, the refining process and oil production, the production and operating phase of the vehicle, and the refuelling infrastructure, as graphically shown in Figure 1. The various reviewed papers consider different boundaries for the analysis.

After the aggregation and summarization process, 41 criteria remain, as listed in

Table 8. Criteria listed by paper and occurrence.

Criteria	Aspect Concerned	Occurrence	Maciol and Rebiasz [29]	Büyüközkan et al. [30]	Sehatpour et al. [31]	Osorio-Tejada et al. [33]	Mukherjee [34]	Cai et al. [35]	Onat et al. [36,37]	Maimoun et al. [38]	Yavuz et al. [39]	Ren et al. [40]	Hayashi et al. [43]	Aydin and Kahraman [44]	Ziolkowska [45]	Tsita and Pilavachi [46]	Streimikiene et al. [47]	Ren et al. [15]	Tsita and Pilavachi [48]	Traut et al. [49]	Heo et al. [12]	Turcksin et al. [51]	Turcksin et al. [52]	Perimenis [53]	Fazeli et al. [54]	Chang et al. [11]	Safaei Mohamadabadi et al. [55]	Rogers and Seager [56]	Pilavachi et al. [14]	Dinh et al. [57]	Quintero et al. [58]	Papalexandrou et al. [59]	Zhou et al. [60]	Tzeng et al. [23]	Yedla and Shrestha [61]	Poh and Ang [62]
GHG	Env.	34	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x	x
Final cost of fuel	Eco.	24		x	x	x	x	x	x	x				x	x	x	x	x	x			x		x	x	x	x		x	x	x	x	x			x
Costs of vehicle (production)	Eco.	18		x		x	x	x		x	x		x	x			x		x	x		x	x		x		x							x	x	x
Air pollution	Env.	16	x	x	x	x	x	x		x			x	x			x					x	x	x	x			x						x
Safety issues	Social	13			x				x		x		x	x	x			x					x		x	x				x					x	x
New employment & economic growth	Social	13			x	x			x		x	x	x		x	x		x	x		x		x	x
Capital/transition cost refineries & fuel production	Eco.	12			x											x		x	x		x	x	x	x	x	x			x		x
Operating cost refineries & fuel production	Eco.	12			x										x	x		x	x		x	x	x	x		x			x		x
Social acceptability	Social	12			x	x					x			x		x		x			x	x	x			x									x	x
Resource & energy security/dependency	Social	12		x	x				x				x			x			x		x		x		x							x			x	x
Availability of feedstock or fuel in general	Eco/Tech.	12		x		x	x							x				x				x	x		x							x		x	x	x
Feedstock cost & stability	Eco.	11			x					x		x			x	x			x			x		x		x			x		x
Availability of refuelling infrastructure	Eco/Tech.	11			x	x		x		x	x											x	x	x	x		x								x
Capital & transition cost fuelling infrastructure	Eco.	10			x			x								x			x	x	x	x		x	x									x
Environmental issues of feedstock production	Env.	11	x		x			x	x	x		x			x								x	x				x		x
Social welfare/support of local communities/economic value added	Social	9			x				x		x	x	x		x			x	x				x
Energy conversion efficiency fuel production	Tech.	9		x			x		x									x						x		x						x	x		x
Maintenance costs of vehicle	Eco.	9		x		x	x	x			x			x											x									x	x
Availability of refineries & processing infrastructure	Eco/Tech.	8			x											x					x	x	x	x	x										x
Productivity of feedstock production/land required	Tech.	8							x						x			x					x	x		x				x		x
Technological progress & maturity & development potential	Tech.	8														x		x	x		x	x		x	x	x
Environmental impact of refineries & fuel production & fuelling stations	Env.	9	x					x	x			x								x			x	x				x			x
Capital & transition cost of new motor production	Eco.	7														x			x	x		x			x									x		x
Market acceptance & consumer welfare	Eco.	7				x					x				x			x			x		x			x
Sense of comfort & fuelling convenience	Tech.	7		x		x	x				x			x									x											x
Fuel range/energy content	Tech.	6			x					x	x			x											x									x
Regulation & taxes	Social	6				x		x	x									x				x	x
Noise	Social	6		x	x	x	x							x																				x
Reliability & vehicle life	Tech.	5				x					x			x				x							x
Energy conversion efficiency motor	Tech.	5		x										x																		x		x	x
Environmental impact of motor production/Recyclability	Env.	6	x					x	x					x						x								x
Vehicle characteristics	Tech.	5		x			x							x											x									x
Food security	Social	4										x			x	x							x
Availability of vehicle production facilities	Eco/Tech.	4			x											x							x												x
Renewable share & resource depletion	Env.	4																				x			x			x					x
Farmers’ income	Eco.	3											x		x								x
Technical compatibility & number of vehicles available	Tech.	3																					x				x									x
Relationship to other industries	Eco.	3																			x		x											x
Technological dependency	Social	2																			x					x
Specific fuel properties	Tech.	1																												x
Fairness & Level playing field	Social	1																					x

. The criteria are sorted by their occurrence and are tagged with the type of aspect concerned (environmental, economic, social, and technical). These types of aspects are chosen similar to Wang et al. [25] and should help to better understand the different characteristics of each criterion.

Table 8 shows that environmental criteria are most often considered, followed by economic and social criteria. Technical issues like the production efficiency or the land productivity seem to play a lesser role. It is also shown that, apart from some main criteria, authors seem to rely on a huge variety of criteria with differently defined system boundaries.

The literature reviews presented in Section 4.1 do not categorize the identified list of evaluation criteria by the system boundaries, arguably because their review fields, e.g., sustainable energy or transportation systems in general, are more broadly defined than our research field. Nevertheless, a comparison of the results reveals that the main evaluation criteria are similar. Wang et al. [25] also observe a comparable occurrence of criteria in their review on MCDA aid in sustainable energy decision-making, whereas Arce et al. [28] find similar criteria, but with different occurrences. Contrary to our findings, though, technical efficiency criteria are relatively often used in the papers appeared in the review by Arce et al. [28] whereas only one social criterion, called the human/technological impact, is included in their list.

5.3. Rationale for Selection of Alternatives and Criteria

Many authors seem to only partly explain how and why they select their evaluation alternatives. Some authors justify their selection by limiting their alternatives to the most common feedstock and production methods in a specific country. For instance, Quintero et al. [58] compare the two most important feedstocks for ethanol production in Colombia and Ziolkowska [45] the most common feedstocks for ethanol and biodiesel production in the USA. Heo et al. [12] confine their alternatives to the hydrogen production methods available in Korea in the near future and Turcksin et al. [52] consider only the most important biofuel options in Belgium. Yavuz et al. [39] limit their options to the alternative fuel vehicles available to a specific home health care provider. Osorio-Tejada et al. [33] choose their alternatives based on a study on clean power for transportation from the European Commission. In many papers, however, the reasons for the selection of the specific evaluation alternatives seem not adequately explained and this is also the case for the selection of the evaluation criteria.

The evaluation criteria determine the outcome of the MCDA analysis to a large extent. Thus, the selection of the appropriate criteria is one of the major difficulties faced by MCDA researchers [12]. Surprisingly, less than half of the authors of the reviewed papers seem to provide for a complete justification of why and how they select their specific evaluation criteria. Turcksin et al. [52] use the MAMCA framework, i.e., various stakeholders, to identify relevant criteria, whereas Turcksin et al. [51] rely on expert opinion to find policy-relevant criteria. Heo et al. [12] rely on the BOCR approach and a review by experts to select their criteria. Other authors use a Life Cycle Sustainability Assessment (LCSA) or Life Cycle Assessment (LCA) approach to define their evaluation criteria [13,29,35,36,37,40], while Hayashi et al. [43] base their selection on the indicators in the Global Bioenergy Partnership. Perimenis et al. [53] and Büyüközkan et al. [30] rely solely on expert opinion to identify relevant criteria. Furthermore, Streimikiene et al. [47] use the EU Transport Policy as background for their selection and Papalexandrou et al. [59] base their evaluation on a well-to-wheel analysis, which also defines the criteria and alternatives in their study. Osorio-Tejada et al. [33] choose their criteria based on the findings of Wang et al. [25] and Ziolkowska [45] uses the criteria developed by Silva Lora et al. [66].

To the extent it is comprehensible and explained in the papers, the remaining authors either base their selection on a survey of the criteria found in previous literature, with no explicit justification for their final choice of criteria, or provide no obvious explanation for their selection. This different handling of criteria selection leads to dissimilar analysis setups and consequently to divergent results. A remedy to this problem could be provided by a systematic methodological framework for criteria selection. As previously discussed, some of the authors rely on LCSA for criteria selection as it provides a holistic approach to assess the sustainability of specific products [4]. We think that LCSA can serve as a basic framework for a systematic selection of evaluation criteria with consistently defined system boundaries. Thus, a “level playing field” would provide for more comparable evaluation setups, which can be customized to specific study problems by setting the weights accordingly. Similar aspects are discussed by Hannouf and Assefa [67], who summarize the features of MCDA which are useful for LCSA-based decision-analysis and vice versa.

The next section discusses the results obtained in papers with similar analyses in terms of alternatives and criteria and confirms that often dissimilar evaluation setups are the reason for diverging results.

6. Comparison of Results of Similar Analyses

Polatidis et al. [68] point out that “it looks like the literature lags behind in comparisons of applications of different MCDA techniques in the context of real data sets in renewable energy planning, with similar input data and related parameters”. Our review on MCDA analyses with similar evaluation setups in terms of alternatives and criteria offers the possibility to make such comparisons. This allows gaining a better understanding of the functionality of MCDA methods and the importance of the factors going into the analysis. Further, it is possible to detect generally observable ranking results.

The most meaningful way to compare the ranking results of different MCDA methods is to compare evaluations with the same data set of criteria and alternatives. Papers which apply several MCDA methods on the same data set most often find relatively consistent rankings [23,29,30,35,38]. Similar to such analyses, Lanjewar et al. [41] and Vahdani et al. [50] apply different MCDA methods on data sets already used and ranked in previous publications. In particular, Lanjewar et al. [41] apply their newly developed MCDA method GTMA on three examples, each based on problems defined in other reviewed papers, namely Zhou et al. [60], Safaei Mohamadabadi et al. [55], and Tzeng et al. [23]. Apart from some minor differences, their results are similar to the rankings found in the source papers.

Contrary to this outcome, Vahdani et al. [50] find almost opposite results to the source paper of Tzeng et al. [23]. Comparing the specific data in the papers it turns out that the criteria weights provided by experts in Vahdani et al. [50] are substantially different to the weights used by Tzeng et al. [23]. For instance, Tzeng et al. [23] attach a relatively high weight to air pollution, whereas the weight assumed by Vahdani et al. [50] for this criterion is one of the smallest. Also differently to Tzeng et al. [23], the noise criterion is weighted very low in their setting. This shows how sensitive MCDA tools can be to the subjective evaluation of criteria’s weights.

Other reviewed papers do not evaluate the same alternatives and criteria with different MCDA methods. Nevertheless, it is possible to compare the relative rankings obtained for specific alternatives, although they might not be rated against the same options or under the same set of criteria. The following general results are observed.

CNG and LPG are assessed rather positively for trucks or buses [23,30,32,44,50], and rather negatively for light vehicles [39,47,55]. Also diesel scores well for heavy vehicles like waste collection vehicles or buses [38,50]. Otherwise, diesel and gasoline are mostly rated in the middle or the lower third of the rankings [29,31,46,47,52,55,60].

Most papers rank ethanol fuels higher in comparison to other fuels [46,48] or consider them better suited for light vehicles [31,47,52,54,59,60], but less for buses or trucks [32,44]. Biodiesel, HVO and BTL [32,39,47,48,52,54,55,56], as well as BEV, HEV and PiHV, score relatively well [23,29,32,34,35,39,46,47,55,56].

Fuel cell vehicles are rated consistently into the middle or the lower third of the rankings [23,30,31,34,44,46,48,50,54]. Likewise, methanol is quite consistently low rated [23,30,31,34,50,60].

Exemptions to these generally observed results are mainly caused by two factors. Either the authors use different criteria or weight sets, or the regional background of the evaluation causes the results to differ.

The results of comparable papers concerned with hydrogen production reveal a similar pattern. With the exception of Heo et al. [12], the rankings are relatively similar [11,13,14]. Heo et al. [12] rank hydrogen production from fossil fuels like natural gas and coal high, while placing methods via electrolysis low. The results found by Chang et al. [11] are almost opposite to Heo et al. [12], although the countries for which the evaluations are made—Taiwan and Korea, respectively—appear relatively similar in economic aspects and the absence of natural resources. Chang et al. (2011) assess hydrogen produced via electrolysis from wind power as the most attractive option, whereas steam reforming of natural gas is placed in the midfield of the ranking. The reason for these striking differences seems to be that Heo et al. [12] select different evaluation criteria in comparison to Chang et al. [11]. As shown in Table 8, they include “New employment & economic growth”, “Resource & energy security/dependency”, “Capital & transition cost”, “Availability of refineries & processing infrastructure”, and the “Relationship to other industries”, i.e., technological and socio-economic side effects to other industries, into the analyses. None of these criteria is taken into account in the other hydrogen related papers. Their specific selection of evaluation criteria makes new technologies like wind power less attractive due to their high transition costs and their dependency on foreign technology. This example further demonstrates the importance of the criteria selection and how it affects the results of an MCDA analysis.

Concluding, our discussion of the results of comparable analyses allows to generalize the results and detect commonly preferable options. One may say that the preferable alternatives to conventional fuels for light vehicles are ethanol or a switch from vehicles with IC engines to electric drive vehicles. The preferable alternative fuel for heavy vehicles, like buses, is gaseous fuels. Further, our analysis provides evidence that evaluations with MCDA methods are highly dependent on the evaluation criteria (e.g., Maimoun et al. [38] and Heo et al. [12]) and weight settings (e.g., Aydın and Kahraman [44] and Vahdani et al. [50]). Also the context in which the analysis is embedded, in particular the regional background (e.g., Iran in Sehatpour et al. [31]) and the vehicle type assessed (e.g., light vehicles vs. buses), play a vital role. In this sense, variances in the results can often be traced back to specific factors and criteria going into the evaluation. In general, applying different MCDA methods on similar setups and contexts leads to similar results. Apparently, the MCDA method used for the analysis has less impact on the final outcome than the factors going into the analysis.

7. Conclusions and Outlook

A systematic review of 40 papers that apply MCDA methods on the evaluation of RTFV is conducted. This specific focus of our review and subsequent analysis allows us to contribute to the existing literature by comparing studies with similar evaluation setups, something that has not been done in comparable literature reviews. The following research questions are addressed: Which evaluation criteria are used most frequently? Which rationales do authors use to select the evaluation criteria and evaluated alternatives? Do related and comparable MCDA studies obtain the same result regardless of the MCDA method used and are there any specific reasons for striking differences in the ranking results? To answer these questions we systematically group the criteria used, examine the process applied to select alternatives and evaluation criteria and detect generally preferable options, based on the results of comparable studies.

A number of 41 distinct evaluation criteria can be traced by our literature review. These criteria are defined in order to reflect the different system boundaries envisaged. GHG emissions is the most common evaluation criterion. Final cost of fuel and safety are the most important economic and social criteria, respectively. Technical aspects like productivity and efficiency issues are not as often used as might be expected. The list of criteria established in this paper can serve as a pool for criteria selection as well as basis of discussion and source of inspiration for future research.

Our analysis revealed that, in most papers, authors rely on a variety of criteria with differently defined system boundaries, which makes the comparability of results more complicated. The choice of the evaluation alternatives and criteria seems, therefore, to lack a rigorous scientific justification and appears to be only partly based on an obvious and consistent selection process. In the papers providing a justification for the selection of the evaluation criteria, the LCSA framework is most often used as guidance in this process.

The results obtained with MCDA evaluations in papers with similar analyses are quite consistent, providing relevant information for decision makers. One may say that good alternatives to the conventional fuels are electric and electrified vehicles or ethanol in case of light vehicles, and gaseous fuels in case of heavy vehicles like buses. Major differences in the evaluation results can often be explained with significant differences in the evaluation context, the evaluation criteria considered and the relative weights attributed to them. Accordingly, the choice of the MCDA method seems less important than the specific factors going into the analysis.

In conclusion, the criteria selection is a determinant factor for the final outcome of the MCDA evaluation of RTFV. Despite its importance for the final evaluation results, the process of criteria selection partly suffers from a lack of scientific foundation and standardization. This limitation leads us to recommend a combination of MCDA and LCSA methods which could offer an interesting alternative for a methodological framework to select evaluation criteria. Meanwhile MCDA is already used to integrate the results of the three sustainability measures under LCSA to a more holistic sustainability result, as discussed for instance in Hannouf and Assefa [67] or Ekener et al. [69], more research is needed on how LCSA and MCDA can be systematically combined to obtain consistent and transparent MCDA problem setups with a standardized consideration of the aspect of sustainability.