Multicriteria Decision Making for Evaluating and Selecting Information Systems Projects: A Sustainability Perspective

Abstract

This paper develops a multicriteria analysis method for effectively making the selection decision on information systems (IS) projects for project management in organizations from a sustainability perspective. The triple bottom line principle of sustainability in organizations is carefully considered in formulating the IS project selection process as a multicriteria analysis problem. The subjectiveness of the decision-making process is modelled by linguistic variables approximated by fuzzy numbers. The positive and the negative ideal solution concepts are used to calculate the overall sustainability performance of individual IS projects in a comprehensive manner. A decision support system framework is then constructed with the use of the developed method for facilitating the IS project selection process. Such a system can provide organizations with an effective mechanism for comprehensively evaluating available IS projects from a sustainability perspective. An example is presented for demonstrating the flexibility and effectiveness of the proposed method in solving the IS project selection problem.

1. Introduction

Sustainability is concerned about meeting the needs of the present generation without compromising the ability of future generations to meet their own needs [1,2,3]. It is becoming increasingly important across the world. The popularity of sustainability in organizations is due to the rapid growth of emerging economies, the increasing expansion of the world population, the accelerated consumption of nonrenewable energy and the deteriorating situation of the world environment [2,4].

The increasing recognition of the importance of sustainability across the world motivates—and to some degree, forces—individual organizations to actively pursue sustainability development under various circumstances [2]. There are growing pressures on individual organizations worldwide to consider the environmental concerns of various stakeholders, to meet environmental standards and regulations and to reduce the environmental impact of the operations of individual organizations [5] while improving their market competitiveness and organizational performance [6]. To effectively handle such pressures and improve organizational sustainability, the evaluation and selection of appropriate information systems (IS) projects for development from the sustainability perspective is of critical importance to individual organizations [7,8].

Much research has been done on organizational sustainability through project management in the literature. There are, however, few studies that look at the sustainability issues in regard to information systems (IS) project management [6,9]. Existing studies that investigate sustainability development in project management focus on evaluating the environmental impact of individual projects, in particular construction and engineering projects [6]. Limited research has been conducted on evaluating the sustainability performance of IS projects [8,9] in the process of selecting specific IS projects for development in organizations. This creates an enormous gap in the literature as to how to incorporate sustainability assessment into the process of evaluating and selecting IS projects in organizational sustainability development.

Evaluating and selecting IS projects is important and complicated in modern organizations [8]. This is because industrial production, service provisioning and business administration are heavily dependent on the smooth operations of IS, which are expensive to develop, complex to use and difficult to maintain [3]. The availability of numerous IS projects, the increasing complexity of these projects and the pressure to make timely decisions in a dynamic environment further complicate the IS project evaluation and selection process [8,9,10].

Evaluating the sustainability performance of available IS projects is challenging. Multiple evaluation criteria are usually present, and subjective assessments are widely used. Much research has been done in developing appropriate multicriteria methods for assessing the performance of IS projects. Lee and Kim [10], for example, presented a comprehensive multicriteria method using the Delphi technique, analytic network process [11] and zero-one goal programming for solving the IS project selection problem. Wei et al. [12] proposed an integrated framework involving strategic objective analysis, system analysis and group decision-making for evaluating and selecting IS projects in organizations. Deng and Wibowo [13] developed a decision support system for facilitating the IS projects selection process in organizations. Yeh et al. [14] proposed a fuzzy multicriteria group decision making method for solving the IS project evaluation and selection problem. Dutra et al. [15] developed a probabilistic method for solving the IS project selection problem. The above studies demonstrate that the development of multicriteria decision-making methods for evaluating the sustainability performance of IS projects is of great significance. These studies, however, are not entirely satisfactory for addressing the subjectiveness and imprecision inherent in the evaluation process of IS projects. Furthermore, these methods do not specifically consider the sustainability performance of individual IS projects. As a result, the development of an ease-of-use method capable of addressing the above weaknesses is desirable.

This paper develops a multicriteria analysis method for effectively making the selection decision on IS projects for project management in organizations from a sustainability perspective. The triple bottom line principle of sustainability is carefully considered in formulating the IS project selection process as a multicriteria analysis problem. The subjectiveness of the decision-making process is modelled by linguistic variables approximated by fuzzy numbers. The positive and the negative ideal solution concepts are used to calculate the overall sustainability performance of each IS projects in all selection criteria and their associated sub-criteria. A decision support system framework is constructed with the use of the developed method for facilitating the IS project selection process. An example is presented for demonstrating the applicability of the developed method for solving the IS project evaluation and selection problem from the perspective of organizational sustainability.

In what follows, the IS project sustainability evaluation and selection process is formulated as a fuzzy multicriteria decision problem with a careful considering of the sustainability criteria and sub-criteria. A multicriteria decision-making method is developed for effectively solving such a problem, followed by the description of a decision support system framework for evaluating and selecting IS projects in organizations. Lastly, an example is presented for demonstrating the effectiveness of the approach in solving the IS project evaluation and selection problem from the perspective of sustainability.

2. Sustainability Performance Evaluation of IS Projects

The concept of sustainability is widely accepted as the one based on the integration of three dimensions of organizational performance in today’s dynamic environment with respect to the economic, environmental and social objectives, known as the triple bottom line [2,9,16]. To effectively achieve the sustainability objectives in an organization in a competitive and fast-changing environment, the incorporation of three sustainability dimensions into the IS project evaluation and selection process is not only desirable but necessary [8].

Much research has been done for evaluating the sustainability performance of individual IS projects in organizations from different perspectives including economic evaluation and environmental evaluation [6,8,17]. The economic evaluation of IS project selection focuses on assessing the economic benefits that an individual IS project brings to the organization and the society [8]. Such an evaluation has been the focus of IS project performance evaluation and selection for a few decades [6,17]. This leads to the development of specific economic models based on various financial criteria including the return on investment, the net present value, the internal rate of return, the discounted cash flow and the opportunity cost for the evaluation and selection of IS projects in a given situation [6,9,16]. This kind of evaluation is popular due to its ease of using the quantifiable monetary indicators that are capable of providing an overview of the economic feasibility of individual IS projects under evaluation and selection. The economic evaluation, however, is often criticized for neglecting the environmental and social perspectives of the sustainability assessment in IS project evaluation and selection in organizations [2].

The environmental evaluation concentrates on examining the energy consumption of IS projects under evaluation and selection and the consequent environmental impact of individual IS projects [17]. Such an evaluation assumes that well-designed IS projects are capable of effectively reducing the negative environmental impact of the operations of an organization and adequately eliminating the potential environmental hazards [4]. This leads to the development of specific methods for evaluating and selecting IS projects under various circumstances. Chen et al. [18], for example, assess the sustainability performance of IS projects using the environmental evaluation from the perspective of eco-efficiency, eco-equity and eco-effectiveness. Watson et al. [19] evaluated the IS project sustainability performance from the environmental perspective with respect to a comprehensive energy informatics framework. Such studies provide insightful information on evaluating the sustainability performance of IS projects from the environmental perspective. They, however, fail to consider the social and economic performance of IS projects in the IS project sustainability assessment.

The social dimension of IS projects is often underexamined in the project management literature [16]. This dimension, however, needs to be properly incorporated into the sustainability assessment in today’s dynamic environment [20,21]. Valdes-Vasquez and Klotz [20], for example, stated that a sustainable project must consider its social influence in relation to the safety, health, and education on end users, the community and other stakeholders involved. Modern organizations put great effort on being outstanding corporate citizens under the demanding pressure from the community on the organization. Such a pressure drives individual organizations to carefully consider their corporate social responsibilities when they decide to adopt a specific IS project. This is because the selection of a specific IS project in an organization can improve its sustainability performance, leading to the enhancement of the sustainability image and reputation of the organization [9].

Existing studies as above often adopted an unbalanced view of the triple bottom line when evaluating and selecting the sustainability performance of individual IS projects [9,17,21]. These studies often concentrated on evaluating the sustainability performance of projects from a certain perspective. This shows the need to incorporate all the three dimensions of sustainability including the environmental dimension, the social dimension and the economic dimension into the IS project evaluation and selection process in organizations [6,16].

With the discussion as above, three main criteria for evaluating available IS projects from the sustainability perspective can be identified including the economic sustainability (C₁), the environmental sustainability (C₂) and the social sustainability (C₃). Figure 1 presents the hierarchical structure of the IS project sustainability performance evaluation and selection problem.

The economic sustainability (C₁) of an IS project focuses on the profit maximization, the cost reduction and the quality improvement of individual IS projects, which are the traditional business imperatives [4,8]. It is measured by the direct financial benefits (C₁₁) and the indirect financial benefits (C₁₂) that an IS project has to an organization. The direct financial benefits (C₁₁) are related to the tangible profit due to the effective selection and adoption of IS projects in an organization. The indirect financial benefits (C₁₂) concerns with the new opportunities for business that arise from the adoption of specific IS projects in a specific situation.

Environmental sustainability (C₂) is concerned with the impact of a specific chosen IS project on the environment [4,5]. This is assessed by procurement (C₂₁), energy (C₂₂), and waste (C₂₃) that an IS project is related to after its implementation. The procurement (C₂₁) is related to the selection of suppliers and the sourcing of project materials for delivering the project in a sustainable manner. The energy (C₂₂) concerns with the compliance with the policies and regulations regarding the energy consumption for IS projects. The waste (C₂₃) focuses on the way that the waste is dealt with during the implementation of IS projects in the organization.

Social sustainability (C₃) focuses on the impact of a specific chosen IS project on the social responsibility of the organization in a given situation [8,21]. This is measured by the labor practices in the workplace (C₃₁), human rights (C₃₂), public acceptability (C₃₃) and the organization’s reputation (C₃₄). The labor practices in the workplace (C₃₁) are related to health and safety, training and education, values and ethics and organizational learning in the workplace. Human rights (C₃₂) reflect on organizations’ nondiscrimination and the freedom of the association’s culture. Public acceptability (C₃₃) refers to the attitude of the public toward the IS projects. The organization’s reputation (C₃₄) concerns with the level of satisfaction of key stakeholders regarding the IS projects in the organization.

With the use of sustainability criteria and sub-criteria identified above, each and every available IS project needs to be comprehensively evaluated for calculating the overall sustainability performance in all criteria and their associated sub-criteria in order to select the most appropriate IS project in a given situation. To solve this problem effectively, a fuzzy multicriteria method is proposed in the following section for facilitating the evaluation and selection of the most appropriate IS project from the sustainability perspective.

3. A New Method

Multicriteria analysis is effective in tackling the decision problem involving in the evaluation and selection of available alternatives in the presence of multiple and often conflicting criteria [22,23,24]. The multidimensional nature of the IS project sustainability evaluation process justifies the use of the multicriteria analysis methodology for solving the IS projects evaluation and selection problems from the sustainability perspective.

A typical IS projects evaluation and selection problem involves in several steps. First, all the IS project alternatives A_i (i = 1, 2, …, n) need to be identified, followed by the identification of sustainability evaluation criteria C_j (j = 1, 2, …, m) and their respective sub-criteria C_jk. Second, the performance rating of each IS project X_ij in regards to the sustainability criteria and sub-criteria needs to be obtained. Finally, the performance ratings and the criteria weights W_j need to be aggregated to determine the overall performance of every IS projects on which the most appropriate IS project can be selected.

Subjective assessments are always inherent in human decision-making [9]. In order to comprehensively model the imprecision and subjectiveness involved in the IS project evaluation and selection process, linguistic variables are used to represent the decision maker’s subjective assessment of the criteria weights and the performance ratings of alternative IS projects in assessing the overall performance of individual IS projects in a specific situation.

To ensure the efficiency of the computation process in evaluating and selecting specific IS projects, triangular fuzzy numbers are used to approximate these linguistic variables. A triangular fuzzy number is determined by three real numbers, denoted as (a, b, c), where b represents the most possible assessment value, and a and c are the lower and upper bounds, respectively, to represent the fuzziness of the assessment of the decision maker [11].

Table 1. Linguistic variables for evaluating performance ratings and criteria weights.

Linguistic Variables	Very Poor (VP)	Poor (P)	Fair (F)	Good (G)	Very Good (VG)
	Very Low (VL)	Low (L)	Medium (M)	High (H)	Very High (VH)
Membership Function	(0.0, 0.0, 0.3)	(0.1, 0.3, 0.5)	(0.3, 0.5, 0.7)	(0.5, 0.7, 0.9)	(0.7, 1.0, 1.0)

shows the linguistic variables and their approximate distribution for decision makers to measure the alternative performance rating of individual IS projects and the criteria weighting in the IS project evaluation process.

The proposed method starts with identifying the performance ratings of available IS projects with the use of the linguistic variable presented in Table 1. The fuzzy decision matrix is defined in (1),

$$X=\left[\begin{array}{llll}{x}_{11}& x_{12}& \dots & x_{1m}\\ x_{21}& x_{22}& \dots & x_{2m}\\ \dots & \dots & \dots & \dots \\ x_{n1}& x_{n2}& \dots & x_{nm}\end{array}\right]$$

(1)

where x_ij represents the subjective assessment of the decision maker with respect to the sustainability performance of alternative IS project A_i (i = 1, 2, …, n) in relation to the sustainability criteria C_j (j = 1, 2, …, m). It is usually given by the decision maker using the linguistic variables defined in Table 1 or aggregated from a lower-level decision matrix (2) for its associated sub-criteria.

If sub-criteria C_jk (k = 1, 2, …, p_j) is existent for criteria C_j, a lower-level decision matrix can be determined as in (2). In this matrix, y_jk represents the subjective assessment of the decision maker on the performance of alternative IS project A_i in relation to sub-criteria C_jk.

$$Y_{C_j}=\left[\begin{array}{llll}{y}_{11}& y_{21}& \dots & y_{n1}\\ y_{12}& y_{22}& \dots & y_{n2}\\ \dots & \dots & \dots & \dots \\ y_{1p_j}& y_{2p_j}& \dots & y_{n{p}_j}\end{array}\right]$$

(2)

In a similar manner, the decision maker can determine the weight vectors for the sustainability evaluation criteria C_j and its associated sub-criteria C_jk using the linguistic variables defined as in Table 1, as presented in (3) and (4),

W = (w₁, w₂, …, w_j)

(3)

W_j = (w_j1, w_j2, …, w_jk)

(4)

The decision vector X_ij in all the alternative IS projects A_i with respect to criteria C_j in (1) can be determined by

$$(x_{1j},x_{2j},\dots ,x_{nj})=\frac{W_j{Y}_{Cj}}{{\displaystyle \sum _{k=1}^{p_j}{w}_{jk}}}$$

(5)

The overall objective for solving IS project evaluation and selection problem from the sustainability perspective is to prioritize alternative IS projects in a comprehensive manner. Such a prioritization process is based on the assessment of the overall performance of each alternative IS project with respect to all the sustainability criteria and their respective sub-criteria. The process of determining the overall performance of each alternative IS project starts with computing the overall weighted performance matrix of the IS project alternatives with respect to multiple evaluation and selection criteria by multiplying the criteria weights w_j and the alternative performance rating x_ij as follows:

$$Z=\left[\begin{array}{llll}{w}_1{x}_{11}& w_2{x}_{12}& \dots & w_m{x}_{1m}\\ w_1{x}_{21}& w_2{x}_{22}& \dots & w_m{x}_{2m}\\ \dots & \dots & \dots & \dots \\ w_1{x}_{n1}& w_2{x}_{n2}& \dots & w_m{x}_{nm}\end{array}\right]$$

(6)

To optimize the computational process in fuzzy multicriteria decision making [14], the defuzzification method based on the geometric center of a fuzzy number [23] can be applied to (6), as determined by (7)

$$r_{ij}=\frac{{\displaystyle {\int }_{S_{ij}}^{}x{\mu }_{w_j{x}_{ij}}\left(x\right)dx}}{{\displaystyle {\int }_{S_{ij}}^{}{\mu }_{w_j{x}_{ij}}\left(x\right)dx}}$$

(7)

where S_ij is the support of fuzzy number w_jx_ij in (6).

For a triangular fuzzy number (a, b, c), (7) can be simplified as (8)

$$r_{ij}=\frac{a+b+c}{3}$$

(8)

A weighted performance matrix for all the alternative IS projects with respect to the evaluation and selection criteria and the associated sub-criteria in a crisp value format can then be obtained as follows:

$$R=\left[\begin{array}{llll}{r}_{11}& r_{12}& \dots & r_{1m}\\ r_{21}& r_{22}& \dots & r_{2m}\\ \dots & \dots & \dots & \dots \\ r_{n1}& r_{n2}& \dots & r_{nm}\end{array}\right]$$

(9)

To effectively prioritize all of the alternative IS projects, the positive-ideal solution and the negative-ideal solution concepts can be used [2]. The positive-ideal solution A⁺ represents the best possible results among the alternatives across all the sustainability selection criteria. The negative-ideal solution A⁻ is the worst possible results among the alternatives across all the sustainability selection criteria, determined as:

A⁺ = (r₁⁺, r₂⁺, …, r_m⁺), A⁻ = (r₁⁻, r₂⁻, …, r_m⁻)

(10)

where

r_j⁺ = max (r_1j, r_2j, …, r_nj), r_j⁻ = min (r_1j, r_2j, …, r_nj)

(11)

With the use of the Euclidean distance concept, the closeness between alternative A_i and the positive-ideal solution and the closeness between alternative A_i and the negative-ideal solution can be calculated using (10) and (11) respectively as follows:

$$\begin{array}{cc}{d}_i^{+}=\sqrt{{\displaystyle \sum _{j=1}^m{(r_j^{+}-r_{ij})}^2}};& d_i^{-}=\sqrt{{\displaystyle \sum _{j=1}^m{(r_{ij}-r_j{}^{-})}^2}}\end{array}$$

(12)

An IS project alternative is preferred if it is closer to the positive ideal solution and at the same time further away from the negative ideal solution [2]. With the adoption of such a principle, an overall performance index value for each alternative IS project A_i can be determined by:

$$P_i=\frac{d_i^{-}}{d_i^{+}+d_i^{-}},i=1,2,\dots ,n$$

(13)

The preferred alternative is the one that has the largest performance index value in a given situation.

With the discussion as above, the multicriteria decision method presented above can be summarized in an algorithmic format as follows:

• Step 1: Determine the fuzzy decision matrix as presented in (1).
• Step 2: Assess the performance ratings of alternative IS projects in relation to each evaluation criteria assessed by the decision maker, as presented in (2).
• Step 3: Determine the fuzzy weighting vector as presented in (3) and (4).
• Step 4: Generate the fuzzy weighted performance matrix using (6).
• Step 5: Transform the weighted fuzzy performance matrix into a crisp performance matrix as presented in (9).
• Step 6: Determine the positive-ideal and the negative-ideal solutions using (10) and (11).
• Step 7: Calculate the distance between each IS project and the positive-ideal solution and the negative-ideal solution using (12).
• Step 8: Determine the overall performance index value for each IS project using (13).
• Step 9: Rank all the IS projects in descending order of their performance index values.

4. A Decision Support System Framework

A decision support system is an interactive computer-based information system for solving semi-structured decision-making problems to improve the effectiveness of human decision making [14]. It provides decision makers with a mechanism to better understand the decision-making issues and the implication of their decisions by allowing them to exchange information interactively between the system and themselves. To help the decision maker solve the IS project evaluation and selection problem from the sustainability perspective in a systematic and user-friendly manner, a multicriteria decision support system framework is introduced in this section.

The proposed decision support system framework consists of four main components, including (a) the database management subsystem, (b) the knowledge base management subsystem, (c) the model library management subsystem and (d) the user interface management subsystem. The database management subsystem contains a relational database for providing data retrieval, updating and appending. This database is used to collect a variety of data and information from various applications. The knowledge base management subsystem contains problem-specific rules and facts in the form of “IF-THEN” rules related to each decision problem. The information is collected from the previous cases. The model library management subsystem provides an environment for storing, retrieving, and manipulating models with respect to the characteristics of a specific IS project evaluation and selection situation. Such a system links decision makers with appropriate methods and models, including the method developed above.

The model library management subsystem collects the input information from the database management subsystem for model execution, and in return, sends the results of the model execution back to the database management subsystem for storage [14]. The user interface management subsystem provides the decision makers with ways to (a) interact with the decision support system, (b) access to the database, the model library, and the knowledge base, (c) store information, for example, performance characteristics of IS projects, (d) analyze and evaluate alternative decisions and (e) display the outputs.

The use of the proposed decision support system in the evaluation and selection of IS projects involves in five phases, including (a) problem definition, (b) alternative, criteria and sub-criteria identification, (c) alternative performance rating, (d) criteria and sub-criteria weighting and (e) project selection as shown in Figure 2. Problem definition is about the identification of the requirements of the decision maker in the decision-making process for IS project evaluation and selection. Alternative, criteria and sub-criteria identification is related to the formulation of the IS project evaluation and selection problem with the description of the selection criteria, sub-criteria and available IS project alternatives. Alternative performance ratings are about assessing the sustainability performance of each available alternative with respect to a specific criterion or its sub-criteria by the decision maker. Criteria weighting is related to the decision maker determining the relative importance of the evaluation criteria and their associated sub-criteria in the evaluation process. Project selection involves applying the fuzzy multicriteria decision-making method for evaluating and selecting the most appropriate IS project based on the overall sustainability performance of all the available alternatives. This leads to the selection of the most suitable IS project for development in organizations.

5. An Example

To demonstrate the effectiveness and applicability of the above proposed multicriteria method in solving the IS project evaluation and selection problem from the perspective of sustainability in an organization, this section presents an example of evaluating and selecting the most suitable IS projects for an organization from the sustainability perspective.

A leading financial service organization plans to implement an IS project for achieving the competitive advantage within the industry. There are four alternative IS projects for evaluation in which the organization needs to select the most appropriate one. This organization has been trying to integrate the sustainability principles in its operations due to the pressure imposed by the government and other stakeholders. Sustainable development is therefore incorporated in the mission statement and the strategic objectives of the organization. As a result, the organization wants to ensure that such principles are followed when evaluating and selecting IS projects in this situation.

The IS project evaluation and selection process starts with determining the performance rating of each IS project with respect to the sustainability evaluation criteria and the relative importance of these criteria with the use of the linguistic variables as defined in Table 1. A Delphi process is used to determine the performance rating of each IS project with respect to the sustainability evaluation criteria and the relative importance of the IS project sustainability evaluation criteria. Such a process helps reach a consensus among several decision makers in the organization about the performance ratings of individual IS projects and the criteria weights used for evaluating the sustainability performance of IS projects. This consensus process facilitates the acceptance of the decision-making outcome among the key stakeholders in this situation.

Table 2. Performance assessment results for each sub-criterion.

Criteria	Project A1	Project A2	Project A3	Project A4
C11	VG	F	P	F
C12	P	G	VG	P
C21	F	VG	P	P
C22	F	G	G	G
C23	VG	F	F	F
C31	G	P	G	P
C32	G	VP	VG	VG
C33	P	F	G	VG
C34	G	F	VG	G

Legend: VG: Very Good; G: Good; F: Fair; P: Poor; VP: Very Poor.

shows the assessment of four alternative IS projects with respect to each criterion.

The relative importance of the IS project sustainability evaluation criteria and their associated sub-criteria is determined by applying the linguistic variable Importance shown as in Table 1.

Table 3. Criteria and sub-criteria weights for IS projects sustainability performance evaluation.

Criteria	Criteria Weights	Fuzzy Numbers
C1	H	(0.5, 0.7, 0.9)
C11	VH	(0.7, 1.0, 1.0)
C12	L	(0.1, 0.3, 0.5)
C2	M	(0.3, 0.5, 0.7)
C21	H	(0.5, 0.7, 0.9)
C22	M	(0.3, 0.5, 0.7)
C23	VH	(0.7, 1.0, 1.0)
C3	VH	(0.7, 1.0, 1.0)
C31	M	(0.3, 0.5, 0.7)
C32	VH	(0.7, 1.0, 1.0)
C33	H	(0.5, 0.7, 0.9)
C34	M	(0.3, 0.5, 0.7)

Legend: VH: Very High; H: High; M: Medium; L: Low; VL: Very Low.

shows the weightings of the sustainability criteria and their associated sub-criteria for the IS project sustainability evaluation and selection problem.

To construct the fuzzy performance matrix for all the alternatives with respect to multiple sustainability evaluation and selection criteria as in (1), lower-level fuzzy performance matrices of all the alternative IS projects with respect to the sub-criteria determined from Table 2 are aggregated with respect to each criterion with respect to the weights of the sub-criteria in Table 3 using (5). To facilitate the understanding of the calculation process for determining the performance matrix, the performance rating of IS Project A₁ with respect to the selection criterion C₁ is calculated below as an example.

$$X_{11}=\frac{w11y11+w12y12}{w11+w12}=\frac{\left(0.7,1.0,1.0\right)\times \left(0.7,1.0,1.0\right)+\left(0.1,0.3,0.5\right)\times \left(0.1,0.3,0.5\right)}{\left(0.7,1.0,1.0\right)+\left(0.1,0.3,0.5\right)}=(0.33,0.84,1.56)$$

Following the same process as above, the fuzzy performance matrix for all the IS project alternatives with respect to the sustainability criteria can be determined.

Table 4. Fuzzy decision matrix for IS projects sustainability performance evaluation.

	C1	C2	C3
Project A1	(0.33, 0.84, 1.56)	(0.28, 0.73, 1.41)	(0.21, 0.60, 1.45)
Project A2	(0.17, 0.55, 1.44)	(0.27, 0.70, 1.49)	(0.08, 0.28, 0.98)
Project A3	(0.09, 0.46, 1.25)	(0.16, 0.48, 1.19)	(0.33, 0.87, 1.74)
Project A4	(0.15, 0.45, 1.19)	(0.16, 0.48, 1.19)	(0.31, 0.81, 1.60)

shows the aggregated fuzzy performance matrix of all the alternative IS projects with respect to the IS project sustainability evaluation and selection criteria.

The overall weighted IS project sustainability performance matrix of all the alternatives with respect to all the evaluation and selection criteria can then be calculated using (6). Such a weighted fuzzy performance matrix can be converted into a crisp performance matrix by (8). The results are shown in

Table 5. The weighted performance matrix in crisp numbers.

	C1	C2	C3
Project A1	0.72	0.48	0.73
Project A2	0.59	0.49	0.44
Project A3	0.50	0.37	0.95
Project A4	0.49	0.37	0.88

. To facilitate the understanding of the calculation process, the weighted performance rating of IS Project A₁ with respect to the evaluation and selection criterion C₁ is calculated as an example as follows.

z₁₁ = w₁ × X₁₁ = (0.5, 0.7, 0.9) × (0.33, 0.84, 1.56) = (0.17, 0.59, 1.40)

The weighted fuzzy performance rating as above can then be converted into a crisp value based on (8) as 0.72. Following the same process, the weighted performance matrix of all the IS projects in crisp numbers can be determined as in Table 5.

To calculate the overall performance index value for each IS project, (9) to (13) are used.

Table 6. Performance index and ranking for IS projects.

Alternatives	Distance		Performance Index Value	Rank
	A+	A−	Pi
Project A1	0.22	0.39	0.64	2
Project A2	0.53	0.16	0.23	4
Project A3	0.25	0.51	0.67	1
Project A4	0.27	0.44	0.62	3

shows the results. Alternative IS project A₃ is the most appropriate choice under evaluation as it has the largest performance index value.

6. Discussion

Evaluating the performance of alternative IS projects from a sustainability perspective is complex and challenging as it involves multiple evaluation criteria in the presence of subjective and imprecise assessments in a given situation. The above example has demonstrated the applicability of the proposed multicriteria decision-making method for adequately evaluating and selecting IS projects from the sustainability perspective. Based on the identified sustainability criteria and sub-criteria in Section 2 as shown in Figure 1, all available IS projects can be comprehensively evaluated, and their overall performance across all the sustainability evaluation criteria can be determined. This leads to the selection of the most appropriate IS project from the sustainability perspective in a given situation.

This study makes a major contribution to IS project sustainability performance evaluation research from both the theoretical and the practical perspectives. IS projects perform an important role in the sustainability development of an organization in modern society [8]. Limited research, however, has been conducted on evaluating the sustainability performance of IS projects in the process of selecting specific IS projects for development in organizations. Theoretically this study fills this gap by proposing an effective method with the use of the triple bottom line principle of sustainability for assessing the sustainability performance of IS projects.

This study offers an effective method for assisting decision makers in the organization in their evaluation and selection of the most appropriate IS project for achieving organizational sustainability objectives. The proposed method effectively incorporates three sustainability dimensions into IS project evaluation and selection process while adequately handling the subjectiveness and imprecision in the IS project performance evaluation and selection process. At the same time, this approach reduces the cognitive demands of the evaluation process on the decision maker.

7. Conclusions

This paper has presented a multicriteria analysis method for effectively evaluating the sustainability performance of IS projects under uncertainty in an organization. The triple bottom line principle of sustainability is carefully considered in formulating the IS project selection process as a multicriteria decision making problem. The subjective assessments of the decision maker in the selection process are represented by linguistic variables approximated by fuzzy numbers. The geometric center-based defuzzification method is used for transforming the weighting fuzzy performance matrix into the crisp performance matrix on which the concept of the positive-ideal solution and the negative-ideal solution is applied for calculating the overall sustainability performance of individual IS projects across all the selection criteria and their associated sub-criteria. The developed method is then incorporated into a decision support system framework for facilitating the project evaluation and selection process. Using an example, the proposed method has demonstrated various advantages for adequately dealing with the problem of evaluating the sustainability performance of alternative IS projects in an organization including the capability to adequately handle the multiple and usually conflicting sustainability criteria and the ability to deal with the subjectiveness and imprecision inherent in the IS projects performance evaluation process. The method is found to be effective and efficient due to the comprehensibility of its underlying concepts and the straightforward computation process.

There are several limitations in this study. One is to do with the inclusion of only one decision maker in the IS project performance evaluation process. Another is about the need for considering specific characteristics of individual IS projects when evaluating the sustainability performance of IS projects. Future research can be carried out to better address these two issues through formulating such a decision problem as a group decision-making problem that considers specific characteristics of an individual IS project to address its sustainability performance evaluation problem.