Prioritizing Indicators for Sustainability Assessment in Manufacturing Process: An Integrated Approach

Abstract

Sustainable manufacturing has renewed attention among researchers to address various sustainability challenges in manufacturing industries. Sustainability assessments of manufacturing organizations help minimize the negative environmental impact and enhance reputation among public and regulatory agencies. To assess the sustainability of the manufacturing process; it is indispensable to investigate the structured set of triple bottom line (3BL) indicators. Moreover, there is no comprehensive and structured set of 3BL indicators that can effectively assess the sustainability of any organization’s manufacturing process. This research aims to identify and prioritize experts’ consensus structured set of 3BL indicators. The 3BL indicators were identified through an open-ended questionnaire. The prioritization was performed through the Best-Worst Scaling (BWS) approach. Further, Multi-Criteria Decision Analysis (MCDA) method was utilized to draw the consensus ranking of sustainability indicators in manufacturing. The findings indicated that the release of greenhouse/harmful gas is the best indicator in the perspective of environmental criteria followed by the rate of contribution to society and operational cost are the most important critical indicator in the case of social and economic sustainability criteria. The outcome of the present study will facilitate researchers and practitioners in developing suitable readiness and operational plans for the sustainability assessment of the manufacturing process.

1. Introduction

Rapidly growing manufacturing companies around the globe have contributed to improved quality of life and adversely affected the environment [1,2]. Manufacturing organizations release a considerable amount of air pollutants that affect the environment and well-being of life forms [3,4]. Organizations have received incremental pressure from governments and other non-government organizations to take action and minimize the environmental damage and enhance safety [5,6]. Sustaining a process can reduce the negative environmental effect of an organization [7,8]. Considering these perspectives, several organizations focus on redesigning manufacturing processes and products to make them highly sustainable [5]. However, changing current operations based on the redesigning processes is expected to have various obstacles from a technical standpoint. Redesigning the manufacturing process alone does not guarantee sustainability from the perspective of the triple bottom line (3BL). However, sustainable manufacturing processes and products can significantly reduce emissions and harmful gas, limiting the negative impacts on the environment, waste generation and conserving natural resources and energies [9]. Therefore, it is crucial to evaluate the sustainability of the manufacturing process and develop amenable strategies for sustainable development.

An evaluation of the 6Rs approach (reduce, reuse, recycle, recovery, redesign, remanufacture) is recommended to accommodate the sustainability of a manufacturing process. However, the sustainability of technologies and products can be evaluated through additional methods such as the social life cycle assessment (SLCA) and techno-economic assessment (TEA) [10,11]. Further, corporate sustainability at the company level can be assessed through various global standards, including social accountability 8000 (SA 8000), ISO 4000 series and global reporting initiative (GRI) sustainability guidelines [12,13,14]. There is no specific method available in the literature to assess the sustainability of a manufacturing process. No consensus list of sustainability indicators is available that should be used for assessment purposes [14,15]. Thus, there is an opportunity to develop a consensus list of sustainability indicators that consider the 3BL approach of sustainability dimensions (i.e., environmental, social and economic). However, the identification and selection of a consensus list of sustainability indicators require a systematic approach and method [16]. While the identification of sustainability indicators itself/alone does not guide managers to choose the right indicators based on their importance. Thus, the prioritization of those indicators based on their importance is needed. Therefore, there is also a need to search for an appropriate method that systematically evaluates and prioritizes the consensus list of sustainability indicators in 3BL perspectives. An in-depth analysis of the 3BL dimensions of sustainability indicators and existing gaps within the sustainability evaluation of a manufacturing process was performed [14,17]. Past literature reveals that a comprehensive and complete set of sustainability indicators in 3BL perspectives for evaluating a process does not exist. The available set of sustainability indicators was determined to be incomplete, lack a holistic view, lack interlinking with each other and need more focus and detailed evaluation [14]. There is also an absence of environmental and social sustainability indicators and assessments using an appropriate number of economic sustainability indicators. Moreover, no studies prioritized the sustainability indicators that assess the manufacturing process in organizations.

This study aims to examine a structured set of sustainability indicators and evaluate their consensus ranking to assess various manufacturing processes in Asian organizations. This research is the first to develop a comprehensive and structured set of 3BL sustainability indicators for the sustainability assessment of a manufacturing process. A consensus prioritization of sustainability indicators can lay the foundation for the harmonization of sustainability assessment of the manufacturing process within a manufacturing environment. This study contributes to the development of an integrated mixed-method using qualitative and quantitative tools, which can deal with the list of sustainability indicators in a structured way. In this context, the list of 3BL sustainability indicators has been identified by piloting an open-ended questionnaire among a group of experts and prioritizing them by utilizing the Best-Worst Scaling (BWS) approach. Furthermore, Multi-Criteria Decision Analysis (MCDA) method has been applied to draw the consensus ranking of those sustainability indicators. The findings of this study will facilitate the researchers and practitioners in developing suitable readiness and operational plans for the sustainability assessment of the manufacturing process through a detailed understanding of the structured 3BL sets of indicators. This study will also help decision-makers and policymakers to better understand the set of 3BL indicators and their prioritization to assess sustainability in the manufacturing environment.

The rest of the article’s structure is as follows: the literature review about the triple bottom line, sustainable manufacturing and sustainability indicators in the manufacturing perspective is discussed next. Section 3 deals with the research methodology applied to prioritize sustainability indicators. The results are presented in Section 4. The detailed discussion followed by managerial implications, challenges, limitations and direction for future research is provided in Section 5. Finally, the conclusion is presented in Section 6.

2. Literature Review

This section contains a summary of the literature available on Triple bottom line, Sustainable manufacturing, Review of sustainability indicators. To find out sustainability indicators in the context of manufacturing, the articles from year to year have been considered for analysis in this study. These research articles have been explored using reputed electronic databases like; Web of Science, Scopus and EBSCO so that all relevant articles must be included in the study. To identify the articles for the present study, keywords including sustainability, sustainable manufacturing, TBL and sustainability indicators were included. Articles other than those in the English language, gray research literature such as unreputed conferences were excluded from the study.

2.1. Triple Bottom Line

The triple bottom line (3BL) is an accounting framework that has three different viewpoints such as financial, social and environmental [18,19,20]. Several industries have implemented the 3BL framework to assess their sustainability performance in the broader perspective to create higher business value [21,22,23]. This framework (3BL) is different from the traditional framework as it integrates social and environmental measures that are difficult to assign appropriate means of measurement [24,25]. These 3BL dimensions are also known as 3Ps (i.e., Profit, People and Planet) [15,25]. The phrase 3Ps to describe the 3BL and the sustainability goal was coined by John Elkington in 1994 [26]. Profit, the economic bottom line: The profit or financial bottom line deals with the economic value created by the organization after deducting the cost of all inputs, including the cost of the capital tied up. The people, social equity, or human capital bottom line pertains to fair and beneficial business practices toward labor and the community and region in which a corporation conducts its business.

The planet, environmental bottom line, or natural capital bottom line refers to sustainable environmental practices. A TBL company endeavors to benefit the natural order as much as possible or at the least does not harm and minimize the environmental impact.

2.2. Sustainable Manufacturing

Sustainable manufacturing (SM) can be defined as the production of items with low or negligible environmental emissions, good resources conservation ability and low cost [27]. It mainly acclaims an industrial establishment that reduces negative environmental impact, energy consumption, waste and improves sustainability dynamics [28]. “SM is the integration of systems and processes capable to produce high-quality products and services by utilizing less and more sustainable resources such as energy and material, being safer for societies, employees, consumers, stakeholders and being able to mitigate social and environmental impacts throughout its whole life cycle” [29]. Reducing negative environmental impact without compromising cost-effective quality production has now become a key duty of manufacturing firms. SM has been applauded since its evolution for the substantial benefits at societal, financial and environmental level indicators with the main objective to achieve a zero-carbon society [30]. SM has several benefits such as cost reduction by improving resource efficiency and regulatory compliance, new market access, better brand reputation, minimum labor turnover and long-term business approach through developing financial and capital access [31,32,33].

The authors of [31] define the scope of SM in the following four different areas with its concerning discipline:

• Manufacturing technologies (i.e., how the products are manufactured in the industries) with a focus on equipment and process (i.e., use of machine tool, equipment, facilities); the associated discipline includes operations management, production engineering, factory planning.
• Product life cycle (i.e., what product or services to be produced) with a focus on design; the associated discipline is engineering design.
• Networks (i.e., value creation) focus on manufacturing industries networks; the associated discipline includes knowledge management and business economics.
• Global impacts (transition mechanism towards SM) focus on work related to impacts on the world, including environment, economy and society.

Based on the above discussion, the concept and strategy of SM can be easily understood by practitioners, researchers and decision-makers. However, the turn from the traditional manufacturing process to SM is not an easy and smooth journey without the assimilation of triple-bottom-line (3BL) sustainability indicators [24,25]. Therefore, a comprehensive and structured set of sustainability indicators must be identified. The 3BL indicators help to assess the sustainability level of the process that entails whether the current process is sustainable or not [15].

2.3. Review of Sustainability Indicators

This section discusses the sustainability indicators that are used to assess the manufacturing process. The list of sustainability indicators was identified in previous studies [14,15,17,34,35,36,37,38]. Many authors have discussed the sustainability indicators to assess the manufacturing process, such as [39] suggested sustainability indicators such as 3R’s (reduce, recycle and reuse), Hazard material consumption and biodiversity factors. Ref. [40] identified several costs such as equipment, service, inventory, stock and transportation to evaluate economic sustainability. Similarly, Ref. [19] utilized indicators such as product life cycle analysis, waste treatment, waste segregation, employee salary, labor availability and skilled labor for evaluating the environmental and operational performance in Brazilian automotive companies. Ref. [35] developed a modeling and simulation-based life cycle evaluation for sustainable manufacturing using several indicators such as the 3 R’s system, Energy consumption, Turnover, Absenteeism, Service cost, Material cost and Return on investment. Further, Ref. [36] Assessed environmental emissions through sustainability indicators that include: Material consumption, Water consumption, Air emission and energy consumption for sawmilling activity in Malaysia. Ref. [37] improved wine production’s environmental performances by analyzing a life cycle assessment using the 3 R’s system, Material consumption, Water consumption, Air emission and energy consumption sustainability indicators. Ref. [41] identified the different economic indicators such as cycle time, changeover time, uptime, Lead time, value-added time and cost for economic sustainability assessment. Ref. [14] grouped indicators in triple bottom line perspectives and assessed the manufacturing process’s sustainability. Ref. [17] reviewed the literature on sustainability indicators and discussed the set of sustainability indicators. Ref. [15] considered different indicators such as Operational cost, Effective cost, water/oil consumption, energy consumption, training opportunity to employees, Accident rate and Waste segregation for the sustainability assessment of Indian automotive components manufacturing organization. The summary of sustainability indicators in triple bottom line perspectives identified in the literature review is presented in

Table 1. Sustainability indicators from the literature.

Dimensions	Sustainability Indicators	Author	Year
Environmental	3 Rs (Reduce, Reuse, Recycle) culture	[39]	2009
		[35]	2014
		[37]	2016
	Product life cycle analysis	[19]	2013
		[14]	2017
	RES consumption	[19]	2013
		[36]	2015
	Hazard material consumption	[39]	2009
		[38]	2017
	Water consumption	[34]	2013
		[36]	2015
		[37]	2016
	Energy consumption	[35]	2014
		[38]	2017
		[15]	2020
	Waste treatment	[19]	2013
		[38]	2017
		[17]	2019
	Biodiversity	[39]	2009
		[4]	2013
	Fossil fuel consumption	[34]	2013
		[37]	2016
		[17]	2019
	Waste segregation	[19]	2013
		[36]	2015
		[15]	2020
	Air emission	[36]	2015
		[38]	2017
	CFC emission	[38]	2017
		[17]	2019
	Biological oxygen demand	[4]	2013
		[17]	2019
	GHG emission	[4]	2013
		[38]	2017
		[17]	2019
Social	Accident rate	[14]	2017
		[15]	2020
	Employee salary	[19]	2013
		[14]	2017
	Turnover	[35]	2014
		[14]	2017
	Employee satisfaction	[14]	2017
		[15]	2020
	Absenteeism	[35]	2014
		[14]	2017
	Labor availability	[19]	2013
		[15]	2020
	Skilled labor	[19]	2013
		[15]	2020
	Ergonomics	[35]	2014
		[14]	2017
	Community development	[14]	2017
		[15]	2020
	Employee training hours	[19]	2013
	Noise level	[19]	2013
		[14]	2017
Economic	Equipment cost	[40]	2009
		[34]	2013
	Service cost	[40]	2009
		[35]	2014
	Material cost	[7]	2012
		[35]	2014
	Return on investment	[34]	2013
		[35]	2014
	Operation cost	[14]	2017
		[17]	2019
		[15]	2020
	Inventory and stock cost	[40]	2009
		[14]	2017
	Cycle time	[21]	2008
		[7]	2012
		[42]	2020
		[15]	2020
	Overall equipment effectiveness	[21]	2008
		[7]	2012
		[42]	2020
	Transportation efficiency	[40]	2009
		[19]	2013
	Value added time	[21]	2008
		[19]	2013
		[42]	2020

.

2.4. Review of Analysis Approaches Used for the Selection and Prioritization of Sustainability Indicators in Manufacturing Environment

This section discusses the various analysis approaches used by the researchers for the selection and prioritization of sustainability indicators in the manufacturing environment. Many researchers have applied the multi-criteria decision-making approaches for the selection and prioritization of sustainability indicators, such as Ref. [43] have evaluated key performance indicators (KPIs) for the adoption of sustainability practices in footwear Supply Chains. The authors applied the best-worst method to prioritize the key performance indicators. Ref. [44] have identified and ranked the environmental sustainability indicators using Pareto analysis cum best-worst method in the manufacturing sector. Ref. [45] have analyzed the green innovation practices for the assessment of sustainability performance in a Chinese manufacturing industry. The authors applied fuzzy AHP and TOPSIS approaches to prioritize green innovation practices. Ref. [46] have evaluated the right welding process for the welding of two metal plates. The authors evaluated the welding process based on a multi-dimensional sustainability assessment model. This study used integrated COPRAS, TOPSIS and GRA methods to select the welding process. Ref. [47] have assessed the sustainability of manufacturing operations based on key performance indicators. A TOPSIS method is applied for the consideration of a number of KPIs related to economic, social and environmental sustainability. Ref. [48] have developed a decision support method for the sustainability and productivity assessment of machining process and operation plans. The authors used simulation and optimization techniques to select the optimized alternatives. Further, sensitivity analysis was performed to determine the sustainability parameters. Ref. [49] have developed a general model for the sustainability assessment of manufacturing processes. The general model was developed based on sustainability indicators. The selection of relevant indicators, their quantification and ranking of alternatives were performed through different MCDM methods including AHP, GRA, COPRAS and ELECTRE. Ref. [50] have proposed the process to evaluate the life cycle sustainability. The authors used an indicator-based approach to evaluate the sustainability of petroleum refinery industry projects. The alternative of the project was prioritized through the fuzzy AHP method. Ref. [51] have presented seven sustainability indicators to assess the potential oven investment. The study used fuzzy set theory and monte Carlo simulation to evaluate the sustainability of improvement projects. Ref. [52] evaluated the sustainability performance of a manufacturing company. The authors used the TOPSIS method to evaluate the performance of the concerned manufacturing company.

Many researchers have discussed the sustainability indicators, few of them have been categorized into different categories such as economic, social and environmental. Further, few researchers have evaluated the sustainability indicators and prioritized them using different MCDM methods. The findings of the literature review revealed that no researchers have provided a structured and complete set of sustainability indicators that can help to assess the manufacturing process. The key reason is inappropriate identification and selection strategy applied by them to examine sustainability indicators. A consensus set of sustainability indicators helps assess the sustainability of a manufacturing process whereas the incomplete set cannot evaluate the manufacturing process properly. Furthermore, poor selection strategy cannot find the consensus list of sustainability indicators and thus, cannot provide the right guidelines to prioritize them. To overcome this problem, this study identified a list of empirically tested sustainability indicators through an open-ended survey from a group of experts. Thereafter, a mixed methodology, integrated with qualitative and quantitative approaches are utilized through a Delphi study and multi-criteria decision analysis (MCDA) to evaluate the sustainability indicators.

3. Research Methodology

The research objective requires a structured methodological approach that consists of (1) gathering and defining information related to sustainability indicators for the manufacturing process and (2) prioritizing the indicators to assess manufacturing process sustainability. In this study, we have utilized the Delphi technique with an integrated MCDA approach to identify and prioritize the sustainability indicators. Delphi technique is the most suitable qualitative approach to collect data from a group of experts working in the relevant domain rather than other data collection strategies for the survey. However, the application of mixed-method (i.e., integration of qualitative and quantitative approaches) provides better insights and synthesize findings than utilization of a single approach in the case of survey. For quantitative decision-making, MCDA techniques are more popular as they give efficient outcomes [8,29,53,54]. Previous research studies revealed that the integration of Delphi and MCDA approaches can resolve issues related to decision-making in complex and uncertain situations [55,56]. The Delphi technique is used to gather relevant data from a group survey based on experts’ experiences. This technique pools the experts’ talents to provide structured feedback [53,57]. The feedback received from group experts helps researchers develop a questionnaire for the next round [54,58]. According to [59,60], the Delphi technique is also helpful for sustainability indicator selection and prioritization for complex scenarios. The indicator selection through the Delphi approach utilizes a qualitative survey. The survey methodology contributes greater efficiency to the quantitative method such as MCDA [61,62]. MCDA is a methodology used for making decisions when multiple criteria/variables or objectives need to be considered together to select or rank among the alternatives being evaluated [16,63]. The integration of the Delphi approach and MCDA has widely been used in sustainability-related research [16,64,65,66].

The present study utilized the Delphi technique to conduct a two-round survey to select and prioritize sustainability indicators of the manufacturing process. The first-round survey consisted of an open-type questionnaire about leading sustainability indicators involved in evaluating a manufacturing process; whereas the second round consisted of a closed-type survey that aimed to prioritize the selected indicators. The structured questionnaires were developed using the Qualtrics Software 2018 [67] version and distributed among related area experts via electronic mail. A set of questionnaires used in this study can be provided by the corresponding authors on request. The panel of experts was carefully chosen based on their qualifications and experience in the research field. The selected panel experts were clustered into three groups, including experts from manufacturing industries, certification bodies and academia/research centers. Based on the literature survey, a panel of a minimum of 10 anonymous experts can provide effective feedback for a Delphi study [58]. In this study, we utilized a group of 261 experts (121 from manufacturing industries, 87 from certification bodies and 53 from academicians/researchers) from various Asian countries. The methodological process is discussed in the next section and the research steps are illustrated in Figure 1.

3.1. Delphi Study First Round

The first round of study consists of open questions related to identifying sustainability indicators. The questionnaire asked group experts to brainstorm the list of leading indicators for evaluating the sustainability of the manufacturing process. The response rate was 75% in the first round with 196 complete responses (83 from manufacturing industries, 64 from certification bodies and 49 from academician/researchers). The experts represented 15 different countries from Asia, which indicates a broad representation.

Further, 72.33% hold a doctoral degree, indicating sufficient knowledge in the research field. The responses received were analyzed by open coding through NVivo 15 software [68]. Open coding reduces the difficulties in responses duplicity and handles qualitative data for analysis purposes. The final responses received from the group of experts were then merged with the results of the detailed literature reviews performed by [14,17]. To select the prominent indicators, a simple questionnaire was formed and asked to provide the panel experts input on the Likert scale of 1 to 5, where 1 represents “not important at all” and 5 represents “most important”. The authors analyzed the score of each indicator in each group for final selection. The average score of finalized indicators was more than 4.6 and the list of finalized indicators is presented in

Table 2. List of sustainability indicators used for manufacturing process assessment.

Environmental Sustainability Indicators	Social Sustainability Indicators	Economic Sustainability Indicators
Process coolant/oil consumption	Accident rate	Operational cost
Electricity consumption	Time-weighted average to record noise exposure	Labor cost
Raw material consumption	Absenteeism ratio	Management cost
Energy consumption per unit	Gender ratio	Facilities and depreciation cost
Greenhouse/harmful gas release	Employee turnover ratio	Effective cost
Toxic discharge to water	Training opportunity for employees	Stock cost
Reuse/recycle raw material ratio	Employee satisfaction rate	Takt cost
Waste segregation percentage	Post-parental leave retention	Inventory holding cost
Net green area impact	Contribution to society rate	Cycle time
Net CO2 emission impact	Local business support index	Changeover time
Net solid waste generation	National production rate	Uptime
Net water footprint	Gender salary ratio	Level of work in process inventory
Scrap rate	Volunteer sustainability initiatives ratio	Overall equipment effectiveness
	Staff incentives/commission/benefits	Machine availability
	Staff salary level	Machine performance
		The acceptance rate of product
		Value-added time ratio
		Value-added cost ratio
		Value-added time
		Value-added cost
		Total productive maintenance ratio
		Return on investment on innovation
		Transportation efficiency ratio

. To ensure the consistency of the questionnaire, the reliability test was performed through SPSS 20 software package. The Cronbach’s alpha yield was determined to be 0.831, indicating a relatively high consistency of the questionnaire. These responses provided a set of comprehensive sustainability indicators that were used as input data for the next round of the study. A detailed description of each of the 3BL indicators was provided to group experts in the second-round study. During the first round of the study, the experts were asked to give a preference of evaluation tool (i.e., which evaluation tool they prefer to assess the sustainability of the manufacturing process). The options included a single index, multiple indices or both.

3.2. Delphi Study Second Round

In this round, the prioritization of the selected sustainability indicators was performed. The MCDA technique was chosen to prioritize the indicators. “The main objective of the MCDA technique is to study the decision problems in which several points of view must be taken into consideration” [69,70,71]. The decision-making problem for this research involves more than 10 indicators in each of the 3BL dimensions. Data collection and analysis through MCDA approaches are complex when many attributes/criteria (i.e., here indicators) are involved during the process [16,72,73]. MCDA approaches such as Analytical Hierarchy Process (AHP), Analytical Network Process (ANP), Weighted Sum Method (WSM), Weighted Aggregated Sum Product Assessment (WASPAS), Simple Multi-Attribute Rating Technique (SMART), Multi-Objective Optimization Ratio Analysis (MOORA), Weighted Product Method (WPM), Simple Additive Weighting (SAW), etc. are the weight evaluation approaches widely used for the ranking or prioritizing criteria or attributes [69,72,74]. Previous research on MCDA techniques suggests the maximum utilization of AHP, fuzzy AHP, ANP, TOPSIS for the weight calculation of attributes [50,52,64,69,74]. Though these techniques are being utilized in the majority of research, the inconsistencies in the results cannot be neglected due to its large and complex pairwise comparisons (i.e., due to the large no. of attributes), which lead to ambiguous outcomes [75]. The BWS is a survey method used for assessing individual priorities and has been noted for cognitive and administrative simplicity [76]. Further, this method is considered superior to rating scales because it avoids common biases and provides consistent ranking with numerous attributes [77]. Thus, the best-worst scaling (BWS) is a suitable MCDA technique to collect and analyze the data with numerous indicators in the decision-making problem [78,79]. Therefore, in the present research, we used the BWS technique, first introduced by the authors of [80] for the second round of the Delphi study. The BWS method helps to obtain granular details from experts and eliminates rating using an integer-based scale [80]. Rather, the BWS provides ratings in two sentences indicating “best” and “worst” based on attributed nature [72,73,81] and preference score of the selected list of 3BL indicators from the experts.

A balanced incomplete block design (BIBD) was built for the BWS exercise to collect relevant data. The block design was made using Sawtooth software [82]. Further, three different questionnaire versions with three different block designs were created using the 3BL perspectives (i.e., environmental, social and economic). Each questionnaire design consisted of 20 questions, and three attributes were presented per question. Simple choice-based questions were created in the questionnaire. For the development of the questionnaire, the design algorithm was similar to those of choice-based conjoint (CBC) and the questions were created based on multiple variables in triple-bottom-line perspectives. The questionnaire sets were distributed randomly through e-mail to the same groups of 196 experts who participated in the first round of the Delphi survey.

A total of 171 experts completed the questionnaire, resulting in an 87% response rate. The survey response data (i.e., preference score) were used to prioritize the sustainability indicators by respondents. Each respondent’s preference score was estimated through the Hierarchical Bayes (HB) method. HB is a “data borrowing” approach and is defined as a method to “stabilize part-worth estimates for each individual through borrowing information from other respondents within the same data set” [83]. To prioritize the sustainability indicators based on preference score, two ranking methodologies namely HB average ranking and HB frequency ranking were used for the ranking process of indicators to evaluate the sustainability of the manufacturing process. The preference score is used as an input to rank the sustainability indicators. A higher preference score from a respondent for a particular indicator helps to prioritize the sustainability indicator. The ranking process of indicators using this methodology was performed in the Sawtooth software. The ranked data were used as input to calculate the final consensus ranking through the Aggregate Uni-Criterion Ranking into One Ranking (AURORA) method, which is an MCDA technique. According to [84], the “AURORA method merges and compares the experts ranking, respecting the ordinal character”.

HB AURORA ranking was utilized to compare and improve the effectiveness of the prioritization results. HB AURORA ranking was performed by developing a structured branch and bound algorithm (BBA) using the C++ programming language. The pseudo-code is shown in Appendix A. AURORA requires a pairwise comparison between respondents and alternatives ranking per respondent [16]. The ranking per respondent is calculated first with the help of the HB preference score. The higher score provided by the respondent to a certain indicator gives a higher position to that indicator. Kendall’s approach is known as the correlation coefficient, τ, and helps measure consensus ranking between respondents and alternative ranking per respondent [85]. The value of τ can be calculated through Equation (1).

$$\mathsf{\tau }=\frac{2*\left(\mathrm{C}-\mathrm{D}\right)}{{\mathrm{n}}^2-\mathrm{n}} \mathrm{and} \mathrm{C}+\mathrm{D}=\frac{{\mathrm{n}}^2- \mathrm{n}}{2}$$

(1)

where, C = concordant pairs, D = discordant pairs and n = number of alternatives

The calculated value of τ can be obtained between the ranges of −1 to 1, which is considered perfect disagreement to a perfect agreement [85]. According to [85], the median of τ can be calculated after every iteration and maximized until potential consensus ranking.

4. Results

The sustainability indicators selected during the first round of the Delphi study are represented in Table 2. These sustainability indicators were used as inputs for the second round Delphi study analysis.

During the first round of the study, the experts were asked to provide a preference of evaluation tool (i.e., which evaluation tool they prefer to assess the sustainability of the manufacturing process). The options included a single index, multiple indexes, or both. After analyzing the responses, the results indicated that 182 respondents preferred multiple indexes, while three respondents preferred a single index, and 11 respondents preferred a combination of both. Based on most responses, the experts concluded that multiple indexes were highly effective in evaluating the sustainability of any manufacturing process. The conversation among the experts indicated that assessing the manufacturing process sustainability using a single index was too complex. Instead, utilizing multiple indicators helps investigate the trade-offs between different sustainability impacts.

In the second round of study, the BWS exercise responses were analyzed through HB. The analysis of responses revealed that all experts reaching fit statistics should have a root likelihood greater than 0.167 [82]. The outcomes of the BWS HB analysis for the 3BL approach, the counting analysis and the final census rank are presented in

Table 3. BWS HB analysis, counting analysis and final census rank outcomes for environmental indicators.

Sustainability Indicator	BWS HB Analysis			Counting Analysis		Final Census Rank
	Average Rank	Frequency Rank	Preference Score	Proportion of Best Count	Proportion of Worst Count
Greenhouse/harmful gas release	1	1	16.424	0.617	0.009	1
Net solid waste generation	2	3	15.398	0.438	0.058	3
Net green area impact	3	2	12.289	0.468	0.079	4
Net CO2 emission impact	4	4	11.010	0.336	0.037	5
Toxic discharge to water	5	5	9.204	0.343	0.201	2
Process coolant/oil consumption	6	6	8.320	0.235	0.089	8
Electricity consumption	7	8	7.098	0.226	0.184	9
Energy consumption per unit	8	7	6.620	0.177	0.153	10
Raw material consumption	9	10	5.021	0.061	0.128	7
Waste segregation percentage	10	9	4.921	0.014	0.188	6
Reuse/recycle raw material ratio	11	11	2.369	0.049	0.490	11
Scrap rate	12	12	1.203	0.031	0.461	12
Net water foot print	13	13	0.171	0.000	0.800	13

,

Table 4. BWS HB analysis, counting analysis and final census rank outcomes for social indicators.

Sustainability Indicator	HB Analysis			Counting Analysis		Final Census Rank
	Average Rank	Frequency Rank	Preference Score	Proportion of Best Count	Proportion of Worst Count
Contribution to society rate	1	1	14.102	0.457	0.048	1
Local business support index	2	2	11.034	0.407	0.092	2
Gender ratio	3	3	10.035	0.461	0.118	5
Time weighted average to record noise exposure	4	5	10.199	0.469	0.170	4
National production rate	5	4	8.380	0.307	0.141	3
Staff incentives/commission/benefits	6	6	8.065	0.239	0.099	8
Employee satisfaction rate	7	7	7.142	0.298	0.102	10
Accident rate	8	8	6.302	0.191	0.153	6
Training opportunity to employees	9	11	5.302	0.190	0.128	9
Employee turnover ratio	10	10	5.120	0.190	0.204	11
Absenteeism ratio	11	9	4.502	0.173	0.319	12
Gender salary ratio	12	12	3.730	0.120	0.172	13
Staff salary level	13	13	3.195	0.091	0.121	7
Post parental leave retention	14	14	1.990	0.051	0.263	14
Volunteer sustainability initiatives ratio	15	15	0.901	0.033	0.250	15

and

Table 5. BWS HB analysis, counting analysis and final census rank outcomes for economic indicators.

Sustainability Indicator	HB Analysis			Counting Analysis		Final Census Rank
	Average Rank	Frequency Rank	Preference Score	Proportion of Best Count	Proportion of Worst Count
Overall equipment effectiveness	1	1	17.508	0.569	0.034	3
Operational cost	2	2	17.038	0.436	0.055	1
Acceptance rate of product	3	3	16.567	0.328	0.049	2
Level of work in process inventory	4	4	5.171	0.283	0.129	4
Inventory holding cost	5	6	5.025	0.278	0.130	7
Value added time	6	7	4.109	0.294	0.099	8
Value added cost	7	5	4.033	0.253	0.088	9
Machine performance	8	8	3.207	0.320	0.077	5
Machine availability	9	9	3.110	0.397	0.052	6
Labor cost	10	11	3.040	0.206	0.204	10
Management cost	11	10	2.437	0.179	0.319	11
Facilities and depreciation cost	12	12	2.326	0.132	0.172	12
Effective cost	13	14	2.213	0.102	0.121	15
Stock cost	14	13	2.101	0.231	0.263	13
Takt cost	15	15	2.093	0.108	0.250	14
Value added time ratio	16	16	2.001	0.126	0.156	19
Value added cost ratio	17	17	1.413	0.107	0.133	18
Total productive maintenance ratio	18	20	1.220	0.096	0.210	17
Return on investment on innovation	19	18	1.141	0.084	0.419	16
Changeover time	20	19	1.132	0.069	0.331	20
Cycle time	21	21	1.102	0.042	0.457	21
Uptime	22	22	1.007	0.022	0.553	22
Transportation efficiency ratio	23	23	1.005	0.000	0.349	23

for the environmental, social and economic indicators. The counting analysis indicates the proportion of an indicator is picked as best and/or worst [16]. The present study results showed that the few sustainability indicators were never picked as best indicators by experts such as net water footprint, volunteer sustainability initiatives ratio and transportation efficiency ratio. Based on the analysis, 80% of the experts selected the net water footprint as the least important indicator.

The maximum importance was given to greenhouse/harmful gas release with a 16.42 preference score, followed by net solid waste generation with 15.40 and net green area impact with a 12.29 score in the environmental indicators of sustainability. The lowest preference score was for net water footprint, with a score of 0.17. Moreover, the overall results indicated that the preference score of the environmental indicators gradually decreases compared to the other two sustainability indicators. In the case of economic indicators, the results show stable preference scores between 2.44 and 2.00 for the sustainability indicators of management cost, facilities and depreciation cost, effective cost, stock cost, takt cost and value-added time ratio. The highest scores among all 3BL indicators were given to economic indicators such as overall equipment effectiveness (OEE), operational cost and product acceptance rate with preference scores of 17.51, 17.04 and 16.57, respectively. The maximum importance scores for the social indicators were contribution to society rate, local business support index, gender ratio and time-weighted average to record noise exposure. Based on the expert input, these four social sustainability indicators account for 45.37% of the total importance scores.

The first sustainability indicators prioritizing method (i.e., HB analysis average rank approach) was applied. The rank is provided based on the average utility of each indicator. For example, greenhouse/harmful gas release, contribution to society rate and OEE ranked first. Whereas net water footprint, volunteer sustainability initiatives ratio and transportation efficiency ratio ranked last for the environmental, social and economic sustainability dimensions, respectively. The average utility scores provided by the group of experts were used to create the average ranking. The rank is assigned based on the frequency of an indicator placed at a particular prioritizing order. The individual rank was assigned by utilizing the individual preference score of the HB analysis. To further elaborate on the frequency ranking, an example is provided in Figure 2 representing a comparison of the three environmental sustainability indicators frequencies at a particular rank position. This type of frequency analysis offers the best way to draw the final consensus of ranking and avoid averaging. According to [16], one drawback has been observed with this approach: it is difficult to assign a rank to indicators when the maximum number of attributes involved in the analysis have the same frequencies. As such, Figure 3 shows an example of how difficult it is to assign a rank when the maximum number of attributes is involved. A suitable model was designed based on the HB AURORA approach to overcome these difficulties and increase the validity of the final ranking of the sustainability indicators. The HB AURORA ranking approach was chosen to develop a reliable consensus ranking as a third-ranking approach.

To perform the third-ranking approach (i.e., HB AURORA), an algorithm was developed using the C++ programming language to determine the ranking of sustainability indicators. The methodology performed by [84] was used to write the Branch-and-Bound algorithm. From this perspective, the median Kendall’s value, τ, was calculated for each dimension of the sustainability indicators. The median of Kendall’s value was maximized to determine the best ranking fit [85]. After running the algorithm, multiple optimal solutions were observed for each sustainability dimension. The algorithm found 29 optimal solutions for the environmental dimension indicators, 324 for the social dimension indicators and 2 for the economic dimension indicators. Figure 4 shows an example of the 29 possible optimal solutions compared to the environmental dimension. The analysis observed the same value of τ for every optimal solution with 0.5829 for the environmental dimension, followed by 0.5135 for the social dimension and 0.6421 for the economic dimension of sustainability. The calculated value for the environmental dimension revealed that at least 58% of the respondents have a rank correlation coefficient of 0.5829 or more for the optimal solution.

When the complexity increases with a decision problem, the AURORA approach generates maximum optimal solutions with a lower value of Kendall’s τ, which results in a lack of consensus in the results [16]. Kendall’s value, τ, was calculated using Equation (1). The overall ranking result reveals that the social dimension has less consensus among the three sustainability dimensions due to the low median Kendall’s value (τ = 0.5135) with a maximum number of optimal solutions (i.e., 324). Moreover, the results indicate that the social indicators are given less attention by manufacturing industries in the Asian continent; therefore, social sustainability indicators should be quantified through proper formulation. The formulation of these indicators to assess social sustainability levels should be clear and easy to understand.

5. Discussion

The results obtained from the three different ranking approaches are discussed with the supporting relevant literature and experts’ feedback. All ranking methods determined that greenhouse/harmful gas release was the most relevant sustainability indicator when looking into environmental dimension indicators. The results also indicated that several respondents suggested greenhouse/harmful gas release is a widely accepted sustainability indicator for manufacturing process assessment. Past studies supported these findings by considering this indicator (i.e., greenhouse/harmful gas release) as the most important indicator to assess environmental sustainability [36,37]. The continual increase of releasing greenhouse gases has resulted in a hole in the ozone layer, which negatively affects human life and other species [86,87]. Concerning the environmental sustainability indicators, the second-highest ranking was toxic discharge to water. According to a global report in 2016, more than 70% of industrial wastes (e.g., toxic chemicals, garbage, sewage, industrial sludge) are dumped directly into the water resulting in decreased oxygen levels, birth of non-native species and bacteria that directly affect human health [88]. Most of the previous studies [17,19,38] suggested that “toxic discharge to water” is an important indicator for assessing environmental sustainability. The third-ranked factor was the net solid waste generation. This indicator influences both social and economic perspectives. For example, the generated industrial nonhazardous or solid wastes are managed by proper recycling, reuse, storage, treatment, or disposal, which influences economic activity. On the other hand, the management of such wastes directly affects human health and the environment by releasing chemicals and odor [89,90]. The frequent appearance and priority of solid waste generation in past literature proved that is an important indicator for environmental sustainability assessment [91,92,93].

The contribution to society rate was ranked first among all indicators of social sustainability, which considers employment opportunities provided by the organization. Based on the expert consensus, it primarily indicates the level of social contribution by the industry where they work [94,95,96]. This outcome is also supported by the previous literature which stated that the “contribution to society” indicator is a leading indicator for assessing social sustainability performance [14,15] The Local business support index was ranked second for the social dimension, indicating the opportunity level provided to local suppliers. This indicator is also considered an important indicator by authors [97]. The national production rate was ranked third. The national production rate directly assesses the industry’s contribution to the community by calculating the domestic rate used in the manufacturing process. An increase in this factor represents economic and social development in the community where the industry is located [14,98].

For the economic sustainability domain, the top three sustainability indicators changed their ranking position when compared to applied ranking methods. The indicators OEE, operational cost and an acceptance rate of the product were ranked first, second and third, using HB average and HB frequency methods. However, when applying the HB AURORA methodology for consensus ranking, operational cost was ranked first, followed by acceptance rate of product and OEE in second and third, respectively. In the manufacturing process, operating cost is considered an important aspect to reduce the overall cost and, subsequently, reduce the selling price of the product to satisfy customers. Based on the Lean manufacturing concept, the overall cost of a product can be easily reduced by removing non-value-added activities [14,21]. Several research studies [17,19,38] suggested that “operational cost” is one of the most important indicators to assess economic sustainability. The acceptance rate of the product is highly related to economic and social perspectives. Suppose the acceptance rate of the product is high. In that case, the organization receives a higher profit, which may further lead to increased benefits for employees in terms of incentives or salary [53,99]. A high product acceptance rate also decreases waste from the manufacturing process, which reduces the environmental impact.

Further, OEE was ranked third using the HB AURORA ranking for the economic sustainability domain. OEE is directly related to manufacturing operations. It identifies the truly productive time from the total time provided for manufacturing the product [100]. The frequent appearance and priority of the overall effectiveness assessment indicator in past literature have proved that is an important indicator for economic sustainability assessment [7,21,42,101].

In addition to the high-priority sustainability indicators, it is essential to explore the lowest indicators. The results reveal that the reuse/recycle raw material ratio, scrap rate and net water footprint had the lowest rankings for the environmental sustainability domain. Further, gender salary ratio, post parental leave retention and volunteer sustainability initiatives ratio were the lowest-ranked social sustainability indicators. Finally, cycle time, uptime and transportation efficiency ratio were the lowest-ranked indicators for the economic domain.

The group experts used the argumentation method to understand the lower-ranked 3BL sustainability indicators. The experts clarified that the sustainability indicator evaluation is very case-specific. The general consensus ranking of the sustainability indicators does not mean the lower-ranked indicators are irrelevant for assessing the sustainability of a manufacturing process. For example, the transportation efficiency ratio is the lowest-ranked (Table 5). This indicator can act as a key sustainability indicator in certain cases where most of the manufacturing processes are performed by outside vendors.

5.1. Theoretical Implications

Sustainability assessment is a major concern; manufacturing organizations need to assess their current process sustainability and develop solutions to improve further if required. The literature suggests that none of the previous studies have provided a comprehensive and complete set of sustainability indicators in 3BL perspectives to evaluate a manufacturing process. The present study examined a structured set of sustainability indicators and evaluated their consensus ranking to assess the manufacturing process of the organization. These indicators help assess the current sustainability of the manufacturing process in 3BL perspectives. So, organizations can develop possible solutions for achieving higher sustainability.

The literature suggests this study is unique because there was no evidence of applying the Delphi technique with MCDA approaches to identify the consensus ranking of sustainability indicators for manufacturing sectors. Since the qualitative survey technique contributes to the maximum efficiency of quantitative approaches, it makes the results more consistent by reducing the possibility of vagueness in the findings. This study has identified and developed a consensus general structured set of 3BL sustainability indicators for assessing the manufacturing sustainability of Asian manufacturing industries that were not present in the previous studies. The literature also suggested that in the majority of the sustainability indicators studies, researchers have not provided a comprehensive set of 3BL indicators. Ref. [14] considered a pool of 17 sustainability indicators in 3BL perspectives to assess the sustainability of the manufacturing process of three different manufacturing organizations located in Brazil. To analyze the process sustainability, only six indicators were considered and assessment was performed from the perspective of Indian manufacturing organizations [15]. Ref. [35] considered only 11 indicators in triple bottom line perspectives for the life cycle evaluation of the sustainable manufacturing process. In the Malaysian sawmill sector, environmental sustainability was assessed through the evaluation of energy consumption [36]. Ref. [37] considered only five indicators to improve the environmental performances of wine production. Ref. [21] considered only six economic indicators to assess the economic sustainability of processes in one Indian automotive component manufacturing organization. The identification and prioritization of an experts’ consensus structured set of 51 triple bottom line (3BL) indicators can be considered a significant contribution to assessing any manufacturing process’s sustainability. The literature suggested that none of the previous studies have identified such a large set of empirical indicators and prioritized the BWS approach, which facilitates sustainability assessment effectively in 3BL perspectives.

5.2. Managerial Implications

This study aims to identify and prioritize the experts’ consensus structured set of triple bottom line (3BL) indicators for assessing the sustainability of any manufacturing process. The difficulties in selecting appropriate indicators for the sustainability assessment of a manufacturing process were overcome by prioritizing the empirically tested sustainability indicators. Further, the biasedness in a predefined questionnaire survey was overcome by utilizing an open-ended questionnaire approach. Open coding is used to reduce the difficulties in response duplicity in qualitative data for further analysis and best-worst scaling (BWS) is a suitable MCDA technique to collect and analyze the data when there are numerous indicators. Furthermore, integration of HB average, HB frequency and HB AURORA ranking method help to compare and improve the effectiveness of the prioritization results. The proposed approach can be applied in any manufacturing organization to help practitioners, decision-makers and managers to identify the indicators for real industrial scenarios and prioritize the indicators based on the problem of the applied industry. The result of the present study helps researchers, practitioners, decision-makers, and industrial managers to identify and prioritize the triple bottom line (3BL) indicators for sustainability assessment in a manufacturing environment.

5.3. Challenges, Limitations and Directions for Further Research

The present work utilized the BWS method to collect the data from a Delphi study, which generally eliminates scaling bias and discrimination [102]. In addition, a detailed, brief description of each sustainability indicator was provided in the questionnaire, which shows a strong contrast in a complex situation. To avoid misinterpretation of any indicator description, a comprehensive definition of each indicator was provided to each respondent at the start of the survey. The questionnaire included a comment box to encourage respondents to report ambiguity. The validity of the research could be further improved by gathering information on the applicability of these indicators in an Asian case study. Furthermore, the result of the three ranking methods shows significant similarities, indicating the robustness of the result. The ranking results reveal that the top and bottom-ranked sustainability indicators are almost the same as only some minor changes appeared when applying other ranking methods. The present study used the BBA in the HB AURORA method, which does not allow for ties. This limitation ensures a precise ranking for decision-makers. However, permitting ties could increase the consensus in the ranking and cluster sustainability indicators [84]. A future study could fill this gap by extending the current BBA and exploring the effects of permitting ties into the model.

Besides methodological concerns, follow-up research is essential to verify the sustainability indicators are applicable for manufacturing process assessment. At present, there is a lack of a standardized method and a consensus set of sustainability indicators to assess the sustainability of a manufacturing process [14,17]. To perform the sustainability assessment, it is necessary to develop measurement methods to fill the existing literature gap. Subsequently, the 3BL sustainability indicators identified through the Delphi technique are broadly defined and might need proper formulation for statistical testing and verification. The execution of a systematic case study can further explore the need to formulate indicators.

Finally, the present study developed a consensus general structured set of 3BL sustainability indicators for assessing the manufacturing sustainability of Asian manufacturing industries. Prioritization of these consensus indicators can be helpful for practitioners, decision-makers and researchers, mainly when resources are limited. To adopt this consensus general set of 3BL sustainability indicators to a specific organization, an iterative stakeholder-based process is recommended to practitioners. In this context, decision-makers should identify the general guidelines provided by the case industry and propose modifications to conform to all of the critical sustainability indicators for the final analysis. After completion of the first-round calculation of indicators, stakeholders of the organization should once again check the validity of the primary outcomes. The developed consensus general structured set of 3BL sustainability indicators in the present study provides a foundation for practitioners to focus on the sustainability assessment of a manufacturing process.

6. Conclusions

Sustainable manufacturing has gained interest and has the potential to tackle some of the sustainability challenges that manufacturing organizations must endure. Sustainability needs to be assessed and monitored to minimize the negative environmental impact and enhance reputation among public and regulatory agencies. The objective of the present study is to investigate the consensus list of sustainability indicators needed for sustainability assessment of the manufacturing process and prioritized them based on their preference. The present study considered a two-stage Delphi-based study approach for prioritizing the consensus result of 3BL sustainability indicators. The approach was particularly developed to evaluate the sustainability of any manufacturing process. The experts from various sectors such as manufacturing industries, certification bodies and academia were selected to perform the Delphi study. The final ranking of these indicators represents how experts extended their experience to adopt the sustainability concept within manufacturing processes and prioritize sustainability indicators to new researchers, practitioners and decision-makers to assess the sustainability within the Asian continent. Three different methods were utilized to rank sustainability indicators such as HB average, HB frequency and HB AURORA. The outcomes of these three methodologies were compared for the consensus ranking and HB AURORA was found to provide the best results to reach experts consensus. The consensus of experts was measured by calculating median Kendall’s value, τ; which obtained positive values for all 3BL sustainability domains. The present study observed a strong consensus with the economic domain of sustainability with a 0.6421 median Kendall’s value, followed by the weakest consensus observed in the social sustainability domain with a value of 0.5135.

The study’s outcome indicates greenhouse/harmful gas release, contribution to society rate and operational cost were the most critical indicators for environmental, social and economic sustainability aspects. Existing literature lacked a structured consensus set of 3BL sustainability indicators for sustainability assessment of a manufacturing process. With the use of the prioritized consensus set of indicators, practitioners can assess the sustainability of their manufacturing processes to drive further sustainability improvements. In addition, ranking the indicators through the MCDA approach helped assign weights/select indicators for when resources are limited or unavailable (i.e., money, time and data). However, minimizing the number of sustainability indicators may increase risk and weaken the analysis.

The present study also provided stepwise guidelines for evaluating and prioritizing 3BL sustainability indicators in the manufacturing context; however, the prioritization of these indicators might differ with the case approach. Therefore, future studies should embed these indicators set in different cases to formulate the indicator’s statistical tests and verify the resulting sustainability assessment. Future research can also test and validate the results by applying new MCDM methods.