1. Introduction
The nature of each construction project is unique and dynamic as it involves numerous operations with multiple intricacies and various techniques that are used throughout one single project. Construction projects encompass various risks, and it is essential to evaluate and manage those risks to guarantee the success of a project or firm [
1] . Ref [
2] stated that the construction industry has a poor reputation in risk classification and analysis compared to other industries. Many variables and complex relationships that exist in construction projects make the execution of a decision complicated. Efficient and effective go/no-go decisions will provide benefits for owners or investors for future business development. There are several kinds of risk decisions in construction projects, starting with the concept phase, continuing with the planning and execution phases, and ending with the completion phase. In each phase, various stakeholders (owners, CMs (construction management firms), contractors, consultants, suppliers) are involved in the uncertain and complex decision-making process in order to attain the greatest value of project objectives. Project decision management is a repetitive process, which is advantageous when implemented in a systematic manner throughout the lifecycle of a project, especially before the execution phase, at which the owner decides to go or not to go with the execution of a decision. This paper introduces, using QUEST (Quick, Unbiased, Efficient Statistical Tree) and exhaustive CHAID (Exhaustive Chi-square Automatic Interaction Detector) decision tree algorithms, the development of go/no-go models for owners/CMs. The model is supported by the attitude of the owner in relation to his/her competence, as well as by the risk attached to the project, as the owner must evaluate possible risks and his/her capability in the planning phase before the execution of a project. The proposed model is expected to identify certain strategies that will meet the financial targets of an owner based on four main go/no-go risk groups (organizational, project/technical, legal, and financial/economic) that include 23 go/no-go factors. Before applying the go/no-go decision tree models, owners should assess the risk level of each go/no-go group by calculating the average risk of all related factors of each group. Therefore, the decision tree of go/no-go models should be able to predict the feasible decision in the planning phase so that the owner can move on to the execution phase. The contribution of this study to the existing body of knowledge is through its recognition of the most influential factors affecting the go/no-go decision (invest/don’t-invest) of the owner, as well as the introduction, for the first time, to the best knowledge of the authors, of an exhaustive CHAID and QUEST decision tree model for go/no-go decisions. This study is important from the “sustainability” perspective because the owner’s decisions based on a robust model would yield sustainable and successful projects.
2. Literature Review
A number of previous studies investigated the decision-making process in the construction sector [
3,
4,
5,
6,
7]. Additionally, go/no-go decision models for decision-making concerning different sectors are also available [
8,
9,
10,
11,
12,
13,
14]. It can be concluded from previous studies that a considerable amount of literature has been published on contractor decisions, while little literature has been published on decisions of owners or construction management firms. Moreover, certain models require complicated inputs and an advanced understanding of mathematics and require software to run the suggested model, which are not practical for the owner. Additionally, some models do not address the importance of risk assessment in the early stages of the project and focus on the bidding stage, although early-stage decision-making is much more critical. Moreover, some previous feasibility models exclude some factors affecting the owner’s decision and focused mainly on financial factors. A combination of all expected factors will result in a better decision strategy for owners. Thus, developing a practical, easy, and fast model that supports the owner’s decisions in the early stages of a project is a significant need.
Numerous researchers and studies have contributed to defining go-decision attributes in the construction industry. The literature on go/no-go decision factors was studied to identify the key factors that influence decision-makers’ judgments. Extensive studies by [
3,
15,
16,
17,
18,
19,
20,
21,
22,
23,
24,
25,
26,
27,
28,
29,
30,
31,
32,
33,
34,
35,
36,
37,
38,
39,
40,
41] identified the most influencing factors affecting go/no-go decisions. In this study, a draft questionnaire with 23 key risk factors was prepared based on the existing literature and divided into four main groups (organizational, project/technical, legal, and financial/economic).
Table 1 demonstrates the twenty-three go/no-go decision attributes, with their corresponding literature references.
3. Research Methodology
This part presents the research methodology that was used in this study to achieve the research objectives. A questionnaire was designed to rank the 23 go/no-go decision factors collected through the literature review. The questionnaire was structured in such a way that it would determine the most significant risk factor based on the practitioners’ observations as well as the relative significance of each risk group. Each respondent was requested to rank each risk factor on a scale from 1 to 5 by considering its importance, 1 signifying the lowest degree of importance; 100 responses were collected from construction industry professionals. The collected data were analyzed using Spearman’s rank correlation and Tukey’s test to extract the important features of the respondents’ responses. Spearman’s rank correlation captures the statistical differences among categories, whereas the Tukey test covers the differences among factors. In the second section of the questionnaire, the respondents were requested to indicate how often their company takes go decisions in the early stages, after the initial design is completed, for different scenarios of the four aforementioned categories (organizational, project/technical, legal, and financial/economic). Then, 80 scenarios were thereby developed and distributed into two forms (40 scenarios per questionnaire).
Following that, a decision tree based on exhaustive CHAID and QUEST was introduced for a go/no-go decision model for owners in the construction sector. The decision tree displays the soft and hot spots between independent and dependent variables, which leads to better decision-making. Decision tree models display the result effectively in visual terms so that it becomes easier to understand and apply. To the best of our knowledge, this is the first study in the literature to introduce an Exhaustive CHAID and QUEST decision tree model for go/no-go decisions to owners.
Figure 1 below shows the steps of the methodology used in this study.
3.1. Decision Trees
Decision trees have been identified as the most influential and useful classification technique that is utilized in the field of data mining. A decision tree is considered to be an instrumental tool that supports the decision-making process as it translates inputs into a tree-like model with their different outcomes, which include “utility, costs of resources, and chance-event outcomes” [
43]. Decision trees are useful in managing two types of data sets: categorical data and numerical data [
44]. It is considered the easiest and least complex technique because it provides result interpretation capabilities to the parties involved.
According to [
45], a decision tree is a model that has the shape of a tree with different branch nodes. The tree starts with the root node at the top, with branch nodes underneath it, where the data are presented by different branches, using some section measures. Each branch node is responsible for representing a choice among alternatives based on the number of substitutes available. The leaf node in the model is responsible for representing a categorical or numerical decision [
45]. The complexity of the tree can be measured by either counting the total number of leaves or nodes or by the depth of the tree used. The construction of a decision tree can be explained as follows; firstly, the attribute selection is placed in the root node, and then two or more branches are created for each option value, which is followed by the split process to form subsets for every value of the attribute. This process is repeated till the node has the same classification value. Ref. [
46] pointed out that the main differences among the various decision-tree-building algorithms are identified by the attribution that produces the best split in the data. Each decision tree algorithm has its own measures to select the attributes at each step while growing the tree.
The benefits of decision trees may be explained as follows: Decision trees are known to be flexible and adaptable as they are made compact in order to become more understandable. In simple terms, decision trees can be understood by nonprofessional users too because the model is straightforward and easy to understand. Furthermore, they utilize sets of rules, making them clearer and more coherent. Decision trees are useful in classifying knowledge into trees, and, thus, they facilitate the decision-making process.
According to [
47], the Exhaustive CHAID algorithm was found to be a good and significant predictor. Additionally, Ref. [
43] examined four prediction algorithms in terms of accuracy and concluded that Exhaustive CHAID and QUEST techniques proved to be more efficient and accurate than other prediction techniques. Thus, this study applies the most appropriate combination of Exhaustive CHAID and QUEST models to forecast the potential risks of a proposed project at an early stage and to compare the prediction performance in terms of accuracy and efficiency.
The tree models were generated using a training sample and tested on a selected sample. The training sample was a percentage of the total sample size, and the remaining portion was used for the test sample; 80% of the collected data were used for developing the go/no-go decision tree models, and the remaining information was used for the test sample.
3.1.1. Exhaustive CHAID
CHAID stands for Chi-squared Automatic Interaction Detector. The Exhaustive CHAID algorithm is a modified version of the CHAID decision-tree algorithm, which was developed by [
48] to overcome some of the weaknesses of the latter. The main difference of Exhaustive CHAID is that it examines all possible splits on each node, and it does not stop the splitting process even if the optimal split is reached. It keeps merging the categories of the predictor variable till only two subcategories are left. It has three core steps: merging, splitting, and stopping [
49]. In a decision tree, these steps are repeated on each node, initiating from the root node.
Merging: The method used to calculate the p-value depends on the measurement level of Y. The F-test will be used if Y is continuous; the two-way cross tabulation test will be used if Y is nominal; the likelihood-ratio test will be used if Y is ordinal. The pairs that have the greatest p-value will be combined to develop the compound category. The previous step will be repeated till two categories are present. The set of categories having the smallest p (index) will be located.
Splitting: The independent variables with the least p-value will be used as a node separator. If this p-value is lower than or equal to the α split identified by the user, the node will be subjected to splitting with the given predictor.
Stopping: The stopping process is dependent on four factors: split, depth of the decision tree, the least number of parent nodes available, and the least number of child nodes available.
3.1.2. QUEST
QUEST is the decision tree algorithm that is responsible for the classification of data. It has a splitting rule, which assumes that the targeted variable is continuous or uniform. In terms of calculation speed, it is efficient and fast compared to some other methods. It also can neglect the bias that is prevalent in other decision tree algorithms. It is generally believed that this algorithm is more appropriate for multiple-category variables. However, it can be used to process binary data only.
QUEST stands for Quick, Unbiased, Efficient statistical tree, which is based on the binary split decision tree algorithm. It is used for data classification and mining. It can be used in a variety of combinations. These include linear or univariate combination splits. The unique aspect of QUEST is that the bias in its attribute selection method is negligible. The construction process of a QUEST tree comprises split predictor selection, split point selection for the split predictor, and stopping.
Loh [
50] asserts that QUEST has two steps that are based on the “significance tests to split each node”. During the first test, the association of each X with Y is tested. The variable selection is based on the level of significance. The highest significant variable is selected. If each X is independent of Y, then each X has the same selection chance. As a result, selection bias is not present in this approach. QUEST uses different tests based on the nature of the variables [
50]. For categorical variables, it utilizes chi-squared tests. For ordered variables, analysis of variance tests are utilized.
For categorical prediction, the chi-square test result for Y and X independence needs to be calculated. The p-value is determined by means of the chi-square test. The least p-value predictor that is located and represented as X is chosen as the split predictor for the node.
Quadratic discriminant analysis (QDA) is used to find the best split point. QUEST is considered as a binary tree, which has a maximum of two splits from each node. However, if there are problems with more than two classifications and clustering, a two-means clustering method will be applied to group them into two superclasses by calculating the mean vector for all classifications. The splitting will form Group A, where the mean of all classifications is identical, while the rest will form Group B.
The QDA estimates the distribution of the two formed groups (
A,
B) by calculating the means and standard deviations from the samples and determines the split point as the point of intersection of the two Gaussian curves, being a root of the equation:
where
N is the node being split;
XA,
XB are the means;
SA,
SB are standard deviations of groups (
A,
B). Thus, a quadratic equation can result from the previous equation, as following:
where
For QUEST, the following stopping rules have been identified: for a node that is pure, the cases are grouped in the category of the same dependent variable within that node. In this case, the node will not be subjected to splitting. Additionally, if the predictor values are identical in the node, the node will not be subjected to splitting. Moreover, the node will not be subjected to splitting if the depth is prespecified and reaches its limit.
3.2. Population and Sample Size
In this study, beta-probability distribution was chosen to estimate sample size
N of the selected respondent (the participants’ years of experience will be used to determine the sample size). According to Chisala [
51] and Wanous [
52], the following equation is used to calculate sample size
N:
- -
is the required sample size, is the z-value for 99% confidence interval, and is the margin of error.
- -
The standard deviation of normal distribution can be estimated as follows:
For this study, maximum years of experience is assumed to be 45 years, 5 years of experience is the minimum number, and the margin of error is assumed to be 2 years; σ = 6.66; Z = 2.58, and ε = 2.0, hence, N = 74. The response rate of 30% was expected to be reached by the respondents. Thus, 200 questionnaires were distributed randomly among construction professionals; 100 out of 200 questionnaires were replied to by the respondents. The actual response rate was higher than the expected response (50%). The number of collected responses (100 participants) was more than the required sample size (74 participants). Therefore, the sample size of this study is valid.
3.3. Relative Importance Index (RII) Technique
To identify the importance of each risk factor that affects the owner’s decision, respondents were asked to rate each factor from 1 to 5 for go/no-go decisions. This scale was later transformed to a relative importance index (
RII) for all factors. A relative importance index (
RII) can be defined as a method that is used to analyze the relative importance of each factor that affects certain phenomena in the data collected by means of the questionnaire. Each risk factor is calculated by multiplying its impact by its frequencies based on the respondents’ view. [
53] used the formula below to calculate
RII as follows:
where
W: the weight given to each factor by the respondents (ranges 1 to 5);
A: the highest-ranking factor available, which is 5;
N: the total number of respondents that have answered the questionnaire.
3.4. Spearman’s Correlation
Spearman’s correlation can be defined as a statistical measure of the strength of a relationship between paired data. The value of the correlation coefficient
range is constrained between
. The closer the value to −1 or +1, the stronger the relationship, while 0 indicates no relationship between the variables [
54]. A comparison between all go/no-go categories (organizational-related factors, project-related factors, legal factors, financial factors) and the total level of importance is performed to determine the type of relationship between the two groups. According to [
29], Spearman’s correlation 𝒓
𝒔 is calculated by the following equation:
where
d: the difference between ranks assigned to variables for each factor;
n: the number of rank pairs (which is equal to the number of go/no-go factors, which is 23).
3.5. One-Way ANOVA Test
An analysis of variance (ANOVA) is identified as a statistical method that focuses on the comparison of the means of different samples. The purpose of ANOVA is to find whether a significant difference can be found between the class means of two or more independent groups [
55]. It is primarily utilized to analyze variances when the data is subjected to division in order to form different groups or classes by a single factor [
55].
3.6. Tukey’s Method
Tukey’s method is used for comparing the means of pairs. Since this research uses ANOVA, the difference found in the significance values of the groups needs to be compared in order to identify all possible pairs to determine the mean that is significantly different [
56]. Tukey’s method compares pairs as this technique increases the efficiency of significant differences in pairs.
4. Descriptive Statistics of the Respondents
The targeted respondents for this study were the experienced professionals in the construction sector. The data was collected from construction industry professionals worldwide. According to the respondent demographics, respondents with over 5 years of experience in construction made the largest portion of respondents, with 58%; 15% of the respondents had between 6–10 years of experience in construction; additionally, 15% and 12% of the respondents had between 16 to 20 and over 21 years experience in construction, respectively; 53% of the respondents represented private company owners, whereas 47% of the respondents represented public owners. In terms of company size, it is found that approximately two-thirds of the participants (65%) worked in companies with more than 500 employees. The majority of the respondents (57%) worked in projects worth $101–500 million. Finally, the participants were asked when to apply the proposed decision model. Almost half of the participants believed that the go/no-go model should be performed at the end of the conceptual design stage.
5. Ranks of the Factors Affecting the Go/No-Go Decision in the Early Stage of a Project
In order to observe the feedback of construction professionals, a questionnaire, as shown in the
Appendix A,
Appendix B and
Appendix C, was structured as a form that is necessary to build the decision tree model. The questionnaire was accompanied by a cover letter. The cover letter indicated the research objective and explained to the respondent that the proposed model will improve the owners’ ability to analyze and estimate risks and strengthen his decision based on the identified data.
The survey is composed of three parts, as following:
Basic personal and organization profiles (e.g., years of experience, company size, work volume) in order to have different groups of respondents for comparison and to develop the go/no-go decision model.
Risk factors affecting go/no-go decisions after the project definition and planning stage of construction projects. The questionnaire is structured to examine the most significant risk factor based on the practitioners’ observations and to determine the relative significance of each risk group. A quantitative weighting approach is adopted in this study to calculate the relative significance of project risks.
A scenario of a go/no-go decision. In this section, the respondents were requested to indicate how often their company takes go-decisions in the early stages after initial design is completed for different scenarios of the four categories (organizational, project/technical, legal, and financial/economic); 80 scenarios were developed and distributed into two forms (40 scenarios per questionnaire).
Table 2 below shows the relative importance index (RII) values and the ranking of key go/no-go factors. The respondents were requested to rate the level of importance of the 23 factors that influence the go/no-go decision in the early stage of a project. Likert scale 1 to 5 were defined as follows: 1 = very low importance, 2 = low importance, 3 = medium importance, 4 = high importance, and 5 = very high importance.
The top-three ranked factors in the organizational category were
- (1)
Financial stability of the owner;
- (2)
Consultants’ and suppliers’ reliability and experience in construction;
- (3)
Qualifications of designers and planners.
The 3 top-ranked factors in the project/technical category were
- (1)
Availability of materials and equipment;
- (2)
Tight schedule;
- (3)
Safety level required.
The top-ranked factor in the legal category was “excessive approval procedures in administrative government departments”.
The last category was the financial/economic category and the top-3 ranked factors in this category were as follows:
- (1)
Underestimated budgeting;
- (2)
High overhead cost;
- (3)
Expected return level/project profitability.
The second part of the analysis is calculating the average RII value per category. The RII values for each category were quite close to each other; however, the organizational category had the highest importance value, which was equal to 0.756. The legal category was marked as the fourth category as it had the lowest RII value, which was 0.705.
6. Ranking Comparison amongst Go/No-Go Category Factors
An inferential statistical test (Spearman’s correlation) was conducted between all go/no-go categories (organizational-related factors, project/technical-related factors, legal-related factors, financial- and economic-related factors) and the total level of importance to test the strength of the relationship between the categories and then to find out which category has the strongest correlation. The Spearman correlation test results are shown in
Table 3.
Table 3 indicates a strong positive correlation between project factors and legal factors. Overall, the Spearman’s correlation coefficient is higher than 0 and positive for the entire comparison, and the
p-value is smaller than 0.01 for all comparisons. It means that a positive relationship exists between each and every pair. The correlation between the financial/economic category and the organizational category is the smallest, with a value equal to 0.268. The table also shows that there is a strong and positive relationship (r = 0.886,
p-value < 0.01) between project/technical factors and the total level of importance, which is the strongest relationship in the whole comparison. There is also a strong and positive correlation (r = 0.749,
p-value < 0.01) between organizational factors and the total level of importance.
7. One-Way ANOVA Test Results
The primary target of applying the one-way ANOVA technique was to examine the potential differences and the degree of disagreement among the respondents based on their company sector, company size, project size, and years of experience.
Table 4 lists the results of the one-way ANOVA test.
As can be seen in
Table 4,
p-values among the groups of company size, company sector, and years of experience are higher than the threshold of 0.05. This indicates that there are no differences between the groups of company size and company sector on the importance level of the go/no-go factors. However, based on size, the
p-value between the groups concerning the legal risk category is smaller than 0.05. Additionally, the
p-value of financial and economic factors (
p-value = 0.012) is smaller than 0.05, which means that there are differences concerning the opinions of experienced respondents on the importance level of the go/no-go factors for the financial and economic risk category.
Based on the significant
p-value for the legal risk category, multiple comparisons were made to determine the disagreements among size categories using Tukey for the legal factors and the project size; they are given in
Table 5.
There is a significant difference between the opinions of respondents who work in small companies and those who work for medium companies (project size =
$1–5 million) regarding the level of importance of some legal go/no-go related factors. Additionally, the mean difference between respondents who were working on small projects and those who were working on very large ones was extremely high compared to other groups. This indicates that the level of importance of legal-related factors (at least one factor or more) in the early-stage process significantly differs regarding project size. Lastly, significant legal factors for the significant
p-values after running Tukey tests (see
Table 5) are presented in
Table 6.
It is obvious that the factor of lack of legality and standard dispute settlement procedure is the main reason behind the differences in opinions of people who are working on mega projects and those working on small projects in the legal category. This result can be translated as small projects being rarely prone to legal disputes because of their nature. On the other hand, it is logical that mega projects are more prone to legal disputes between the parties involved. Additionally, the standards and codes followed in mega projects are extensively complicated compared to simple projects.
Similarly, significant financial and economic risk factors for the significant
p-values listed in
Table 5 are presented in
Table 7 after running Tukey tests.
8. Decision Tree Model for the Owner
8.1. Using the Exhaustive CHAID Method for the Owner
The visually structured model of the Exhaustive CHAID tree-based algorithm used for the prediction of go/no-go decisions is presented in
Figure 2.
Financial/economic risk, legal risk, project/technical risk, and organizational risk are the only independent variables included.
The modality with the highest value is the one that is highlighted with grey inside the node.
The maximum tree depth that contains the most significant predictors of go/no-go decisions is three.
The best predictor of a go/no-go decision for the owner is the financial/economic risk category.
Node 0 in
Figure 2, at the top of the constructed tree, gives more than half of the prediction (55%).
In the first depth of the tree, the root node is divided into Node 1, Node 2, and Node 3—low, medium, and high categories.
The next best predictor is the legal risk level.
If the financial and economic risk and risk levels are high, the model shows 82% in favor of the no-go decision.
Model Validation—Exhaustive CHAID Method for the Owner
Split-sample is an evaluation technique used in predictive decision tree models that divides the original sample into a training set to train the model and a test set to evaluate it. In this model, 20% of the sample size was used to measure the accuracy of the model for future cases.
Table 8 shows the risk table for the model accuracy of the Exhaustive CHAID algorithm.
According to the table, the risk estimate for the training sample (0.247) indicates that the predicted value by the model (go or no-go) is wrong for 24% of cases. In other words, the risk of misclassifying a go-decision is approximately 24%. The estimated risk of the sample test is higher (26.3% of sample size), with a standard error of 0.015.
Table 9 shows the classification table for the exhaustive CHAID algorithm.
The result of the classification shows that the model correctly accounts for 75.3% of the training sample, while it accounts for a slightly lower percentage of 73.7% for the test sample. However, the result is considered acceptable, and it indicates that the model predicts the dependent variable with 74% correctness.
8.2. Using the QUEST Method for Owners
The QUEST algorithm deals with several sequences of rules in the evaluation of nodes based on a significant test, unlike the Exhaustive CHAID method. The visually structured model of the QUEST tree-based algorithm used for the prediction of go/no-go decisions is presented in
Figure 3.
Financial/economic risk, legal risk, project/technical risk, and organizational risk are the only independent variables included.
The modality with the highest value is the one that is highlighted with grey inside the node.
The maximum tree depth, which contains the most significant predictors of go/no-go decisions, is four.
The best predictor of go/no-go decisions for the owner is the financial and economic risk variable (similar to the result in Exhaustive CHAID).
Node 0 at the top of the constructed tree gives more than half of the prediction (57%) to not go ahead with the project.
In the first depth of the tree, the root node is divided into two nodes since it is a binary classifier. Node 1 will include low and medium categories, while Node 2 will include the high category.
Node 2 incites the next best predictor as the level of legal risk, while Node 1 is terminated from the tree as its outcomes are not beneficial.
If financial/economic risk is high, the model shows 84% for the no-go decision.
In a scenario where financial/economic risk is medium/low, the model in Node 2 will predict 57% for the go-decision.
Other scenarios can be predicted using the decision tree from the root node to the bottom nodes.
Model Validation—the QUEST method for the owner
Likewise, as with the Exhaustive CHAID method, a similar analysis (
Table 10) shows the risk table for the model accuracy of the QUEST algorithm.
According to
Table 10, the risk estimate for the training sample (0.248) indicates that the predicted value by the model (go/no-go) is wrong for 25% of cases. In other words, the risk of misclassifying a go-decision is approximately 25%. The estimated risk of the sample test is higher (24.9% of sample size) with a standard error of 0.008.
Table 11 shows the classification table for the QUEST algorithm.
The result of the classification shows that the model correctly accounts for 75.2% for the training sample, while it accounts for a slightly lower percentage of 75.1% for the test sample. However, the result is considered acceptable and indicates that the model predicates the dependent variable with a 75% level of correctness.
9. Case Study for the Developed Decision Tree Algorithms
The developed models will be validated by a real case study in Qatar. The case study involves the owner, the main contractor, and the government sector. The project is about a shopping mall located in a critical intersection in Qatar. The new design of an intersection affected the income of the owner badly as the construction of the intersection limited the visitors to the mall. The owner requested the Public Works Authority to develop a direct passage to the mall. To this request, the local authority responded and took action by asking a project management company to prepare a conceptual design and complete the evaluation process for adding a tenth bridge to directly connect the expressway highway to the mall to increase the number of mall visitors, as was in the past. The construction of the intersection is ongoing to develop a free-stopping intersection by building nine bridges for smooth traffic flow. The proposed bridge will be the extra bridge, which is considered a challenge due to its complex design. The main critical factors that drive the project are the need for the project, the budget for the extra work, the size and complexity of the project, and the risks associated with the additional work.
9.1. Project Description
The project includes grade-separated interchanges, with crossroads, frontage roads, overpass and underpass structures, retaining walls, and other related infrastructure. The intersection is considered the most complex intersection in Qatar, which includes nine bridges in one area for easy traffic flow. The proposed additional bridge will only serve the mall; thus, the cost will be on the owner of the mall, as discussed with local authorities. The overall total area required to construct the additional bridge is around 1350 m2. The additional bridge will result in the removal of six existing residential buildings. The proposed contract is design-build, assigned to the same main contractor that is working on this intersection, subjected to advanced approval from the mall owner. The contractor provided a conceptual design for the additional bridge, with a total cost of USD 30 million to the mall owner. The proposed time is extremely tight as the additional bridge should not impact the completion day of the ongoing project. The local authorities have denied any postponement of the opening day as it will have a high impact on the neighborhood and local society. As stated previously, the site is already busy with the ongoing project, and, moreover, additional work is required. A relatively high level of logistics and planning is needed to achieve the target without delaying the works of the remaining bridges. From the mall owner’s perspective, the market is classified as highly competitive as there are two other malls located at the same intersection. The proposed conceptual design consists of a single lane bridge, including MSE (mechanically stabilized earth) walls, six piers, and a retaining wall. A conceptual design has been prepared for a clear overview of constructability. The designer had already designed the previous nine bridges, and he aggregated the tenth one into the same intersection. The proposed construction work will be assigned to the same main contractor working in the intersection as the contractor is already experienced with the sequence of the work and the requirements. This will minimize the risks and control problems.
9.2. Risk Assessment
The evaluation of the risk factors for the case study is presented in
Table 12.
After the assessment of 23 go/no-go factors, an analysis was made to determine the categories’ risk level; the following result was found:
Organizational risk factors (Owner, CM, designer, planner, contractor) are considered low risk.
Project/technical risk factors are considered high risk.
Legal risk factors are considered high risk.
Financial and economic risk factors are considered high risk.
9.3. Go/No-Go Decision Using Exhaustive CHAID
The Exhaustive CHAID tree model in
Figure 2 should be tracked from the top “root node” to the bottom “child node” by the decision-maker. The present case study scenario is as follows: financial and economic risk is high (82.2% for no-go), legal risk is high (68.5% for no-go), project/technical risk is high (59.3% for no-go), and the organizational risk factors are low (74.1% for no-go). Although the last category has a low level of risk, the “no-go” decision is recommended according to the model.
9.4. Go/No-Go Decision Using QUEST
The QUEST decision tree model in
Figure 3 should be tracked from the top “root node” to the bottom “child node” by the decision-maker. According to the QUEST algorithm, financial and economic risk is high (84.2% for no-go), legal risk is high (65.7% for no-go), and project/technical risk is high (82.4% for no-go). No further nodes are required because the no-go decision is recommended; thus, the evaluation process should stop at this point.
10. Discussion
Twenty-three key go/no-go factors were identified from past research, categorized into four main groups, and ranked based on the relative importance index. The top five go/no-go factors are (1) financial stability of the owner, (2) underestimated budgeting, (3) high overhead cost, (4) availability of materials and equipment, and (5) consultants’ and suppliers’ reliability and experience in construction. Following that, Spearman’s correlation test was applied to investigate the relationship between the paired groups. Consequently, an ANOVA test analysis was performed amongst different groups of respondents, including such variables as company size, company sector, project size, and the respondents’ number of years of experience. It can be concluded that the majority of the comparisons did not have a statistical difference between the go/no-go groups. A slight difference was found between the respondents working on mega projects and those working on small projects in the legal category.
In this research, two decision tree algorithms (Exhaustive CHAID and QUEST) were structured to aid the owner in making the right decision during the early stage of a project. The primary goal of using two algorithms was to study the prediction accuracy of each algorithm and the percentage of error. The best predictor of the go/no-go decision for the owner is the financial and economic risk. The structure of the Exhaustive CHAID tree consists of 28 nodes, including 17 terminal nodes and a maximum of three depth levels. The second algorithm is QUEST, which consists of 15 nodes, including 8 terminal nodes and a maximum tree depth equal to four levels. The best predictor of the go/no-go decision for the owner is the financial and economic risk in this algorithm as well. The split-sample validation technique was applied to the decision tree models to check the accuracy of the go/no-go Exhaustive CHAID and QUEST models. All the charts indicate that the proposed models are good models; however, the QUEST technique has proved to be more efficient and accurate than the Exhaustive CHAID technique.
For an in-depth evaluation of the proposed models, a construction case study was implemented to test the results of the models. The case is interesting as it involves the owner, the main contractor, and the government sector. The findings obtained from the case study tested the proposed decision models’ ability in go/no-go decisions during the early stage, after conceptual design. Both models resulted in the same conclusion—a no-go decision. The conclusion assists the owner’s decision and prevents him from unwanted losses. The result strengthens the evaluation provided to the mall owner and the difficulties expected over the risk factors.
11. Conclusions
The goal of this study was to identify and categorize the most relevant risk factors influencing an owner’s go/no-go decision through a systematic literature review. The go/no-go variables were rated in a questionnaire by construction professionals according to their perceived importance. After that, using QUEST and Exhaustive CHAID algorithms, a decision tree model was developed. The outcome of this research benchmarks the existing evaluation models and provides a clear and user-friendly decision model for owners. It is expected that the model will determine anticipated risk factors in the project and decrease the degree of uncertainty. A case study validated the Exhaustive CHAID and QUEST designs. This paper adds to the existing body of information by defining the variables that have the greatest influence on the decision of an owner as well as applying the best knowledge of authors for the first time to the Exhaustive CHAID and QUEST decision tree models for go/no-go decisions.
12. Limitations and Future Research
This research can be updated periodically to capture the latest market conditions and risk factors. Additionally, several data-mining techniques, such as CRUSE and MARS algorithms, can be examined and compared to the proposed models in this paper. Finally, software algorithms based on QUEST and Exhaustive CHAID growing algorithms can be developed for the proposed go/no-go decision models in this study.
Author Contributions
Conceptualization, M.G. and H.M.A.L.; Methodology, M.G. and H.M.A.L.; Software, H.M.A.L.; Validation, M.G. and H.M.A.L.; Formal analysis, M.G. and H.M.A.L.; Investigation, M.G. and H.M.A.L.; Resources, H.M.A.L.; Data curation, H.M.A.L.; Writing—original draft preparation, M.G. and H.M.A.L.; Writing—review and editing, M.G. and H.M.A.L.; Visualization, H.M.A.L.; Supervision, M.G.; Project administration, M.G. All authors have read and agreed to the published version of the manuscript
Funding
This research received no external funding.
Institutional Review Board Statement
Not applicable.
Informed Consent Statement
Not applicable.
Data Availability Statement
All data, models, and code generated or used during the study are available from the corresponding author by request.
Conflicts of Interest
The authors declare no conflict of interest.
Appendix A. Basic Personal and Company Information
- 1.
How many years have you worked in the construction sector?
![Sustainability 13 00815 i001]()
6–10
![Sustainability 13 00815 i001]()
16–20
- 2.
Which of the following describes your company sector?
![Sustainability 13 00815 i001]()
Private Owner
![Sustainability 13 00815 i001]()
Public Owner
- 3.
Which of the following describes your company size (number of employees)?
![Sustainability 13 00815 i001]()
1–50
![Sustainability 13 00815 i001]()
50–100
![Sustainability 13 00815 i001]()
101–500
![Sustainability 13 00815 i001]()
More than 500
- 4.
Which of the following describes your company’s work volume in the last 5 years (USD)?
$1–5 million
$6–50 million
$51–100 million
$101–500 million
![Sustainability 13 00815 i001]()
More than
$500 million
- 5.
At which stage of the project lifecycle do you believe the final go/no-go decision should be performed?
![Sustainability 13 00815 i001]()
At the end of Conceptual Design
![Sustainability 13 00815 i001]()
At the end of Preliminary Engineering
![Sustainability 13 00815 i001]()
At the end of Final Design
Appendix B. Risk Factors Affecting Go/No-Go Decisions in the Early Stages of the Project, before Execution
How important do you think the following risk factors affect go/no-go decisions in the early stage of the project, before execution? (Please mark only one answer per row). |
GNG Risk Factors | Level of Importance on execution decision 1 = Very Low, 2 = Low, 3 = Medium, 4 = High, 5 = Very High |
1 | 2 | 3 | 4 | 5 |
Organizational Risk Factors (Owner, CM, designer, planner, contractor) | | | | | |
1. Financial stability of the owner | | | | | |
2. Consultants’ and suppliers’ reliability and experience in construction | | | | | |
3. Design errors and omissions (Rush design) | | | | | |
4. Qualification of designers and planners | | | | | |
5. Availability of skilled labor | | | | | |
6. Availability of reliable and experienced contractors | | | | | |
Project/Technical Risk Factors | | | | | |
1. Availability of resources (materials and equipment) | | | | | |
2. Geological conditions of the construction site | | | | | |
3. Availability of construction technologies and/or skills | | | | | |
4. Size and location of the project | | | | | |
5. Safety level required | | | | | |
6. Complexity of the design and scope | | | | | |
7. Site space constraints | | | | | |
8. Tight schedule | | | | | |
Legal Risk Factors | | | | | |
1. Excessive approval procedures in administrative departments | | | | | |
2. Specifications and standards required | | | | | |
3. Dispute settlement procedure | | | | | |
Financial and Economic Risk Factors | | | | | |
1. Underestimated budgeting | | | | | |
2. Inflation and deflation | | | | | |
3. Price of raw materials | | | | | |
4. Expected return level/Project profitability | | | | | |
5. High overhead costs. | | | | | |
6. Forecast about market demand/Potential level of competition | | | | | |
Appendix C. Scenario of Go/No-Go Decision
How does your company/organization take go-decisions in the early stages, before execution, in different scenarios. Please click on the go-decision (go/no-go) below for different scenarios.
Form (A)
Scenario | Organizational Risk | Project/Technical Risk | Legal Risk | Financial/Economic Risk | Go Decision |
GO | /NO-GO |
1 | Low | High | Medium | Medium | | |
2 | High | Low | Low | High | | |
3 | Medium | Medium | Medium | Low | | |
4 | High | Medium | Medium | Medium | | |
5 | High | Low | Medium | Medium | | |
6 | Low | Medium | High | Medium | | |
7 | Low | Low | High | Medium | | |
8 | Low | Medium | High | Low | | |
9 | Low | Medium | Low | Medium | | |
10 | High | High | Medium | Low | | |
11 | High | High | Medium | Medium | | |
12 | Medium | High | Medium | Low | | |
13 | Low | Medium | High | High | | |
14 | High | Low | Medium | High | | |
15 | Medium | Low | High | Low | | |
16 | Medium | Medium | Low | High | | |
17 | Medium | Low | Low | High | | |
18 | Medium | Medium | Medium | Medium | | |
19 | Low | Medium | Low | Low | | |
20 | Low | High | High | High | | |
21 | High | Medium | High | Medium | | |
22 | Low | Medium | Low | High | | |
23 | High | High | High | High | | |
24 | Medium | Low | Medium | Medium | | |
25 | Medium | High | Low | High | | |
26 | Low | Low | Low | Medium | | |
27 | Medium | High | High | Low | | |
28 | High | Low | Low | Medium | | |
29 | Medium | Medium | High | Medium | | |
30 | High | High | Medium | High | | |
31 | High | High | Low | High | | |
32 | Medium | High | Medium | High | | |
33 | High | Medium | Low | Low | | |
34 | Medium | Low | Medium | High | | |
35 | Medium | High | Low | Medium | | |
36 | Medium | Low | Low | Low | | |
37 | High | High | Low | Low | | |
38 | Medium | High | High | Medium | | |
39 | High | Low | Low | Low | | |
40 | Medium | Medium | Medium | High | | |
References
- Viswanathan, S.K.; Jha, K.N. Critical risk factors in international construction projects: An Indian perspective. Eng. Constr. Archit. Manag 2020, in press. [Google Scholar] [CrossRef]
- Nawaz, A.; Waqar, A.; Shah, S.A.R.; Sajid, M.; Khalid, M.I. An Innovative Framework for Risk Management in Construction Projects in Developing Countries: Evidence from Pakistan. Risks 2019, 7, 24. [Google Scholar] [CrossRef] [Green Version]
- Firmansyah, B.A.; Veronika, A.; Trigunarsyah, B. Risk analysis in feasibility study of building construction project: Case study—PT. Perusahaan Gas Negara Indonesia. In Proceedings of the Tenth East Asia-Pacific Conference on Structural Engineering and Construction, Bankok, Thailand, 3–5 August 2006. [Google Scholar]
- Chillingworth, L. Innovation and the Feasibility Formula™. Ph.D. Thesis, University of Szent István, Gödöllő, Hungary, 2015. [Google Scholar]
- Elhassan, S.M.; Zawawi, N.A.; Ghazali, Z.B. Decision Making Framework for Optimizing Construction Management Objectives: A review. Economics 2012, 23, 285–294. [Google Scholar]
- Jónsson, H. Feasibility Analysis Procedures for Public Projects in Iceland. Ph.D. Thesis, Reykjavík University, Reykjavík, Iceland, 2012. [Google Scholar]
- Kim, S.-C.; Yoon, J.-S.; Kwon, O.-C.; Paek, J.-H. Feasibility Analysis Simulation Model for Managing Construction Risk Factors. J. Asian Arch. Build. Eng. 2005, 4, 193–200. [Google Scholar] [CrossRef] [Green Version]
- Taylor, M.D.; Wamuziri, S.C.; Smith, I.G.N. Analysis of Risk in Construction Automation Investment; School of the Built Environment: Edinburgh, UK, 2000. [Google Scholar]
- Han, S.H.; Diekmann, J.E. Approaches for Making Risk-Based Go/No-Go Decision for International Projects. J. Constr. Eng. Manag. 2001, 127, 300–308. [Google Scholar] [CrossRef]
- Ock, J.H.; Han, S.H.; Park, H.K.; Diekmann, J.E. Improving decision quality: A risk-based go/no-go decision for build–operate–transfer (BOT) projects. Can. J. Civ. Eng. 2005, 32, 517–532. [Google Scholar] [CrossRef]
- Won, J.W.; Lee, C.; Jang, W.; Han, S.H. A Two-Stage Model to Support Go/No-Go Decision Making in the International Construction Market. In Proceedings of the Creative Construction Conference, Budapest, Hungary, 25–28 June 2016. [Google Scholar]
- Chen, C.; Yu, J.; Yan, T. Modeling the Go /No-Go Investment Decision for China’s PPP Projects Based on Fuzzy Preference Relations. Destech Trans. Soc. Sci. Educ. Hum. Sci. 2017. [Google Scholar] [CrossRef] [Green Version]
- Utama, W.P. The Overseas Expansion of Indonesian Contractors: Motivations, Entry Modes Choice and a Neuro Fuzzy Based Decision Support System. Ph.D. Thesis, The Hong Kong Polytechnic University, Hong Kong, China, 2018. [Google Scholar]
- Chuang-Stein, C.; Kirby, S.; French, J.; Kowalski, K.; Marshall, S.; Smith, M.K.; Bycott, P.; Beltangady, M. A Quantitative Approach for Making Go/No-Go Decisions in Drug Development. Innov. Regul. Sci. 2011, 45, 187–202. [Google Scholar] [CrossRef]
- Bahamid, R.A.; Doh, S.I.; Al-Sharaf, M.A. Risk factors affecting the construction projects in the developing countries. IOP Conf. Ser. Earth Environ. Sci. 2019, 244, 012040. [Google Scholar] [CrossRef]
- Abd-Eltawab, A.I.; Kandil, K.A.; Hussein, G.; Badawy, M. Modelling Risks of Road Construction in Real Estate Projects. Int. Res. J. Eng. Technol. 2018, 5, 1025–1031. [Google Scholar]
- El-Karim, M.S.B.A.; El Nawawy, O.A.M.; Abdel-Alim, A.M. Identification and assessment of risk factors affecting construction projects. HBRC J. 2017, 13, 202–216. [Google Scholar] [CrossRef]
- Yucelgazi, F.; Yitmen, I. Risk Assessment for Large-Scale Transport Infrastructure Projects. In Proceedings of the 3rd World Multidisciplinary Civil Engineering, Architecture, Urban Planning Symposium (WMCAUS 2018), Prague, Czech Republic, 18–22 June 2018; Institute of Physics (IOP): London, UK, 2019; Volume 471, pp. 1–10. [Google Scholar]
- Amoatey, C.T.; Ameyaw, Y.; Adaku, E.; Famiyeh, S. Analysing delay causes and effects in Ghanaian state housing construction projects. Int. J. Manag. Proj. Bus. 2015, 8, 198–214. [Google Scholar] [CrossRef]
- Bagaya, O.; Song, J. Empirical study of factors influencing schedule delays of public construction projects in Burkina Faso. J. Manag. Eng. 2016, 32, 05016014. [Google Scholar] [CrossRef]
- Sharaf, M.M.M.; Abdelwahab, H.T. Analysis of Risk Factors for Highway Construction Projects in Egypt. J. Civ. Eng. Arch. 2015, 9, 526–533. [Google Scholar]
- Issa, U.H.; Farag, M.A.; Abdelhafez, L.M.; Ahmed, S.A. A Risk Allocation Model for Construction Projects in Yemen. Civ. Environ. Res. 2015, 7, 78–89. [Google Scholar]
- Zidane, Y.J.-T.; Andersen, B. The top 10 universal delay factors in construction projects. Int. J. Manag. Proj. Bus. 2018, 11, 650–672. [Google Scholar] [CrossRef]
- Kishan, P. A Study of Risk Factors Affecting Building Construction Projects. Int. J. Eng. Res. Technol. 2014, 3, 831–835. [Google Scholar]
- Asadi, S.S.; Rao, V. An Integrated Approach to a Critical Analysis of Risk Management in Construction Projects. Int. J. Civ. Eng. Technol. 2018, 9, 20–28. [Google Scholar]
- Singh, M.K.; Deep, S.; Banerjee, R. Risk management in construction projects as per Indian scenario. Int. J. Civ. Eng. Technol. 2017, 8, 127–136. [Google Scholar]
- Al-Hazim, N.; Abu Salem, Z.; Ahmad, H. Delay and Cost Overrun in Infrastructure Projects in Jordan. Procedia Eng. 2017, 182, 18–24. [Google Scholar] [CrossRef]
- Sharafi, P.; Rashidi, M.; Samali, B.; Ronagh, H.; Mortazavi, M. Identification of Factors and Decision Analysis of the Level of Modularization in Building Construction. J. Arch. Eng. 2018, 24, 04018010. [Google Scholar] [CrossRef]
- Jarkas, A.M.; Mubarak, S.A.; Kadri, C.Y. Critical Factors Determining Bid/No Bid Decisions of Contractors in Qatar. J. Manag. Eng. 2014, 30, 05014007. [Google Scholar] [CrossRef]
- Zou, P.X.; Zhang, G.; Wang, J.Y. Identifying Key Risks in Construction Projects: Life Cycle and Stakeholder Perspectives. Int. J. Constr. Manag. 2014, 9. [Google Scholar] [CrossRef]
- Kadry, M.; Osman, H.; Georgy, M. Causes of Construction Delays in Countries with High Geopolitical Risks. J. Constr. Eng. Manag. 2017, 143, 04016095. [Google Scholar] [CrossRef]
- Horine, G. Absolute Beginner’s Guide to Project Management, 2nd ed.; Que Publishing: Indianapolis, IN, USA, 2009. [Google Scholar]
- Diab, M.F.; Varma, A.; Nassar, K. Using risk assessment to improve highway construction project performance. In Proceedings of the 48th ASC Annual International Conference, Birmingham, UK, 11–14 April 2012. [Google Scholar]
- Gavit, S.; Pitroda, J.; Makwana, A.H. Risk Management in High Rise Construction Projects in Surat City. Constr. Manag. Approach 2019, 5, 160–167. [Google Scholar]
- Sakthiganesh, R.; Suchithra, S.; Saravanakumar, S. A Study on Critical Risk Factors involved Life Cycle of construction Projects. Int. J. Eng. Technol. Manag. Appl. Sci. 2017, 5, 32–36. Available online: http://www.ijetmas.com/admin/resources/project/paper/f201703071488864063.pdf (accessed on 14 January 2021).
- Gondia, A.; Siam, A.; El-Dakhakhni, W.; Nassar, A.H. Machine Learning Algorithms for Construction Projects Delay Risk Prediction. J. Constr. Eng. Manag. 2020, 146, 04019085. [Google Scholar] [CrossRef]
- Jang, W.; Lee, J.K.; Lee, J.; Han, S.H. Naive Bayesian Classifier for Selecting Good / Bad Projects during the Early Stage of International Construction Bidding Decisions. Math. Probl. Eng. 2015, 2015. [Google Scholar] [CrossRef] [Green Version]
- Wu, Z.; Nisar, T.M.; Kapletia, D.; Prabhakar, G.P. Risk factors for project success in the Chinese construction industry. J. Manuf. Technol. Manag. 2017, 28, 850–866. [Google Scholar] [CrossRef]
- Mishra, A.K.; Mallik, K. Factors and Impact of Risk Management Practice on Success of Construction Projects of Housing Developers, Kathmandu, Nepal. Int. J. Sci. Basic Appl. Res. 2017, 36, 206–232. [Google Scholar]
- Hastak, M.; Shaked, A. ICRAM-1: Model for International Construction Risk Assessment. J. Manag. Eng. 2000, 16, 59–69. [Google Scholar] [CrossRef]
- Chua, D.K.H.; Wang, Y.; Tan, W.T. Impacts of Obstacles in East Asian Cross-Border Construction. J. Constr. Eng. Manag. 2003, 129, 131–141. [Google Scholar] [CrossRef]
- Liu, J.; Zhao, X.; Yan, P. Risk Paths in International Construction Projects: Case Study from Chinese Contractors. J. Constr. Eng. Manag. 2016, 142, 05016002. [Google Scholar] [CrossRef]
- Leka, S.; Caushi, E. Statistical Methods for Identification, Analysis and Prediction of Customer Attrition. Ph.D. Thesis, Catholic University, Albania, 2019. [Google Scholar]
- Rajalakshmi, R.; Uma, M.; Thangadurai, K.; Punithavalli, M. Impact of Datamining Techniques in Forecasting Plant Disease. Int. J. Adv. Res. Comput. Sci. 2012, 3, 187–190. [Google Scholar]
- Magesh, N.; Thangaraj, P.; Sivagobika, S.; Praba, S.; Priya Mohana, R. Evaluating The Performance of an Employee Using Decision Tree Algorithm. Int. J. Eng. Res. Technol. 2013, 2, 2814–2830. [Google Scholar]
- Vandamme, J.-P.; Meskens, N.; Superby, J.-F. Predicting Academic Performance by Data Mining Methods. Educ. Econ. 2007, 15, 405–419. [Google Scholar] [CrossRef]
- Ali, M.; Eyduran, E.; Tariq, M.M.; Tirink, C.; Abbas, F.; Bajwa, M.A.; Baloch, M.H.; Nizamani, A.H.; Waheed, W.A.; Awan, M.A.; et al. Comparison of Artificial Neural Network and Decision Tree Algorithms used for Predicting Live Weight at Post Weaning Period from Some Biometrical Characteristics in Harnai Sheep. Pak. J. Zool. 2015, 47, 1579–1585. [Google Scholar]
- Biggs, D.; De Ville, B.; Suen, E. A Method of choosing multiway partitions for classification and decision trees. J. Appl. Stat. 1991, 18, 49–62. [Google Scholar] [CrossRef]
- Novita, R.; Sabariah, M.K.; Effendy, V. Identifying factors that influence student failure rate using Exhaustive CHAID (Chi-square automatic interaction detection). In Proceedings of the 2015 3rd International Conference on Information and Communication Technology (ICoICT), Nusa Dua, Bali, 27–29 May 2015. [Google Scholar]
- Loh, W.-Y. Classification and regression trees. Wiley Interdiscip. Rev. Data Min. Knowl. Discov. 2011, 1, 14–23. [Google Scholar] [CrossRef]
- Chisala, M.L. Quantitative Bid or No-Bid Decision-Support Model for Contractors. J. Constr. Eng. Manag. 2017, 143, 04017088. [Google Scholar] [CrossRef]
- Wanous, M.; Boussabaine, A.H.; Lewis, J. Tendering factors considered by Syrian contractors. In Proceedings of the 14th annual ARCOM conference, Reading, UK, 9–11 September 1998; pp. 535–543. [Google Scholar]
- Gunduz, M.; Nielsen, Y.; Ozdemir, M. Quantification of Delay Factors Using the Relative Importance Index Method for Construction Projects in Turkey. J. Manag. Eng. 2013, 29, 133–139. [Google Scholar] [CrossRef]
- Faridi, A.S.; El-Sayegh, S.M. Significant factors causing delay in the UAE construction industry. Constr. Manag. Econ. 2006, 24, 1167–1176. [Google Scholar]
- Ostertagová, E.; Ostertag, O.; Kováč, J. Methodology and application of the Kruskal-Wallis test. In Applied Mechanics and Materials; Trans Tech Publications: Zurich, Switzerland, 2014; Volume 611, pp. 115–120. [Google Scholar]
- Abdi, H.; Williams, L.J. Newman-Keuls test and Tukey test. In Encyclopedia of Research Design; Sage: Thousand Oaks, CA, USA, 2010; pp. 1–11. [Google Scholar]
| Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).