Riesgo: A Knowledge-Based Qualitative Risk Assessment System for PPP Projects

Abstract

A successful public-private partnership (PPP) relies heavily on effective risk assessment, given the intricate risk factors and contractual arrangements involved. While quantitative risk assessment methods have received significant attention in the PPP literature, qualitative risk assessment, the sector’s predominant preference, remains underexplored, causing a low level of applicability of academic studies and indicating a noticeable research gap. A qualitative risk assessment tool prototype, Riesgo, is developed in this paper as a customizable, knowledge-based digital risk register incorporating a pre-defined template that guides users using PPP risk factors, compensation and mitigation options, project information requirements, and risk register items. This paper presents the proposed system architecture, explains the research steps adopted in determining the system elements, and delineates the system functions through a use case developed to illustrate the process and information flows. The prototype was verified by 13 PPP experts who employed it for risk assessment, and their feedback was utilized for further development. A validation survey of 21 professionals affirmed Riesgo’s usability and applicability in the industry. The customizable and knowledge-based prototype has the potential to streamline effective risk assessment and guide the users across various PPP phases, such as early risk assessment, feasibility studies, contract preparation, and monitoring.

1. Introduction

A public-private partnership (PPP) is a contract established between public and private parties, wherein the project is financed and delivered for public services through a capital asset by the private party (OECD, 2022) [1]. The public party can be a government entity such as a ministry, municipality, or state-owned enterprise, and the private party may be a company possessing technical expertise and investment capacity to achieve project execution (Felsinger, 2008) [2]. Common features of PPP projects, such as including many stakeholders with different goals, complicated contracts between parties, long project durations, and broad project scopes, increase the risks in PPP projects (Kuru and Artan, 2020) [3]. For the successful completion of PPPs, the crucial task is to ensure that these risks are adequately assessed and transferred to the party capable of managing them most effectively (OECD, 2012) [4]. The risk management standard ISO 31000 (2009) [5] depicts risk assessment as the combination of processes involving risk identification, analysis, and evaluation, which is a frequently examined topic in the PPP literature (Osei-Kyei, 2023) [6]. The academic interest in this area arises from the industry’s experiences with negative impacts of risks. The Suez Canal Project, recognized as the first build–operate–transfer PPP project in the modern world (Levy, 1996) [7], encountered a substantial cost overrun of 1900%, and factors such as the inherently risky structure of the PPP and misinformation about the risks were cited as causes of this adverse impact (Flyvbjerg, 2014) [8]. Similar issues are extensively reported in the literature, for instance, Li and Wang (2023) [9] empirically demonstrated that inflation risk significantly influences project failures. Castelblanco et al. (2023) [10] highlighted the intricate correlation between the project risk profile and the concession period, revealing that risk assessment is critical for PPPs.

Following the identification of risks, qualitative or quantitative tools and techniques are employed for risk assessment in PPP projects. Various models have been proposed in prior studies, the majority of which predominantly incorporate quantitative methods. A risk assessment framework for PPP megaprojects using structural equation modeling was developed by Ahmadabadi and Heravi (2019) [11]. Liu and Sun (2020) [12] presented a PPP risk assessment method based on the improved matter element model. Moradi Shahdadi et al. (2023) [13] utilized the measurement alternatives and ranking according to the compromise solution (MARCOS) technique to evaluate risk factors in water and wastewater PPPs in Iran. Wang et al. (2023) [14] established a best–worst multi-criteria decision-making and comprehensive fuzzy evaluation model for optimizing risk assessment in waste recycling PPPs. Nevertheless, it is expressed by the World Bank (2017) [15] that implementing agencies typically prefer to apply qualitative techniques and tools for risk assessment during PPP appraisal in the industry. For this purpose, risk matrices and risk registers are utilized to assess and allocate PPP risks (World Bank, 2008; Global Infrastructure Hub, 2019) [16,17]. Qualitative risk assessment requires an extensive collection of information regarding PPP project risks. However, as the risk matrices and risk registers used in the current practice utilize limited information as input, misinterpretation issues and improper risk allocation are very common due to insufficient information or unavailable critical details regarding PPP projects (Tariq and Zhang, 2020) [18]. Hence, for adequate risk management, it is necessary to determine the critical information items and assessment requirements for PPP risks and incorporate these into a more comprehensive risk register model. Moreover, the current risk matrices and registers are standardized documents that are unsuitable for customization based on project requirements. The customization of risk assessment to the specific attributes of projects in the PPP sector is recommended by the Global Infrastructure Hub (2019) [16]. However, the risk matrix provided by the Global Infrastructure Hub (2019) [16] also adheres to a standard structure, and practical revision of this matrix according to the needs of a particular PPP project is not feasible. The PPP risk assessment process in different project phases can be enriched by a customizable risk register, offering users additional functions and more comprehensive utilization of project information.

The relationship between risk registers, risk matrices, and contracts is emphasized in the literature. Farquharson et al. (2011) [19] express that the recording of risk assignments, which will be implemented in the PPP contract, can be accomplished using risk registers and matrices. This contributes to clarifying any aspect related to risks in the contract, and an agreement is reached among the parties regarding actions to be taken for the risks. The connection between the contract and risks is also underscored by the World Bank (2017) [15] suggesting that the determined risk allocation should be incorporated into the PPP contract by enhancing it with additional information. The importance of practical risk assessment models for successful PPP contract formation and implementations is also emphasized by previous researchers (Xu et al., 2010; Chan et al., 2011) [20,21]. However, despite the sector’s predominant preference for qualitative and expedient solutions, the tools and techniques explored in prior studies are mostly quantitative, and a comprehensive digital risk register tool for PPP risk assessment is lacking in the literature. As a result of this neglect in the current studies of the needs of the PPP sector, the level of transition of academic studies to the industry is deemed low. Bruneel et al. (2010) [22] state that a lack of understanding of working practices and sectoral expectations acts as a barrier to the practical applicability of academic research. Furthermore, the complexity of the quantitative risk assessment models developed in the literature hinders their usability by PPP professionals (Chan et al., 2011) [20]. As the result of their systematic literature review, Rasheed et al. (2022) [23] stated that despite the proliferation of quantitative risk management models in the PPP literature, qualitative methods are preferred in practice, and that the development of practical knowledge-based models that can guide users through the PPP risk assessment process is essential to overcome this problem. Jiang et al. (2023) [24] also underlined the need for adequate information management for the assessment of PPP risks. The main shortcomings in PPP risk assessment identified as a result of the presented review include (1) the current tools utilize very limited information and lack customization, (2) the complexity of the existing risk assessment models lack a knowledge base and practical guidance, and (3) academic studies are mainly grounded in quantitative or semi-quantitative approaches, despite the sector’s needs for qualitative tools.

This study aims to present a qualitative risk assessment tool prototype, Riesgo, which is a digital model designed to advance the risk assessment practices in PPP projects. Addressing critical deficiencies highlighted in the current models, Riesgo offers a customizable, knowledge-based qualitative risk assessment system explicitly tailored for PPPs. By rectifying shortcomings such as limited information utilization, lack of customization, and the absence of practical solutions based on sectoral needs, Riesgo aims to bridge the gap between theoretical PPP risk assessment models and real-world applications. By guiding PPP professionals in the utilization of the required PPP information and risk register items, the proposed prototype enhances information management in PPP projects. Moreover, by providing knowledge of PPP risk factors, as well as compensation and mitigation options to manage these risks, Riesgo offers the users practical solutions to support the risk assessment process. Riesgo is developed to be used by the public party and its consultants in various project phases, with the objectives of (1) assessing project risks during the early phases of the project, (2) assessing the risks that will be addressed in the contract, (3) assessing the risks inherent the contracts before project commencement, and (4) monitoring project risks throughout the construction and operation stages in ongoing projects. As the model includes information items applicable to all types of PPPs, regardless of project location, it was developed with a generic approach targeting all PPP projects. The processes detailed in this paper include (1) identifying risk register items and system requirements for the prototype, (2) developing of the system architecture by incorporating PPP risk factors, PPP information requirements, risk register items, and system functions into the model, and (3) conducting verification and validation processes for the prototype.

2. Literature Review

2.1. Previous Studies on PPP Risk Assessment and the Literature Gap

The bibliometric analysis of the PPP literature reveals that risk assessment ranks among the top ten keywords noted, based on occurrence frequency (Azarian et al., 2023) [25]. Numerous studies focus on PPP risk assessment using diverse scopes and techniques. Regarding scope, some studies concentrate solely on assessing specific risk factors, without presenting a model encompassing the entirety of the project’s risks. Liu et al. (2017) [26] presented a framework for the assessment of revenue risk in PPPs. The performance risk of PPPs was the focus of Chen et al. (2020) [27], who integrated structural equation modeling and fuzzy cognitive mapping. Liu et al. (2021) [28] utilized Bayesian Network calculations to assess the construction risks in urban rail transit PPPs in China. Alasad and Motawa (2015) [29] developed a system dynamics model for assessing demand risk in PPP road projects. Sun et al. (2023) [30] assessed the investment risk of urban rail transit PPPs using the system dynamics approach. On the other hand, the three types of techniques used in the risk assessment literature include qualitative, quantitative, or semi-quantitative approaches (Chinyio, 2003) [31], and numerous models have been developed in the literature using these approaches. Monte Carlo simulation is often employed as a quantitative approach in financial risk assessment models (Kumar et al., 2018) [32]. Quantitative risk assessment models are widely utilized in the calculation and assessment of various values or formulas (e.g., demand guarantees/volume thresholds, tariff indexation) included in the PPP contracts (ADB et al., 2016) [33]. However, PPP contracts also contain complicated verbal details concerning the qualitative aspects of the projects, which may be overlooked in the quantitative models. Therefore, using solely quantitative models may not be sufficient to conduct a comprehensive risk assessment and comprehend generic project risks, and they are better suited as contributory tools. As the second approach, semi-quantitative studies incorporate linguistic statements, yet they lean towards a more mathematical perspective (Xu et al., 2012, Wu et al., 2018, Niazmandi et al. 2024) [34,35,36]. Niazmandi et al. (2024) [34] proposed a PPP risk and productivity assessment model by using a fuzzy interference system. Another example is the risk assessment model developed by Xu et al. (2012) [36]. The Delphi survey and the fuzzy set theory were preferred for the weighting and membership functions of the risk factors, and fuzzy synthetic evaluation was then applied. Thus, both the qualitative and quantitative steps are encompassed. However, PPP professionals’ unfamiliarity with the techniques employed in semi-quantitative studies hinders the practical application of these models in the industry (Chan et al., 2011) [20].

The third and final approach involves the qualitative technique. The recent studies employing qualitative research primarily involve probability and impact-based risk assessment and ranking studies (Ameyaw and Chan, 2015; Osei-Kyei and Chan, 2017; Kukah et al., 2024) [37,38,39]. Ranking the risks aids in identifying the most significant project risks, allowing PPP professionals to focus more on these critical areas. However, conducting risk analysis using solely probability and impact assessment may result in overlooking noteworthy information regarding risks in PPPs. For instance, the unprecedented stress of the COVID-19 pandemic on PPPs presented challenges. While it was anticipated that contractual provisions related to force majeure could be linked to the pandemic risk, further uncertainties concerning the applicability of this provision were encountered in practice (Casady and Baxter, 2020) [40]. In such circumstances, the assessment remains incomplete if the force majeure risk is evaluated solely through probability-impact assessment and lacks enrichment with more qualitative information. Further qualitative analysis is needed to assess the conditions required for the pandemic risk to fall within the scope of the force majeure risk and to determine the compensation mechanisms that can be applied in the case of a pandemic. Ward (1999) [41] and Taroun, (2014) [42] also underlined the inadequacy of probability and impact-based assessment processes for a comprehensive qualitative risk evaluation.

In addition to academic studies; various models, tools, and techniques are also developed for risk assessment in the industry. For example, a quantitative fiscal risk assessment model known as PFRAM was presented by the IMF and World Bank Group (2022) [43]. PFRAM is an Excel-based fiscal model designed for use by public authorities, such as PPP units and public corporations. The World Bank (2008) [17] offers a qualitative tool in the form of a risk matrix, serving as a risk assessment and allocation tool. This matrix encompasses information on risk groups, risk descriptions, cost drivers, risk allocation, and treatment recommendations. Similarly, the Global Infrastructure Hub (2019) [16] developed risk matrices for PPPs that include risk factors, preferred risk allocation, mitigation measures, actions, and recommendations based on the type of market in which the project is conducted. Omurzakova (2022) [44] suggests that a risk matrix can be employed during project assessment to determine risk significance, relationship, and priority. Another qualitative tool is the risk register, as emphasized by Williams (1994) [45], who states that the risk register functions as an information repository and usually contains more fields for recording information when compared to a risk matrix. Due to its simplicity, the risk register is often preferred in various sectors. For instance, the risk register used by the U.S. Department of Transportation is the agency’s most commonly employed risk management tool (Curtis and FHWA, 2012) [46]. Farquharson et al. (2011) [19] recommend continuous review and monitoring of PPP risks with the assistance of risk registers and risk matrices developed during the project preparation. Since PPPs typically have long durations, the risk registers and matrices allow for their ongoing review and update during the project (O’Har et al., 2017) [47].

Despite their suitability for PPPs, there are crucial areas for improvement in PPP risk registers and matrices. Firstly, these tools often include only standard items (e.g., risk description, compensation, mitigation). There is a potential to enhance the items regarding risk registers and collect more detailed information about the risks. Secondly, despite the need to develop such tools for PPPs, the topic has not been extensively studied in academic literature. A search on the Scopus database for “Search Within the Article Title: public-private partnership” and “Search Within All Fields: register” and “Search Within the Abstract: risk assessment” returns only one study for PPPs (Mazher et al., 2018) [48]. Moreover, in this study, the term “register” refers only to the “list of risk factors” and does not include a qualitative assessment tool. It is evident that PPP risk assessment based on the risk register approach has been neglected. This identified gap in both the literature and industry emphasizes the need for a comprehensive qualitative risk assessment tool based on the detailed risk register approach. Therefore, there is a need for a (1) qualitative, (2) knowledge-based, and (3) customizable risk assessment tool developed with the risk register/matrix approach. It is anticipated that a model developed in line with this approach would exhibit greater industry applicability compared to those found in other studies in the literature.

2.2. Risk Register Items in the Literature and System Functions Used in the Previous Tools

In the scope of developing a digital qualitative risk assessment system prototype, the risk register items and system functions were determined. The term “risk register item” is used to describe the information that can be held and stored in a risk register (Ward, 1999; Patterson and Neailey, 2002) [41,49]. A literature review was undertaken to identify the risk register items to be used in the prototype. System functions were determined based on the analysis of previous tools developed for risk assessment in various domains.

During the determination of risk register items, a query was executed for the term “risk register” and “public-private partnership” in the abstract, resulting in only three relevant results (Ke et al., 2010; Mazher et al., 2018; Erfani et al., 2021) [48,50,51] in the Scopus database. In these studies, the term was used to refer to a list of risk factors, but they did not provide detailed sample risk registers or typical items found in registers, rendering them unsuitable for this research. Subsequently, the search was broadened to include sources regarding risk registers, without specifying the “public-private partnership” keyword. This choice was made on the premise that items in risk registers used in other sectors can also be applicable in PPP registers. A query was also made for the risk matrix term. Filtering “risk matrix” and “public-private partnership” in the abstract resulted in eight articles meeting these criteria. While half of these studies used the term “risk matrix” in the context of this paper, they did not provide sample risk matrices or information about the items in a typical risk matrix. Consequently, instead of academic studies, the review primarily considered sectoral documents from sources such as the NDFA (2007), the World Bank (2008), and the Global Infrastructure Hub (2019) for insights into risk matrices [16,17,52]. Since risk matrices generally contain fewer items than risk registers, and their items are more standardized, the number of references for determining risk matrices was relatively low compared to the references reviewed for risk register items. The risk register items and their references are presented in

Table 1. Risk register items found in the literature.

	Risk Register-Based Studies													Risk Matrix
														Based Studies
Risk Register Items	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18
Risk ID
Risk contract clause ID
Category (Type)
Project phase
Description
Probability score
Impact score
Probability x Impact
Impact type
Risk cost impact
Risk time impact
Strategy
Allocation
Risk owner
Mitigation/Prevention
Compensation
Causal risk (cause, source)
Consequential risk
Residual risk
Secondary risk
Author
Risk entry date
Risk revision date
Risk status

(1) Williams (1994) [45]; (2) Ward (1999) [41]; (3) Patterson and Neailey (2002) [49]; (4) Sillars and O’Connor (2009) [54]; (5) Dunovic et al. (2013) [53]; (6) Govan and Damnjanovic (2016) [55]; (7) Leva et al. (2017) [56]; (8) Ou-Yang and Chen (2017) [57]; (9) Ahmad et al. (2018) [58]; (10) Castro-Nova et al. (2018) [59]; (11) Hossain et al. (2018) [60]; (12) Richards (2018) [61]; (13) PMI (2021) [62]; (14) Grimsey and Lewis (2004) [63]; (15) National Development Finance Agency NDFA (2007) [52]; (16) World Bank (2008) [17]; (17) Fantozzi et al. (2014) [64]; (18) Global Infrastructure Hub (2019) [16]. Colored fields represent the presence of the risk register item in the related reference.

. References 1 to 13 come from studies related to risk registers, while the remaining references (14 to 18) pertain to risk matrices. The references included in the table contain a risk register/risk matrix or comprise the items that can be seen in a risk register or risk matrix. The table contains both PPP-related references and references from outside the PPP literature. This is due to the scarcity of studies on the risk registers within the PPP field, necessitating the exploration of studies in other construction or project management domains (Dunovic et al., 2013; Patterson and Neailey, 2002) [49,53].

Some references listed in Table 1 contain a risk register that aligns with the objectives of the research. Govan and Damnjanovic (2016) [55] conducted a case study on a compressor station and pipeline project, and they presented the risk register used in their research. Ahmad et al. (2018) [58] explored the impact of building information modeling on risk management, providing a post-BIM-based risk register of the project. Ou-Yang and Chen (2017) [57] identified and analyzed risks in a petrochemical plant project in Taiwan, presenting their risk register findings. On the other hand, some of the other studies in Table 1 discussed the concept of risk registers, but did not present a risk register for a specific project. Ward (1999) [41] investigated the role of risk registers in risk assessment. Williams (1994) [45] devised an integrated risk management scheme centered around the register approach. Dunovic et al. (2013) [53], Patterson and Neailey (2002) [49], and O’Har et al. (2017) [34] studied risk register development. In addition to the risk register-based references, Table 1 also consists of risk matrix-related references, and all those references contain risk matrices in varying degrees of detail. While Fantozzi et al. (2014) [64] used a small risk matrix to assess the risks they studied, the Global Infrastructure Hub (2019) [16] presented a more detailed risk matrix that can be used during the risk assessment of PPPs.

All items in Table 1 were incorporated into the risk register in the proposed prototype, without applying a criterion based on the frequency of occurrence, since all these items contain information useful for PPP project risk assessment. Even though “project phase” is mentioned in only two references, it was included in the prototype’s register because it allows users to classify and filter risks based on the project phase. To illustrate, in a PPP project with numerous risk factors, PPP professionals may prefer to view risks in the construction phase, and the inclusion of the project phase item on the risk register can facilitate filtering the related risks. Another example is the “consequential risk”. Although it was mentioned in only one reference, it can aid in defining the consequences of a PPP risk factor, facilitating a more efficient interpretation of risk factors, and establishing connections to the contract.

Table 2. System functions used in the previous tools.

System Functions	S 1	S 2	S 3	S 4	S 5	S 6	S 7	S 8	S 9	S 10	S 11	S 12	S 13	S 14	S 15	S 16	S 17	S 18	S 19	S 20	S 21	S 22	S 23	S 24	S 25	S 26	S 27	S 28
Templates *
Risk Hierarchy *
Risk Search *
Filters *
Risk Register *
Risk Matrix (Heat Map) *
Risk Visualization *
Risk Ranking *
Relationship Diagram *
Document Library *
Export Options *
Bow-Tie Method
Monte Carlo Simulation
Sensitivity Analysis
Scenario Analysis
Stress Test
Efficient Frontier Analysis
Genetich Algorithm
MS Project Integration
Reviewed systems	(S1) Palisade @Risk [70]; (S2) A1 Tracker [65]; (S3) Active Risk Manager [72]; (S4) Aperitisoft [67]; (S5) Aptien [73]; (S6) Auditboard [74]; (S7) Auditcomply [75]; (S8) Barnowl [76]; (S9) Corporater [77]; (S10) Decision Time [78]; (S11) ERM Essential [79]; (S12) Execview [66]; (S13) Goat [16]; (S14) Hyperproof [80]; (S15) Risky Project Professional [71]; (S16) iRisk [81]; (S17) JCAD Core [82]; (S18) Logic Manager [83]; (S19) Mango [84]; (S20) nTask Manager [85]; (S21) Optial Smart [86]; (S22) Omega 365 Pims Risk Management [87]; (S23) Pirani Riskment Suite [88]; (S24) Project Risk Manager [69]; (S25) SAP Risk Management [68]; (S26) Sectera [89]; (S27) Symbiant [90]; (S28) Dmaze [91].

* System functions that are integrated into the proposed prototype. Colored fields represent the presence of the system function in the reviewed system

presents the system functions utilized in previous risk management software tools. The list of software reviewed for the determination of system functions in the proposed prototype is given in Supplementary Material A. The software systems featured in Table 2 were not chosen from those specifically developed for PPPs, as no specialized software for PPP risk assessment was identified in the academic databases and search engines. Therefore, risk assessment software systems developed in other domains were analyzed to select functions suitable for qualitative risk assessment in PPPs. The software models that featured supporting materials on their websites and possessed at least three functions suitable for risk assessment were added to Table 2, and the system functions requirements table was compiled using 28 software systems. These software systems encompass various scopes and functions, all focused on the management and assessment of risks. The table highlights that heat maps, registers, and visualization features are commonly integrated into most risk management systems. While certain systems like A1 Tracker [65] and Execview [66] primarily offer basic risk assessment functions, others include additional features such as the Risk Hierarchy function in the Apertisoft [67] and SAP Risk Management [68], or the Filters function in Project Risk Manager [69]. Apart from system functions that can be comprehended easily by many users, some systems also incorporate advanced features that may necessitate greater user expertise and knowledge. For instance, Palisade @Risk [70] and Risky Project Professional [71] include functions like Monte Carlo simulation and sensitivity analysis.

The system functions selected for the proposed model are highlighted in bold in Table 2. The selection of these functions is based on their expected contributions to the model. The integration of the “templates” feature allows users to utilize a specific pre-established PPP template with predefined risk factors. The “risk hierarchy” facilitates the easy visualization of project risks within distinct risk groups through a structured hierarchical format. The inclusion of “risk search” and “filters” aims to simplify the finding of searched risks within PPP projects that involve numerous intricate risk factors. Given that the primary objective of the model is to assess project risks, the “risk register” function is chosen to retain the risk information. The “risk matrix (heat map)” and “risk ranking” functions demonstrate the distribution and ranking of risks based on their importance. A “relationship diagram” is integrated into the prototype to aid in understanding risk relations. A “document library” is included to store documents related to the project, and “export options” are added to provide users with Excel and PDF outputs based on the data entered into the model. All these functions are suitable for the qualitative assessment of PPP risks, given that PPP risks often involve many details, and these functions empower users to define and manage those details with ease. Hence, these functions are integrated into the developed qualitative risk assessment prototype. While the bow-tie method is also ideal for qualitative assessment, it was omitted from the prototype since the relationship diagram serves a similar function. The remaining system functions in Table 2, such as Monte Carlo simulation and sensitivity analysis, are suitable for quantitative analysis; hence, they were not included in the prototype. The selection of system functions was validated via interviews conducted with PPP experts, and the validation process is detailed in Section 6.

3. Methodology

The risk assessment prototype system was developed through a research process comprising three steps: (1) the determination of PPP risk factors that must be analyzed by the public party, (2) the determination of the information requirements for risk assessment by the public party in PPPs, and (3) the determination of risk register items and system functions, the development of the prototype, and validation and verification (

Table 3. Methodology.

Research Step	Activity	Method	Input	Output
Step 1—Presented by Authors [92]	Determination of PPP risk factors that must be analyzed by the public party	Structured literature review	Academic journals, books, sectoral documents	Validated PPP Risk Factors List (Supplementary Material B)
		Validation of the risk factors by the Delphi method using 13 experts	Expert input
Step 2—Presented by Authors [93]	Determination of PPP information requirements for the public party	Literature review	Academic journals, books, sectoral documents	Validated PPP Project Information Requirements List (Supplementary Material C) Validated PPP Financial Information Requirements List (Supplementary Material D)
		Validation of the requirements by Delphi method using 12 experts	Expert input
Step 3 (Current paper)	Determination of the risk register items	Literature review	Academic journals, books, sectoral documents	Risk Register Items Table (Table 1)
	Determination of the risk software functions	Review of the risk software used in the industry	Sectoral software	Risk Software’s Functions Table (Table 2)
	Development of the system architecture and GUI design	Use case	Validated PPP Risk Factors List Validated Project Information Requirements List Validated PPP Financial Information Requirements List Risk Register Items Table Risk Software’s Functions Table	Draft GUI design
	Development of the prototype	Microsoft Winform App, C# Language, Devexpress component	Draft GUI design	Prototype
	Testing the prototype	User acceptance tests	Prototype	Bug reports
	Verification	Use of the software by 13 PPP professionals and subsequent feedback collection	Semi-structured interviews	User feedback
	Validation	Via usability questionnaire administered to 21 users	1–5 Likert usability survey	Average usability scores

).

In the first step of the research, which was previously presented by Authors [92], PPP risk factors that must be analyzed by the public party in PPPs were identified. Initially, the literature was reviewed by examining academic journals, books, and sectoral documents [16,21,37,39,94,95,96,97,98,99,100,101,102,103,104]. The risks were categorized in the PPP risk factors draft list, and a Delphi study was conducted involving 13 experts from the PPP sector for validation. The experts, specializing in assessing PPP risks for the public party, were lawyers, each possessing at least five years of experience regarding PPPs, with a minimum of three PPP project experiences. The group’s average PPP experience duration exceeded 12 years, and the average PPP project count per person surpassed 17 projects (Supplementary Material E). Based on the Delphi results, certain risk factors were excluded from the draft list, and the remaining components were consolidated under the validated risk factors list as the outcome of the first step (Supplementary Material B). The identified PPP risk factors are utilized in the proposed prototype in this paper.

The second step, as presented by Authors [93], involved determining PPP information requirements for the public party. The term “requirements”, described by PMI (2016) [105] as “conditions or capabilities necessary in a product, service, or result”, was used to emphasize the role of requirements in meeting contractual obligations. In this context, information requirements refer to the information that can be recorded or collected regarding the project, contract, and project risks, and that can be used in assessing and monitoring project details and risks. A methodology similar to Step 1 was employed. Initially, a literature review was conducted to identify PPP information requirements, which were then categorized into two groups. The qualitative requirements acquired from the literature [33,61,62,103,106,107,108,109,110,111,112,113,114] were labeled “project information requirements”, while the quantitative factors obtained from [43,63,107,115,116,117], focusing on the project finance aspect of PPPs, were named “financial information requirements”. Subsequently, a Delphi study, involving 12 experts from the PPP sector, was conducted to validate these draft requirements. These experts were different from those in Step 1, but the criteria for selecting them remained the same. The experts were lawyers specializing in assessing PPP risks for the public party, each with a minimum of five years of experience in PPPs and at least three PPP project experiences. The average PPP experience within this group exceeded 11 years, and the average PPP project count per person surpassed 14 projects (Supplementary Material F). Following the Delphi process, some project information requirements and financial information requirements were excluded from the draft lists and the remaining were consolidated into the validated information requirements and financial information requirements list as the outcome of the second step (Supplementary Material C and D). In this paper, the identified project and financial information requirements are utilized in the proposed prototype.

The third step presented in this paper involves the development of the system architecture, as well as the research steps adopted in determining the system elements (i.e., risk register items, system functions, and graphical user interfaces (GUIs)) and delineates the proposed risk assessment process through a use case developed to illustrate the utilization of the prototype. Verification and validation of the proposed prototype by the PPP experts were also undertaken in this step.

Qualitative risk assessment methodology is adopted in the development of the proposed risk assessment process. Academic studies often employ quantitative or semi-quantitative methodologies for PPP risk assessment. Qualitative risk assessment, the sector’s predominant preference (World Bank, 2017) [15], remains underexplored, causing a low level of applicability of academic studies and indicating a noticeable research gap (Rasheed et al., 2022; Jiang et al., 2023) [23,24]. On the other hand, tools used in practice encompass both qualitative and quantitative methods, yet none have been tailored to meet the needs of PPP projects. The proposed prototype, Riesgo, stands out not only by responding to the specific needs of PPPs through its qualitative risk assessment methodology but also by utilizing a knowledge-based approach. The risk register featured in the proposed prototype offers comprehensive information for PPP practitioners. The pre-populated template encompasses PPP risk factors, required information items for risk assessment, comprehensive risk explanations, compensation details, and mitigation recommendations for the identified risks. Moreover, Riesgo is suitable for all PPPs, regardless of contract type, location, or other variables via the customizable structure adopted in its development.

The third step commenced with identifying items typically found in a risk register. A comprehensive literature review was conducted to define the risk register items. Initially, studies related to risk registers and risk matrices for PPPs were reviewed. However, due to the lack of studies for risk registers in the PPP field, studies on the risk registers in the construction and project management field were also reviewed. The identified items for both risk matrices and risk registers are presented in Table 1.

After the identification of risk register items, the next step was the identification of system functions. The prototype’s system functions were determined based on a review conducted on a sectoral software website called Capterra (Capterra, 2023) [118]. Capterra is a digital platform that compiles software listings across various fields and facilitates connections between software vendors and users. The platform offers numerous software options based on selected expertise fields, and filter options are available to find suitable software for various domains. Employing risk management and risk assessment filters, over 300 software options were initially listed. In the process of determining system functions, the websites of the software models were scrutinized. Software models lacking explanatory documents or demo videos showcasing system functions on their websites were excluded. The system functions integrated into the prototype can be seen in Table 2.

After the determination of risk register items and software functions, the subsequent step involved the design of system architecture and graphical user interfaces (GUIs). A use case was developed to illustrate the proposed PPP risk assessment process, and accordingly, digital design software was employed for creating the GUIs (Section 5.2). The validated PPP risk factors list (Supplementary Material B), project information requirements list (Supplementary Material C), financial information requirement list (Supplementary Material D), risk register items (Table 1), and system functions (Table 2) were utilized in designing the algorithm and GUIs. Draft designs were generated, and consequently, the prototype development process was initiated. The prototype was developed as desktop software using the C# language and incorporating Devexpress components.

Upon the completion of the development phase, the researchers conducted user acceptance tests (UAT) to identify any encountered system bugs. These bugs were categorized based on their priority levels. While some bugs were straightforward and related to design aspects, others pertained to the technical facets of the prototype. Bugs influencing more than one prototype module were deemed more critical and were addressed with high priority. After resolving all reported bugs, the system underwent further scrutiny, signifying the comprehensive completion of both the development and testing phases.

For the verification of the developed prototype, semi-structured interviews were conducted with professionals possessing expertise in PPPs, contracts, and project risks. During these interviews, a detailed presentation elucidating the software’s functions was delivered to the professionals. The professionals were also allowed to use the software. Feedback from the professionals was collected through open-ended questions. The interviews incorporated additional questions based on the users’ responses, in addition to pre-defined inquiries. While there is no stringent criterion for determining the sample size in qualitative studies, Boddy (2016) [119] asserts that a sample size of 12 in qualitative research design is adequate for achieving theoretical saturation. A similar sample size is also indicated in various studies (Guest et al., 2006; Hennink and Kaiser, 2022) [120,121]. Data saturation is realized when no novel information or themes emerge in the data after conducting further interviews or case evaluations (Boddy, 2016) [119]. In this study, saturation was achieved after the ninth interview, as the professionals’ responses and recommendations began to replicate. The interviews concluded after the 13th session, given the absence of new feedback for the questions addressing the prototype’s contribution, challenges encountered with the prototype, and recommendations for future developments.

Upon the completion of the verification process, the validation phase was initiated. An online survey form was dispatched to the previous 13 participants and 8 additional participants to validate the prototype. In total, 21 professionals replied to the validation questions. The participant profile is presented in Supplementary Material G. The survey encompassed inquiries regarding the model’s usability and its application in regards to PPPs. The usability questions were derived from the system usability scale (SUS) (Brooke, 1996) [122]. In addition to the usability queries, ten questions about the model’s utilization in PPP projects were incorporated. All questions related to usability and the model’s practicality and applicability were presented on a 1–5 Likert scale, ranging from strongly disagree to strongly agree. A similar sample size was employed in the validation of the prototypes developed within the construction management field in previous studies (Artan et al., 2022) [123]. The results of the validation process are detailed in Table 4.

4. System Architecture and Elements

4.1. System Architecture

The Riesgo prototype is developed as a desktop application. PPP experts utilize project documents and project-related data to complete the pertinent fields within Riesgo (Figure 1). Some menus within the prototype serve primarily as information repositories, aiding users in documenting and monitoring vital project details. Other menus have connections to each other to facilitate an exchange of information among them. These connections will be further elaborated in the information flow section. Based on the user inputs entered into Riesgo during risk assessment, the prototype generates outcomes such as rankings, graphs, pie charts, and visualizations. The information about the risks assessed in Riesgo can be saved, stored in a database, and exported to Microsoft Excel.

4.2. Risk Factors

The Delphi technique was employed to determine the PPP risk factors that must be analyzed by the public party while using the proposed prototype. A total of 83 risk factors were identified and grouped into Land Risks, Financial Risks, Legal and Contractual Risks, Governance Risks, Political Risks, Social Risks, Design Risks, Environmental and Force Majeure Risks, Construction Risks, and Operation and Transfer Risks. The details and methods adopted in the determination of the risk factors using the Delphi technique were presented by Authors [92]. These risks were integrated into the prototype’s template under the Risk Hierarchy menu. Embedded risk factors in the Risk Hierarchy menu are presented in Supplementary Material B. Various implementation modes (e.g., BOT, BOOT, BLT) can be employed for PPPs, each encompassing certain common generic risks. Nonetheless, these modes also entail distinct risks (Akhtar et al., 2023) [124]. Furthermore, the project risk profile can be influenced by the nature and location of the project, and new risks can occur. Hence, “add”, “edit”, and “delete” functions are incorporated into the Riesgo prototype to enable modifications to the predefined risk factors. Thus, risk factors can be updated based on the project characteristics. These features make the prototype dynamic and set it apart from previously described tools, such as risk registers and matrices, providing a distinct advantage.

4.3. Project Information Requirements and Financial Information Requirements

Project information requirements are the information items that need to be collected and entered into the proposed prototype by the users. Since the utilization of limited information was determined as a major drawback of the previous studies, information items required for a comprehensive PPP risk assessment were identified by a detailed process involving the Delphi technique. The details and methods adopted in the determination of the project information requirements using the Delphi technique were presented by Authors [93].

The qualitative information requirements, designated as “Project Information Requirements”, were categorized under Project General Information, Contract General Information, Financial Details, Legal and Technical, Mechanisms and Provisions, and Project General Analysis. Project General Information requirements encompass aspects such as PPP types, scope, description, and stakeholders. Contract General Information pertains to crucial dates for tender and contract, concession period, and expiry date. Financial details address topics like shareholder percentages, loan financing structure, and guarantees. Legal and Technical requirements involve laws, regulations, permits, guideline details, and specifications. Mechanisms and Provisions focus on the types of mechanisms applicable to the contract, such as tariff adjustment mechanisms, revenue sharing mechanisms, dispute resolution mechanisms, and early termination provisions. Finally, Project General Analysis includes requirements like milestones, success criteria, and boundaries. The complete list of project information requirements is presented in Supplementary Material C.

Alongside the provided examples, the quantitative requirements employed to analyze the financial aspect of PPPs were identified as “Financial Information Requirements”. The monitoring of the project’s financial aspect relies on these specific financial values. These requirements have the potential to be incorporated into contracts using threshold values, establishing guidelines for implementing compensation mechanisms and as a result, being linked to project risks. Examples of financial information requirements include demand projection, inflation, net present value, and debt service coverage ratio. The complete list of financial information requirements is presented in Supplementary Material D.

4.4. Risk Register Items

Risk register items were identified through a literature review and are presented in Table 1. These items have been integrated into the Risk Register menu of the prototype. The goal is to facilitate the gathering of diverse information related to the risk factors. Consequently, users can conduct risk assessments by examining numerous risk items. A detailed list of the items featured in the Risk Register, along with their explanations, is provided in Supplementary Material H.

5. Prototype Information Flow and Graphical User Interfaces

5.1. Prototype Information Flow

The illustration in Figure 2 shows the main functions, information flow, and outputs of the prototype. The initial menu of the prototype, All Projects (a), allows users to either create a new project or choose one that was previously created. When initiating a new project, users are presented with two options: generating a project from a pre-populated PPP template or an empty template. While the empty template lacks pre-defined information, the pre-populated template includes predetermined elements for PPPs, such as risk factors, compensation, and mitigation actions.

Upon the creation of a new project, the Project Charter menu (b) is activated. This menu encompasses fields for the PPP information requirements listed in Supplementary Material C (e.g., project description and scope, laws related to the project, milestones of the project) to delineate the general characteristics of the PPP. Subsequently, as the user fills out the Project Charter, the Project Financials menu (c) is opened. This menu facilitates the input of data for the Financial Information Requirements listed in Supplementary Material D (e.g., demand projection, interest rate, project costs). Filling out the fields in these two menus with comprehensive notes and data provides users with a thorough understanding of the project before initiating the risk assessment. Data in both menus can be exported to Excel. In the Risk Hierarchy menu (d), risks are identified. The Pre-Populated Template includes 83 validated risk factors (listed in Supplementary Material B), and users can either use these risks or make modifications, delete existing ones, and add new risks and risk groups. After identifying the risks pertaining to the project being assessed, the Dropdown List Items menu (e) is accessed, where project specific risk details (e.g., project phases, risk management strategies, risk owners) are identified. Subsequently, the risk factors in the Risk Hierarchy menu (d) and the project specific risk details identified in the Dropdown List Items menu (e) contribute to the Risk Register menu (f). The user performs a qualitative risk assessment on the Risk Register menu (f) using the risk factors and project specific risk details. When the user enters information for the risk items in the Risk Register, the information is analyzed and fed to the other menus (g-h-i-j), creating the prototype’s risk assessment output as rankings, graphs, pie charts, and visualizations. For instance, the Probability and Impact Score feed the Risk Matrix menu (g), displaying risks on a risk heat map. Risk ID, Probability and Impact Score, and Mitigation information feed the Risk Ranking menu (h), ranking risks based on Probability x Impact scores. Causal Risk and Consequential Risk information in the Risk Register (f) menu feeds the Risk Schema (i), creating a cause–effect schema illustrating the relationship between occurring risks. Finally, the user’s selections on the Risk Register menu (f) feed the Visualization menu (j), generating pie charts summarizing the entered data. The Documents (k) menu can be employed to upload and store project documents.

5.2. Development of the Use Case and the Graphical User Interfaces (GUIs)

This section delineates the system functions through a use case developed to illustrate the information and process flows in the proposed prototype. Steve, a PPP professional, intends to perform a comprehensive risk assessment on Riesgo. Initially, he generates a new project within the All Projects menu by choosing the PPP Risk Template (pre-populated) and assigning it the name Test Project Riesgo (Figure 3).

Upon entering the project, the Project Charter becomes accessible. This menu encompasses multiple tabs and information fields for Steve to complete using the information he possesses. For instance, in Figure 4, he enters some details into the “Loan Financing Structure” information requirement box under the Financial Details Table. The full list of Project Information Requirements can be seen in Supplementary Material C.

In the following menu, Project Financials (Figure 5), values for project financials are entered by Steve. During his use test, dummy demand projection values are added by Steve, and he observes the trend of the data through graphs (Figure 6). The full list of Financial Information Requirements can be seen in Supplementary Material D.

Upon accessing the Risk Hierarchy menu (Figure 7), all the predefined risk factors within the prototype (Supplementary Material B) are listed. Steve reviews all the predefined risk groups and the predefined risks under these groups. He wants to make sure all potential risks in the PPP project he is working on are listed in the prototype.

For example, he intends to see whether the force majeure risk is listed. Since numerous risks are available in the Risk Hierarchy menu, checking each risk individually can be challenging. Therefore, Steve utilizes the risk search bar for “force majeure” risk (Figure 8). Consequently, he sees a risk group titled Environmental Risks and Force Majeure, and within this group, the force majeure risk is listed.

Subsequently, Steve recalls the significant global impacts of the COVID-19 pandemic on various PPP projects. Consequently, he contemplates including “pandemic risk” as a new risk within the Environmental Risks and Force Majeure group. The prototype facilitates this process by allowing users to add, edit, and delete risks effortlessly. To introduce the new risk, Steve opts for the risk group, utilizes the “Add Risk” function, enters “pandemic risk”, and successfully incorporates the new risk (Figure 9). Also, risks are listed on the left side of the menu, so he can alternatively use add, edit, and delete functions by pressing the icons near these risks.

Next, Steve assesses the risks on the Risk Register menu. During the assessment, he can also utilize the Dropdown List Items menu to expedite the assessment process. Steve establishes project specific risk details using the Dropdown List Items menu. These items are categorized under Project Phases, Impact Types, Risk Strategy, Risk Allocation, Risk Owners, Mitigation Mechanism, Compensation Mechanisms, and Risk Status and are listed in the dropdown lists in the Risk Register menu. Steve retains control over these items, with the ability to delete those unsuitable for his project and add new items under each heading. For example, he adds new project phases named “Phase 1”, “Phase 2”, and “Phase 3”. Similar customizability exists for adding new items under the other headings (Figure 10).

Once all the necessary items have been included in the Dropdown List Items menu, Steve can proceed to the Risk Register menu. The risk list previously established in the Risk Hierarchy menu is visible on the left side of the Risk Register menu. Here, he can choose the risk he intends to assess, prompting the display of a register for the selected risk on the screen. For instance, when Steve opts for the “Changes in the Law” risk, the risk register for this specific risk becomes visible. Knowledge embedded in the prototype pertaining to this risk factor, including information such as Explanation, Compensation, and Mitigation actions, are presented to Steve. He can revise this information, if he chooses (Figure 11 and Figure 12).

Steve fills out the remaining fields based on the project (Figure 13). He manually enters the Risk ID and Contract ID. He provides a Probability and Impact value, based on his subjective judgement. Project Phase, Strategy, Risk Allocation, Risk Owner, Mitigation, Compensation, and Status are completed with the selections from the Dropdown lists. For example, when Steve clicks on the field under Project Phase, a dropdown list containing previously defined project phases from the Dropdown List Items menu pops up. Then, Steve assigns the Construction, Operation, and Maintenance phases to the Changes in the Law risk (Figure 14). Causal Risk, Consequential Risk, Residual Risk, and Secondary Risk are also determined by Steve using dropdown menus. For instance, by clicking the field under Consequential Risk, all risks outlined in the Risk Register menu are displayed in a dropdown menu. Then, Steve designates Regulatory Compliance risk as a consequential risk for the Changes in the Law risk. The system automatically fills the remaining fields, such as risk author, revision date, and risk entry date. The risk status is also assigned by Steve.

Utilizing the explained approach, Steve assesses ten risks. Then, he opens the Risk Register menu and observes the assessed risks displayed on a 10 × 10 Probability Impact Risk Matrix, depicted in Figure 15. The risks are represented on the risk matrix according to the risk IDs assigned by Steve in the Risk Register. Positions of the risks are also determined based on the Probability and Impact Scores that Steve determined with his subjective assessments in the Risk Register. For example, Changes in the Law risk (Risk ID: CHA) had been rated by Steve with a Probability Score of 6 and an Impact Score of 7, as evident in the Risk Register in Figure 13. Hence, in Figure 15, the representation of CHA is illustrated at the intersection of Probability Score: 6 and Impact Score: 7. Steve can change the bars’ position on the right side of the screen and determine red, yellow, and green fields based on his subjective risk perception.

In his assessment, Steve opts to maintain the intervals as unchanged and proceeds to the subsequent menu, Risk Ranking (Figure 16). Within this menu, the ten risks assessed by Steve are arranged in a list based on their Probability and Impact Scores. For instance, based on his assessment, the Delay risk is positioned at the top. The Risk ID and Contract ID (i.e., related contract clauses for a specific risk) are also provided to facilitate Steve’s understanding of the relationship between the most significant risks and the project contract. The prototype further assesses the Risk Level and the availability of mitigation, presenting Steve with a priority value for each risk based on an “if–then” rule.

In the following menu titled Risk Schema (Figure 17), Steve observes the cause-and-effect connections among the risk factors. Riesgo generates this relationship scheme automatically, relying on Steve’s input in the Risk Register menu under the risk items of Causal Risk and Consequential Risk. In the Risk Register, Improper Legal/Law System was designated as a Causal Risk for the Changes in the Law risk by Steve, and the Regulatory Compliance Risk was specified as the Consequential Risk (Figure 13). Analyzing this data, the prototype visually demonstrates the relationships between the risks, and it can be easily inferred that the occurrence of the Changes in the Law risk can be attributed to the Improper Legal/Law System, and the Regulatory Compliance Risk may arise as a consequence of the Changes in the Law (Figure 17). This visualization enhances decision making in risk assessment by enabling a thorough observation of the relationships between risks, which may be particularly insightful in complex PPPs comprising multiple cause–effect relationships among the risks.

In the subsequent menu, called Risk Visualization, Steve accesses pie-chart representations derived from the risk data he entered into the Risk Register. Pie charts for Risk Category, Project Phase, Impact Type, Strategy, Risk Allocation, Mitigation, Compensation, Risk Owner, and Risk Level are automatically generated, analyzing the data entered by Steve in the Risk Register. For example, Figure 18 illustrates the distribution of risks based on their categories.

In the final menu, Documents (Figure 19), Steve can upload project-related files. For example, he can upload an XSLX file containing Exchange Rate projections to examine during the assessment of exchange rates. In this use test, Steve uploads files exported from various menus within the Riesgo prototype. Project Charter, Project Financials, Dropdown List items, Risk Register, and Risk Ranking are suitable for Excel export since they contain a database. Risk Hierarchy, Risk Matrix, Risk Schema, and Risk Visualization are suitable for PDF export since they contain visual output.

6. Validation and Verification of Prototype

Verification and validation were conducted in the final step to examine the developed prototype’s functionality, as well as its applicability, usability, and practicability in PPP projects. The prototype was verified by 13 PPP experts with experience in the fields of PPP, risk management, and contracts who used it for risk assessment, and their feedback was utilized for further development. The validation process was undertaken with 21 PPP experts through a usability survey. The validation and verification results are explained in this section.

The feedback received from professionals during verification can be summarized as follows: (1) Riesgo is easy to use due to its user-friendly graphical interface. (2) The Pre-Populated PPP Template, containing predefined risk factors, provides a time-saving advantage to the professionals during the risk assessment, and the template also works as a checklist to prevent the oversight of significant risks. (3) Since the PPPs comprise a multitude of risk factors and risk items that can vary from one project to the other, the customizable structure of the prototype is identified as another noteworthy advantage. This structure helps to facilitate the revision of a project in Riesgo based on the needs of a real PPP project. (4) The visualizations provided by Riesgo for the PPP risk assessment deliver easily comprehensible outputs. (5) Riesgo can expedite risk and contract management operations. Its proactive risk assessment approach can prevent significant cost and time losses arising from risk factors. (6) All main PPP stakeholders, including the public party, private party, and financiers, can utilize the prototype for PPP risk assessment (7) The utilization of Riesgo in real PPPs promotes transparency and unity among stakeholders. (8) Considering the absence of dedicated software for PPPs’ qualitative risk assessment, this prototype can fill a gap in the PPP sector.

Challenges were also reported by professionals during verification, mainly due to the numerous information fields in the prototype. To overcome the uncertainty regarding which information fields should be completed by the users, it was suggested by professionals that the model should show the mandatory data entry fields to the users. This way, users can initially focus on the mandatory fields to obtain Riesgo outputs. In addition, a simple diagram illustrating the relationships between information fields in the Riesgo can be added to the model. This clarifies which information fields contribute to which menus. Another difficulty mentioned by some professionals was that, despite the user-friendly graphical interface of the prototype, becoming accustomed to new software in a business process might require a period of adjustment. Concerning the second challenge, professionals recommended the development of additional user documents and the inclusion of a video within Riesgo. These Supplementary Material can facilitate the resolution of issues encountered by new users and promote their adaptation to the new software.

To validate the prototype, as outlined in the methodology section, professionals from the sector completed a survey comprising 20 questions. Ten questions focused on the general usability of the prototype, while the remaining questions assessed its applicability in PPPs. Average scores were computed based on the responses. The statements in questions 1, 3, 5, 7, and 9 incorporated positive language, while those in questions 2, 4, 6, 8, and 10 comprised a negative structure. Hence, higher scores for questions 1, 3, 5, 7, and 9 and lower scores for questions 2, 4, 6, 8, and 10 are expected. The average scores are presented in Table 4.

The average scores for questions 1, 3, 5, 7, and 9 ranged from 3.81 to 4.33. The participants indicated a strong intention to use the system frequently (Q1 avg = 4.33), acknowledged its user-friendly nature (Q3 average = 4.29), and praised the integration of functions within the system (Q5 avg = 4.24). Questions 7 and 9 assessed the ease of learning and user confidence, with both averaging 3.81. Despite these scores being lower than the others, they still indicate that the users found the system easy to learn. Furthermore, analyzing the averages of questions with negative statements revealed that the users believed the system could be learned without technical support (Q4, avg = 1.71) and that it did not require extensive learning (Q10, avg = 2.05). Additionally, Q2 (Average = 1.71) suggested that the system was not complex, which was supported by Q6 (avg = 1.43), indicating low system inconsistency. Q8 (avg = 1.95) highlighted the system’s efficiency and showed that it is not cumbersome.

For the remaining questions related to the model’s applicability and practicability for PPPs, average scores ranged from 4.19 to 4.52. Participants agree that the model facilitates data collection for projects (Q11 avg = 4.19), enables the retention of more information compared to sector tools (Q12 avg = 4.29), and has the potential to prevent the overlooking of critical information during risk assessment (Q13 avg = 4.29). They also affirm the suitability of selected functions for risk assessment (Q14 avg = 4.48), stating that it can facilitate (Q15 avg = 4.48) and expedite (Q16 avg = 4.29) PPP risk assessments. Additionally, they endorse the model’s applicability in real PPPs, suggesting its use in the feasibility phase (Q17 avg = 4.48), during PPP contract preparation (Q18 avg = 4.52), for assessing contract risks during contract preparation (Q19 avg = 4.48), and for tracking and monitoring project risks during project implementation (Q20 avg = 4.24).

Table 4. Validation survey results.

	Questions Regarding General Usability	# 1	# 2	# 3	# 4	# 5	# 6	# 7	# 8	# 9	# 10	# 11	# 12	# 13	# 14	# 15	# 16	# 17	# 18	# 19	# 20	# 21	Avg
Q1	I think that I would like to use this system frequently.	4	4	4	3	4	5	4	4	5	5	4	5	5	4	5	4	5	4	4	5	4	4.33
Q2	I found the system unnecessarily complex.	1	2	2	2	1	2	3	1	2	1	1	2	2	2	1	2	2	2	1	1	3	1.71
Q3	I thought the system was easy to use.	4	4	4	4	5	5	4	4	4	5	4	5	5	4	5	4	5	4	3	5	3	4.29
Q4	I think that I would need the support of a technical person to be able to use this system.	1	1	2	4	1	2	1	1	1	1	2	1	2	2	2	2	2	2	2	2	2	1.71
Q5	I found the various functions in this system were well integrated.	4	3	4	3	5	5	3	5	5	5	4	5	5	4	5	3	5	4	3	5	4	4.24
Q6	I thought there was too much inconsistency in this system.	2	1	2	2	1	2	2	1	1	1	1	1	1	2	1	2	1	1	2	1	2	1.43
Q7	I would imagine that most people would learn to use this system very quickly.	4	4	4	4	5	4	3	5	4	4	3	4	4	4	4	2	4	3	3	5	3	3.81
Q8	I found the system very cumbersome to use.	1	2	2	2	1	2	4	1	1	2	1	2	3	2	1	2	2	2	3	1	4	1.95
Q9	I think I will feel confident using the system.	4	4	3	3	5	4	4	4	5	4	3	4	4	4	4	2	4	4	3	5	3	3.81
Q10	I need to learn a lot of things before I get going with this system.	1	1	2	3	2	2	2	1	1	2	2	2	2	2	2	4	2	2	2	2	4	2.05
	Questions regarding the usage of model in PPPs	# 1	# 2	# 3	# 4	# 5	# 6	# 7	# 8	# 9	# 10	# 11	# 12	# 13	# 14	# 15	# 16	# 17	# 18	# 19	# 20	# 21	Avg
Q11	The model facilitates the data collection associated with the project.	4	3	4	4	5	4	4	4	5	5	4	5	5	4	5	3	5	4	2	5	4	4.19
Q12	The model enables the keeping of more data/information related to risks compared to alternative tools commonly used in the industry such as Risk Registers and Risk Matrices.	4	5	3	4	5	4	5	3	4	5	4	4	5	4	5	3	5	5	3	5	5	4.29
Q13	The model may prevent critical points from being overlooked in qualitative risk analysis/assessment processes.	2	4	4	3	5	4	5	5	5	5	4	4	4	4	5	4	5	5	4	5	4	4.29
Q14	The functions of the model can provide a comprehensive/versatile view for qualitative risk analysis/assessment processes.	4	4	4	3	5	4	5	4	5	5	4	5	5	4	5	4	5	5	5	5	4	4.48
Q15	The model has the potential to facilitate qualitative risk analysis/assessment processes.	4	4	4	4	5	4	4	5	5	5	5	5	5	4	5	3	5	5	4	5	4	4.48
Q16	The model has the potential to expedite qualitative risk analysis/assessment processes.	4	2	4	4	5	4	4	4	5	5	4	5	5	4	5	3	5	5	4	5	4	4.29
Q17	The model can be used for general risk assessment in the feasibility processes of PPP projects.	3	4	4	3	5	4	4	5	5	5	4	5	5	4	5	4	5	5	5	5	5	4.48
Q18	The model can contribute to the contract preparation processes of PPP projects.	5	4	4	3	5	4	4	5	5	5	4	5	5	4	5	3	5	5	5	5	5	4.52
Q19	The model can be utilized for conducting risk assessment through PPP project contracts.	4	5	4	4	5	4	4	5	5	5	4	5	5	4	5	3	5	5	3	5	5	4.48
Q20	The model can be used for tracking/monitoring risks in PPP projects.	5	4	3	3	5	4	4	2	5	5	3	5	5	4	5	4	5	5	3	5	5	4.24

Table 4. Validation survey results.

	Questions Regarding General Usability	# 1	# 2	# 3	# 4	# 5	# 6	# 7	# 8	# 9	# 10	# 11	# 12	# 13	# 14	# 15	# 16	# 17	# 18	# 19	# 20	# 21	Avg
Q1	I think that I would like to use this system frequently.	4	4	4	3	4	5	4	4	5	5	4	5	5	4	5	4	5	4	4	5	4	4.33
Q2	I found the system unnecessarily complex.	1	2	2	2	1	2	3	1	2	1	1	2	2	2	1	2	2	2	1	1	3	1.71
Q3	I thought the system was easy to use.	4	4	4	4	5	5	4	4	4	5	4	5	5	4	5	4	5	4	3	5	3	4.29
Q4	I think that I would need the support of a technical person to be able to use this system.	1	1	2	4	1	2	1	1	1	1	2	1	2	2	2	2	2	2	2	2	2	1.71
Q5	I found the various functions in this system were well integrated.	4	3	4	3	5	5	3	5	5	5	4	5	5	4	5	3	5	4	3	5	4	4.24
Q6	I thought there was too much inconsistency in this system.	2	1	2	2	1	2	2	1	1	1	1	1	1	2	1	2	1	1	2	1	2	1.43
Q7	I would imagine that most people would learn to use this system very quickly.	4	4	4	4	5	4	3	5	4	4	3	4	4	4	4	2	4	3	3	5	3	3.81
Q8	I found the system very cumbersome to use.	1	2	2	2	1	2	4	1	1	2	1	2	3	2	1	2	2	2	3	1	4	1.95
Q9	I think I will feel confident using the system.	4	4	3	3	5	4	4	4	5	4	3	4	4	4	4	2	4	4	3	5	3	3.81
Q10	I need to learn a lot of things before I get going with this system.	1	1	2	3	2	2	2	1	1	2	2	2	2	2	2	4	2	2	2	2	4	2.05
	Questions regarding the usage of model in PPPs	# 1	# 2	# 3	# 4	# 5	# 6	# 7	# 8	# 9	# 10	# 11	# 12	# 13	# 14	# 15	# 16	# 17	# 18	# 19	# 20	# 21	Avg
Q11	The model facilitates the data collection associated with the project.	4	3	4	4	5	4	4	4	5	5	4	5	5	4	5	3	5	4	2	5	4	4.19
Q12	The model enables the keeping of more data/information related to risks compared to alternative tools commonly used in the industry such as Risk Registers and Risk Matrices.	4	5	3	4	5	4	5	3	4	5	4	4	5	4	5	3	5	5	3	5	5	4.29
Q13	The model may prevent critical points from being overlooked in qualitative risk analysis/assessment processes.	2	4	4	3	5	4	5	5	5	5	4	4	4	4	5	4	5	5	4	5	4	4.29
Q14	The functions of the model can provide a comprehensive/versatile view for qualitative risk analysis/assessment processes.	4	4	4	3	5	4	5	4	5	5	4	5	5	4	5	4	5	5	5	5	4	4.48
Q15	The model has the potential to facilitate qualitative risk analysis/assessment processes.	4	4	4	4	5	4	4	5	5	5	5	5	5	4	5	3	5	5	4	5	4	4.48
Q16	The model has the potential to expedite qualitative risk analysis/assessment processes.	4	2	4	4	5	4	4	4	5	5	4	5	5	4	5	3	5	5	4	5	4	4.29
Q17	The model can be used for general risk assessment in the feasibility processes of PPP projects.	3	4	4	3	5	4	4	5	5	5	4	5	5	4	5	4	5	5	5	5	5	4.48
Q18	The model can contribute to the contract preparation processes of PPP projects.	5	4	4	3	5	4	4	5	5	5	4	5	5	4	5	3	5	5	5	5	5	4.52
Q19	The model can be utilized for conducting risk assessment through PPP project contracts.	4	5	4	4	5	4	4	5	5	5	4	5	5	4	5	3	5	5	3	5	5	4.48
Q20	The model can be used for tracking/monitoring risks in PPP projects.	5	4	3	3	5	4	4	2	5	5	3	5	5	4	5	4	5	5	3	5	5	4.24

7. Discussion

Various risk management models are available in practice, each serving different purposes and providing different functions. Palisade @Risk [70] possesses an approach that focuses on Monte Carlo simulation, decision tree analysis, predictive neural networks, statistical analysis, and forecasting. Both qualitative and quantitative tools for project or program risk assessment are included in Primavera Risk analysis [125], yet quantitative assessment conducted by Monte Carlo simulation is at the forefront. Similarly, Full Monte [126] executes schedule risk analysis in Microsoft Project [127] or Primavera [125] through the Monte Carlo simulation. Riskonnect (formerly Sword Active Risk Manager) [72] incorporates qualitative functions and cost impact analysis, once again using Monte Carlo. Reviewing alternative risk management tools reveals that (1) none were specifically designed for PPP risk assessment, (2) they do not include knowledge-based templates developed to guide the practitioners through various PPP project phases, and (3) the existing tools aim for versatility across multiple sectors, leading to a general approach rather than to a focus on the specific needs of a sector such as PPP.

The proposed Riesgo prototype provides a qualitative assessment process with a knowledge-based approach. Like other risk software models, Riesgo can also be applied to various project types, but it is particularly tailored to meet the needs of PPP projects. Therefore, the pre-populated PPP template within Riesgo sets it apart from other risk software models, making it a unique model for PPP risk assessment. This template, readily available in the software, comprises PPP risk factors, information requirements, descriptions, and explanations regarding the risks, compensation, and mitigation mechanisms for the risk factors. Additionally, users can customize these elements based on their project requirements. While most software models focus on risk management after project commencement, Riesgo can be also utilized before project initiation to understand project risks and develop a comprehensive PPP contract, mapping the project risks and their details. Thus, the key advantages of Riesgo lie in its customizable and knowledge-based structure, which is suitable for use in all project phases by all stakeholders, with a focus on qualitative risk assessment, which is often not prioritized in many software solutions. The contributions and limitations of the proposed prototype are further discussed below, utilizing the findings from the verification and validation processes.

7.1. Contributions

7.1.1. A Customizable and Knowledge-Based System Utilizing Extensive Project Information for Qualitative Risk Assessment in PPPs

Riesgo boasts a wide array of data entry fields that have been validated by PPP experts, ensuring a thorough consideration of crucial aspects and information related to the project, contract, and associated risks. The design of its user-friendly graphical interface, validated based on Brooke’s (1996) [122] system usability questions, significantly contributes to facile user comprehension of the system.

The Riesgo prototype offers users a predefined PPP risk template, incorporating validated risk factors and detailed information, such as mitigation and compensation mechanisms. In other words, Riesgo presents its users with predefined knowledge and thereby works with its content as a knowledge-based system. This feature allows users to streamline the risk assessment, saving valuable time. Also, it can help knowledge transfer among the PPP projects, which is one of the significant requirements for capacity-building in PPPs (Chileshe et al., 2023) [128]. In addition to presenting predefined knowledge, Riesgo goes a step further by visualizing data, simplifying the interpretation of risks compared to traditional risk matrices/registers, such as Excel sheets, prevalent in the sector,.

The critical risks employed in the prototype were ascertained through expert validation and are presented in Supplementary Material B. Upon review, it becomes evident that experts deemed most factors related to land, financial, legal and contractual, governance, political, social, design, and environmental risks critical for the public party. Conversely, the proportion of critical risks within the construction and operation categories was comparatively lower. This result may stem from the usual allocation of these risks to the private party. However, it is essential to highlight that certain risk factors typically allocated to the private party, such as inflation, interest rate, revenue risk, finance source unavailability, cost increase, and demand risk (Bing et al., 2005; Ibrahim et al., 2006) [95,99] were also identified by the experts as critical risks and were integrated into the model.

In practical applications, if new critical risks are identified, the project team can add new risk factors to Riesgo with the help of its customizable structure. Users can modify the template according to their project requirements, including creating new risk groups and risk factors within these groups. By default, Riesgo does not categorize risks based on their criticality level, as the criticality of PPP risks can vary from one project to another. For example, financial risks such as inflation and exchange rates may not be considered primary risks in mature markets. In contrast, in less developed markets, these risks could be deemed more critical (Global Infrastructure Hub, 2019) [16]. Therefore, in Riesgo, all risks possess the same criticality level at the project’s outset, and users alter this status by determining the level of criticality through the probability and impact scores they assign during their risk assessment.

Riesgo can be used as a standalone risk assessment tool or a contributory tool. The predefined information within Riesgo serves as a practical checklist for professionals, enabling them to cross-reference their project-specific risk matrices/registers with the established information in the system. Although a risk matrix that can be used as a checklist during the PPP risk assessment was created by the Global Infrastructure Hub (Global Infrastructure Hub, 2019) [16], this tool takes a standard form, so it cannot be customized based on the needs of the conducted PPP project. Herein, a notable advantage of Riesgo lies in its customizable structure, empowering users to open a PPP template and modify it according to their project requirements using functions like “add a risk”, “delete risk”, or “edit risk.” Alternatively, users can commence with an empty template and define all project risks from the ground up. With these advantages, Riesgo differs from most of the models presented in the literature, which were developed based on a specific location or PPP type. For instance, Ghimire et al. (2024) [129] assessed the risks of waste-to-energy projects in Nepal using the analytical hierarchical process method. Yang et al. (2024) [130] proposed a risk assessment model for PPP water environment treatment projects. These models are specific and cannot be adapted directly to all PPPs. Yet, Riesgo’s generic approach and a high degree of customizable structure make it possible to use it in all PPP projects, irrespective of location and project type.

7.1.2. Suitable for All Stakeholders and Various Project Phases

PPPs include various risk factors, which must be fully assessed and balanced to attract the private sector (Zhang and Shahid, 2024) [131]. Hence, Riesgo was specifically developed for the public party and its consultants considering the public party’s pivotal role as the primary stakeholder throughout the entire project lifecycle. The risk factors integrated into the model were identified to align with the risks that the public party is obligated to analyze regarding PPPs. Professionals involved in the verification process highlighted that the model’s customizable structure extends its suitability to all major stakeholders in PPPs. Furthermore, a valuable recommendation emerged to transform Riesgo into a unified platform for all stakeholders. In practical terms, in a country engaging in PPPs, the public party could initiate the use of Riesgo and mandate its utilization by other stakeholders. This approach ensures that all parties engage in the risk assessment process using the same tool, fostering collaboration. Such a coordinated approach not only streamlines the risk assessment but also enhances communication among all involved parties.

Certain models proposed in the literature for PPP risk assessment focus on specific risks rather than all project risks. For example, the model by Alasad and Motawa (2015) [29] addresses demand risk, while Liu et al. (2017) [26] concentrate on revenue risk. Thus, such specialized models are associated with particular project phases. In contrast, Riesgo is a generic model designed to evaluate all project risks, and this ability makes it valuable across all the major stages in the typical PPP project cycle outlined by the World Bank (2017) [15]. To illustrate, in the project’s early stages, such as feasibility, Riesgo aids in defining and assessing general project risks, contributing to a prompt assessment of a PPP project included on a country’s PPP agenda. Furthermore, the platform can be utilized to assess alternative projects based on their risks, simplifying the selection process through comparative assessments.

Given that project risks are outlined in PPP contracts, Riesgo proves to be an ideal tool for contract preparation and drafting. Professionals can meticulously assess the risks designated for implementation in the PPP contract, subsequently integrating them into the contract. Riesgo helps to identify crucial risk factors, their interrelations, and their detailed aspects, enriching the contract preparation process and ensuring clarity without ambiguities.

Riesgo can also be used to assess the risks available in a prepared contract. Risk information from the contract can be input for the Riesgo risk assessment. For instance, stakeholders in the private sector can make informed decisions regarding participation in a PPP bid using the qualitative risk assessment conducted in Riesgo based on the prepared project contract.

During the validation process, professionals also highlighted Riesgo’s suitability for monitoring project contract risks. The projects initially created in Riesgo can be easily duplicated and updated in the monitoring phase. The professionals also emphasized knowledge transfer challenges in the PPP sector arising from staff turnover in either public institutions or the private sector. They expressed that utilization of Riesgo can facilitate knowledge and experience transfer, ensuring their continued utilization in future projects.

7.1.3. Fills the Gap in the PPP Sector for a Qualitative Risk Assessment Tool

Despite the proliferation of approaches for quantitatively assessing risks in PPPs (Rasheed et al. 2022) [23], there is a significant gap in the literature and sector concerning qualitative risk assessment for PPPs. The absence of a dedicated tool in the sector for qualitative risk assessment highlights a notable void in managing risk information in PPPs (Jiang et al., 2023) [24], which Riesgo has the potential to fill, thereby enhancing the overall risk assessment process in PPPs. The inadequacies of risk assessments, characterized by suboptimal in-house solutions like Excel sheet-based matrices/registers, can be circumvented through the adoption of a tailored and adaptable solution designed specifically for PPPs, exemplified by the case of Riesgo.

The current version of Riesgo is a prototype validated by the PPP professionals. The absence of a tool like Riesgo was also acknowledged by the professionals, and a desire to utilize it in real-world projects was expressed during the interviews. Furthermore, additional recommendations for future developments were provided. Applying these recommendations in the scope of future works can bolster the likelihood of Riesgo being widely utilized in actual PPP applications.

7.2. Limitations

During validation and verification, the prototype was used by experts involved in real-world PPP projects. These experts provided detailed feedback on the prototype’s effectiveness by testing it on the real-world PPPs. One limitation of this study is that comprehensive testing of Riesgo on an actual PPP project throughout the project’s lifecycle was not feasible, due to the extended time required. After implementing the improvements suggested by the experts on the prototype, the testing of Riesgo on an actual PPP, involving risk analysis at the beginning and collecting feedback on the risks encountered at later stages, can be undertaken as a case study.

In this study, Riesgo was developed as a simple prototype, with the aim to solicit expert feedback concerning the proposed approach. Due to time and budget constraints, the researchers opted to apply the minimum viable product (MVP) concept. This methodology allows for a tool or product’s swift and cost-effective development, along with collecting feedback from potential users (Ries, 2011) [132]. In this regard, a desktop software model was chosen over a cloud-based collaborative web application model. This choice streamlined the development phase and aided in adhering to the project’s budgetary limitations. Based on the feedback obtained during the validation and verification phase, the development of a cloud-based version of Riesgo, which will also enable collaborative project work within the same team and among different teams, is planned in the scope of future studies to enhance the model’s accessibility.

7.3. Future Work

As part of future software developments, plans are in place to enhance accessibility through the creation of a web-based version of Riesgo. The web-based version will be designed to enable collaboration on the same project by multiple individuals within the same team. Alongside fostering collaboration within teams, there are also intentions to implement enhancements that facilitate collaboration among key stakeholders in PPPs. This would enable coordination of the risk assessment process for a project among the public party, private party, and lenders. A log-tracking feature can also be added to the Riesgo, which allows easy identification regarding where and by whom changes have been made. Additionally, a user role authorization feature can be beneficial. This feature allows some users solely to view the project, while others can make changes to it.

In line with recommendations from the professionals, plans include incorporating functions related to artificial intelligence (AI) into the software. One potential improvement involves uploading the project contract into the software and automatically populating relevant fields by extracting information from the contract. This way, risk assessment regarding the contract can be conducted rapidly. Users can have the option to use AI as an assistant, providing automatic suggestions during the risk assessment process. AI can also answer user questions, expediting the assessment process. Users can leverage AI as an assistant to carry out the entire risk assessment process independently or as a supportive function to verify the assessment made.

If risks are assessed before the formation of the project contract, AI can utilize defined risk characteristics to generate relevant contract clauses. Thus, a draft project contract can be created automatically based on the information and data entered into the system. Numeric values in the project contract, also displayed in Riesgo’s Financials menu (Figure 5), can trigger warnings if they surpass predetermined thresholds, notifying the user about the activated mechanisms. For instance, if the estimated annual traffic for a bridge in the contract is 1 million vehicles, and at the end of the year, it only reaches 800,000, Riesgo can report on the guaranteed amount to be paid by the treasury based on the current toll rate, as well as the mechanism that will be activated if this difference is not paid. Past court decisions can be taught to the AI in Riesgo, enabling it to learn the outcomes of similar cases. Thus, AI can inform the users about the legal consequences of risks. All these improvements aim to contribute to making Riesgo a more efficient and evolving model, thereby increasing the likelihood of its use in real projects.

Following the implementation of improvements selected from the suggestions outlined in the this section, communication with professionals will be initiated to test the new version using an actual PPP project. While utilization throughout the entire project lifecycle may not be feasible, implementation at specific stages for a real test can allow for a more in-depth exploration of its impact within the industry.

8. Conclusions

This paper introduces Riesgo, a qualitative risk assessment prototype for PPP projects. The development and research steps are explained in this paper using three main phases: (1) identifying risk register items and system requirements for the prototype, (2) developing the system architecture by implementing PPP risk factors, PPP information requirements, risk register items, and system functions to the model, and (3) conducting verification and validation processes for the prototype. The feedback collected during the validation and verification process reveals that the developed prototype Riesgo (1) is a customizable and knowledge-based system that utilizes extensive project information for qualitative risk assessment in PPPs, (2) is suitable for all stakeholders and various project phases, and (3) can fill the gap in the PPP sector for qualitative risk assessment.

Riesgo is an effective system for qualitative risk assessment in PPPs, guiding its users using knowledge-based content. With a diverse set of validated data entry fields and a predefined risk template, it streamlines assessments, saving time for users. Its customizable structure makes it suitable for all PPP projects, offering versatility as a standalone or contributory tool. It is designed for the public party and consultants in PPPs, and professionals in the sector found it suitable for all stakeholders, thanks to its customizable structure. The tool can be beneficial across various PPP phases such as early risk assessment, feasibility studies, contract preparation, and contract monitoring. Variations exist in PPP applications across different countries. Nonetheless, given that Riesgo encompasses generic PPP risk factors, items, and requirements within a customizable framework, it can be effectively utilized across different PPP models and diverse geographical locations. This advantage renders the model a universal tool, facilitating its application in various contexts. In other words, factors such as the PPP model implemented, the developmental stage of the host country, the legal system in place, or the political structure do not hinder the model’s applicability.

The model can be utilized by all key stakeholders in PPPs, including the public party, private party, and lenders, for various purposes, according to their requirements. For example, the public party can employ Riesgo during the initial stages of the project when making investment decisions. Private parties can evaluate PPP project risks based on draft project contracts and decide whether to participate in a PPP bid following the conducted risk assessment. Lenders can analyze overall project risks and utilize the model for financial risk assessments as an additional tool alongside their existing tools, aiding in the decision-making process regarding project funding. Subsequently, once the project is underway, all stakeholders can utilize Riesgo to monitor and manage project risks. The stakeholders can store and review their previous risk assessments conducted within Riesgo during the earlier project phases. This functionality facilitates the accumulation of PPP risk knowledge, which can also be transferred to future projects.

Riesgo’s ability to act as an information repository facilitates knowledge transfer in the event of personnel changes in an organization. Its customizable design helps to avoid inefficiencies associated with in-house solutions like Excel-based matrices/registers. Riesgo addresses a notable gap in the literature, which is proliferated by quantitative risk assessment models, providing a qualitative risk assessment tool for PPPs to meet the predominant preference of the PPP sector. The positive validation feedback from professionals underscores the demand for such a tool in real world projects.

Future developments, such as the introduction of a web-based version, AI integration, and improved collaboration features, will likely enhance Riesgo’s applicability and effectiveness in real PPP applications. The proposed improvements and ongoing efforts to seek feedback from professionals indicate a commitment to refining Riesgo for broader industry use, potentially transforming the landscape of PPP risk assessment.