Systems Engineering in Front-End Governance of Major Public Investment Projects

Abstract

This paper provides an account of how systems engineering principles are applied by the Norwegian government to improve up-front planning and decision-making of large public investment projects, as well as the effect of these efforts after 15 years of operations. It suggests that the results are promising, both in securing budgetary compliance, but also to ensure conceptual solutions that are economically viable in a life-time perspective.

1. Background

Governments in many countries are struggling with massive cost overruns and delays in major public sector investment projects. A number of international studies have put this problem under the spotlight, and several countries have introduced designated governance regimes for such projects. Norway was an early mover and introduced a scheme to improve the basis for decision-making on such projects in 2000, also known as the State Project Model. The procedure was based on systems engineering principles. At the same time, a research program was initiated to monitor developments and achievements and help improve procedures and practices. This paper summarizes some of the researchers’ findings.

2. Systems Engineering

Systems engineering is a collective term for various methodological approaches brought to bear to identify an optimum solution to a problem. Instead of springing from an assumed best solution, systems analysis considers the problem to be solved in its context, described as a system, and asks which requirement must be fulfilled for the system to function [1]. The solution is that which best satisfies the requirements. Such an open approach makes systems engineering a useful tool as an initial approximation on the road from a problem to a successful project.

The techniques are used primarily in lifetime design of complex projects, such as large infrastructure projects, aircraft and spacecraft design, robotics, etc. Systems engineering is interdisciplinary and focuses on defining customer needs and required functionality early in the development cycle, documenting requirements, then proceeding with design synthesis and system validation while considering the complete problem [2].

A system is a construct or collection of different elements that together produce results not obtainable by the elements alone. The elements, or parts, can include people, hardware, software, facilities, policies, and documents; that is, all things required to produce systems-level results. The results include system level qualities, properties, characteristics, functions, behavior, and performance. The value added by the system as a whole, beyond that contributed independently by the parts, is primarily created by the relationship among the parts; that is, how they are interconnected [3].

The economic value of systems engineering has been documented in numbers of case studies and comprehensive analyses of larger cohorts of projects. For instance, a study conducted by the United States National Aeronautics and Space Administration (NASA) indicated that there is a general correlation between the amount invested in systems engineering and cost overruns in individual projects [4].

Traditional problem-solving often derives from a central actor’s perspective and is characterized by the basic choices having been made before systematic analysis begins. Thereafter, the analysis will, in most cases, be associated with, and restricted to, the choice already made. Systems analysis, however, is a stepwise approach to problem-solving. The concern is with the system and its components, their properties, and the relationships between the components, as they impart the combined properties of the whole system. It focuses on the system in its entirety, that is, its capacity (such as in products or services) and its vulnerability (such as in damages, pollution, or losses), and on those subsystems that enable the whole system to function as well as possible.

The level of methodological sophistication applied may be anything from a simple conceptual model to a dynamic simulation. However, bringing in relevant experience from similar situations or projects is indispensable. Time, that is, the system life cycle, is a principal parameter in systems analysis. A general procedure might progress in the following nine steps [5]:

1.

System definition: Focus on the complexity of the issue and delimit the system.

2.

Identification of needs: Identify involved or affected stakeholders, along with their needs, prioritizations and any changes with time over the life cycle.

3.

Requirement specification: From the needs under consideration, identify the demands placed on the system, including:

• functional—how the system should work
• physical—how the system should be built and what it must sustain
• operational—how the system should run
• economic—the costs of system development and operation

4.

Alternative subsystems: Identify technologies or concepts that may help meet system requirements.

5.

Performance appraisal: Test the subsystems against the requirement specification.

6.

Concept choice: Evaluate alternatives to find the most suitable concept.

7.

Control of choice: Test the concept over the life cycle and more closely examine the financial conditions, the environmental conditions, logistics, impacts of technological change, operation and maintenance, etc.

8.

Vulnerability testing: Simulate the system in various scenarios with changes, errors, and disturbances, such as by using models.

9.

Describe the system.

Various tools and techniques may be used in carrying through such a procedure. They may be databases and networks for securing relevant information on experience in similar projects, Delphi techniques or Monte Carlo simulations to forecast or estimate, scenario tools and mind-mapping techniques to describe and visualize, multi-attribute evaluations to rank alternatives, and dynamic simulation models for evaluating effects and conducting sensitivity testing.

Such a rigorous procedure would preferably be used to design complex projects or programs. One common approach to define complexity is the Cynefin framework [6]. It outlines four categories of problems and possible approaches to solving them, and explains four main domains of complexity as follows:

• Simple, where cause-and-effect relationships are readily apparent, and known best practice can be applied.
• Complicated, where more analysis is required to understand the cause-and-effect relationship, but where good practice rational analysis will clarify this relationship.
• Complex, where fully predicting the relationship between cause and effect in advance is impossible, and the proposed approach is to Probe–Sense–Respond.
• Chaotic, in which there is no apparent relationship between cause and effect, and where the proposed approach is to Act (not just Probe)–Sense–Respond.

Large public investment projects clearly extend beyond the simple and complicated and, depending on the project, they can probably be both complex and chaotic. In such cases, ex ante analyses would be insufficient for rationally determining the best concept. In some cases, the complexity may seem to fit the term “wicked problem”, a term originally used in social planning to describe a problem that is difficult to solve because of incomplete, contradictory, and changing requirements that are often difficult to recognize [7].

3. Front End Governance of Major Public Projects

Governance regimes for major investment projects comprise the processes and systems that need to be in place on behalf of the financing party to ensure successful investments. This would typically include a regulatory framework to ensure adequate quality at entry, compliance with agreed objectives, management and resolution of issues that may arise during the project, etc., and standards for quality review of key governance documents.

Many of the problems facing major public investment projects can be interpreted in terms of deficiencies in the analytic or the political processes preceding the final decision to go ahead, and the interaction between analysts and decision-makers in this process [8].

The more fundamental problems that have to do with the project’s long-term utility and effect could typically be traced back to the earliest preparatory phases of the project, while the more marginal problems of cost efficiency, delays, and cost overrun are management issues that arise during the project’s implementation.

The more fundamental challenges would typically be to deal with problem, such as tactical budgeting in responsible agencies at various levels, done in order to increase the chance to obtain government funding for a project. Such practice is likely to flourish in organizations whose culture includes strong management pressures to avoid revealing bad news, appearing pessimistic, or lacking in confidence. In these situations, the ‘conditional estimate cop-out’ is a useful defensive mechanism, but one that reinforces the culture and can result in a ‘conspiracy of optimism’ [9]. Tactical estimation is also common in projects where so-called perverse incentives occur, for instance, in publicly funded projects where there are no financial commitments for the recipients and, therefore, no incentive to opt for the most cost-effective alternative. In projects where costs are strongly underestimated, there may be an inherent and often undisclosed conflict of interest between the funding and recipient parties, and considerable information asymmetry [10].

Another challenge is to increase the chance that the most relevant project concept is chosen. Yet another challenge is to ensure a transparent and democratic process and avoid adverse effects of stakeholder’s involvement and political bargaining. Finally, to make the process predictable is a major challenge since the front-end phase in large public projects commonly would extend over at least one parliamentary election period.

Public investment projects do not always meet the expectations of different stakeholders. Cost overruns are apparently the most common failure reported in the media. In studying more than 4000 large government funded projects, Morris and Hough found that cost overruns were typically between 40% and 200% [11]. Flyvbjerg et al. analyzed 258 infrastructure projects in 20 countries over a 70-year period and concluded that nine of ten projects had cost overruns [12]. Further, Pinto claims that a culture has developed (in the US) whereby decision-makers no longer see any reason to give credence to figures presented in the early phase of projects and instead acknowledge already at that stage that cost overruns will occur (normalization of deviance) [13].

Another problem, more serious by far, is that projects fail in strategic terms, even if it successfully produces its agreed outputs within its budget and on time. Strategic failure means that the choice of conceptual solution turns out to be the wrong one. It could be the wrong solution to the problem at hand or only a partial solution. In some cases, the project may create more problems than it solves so that these outweigh the benefits. In other cases, the initial problem no longer exists once the project is finalized. Serious problems that cause strategic failure typically result in projects that may not be able to produce the anticipated effect, rendering public resources wasted.

In 1997, in recognition of such problems, the Norwegian government initiated a study to review the systems for planning, implementing, and monitoring large public investment projects. The study reviewed eleven project cases in the transport, defense, and construction sectors, and focused on (1) whether the documentation that provided the basis for decisions was adequate when the project was approved, and (2) whether project implementation was satisfactory. The study found that of the eleven projects, only three were completed within the original budget; cost overruns for the other eight were as high as 84%. Moreover, the underlying documentation was deficient in a number of projects. The study ultimately concluded that failures in the initial phase of projects prior to the decision to proceed were generally the main cause of the low success rate for projects [14].

As a result, the Ministry of Finance introduced in 2000 a mandatory quality-at-entry regime to address the challenges described above. The aim of the regime is to reduce the problems with cost overruns, i.e., to ensure operational success, as well as improve the choice of conceptual solutions in order to ensure that the right projects are started and that unviable projects are rejected, i.e., to improve tactical and strategic success. The regime was designed to improve analysis and decision-making in the front-end phase, particularly the interplay between analysis and decision-making.

Under the Quality-at-Entry regime, highly-skilled external consultants, pre-qualified by the Ministry of Finance, are assigned to perform quality assurance of the decision documents in all public investment projects with a total budget exceeding NOK 750 million (approximately EUR 80 million). It involves two separate decision gates with preceding quality assurance of decision documents regarding (1) the choice of conceptual solution (QA1), and (2) the budget, management structure, and contract strategy for the chosen project alternative (QA2) (see Figure 1).

The regime was based on the notion that the necessary rules for decision-making was already in place, but ministries and agencies are now required to produce documents that are in compliance with certain requirements. These are essentially in accordance with procedures in systems engineering, as outlined in the outset of this paper.

The review process is fairly similar in both QA1 and QA2. The reviewer first receives documentation from the sectorial ministry and its subordinate agency and then examines the documentation to check whether it provides a sufficient basis for decision-making. If the documentation is insufficient, additional information may be requested. The reviewers also conduct independent analyses and calculations (a cost-benefit analysis, and, in QA2, an uncertainty analysis).

QA1 is a qualifying step for QA2, and QA2 is a qualifying step for submission to the budget process. Having QA1 performed on a project does not guarantee that QA2 will be performed, and having QA2 performed on a project does not guarantee that the project will be prioritized by Parliament. The QA1 process begins with a decision in the sectorial ministry responsible for the project and the participation of the Ministry of Finance as a quality body. When the external reviewer submits his report, the case is evaluated by the Ministry of Finance and presented to the Cabinet, which then decide whether to proceed with the project. The sectoral ministry may also decide to stop the process after QA1. If a decision is made to continue with a pre-project, the resulting document will be subjected to external review (QA2).

The current procedure during the QA1 phase is for the responsible Ministry to prepare a Conceptual Appraisal (CA) report or pre-feasibility study of the investment case. This report should include the following documents:

• Needs analysis. In this document, all stakeholders and affected parties are identified, and the relevance of the anticipated investment in relation to their needs and priorities is assessed.
• Overall strategy. Based on the prior analysis, consistent, realistic, and verifiable immediate and long-term objectives are specified in this document.
• Overall requirements. This document identifies all the requirements, such as functional, aesthetic, physical, operational, and economic requirements, that need to be fulfilled.
• Possibilities study. With the “opportunity space” delimited by the needs, objectives, and requirements, this document provides the limits to what is possible and identifies realistic alternative conceptual solutions.
• Alternatives analysis. At least two alternatives and the so-called zero alternative (no project) are analyzed to specify their operational objectives, essential uncertainties, cost estimates, and so forth, and the alternatives are subjected to a full cost-benefit analysis that is reported in this document.
• Guidelines for the pre-project phase. This document includes a suggested implementation strategy for the preferred alternatives.

The CA report is then scrutinized by external reviewers (QA1). In addition, they perform a complete cost-benefit analysis of the alternatives based on guidelines from the Ministry of Finance. The reviewers present their findings in a report containing their assessment and advice regarding the following:

• Uncertainties likely to affect the project
• The anticipated economic benefits and costs of the concepts analyzed
• The ranking of the alternatives
• The management strategy

Finally, the reviewers write a report and present it to the sectorial ministry and the Ministry of Finance. The report is generally then made public.

The purpose of QA1 is to assist the Ministry in ensuring that the decision regarding the choice of the conceptual solution has been subjected to a fair and rational political process. Ultimately, of course, the concept is selected through a political process in which external reviewers play no role. The reviewers’ role is limited to controlling the professional quality of the underlying documents that provide the basis for the decision. As a fundamental requirement, at least two viable alternative concepts, in addition to the zero alternative, should be reviewed.

The Ministry now analyses the documents and presents the case to the political level. Being the project owner, it must determine how to best solve the underlying problem that triggered the project and the associated societal needs. The Cabinet then makes the decision regarding the choice of conceptual solution and decides whether to proceed with the pre-project phase.

The second decision gate is based on a strategy document produced by the responsible ministry/agency after the pre-project phase. The emphasis is on cost, timing, uncertainties, etc.

The document is subjected to external quality assurance. After the QA2 report is delivered, the Cabinet still has two options—either stop the project, or allow it to enter into the budget process, but without any guarantee that it will be prioritized [15].

4. Systems Analysis

Large projects are complex undertakings and difficult to analyze at an early conceptual stage. Objectives may be unclear, information scarce, means of implementation unresolved, etc. Systems analysis and system dynamics offer powerful tools to cope with complexity. They recognize that in any complex system, the many circular, interlocking, sometimes time-delayed relationships among its components, may be just as important in determining its behavior as the individual components themselves.

While dynamic simulation models are used with success to describe complex physical processes that are rooted in the natural sciences, their application in dealing with societal problems is more difficult. To analyze large projects in their societal context adds an order of complexity. More so in attempts to estimate their second and third order effects up-front. Mathematical simulation may be less suitable to analyse self-adjusting, societal processes. The principal difficulty is that, as opposed to a physical system, the elements of a societal system can make their own decisions. A physical system is, in principle, predictable both at the elementary level and at the aggregate level. A societal system can be predictable up to a point at the elementary level but only to a lesser extent at the aggregate level [5].

In the Norwegian scenario described above, therefore, systems analysis, is performed as a series of analytic steps based on quantitative and qualitative information, rather than simulation. The requirements posed to the anticipated project define the system boundaries that delimit what is termed the opportunity space, illustrated in Figure 2. The needs and goals express what we want to achieve. The requirements limit the space by defining the premises for which conceptual solutions are feasible and which are not. At the same time, the opportunity space is bounded by what is rationally feasible and politically possible.

The opportunity space should allow for creative thinking, and at the same time restrict and rule out unfeasible conceptual solutions. If the requirements are formulated in such a manner that they greatly restrict the choice of technology, functionality, or operational issues, the opportunity space is compressed to allow for only a certain type of solution. Unfortunately, such situations are quite common, not least when the requirements have been formulated as sector-specific. Another limiting approach is to anticipate political acceptability when defining requirements and the opportunity space. Different sectors have their own traditions, and defining requirements and alternative concepts within these traditions tends to restrict reasoning. If the restrictions are too few or non-specific, the opportunity space will be too wide and give little guidance regarding what is acceptable and not.

5. Effects of the Regime

After 15 years of operation (2015), nearly 200 projects had been subjected to QA2 reviews, and about 80 of these were completed and in the operational phase. The QA1 scheme had been in operation for ten years, and about 70 projects had been through a conceptual appraisal (CA) followed be an external QA1 review; however, none of these projects have been finalized thus far (2017). To date, only eleven of the projects have undergone both QA1 and QA2.

With few exceptions, the projects subjected to QA1 and QA2 represent major public investments with an expected investment cost above the threshold value of approximately EUR 80 million. For most of the projects, the cost estimates range from EUR 70–300 million; however, some of them have a much higher cost estimate. For example, the acquisition of new fighter aircrafts is estimated to cost about EUR 7 billion.

About half of the projects fall under the purview of the Ministry of Transport (mainly road and rail); the other half are mostly projects under the Ministry of Defense, and construction and ICT projects in different parts of government. The agency for the construction of public buildings (Statsbygg) under the Ministry of Local Government and Modernization is also involved, as is the Ministry of Finance in its role as the manager of the QA scheme.

5.1. Strategic Issues

A decade after the first CA and corresponding QA1 reports were produced, it is still too early to evaluate the effects of the scheme. Our knowledge is so far limited by the type of projects involved, the quality of CA and QA1 reports, and the resulting decisions. Indirectly, one can also infer some of the spinoff effects in government, industry, and academia after the introduction of the scheme.

A review of such documents from the projects that had been analyzed, as per 2013 (57 projects) concluded that there is little doubt that the quality of the CA reports has improved steadily over time and that there is a convergence towards a common practice. The same trend can be observed with the QA1 reports—quality assurers have gained years of experience and shown a positive learning curve [16]. Some reviews in the literature have already examined the performance of the CA/QA1 process in the transport sector and stakeholders’ experience with the scheme; see, for example [17,18,19]. These studies highlight that the CA/QA1 process may consume time and resources, but overall, agencies seem to benefit from the scheme. In particular, the scheme provides a more systematic approach to the early identification of project ideas than in the past. Rather than going straight to selecting road sections and determining a technical solution, planners are forced to take a broader perspective and to discuss societal aspects, which allows ideas to mature and stimulates creativity in the agencies. The process in the scheme also increases the likelihood that the most effective option will be included in the analysis.

The QA1 scheme allows the ministries and government to have a more direct influence in the early stages of the process in comparison to local stakeholders, who have traditionally had a significant influence, especially in road projects. However, there is room for improvement. One in-depth study of 17 projects specifically examines how the opportunity space is defined and utilized in CA reports [20]. A recurrent problem is that the conceptual solution has already been selected before the CA process, either because of path dependency in the agencies or political constraints and limitations. Another study suggests that quality assurers seem to give disproportionate attention to economic considerations and that they should balance economic impacts with the achievement of various political objectives. Finally, some ministries and agencies have drawn attention to the futility of undergoing the full CA/QA1 process in cases where, in their opinion, there are simply no alternatives apart from one feasible conceptual solution [17].

In studying the CA and QA1 recommendations and the resulting decisions for the first 70 QA1 projects in 2015, researchers have found that quality assurers agree with the sectoral ministry on the ranking of concepts in one-third of the cases. In the remaining two-thirds of the cases, the quality assurer and the sectoral ministry disagree on the ranking of concepts. The QA1 reports more often recommend the zero alternative or a more economically feasible concept. In the QA1 report, the quality assurer often criticizes the sectoral ministry for its failure to explore the entire opportunity space, particularly with respect to less expensive concepts, during the CA process. Most of the 70 projects have now been through political treatment by the Cabinet, and almost 80% of them have entered into the pre-project phase with one (or sometimes more than one) concept. In only 6% of the cases, the Cabinet has rejected the project altogether, normally in accordance with the QA1 recommendations. Not surprisingly, we observe that when the quality assurer approves the CA recommendation in the QA1 report, the Cabinet normally follows the recommendation. The QA1 process, thus, increases the confidence that the proposed concept is the most efficient and effective alternative. However, when the recommendations diverge, the outcome is less predictable. In such cases, the Cabinet follows the recommendation by the sectoral ministry more often than the QA1 recommendation but, in some cases, project proposals are withdrawn, sent back to the sectoral ministry for new CA appraisal, or the Cabinet chooses a completely different concept [21]. The results indicate that the Cabinet is now better informed about the consequences of projects, mostly the economic consequences, and that they take QA1 recommendations seriously; however, the choice of a project concept clearly remains a political decision.

A 2015 study examines whether the Norwegian government is better informed when making decisions about major public investment projects today compared with the period before the QA scheme was introduced [22]. Based on a sample of projects from before and after the QA scheme was introduced, as well as interviews with 24 public planners and leaders, the study shows that

• the quality of the decision documents has improved considerably;
• the essential factors for the choice of conceptual solution are covered in the CA/QA1 analyses;
• the premises for making a decision are well documented;
• recommendations are clear and transparent; and
• the projects are sufficiently mature and ready for Parliamentary approval after QA2.

A significant feature of the Norwegian QA scheme is the spin-off effects that it seems to have had on both the government and private sector. In the period after the scheme was introduced, we find a clear trend of improved practices in the areas of cost estimation and budgeting, risk assessment, and strategic planning. Moreover, there is growing awareness in the government regarding the need to improve the quality of decision documents, broaden the scope of analyses to include alternative concepts, and avoid making overly detailed analyses at an early stage. Such awareness has also proliferated into the consulting and construction industries, which have clearly responded positively to the new procedures and requirements in these areas in their role as suppliers to the public. We can also see that front-end management has become an issue within the community of professionals in project management, and training courses are now being offered by a number of institutions and consultants. Improved practices have also been adopted and institutionalized by various government agencies. Sectors not subjected to the QA scheme, notably health authorities, electric utilities, and the Oslo municipal authority, have voluntarily introduced variants of the scheme [23,24,25]. Other countries have also shown interest in the scheme. For example, in Sweden, a variant of the CA report, directly inspired by the Norwegian scheme, was introduced in 2013 as a new step in early planning. The Province of Quebec, Canada, has also introduced a similar scheme.

5.2. Operational Issues

Under the current scheme, cost estimation is based on systematic stochastic analysis. The reason for this is that simple deterministic cost estimates are often systematically skewed and do not provide sufficient assurance that the cost frame eventually adopted by Parliament will hold. By means of stochastic estimation, either based on mathematical-analytical methods or simulation tools, the result is a cumulative probability distribution of investment cost as in Figure 3. The proposed cost frame is normally P85 with deductions for possible simplifications and reductions (reduction list) that can be handled during implementation if the cost frame would be in danger of being exceeded. The agency’s steering frame is lower, normally at the P50-level, in order to avoid incentives to use contingency reserves (the agency should have a project management steering frame, which is even lower).

A total of 78 projects submitted to QA2 review were completed by 2015, and the final cost has been established for 67 of these (84%) [26].

The data show that as much as 53 of the 67 projects, i.e., about 80%, were completed within or below the cost frame, Figure 4. The total net savings for the projects taken as a whole was almost EUR 600 million, or about 7% of the total investment. This is a radical improvement from the pre-1997 scenario. There are improvements in all sectors, however, the roads sector has the largest number of cost overruns (33% of the projects).

One factor that may influence the problem of cost compliance is the date on which the project was commissioned. Indeed, the outcome of a project may be positively affected by learning effects, or positively or negatively affected by economic cycles, in the sense that price changes may be greater than the variation captured by the uncertainty analysis. The data showed a vague tendency for cost overruns to have occurred in the middle part of the period, i.e., 2004–2008. On average, the projects in this period had final costs that were relatively equal to their cost frames (0% deviation), whereas the mean deviation for projects commissioned before and after this period was 13%, i.e., considerable cost savings. This vague tendency may be due to strong cost increases in the construction industry that occurred in this period. Alternatively, the subsequent global financial crisis of 2007–2008 may have had unforeseen consequences.

The executing agency is authorized by the Ministry to operate within a steering frame, which in most cases coincides with the estimated median (P50). With steering frames at this level, we should expect equal numbers of overruns and underruns, and with a sufficiently large portfolio, the average for the whole portfolio should be close to the median.

Data demonstrated that this was as expected. Cost overruns and cost savings were almost symmetrically distributed around the median, indicating that cost control at the portfolio level is good.

Since 2011, extensive ex post evaluations have been carried out of the first 19 projects that have undergone QA2 and are now in their operational phase. The results indicate that, overall, their rate of operational success is high, i.e., not only in financial terms. Only six of the projects exceeded their cost frames, whereas seven exceeded their steering frames. Furthermore, only three experienced delays and four had (insignificant) shortcomings related to quality and functionality. The projects were essentially well organized and executed. Most risk factors that do indeed materialize were identified in the QA2 reports. However, notably, in one of the projects, expensive adjustments and upgrading were necessary in the first few years after it began. This demonstrates the importance of focusing on the life cycle cost, not exclusively the investment cost.

These results indicate that, at the portfolio level, the Norwegian state is now effectively controlling costs in major investment projects. This result is likely due to the introduction of a standardized analytic procedure rooted in systems analysis with subsequent external quality assurance, but also the practice of establishing a lower steering frame for the executing agency (typically at the P50 level).

6. Conclusions

Lessons so far indicate that projects subjected to a systems engineering approach are now largely completed within their cost frames. Hence, at the portfolio level, the state is able to more effectively control the cost of major investment projects. Perhaps the most intriguing aspect, as compared to other studies of cost compliance in projects, is that current figures are consistent, based on the same principle and methodology for all projects and, therefore, comparable. The good results concerning cost compliance should however not be confused with evidence that Norway implements its major projects more cost-efficiently than other countries. What the QA regime cannot prevent is an increase in the project content, scope, and cost during the process that precedes QA2. It should be a topic for future research to investigate whether this is a prevalent problem.

It is, however, too early to determine whether quality assurance of the choice of conceptual solution (QA1) has increased the tactical and strategic success of projects, but it is clear that the systematic appraisal of the choice of conceptual solution is considered useful.

The Norwegian QA regime is a novel and simple approach for improving project governance. With a few exceptions, most notably the lack of requirement for risk capital from those who initiate and benefit from the project, the scheme is in line with Flyvbjerg’s recommendations [12]. It ensures that the projects are mature and well-defined when presented to Parliament, it requires the use of stochastic cost-estimation methodology, and it contains incentives for the agency not to use the whole cost frame. So far the scheme seems to work well in its context. Other countries with different institutional set ups, specific types of projects, and major, long-lasting projects may require other approaches.

Project governance has become an issue in the project management community only recently. To move forward in this field, numerous questions need to be answered. What procedures are applied in different countries and agencies, and what are their effects? What would it take to develop more effective governance regimes at the international, government, or corporate level in order to ensure maximum utility and return on investment for society and investors? What is the optimal mix of regulations, economic means, and information in improved governance regimes for major investment projects? A challenge for the project management community will be to shift the focus beyond the delivery of the project itself to the broader issues of the project’s utility and effects. A recent study compared governance schemes for major public investment projects in Norway and five other countries [27]. There are many similarities between the schemes, but also differences regarding, for example, the number of project phases, decision points, and interventions, and who performs quality assurance. All schemes are of a fairly recent date, and it is too early to explore their effects and compare their success, but this would be an interesting topic for future studies. An increased understanding and sensitivity in this area could be of mutual benefit to both the financing and the implementing parties.