Public Private Partnerships, a Value for Money Solution for Clean Coal District Heating Operations

Abstract

Although district heating is an energy-effective solution for cold countries, the coal-fired district heating sector is still facing significant challenges as regards sustainable development, during its operational stage. In order to achieve better operational performance, public private partnerships (PPP) have been introduced in relation to managing clean coal district heating, as a value for money (VfM) solution. To date, however, studies of the application of clean coal in district heating operations are rare and the lack of an evaluation framework hinders the effectiveness test on this sustainable solution. This research aims to investigate the effectiveness of PPP as a VfM solution to the operation of clean coal district heating, by developing an assessment framework on the basis of three dimensions of sustainability: economic, social and environmental, and discussing the proposal, by means of a case study. The assessment results show that the PPP operation offers good VfM performance but with room for improvement, including more user engagement. This research contributes to the literature on clean coal district heating and PPP by providing a comprehensive assessment framework for evaluating VfM performance of the concept. The application of the framework can help the authorities with a robust PPP assessment and thus, improve the sustainable performance of the clean coal heating operation.

1. Introduction

In some cold countries, district heating is perceived as being capable of lowering peak electricity demand and being more energy efficient in providing heating for large urban areas in winter [1,2,3]. The traditional district coal-fired heating system has, however, been criticised in terms of its low energy efficiency, extensive greenhouse gas (GHG) emissions and heavy air pollution [4]. Cleaner coal (e.g., anthracite) or coal products (e.g., clean coal briquettes) could temporarily serve as substitute in rural household heating districts, in providing sufficient resources and infrastructure facilities [5].

China is the largest coal consumer in the world and, traditionally, coal has been the primary energy source. Although the share of coal in the energy consumption of China has decreased in recent years, it still accounted for 62% of its total energy consumption in 2016 and the total consumption of coal has increased by 14% in the last 10 years [6]. Searching for effective low carbon solutions for coal-fired district heating is equally important to China and to the world, in terms of carbon mitigation. The heating industry, mostly in the form of district heating, is a large coal consumer [7], consuming 241 million tons of coal in 2015 [6]. This dependence of the heating operation on coal will continue for decades, as it has been estimated that the share of renewable energy in China will only reach 30% by 2050 [8]. Recently, the Chinese government introduced clean coal technologies into district heating, as pilot projects for energy saving and carbon reduction purposes. Unfortunately, the application of clean coal district heating has not always been satisfactory due to the lack of professional knowledge and management skills of the local public authorities. Furthermore, the potentially significant financial investment required for purchasing the necessary assets and equipment is one of the major challenges to the application of clean coal technologies [9,10]. In response to these practical problems that constrain the promotion of new technologies, many countries have adopted a public private partnership (PPP) approach as a new solution for infrastructure development, due to its advantages in rapid financing, risk transference, professional whole-life project management and goal of long-term sustainability [11,12,13]. As an innovative way for public facilities to be financed and operated, PPP allows the public sector to define the requirements of public service needs and ensure the delivery of the outputs through contracting with the private sector, which is perceived as having the ability to deliver a better quality of public services and value for money (VfM) [14]. Notwithstanding this, the effectiveness of a PPP as a VfM solution for promoting new and sustainable technologies has rarely been discussed in previous researches.

Although many PPP projects are successful, there are some unsatisfying results revealed in previous studies [15]. In those cases, lack of experience and understanding of the mechanism within PPP to achieve VfM limits the public sector in making an appropriate evaluation of project performance [16,17]. It is believed that, the failure of PPP projects is broadly attributed to the issues involved in project initiation, governance, financial arrangements and life-cycle performance measurement [15,18,19,20]. VfM is an important measure regarding the performance of PPP Projects [21] and an expression of economic satisfaction, efficiency and effectiveness [22]. Despite the advantage of PPPs in raising social capital and promoting the transformation of government functions, the public sector should focus on how to improve the performance of PPP projects and achieve the best VfM. The performance of the PPP project during its long-term operational stage is one of the key factors in assessing VfM of PPP [23].

It has been acknowledged that the development of a whole-of-life cycle evaluation system for PPP operations is urgently needed [24]. The absence of a VfM assessment framework for PPP projects in the operational stage, however, makes it impossible for the public sector to test whether the performance of the facilities constructed by private contractors, could achieve better VfM for the public. In order to improve the sustainable performance of the coal-fired district heating sector, it is critical to understand the efficiency of clean coal technology by observing and assessing its operation in real projects. This research aims to assess and improve the VfM performance of a PPP clean coal district heating operation and fills a gap by developing a VfM assessment framework for the operational stage of such PPP contracts. The research questions, hence, include: (1) What criteria can be used to measure the VfM performance of clean coal district heating, during the operational stage? (2) How can we assess the VfM performance of PPP project, at its operational stage (3) Is PPP a VfM approach to clean coal district heating?

2. Theoretical Background

2.1. Clean Coal and District Heating

There are a number of researches focusing on the technology development and efficiency improvement of clean coal technologies [25,26,27]. A financial analysis in the United States discovered significant financial return for the public on investment in clean coal technology programmes, whilst some management weaknesses were also discovered [28]. It was estimated that the application of clean coal technologies in Nigeria could provide an increase of over 30% in the total electricity generated [29]. A review study on the development status of clean coal technologies in China claimed that the main clean coal technologies have been developed rapidly in recent years and are ready to be put into operation [4]. Clean coal technologies are believed to be one of the key options for China in achieving sustainable development, due to their high efficiency and lower emissions [30]. There are, however, still some constraints on the application of clean coal technology in China, including high capital investment [31]. Suggestions have been provided by researchers, including raising public awareness and enterprises’ enthusiasm for clean coal [31], as well as encouraging the public and industrial participation in promoting the implication of clean coal technologies [32].

Recent research relating to district heating have mostly focused on alternative energy sources and technologies, such as waste heat [33,34], solar heat [35], biomass [36,37,38], heat pumps [39,40,41] and natural gas [42]. In order to handle the low energy efficiency and pollutant issues of coal-fired district heating, a low temperature air source heat pump was proposed as offering a more effective heating system to coal-fired district heating, to be used in Beijing [43]. Other research in Poland also specified the pollutant issue of the coal-fired district heating system and stressed the urgency of upgrading the current coal-fired district heating [44]. Since clean coal is that offered by sub-bituminous coal and lignite coal, the latter should be targeted, to encourage a clean coal policy [43]. After reviewing the current clean coal application in China, it was claimed that lifecycle management for clean coal utilisation was critical, for the success promotion of clean coal technologies [31].

There are very few researchers discussing the application of clean coal in the operation of a district heating sector, except for [45] and [46]. Moreover, the evaluation of the operational performance of district heating with clean coal technology seems to be largely ignored by researchers.

2.2. Public Private Partnerships (PPP) and Value for Money (VfM)

As an innovative procurement method, a PPP provides the public sector with a new way of allowing the private sector to manage the whole lifecycle of public projects, which are anticipated as providing better operational performance for public services. The key features of PPP can be seen from various definitions in literature, including the long-term management, cooperation, VfM and sustainability [11,12,47]. The long-term infrastructure contract partnership has become mainstream in the world [48], as the main drivers for the public sector to adopt the PPP contract for construction efficiency, innovation, risk management and financing [49]. VfM assessment has been widely employed by the public sector in different countries at the project initiation stage, to test the PPP approach as a potential VfM option for their projects. As VfM assessment can ensure that partnerships are continually being developed, in order to reach the desired monetary and social targets it is the driving force behind infrastructure and public service procurement, by which governments can select the procurement method that creates the most VfM [11,50]. Unfortunately, VfM assessment is currently only applied at the feasibility stage of public projects, although the quality of the public service provided by the private sector can only be best observed at the operational stage. Many countries do not yet require ex-post VfM assessment at the operational stage of PPPs [51,52]. [11] suggested that, ex-post VfM assessments should be conducted, to determine whether VfM has actually been delivered, as the ex-ante assessment of VfM is inadequate.

In addition, a positive relationship has been discovered between a PPP project ex-post performance and VfM assessment at different stages of PPP projects, including the initial stage and the operational stage [53]. [54] suggested that a systematic long-term VfM evaluation, including the initial phase, purchase phase, construction phase, operational and maintenance phase, would help the public to draw more definitive conclusions on the merits of VfM. Similarly, [55] believed that, a process-based lifecycle performance measurement that strategically places emphasis on VfM should be introduced into PPPs, to replace traditional ex-ante and ex-post evaluations.

The evaluation of a PPP project is a complex process, which uses absolute time and cost measurements, which do not reflect the complexity associated with PPP delivery [56]. VfM assessment should, thus, cover a wider range of issues within qualitative and quantitative contexts, such as whole-life cost, sustainability, service quality, social benefits and maintainability [48,57,58,59]. For instance, [18] established a VfM framework, which considered three key factors of a sustainable energy system (energy security, environmental sustainability, price affordability), as being the measurement of project outcomes. [16] selected criteria for VfM, which addressed social progress and economic objectives, for the promotion of intra-generational equity.

The objectives of PPP are often linked to the long-term social, environmental and economic benefits to be achieved. Performance improvement and sustainability throughout a PPP project’s lifecycle could be achieved by process-based lifecycle performance measurement and a series of key performance indicators [55]. Consequently, sustainability indicators should be introduced into the assessments of PPP projects. The principles of sustainable development have not been fully incorporated in the theory and frameworks of PPP, although, their adoption could result in better infrastructure, more benefits to the society and VfM improvement [60]. In order to assess the long-term sustainable performance of PPP projects, [13] developed a sustainable project management assessment framework for the operational stage of PPP, which contains the most commonly considered three dimensions: economic, social, and environmental sustainability.

Through a literature review, the indicators relating to VfM assessment during operational stage of PPP were identified. They were categorised into three groups of main criteria: social sustainability, environmental sustainability and economic sustainability dimensions, as shown in

Table 1. Evaluation indicators for value for money (VfM).

Main Criteria	Sub–Criteria	Literatures
C1 Economic sustainability	C11 Operation cost	[16,61,62,63,64,65,66]
	C12 Economic incentives	[61,62,65,67]
	C13 Economic risk management	[16,18,64,67,68]
C2 Social sustainability	C21 Health and safety	[13,68,69,70]
	C22 Social incentives	[61,64,68,71]
	C23 User satisfaction	[64,68,72,73]
	C24 Social risk management	[16,18,67,74]
C3 Environmental sustainability	C31 Environmental management	[18,61,69,70,71]
	C32 Environmental incentives	[61,66,68,69]
	C33 Resources and energy consumption	[13,18,75,76]
	C34 Environmental risk management	[16,18,67,71]

. Most of the sub-criteria are qualitative, with the exception of the operational cost and resources and energy consumption, which are quantitative criteria.

3. Research Methodology

In order to achieve the research objectives, various research methods, including a literature review, a questionnaire survey, an interview survey, a fuzzy analytic hierarchy process (FAHP), and a case study method have been applied in this research. The research process includes four main stages, as shown in Figure 1 below.

3.1. Selection of VfM Evaluation Criteria

The VfM assessment criteria are identified through a structured literature review method, following [77]. Relevant literature published in peer-reviewed international journals from 2008 to 2019 were collected from four main academic databases: Web of Science, ASCE, Elsevier-Science direct, and Taylor & Francis. The search keywords include PPP, value for money, sustainability, PPP operation. A total of 135 articles were chosen after the initial search by reading the abstracts, which were then reduced to the 34 most relevant articles, after screening the full-text of the literature. The criteria were generated and classified on the basis of a literature review. The in-depth literature review was then validated by a questionnaire survey. The membership degree test was used to verify the suitability of the selected criteria and their classification (see Table 1) in assessing the operational stage of PPP projects for VfM assessment purposes [78]. There were 30 experts involved in the membership degree test, who were selected by the following criteria: (1) more than 30 years old, (2) have over 3 years working experiences in project management, (3) titled engineer or above. The experts’ profiles are shown in

Table 2. Participating in interview survey.

	Classification	Number
Age	30–40 years	14
	41–50 years	12
	51–60 years	4
Sectors	Research institute	12
	Consultancy	6
	Government	8
	PPP Contractor	4
Working experience	3–6 years	13
	7–9 years	10
	10–12 years	6
	>12	1
Working title	Senior staff research scientist/research	15
	Senior engineer/associate research fellow	12
	Engineer/lecture	3

. The expertise of the experts covers a wide range of sectors including researchers, consultants, government officers and contractors.

The Blechinger’s 5-point Likert scale ranging from ‘1’ for ‘No importance’ to ‘5’ for ‘Extremely importance’ [79] has been used in this test. The criteria with the average score exceeding 3.0 were selected as the VfM assessment criteria. The suitability of the classification was measured by a 2-point scale (0–No, 1–Yes), indicating whether the sub-criterion was suitable to be classified in the designated main criterion group. The classification was believed to be suitable, when the number of “Yes” responses exceeded 80% of the total number of answers.

3.2. The Calculation of Weighting Coefficient

The fuzzy analytic hierarchy process (FAHP) method provides analysts with an appropriate way of handling both qualitative criteria and quantitative criteria at the same time [80,81], which has been widely utilized in project assessment researches [82,83]. This approach can effectively avoid processing the inaccurate relative importance value of a decision-maker’s subjective judgment as a crisp numerical value and ensure that the results are appropriate and effective.

A structured interview survey was conducted for data collection purposes. 8 out of 30 experts accepted an invitation to take part in the structured interview, for pairwise comparison, which aimed to measure the relative importance of each main criterion and sub-criterion. Each interview with the experts lasted 40 to 60 mins and was tran scripted into text, with the permission of the experts.

The determination of weighting coefficients of the criteria follows 6 steps.

Step 1: Determining criteria and developing fuzzy linguistic scale

The linguistic variable is defined by linguistic terms, in a natural or artificial language [84]. The triangular fuzzy membership function UA(x): X → [0,1] for the fuzzy number $\tilde{C}$ = (l, a, u) is shown at Equation (1).

$${\mathrm{U}}_{\mathrm{A}\left(x\right)}=\{\begin{array}{c}\frac{x-\mathrm{l}}{a-\mathrm{l}},l\le x\le a\\ \frac{u-x}{u-a},a\le x\le u\\ 0,otherwise\end{array}$$

(1)

The 9-point fuzzy scale [85,86], as shown in

Table 3. The relative importance of fuzzy scale.

Linguistic Variables	Relative Importance	Triangular Fuzzy Number
Absolute more important	9	(8,9,9)
Intermediate value	8	(7,8,9)
Very Strongly more important	7	(6,7,8)
Intermediate value	6	(5,6,7)
Essentially more important	5	(4,5,6)
Intermediate value	4	(3,4,5)
Weakly more important	3	(2,3,4)
Intermediate value	2	(1,2,3)
Equally important	1	(1,1,2)

, is employed to evaluate the relative importance of criteria.

Step 2: Pairwise comparison

After each round of pairwise comparison, an expert’s evaluation is converted into a positive fuzzy reciprocal matrix, according to their scores on relative importance. The fuzzy matrix $\tilde{C}$_(k) is the initial positive fuzzy reciprocal matrix of expert k, as shown in Equation (2).

$$\tilde{C}(k) = \left[\begin{array}{ccc}\widetilde{C11}\left(k\right)& \cdots & \widetilde{C1m}\left(k\right)\\ ⋮& \ddots & ⋮\\ \raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{$\widetilde{C1m}$}\right.\left(k\right)& \cdots & \widetilde{Cmm}\left(k\right)\end{array}\right]$$

(2)

where, i = 1, 2, …, m

m—the number of criteria.

Each element in the matrix is in the form of a fuzzy number ${\tilde{C}}_{ij}$= (l_ij, a_ij, h_ij), where l_ij, a_ij, h_ij represent the lower limit, the average and the higher limit of triangular fuzzy numbers.

Step 3: Summarising the fuzzy evaluation results

The geometric mean of the n positive fuzzy reciprocal matrix, Avg $\tilde{C}$, was calculated by Equation (3), to summarise all the results.

$$\mathrm{Avg} \tilde{\mathrm{C}}=\sqrt[n]{\prod {\tilde{C}}_{ij}\left(1\right)\otimes {\tilde{C}}_{ij}\left(2\right)\dots \otimes {\tilde{C}}_{ij}\left(k\right)\otimes {\tilde{C}}_{ij}\left(n\right)}$$

(3)

where, k is the 1 to n experts;

Step 4: Defuzzification

The defuzzification process derives a crisp number for each main criterion and sub-criterion from Avg $\tilde{\mathrm{C}}$. The most widely used defuzzification method, the composite moment method, is applied in this step [87]. The defuzzification of the main criteria follows Equation (4), whilst the defuzzification of the sub-criteria follows Equation (5).

$$X_i=\raisebox{1ex}{$\left(l_i+m_i+h_i\right)$}\!\left/ \!\raisebox{-1ex}{$3$}\right.$$

(4)

$$X_{ij}=\raisebox{1ex}{$(l_{ij}+m_{ij}+h_{ij})$}\!\left/ \!\raisebox{-1ex}{$3$}\right.$$

(5)

Step 5: Calculating the relative weight of criteria and consistency check

The asymptotic normalization coefficient method, as shown in Equations (6) and (7), is used to normalise the defuzzification matrix X, to a standard matrix.

$$\widetilde{Wi} = \frac{X_i}{\sum X_i}$$

(6)

$$\widetilde{Wij} = \frac{X_{ij}}{\sum X_{ij}}$$

(7)

A consistency check is used to ensure that the analysis results in the pairwise comparisons being acceptable [88]. If the consistency ratio (CR) is lower than 0.10, the ratio is at an acceptable level of consistency. The CR can be calculated by Equations (9) and (10).

$$\mathrm{CI}=\frac{\mathsf{\lambda }\max -m}{m-1}$$

(8)

$$\mathrm{CR}=\frac{CI}{RI}$$

(9)

where λ_max is the largest eigenvalue of a matrix,

m is the dimension of the matrix and

RI is a predefined random index that depends on m.

Step 6: Calculating the final weighting coefficient

The final weighting coefficient Wij of each sub-criterion, considering the relative importance comparisons, is calculated by Equation (11).

$$Wij = \widetilde{Wi}\times \widetilde{Wi}j$$

(10)

3.3. The Implementation of the VfM Assessment Framework

In applying this framework to assessing the VfM performance of the PPP operation, the project specific data should be collected against each sub-criterion in the framework. For the quantitative criteria, the score is calculated by Equation (11).

$$S_{ij} for quantitive criterion = \frac{actual performance - planned performance}{planned performance}$$

(11)

The scores of qualitative criteria are derived from expert consensus meeting. The score indication and standards are shown in a VfM assessment sheet, as shown in Supplementary Materials The five-level scale introduced by the MoF in the ‘VfM Assessment Guidance of PPP (Trial version)’ is adopted in measuring the operation performance [89], where the five score ranges (0–20), (21–40), (41–60), (61–80), (81–100) represent ‘poor’, ‘unsatisfactory’, ‘average’, ‘satisfactory, ‘good’ performances.

The average score of each qualitative criterion was calculated by Equation (12).

$$S_{ij} for qualitative criterion=\frac{\sum S_{ij\left(1\right)}+S_{ij\left(2\right)}+\cdots S_{ij\left(n\right)}}{n}$$

(12)

The final VfM assessment score FS is calculated by Equation (13).

$$FS= \sum W_{ij}\ast S_{ij}$$

(13)

3.4. Case Study Design

For demonstration purposes, a large-scale PPP project, the ZK City clean coal district heating PPP project in Northern China, was selected as a case study to evaluate the effectiveness of PPP in achieving VfM.

The ZK clean coal district heating PPP project is one of the typically large-scale stock projects in the first PPP demonstration projects of the Ministry of Finance, China. Since the outset, the original coal-fired district heating system in ZK city experienced low energy efficiency and low user satisfaction. Due to poor performance in operation, the City Council suffered significant losses each year. For example, in 2014, the Council had to compensate to the extent of 16 million RMB, for the operation of the ZK district heating system. In 2015, the Council decided to adopt a PPP mode, to upgrade the conventional district heating of the city to offer a cleaner operation. By introducing the private sector’s technical knowledge and management skills, the Council expected the project to achieve better VfM for the public users. The scope of the project included upgrading the outdated heating system, by introducing clean coal technologies, as well as the operation and management of the district heating system until 2041. The contract price was 415 million RMB. The heating service covered around 6 million km² of households in ZK city. The refurbishment stage of the project started in September 2015 and was completed in October 2015. The partnership mode of the project is shown in Figure 2 below.

The Special Purpose Vehicle (SPV) signed a project contract with the public client to finance, refurbish and operate the ZK district heating system for 25 years. For this case study, the quantitative data were collected from the project’s official website and project database. Meanwhile, the qualitative data were gathered from an expert consensus meeting with four assessors, these being a researcher, a consultant and two government officers, who had previously experienced the interview surveys and understood the objective of this study. They were required to score on the operational performance of this PPP project, against each qualitative sub-criterion, according to the score indication. The meeting outlines the project information and VfM assessment sheet with qualitative criteria for the ZK project prepared and handed out before the expert consensus meeting. Two rounds of consensus were carried out before the experts reached the final result to input to the fuzzy model.

4. Results

4.1. VfM Assessment Framework

The results of the membership degree survey is shown in

Table 4. Results of the questionnaire survey on membership degree.

Main Criteria	Sub-Criteria	Suitability of the Classification	Suitability of Criteria
C1	C11	94%	4.5
	C12	86%	4.0
	C13	92%	3.9
C2	C21	90%	4.1
	C22	83%	3.8
	C23	92%	3.9
	C24	92%	4.2
C3	C31	92%	4.2
	C32	91%	4.1
	C33	96%	4.5
	C34	92%	4.2

, where the main criteria, sub-criteria and their classification passed the membership degree test.

The VfM assessment framework for the operational stage of the PPP project was established, as shown in

Table 5. The VfM assessment framework for PPP operation.

Main Criteria	Sub-Criteria	Assessment Standards	Type
Economic sustainability C1	Operation cost C11	The ratio change of operation cost = $\frac{actual operation cost - planned operation cost}{planned operation cost}$ × 100%. For 1% increase in the operation cost ratio, 1 point is deducted.	Quantitative
	Economic incentives C12	(1) Payment deduction is linked to asset management quality	Qualitative
		(2) Payment deduction is linked to service quality	Qualitative
		(3) The amount and frequency of payment deduction	Qualitative
	Economic risk management C13	(1) The involvement of the third party in assessing the project’s economic status	Qualitative
		(2) Economic risks are control effectively	Qualitative
Social sustainability C2	Health and safety C21	(1) Distribution of health and safety manuals to the public	Qualitative
		(2) Reporting to the public on health and safety issues regularly	Qualitative
		(3) Operation in line with the life cycle safety and health management plans	Qualitative
	Social incentives C22	(1) The construction and operation of project can improve local employment situation	Qualitative
		(2) Increased publicity and community awareness	Qualitative
		(3) Guiding the public for appropriate usage of facilities	Qualitative
	User’ satisfaction C23	(1) Users’ satisfaction on service quality	Qualitative
		(2) Client satisfaction on project operation	Qualitative
	Social risk management C24	(1) Social risks are controlled effectively	Qualitative
		(2) The involvement of the third party in assessing social impacts during project operation	Qualitative
Environmental sustainability C3	Environmental management C31	(1) Private sectors and property management protect the public areas in good conditions	Qualitative
		(2) Guiding users for energy saving	Qualitative
		(3) Effective waste management such as recycling	Qualitative
		(4) Maintaining public and green spaces in good conditions	Qualitative
	Environmental incentives C32	(1) Payment mechanism linking to energy and resource consumption	Qualitative
		(2) Payment mechanism linking to environmental pollution	Qualitative
	Resources and energy consumption C33	The rate of change in resources and energy consumption = $\frac{\mathrm{energy} \mathrm{or} \mathrm{resource} \mathrm{consumption}-\mathrm{planned} \mathrm{consumption}}{\mathrm{planned} \mathrm{consumption}}$ × 100%. For 1% increase in the energy consumed per unit area ratio, 1 point is deducted.	Quantitative
	Environmental risk management C34	(1) The involvement of the third party in assessing environmental impacts during project operation	Qualitative
		(2) Environmental risks are controlled effectively	Qualitative

. The assessment standards for sub-criteria were explained, to ensure consistency when the assessors were using this framework for assessment.

The criteria relating to economic sustainability(C1) of the PPP operation aimed to measure its performance on cost control and ensure the economic stability of the project. The criteria of social sustainability (C2) indicated the overall social contribution of the project to the employment rate, publicity and community awareness, health and safety and user satisfaction. The environmental sustainability (C3) tests the environment management skills of the project team; including maintaining the public areas in good condition, energy saving, waste management, resources and energy reduction, and implementation of new technologies.

4.2. Weighting Coefficient

The paired comparison results of the main criteria and sub-criteria of the 30 experts were collected and summarised in fuzzy matrixes, as shown in

Table 6. The fuzzy matrix of the main criteria.

C	C1	C2	C3
C1	(1.00,1.00,1.00)	(1.00,1.50,2.00)	(0.50,0.75,1.00)
C2	(0.50,0.67,1.00)	(1.00,1.00,1.00)	(0.25,0.40,0.50)
C3	(1.00,1.33,2.00)	(2.00,2.50,4.00)	(1.00,1.00,1.00)

,

Table 7. The fuzzy matrix of the sub-criteria on economic sustainability.

C1	C11	C12	C13
C11	(1.00,1.00,1.00)	(2.00,3.00,3.00)	(3.00,4.00,5.00)
C12	(0.33,0.33,0.50)	(1.00,1.00,1.00)	(1.00,1.50,3.00)
C13	(0.20,0.25,0.33)	(0.33,0.67,1.00)	(1.00,1.00,1.00)

,

Table 8. The fuzzy matrix of the sub-criteria on social sustainability.

C2	C21	C22	C23	C24
C21	(1.00,1.00,1.00)	(0.50,0.67,1.00)	(0.50,0.80,1.00)	(0.50,1.00,1.00)
C22	(1.00,1.50,2.00)	(1.00,1.00,1.00)	(0.50,1.00,1.00)	(0.50,0.60,1.00)
C23	(1.00,1.25,2.00)	(1.00,1.00,2.00)	(1.00,1.00,1.00)	(1.00,1.33,2.00)
C24	(1.00,1.00,2.00)	(1.00,1.67,2.00)	(0.50,0.75,1.00)	(1.00,1.00,1.00)

and

Table 9. The fuzzy matrix of the sub-criteria on environmental sustainability.

C3	C31	C32	C33	C34
C31	(1.00,1.00,1.00)	(2.00,3.00,4.00)	(1.00,2.00,2.00)	(1.00,1.80,2.00)
C32	(0.25,0.33,0.50)	(1.00,1.00,1.00)	(0.50,1.00,1.00)	(0.50,0.75,1.00)
C33	(0.50,0.50,1.00)	(1.00,1.00,2.00)	(1.00,1.00,1.00)	(1.00,1.50,3.00)
C34	(0.50,0.56,1.00)	(1.00,1.33,2.00)	(0.33,0.67,1.00)	(1.00,1.00,1.00)

.

The defuzzification and relative weight-consistency check matrix of the main criteria and sub-criteria are shown in

Table 10. The defuzzification matrix of the main criteria.

C	C1	C2	C3	$\widetilde{\mathit{W}\mathit{i}}$	λmax	CI	CR
C1	1	1.50	0.75	0.32	3.09	0.05	0.081
C2	0.72	1	0.38	0.20
C3	1.44	2.83	1	0.49

,

Table 11. The defuzzification matrix of the sub-criteria on economic sustainability.

C1	C11	C12	C13	$\widetilde{\mathit{W}\mathit{i}\mathit{j}}$	λmax	CI	CR
C11	1	2.67	4.00	0.60	3.10	0.05	0.090
C12	0.39	1	1.83	0.25
C13	0.26	0.67	1	0.15

,

Table 12. The defuzzification matrix of the sub-criteria on social sustainability.

C2	C21	C22	C23	C24	$\widetilde{\mathit{W}\mathit{i}\mathit{j}}$	λmax	CI	CR
C21	1	0.72	0.77	0.83	0.21	4.28	0.09	0.097
C22	1.35	1	0.83	0.70	0.23
C23	1.42	1.33	1	1.44	0.29
C24	1.33	1.56	0.75	1	0.26

and

Table 13. The defuzzification matrix of the sub-criteria on environmental sustainability.

C3	C31	C32	C33	C34	$\widetilde{\mathit{W}\mathit{i}\mathit{j}}$	λmax	CI	CR
C31	1	3.00	1.67	1.60	0.38	4.27	0.09	0.099
C32	0.36	1	0.83	0.75	0.16
C33	0.67	1.33	1	1.83	0.26
C34	0.69	1.44	0.67	1	0.20

, where CR < 0.1 indicates an acceptable level of consistency.

The weight coefficients of all sub-criteria are calculated as listed in

Table 14. The results of weighting coefficient.

Rank	Sub-Criteria	Weighting Coefficient
1	C11	0.19
2	C31	0.18
3	C33	0.12
4	C34	0.10
5	C12	0.08
5	C32	0.08
7	C23	0.06
8	C13	0.05
8	C22	0.05
10	C24	0.05
11	C21	0.04
Sum		1.00

. The weighting coefficient demonstrates the relative importance of each criterion.

The result revealed that, the top four more important criteria were C11-Operation cost, C31-Environmental management, C33-Resources and energy consumption and C34-Environmental risk management. Together, they represented 59% of all the criteria. Criterion C11, was given the highest weighting by the experts. During the interview survey, one expert ranked the ‘check of the private sector’ actual operation cost with the cost budget plan in the contract’, as being the most critical criterion for VfM assessment of the PPP project operation. According to the expert, the cost check, ‘provided a benchmark for the public sector to assess the performance of the private sector during the operational stage of any PPP project’. Criteria C31 and C33 were ranked as 2 and 3 by the experts, showing the concerns of practitioners about PPP objectives in improving service quality and achieving sustainability.

4.3. The Case Study on a VfM Assessment Framework

The investigation of the case project discovered the advantages of using the PPP approach to applying clean coal technology to the district heating system in ZK city and to maintaining the operation of the system. After the establishment of the PPP project, a patented technology adaption was made to the pump system of the existing boiler room, to ensure timely heating, based on users’ need for heating services. The technology transformation achieved a 65% reduction in the power consumption of the one-cycle circulating pump. At the same time, the project employed a boiler flue gas condensation heat recovery technology to reuse the waste heat from the boiler, which improved the operating efficiency of the boiler. The result of the first operation review showed reductions in coal consumption of 20%, electricity consumption by 50% and water consumption by 30%. It was predicted by the SPV that the project revenue could be increased by 20% through efficiency improvement in comparison to the conventional coal-fired district heating system.

The assessment results of the VfM assessment framework, which was implemented on the ZK clean coal district heating PPP project, are shown in

Table 15. VfM assessment results of case study.

Main Criteria	Sub-Criteria	Weight Coefficient	Average Score	Weighted Score
Economic sustainability C1	Operation cost C11	0.19	85.60	16.26
	Economic incentives C12	0.08	81.90	6.55
	Economic risk management C13	0.05	84.80	4.24
Social Sustainability C2	Health and safety C21	0.04	80.10	3.20
	Social incentives C22	0.05	81.70	4.09
	User’ satisfaction C23	0.06	75.10	4.51
	Social risk management C24	0.05	84.90	4.25
Environmental Sustainability C3	Environmental management C31	0.18	85.20	15.34
	Environmental incentives C32	0.08	84.70	6.78
	Resources and energy consumption C33	0.12	90.20	10.82
	Environmental risk management C34	0.10	83.40	8.34
Total:				84.37

.

The final total score of the VfM assessment was 84.37 points, indicating an overall ‘Good performance’ of the PPP operation concerning VfM. Among the three main criteria categories, environmental sustainability gained the highest scores, with an average of 86 points, whilst the social sustainability gained only 80 points on average. The scores for each sub-criterion ranged from 75.1 to 90.2, which suggests the performances of the project on sub-criteria vary (see Figure 3).

Most of the sub-criteria scored over 81 points, indicating ‘Good performance’ of the PPP operation concerning these sub-criteria. The sub-criterion with the highest score was C33-Resources and energy consumption. As the primary goal of the project was energy efficiency and carbon reduction, the SPV paid great attention to this aspect during the operational stage. One of the assessors claimed that, ‘the PPP mode was a good way to help government to implement low carbon and a less polluting heating supply, by introducing private sector professional management skills’. Furthermore, the low carbon and sustainability concepts and practice ‘should be integrated to PPP projects from tendering, design, construction, and operation to disposal’, in order to, ‘create a low carbon and less polluted world’.

The SPV in this PPP project adopted more proactive measures than had the previous public operator of the ZK district heating system, which leads to, ‘more effective cost control and auditing’, as commended by another assessor. The SPV also targeted user management, by giving users instructions on how to save resources during household interviews. These measures prevented energy waste and asset damage due to misuse of the users. As a result, the PPP operation achieved outstanding performance in terms of C11-Operation cost, in comparison to the previous operation that was solely delivered by the public sector. The third highest score was given to C31-Environmental management. The private sector increased environmental investment and adopted new approaches, including a desulphurisation system and emission monitoring, to reduce air pollution.

There were two sub-criteria given relatively low scores: C21-Health and safety, and C23-User’ satisfaction, which shows that the SPV had paid insufficient attention to these issues during the current operation of the clean coal district heating operation. As health and safety may have a negative impact on public acceptance and willingness to engage, this issue should be promptly rectified. The study of [32] also stressed the importance of improving public engagement in clean coal technology. Operational efficiency and user satisfaction will potentially affect the SPV’s revenue, therefore, it was suggested by the assessors to, ‘improve user satisfaction survey and communication’ and ‘strengthen the health and safety training and management’, in future operations.

Given the significance of district heating demand and the situation whereby China is and will still heavily depend on coal as a source of prime energy, the promotion of clean coal in district heating systems is urgently needed for the sustainable development of the heating sector in China. PPP introduces private professional knowledge and management skills to public services which could potentially be an effective approach in achieving VfM as regards public service provision. The evaluation framework to be used periodically throughout the long-term operational stage of clean coal district heating can assist the authority to identify the ineffective and inefficiency issues and gradually improve the VfM performance of PPP projects.

5. Conclusions

This research explores the effectiveness of PPP as a VfM approach to a clean coal district heating operation, by developing a VfM assessment framework for evaluating PPP operations. In answering the first research question, 11 sub-criteria are identified under three main criteria categories, considering economic, social and environmental sustainability. The C11-Operation cost and C31-Environmental management appear to be the more crucial criteria for the VfM assessment of PPP operations. In order to answer the second research question, an assessment framework was developed and demonstrated by a case study of a large clean coal district heating project in China. The assessment result answers the last research question and shows a generally good performance of the PPP operation, with some areas requiring improvement. The private sector of the PPP project is suggested for improving their performance on C23–User’s satisfaction and C21-Health and safety issues. The results show PPP is potentially a VfM solution for the clean cost district heating operation. The research of [90] on tourism PPP provided similar results, showing PPP as acting as a policy driver in the development of the tourism industry.

Previous researchers have pointed out that the lack of evaluation of the process of PPP was responsible for its inefficiency and ineffectiveness [17]. The VfM assessment framework in this research contributes to the literature on the sustainable performance assessment of public projects. The research also provides the researchers with comprehensive indicators that extend the application scope and theoretical structure of VfM assessment. The implementation of the VfM assessment throughout the operational stage can benefit the government in evaluating the sustainable performance of PPP operations and help practitioners to continuously improve the quality of public services provided through the operational stage of PPP.

As the demonstration of the VfM assessment framework was based on a particular case study, future researchers are recommended to test the framework on a wider range of PPP operations.