Advanced Control and Fault Detection Strategies for District Heating and Cooling Systems—A Review

Abstract

Peak shaving, demand response, fast fault detection, emissions and costs reduction are some of the main objectives to meet in advanced district heating and cooling (DHC) systems. In order to enhance the operation of infrastructures, challenges such as supply temperature reduction and load uncertainty with the development of algorithms and technologies are growing. Therefore, traditional control strategies and diagnosis approaches cannot achieve these goals. Accordingly, to address these shortcomings, researchers have developed plenty of innovative methods based on their applications and features. The main purpose of this paper is to review recent publications that include both hard and soft computing implementations such as model predictive control and machine learning algorithms with applications also on both fourth and fifth generation district heating and cooling networks. After introducing traditional approaches, the innovative techniques, accomplished results and overview of the main strengths and weaknesses have been discussed together with a description of the main capabilities of some commercial platforms.

1. Introduction

The world’s level of urbanization is expected to increase from about 55% in 2018 to 68% in 2050, and 90% of this increment is expected to take place in Asia and Africa, which were home to about 90% of the world’s rural population in 2018 [1]. In Europe, 74% of the population lives already in urban areas today, and this percentage is expected to grow up to 84% by 2050 [1]. Furthermore, in urban areas where the heating and cooling demand exhibits the highest density and the largest load simultaneity, a huge amount of low-grade excess heat is wasted. Moreover, for historical reasons, cities and towns have been born along rivers, lakes, and seashores that are all ambient heat sources in which utilization is highly replicable, because it is accessible right where it is needed. Even, in some cases, such as in London [2], the total heat wasted from secondary sources has been estimated larger than the city’s total heat demand.

Six out of ten of the top European heatwaves between 1950 and 2014 have appeared in the latest 20 years [3]. Extreme weather events, such as wind storms and flooding, have increased in number and intensity worldwide [4]. Some scientists claim that climate change’s evolution and consequences have several similarities with the current pandemic crisis of COVID-19, but in slow motion [5,6]. Nevertheless, in the latest years, several public entities recognized the climate crisis and are going to implement serious actions to mitigate global warming effects. As a result, 2019 has been defined the year of the “climate emergency” declaration. In this context, the European Union, which already demonstrated between 1990–2018 that it is possible to decouple gross domestic product (GDP) growth from greenhouse gas emissions [7], set a very ambitious target to achieve carbon neutrality by 2050.

Moreover, the EU admitted that this could not be accomplished with the current commitments under the Paris Agreement that foresee a total greenhouse gas emissions reduction by only 40% in 2030 with respect to 1990 levels. To boost the global warming fighting process, the new European Green Deal [8] set a more ambitious target that corresponds to reach a greenhouse gas emissions reduction by at least 50% by 2030 compared with 1990 levels with dedicated strategies for the different sectors. In particular, to decarbonize the heating and cooling sector and to improve the air quality in urban areas, the European Green Deal Investment Plan will support the implementation of district heating and cooling (DHC) networks [9]. This technology is the most promising to implement circular economy principles in the heat sector by harvesting and distributing local excess heat that otherwise will be wasted. Moreover, it allows boosting the share of modern renewables to cover the heat demand of buildings that was only equal to about 13.6% in 2017 at a worldwide level [10].

The digital roadmap for district heating and cooling (2019) [11], which has been developed as part of the H2020 project STORM [12], carefully presents how digitalization and the concept of Industry4.0 can push forward the efficiency and operation of DHC networks, empowering their role in an integrated smart energy system. The barriers identified that could hinder the rollout of digital technologies in DHC are not technical but are mainly related to limitations in acting on the building substation, absence of business models or dynamic tariffs to stimulate the flexibility that can be provided by buildings and regulations on private data protection. This study, performed in the context of the H2020 REWARDHeat project [13], aims at reviewing the state-of-the-art in scientific publications and to create a common knowledge from different European research projects of two parts that are fundamental to achieve innovative and robust DHC systems in the near future: advanced control strategies and fault detection and diagnosis solutions. These are presented in Section 2 and Section 3, respectively, with a focus on the novel applications in the Fifth generation of DHC systems (5GDHC) that are not available in previous works, such as [14] and [15]. Finally, the positive and negative aspects of the different approaches encountered are discussed in Section 4.

2. State of the Art Control Strategies for District Heating (DH) and District Cooling (DC) Systems

2.1. DHC as an Enabling Infrastructure for Greater Renewable Sources Penetration

DH/DC networks include a variety of subsystems from heat/cold generation units, distribution pipelines and user substations. In the last decades, while renewable energy sources for electricity generation, such as solar and wind, have made considerable progress, it is not the same for renewable thermal energy generation. Even if the technology is mature, only a few countries such as Denmark, Sweden and Lithuania [16] already achieved a good share of renewable thermal energy production in DH/DC systems thanks to their ambitious decarbonization policies. However, it is worth mentioning that DHC is the key solution to scale up renewable thermal energy in the built environment. On one hand, thanks to economies of scale and, on the other hand, by solving issues that are relevant at the building level such as the lack of space or architectural heritage preservation. In Europe, DH cover about 12% of the heating market share [17] and DC only 2% of the cooling market share [18]. Worldwide, district energy systems supply only 6% of the thermal energy demand and it is strongly based on fossil fuels. In fact, in 2018 only 8% of the energy sources employed in DHC were based on renewables with 95% of this consisting of biomass according to [16]. This high share confirms that biomass is the most competitive source for renewable heat supply in DHC systems of several countries. Moreover, it is easy to implement, since the management of the heat generation plant remains very similar to a traditional dispatchable fossil-based one. Biomass boilers can be derived from the upgrade of existent coal-fueled plants and, in some cases, are coupled with steam-based or Organic Rankine Cycles (ORC) combined heat and power (CHP) systems.

The years between 2020 and 2030 have been defined as the “Geothermal Decade” [19], since a large growth is foreseen for the exploitation of geothermal energy in Europe for both electricity production and direct/indirect use. Even if the location of high enthalpy geothermal reservoirs is geographically constrained, medium enthalpy resources are more diffused. Deep geothermal for direct use in DH can meet the heating demand of about 25% of the European population according to the results of the research project GeoDH [20]. With this regard, Iceland is the leader country in the installed capacity of geothermal DH. The projections for coming years foresee important market growth of geothermal DH, especially in France, Germany, Denmark, the Netherlands and Hungary [19]. Deep geothermal for direct use in DH provide dispatchable baseload capacity, achieving large equivalent full-load hours. Shallow geothermal can be exploited almost everywhere but requires centralized or decentralized heat pumps to lift the supply temperature.

Solar thermal has great potential for DH application, but conversely from geothermal and biomass might introduce more effort for the control of the system that can be supported by forecast tools for both the heat demand and the not-programmable solar heat generation. Denmark is the pioneer country in solar district heating (SDH) and the worldwide leader in large-scale solar thermal fields achieving 1.1 GW_th of installed capacity in the first half of 2019 (almost equal to the PV capacity installed in the country) [21]. This was possible thanks to the competitive heat price from solar thermal with respect to gas (affected by high taxes), high penetration and sound experience with DH networks, cheap land prices and willingness of the community-owner DH companies that run a business that is non-profit by law. From the Danish experience [22], the average efficiency of the solar collector fields is about 40%, and the heat production cost ranges from 20 to 40 EUR/MWh. Concerning the solar fraction over the total thermal energy supply, 20% can be achieved without thermal energy storage (TES) installation, whereas a larger solar fraction can be accomplished with seasonal TES. Among the different technologies, water pit and borehole seasonal TES systems result in the most used in conjunction with SDH. Moreover, the installation of the seasonal TES system paves the way for more heat production flexibility and integration of large-scale heat pumps that improve the performance of the overall SDH system [23]. Further penetration of non-programmable renewable power generation can be also achieved thanks to the DHC systems, since excess electricity generated can be used directly or stored in the form of heat or cold in TES systems avoiding the curtailment of PV and wind production. Moreover, sector coupling by means of power-to-heat technologies, such as electric boilers and heat pumps, unlock remunerative balancing services for the power grid such as frequency regulation.

Traditional DH/DC systems and 5GDHC systems are conceived differently, opposing centralized with decentralized energy conversion systems [24]. Thus, in several cases, they exploit different control strategies that are presented in the next sections.

2.2. Basic Control Strategies in Traditional DH/DC Systems

Six independent basic control strategies operate simultaneously in a traditional DH network, as depicted in Figure 1 and listed in the following. These are usually implemented in the form of classical control such as rule-based or PID. According to Frederiksen and Werner (2014) [25], the ones from A to E are implemented at the production and pumping station level, whereas the control strategy F is implemented at the user substation level.

A. Supply temperature control: The goal of this control is to module the thermal power at the heat generation plant to reach a fixed supply temperature set-point. The latter is generally determined by means of a heating curve as a function of the external ambient temperature, and it is high enough to satisfy the temperature level required for all the customer-sited heat emission systems.

B. Minimum supply temperature control: This control is important, especially in summer, when the DH system operates to supply only the DHW load. During the night, for instance, a by-pass between the supply and return pipelines at the end of the network allows maintaining a small recirculation in order that the supply temperature always satisfies the comfort level for the users.

C. Minimum differential pressure control: It is essential to maintain a minimum differential pressure between the supply and the return pipelines at the farthest substation from the network pumping station. In this way it is assured that the pressure head is enough to supply all the substations in the between.

D. Maximum pressure control: The pumps are controlled to avoid over-pressure at the supply line that could damage the network.

E. Minimum pressure control: The minimum pressure of the network that occurs at the suction of the pumps is controlled to avoid cavitation and damage of components.

F. Heat demand and flow control: These control strategies are implemented at the user substation level. For instance, a climatic control can be implemented to set the supply temperature on the substation’s secondary side. A modulating valve is then used to achieve this set-point by controlling the flow rate on the substation’s primary side.

Thermal energy storage (TES) systems based on short-term tanks provide several advantages in traditional DH/DC networks. They allow equalizing the load of the heat production plants, while heat demand by consumers is fluctuating, and covering the peak demand by avoiding that peak generation units switch on with high operational costs. Moreover, TES systems allow operating the system when the heat demand is below the technical minimum of the heat production plant and increasing the flexibility of combined heat and power (CHP) systems by permitting electricity generation during off-peak hours of the heat load without wasting the heat produced. Several temperature meters are usually installed along the height of a tank TES to monitor the temperature stratification and state of charge. To exploit the capacity of TES directly connected to the DH network such as in Figure 1, few further control measures are needed: the charging and discharging phases are activated automatically by generating the right amount of positive/negative differential pressure at the connections of the TES at the supply and return DH pipelines by controlling properly the network and production pumps. Moreover, usually, a control to limit the minimum temperature at the top during discharging and the maximum temperature at the bottom during charging can be beneficial to avoid thermocline destruction. The same basic control strategies shown in this section for traditional DH applications are also valid for DC systems. The latter usually operate with a smaller temperature difference between supply and return concerning DH, with consequent larger flow rates and pumping electricity consumption to supply the same thermal power. However, heat/cold loads forecasting solutions, presented in Section 2.3, are necessary to take advantage of the flexibility provided by the TES system and to optimize the operational management of the production units in both DH/DC systems.

2.3. Basic Control Strategies in 4GDH and 5GDHC Systems

The basic control strategies shown in the previous section are usually applied also in 4GDH systems. Generally, these have a supply temperature of 50–60 °C, while if it is lower such as in ultra-low temperature district heating (ULTDH), small booster heat pumps (HPs) are required at the customer-sited substation to satisfy minimum DHW comfort temperature and limiting legionella proliferation. The challenge that the current 3GDH systems are facing in the transition towards 4GDH is the supply temperature reduction. This is achievable mainly in those areas with new or refurbished buildings by zoning the DH network, maintaining a high-temperature backbone and low-temperature subnetworks. In this regard, a prototype of a mixing shunt system in a containerized solution has been installed in the DH network of Brescia (Italy) in the framework of the H2020 TEMPO project [26]. In contrast, a commercial solution has been recently installed in the DH network of Albertslund (Denmark) in the framework of the H2020 REWARDHeat project [27], as shown in Figure 2. These shunt systems usually include the minimum supply temperature and the minimum differential pressure controls for the subnetwork and unlock the possibility to absorb local peak loads by operating the subnetwork according to a dynamic heat storage strategy. Other benefits of the network supply temperature reduction are:

• The diminution of the distribution heat losses since also the return temperature decreases.
• The increment of the DH network capacity thanks to the larger temperature difference between supply and return at the backbone. This means that more customers could be connected to the existing infrastructure and that a lower flow rate is needed to supply the same thermal power.
• The increment of the life of some DH components thanks to the lower operating temperature.
• The increment of the efficiency in condensing boilers, CHP plants, HPs and solar thermal fields thanks to the lower return temperature.

Nevertheless, to maximize these benefits, it is essential to reduce the return temperature of the subnetwork by acting on the customer’s substation and heating system. Apart from the substitution of some critical radiators with high panels or the extension of the heat emission surface, it might be useful to mitigate the effect of high flow rate occurring for instance due to night set back, with control using thermostatic radiator valves (TRV) and return temperature sensor [28].

As far as 5GDHC networks are conceived, basic control strategies can differ from traditional DH/DC systems and 4GDH. 5GDHC systems, which usually operate with a network temperature in the range of 0–30 °C, include heat pumps (HPs) at the customer-sited energy transfer station that are in charge of meeting the required heating/cooling supply temperature. For this reason, supply temperature control is usually not implemented at centralized level, and several systems operate with free-floating temperature that is a result of the balance between the heating and cooling loads as well as thermal energy source temperature throughout the year [24]. Minimum supply temperature control is in some cases implemented to limit the risk of very low temperature and possible freezing in the pipes using back-up systems [29,30,31]. Differential pressure control is usually present only when centralized pumping stations are used and, in some cases, together with redundant heat exchangers that separate the HPs from the thermal grid. In other cases, such as in small loops where decentralized hydraulic pumps are installed at the substation level, differential pressure control is absent. Anyhow, the minimum pressure in the network is met by means of pressurization stations.

Remote smart metering is a mature technology, and it is an important aspect to improve the operation in both 4GDH and 5GDHC systems such as with the implementations of demand response (DR) strategies for peak shaving as presented in Section 2.5 as well as to detect faults in substations as described in Section 3.2. Nevertheless, there are still some barriers to overcome to achieve this as the lack of trust between the DHC operators and the users as well as additional services to provide by DHC companies such as substation maintenance [32].

2.4. Heat/Cold Load Forecasting in DHC Networks

To satisfy the users’ comfort, the heat load must be met almost instantaneously with the consequence that DHC systems are usually demand-driven. Since the heat load of the entire DHC system is the result of the aggregation of the heat demand of every single building plus the network distribution thermal losses, it is affected by less variability concerning the heating/cooling demand of every single building. Nevertheless, it is characterized by a seasonal variation, weekly variation between workdays and weekends and a daily one. For instance, the daily and weekly variations for the heat load of a DH system located in Northern Italy during two consecutive weeks in the winter period are shown in Figure 3. Both variations are a consequence of the collective social behavior, such as harmonized working hours, leading to the result that the heat load during weekend days is lower with respect to one of the weekdays. The daily load variation is characterized by two peaks: one in the morning and one in the afternoon. Nevertheless, the night setback control used to cover the space heating (SH) demand makes the morning peak higher than the afternoon one. It aims to shut down the heating system or adopt a lower room temperature setpoint during the night with a time-clock operation that can have the same settings in several buildings. When the DH substation starts its operation, the superposition of two contemporary side effects occurs: a very high flow rate on the primary side, since the valve is fully opened to meet the temperature set point at the secondary circuit, and a peak in the differential temperature on the primary side, because even if the supply temperature on the secondary side increases, the return temperature is still shallow until the building heating system reaches a more steady-state condition.

Recent advances in the DH load forecast have been investigated and tested in two recent closed EU H2020 projects: STORM [12] and OPTi [33]. Different techniques from simple linear regression to machine learning (ML) algorithms have been investigated and applied to the demo case of Rottne (peak capacity of about 2.5 MW) in the framework of the STORM project. The results show that monitored historical data are not always the best option to develop such models. For instance, the weather forecast could tend towards overestimating the outdoor temperature with respect to the measured one in a location [34]; therefore, since the weather forecast is then used as input for making predictions, it is important to use historical weather forecast data to build data-driven models to forecast the DH/DC loads. A feed-forward neural network with a single hidden layer slightly outperforms an extremely randomized trees model in Johansson et al. (2017) [34] with online periodic training and day ahead forecast once per day achieving the lowest mean absolute percentage error (MAPE) equal to 6.84% over one month. In [35], the same authors presented a different strategy by assessing on the same training and testing dataset of 27 months the performance of different eight data-driven models, based on linear regression (LR), extremely randomized trees (ERT) regression, feed-forward deep ANN (with two hidden layers, each with 12 neurons and the ReLu activation function) and support vector machine (SVM), and developing an expert advice prediction tool that can track the best of these different data-driven models. The results showed that ANNs and ETRs are slightly better than the SVMs, achieving a MAPE of about 11.6, 12.4 and 14.7%, respectively, while the one based on LR is the worst model with a MAPE of 17.3% over a test period of seven months. The expert prediction system tracks very closely the ANN model, but it results in being more robust and accurate in the load prediction during the mid-season period. To set the best hyper-parameter of these models in this work, the authors used an exhaustive search over specified values, whereas in the following paper [36], they used the tree-structured Parzen estimator, that is, a Bayesian optimization algorithm. Assessing almost the same models on the same dataset, it results that all the models improved their performance a lot, but the best one is still the deep ANN that achieved a MAPE of 8.08%. Considering the assessment of the same models on the DH network in Karlshamn (peak capacity of about 50–60 MW) that is much larger than the one in Rottne, the performance of all the models improves, and the MAPE is more or less reduced by half.

In the framework of the OPTi project [33], ML-based black-box models (random forest) to forecast the load of buildings connected to the DH system of Luleå (Sweden) have been studied in [37], whereas grey-box models have been developed in [38]. SVM, multiple linear regression (MLR), and feed-forward ANN performed similarly and better than regression trees in Idowu et al. (2016) [39] where on average worse results have been obtained in forecasting loads of commercial buildings with respect to residential ones using these black-box ML models. Dalipi et al. (2016) [40] achieved better results using SVM for hourly load forecasting concerning partial least square (PLS) and random forest (RF). The mean absolute percentage error (MAPE) over the testing dataset of one week achieved with SVM was 3.43%. In [41], an adaptive neuro-fuzzy inferences system (ANFIS) has been developed using only previous values of thermal load, whereas a different approach has been adopted in [42], where it has been demonstrated that a feature fusion long short-term memory (FFLSTM) deep neural network model outperforms seven different ML state-of-art models. Regarding the load forecast in district cooling (DC) systems, Kato et al. (2008) [43] demonstrated the improvement achieved using a recurrent neural network (RNN) with respect to a three-layered neural network (TLNN) at the expenses of a slightly higher processing time. The RNN architecture employs the sigmoid activation function in both inner and output layers and the 33 inputs consist of the previous 24h values of the cooling load, day of the week, and daily minimum and maximum forecasted ambient temperature.

In the past, statistical time series forecasting methods have been also successfully applied to DHC systems. For instance, statistical and physical models have been combined in Nielsen and Madsen (2006) [44] according to a grey-box approach. Auto-regressive models with exogenous inputs (ARX) has been used by Saarinen (2008) [45], whereas a seasonal autoregressive integrated moving average (SARIMA) model have been developed in Grosswindhager et al. (2011) [46] and incorporated into a state space framework where on-line load forecasting is obtained exploiting a Kalman filter. Another solution has been adopted by Chramcov (2010) [47] that applied the Box–Jenkins method to model the stochastic behavior of the social component of a DH load and a cubic function for the outdoor temperature-dependent component.

At the substation level, a linear regression algorithm is proposed in [48] to disaggregate the SH and DHW parts from the total load to be used for forecasting purposes in the framework of the H2020 RELaTED project [49]. However, this model performed worse than an ensemble of decision trees (DT) model. Similarly, a compact black-box model of the building based on a linear system has been developed in Guelpa et al. (2019) [50]. The latter uses the predicted average temperature of the day, the average and the minimum and maximum temperature of the previous day as inputs, and it is able to provide the peak and steady-state load and the peak duration as outputs. The building model has been integrated with the physical models of the distribution and transmission DH networks in a multi-level approach to assess the effect of demand response application on the aggregated load with a high time resolution. For an exhaustive review of models and approaches for district load forecasting, the reader is referred to [51].

2.5. Advanced Control Strategies in DHC Systems

Given the fact that smart DHC grids are becoming more complicated than before due to the exploitation of non-programmable renewable thermal energy sources and sector-coupling with the power grid, traditional strategies, even if they are robust, are not sufficient to cope with these challenges. Smart control strategies at centralized and decentralized levels are of paramount importance and need to be applied on top of the basic control strategies presented in Section 2.1 and Section 2.2. These strategies against challenges such as weather uncertainty, user behavior and customer-sited substation operation are trying to achieve good performance of the system satisfying capacity constraints at some points of the DH network. In recent years, model predictive control (MPC) has become a high-tech solution to several complicated control problems. It attracted research communities’ attention in different fields and in particular in the DHC sector. MPC is based on a predictive process that exploits a dynamic model of the system and allows finding the vector of the manipulate inputs over a time horizon by solving an on-line constrained optimization problem that takes into account the system details [52]. For instance, an MPC controller based on a dynamic programming (DP) optimization algorithm has been presented in Saletti et al. (2020) [53] to supply the thermal energy to a school complex located in Northern Italy. Results show a decrease in natural gas consumption of the boiler up to more than 7% in comparison with a standard PID controller. In De Lorenzi et al. (2020) [54], the same tool and approach are applied in different case studies. The comprehensive smart control strategy consists of pledges efficient energy distribution, flexibility and low-carbon energy integration. Achievements are a 6% reduction in operating cost and up to 34% in energy consumption, whereas meeting consumers’ needs. Moreover, MPC has been applied in Verrilli et al. (2017) [55] by scheduling boilers, TES units and flexible loads. Results show that the MPC succeeded in reducing operating and maintenance costs in a DH power plant. This methodology integrated projection on fluctuating demand within the optimization problem and answered it in a receding horizon fashion. Labid et al. (2014) [56] developed an MPC to improve the thermal storage tank management in a multi-energy district boiler. The results revealed that the controller reduced fossil energy consumption and CO₂ emissions, while the economic profit increased. A model predictive control (MPC) introduced in Lennermo et al. (2019) [57] has been applied to decentralized solar thermal collectors (STC) fields connected to a DH network in the return-supply configuration. The authors showed that the MPC implementation in the real SDH system in Lerum (Sweden) operated in a robust way solving the problems of the previous PID controller that was not capable of limiting severe supply temperature and flow rate cycling variations that can be a cause of fatigue problems in buried steel pipes.

Sometimes for covering the disadvantages of MPC and obtaining the best performance and result, MPC has been merged with other methods. For example, to control the supply temperature in the DH network, authors in Grosswindhager et al. (2013) [58] used the combination of the MPC and fuzzy direct matrix control (FDMC). Findings confirm that FDMC can command the inherent nonlinearity in the acknowledgment characteristics of DH systems by considering the volume flow rate at the plant as a fuzzy variable. Additionally, the trade-off between pumping and heat loss costs can significantly impact minimizing operational costs. A tool named XEMS13 using MPC strategy has been introduced in Lazzeroni et al. (2019) [59] in order to optimize operational costs by considering technical and operational constraints. This tool is developed by the Energy Department of the Politecnico di Torino and LINKS. Different configurations with increasing cost-saving, installation costs, renewable energy source generation and primary energy saving are presented, and results demonstrated the success of the method.

Linear programming (LP) and mixed-integer linear programming (MILP) are commonly used to individuate complex energy systems’ best size or scheduling. A recent application of LP related to this scope for a 5GDHC system supplying several buildings of a university campus can be found in Wirtz et al. (2020) [60], where the problem has been set up in Python and solved by means of the solver Gurobi. A MILP model for the design and operation of an urban energy system in the center of England has been presented by Samsatli and Samsatli (2018) [61]. This model is based on a flexible value web framework for representing integrated networks of resources and technologies. It can be used for different temporal and spatial scales. Researchers in [62] have applied MILP to minimize the total annual cost for an industrial area located in the northeast of Italy with the integration of solar thermal production. Their results showed that the yearly total cost and primary energy consumption reduce 5 and 15%, respectively. In the framework of the H2020 FLEXYNETS project [63], to reduce the overall heat generation cost of a DH supplied by a large share of low-grade excess heat, Vivian et al. (2017) [64] indicate that the suggested MILP-based controller managed to reduce the system operational costs by 11% with respect to a standard rule-based control. Similarly, in Schütz et al. (2015) [65], the authors applied a MILP method to minimize the total costs. To this end, four different strategies for calculating temperature stratification inside thermal storage are performed. Procedures are tested with an operation scheduling use case that aims at minimizing operational costs by achieving a reduction of 6–7%. MILP has been applied in the work of Giraud et al. (2017) [66] using the commercial solver CPLEX in a receding horizon fashion with a timestep of 15 min and a prediction horizon of 24 h. Differently from other works, here, the optimal scheduling of the generation units assessed takes into consideration the effects on the DH network with the integration of a dynamic model in Modelica. The simultaneous optimization of DH production and distribution resulted in the use of the DH network as storage by increasing the supply temperature prior to a peak of the demand, reducing the differential pressure and avoiding the start-up of expensive peak units. In [67], a more complete overview of the results is provided by the authors showing that with respect to an empirical piling method, the production optimization only was capable of achieving 6.4% of savings on the production costs and no energy savings, whereas with both production and distribution optimization, it was possible to obtain 8.3% of savings in production costs and 1.7% reduction in the energy consumption.

Adequate optimization algorithm depends on the problem formulation; likewise, how to formulate the problem depends on the optimization algorithm to use. In some cases, the execution time increases exponentially according to the problem dimensions and deterministic optimization algorithms cannot solve challenging and complicated issues. Meta-heuristic algorithms are a kind of random algorithm that are used to find an optimal response to this problem. Genetic algorithm (GA), particle swarm optimization (PSO) and ants colony optimization (ACO) are some of the most used population-based meta-heuristic algorithms. Although these methods’ performance and ability have been proved in many references, they have rarely been used directly in DH application. For instance, a two-level optimization algorithm is proposed in Urbanucci et al. (2019) [68]. MILP formulation has plenty of pros for optimal operation problems while overcoming its disadvantages, GA is applied. In the case study, the cogeneration unit directly reaches 70% of both the thermal and electric yearly demand, while 16% of the subsequent is met by storage. Likewise, a hierarchical optimization strategy is used in Casisi et al. (2018) [69]. The higher layer for binary decision is dedicated to the genetic algorithm (GA), while the MILP algorithm is used in the lower to choose the system’s optimal operation. Results show a remarkable reduction in computing time.

The multi-agent system (MAS) approach benefits several critical computer applications such as communication network configuration, process control, planning or concurrent systems. MASs are formed by several agents that have a considerable number of communications with each other. In general, agents act to support the users and have different goals and motivations. An agent is defined as an autonomous entity that can be viewed as perceiving and acting upon its environment. An agent must be able to communicate with other agents [70]. To interact successfully, agents must have the ability to cooperate, coordinate and negotiate with each other, almost as people do. To improve the monitoring and control of a DH system through the use of agent technology, Wernstedt and Davidsson (2002) [71] developed a MAS approach. In order to shave the peaks and prevent entering the boilers in the circuit, the authors of [72] have applied the MAS approach. Yielded results show that MAS was capable of decreasing the peak loads by up to 20%. In the same way, Lacroix et al. (2012) [73] used the MAS approach to control a compressed unit providing heating, ventilation and domestic hot water production in a low-energy building. However, findings show that the suggested methodology managed to increase costs by only 2.5%, but they improve thermal comfort by 35%. MAS and PSO’s combination is the main idea in Wang et al. (2012) [74]. The aim is to maximize the comfort index using minimum power consumption, and the findings indicate the effectiveness of the proposed method.

The MAS approach has been recently applied to 5GDHC systems that enable the concept of decentralized prosumers for all the connected substations. In particular, in Bünning et al. (2018) [75] the performance of two 5GDHC systems located in San Francisco and Cologne exploiting an agent-based control have been assessed in comparison with a traditional gas-fired DH and individual chillers solution for space cooling. By exploiting the coordinated balancing effects of several decentralized prosumers with heat pumps and chillers, this control strategy was able to maintain the average temperature of the system within 2 K around the setpoint all year around showing that this approach is beneficial with respect to free-floating temperature control. Future technology could match blockchain with the MAS approach. Blockchain application in 5GDHC have been investigated in the framework of the D2Grid project [76] and is going to be demonstrated at the demo case of Paris-Saclay (France) [77]. The goal is to apply a new business model where smart contracts can be signed automatically and with the exploitation of dynamic energy tariffs to flatten the peaks in the heating and cooling demands by promoting flexibility and demand response (DR) strategies at the customer level. In this regard, an overview of DR applications in the DHC sector is presented in the next section.

2.6. Demand-Side Management (DSM) in DHC System

Electrical/thermal demand is mainly uncontrollable, and its quantity varies during the day and along the season. Accordingly, to flatten the load curve, system operators usually implement demand-side management (DSM) strategies that consist of a portfolio of practices aiming at modifying the demand side of an energy system by promoting information programs, energy efficiency, energy saving but also with the implementation of demand response (DR) programs. Peak-shaving and load management are two main applications of DR that add flexibility to an energy system by making the demand more elastic. In general, DR strategies are classified into price-based and incentive-based programs [78]. To the first group belongs all the solution that aims at influencing end-user choices with time-dependent tariffs such as real-time pricing (RTP) or common time-of-use (TOU) tariffs. Differently, incentive-based programs are based on contractual agreements with grid operators or utility companies and foresee a reward to the customer for the reduction in their load upon request or with the direct remote control of some equipment. So far, DR has widely studied to shave the peaks in power grids and on pooling of individual air-source heat-pump-based heating ventilation and air conditioning (HVAC) systems as in [79], [80] and [81]. Recently, aggregation of the HP-based substations in the 5GDHC network of Wüstenrot has been analyzed for the implementation of power-to-heat strategies in the context of the Sim4Blocks H2020 project [82]. In particular, heuristic optimization of HP operation for the maximization of PV electricity self-consumption and exploitation of flexible electricity tariffs have been mainly investigated in Brennenstuhl et al. (2019) [83], whereas cluster aggregation for power market participation such as frequency restoration reserve (FRR) has been analyzed in Romero Rodríguez et al. (2019) [84]. The latter showed how the requirement of a short activation period in the FRR market can be in contradiction with the minimum running time of the HPs for both technical and economic aspects.

As explained in Section 2.3, the critical issue in DHC networks is the presence of peak requests, particularly in the morning. Here, DR can play a vital role on the demand side to shave these peaks and avoid large investments in centralized TES systems. Thermal peak shaving using DR in a real test case of a distribution network of the Turin DH system is investigated in Guelpa et al. (2019) [85]. A genetic optimization algorithm has been adopted to select the binary values of on/off switching every 10 min with strong restrictions such as maximum anticipation of 20 min and acting only on 32 out of the 104 buildings connected to the distribution network. The field results revealed a reduction of 5% of the global load. However, the authors demonstrated with a simulation activity that if the controller acts on all connected buildings and with maximum anticipation of 90 min, a peak reduction up to 37% can be obtained. In a previous work [86], the same authors assessed the fact that modeling both supply and return pipelines provide a more precise assessment taking into consideration how the on/off switching of some substations could affect the delay in the temperature distribution to the farthest consumers. Moreover, Guelpa and Marincioni (2019) [87] proposed a simplified approach of DSM to shave the peak of DH substations by means of a modification in the proportional–integral control logic that regulate the flow rate on the primary side. A differential of return temperatures (DRT) control strategy is introduced, and it aims at maintaining the temperature difference between the inlet of the secondary side and the outlet of the primary side below a threshold so that during the start-up the valve opens slower, the flow rate on the primary side is reduced and the efficiency of the heat exchanger is increased. Findings indicate that the achieved decline in the peak load of a distribution network supplying 62 buildings is equal to about 24% during a typical winter day.

An integrated demand–supply co-optimization methodology is presented in Romanchenko et al. (2019) [88] and applied to the DH system of Gothenburg (Sweden). The space heating demand in buildings is modeled with one thermal zone and the dispatch of heat production units and internal temperature set point variation in the buildings (only upwards) is obtained by mixed-integer linear programming in GAMS with hourly time resolution. The aim is to investigate the flexibility potential of DR exploiting the thermal mass of buildings for a DH network whose characteristics and limitations have been here neglected. All suggested scenarios manage to shave the peaks achieving a reduction in on–off switching of the peak units up to 80%.

In the context of the H2020 project OPTi [33], some specifications for the implementation of automated DR in DHC system are provided in [89], whereas in the H2020 project E2District [90], the authors in Beder et al. (2019) [91] applied behavior DR to the demo case of the CIT Bishopstown campus (Ireland). A couple of behavioral parameters such as action–regulation theory, high-performance cycle and planned behavior theory have been embedded in their methodology, and yielded results show that the proposed model can increase the energy-saving up to 4.5%. Cost-saving and emission reduction considering thermal comfort during DR programs have been investigated in Wu et al. (2020) [92] where also dedicated experimental tests using a thermal manikin to assess the effect of draught from cold windows have been performed. The MPC algorithm exploits a dynamic DH price model and included constraints to restrict the temperature of cold windows.

Sector-coupling between electrical and thermal grids is also possible by exploiting the presence of HPs and TES systems at the customer sited-station level and allowing their participation in more common electrical DR programs. In Knudsen and Petersen (2017) [93], a 4GDH application with ULTDH and a booster HP for DHW production is analyzed. Peak shaving and energy-cost saving are the main goals of the suggested MPC strategy that has been implemented in the form of non-convex quadratic programming (QP) and solved with the optimization engine CPLEX. Similarly, in a 5GDHC application, an ANN-MPC algorithm has been developed in [94]. The binary version of PSO has been applied to minimize the user energy bill and the state change with different time-of-use (TOU) DR scenarios exploiting a composite ANN model of the integrated HP and TES systems. The results show that up to 14.2% of electricity consumption has been shifted from peak to off-peak hours.

3. Overview of Fault Detection and Diagnosis (FDD) in DHC Systems

It is evident that the exploitation of innovative DHC solutions requires the whole system to operate as efficiently as possible [95]. However, several authors [96,97,98,99] have proved that faults taking place during the operation of energy systems can normally be responsible for up to 40% of their total energy use. Several factors, such as compensation actions triggered by control algorithms or lack of proper maintenance practice, can make these faults remain undetected. Indeed, the manual identification of these faults gets very complicated even if the suboptimal operation of the system is known. This makes the tasks of human maintenance operators really costly, since they only take actions on the system when the indoor environmental thresholds are not met. In this context, automated fault detection and diagnosis (FDD) methods and tools play a key role to assist building and DHC system operators [100].

One typical operating problem in district heating systems indicated by Frederiksen and Werner (2014) [25] is the heat carrier loss through water leakage. Water losses occur for several reasons and the corresponding magnitudes of losses also vary. A review of FDD methods to address this problem is presented in Section 3.1. Other operating problems are higher temperature levels due to high return temperatures caused by typical malfunctions such as set-point errors in substations and customer heating systems or short-circuit flows in the thermal grid [25]. Since the reduction in the supply temperature is the main objective of 4GDH and 5GDHC in order to exploit renewable and low-grade excess heat sources, assuring a low return temperature is a key aspect in these innovative DHC solutions [24].

In order to avoid malfunctions and faults, maintenance planning becomes crucial to achieve a good service for customers and maintain an economical retrofit for owners. The concept of maintenance includes the administration, control, implementation and quality of those activities, which will ensure that design availability levels and asset performance are achieved in a reasonable way to meet economical and functional objectives. Some definitions of classic maintenance strategies that can be applied are:

• Corrective maintenance is performed to determine, separate and fix a fault so that the failed equipment or facility can be brought back to an operational condition, which lies within in-service operations tolerances.
• Preventive maintenance is performed on a regular basis on a piece of equipment in order to reduce the probability of failure, and it involves a systematic check-up of equipment, thus enabling to detect and correct potential problems.
• Condition-based maintenance consists of a strategy different from preventive maintenance because the maintenance action relies on the actual condition of an asset, rather than average or expected life statistics, to decide what maintenance needs to be done. It imposes that maintenance should only be performed when some indicators show marks of decreasing performance or imminent failure.
• Predictive maintenance is an extension of condition-based maintenance where precise techniques and formulas are used to detect incipient faults and predict their evolution, so the maintenance action can be scheduled before the critical failure in the equipment occurs. Predictive maintenance generally applies non-destructive testing technologies and other specific online methods depending on the type of equipment or process being monitored.
• Proactive maintenance sets corrective actions focused on failure root causes, not on failure symptoms, unlike predictive or preventive maintenance.

There is a huge range of different fault detection (FD) techniques as classified by Granderson et al. (2017) [101]. FD methods may be model-based or based purely on process history data, both of them are also called “internally based” methods. The model-based methods depend on knowledge of the basic physical processes and principles governing those system(s) being the target of the analysis. Quantitative model-based approaches are, currently, not frequently used in commercial tool offerings, however qualitative model-based approaches including rule-based fault detection, have been largely applied to industrial environments and provide intuitive representations of engineering principles. The process history-based, also referred as “data-driven FD” is an innovative approach that does not rely upon knowledge of first principles, but on the data from the system in operation from which they may leverage some degree of engineering knowledge. These include statistical regression models, artificial neural networks (ANN), etc. Anyway, a combination of both approaches can be also found in several FDD applications. Moreover, there also exist “externally based” or hardware methods [102], such as visual inspection, infrared image processing or cable methods that usually are not suitable to be included in an automatic fault detection system. The following sections present a review of the state-of-the-art fault detection approaches and algorithms that are mainly applied in DHC systems.

3.1. Leakage Detection in DHC Networks

Failures on district heating pipes are often caused by water leaks due to corrosion, mechanical impacts and insufficient or deteriorated performance of the thermal insulation solutions, as indicated by Hallberg et al. (2012) [103]. However, some degree of leakage is impossible to be avoided during extended operation, since pipeline performance degrades over time. Therefore, an anticipated diagnosis of leakage occurrence is highly necessary to improve efficiency, reduce operating costs and protect the environment. In comparison with DHC networks, both oil/gas and water distribution networks have a longer industrial history. Thus, many established leak detection research results and applications were first applied to such applications. Some representative techniques for pipelines developed for specific fluids (oil, gas or water), different layout patterns, several lengths of pipelines as well as for a certain range of different operating conditions, can be found in [104].

As mentioned above, those methods found in literature are usually divided as “internally-based” (or “software-based”) and “externally-based” (or “hardware-based”), as presented in the review of Zhou et al. (2018) [105] for leakage detection in DH networks. Zaman et al. (2020) [102] developed and compared “software-based” solutions, both model-based and data-driven, applied through a leakage detection algorithm, whereas a good example of a physical model-based algorithm can be found in Liu et al. (2019) [106]. The latter includes a dynamic monitoring module (DMM) and a static testing module (STM): the DMM can detect larger leakages analyzing pressure waves through amplitude propagation and attenuation models; the STM, based on the pressure loss model, can detect micro-leakages, thus being able to act as an effective compensation for the DMM.

As far as data-driven methods are concerned, an application can be found in Xue et al. (2020) [107]. It consists of training a decision-tree-based ensemble ML algorithm called XGBoost, using data generated by a simplified physical model and using it to detect leakage in pipes through the collected data from pressure and flow sensors present in the DHC network and substations. A potential obstacle to replicate this approach is the fact that pressure sensors are not always available in the system.

Two interesting proprietary “hardware-based” solutions deserve mentioning. One of them has been developed by the smart meter brand Kamstrup and consists of a leakage detection system based on the analysis of the signal coming from ultrasonic flowmeter installed in substations [108]. The other one is based on the well-known impedance method using sensing cables. When a leak takes place, the cable gets saturated with fluid, thus altering its impedance [109]. The advantages include high accuracy in determining leak location and easy configuration and maintenance. In contrast, the installation has very high costs and wiring requirements.

In the field of image processing, infrared (IR) sensors are able to capture variations in the heat flow caused by underground fluid leaks, and then show them as hot spots in the DHC system route. This process can be accomplished on the ground, but the availability of high thermal sensitivity and spatial resolution thermal imaging systems mounted on an aerial platform has become the most effective procedure. For instance, data collection can be conducted by an aircraft or drone, which flies over the target area with a camera mounted to the airframe and looking straight down to the ground. This way, thermography reveals sources of heat and the relative differences in temperature from one object to another, as presented in [110]. In contrast, postprocessing of IR images may be computationally expensive, and their analysis could lead to false negatives because some color differences caused by a leakage could be almost inappreciable. Some authors such as Zhong et al. (2019) [111] and Hossain et al. (2020) [112] have developed ML algorithms to improve postprocessing and satisfactorily make the difference between true leakages and other potential causes.

3.2. Fault Detection in Substations and Customer Facilities

Nowadays, a current preconceived idea considers that most of the end-use substations in district heating systems work well. This means that it is taken for granted that the facilities deliver or use exactly the right amount of energy to cover customer’s needs. Gadd and Werner (2015) [113] showed that this is not the case and almost three-quarters of the substations analyzed present faults or symptoms of faults, which could lead to higher return temperatures. This fact is unacceptable especially in 4GDH and 5GDHC, where it is important to achieve a very low return temperature.

In an analysis of the most common faults in DH substations performed as part of the H2020 project TEMPO [26], a survey-based study by Månsson et al. (2019) [114] found that the largest fault category was leakages (33%), closely followed by faults in the customers’ internal heating systems (31%). This fact reveals that it might be difficult for the energy utilities to get access to all faults present in the customers’ facilities because the DH operators are only usually allowed to access to the substation and not to the internal heating system. Therefore, utilities must make important efforts to establish a good relationship with customers. Common practice to achieve this is to have maintenance contracts with the customers or to include free of charge inspections in the DH agreement. In addition, a proper fault detection system must include customer-sited substations as the main element to inspect. This can be done by analyzing the components of the substation individually, or/and looking at the customer heat load patterns. In the following sections, a literature review of FDD methods for different parts of customer-sited DHC stations is presented.

3.2.1. Heat Load Patterns-Based Methods

The heat load in a DH/DC system is the sum of individual heat loads from all customer-sited substations connected to the network and the distribution heat losses. DH/DC heat generation plants might be affected by malfunctions in customer substations and building HVAC systems, which are propagated through the network. The operation of the HVAC system varies depending on the building’s end use, so the resulting profile of the heating/cooling load will vary among the different types of buildings. Gadd and Werner [115] made a study of 141 different buildings where they first defined two indicators (annual relative daily variation and annual relative seasonal variation) and then used them to detect failures in the DH substations based on high or low variations of these parameters depending on the type of building. Irregularities of the heat load pattern or no correlation between outdoor temperature and heat demand can also be used to detect that the substation is not working properly.

The main challenge related to heat load patterns-based methods is how to deal with such different profiles for various types of buildings or how to create reliable predictions of them. Concerning the first point, a data-driven algorithm enabling large-scale automatic analysis of district heating load patterns was developed by Calikus et al. (2019) [116] using an initial dataset of 19.6 million hourly measurements. The algorithm applies clustering techniques to aggregate profiles of customers into different groups and extracts their representative behavioral patterns in terms of heat load. In this way, it is capable of detecting unusual customers whose profiles deviate significantly from the rest of their group. These outliers can be analyzed in depth in order to find problems in the corresponding substations or customer facilities. The application of the algorithm allowed to detect abnormal heat load profiles due to, for instance, a mismatch between the real and designed use of the buildings, problems in the HVAC system that resulted in sharp and irregular afternoon peaks, and summer loads higher than mid-seasons’ ones due to substation faults.

In the framework of the H2020 project RELaTED [49], two tools have been developed for automatic fault detection in DH substations based on ML algorithms: DH doctor and DH Autotune [48]. The first one exploits clustering, and it is based on daily averaged readings. Anomalies can be detected by measuring the distance among the clusters and following the evolution of the centroids related to a particular variable over time. Moreover, it exploits an ensemble of decision tree (DT) algorithms to make predictions that allow assessing deviations of a monitored variable. The second tool is based on hourly averaged readings and allows the prediction of the load as mentioned in Section 2.3, but also a fast reaction is triggered if abnormal behavior occurs. Alarms are activated if some KPIs, such as MAPE, exceed a threshold. Further applications concerning the prediction of substation heat demand patterns through ML algorithms using the usual metering variables such as flows and temperatures can be found in [117,118,119].

3.2.2. Fouling Detection in Heat Exchangers

When focusing on operational faults specifically related to heat exchangers, the literature shows that most of them commonly involve fouling formation [118]. This can be described as the accumulation of deposits on heat transferring surfaces, which cause a higher thermal and hydraulic resistance in the heat exchanger. An automatic method using the usual metering variables such as volumetric flow and temperatures in the substation primary and secondary circuits has been developed by Guelpa and Verda (2020) [120]. It consists of the indirect calculation of the global heat transfer coefficient and monitoring its change during the fouling process. This method is easy to implement but involves important calibration challenges when it comes to different kinds of heat exchangers.

3.2.3. Detection of Regulation Valves Malfunctioning

The actuator of the control valves or the valves themselves may wear and tear during operation or after large periods without use. This causes uncontrolled flows in the installation and instability of the flow rate. An analysis of frequency variation and stability of the flow in the primary circuit of the substation was performed by Fabre et al. (2020) [121] leading to a simple and easy method to detect this kind of fault.

3.2.4. Malfunction in Heat Pump Components

Heat pumps (HPs) at the customer-sited substations are used in 4GDH systems as booster stations for DHW production, whereas in 5GDHC systems, they are needed to supply both space heating/cooling and DHW loads at the right temperature for the distribution and emission system. Due to that, it is really important to assure an efficient operation detecting possible faults and to prevent them. A comprehensive study that analyzes the most important faults reported by both original equipment manufacturers (OEMs) and insurance companies in Sweden is presented in [122]. The results state that the issues in control and electronics are one of the most common and costliest faults in all types of HPs. According to OEMs, the shuttle and shunt valves are the second most common faults that occurred in ground source and exhaust air HP systems, respectively. Unfortunately, there are not any investigations in the current literature about fault detection applied to these specific parts. Moreover, an additional impediment for the development and implementation of the HPs fault detection algorithm is the fact that the equipped software is often a closed system that can only be accessed by the manufacturer. However, there exist some studies leading to detect leaky check and reversing valves using their own test benches or modified commercial heating pumps to be able to measure internal temperatures and mass flow of the refrigerant [123].

It is proved that for HP systems, there is a maximum coefficient of performance (COP) at the optimal charge amount and refrigerant leakages cause performance degradation and a decrease in thermal comfort [124]. However, several heat-pump-based units usually do not have an optimal amount of refrigerant. For this reason, it is important to detect these leakages in an early phase to be able to fix them without energy losses and to limit their contribution to global warming. Several studies have been developed in this direction. For instance, Eom et al. (2019) [125] proposed a novel refrigerant charge fault detection strategy for HPs using convolutional neural networks trained using a real commercial HP system and the variables used for internal control provided by the manufacturer. Sun et al. (2020) [126] defined the sub-health operation concept of HP systems, which is used to define the intermediate state between normal and fault. Moreover, an online undercharge sub-health diagnosis method was proposed that analyses the theoretical behavior of the system facing a refrigerant leakage.

Even though it seems that every time more algorithms used for fault detection are based on neural networks and machine learning techniques, a recent study concludes that some of these approaches are not useful when working with real data series from heat pumps [95].

3.3. Diagnostics of Sensors and Actuators

The digitalization of the DHC sector is becoming crucial. Moreover, in order to control efficiently the systems and to be able to detect malfunctions, the introduction of more and new sensors and actuators is necessary. However, these components may also fail, and it is important to be able to detect it. There are several simple complementary methods, which should be implemented in all sensors and actuators of the whole facility:

• Monitoring of raw voltage/current sensor signals to detect short circuits to detect out-of-range values.
• Monitoring of incoherent values of the measures such as instabilities or impossible values to reach (e.g., ambient temperature above 70 °C).
• Considering the size of deviation, duration of the fault and average frequency of appearance.
• Creation of strategies identifying when the actuators and sensor will be tested, taking advantage of specific operation points such as stationary behavior, opening/closing of valves, etc. The continuous diagnosis of some variables may lead to false fault detection, which may suppose an extra cost for maintenance companies.

In the literature, some specific methods applied to heat pump (HP) sensors are investigated. Zhang et al. (2019) [127] propose a data-driven statistical model optimized and applied for sensor fault detection and diagnosis (FDD) using subtraction clustering and k-means clustering combined to identify and classify modelling measurements of unsteady operating conditions. Moreover, in order to calibrate the HP sensors, a method called virtual in situ calibration (VIC), based on the Bayesian inference and Markov Chain Monte Carlo (MCMC), was very effective in detecting and correcting the systematic and random errors of various sensors installed in a PVT/heat pump system [119]. As the VIC method can effectively improve the measurement certainty, the sensors with relatively low accuracy can be used to achieve a higher precision, which is able to significantly reduce the cost of equipment. VIC uses a certain grade of modelling, and it seems not so hard to implement.

4. Discussion

4.1. Advanced Control Strategies

The implementation of advanced control strategies in DHC systems is challenging, since they are affected by time-varying nonlinear dynamics, time-varying set-points and disturbances. Likewise, the implementation in HVAC systems, advanced control strategies applied in DHC can be classified as hard, soft and hybrid control [128,129,130,131]. In the following, some more details are provided about hard and soft implementation of model predictive and agent-based control strategies highlighting their strengths and weaknesses. The focus by the authors on these two main approaches is motivated by a bibliographic study that show the growing number of publications related to the application of these control approaches in DHC systems between 2010 and 2020. The bibliographic analysis, whose results are plotted in Figure 4, has been performed through the database Scopus considering both peer-reviewed and conference papers. The results are also listed in Appendix A with the string used to query the database. It is also evident the growing number of works related to applications of neural networks, whereas in very few cases, robust control and reinforcement learning have been studied.

MPC can be considered a cutting-edge technology for improving the operation of DHC systems. It requires a surrogate model of the plant that it is used to simulate the behavior of the system in a dynamic optimization problem. The latter is implemented to minimize a cost function in a receding horizon fashion. This methodology consists of a time window that moves forward by one time-step at each control timestep t according to the schematic shown in Figure 5. The length N of this time window is constant and called prediction horizon. It represents the future time interval in which the outputs of the model (y(t)) are forecasted according to the trajectory of inputs received (u(t)). In order to speed up the optimization algorithm, some “blocking” methodologies can be applied. For instance, according to Figure 5, one can adopt a control horizon (M) that is shorter than the prediction horizon (N). In this way, the last value of the control variable in the control horizon is kept constant and used for the rest of the simulation. Only the first element (u₀^*) of the optimal input trajectory provided by the optimization algorithm is sent to the plant, whereas the rest is discarded. The optimization process restarts again at the next control timestep after updating the state of the surrogate model with monitored data.

According to the scientific literature reviewed, in several cases, the control problem is implemented in the form of economic MPC with the objective of minimization an operating cost function that can be formulated according to the following general expression for a generation system that supplies a DH network with CHP and boiler units:

$$\min \left\{\begin{array}{c}\mathrm{cost} \mathrm{of} \mathrm{heat} \mathrm{production} \mathrm{by} \mathrm{fossil} \mathrm{fired} \mathrm{CHPs}\\ -\mathrm{revenues} \mathrm{from} \mathrm{electricity} \mathrm{production} \mathrm{by} \mathrm{CHPs}\\ +\mathrm{cost} \mathrm{of} \mathrm{heat} \mathrm{production} \mathrm{by} \mathrm{fossil} \mathrm{fired} \mathrm{boiler} \mathrm{units}\\ +\mathrm{cost} \mathrm{for} \mathrm{buying} \mathrm{excess} \mathrm{heat}\\ +\mathrm{cost} \mathrm{for} \mathrm{electricity} \mathrm{consumtion} \mathrm{by} \mathrm{auxiliaries}\end{array}\right\}$$

(1)

where constraints are foreseen for the minimum and maximum capacity of the units as well as their ramp-up rates. Auxiliaries can be, for instance, hydraulic pumps, and the revenues from the electricity production by CHPs can include subsidies such as in [65]. However, some variants of the above general formulation are possible according to the problem and the system analyzed. For example, when low-temperature networks with decentralized heat pumps are present at the user substations such as in [64], in the problem formulation, there is also included the electricity costs for their consumption. In [55], since controllable loads are present, an additional cost is considered in the optimization problem for the curtailment of the flexible loads together with start-up and shut down costs of the boilers and TES charging. In a few cases, the effect of the DH supply temperature on the thermal energy loss through the pipes is taken into account such as in [55,132]. In some cases, energy consumption minimization is the goal of the problem formulation such as in [53,132], while application of minimization of CO₂ emission or non-renewable primary energy consumption has not been encountered. Moreover, formulations that may be relevant in real applications to also take into account the constraints on the electricity production allocated in the day-ahead electricity market the day before for the following one have not been encountered.

As a synthesis of the material examined in this study, the following strengths (+) and weaknesses have been identified for the application of MPC:

(+) Instead of corrective operations, it employs a proactive approach with anticipatory control actions.

(+) Load forecast and stochastic disturbances can be handled.

(+) Systems with delays and operational constraints are taken into account.

(+) Multiple objectives can be formulated in a cost function and achieved by exploiting advanced optimization algorithms.

(+) It can be formulated in both centralized and distributed fashion but also at the master or slave level.

(−) Time-consuming for the implementation and model identification phase.

(−) Non-technical users require specific background knowledge of the method.

(−) Require significant higher expenditures, which may not be repaid by additional savings in a short period.

Among the different MPC formulation in DHC application, mixed-integer linear programming (MILP) resulted in one of the most used, since it is able to handle both continuous and binary variables that are needed for set-points definition and on–off planning [60,61,62]. This hard-implementation sometimes called also hybrid MPC, which is characterized by [133,134,135]:

(+) Good and exact optimal solution.

(+) Rigorousness, flexibility and extensive modelling capability.

(−) Time-consuming particularly for large problems.

(−) It is known to often have weak linear programming relaxations.

(−) Loss of original discrete structure.

(−) Introduction of auxiliary binary variables.

With the growth of artificial intelligence (AI) research field, the development of MPC with soft and hybrid control technique are also appearing in DHC applications. In particular, black-box machine learning (ML) models are mainly used to cope with system non-linearities, and meta-heuristic optimization algorithms are used to solve the constrained optimization problem. The latter are usually population-based biological-inspired algorithms and can be mainly distinguished by [136]:

(+) Straightforward coding in any programming languages.

(+) Very quick convergence.

(+) No need to do complicated mathematical operations, since they are gradient-free.

(−) Easy to drop into local optimum.

(−) Possible stagnation after some initial stages.

(−) Efficiency reduction with an increase in problem dimension.

Adaptive control is a technique that is also applied in some cases in DH. In particular, their application in controlling the supply [137] or the return [138] temperature from substation heat exchangers emerges. It is a methodology that tries to overcome the problem of low performance of conventional feedback controllers during changes in the process dynamic. In fact, they foresee the self-adaptation of the control law to the changing conditions by means of the correction in the time-variant parameters. In [139], in the framework of the project OPTi, it has been demonstrated for the Luleå district heating system how the multi-input multioutput (MIMO) adaptive controller outperforms a standard multi-loop PID implementation, reducing the oscillations in the controlled variables that is a consequence of the different interaction of components such as generation units, pumps and valves in substations that are present in DH systems.

Finally, conversely from traditional MPC, multi-agent systems (MAS) control can be considered a hierarchy-free solution where different entities interact, e.g., as in a peer-to-peer market and operate without mandatory signals from a centralized higher-level controller. Even if they are not so widely applied in the DHC sector, some interest is appearing for the operation of decentralized active substations in 5GDHC with good results [75]. Nevertheless, for the MAS control approach, the following positive (+) and negative (−) aspects can be derived [140,141]:

(+) Based on the goals, agents can act autonomously in their environment, which could be cooperative or competitive.

(+) Decreased need for massive data manipulation.

(+) The other agents modify and continue the system functions if one of any controller loses.

(+) According to some rules, allow the manufacturers or loads to embed programmable agents in the equipment controllers.

(+) Agents are able to learn from their behaviors and past activities.

(−) Communication languages, protocols and the design of agents’ ontologies should be based on common standards.

(−) Since plenty of multi-agent platforms have been developed, selecting the most appropriate is a tricky task.

(−) The design of an intelligent agent is challenging.

4.2. Fault Detection and Diagnosis (FDD) Approaches

In general terms, algorithms used in FDD are divided into physical model-based and data-driven methods. Both can be used to compare the real and the predicted value of one or several variables and deploy an action when this difference reaches a threshold that has to be calibrated for each application. The action to perform can vary from a simple alarm to a complex control action to minimize safety and performance problems, and it can be applied at the component level but also at the facility level. To summarize the main findings from the works reviewed, in the following, the main pros (+) and cons (−) of the two approaches are presented.

As far as physical model-based approaches are conceived:

(+) They are made up of simple and usually well-known algorithms based on physical laws and equations.

(+) Once the models are calibrated, online data can be used directly to predict some indicators and compare them with the real situation without the need for training the model periodically.

(−) The integration of physical models from different components to create a complete facility model may suppose an important effort to be assumed by developers and testers.

On the subject of the use of data-driven methods, the approach is slightly different:

(+) Can be useful to represent complex real phenomena that are not easy to explain with equations based on physics.

(+) Can be very accurate in the prediction of the behavior of a system.

(−) The models have to be trained using usually different datasets for the different states of the system, one for a regular state and one for each possible failure state or even combination of several failures at the same time.

(−) It is necessary to have a huge amount of experimental data that in most of the cases are not available, with the consequence to resort to using small not representative experimental datasets or data coming from simulations or physical models.

By way of conclusion, even though it seems that every time more algorithms used for fault detection are based on machine learning (ML) techniques such as artificial neural networks (ANNs), a recent study [95] concludes that some of these approaches are not useful, since models trained with laboratory data or data coming from simulations do not achieve a good enough performance when working with online data. On the other hand, these kind of FDD applications show a very promising growth and may be a good option to solve complex FDD problems in the near future.

4.3. Overview of Commercial Platforms Capabilities Available in the Market

Conversely from HVAC systems in buildings, in DHC applications, there are more chances to find a high-level supervisory control and data acquisition (SCADA) infrastructure in place that allows taking advantage of innovative platforms available in the market for load forecasting, operational planning and anomalies detection. Even if it is out of the scope of this work to perform a detailed technical review of the platforms available in the market, the authors performed a high-level analysis based on a few publicly available information gathered and without pretending to be exhaustive. The capabilities analyzed and the commercial platforms surveyed have been listed in

Table A2. Overview of the capabilities of the commercial platforms for planning, advanced control and fault detection in DHC systems.

Proprietary Platforms/Capabilities	TERMIS by Schneider Electric [142]	DANFOSS [154,155,156,157]	OPTIT [158]	Artelys [159,160]	NODA [161]	ENFOR [149]	INeS by DCbrain [162]	Gradyent [163]	EA-PSM by Energy Advice [151]
1-DH network planning and design		Energis Designer	OptiTLR
2-DH production optimization	Production Scheduler	Mentor planner	OptiEPM	Crystal Energy Planner		HeatPO
3-Supply temperature optimization	Temperature Optimization	Energis Operator, Mentor planner				HeatTO
4-DH load forecast	Load Forecaster	LeanheatA Mentor Forecast™		Crystal Forecast		HeatFor
5-Local optimized weather forecast						MetFor
6-Demand side management		LeanheatAI
7-Fault detection and diagnosis		LeanheatAI

of Appendix B. It has been considered the possibility for the exploitation of a physical thermal-hydraulic model of the DH network for design purposes, the possibility to run an advanced optimization algorithm for optimal production scheduling, supply temperature optimization, local weather forecast, DH load forecast, demand response operation and fault detection capabilities. Only for a few platforms has some detailed information been found. Termis Engineering by Schneider Electric [142] seems to be one of the most complete and can be applied to both DH and DC systems. It has also dedicated modules for supply temperature, pressures, pumping and TES optimization. The physical model of the DH network in Termis is based on the quasi-dynamic assumption with a static model for the evaluation of pressures and flows, and a fully dynamic temperature assessment [143]. It can be used offline for scenario analysis, but it can also run online predicting the impact of some control inputs in the DH system in real-time applications.

Danfoss after the acquisition of the companies OE3i [144] and Leanheat [145] is capable of providing an advanced modular-based software solution covering almost all the capabilities investigated.

• Overall, from the analysis performed on main European players it emerges that:
• Several platforms are conceived in hybrid solution by implementing a digital twin of the network based on physical models but also exploiting artificial intelligence algorithms (Danfoss, DCbrain, Gradyent).
• More and more platforms are using data-driven machine learning models for load forecasting and in some cases also at the building level (NODA, Danfoss).
• Termis, Danfoss and Gradyent have the capability to use the thermo-hydraulic model of the network (in some cases simplified) on-line as an operational support tool calculating optimal hydraulic parameters of the DH network (temperatures, flows and pressures) according to the forecasted boundary conditions.
• Some companies, even if they have developed thermo-hydraulic models for DH design and planning, they do not integrate them with their optimal dispatching tools.
• Stochastic optimization is handled by the dispatch optimization engines provided by Artelys and ENFOR.
• In [146], the developers of OptiEPM highlighted their choice in implementing the Matheuristics algorithms to achieve a better performance than a direct MILP approach from the computational time point of view that can be relevant in complex problems with a large number of control variables.
• Almost all DH production optimization tools surveyed are used to solve the unit commitment problem considering both long-term and short-term planning of the generation plants and considering the participation to the electricity markets (e.g., with CHP units).
• Few companies such as NODA and Danfoss focus on the demand side of the DH system for the implementation of demand-side management solutions for peak shaving.
• Some companies such as DCbrain, NODA, Danfoss and Gradyent apply artificial intelligence algorithms to provide predictive or condition-based maintenance and fault detection services.
• Termis Temperature Optimization is capable to transform the non-linear dynamic optimization problem in a linear programming one and to solve it [147]. Among the case studies reported about its application in Hjørring and Hørning [148], DH systems’ heat loss reduction of about 10% (from 23 to 20.7%) has been achieved.
• Only ENFOR [149] provide an additional tool called MetFor™ based on machine learning that is able to optimize the local weather forecast up to 10 days ahead from historical weather data and meteorological models, but also capable of detecting short-term deviations (12 h ahead) using real-time weather data.

Another tool that is also emerging in the academic literature is energyPRO [150], which can be used for technical and economic analysis of complex generation systems as well as for short-term operation scheduling. Energy Advice [151] developed a platform that allows for the creation of a thermo-hydraulic model of the network and to use it to improve the control of the DH system. Finally, it is worth mentioning that some companies developed advanced data analytics and visualization tool such as Tango [152] with the aim of getting more insight into information and future trends, but also integrating applications available by third party. A new start-up called Arteria [153] was born recently and intends to differentiate itself from the other companies by developing an innovative platform based on a physical model of the DHC network, thermoeconomics optimization and the assessment of the exergy flows in the DHC system.

5. Conclusions

With the growth of the world’s population and an increase in urbanization, district heating and cooling (DHC) has been identified as a promising technology to cover the thermal energy demands in urban areas. Notwithstanding, DHC is evolving towards lower distribution temperatures, the exploitation of decentralized and non-programmable renewable sources and sector-coupling with other energy carriers. However, the lack of optimal control strategies and fault detection tools leads DHC systems to waste energy and resources. Moreover, new challenges such as demand-side management, weather uncertainty and environmental efficiency have emerged. Those have attracted more and more attention to the design of intelligent, robust control platforms as well as diagnostic methods. These advanced solutions, reviewed in this paper with a focus on low-temperature DH studies and EU research projects, must be able to predict events, make real-time decisions and be integrated with SCADA systems in order to push the DHC sector forward in the digital transition.

As far as advanced control in DHC is conceived, most of the applications follow purposes such as decreasing fossil fuel consumption, energy-savings maximization, emissions and operating costs reduction. Two main different approaches have been identified that are model predictive control (MPC) and multi-agent systems (MAS) control. The latter has been applied in very few cases but has the potential to reduce the need for extensive data manipulation and enhance the openness of the system to third parts and peer-to-peer markets, but there is still a lack in the standardization of communication and development. MPC has a wider application than MAS and can be considered a cutting-edge technology to improve DHC operation. On one hand, it has the main ability to consider weather forecasts, occupancy profiles, together with stochastic disturbances taking into account system constraints in real-time optimization. On the other hand, it can be time-consuming for the identification of the system model and require high-skilled personnel for development and use.

Among the hard control implementation of MPC, mixed-integer linear programming (MILP) is one of the most used thanks to some features such as rigorousness, flexibility, finding the exact optimal solution and extensive modelling capability. With the advancement in swarm intelligence and machine learning, some soft and hybrid forms of MPC have also appeared in DHC applications. In particular, gradient-free meta-heuristic optimization methods, such as genetic algorithms (GA) and particle swarm optimization (PSO), are becoming more and more popular, since they are capable of quick convergence, flexible control parameters and do not need complex mathematical operations. However, dropping into local optimum, stagnation after some initial stages and reducing the efficiency with increasing dimensions are the principal disadvantages of these methods.

Similarly, as in predictive control strategies, in fault detection and diagnosis (FDD), two main approaches emerged: physical-based versus data-driven modelling. Models here are useful in software-based solutions to detect malfunctions from the deviation of the system from standard operation. Alarms can be activated when some calibrated thresholds are exceeded. The use of machine-learning-based FDD tools is growing in DHC, since they can handle complex systems with a large number of variables and have excellent performance in the behavior prediction of non-linear systems and in pattern recognition. However, these solutions have some drawbacks, since large datasets are needed for training over different operating conditions to avoid low performance in the extrapolation in real-time operation. Among hardware-based FDD approaches, infrared thermography found applications for leak detections in DHC pipeline and can be further boosted by means of automatic image recognition algorithms. Predictive control and fault detection sometimes share similar approaches and from the survey performed emerges that some companies developed innovative platforms based on artificial intelligence for both applications.