Identification of Gaps and Barriers in Regulations, Standards, and Network Codes to Energy Citizen Participation in the Energy Transition

Abstract

The success of the energy transition in Europe depends on the engagement of citizens and the sustainable replacement of conventional generation with renewable production. Highlights of the PAN European Technology Energy Research Approach (PANTERA) project, a H2020 coordination and support action, are presented in this paper. In broad terms, PANTERA offers a forum for actors in the smart grid to support the expansion of activities in smart grid research, demonstration, and innovation, especially in the below-average spending member states in the European Union (EU). The focus of this paper is on those activities of the project consortium related to the identification of gaps and barriers in regulations, standards, and network codes that hinder the sustainable engagement of energy citizens in the energy transition. The paper summarises the challenges to citizen engagement in the energy transit and considers the enablers that make the engagement of citizens viable, e.g., demand response (DR), renewable energy resources (RESs), and modern designs for local energy markets (LEMs). We focus on the barriers to the enablers that are explicitly and implicitly related to regulations, standards, and network codes and explore aspects of the relevant regulations and standards of the sample below-average spending member states. A specific case study of a research and demonstration project in Ireland for updating the network codes is also presented to demonstrate the ways in which member states are attempting to remove the barriers and enable citizen participation in the smart energy transition. Finally, the opportunities to foster smart grid research and innovation through shared knowledge and insights offered by the PANTERA European Interconnection for Research Innovation and Entrepreneurship (EIRIE) platform are highlighted.

1. Introduction

The European Union (EU) is striving to reduce greenhouse gas emissions by 80–95% by 2050 and has set a 97% renewable energy production target to decarbonize the energy sector [1,2]. The successful energy transition of Europe is contingent upon successful and sustainable replacement of conventional fossil-fuel generation with renewable production. Given the variability of renewable energy and the unavailability of conventional generation, new sources of flexibility are sought. Demand-side energy management has shown promise in fulfilling this requirement. As this paper will show, a wide range of challenges hinder the successful integration of energy-enabled consumers, known as prosumers, into the energy transition to achieve the required flexibility. The successful energy transition, therefore, depends on the identification of the important gaps and barriers to the sustainable integration of consumers and directing the global efforts to effectively bridge such gaps and remove those barriers. The gaps and barriers are rooted in policies, regulations, codes, standards, and technological challenges.

PAN European Technology Energy Research Approach (PANTERA) is a Horizon 2020 coordination and support action (CSA) aimed at supporting smart grid research and innovation (R&I) in the member states (MSs) that spend less per capita on R&I in the energy domain (as identified by [2]), including Cyprus, Bulgaria, Ireland, Italy, and Latvia. PANTERA offers a forum to connect stakeholders, holding workshops, and making results and data about EU smart grid R&I available through a central IT platform [3]. The identification of barriers to the efficient and sustainable engagement of European citizens in the energy transition is one of the main targets of PANTERA.

Among the challenges identified by PANTERA, the barriers and gaps in regulations, codes, and standards (RCSs) [4] are the focus of this paper. Dealing with the gaps in RCSs are of special importance for implementing smart grid projects as RCSs facilitate the engagement of energy citizens in the energy transition and control what products/services can be sold and bought. Service quality assurance is out of reach if effective standards are not addressed. By following industry standards, adherence to the prevailing codes, and complying with local regulations, various entities can participate in the energy transition to not only fulfill their decarbonization role but also to increase/decrease revenue/cost. A review of RCSs was conducted by PANTERA to understand the challenges and prospects for smart grid research and innovation in Europe [5]. This paper presents a high-level overview of PANTERA’s European RCS review which focuses on technical and legal barriers in RCS to support the future functionalities of energy systems as defined in the European Technology and Innovation Platform on Smart Networks for the Energy Transition (ETIP SNET) 2050 roadmap [6].

We use a four-step methodology to identify the contemporary RCS-related challenges in the successful integration of energy citizens into the energy transition.

First, a high-level literature review is conducted to identify all reported challenges to citizen engagement in implementing smart energy systems. Then, “enablers” of citizens’ empowerment in the energy transition identified in the PANTERA project are considered. These enablers include energy efficiency measures, demand response (DR), energy management systems (EMS), distributed energy resources (DERs), renewable energy resources (RESs), energy storage systems (ESSs) and local energy markets (LEMs). A literature review is conducted to provide a comprehensive list of possible barriers to the enablers. The output of this step is a set of barriers to each enabler that hinders the citizens’ engagement in the energy transition.

The second step is to trace the barriers back to the gaps across five themes: “policies and policy conflicts”, “regulations, codes, and standards”, “technical advancements”, “social behavior “, and “financial support”. The output of this step is a set of gaps in each theme.

In the third step, the barriers linked to “RCSs” are filtered. Some of these barriers are more general and hinder the effective energy transition in most of the MSs. Some other barriers are no longer a real concern to the more active countries in the energy transition but are yet hindering the sustainable engagement of consumers in the energy transition in below-average spending countries.

The fourth step is dedicated to investigating to what extent each of the RCS barriers hinders the citizens’ engagement in EU member states. The investigation is performed for a subset of below-average spending MSs based on their national codes and standards already in place, local documents, the national energy and climate plan (NECPs), and, if available, their plan for upgrading national RCSs.

Finally, a specific case study of a research and demonstration project [7] based on the project trials in Ireland [8] is presented to showcase how research and innovation projects can provide insight for updating the network codes.

The remainder of this paper is organized as follows: Section 2 provides in in depth literature review. Section 3 presents the barriers to accomplishing the enablers of citizens’ participation in the energy transition, i.e., building energy efficiency, DR, building EMSs, DERs, RESs, ESSs, and LEMs. This section also categorizes the barriers into general and local barriers. Section 4 filters the barriers related to RCSs. These challenges are traced in current and planned-for-future policies, regulations, codes, and standards of some low-spending MSs. This section also presents a specific case study of providing insight for updating the network codes based on the results of some trials in Ireland. Conclusions are drawn in Section 5.

2. Literature Review

The remainder of this Section is dedicated to reviewing the academic publications, grey literature, i.e., related policies, and regulatory and engineering reports that focus on various or some certain challenges and barriers to the sustainable realization of the enablers introduced earlier in this section.

2.1. Energy Efficiency

In terms of improving energy efficiency, a literature review in [9] presents the general barriers to the improvement of citizens’ energy efficiency. Building renovations can improve energy efficiency, directly and indirectly, reduce emissions, and enhance human health. More than 97% of the buildings in the EU hold an energy rating below-A. Reference [10] reviews the social barriers that hinder building energy efficiency. This reference provides a detailed assessment of the opportunities, costs, and benefits of building energy efficiency projects as well as an approach to review what options are available and applicable and the relative costs and benefits of such options. Another study [11] is focused on social behavior and building renovation. This study identified the main barriers at technical, statutory, financial, and behavioral levels. The lack of standards delineating the minimum level of renovation in different classes of buildings that can guarantee compliance to the climate neutrality objectives is studied in an ongoing study, i.e., “Renovate2Recover” [12], which analyzes the progress of such standardization in the member states.

2.2. Demand Response

Demand response programs pervade the NECPs of all member states as a key enabler of the energy transition in Europe. Exploiting the flexibility of residential demand has become more challenging and, at the same time, more advantageous as electric heating and transport are gaining popularity, and, in turn, electricity demand is increasing. Recent research in [13] bolsters the key message that boosting the flexibility level of the distribution system is a key enabler in reducing the costs of integrating RESs. While industrial and commercial customers currently contribute more to DR programs in most European countries and the UK [14], from a theoretical viewpoint, residential customers represent a larger source of flexibility with a more potential effect on the decarbonization of energy systems [15]. On the other hand, while it is undeniable that some citizens are ready to engage in at least some forms of DR programs, the research shows that the customers’ engagement could be variable [16]. Investigating the consistency of customers in providing flexibility helps identify the gaps and barriers, predict DR potential, remove the barriers to boosting the consumers’ participation in DR programs. Based on a systematic review of demand response trials and best engineering practice across Europe, it becomes clear that the barriers of engaging residential consumers are rooted in either consumers’ participation (being enrolled in DR programs), performance (delivering the service in a desired manner), and, finally, the persistence of service provision over time [14]. The EU level and national regulations and policies have the greatest impact on user engagement in residential DR programs. Policies may pave the way to reduce marketing costs to engage the consumers who are likely to offer their best performance, and regulations can help expedite the connection to the grid and the continuity of service.

Reference [14] conducted a literature review to find the key factors affecting the engagement of residential consumers in DR programs from their viewpoint. Among motivations, monetary and environmental concerns were the most frequent motivations identified, while the monetary benefits were typically given the highest importance. This result is confirmed by [17]. The enrolment rates in various sub-categories of DR programs differ mostly because of their effect on the bill reduction. The effects of flexibility provision on emission reduction need to be better presented to citizens to remove ambiguities [14,18]. One of the gaps in this regard is the lack of focus on promoting the benefits of engaging with DR programs to increase the penetration level of renewable energy resources.

There are some pieces of evidence that, after enrolling in DR programs, citizens continue to compare the potential monetary benefits against the effort, time and loss of comfort when deciding whether to stay active or not. At this stage, it is very important to provide tools that facilitate responding to dynamic pricing signals. The level of citizens’ trust can generally be enhanced by measures that improve clarity around DR. Such measures include providing information on DR from independent sources [14], communicating how different parties, such as participating citizens and energy providers, benefit from DR [19], and regulations that ensure notifying DR participants users of any direct load control actions taken [20]. It is shown in [21] that the citizens prefer incentive-based DR over time-varying pricing because of the higher risk of the latter. However, as summarized in [17], in a series of trials in the US, little difference is observed in actual rates of enrollment in variable pricing and rebate-based DR.

Energy and network tariffs that do not change enough for different periods are volumetric, capacity-based, or sometimes fixed and can make the energy citizens indifferent to the level and pattern of energy use. Such tariffs are not able to incentivize the citizens to engage in time-of-use demand response programs. Reforming tariff structures is a crucial factor in motivating the consumers to switch to electrified transport and e-mobility and the use of electrical heating, e.g., through heat pumps, as researched in [22]. The available schemes for remunerating distribution system operators (DSO) only or mostly motivates wire solutions and non-wire solutions, such that the support provided from DR is not well incentivized. The related issues are studied in [23].

2.3. Local Energy Markets

Local energy markets (LEMs) are the next enabler that facilitates the citizens’ engagement in the energy transition. The EU is being prepared to set an exceptional standard by ratifying the role of citizens and communities in the energy transition. Nearly half of all EU citizens could be engaged in producing renewable energy by 2050, about 37% of which could be realized through the involvement of such citizens in citizen and renewable energy communities (CECs and RECs). The market design initiatives must put strong regulations in place to acknowledge, allow for and offer rights to households that want to participate in energy communities and, ultimately, in energy markets. Essentially, the electricity market design should be coherent with other regulations in the clean energy package (CEP) that have already been decided by the European Parliament and the Council. The European Commission has acknowledged that new policies and structures for electricity markets are required to expedite the energy transition, better fit to current advancements, encourage prosumers rather than consumers, motivate the citizens to further participate in collective projects and electricity markets, and hold demand-side flexibility as one of the main priorities. The EU Electricity Directive and Renewable Energy Directive indicate the vision of the EU on the future structure of the energy system. For the future of energy communities in Europe, how these directives will be interpreted into national regulations matters the most. Local energy trading, for instance, through peer-to-peer trading or a community energy market if the related barriers are effectively dealt with. This paper also summarizes the barriers to the successful and sustainable introduction of demand-side products and services into electricity markets.

Reference [24] reviews various market roles that can be identified in electricity markets that are being designed for the near future. These roles include prosumers, facilitators, producers, energy service companies (ESCos), aggregators, suppliers, DSOs, transmission system operators (TSOs), and balance responsible parties (BRPs). Many challenges in the implementation of an effective and enduring structure for the energy market root in conflicts between and the lack of accurate definitions for these roles. Reference [25] provides a range of market design challenges along with some solution approaches proposed in the academic literature and industry.

Other barriers stem from new technical responsibilities that nontechnical organizations must address. For instance, as aggregators benefit from demand-side flexibility, they should handle the power quality issues. Reference [26], as one of the early research papers on the subject, presents the related challenges and a method to deal with such issues. This method coordinates the actions of the aggregators with DSO operations by assuming that there is another role as the distribution network operator (DNO) that provides the results of state estimation and a set of sensitivity coefficients, using which the operational limits can be modeled. In practice, however, such an assumption does not hold. The challenge of multi aggregators and fairness in dividing the technical responsibilities between these aggregators yet remains to be addressed. Aggregators may play an important role to allow groups of energy citizens or renewable energy communities to participate in LEMs.

DOMINOES, a European research project, reviews the policies and regulations in the CEP to provide an overview of the concept of LEMs in [27]. They highlight the opportunities to new and existing stakeholders and identify advantages and challenges of, and barriers to, the effective and sustainable development of LEMs. Some barriers in [27] are no longer a real concern, especially to active countries in the energy transition. However, new challenges arise, as the business environment is always evolving.

2.4. Smart Meters and Technical Issues

Other research studies identify much more specific challenges. Smart meters are another key component to the operation and management of market flexibility. Luckily, the penetration of smart electricity meters in the EU has passed the 50% mark in 2020, owing to expanded investments in grid digitalization. However, smart meters have not yet been installed for many households; there is a need to unify and tighten standardization in metering schemes. More context on the plans for successful implementation of the smart meter systems; the respective challenges and barriers are provided in the USmartConsumer project and reference [28].

The barriers to the safe operation of microgrids, which, in some structures of LEMs, play an important role, are discussed in [29]. These challenges are mostly technical and related to operation codes, e.g., for islanding mode of operation and switching modes. Some other barriers are related to the long administrative procedures and delays of small-scale rural DERs. For instance, in some member states, there are no (or no expediting) regulations for the connection of small-scale renewable generation in rural zones. These challenges are identified in [30].

2.5. Taxation

Tax regulations also play an important role in achieving the climate and energy targets. The rules set under the Directive 2003/96/EC, i.e., Energy Taxation Directive (ETD) aim at ensuring the proper implementation of the internal markets. However, the climate and energy policies have radically been changed and ETD is no longer in line with EU policies. The ETD was evaluated in 2019 [25], where the taxation barriers to the energy transition were discussed at length. Subsequently, the Council invited the Commission to revisit the ETD [31].

3. Possible Barriers to Accomplishing Enablers of Energy Citizens in Energy Transition

In this section, we explore barriers to the enablers identified in Section 2.

3.1. Barriers to Citizens’ Energy Efficiency Improvement

Building energy efficiency is linked to building standards and policies and is becoming more related to renewable energy policy and network policy as domestic buildings are increasingly integrated into the energy systems as not only end-users of energy but also the providers of a wide range of services such as energy provision, storage, and flexibility.

For an energy citizen willing to be a part of the energy transition, it is important to put energy efficiency as the priority, before engaging in service provision. Hence energy management in citizens’ buildings must prioritise energy efficiency at the center of new and revised building energy policies. This means that citizens should not use more energy than necessary and energy policies should ensure that building residents have a comfortable and healthy life and workspaces. Building renovations can improve energy efficiency directly and, thus, can indirectly reduce emissions; moreover, it enhances human comfort and health. Many activities have focused on building renovation for energy saving; however, the expected saving has not been realized (see Section 2.1). The EU encourages building renovations. The success of European renovations will be measured not only by the quantity but also by the quality of these renovations. The European Commission considers renovations that result in more than 60% savings to be “deep renovations”. Unfortunately, despite the climate crisis and the rise of energy prices, many renovations are far from reaching the 60% mark, let alone the 80% energy savings that are technologically feasible for most buildings [32]. Analysis of the National Recovery and Resilience Plans (NRRPs) shows that most of the renovation investments planned under the Recovery Fund will deliver only 30% energy savings.

The remainder of this subsection lists the barriers to the sustainable enhancement of energy efficiency, including those that most hinders effective renovations:

1-1

Complexity of associated renovation and lack of skills in the supply chain of renovation works. This indicates the necessity of sharing the information and experience gained in accomplishing successful renovation projects.

1-2

Quality of renovation. Sometimes renovations are cosmetic fixes only. The lack of monitoring bodies that enforce building energy efficiency certification can be an important barrier in this regard. The citizens’ awareness about certification should also be improved.

1-3

Institutional and legal frameworks that slow down renovation projects. An example is the resistance of groups involved in urban decision-making, as they believe this may distort the buildings’ view (an RCS conflict barrier).

1-4

Lack of access to finance while renovation costs are high. Renovations are often resource-intensive, both in terms of financing and time. Sometimes, the energy performance improvement achieves less value than the required investment costs. Discrepancies between predicted and actual savings also reduce the citizens’ trust in energy efficiency projects. Therefore, it is important to provide a clear picture of the cost and saving for citizens. Financial incentives are weak and external risks such as price volatility give rise to the lack of citizens’ motives.

1-5

Lack of standards delineating the minimum level of renovation in different classes of buildings. A “deep renovation” standard in the Energy Performance of Buildings Directive (EPBD) is vital for more highly energy-efficient renovations. The Commission “deep renovation” standard can be bolted onto the EPBD, which is promised to be updated later this year. National standards also need to be updated according to the EC EPBD. Currently, such national revisited standards do not exist in most European countries. An ongoing study, i.e., “Renovate2Recover” [16], is analyzing the progress of such standardization in the MSs.

1-6

Split incentives, lack of communication between buyers and building constructors, and a fragmented real estate market [14]. An example is different and sometimes opposing interests of constructors and final building buyers. For instance, the constructor may favor low cost over the efficiency of the equipment. The construction and renovation industries and be overly conservative and rely on traditional methods or can oversize equipment. Building energy policies should be revisited to avoid such barriers.

1-7

Barriers related to citizens’ behavior such as the lack of shared objectives among citizens, and inertia, e.g., aversion to change and lack of understanding about how renewable energy communities operate and share access to RES fairly.

1-8

Lack of information and knowledge regarding energy efficiency and sustainable products. These barriers continue to be an important cause hindering energy efficiency. The citizens’ perception of high investments and long return times is an important issue that should be clarified for end-users.

The above barriers to investment in building energy efficiency highlight broad gaps that can be examined in more detail by looking more closely at specific barriers or certain sections of the building and construction industry. However, many of the important barriers can be mitigated by taking effective measures. The above barriers only include those reviewed in the literature and are still a real concern. The target of domestic building energy efficiency measures is to reduce the overall consumption permanently. Nevertheless, in contrast to energy efficiency improvement, demand response means an action that changes the consumption pattern and does not necessarily lead to an overall decrease in demand. Analyzing the barriers to the effective implementation of demand response programs is the subject of the next subsection.

3.2. Barriers to Sustainable Engagement of Citizens in Demand Response Programs and BEMSs

Demand response programs pervade the European NECPs as a key enabler of the energy transition. In the case of electricity, DR means changing the pattern of the daily load. Demand response programs can help to reduce the size of peak demand. Reducing peak demand, in turn, reduces the power and transmission capacity needed to serve the demand. Smoothing the demand curve, using the existing network and, in general, using the available energy infrastructure more efficiently is crucial when we heed the expected growth of overall electricity demand due to the electrification of urban activities, including transportation (electric vehicles) and heating (heat pumps). DR entails managing the consumers’ demand profiles based on grid requirements. Therefore, it is more complicated to implement, requires more effective regulations/policies to become ubiquitous, poses more challenges on the grid operation, and is harder to be fully explained to the residential consumers. Nevertheless, DR helps increase the penetration of RESs, especially through energy communities or by energy citizens and, therefore, paves the way towards further decarbonization of the energy sector.

By reviewing the reports of trial projects and best engineering practice across Europe, it becomes clear that the barriers of engaging residential consumers can be rooted in either consumers’ participation (being enrolled in DR programs), performance (delivering the service in a desired manner), and, finally, the persistence of service provision over time [14]. These steps are summarized in Figure 1.

Reference [33] classifies DR programs into price-based DR (including time of use (ToU) tariffs, real-time pricing (dynamic pricing scenario), and peak load pricing), and incentive-based DR programs (including direct load control, curtailable load, and demand-side bidding in the capacity market and ancillary services). Figure 2 presents such a classification for residential DR programs. A residential building can participate in all price-based programs [34]. Smart meters and other advancements have made multi-tariff plans and dynamic pricing viable. In terms of incentive-based DR programs, the direct participation of individual buildings in the energy market is not viable. However, the participation of residential consumers can be aggregated and offered to the market. In direct load control programs as the next incentive-based program, the building owners hand over the control of certain equipment, e.g., freezers and beverage refrigerators, to the utility operator for additional remuneration. The associated slight alteration of the load is included in the uncertainties, as the decision-makers, i.e., the building owners, have no control over it. Curtailable loads are also another service that can be offered by residential consumers for a certain portion of their load. The classification of DR programs highlights some important barriers and bottlenecks for engaging the consumers in DR programs.

Consumers’ engagement in DR programs is more effective if other supporting technologies, e.g., energy storage systems, DERs, building energy management systems, smart meters, and the internet of things, are also adopted. Therefore, other than policy and regulations barriers in the promotion of consumers’ engagement in DR programs, the immaturity of such technologies on a domestic scale may also hinder the response and, especially, the persistence of the consumers’ engagement in DR programs (see Figure 1). The technologies that support the successful implementation of each type of DR program are also presented in Figure 2. Even though DR and generally demand-side management can be considered as a technology, its successful implementation is rooted in the readiness of many other technologies, including, but not limited to, those introduced in Figure 2. This indicates that the immaturity of these technologies can be one of the important barriers in the enrolment, responsiveness, and persistence of energy citizens in DR programs. For more details of PANTERA methodology to analyze the technology readiness, see [4].

Reviewing the academic publications, grey literature, i.e., related policies, and engineering reports, may also reveal more barriers to the successful implementation of DR programs. Reference [14] conducts a systematic literature review to find the key factors affecting the engagement of residential consumers in DR programs from the viewpoint of these consumers. As discussed in Section 2.2, among such motivations, monetary and environmental concerns are the most frequent motivations identified, while the monetary benefits are typically given the highest importance. In this regard, some consumers state that bill reduction is much more appealing than bonuses or other financial incentives. Therefore, as expected, the enrolment rates in various sub-categories of DR programs are different mostly because of their effect on the bill reduction. As much as the importance level of the financial aspects seemed to be obvious, with the national and EU-level programs for improving the awareness of the citizens of environmental concerns in the energy sector, it was expected that the environmental concerns be of much more important for the consumers than what found in the related research, e.g., [14]. One of the reasons is that most of the impacts of citizens’ engagement in DR programs on pollution reduction and climate change are not obvious for the citizens. For example, because in most DR programs the total electricity use will not necessarily decrease [18], the citizens might not comprehend the potential environmental benefits of participating in DR. In this regard, one of the gaps is the lack of enough focus on promoting the effects of engaging with DR programs on the penetration level of renewable energy resources. Most of the citizens are not aware of such a potential, as the focus of the programs provided for improving the awareness of the citizens has been on the detrimental effects that conventional energy generation might have.

There is some evidence that, after enrolling in DR programs, citizens continue to compare the potential monetary benefits against the effort, time, and loss of comfort when deciding whether to stay active or not. At this stage, it is very important to provide them with some tools that facilitate responding to dynamic pricing signals or make this challenge enjoyable for them, for instance, by presenting this challenge as a game to children [14].

Barriers 1.1–1.8 identified challenges for building energy efficiency improvement. The successful and sustainable engagement of energy citizens in demand response programs requires those and additional barriers to be addressed as without being able to optimize the use of energy in buildings, citizens do not have the potential to take part in demand-side support.

In addition, there are many social barriers, e.g., lack of trust which stems from gaps in policies, e.g., the lack of trust can be (partially) mitigated by the policies that encourage clear communication with citizens. Other common barriers to DR are related to regulatory and specifically market barriers (see Section 3.3). We summarize the main barriers to the sustainable engagement of citizens in demand-side management programs as:

2-1

Citizens’ unfamiliarity and mistrust

-

Unfamiliar technology/technical terms

-

Lack of transparency around what DR entails and whom DR benefits

-

Mistrust in community-based mechanisms

2-2

Perceived loss of control and associated risk

-

Long-term time-varying pricing may hinder enrollment

-

Fear of loss of control of the citizens over their demands/tasks

-

Unpredictable short-term prices that may deter citizens’ persistence

-

The prices should be predictable, but variable enough to guarantee the earning

-

Lack of DR models that are understood by citizens and offer acceptable control

2-3

Complexity and effort

-

Inconvenience and discomfort associated with demand shift

-

Low reimbursement compared to the underlying decrease in comfort level

-

Complexity and required effort of responding to time-varying prices

-

Unpredictability of the weather in Western European countries

2-4

Need to install new equipment and technologies

-

High cost of such technologies

-

Space required

-

Disruption of services while installing the required equipment

-

Lack of trust in additional technology

-

Associated complexity and risk of failure of new technologies

2-5

Insufficient wholesale price variation discourage engagement in dynamic pricing DR

-

Conflict with other conventional use cases that favor low variation in prices

2-6

Energy and network tariff structure does not support demand shift in time

-

Lack of motivation to switch to e-mobility and the use of electrical heating

2-7

Distribution System Operators (DSO) remuneration approach

-

Preferring wire solutions over non-wire solutions

-

Lack of policies for the gradual transition from old DSO remunerating models

2-8

Necessity to give access to third-party actors

-

Low weight of the demand-side stakeholders in policymaking

-

Concerns about privacy and security

2-9

Ambiguous or no definitions for rights for direct control of citizen’s loads

-

Since different entities might make use of customers’ load control for different purposes, there is a need to define certain rights and obligations which are applicable to the parties responsible for power balance.

From a higher-level point of view, most of these barriers are rooted in, policymaking gaps, e.g., the higher weight of the supply-side stakeholders in decision-making for the development of regulations. Many of these barriers also stem from supply chain barriers, e.g., energy markets have been designed from a supply-side perspective. The stakeholders on the supply side have no genuine incentive to support demand-side services. Further, they may even have inducements to hinder the successful implementation of DR. Some other barriers originate from regulations interaction, an example of such a barrier is the conflicting objectives or priorities of different actors when making the supporting policies. These categories of gaps are added here to the list of barriers to sustainable engagement of energy citizens in the energy transition. These gaps may also be the main cause of many other barriers to the other enablers that will be discussed in the upcoming subsections of this Section.

2-10

Policymaking barriers

-

Potentially higher weight of the supply-side stakeholders in decision-making

2-11

Supply chain barriers

-

Old design of energy markets from a supply-side perspective

2-12

Regulation interaction barriers

-

Conflicting objectives or priorities when devising supporting policies

-

Conflict between price variability to motivate price-based DR and the need to stabilize the prices and make them predictable for other consumers.

3.3. Barriers to Efficient Local Energy Markets

The market design initiatives must put strong regulations in place to acknowledge, allow for, and offer rights to households that want to participate in energy communities and, ultimately, in energy markets. A variety of entities and organizations, e.g., local authorities, renewable energy cooperatives, NGOs, members of the renewable energy industry, and community energy coalitions are working together to provide a fair deal for households as “energy citizens” as envisaged in the EU’s CEP, and to ensure nobody is left behind in the energy transition of Europe.

As co-legislators, the European Council and European Parliament have been tasked with ensuring that all European citizens can harness this potential. Essentially, the electricity market design should be coherent with other regulations in the CEP that have already been decided by the European Parliament and the Council. Particularly, such regulations should not be inconsistent with provisions in the Renewable Energy Directive. Such provisions include: 1. definitions of RECs and self-consumers, 2. allowing for this citizens’ right to take part in the energy transition as an active customer or an energy community without losing their rights as consumers, 3. acknowledging the right to access all types of relevant markets without idiosyncrasy or disproportionate treatment, 4. considering a right to sell energy through suppliers and peer-to-peer energy sharing and, finally, 5. acknowledging the benefit that energy citizens and energy communities can bring to the energy system and remunerate their participation. The European Commission has acknowledged that new policies and structures for electricity markets are required to:

-

Expedite the energy transition

-

Better fit to current advancements

-

Encourage prosumers rather than consumers

-

Motivate citizens to participate in collective organization/electricity markets

-

Put demand-side flexibility as one of the main priorities

The EU Electricity Directive and Renewable Energy Directive indicate the vision of the EU on the future structure of the energy system. For the future of energy communities in Europe, how these directives will be interpreted into national regulations matters the most. A variety of roles and functionalities beyond energy efficiency improvement and saving and energy generation might become possible, feasible, and appealing for such communities. This subsection presents various roles in the current and futuristic scenarios of electricity market structures in the EU. It also describes which roles can be taken by energy communities and energy citizens.

Many businesses and citizens are installing RESs. Since traditional flexibility sources such as conventional power plants are going offline due to environmental concerns and their impact on climate change, the flexibility must take new forms. In this regard, a demand-side flexibility provision is essential for the EU to meet its sustainable energy targets. Active customers have the potential to provide the new form of flexibility that electricity networks need. The flexibility that they individually offer might be insignificant but when pooled or “aggregated” such flexibility could be sufficient to cater for a considerable amount of flexibility that a power system might need. Various market roles can be identified in some of the current electricity markets and those which are being designed for the near future. Such roles are outlined below, as defined in [24]:

-

Prosumer: Consumes energy, produces energy and may provide flexibility. An example of such prosumers is the citizens that have PV panels on their roofs.

-

Facilitator: Facilitates implementation of DERs, RESs, RECs, CECs, and so on. In many energy communities, one of the reasons to establish such a community is to facilitate the uptake of RESs and other energy generators in their community by, for example, providing help with financing, awareness increasing, joint purchasing, and knowledge sharing.

-

Producer: Generates energy and feeds this energy into the electricity network. If RECs have invested in a collective generation project, such as a collective rooftop PV system or a wind park, they are taking the role of a producer.

-

Energy Service Companies (ESCos): Provide energy profile optimization tools and services. An example of such providers is a company that offers cloud-based building EMSs. An energy community might also be able to provide technologies/management systems that optimize energy profiles in response to varying external signals, e.g., energy or flexibility prices.

-

Aggregator: Pools and sells the flexibility that citizens and communities might be able to offer. An energy community itself can combine the flexibility of multiple households and together as a single “package” introduce the collative flexibility to the energy market and perhaps directly sell it to another party that may want to buy flexibility. Aggregators pool enough flexibility from multiple flexibility suppliers (who can be energy citizens or energy communities) to provide a worthwhile amount of flexibility to its users, such as distribution system operators (DSOs), transmission system operators (TSOs), and balance responsible parties (BRPs). The need for demand response aggregation and the aggregator role has been highlighted in the European CEP. This package also provides a series of directives defining such a role in electricity markets. Specifically, Directive 2019/944, Article 17 presents the new features of electricity market designs that deal with demand response through aggregation. This article requires all MSs to develop the necessary regulatory framework for (independent) aggregators and demand response to participate in energy and flexibility markets. In addition, article 32 seeks to motivate the use of the flexibility provided by the aggregators in distribution networks. It encourages the MSs to develop the essential regulatory framework to let the TSOs and DSOs deploy such flexibility to alleviate congestion (for adherence to both line power carrying limits and statutory voltage constraints) in their networks.

-

Supplier: Buys/sells the extra energy produced by citizens/communities. Note that an aggregator or the community itself might also take the role of a supplier.

-

DSO: Effectively manages the distribution systems at low- and medium-voltage (LV and MV) levels. DSO is generally responsible for regional grid stability and adherence to the power quality standards. In the futuristic scenarios for energy markets, energy communities might be permitted to operate their LV distribution (micro) grid. In the new generations of electricity markets, it has been proposed to regard the DSOs as fully independent bodies and even remove their technical role. Distribution network operators (DNOs) might take this role.

-

TSO: Actively manages transmission grid. TSO is regarded as responsible for system balance and adhering to the security and power quality standards at the HV level. The physical extent of TSO’s remit is most often large and beyond the capabilities of energy communities to fulfill. However, in the new setup of energy markets, TSOs are still attractive associates for aggregators/communities as the buyers of flexibility.

-

BRP: Manages and is responsible for the balance of demand and supply in its portfolio. This party is responsible for and manages a very large portfolio. Thus, it is interesting for energy communities to collaborate with.

An example of the realization of incentive-based and price-based DR are schematically presented in Figure 3 and Figure 4, respectively, through demonstrating the money, flexibility, and information flow among the abovementioned roles. Sometimes, there are no clear boundaries between associated functionalities. As demand-side flexibility can be provided by households and communities, new chances arise for such parties to collectively take up new roles, e.g., aggregator and ESCo. Figure 5 presents a widely discussed structure for LEMs [25] in the presence of CECs. However, in practice, there might be a variety of structures for money, information and flexibility flow among the participants of these markets. An aggregator might also be a supplier, or most of the functionalities of aggregators and suppliers might be provided by energy communities in some instances. Regulators, system operators, market participants, and other parties in the current energy landscape all have different viewpoints on flexibility, and therefore approaches to procure the required levels of flexibility. However, a very transparent and integrated flexibility market requires more coordination of the abovementioned roles, as well as clear market mechanisms. This is an important challenge as it is still not clear how these roles and responsibilities will work in practice.

According to the regulations, currently, some roles are not allowed to be taken by one organization. An example is that the role of DSO and supplier cannot be both played by a single organization. In the future, energy citizens and energy communities can take roles that they could not previously adopt. This highlights opportunities to contribute to new activities that pave the way towards achieving citizens’ economic, environmental, and social targets. Examples of such activities are prosumer, producer, aggregator, ESCo, and supplier roles. The abovementioned roles, described in [25], reflect the current electricity markets and show the pathway that has been discussed in the past years for these markets.

In the remainder of this subsection, the main barriers to promising designs of LEMs, which mostly have their roots in the policies, regulations, and conflict among these roles are enumerated.

3-1

Financial risk due to the presence of giant investors: Even today, it seems that change is on the way. It is possible that new roles emerge or that the roles listed above are changed. All these changes are not happening in response to new national and EU policies. The industry is striving to get its share from the available opportunity to make a profit. New investors and companies with new specialties are getting interested in investing in the new structure of energy and flexibility provision in power systems. Not that the presence of new investments in the energy sector can necessarily be a challenge, the abovementioned point might distort the whole prospect. Some giant enterprises have started developing business models to take the place of the available organizations [35]. An example is Tesla, with a new “Energy Plan” to offer low electricity rates for citizens [35]. They plan to provide energy to households by rooftop PV systems and Powerwall VPP technology. In turn, it is sought that the households hand over the control of these resources to Tesla. They are targeting both energy and flexibility markets. An uncertain future affects citizens’ choices and the required design of future energy markets and related regulations.

According to USEF (a solid foundation for smart energy futures), a wide range of complexities should be first addressed, before the aggregators can achieve their functionalities. The barriers to the functionality of aggregators, i.e., the complexities that should be dealt with to make the aggregator role functional are presented in 3.2–3.5 below.

3-2

Complexity of measurement, validation, and baseline methodology: In remuneration of demand-side flexibility, a baseline is the value of demand/generation of flexibility providers before they change it based on the aggregator’s request. A baseline methodology is required to quantify the performance of flexibility service providers towards the customers of the flexibility. How to define appropriate baseline methodologies, roles, and responsibilities is an open question. Frameworks are needed for ensuring accurate and dependable data. It should be clear how to measure or calculate flexibility.

3-3

Joint remuneration of price-based and incentive-based demand response: It is important to find a method to effectively separate the share of price-based and incentive-based demand response when a consumer/energy community changes its demand/generation. In many cases, a flexibility resource may be subject to both price-based and incentive-based demand response. To remunerate the providers, the impacts of the two forms should be separated unambiguously.

3-4

Data confidentiality vs. transparency: A balance between transparency and confidentiality is hard to find. For efficient demand response, each participant in the new structure of the energy market needs some information from others. An example of this is aggregators who need demand, demand reduction capability, and demand reduction data to be able to accurately forecast the demand response, as well as for billing purposes. Nevertheless, some of this information might be commercially sensitive. Finding a balance between transparency and confidentiality is critical for deciding what information can be shared, as well as when and at what aggregation level this information is useful and can be passed to the respective bodies.

3-5

Data security challenges: As discussed above, local energy markets involve dynamic gathering and transferring significant amounts of data. Much of such data is of a sensitive and confidential nature. Secure data handling and protection from cyber security threats in this context are the main concerns. The respective challenges should be dealt with by ensuring a clear definition of responsibilities and updating the data exchange systems of local energy markets.

Some approaches to the above barriers are proposed in the academic literature and industry reports [15]. There are many additional barriers that stem from the transition from the old structure of electricity markets to the ones that are being put into action, and from the ambiguity about the associated new roles needed to assure the required functionalities. For most of these barriers, there have not been any effective remedies proposed. Most of the proposed approaches are based on oversimplifications. A concise list of such barriers is provided below:

3-6

Technical responsibilities for nontechnical organizations: To best utilize demand-side flexibility, according to the CEP, aggregators are supposed to conflate the capabilities of a large group of households in a DR pool and join, as a single participant, in the electricity market. To this end, aggregators need to consider the operational constraints of the local LV grids, including the voltage statutory limits. Otherwise, the power quality might be jeopardized. Neglecting the technical limitations, DR potential might be overestimated. This could lead to instability of the market and power systems, as the aggregators are not able to alter their demand/production when called on to do so. Reference [26], as one of the early research papers on the subject, presented a method to deal with such an issue. This method coordinates the actions of the aggregators with DSO operations and assumes that there is another role as the DNO with the functionality of providing the results of state estimation and a set of sensitivity coefficients, using which the operational limits can be modeled. In practice, however, such an assumption does not hold. Some experts believe maintaining the power quality is not a task to be assigned to aggregators. However, if the aggregators benefit from demand-side flexibility, they should handle the power quality issues. The problem is that they do not have the technical knowledge and the required data. This is classified as a conflict barrier, as the DSOs do not willingly help the aggregators if they are not receiving monetary benefits. Regulatory policies need to be amended to remove this conflict.

3-7

Technical limitations and fairness: Reference [26] assumes only one aggregator is in charge of pooling the demand-side resources in the LV feeder. In practice, however, many energy communities that can play the role of an aggregator might be available along with other aggregators. It is not clear which aggregators should share the task of solving the possible power quality issues and to what extent. In other words, there is no agreement on who is in charge of assuring the adherence to the statutory standards among the aggregators in an LV grid. There are other technical issues, such as voltage variations, high system peak levels, congestion and phase imbalances that are identified as the most common [36].

3-8

Recognition of user characteristics for market-oriented DR: Even though price-based DR programs, e.g., critical peak pricing, dynamic pricing, and time of use pricing (for which the challenges and barriers were reviewed in Section 3.2), have been implemented for many years across the globe, market-oriented DR is still taking its early steps. Considering citizens’ intended tasks, their purposes, and electrical safety, demand-side aggregators have no right to regulate user loads, e.g., by forcing the power-producing users to change their production patterns. On the other hand, the ambiguity in the citizens’ manual load alteration might lead to the deviation of the amount of increase/decrease in the production or consumption from the level that has been promised by aggregators. For aggregators, this can be interpreted as the (partial) loss of revenue. The limited data on citizens’ demand response also puts the aggregators far away from the true recognition of citizens’ DR characteristics. This leads to flawed decision-making by aggregators.

3-9

No distribution network operation role is allowed for energy communities: Currently, only industrial or commercial consumers can get exemptions regarding the operation of “closed distribution systems”. Domestic consumers and energy communities are not allowed to get such an exemption. In the future structure of the European energy markets, communities may be permitted to operate their community distribution network. Article 16 of the Electricity Directive should set the regulations to provide energy communities with a solid set of rights, involving an equal playing field and a right to build, keep, operate, and manage distribution networks or micro-grids or coordinately manage public distribution systems as well as ‘community networks’ (known as closed distribution systems, or microgrids). This right should not be discretionary. For this provision to be meaningful, it must be mandatory. On the other hand, the Parliament’s proposal to ensure compliance with national concession rules needs to be supported. However, the MSs should revisit the related regulations in their RCSs.

3-10

Legal issues related to new specially designed grid for energy communities: Local energy systems might require new distribution infrastructure, e.g., to connect the consumers for collative consumption. Such grids might be expanded to private properties, which do not necessarily belong to community members or to publicly owned lands. This gives rise to legal issues that must be anticipated in policies [27]. Further on this subject, it can create conflicts of interest, when the new grid intersects available distribution network rights-of-way.

3-11

Supplier license for sharing energy: Energy sharing within communities is very difficult to organize considering the current hindering legislations/Regulations. One reason is that each party that supplies energy is obliged to have a supplier license. It is sought that, in the futuristic scenarios for energy markets, energy sharing can be accomplished within a community.

3-12

Taxation barriers: The taxation of electricity plays an important role in achieving the climate and energy targets. The rules set under the Directive 2003/96/EC, i.e., Energy Taxation Directive (ETD) aim at ensuring the proper implementation of the LEMs. However, since 2003, the climate and energy policies have been changed radically and ETD is no longer in line with EU policies. More importantly, the ETD is no longer ensuring the proper functioning of the internal markets. Changing the ETD is a part of the European Green Deal (EGD) and the “Fit for 55” legislative package. The former focuses on tackling environmental-related challenges and achieving the EU’s domestic greenhouse gas reductions objectives. In the EGD the European Commission committed to revising the ETD to ensure that energy taxation is in line with climate targets. Taxation plays a direct role in supporting the energy transition by sending the right price signals and providing the right incentives for sustainable consumption and production. The ETD was evaluated in 2019 [25]. Subsequently, the Council concluded that energy taxation plays an important role in steering successful energy transition [29] and, hence, invited the Commission to revisit the ETD. The current ETD, however, hinders the effective energy transition, raises a series of issues linked to its disconnection from climate and energy objectives and its shortcomings regarding the functioning of the internal market. For instance, in Finland, owners of electric storage systems pay taxes for the charging electricity. This not only does not motivate a sustainable energy transition but also leads to double taxation, as consumed electricity from storage is equally taxed [27]. In addition, there are some aspects of the ETD that lack clarity and lead to legal uncertainty, e.g., the definition of taxable products and uses that are out of the scope of the ETD.

3-13

Outdated wholesale market mechanisms: A market clearing mechanism should be fair to aggregators, large renewable producers, and conventional producers, encourage flexibility providers, avoid spillage or renewable energy if it reduces consumers’ payment, and does not cause technical issues. The available cost minimization wholesale market structures should be revisited to achieve these targets.

3-14

Separate Power Exchange and Flexibility Market: In the continuous effort to achieve the targets of the energy transition, the variable energy sources are becoming more prevalent. The relevance of co-optimization of energy and ancillary services, e.g., flexibility reserve, pervades the electricity market structures in Europe. In the US, the integration of transmission constraints in energy markets was underpinned by the advent of electricity restructuring and later led to the integration of ancillary services in the market. However, restructuring in most European countries does not co-optimize energy and reserve and other services [37]. A new EU-wide agent called “European Market Coupling Operator” deals with the transmission but not yet with ancillary services. The growing reliance on renewable energy generation and the services provided by the energy communities and citizens provides the reasons for revisiting the role of co-optimization of energy and services. In addition, energy and ancillary markets obey different rules in different member states and are not subject to EU-wide regulations.

3-15

Pressure of traditional market players: Innovative DER- and customer-centric business projects put pressure on conventional market participants, such as centralized generation companies and operators, to change their business plans and models until they finally reach new market equilibrium. Increased self-generation and share of local energy markets can threaten the ability of DSOs to invest in network expansion and maintenance if their income is reduced from network assets. This leads to increased electricity prices and network costs for citizens who do not engage in energy provision. It is also important to note that the local markets also need the distribution grid for delivering locally generated energy to the consumers. Traditional energy market players are also likely to resist the increased share of local markets as they may fear losing their position in the market. Although new opportunities will arise for these important and experienced market players to offer new types of services at the early stages of the energy transition, they may resist it.

Some other challenges mostly stemmed from the complexity of control or the lack of effective management strategy. Such technical challenges are detrimental to upgrading the structure of energy markets. Many challenges in this regard are subject to academic and industrial research.

3-16

Unavailability of network codes and effective standards for switching between grid-connected and island modes: Such switching entails a complex sequence of actions and requires special care about frequency and voltage control, due to the imbalances of generation and loads [27].

3-17

Managing instantaneous active/reactive power balances between upstream and downstream networks: is problematic under various voltage profiles [38]. TSO-DSO coordination needs to be revisited to cope with power and frequency control requirements since a significant extent of the generation in downstream comes from intermittent sources.

Other than the abovementioned challenges and barriers, which hinder the effective and sustainable upgrade in energy market design in almost all European countries, some barriers are no longer a real concern to the high spending/active countries in the energy transition but are still hindering in some other countries.

3-18

Unavailability of Smart meters and lack of standardization on smart metering: Smart meters are the other key components to the operation of and to market flexibility management. Luckily, smart meter rollout is getting momentum in most Member States. The penetration of smart electricity meters has passed the 50% mark in 2020 owing to expanded investments in grid digitalization by utilities in Europe. In 2020, about 150 million smart electricity meters were installed with the bloc recording a 49% penetration rate. However, firstly, such meters have not yet been installed for many other households. On the other hand, there is a need to unify and tighten standardization in metering schemes. Administration of the aspects linked to the data available from such smart meters should be better studied. An example is a need for the analysis of the data that should be availed to citizens to enable them to manage their demand based on the signal of the market price. The need for standardization of the data to be exchanged among the agents, or the plans for taking actions with regards to the access and protection of such data are the issues that should be tackled before causing escalating problems.

3-19

Regulation barriers hindering the effective operation of RESs and ESSs: In some MSs, some other regulatory barriers hinder the development of LEMs. Most of these barriers stem from blocking the effective operation of DERs, RESs, and ESSs that was discussed in the previous subsections. For instance, in some Member States, it is not legal to blend energy generation with storage in the customer premises. In some other states, it has not been viewed in the regulations to feed the citizens’ generated electricity to the grid. These challenges hinder the energy transition and are detrimental to both sustainable adoption of RESs and ESSs and upgrade of market design.

3-20

The regulators often do not permit microgrid islanding: Typically, to avoid resynchronisation issues, voltage stability problems and other challenges related to the safe operation of microgrids, the islanding mode of operation is prohibited for microgrids [30]. To face this, the policymakers and other decision-makers need to push regulatory bodies to accelerate compliance with bi-directionality requirements, at the point of common coupling (PCC), where many technologies should be adopted to assure voltage and frequency stabilities as well as protection coordination. These technologies range from fault current limiters to new methods that have been recently proposed for dynamic stability based on the inverters of RESs. Many required changes in the regulations have been presented in [30].

3-21

Inconsistency of market instruments for incentivizing renewables and the need for further investment in these technologies: Regulations constantly change concerning prosumer feed-in tariffs and the models that decide the level of such tariffs. Such regulations also vary among the MSs. Even though this gives rise to uncertainty of the business model from the perspective of citizens, it is understandable when analyzing the problem from the viewpoints of incentivizing the citizens for the adoption of such technologies and the need for such energy production. What is not rational, however, is that, in some MSs, the feed-in tariffs/premiums are not considered for citizens and energy communities, while the renewable share in their energy markets is way lower than the amount provided in CEP. Except for such inconsistency, in some MSs, there are no customer remuneration schemes for surplus electricity generation. In other cases, it is not possible based on the local regulations to export electricity produced by energy communities to the grid, which keeps these communities away from minimum revenues for market participation. In such situations, eliminating the chance of receiving extra remuneration through such premiums for self-consumption and the unavailability of an effective mechanism to adjust feed-in tariffs/premiums demotivate citizens. In less active countries, the operation of local energy markets entails well-determined and harmonized regulations geared towards permitting citizens to trade surplus electricity with grid operators or other customers [27].

3-22

DSOs regulations motivating investment in wired solutions and conventional production not in demand response and renewable production projects: It was discussed that the economic regulations of DSOs usually lead to their tendency towards employing the products of conventional generation companies since they are remunerated for providing the required assets that make it viable to deliver the power to end-users. As a side effect, such regulations also incentivize infrastructure expansion investments over RESs and demand response. Such legislative frameworks differ considerably across the Member States, and also globally, and will affect the development of efficient local energy markets to make the energy transition possible.

3-23

Long administrative procedures can be an important barrier in getting the rights and incentives to install DERs. Usually, different plants of different sizes are subjected to different authorization requirements and the process can last for a different number of years. Moreover, in some Member States, there are no (or no expediting) regulations for the connection of small-scale renewable generation in rural zones. This is likely to lead to a long administrative process and delay. There should be some mechanisms for obtaining the approvals for starting such a project. An example of such absent regulations is that it is not clear who pays for connecting the small-scale resources to the distribution grid. Another example is the ambiguity around the entities that are responsible for potentially required grid reinforcements [31]. Along with the already unclear policy settings around this subject, such an uncertainty introduces additional risks for shareholders. This accumulated risk negatively affects cost-benefit analyses and reduces the number of potential prosumers in the future of energy systems in the Member States.

4. Barriers Concerning Regulations, Codes, and Standards

The barriers 1-1–3-23 to enablers of citizens’ engagement in the energy transition cover a wide range of scopes and areas. PANTERA tracks back these barriers to their roots in policies, RCSs, social behavior, technical advancements, and/or financial support. Relating the barriers to the gaps and bottlenecks in these sectors makes it easier to draw rational conclusions. In this section, the barriers concerning the RCSs are presented for each enabler. Generally, regulatory, code, and standardization gaps refer to the barriers in the energy system operational routines, or network control and management protocols. Such gaps might exist, for example, when current regulations impede the choice of demand-side resources as an alternative to supply-side resources. Another example is when current regulations allow demand-side resources but create a bias in favor of supply-side resources. An example of such a situation is a building energy code that might prefer renewable energy supply over end-use energy efficiency improvement. Other important points in regulations include planning lead time, integrated planning regulations, access to resources, grid access, ownership models, and local benefit frameworks.

It is crucial to note that the categories of gaps and bottlenecks are not exclusive; in practice, a certain barrier can be related to several sectors. For instance, many regulatory gaps may stem from political decisions and, therefore, might be linked to policy gaps. In this case, the main issue is that the role of demand-side resources has not been considered properly when making the policies nor, subsequently, when designing the regulation. In this fashion, such gaps are also linked to the lack of knowledge or expertise.

This section also traces these barriers in the RCSs in a sample of EU member states with below-average spending on Smart Grid R&I. We highlight examples of the challenge and responses in Bulgaria, Cyprus, Ireland, Italy, and Latvia. Finally, a case study of recommendations for upgrading some of the network codes based on a few trial sites in Ireland is briefly presented to showcase how RCSs need to be modified to cope with the activities around the energy transition.

Based on the barriers, gaps, and challenges related to each enabler, and the discussions provided in Section 3.1, Section 3.2 and Section 3.3,

Table 1. Barriers related to RCSs.

EE, DR, and BEMSs	LEMs
1-2, 1-3, 1-5, 2-5, 2-6, 2-7, 2-9, 2-12	3-2, 3-4, 3-6, 3-7, 3-8, 3-9, 3-13, 3-14, 3-16, 3-17, 3-18, 3-19, 3-20, 3-21, 3-22, 3-23

summarizes the challenges explicitly and implicitly related to RCSs for accomplishing the functionalities of each enabler introduced in this paper.

4.1. Bulgaria

The unbundling process set many challenges in the electrical power system sector in Bulgaria concerning the TSO and DSO interaction regarding the power system stability and control. The high CO₂ emissions pricing leads to gradual lignite plants shutdown, combined with an increasing share of renewable energy DER integration. The lack of clear network codes and regulations on maintaining stability and grid support services that were previously inherently provided by the conventional generation poses some major threats to the power system stability. The transition from TSO coordinated centralised control of large conventional centralised generation (in the past) to a new control and stability support architecture that will deal with converter interfaced DER integration at the DSO level (in the future) requires intensive R&I activities that could further enable adequate codes. Specifically, the lack of regulations concerning participation of the TSO in the DSO MV terminals voltage regulation (using their ULTCs), as well as regulations dealing with TSOs option to request frequency stability supporting services from distributed generators connected in DSO networks, is evidently problematic [39,40].

The present regulation and codes basis in Bulgaria [41] do not provide any technical guides on distribution network interconnection criteria in the case of the presence of distributed generation. The lack of settlement of “fair” allocation and sharing of the maximum admissible voltage rise/drop between DER producers and consumers hinder the renewable energy integration.

On the other hand, the presence of laws and regulations that are forcing the DSOs to connect small generators without considering the physical and technical limitations of the electrical network allows the investors to install at low-cost significant generation in poor rural areas with low demand and weak networks. This corrupts the smart energy concepts that stimulate efficient generation close to the load and implies major and costly network reinforcements that are socially not well accepted in these poor (low income) regions. Additionally, since commonly presented as an “EU initiative”, this further increases the EU skepticism among the citizens and the energy stakeholders.

4.2. Cyprus

The Third Energy Package has not yet been fully implemented in Cyprus. The reason behind this is that only one state-owned company, the Electricity Authority of Cyprus (EAC), generates and supplies electricity within this island country. Therefore, there is no wholesale market and there are no cross-border links at present. Liberalisation of the Cyprus electricity market began under the provisions of the First Electricity Directive and the Second Electricity Directive since 2009, including all “non-domestic” consumers being able to select their supplier according to what is in their best interest [42]. Nevertheless, EAC remains the dominant producer of electricity and the owner of both the electricity transmission and distribution systems in Cyprus. The opening of the electricity market to all customers has been delayed and should be implemented by the year 2022. These market conditions hinder further formulation of innovative LEM structures that would empower citizens’ participation, e.g., through citizen energy communities. The same stands for aggregators schemes and explicit demand response programs that depend upon the participation of consumers and are not viable under no market conditions.

On the other side, EAC has committed to support the integration of RES plants (solar) in the power generation system through schemes for net-metering and self-generation for all consumers. Support scheme “Solar Energy for All” for on-the-site production and consumption of RES for their own use provides:

(a)

the installation of net-metering photovoltaic systems with a capacity of up to 10 KW connected to the grid for all consumers (residential and non-residential). Net metering will be converter to net-billing after 2023, and

(b)

the self-generation systems with capacity up to 10 MW for commercial and industrial consumers.

There was a debate during the public consultation regarding the self-consumption fee, which is something that needs to be examined in more detail, considering the study contacted from JRC, under the administrative arrangement of SRSS/C2017/077 that the existing framework for network charges must change, moving towards a usage-based capacity charging system [43].

Renewables up until now are entitled to dispatch priority. The current call, however, as well as future ones, will require prospective RES generators to operate through the market rules similar to any other generator. This, together with the non-interconnection status of Cyprus, highlights the importance of the storage installations for the island case of Cyprus. At the time being, no regulations exist regarding storage. It should be mentioned, however, that behind-the-meter storage could be profitable for end-consumers under a net-billing plan and in case time-of-use electricity tariffs are adopted in the future.

To sum up, the main barriers for citizens’ empowerment are related to the structure and the size of the electricity market in Cyprus, which is small with small volumes and, thus, the absence of competition.

4.3. Ireland

In 2018, the Irish Government approved the high-level design of the Renewable Electricity Support Scheme (RESS), including a community energy provision [44]. The design aims to facilitate energy communities, which are viewed as a key function of the recast Renewable Energy Directive of the EU Clean Energy Package and are a component of the Programme for Government and the Climate Action Plan 2021. Ireland’s RES-E target is for at least 70% renewable electricity by 2030.

The RESS provides pathways and supports for communities to participate in renewable energy projects. The community enabling framework in the RESS aims to provide end-to-end support to create a community energy sector in Ireland. Community-led projects can apply for RESS if they meet certain criteria. The project size must be between 0.5 and 5 MW. The project must be fully owned by a renewable energy community (REC) whose primary purpose is community benefit (environmental, economic or social) rather than financial profit. There are several legal forms the REC can take; however, the crucial characteristic is that the REC must be based on open and voluntary participation of natural persons based on the local domicile (within close proximity to the RESS project).

Support under RESS is allocated by way of auctions. RESS auctions are delivered by the Department of Environment, Climate and Communications (DECC) with the support of the Commission for the Regulation of Utilities (CRU) and EirGrid, the Transmission System Operator (TSO). RESS uses a competitive auction process to determine which generators receive support. For projects that are successful in the RESS 1 Auction, this support typically applies for approximately 15 years. Seven community projects were successful in RESS 1, the first such auction in 2020. 3.4 MW of onshore wind, and 20.95 MW of solar. [45].

In addition to technical requirements, RESS projects must establish a community benefit fund to support the wider economic, environmental, social, and cultural well-being of the local community where the project is located. The contribution is €2 per Megawatt hour of generation of the RESS Project. The objective is to benefit the local community and incentivise investment in local renewable energy, energy efficiency measures and climate action initiatives in the locality.

4.4. Italy

Italy has, in recent years, been facing a decreasing pace of deployment of new renewable power plants. Accounting for 56 GW of RES capacity in total, in the last year (2020) the new renewable power installed was 784 MW, approximately 35% less than the installed RES during the same interval of 2019. This has been driven especially by a substantial decrease in new wind power plants. The COVID-19 pandemic certainly had a key role in this significantly slowing down of RES deployment; however, it isn’t the only cause. Moreover, the years before 2019 were characterized by a decelerating deployment of new RES. Criticalities can be found also in the administrative and authorization process that new power plants need to undertake [46,47]. The complexity of the process, depending on the type and size of the power plants, involves several steps, and authorization from different entities can take very long times (a different number of years in some cases). This leads to a situation where the number and the amount of capacity waiting for authorization is huge, but the actual concession is a much smaller fraction. The completion of the authorization process is a prerequisite needed to access the auctions set up to grant incentives to renewable plants above a certain size. Therefore, the difficulties in a timely completion of the authorization process could constitute a barrier in getting the incentives established by the government. The last auctions organised failed in granting all the available budget of incentives. Finally, it is important to note that, in some regions, the possibility to occupy land is severely limited. Soil protection is certainly to be recognised as an important factor in the development of a country; however, the importance of discussing soil protection in relation to the need to decarbonise the energy system needs to be recognized. Besides pointing out these barriers, it should also be noted that, recently, the Italian government introduced some changes in the process to retrofit existing power plants that have sped up fostering the renewal of the old (especially wind) power plants.

Besides what has been pointed out so far regarding the deployment of RSE power plants, it is important to briefly mention that, as pointed out in the other sections of this article, the process of customers’ engagement in providing flexibility services is still far from being accomplished. Even if, in Italy, the smart meter rollout process has been very successful, this is not the only enabler needed to make customers active. In this view, the second generation of smart meters that, besides others, introduces relevant novelties in terms of data exchange between user devices is now under deployment, thus, making available (to the user) pointy consumption and generation data that could be exploited in different and emerging commercial services. Besides, it is important to note that they are under development standards and architectural solutions to enhance the user participation and provision of services to the grid. It is under discussion, within the relevant national technical committees, how the aggregators should interact with user devices and the technical rules for the home private electric vehicle charging points. However, this is not to be seen as a barrier, but as an opportunity for the future, since these are new solutions that need to be discussed by the relevant bodies before a real market uptake can happen. It is, however, important to point out that, due to the complexity of the field, several committees are involved in the discussion; a lack of coordination should be avoided, otherwise the standardisation process slows down, resulting in a relevant barrier, as already highlighted.

4.5. Latvia

Despite the high share of electricity produced from RES, stagnation can be observed in Latvia’s local electricity production from renewable energy sources. Energy produced by solar panels and wind generators makes up only 2% of the amount of electricity produced by RES [48]. This stagnation is also reflected in ensuring energy prosperity, especially in the prevention of energy poverty (Latvia has the 5th highest level of energy poverty in the EU) [49].

Research shows that one of the key solutions to prevent high energy poverty levels would be to promote local generation [50], offering consumers the possibility to become energy producers (prosumers) or to provide non-discriminatory participation opportunities to participate in the energy community. In addition, consumer participation in local energy production would motivate them to significantly change their usual electricity consumption, thus, promoting energy efficiency and development of new services, such as demand response activities [51].

Conducting several studies and a broad discussion on the promotion of energy communities in Latvia has made it possible to plan a number of amendments to the Energy Law.

Thereby, the main barrier to the further development of local generation in Latvia is the lack of relevant legislation. Respectively, a regulatory law on the establishment of energy communities, rules for participation and responsibilities has not yet been developed [52]. This is also indicated by Latvia’s National Energy and Climate Plan for 2021–2030 [53]: there is a lack of incentives and regulations that do encourage RES initiatives, thus, increasing the cost and payback period for energy community and prosumer based projects. In other words, appropriate further steps will also address the problem of energy poverty, significantly improve energy efficiency and demand response activities to a wider range of stakeholders.

4.6. Summary of the Studies on the Sample Below-Average Spending MSs

The study on these sample MSs revealed that, along with the abovementioned barriers and gaps, some other important challenges should receive much more attention to stay on track to achieve the main objectives of the energy transition. Some of the below-average spending MSs, such as Italy, have ambitious targets for renewable energy generation; however, there are still some bottlenecks limiting the deployment of RESs (such as a slow and complex authorization process). Besides, the process of customers engagement in providing flexibility services still needs to be further developed. Smart meters rollout has been very successful; however, this is not the only enabler for smart customer engagement. In this view, solutions such as a common architecture for providing services to the grid and standardized data exchange protocols between devices need to be deployed.

For some other below-average spending MSs such as Cyprus, the main challenges stem from the nature of the grid that is a non-interconnected islanded grid. This makes the electricity market a small non-competitive market with small volumes and low business interest for new actors, such as those presented in Section 3.3. In the absence of these business cases, regulations for DR providers and other actors, such as aggregators, are stalled. On top of that, the absence of storage operation standards that are important for the islanded grid mode prevents RESs from operating through the market rules.

In some below-average spending MSs such as Latvia, the main barrier to the further development of local generation is the lack of relevant legislation. Respectively, a regulatory law on the establishment of energy communities, rules for participation, and responsibilities has not yet been developed. There is a lack of incentives and regulations in MSs such as Latvia. Although this encourages RES initiatives, at the same time, it increases the cost and payback period for CEC projects.

For most below-average spending MSs, the other barriers to the sustainable integration of energy community projects include: No standard on TSO and DSO interaction regarding stability and control. The present RCSs do not provide any guides on distribution network interconnection criteria in the case of the presence of DERs. The lack of settlement of “fair” allocation and sharing of the maximum admissible voltage rise/drop between DER producers and consumers. The presence of laws and regulations which are forcing the DSOs to connect small generators without considering the technical limitations allows the investors to install at low-cost significant generation in poor rural areas with low demand and weak networks.

4.7. Network Connection Code Case Study: RESERVE Project

The PANTERA RCS review outlines the role of the EU processes to develop the RCSs. Starting with primary legislation (treaties and articles), working through secondary legislation (decisions, regulations, and directives), through guidelines and network codes. The principle of subsidiary applies; national and regional guidelines and codes are developed to suit local needs.

Maintaining grid stability (especially, voltage and frequency stability), is of growing importance for grid operators with the increased penetration of RESs and inverter-interfaced DERs. Hence, the grid should offer new capabilities, e.g., inertia improvement, over-frequency generation curtailment strategy, the rate of change of frequency mitigation and frequency response to sudden disturbances. RESERVE, a European project, explored new energy system support services to enable distributed and multi-level control of the energy system. The RESERVE project [8] published models and mechanisms for implementing system support services. The RESERVE project revisited the network codes related to active voltage control (VC), stability VC, and ICT requirements for the successful integration of inverter-interfaced and other types of DERs. Among the project deliverables 3.8 [54], and 3.9 [55] proposed updated ancillary services and network code definitions for integrating many RESs into low voltage distribution systems. Network codes and ancillary services definitions address the dynamic and static active voltage management of RESs and DERs, power factor requirements, and the controllable operating range of a wide variety of DER technologies, such as photovoltaic systems, vehicle to grid and dispatchable generation systems. Based on the results of the simulations and field trials, proposed amendments to the existing network codes were presented in the deliverables of the project.

In terms of VC requirements in active distribution systems, the network codes concerning distributed/decentralised VC, behaviour and constraints of different DER technologies and leading power factor were re-evaluated for the RESERVE project. From the perspective of dynamic voltage stability monitoring, the RESERVE project proposed some amendments associated with dynamic stability margins, requirements of new behaviour of DER inverters, and new requirements for the perturbations injected from RES inverters to best identify the network. The results of implementing the proposed network codes and VC approaches in many trial sites based in Ireland were presented in [8] to demonstrate the effectiveness and viability of the proposals.

To make the results of implementing the network codes recommended by the RESERVE consortium much clearer, an example is borrowed here from the deliverables of this project. More information in this regard can be found in [8]. As a result of applying the static VC algorithm proposed in the RESERVE project (which was referred to as the active voltage management (AVM) algorithm in the deliverables of the project) instead of the 0.95 lagging/leading power flow limitation enforced in Irish network codes, a considerable loss reduction and voltage profile improvement were achieved in the trial sites of the project. As an example, the RESERVE VC solar PV array trial site is located at ESB Network’s National Training Centre in Portlaoise, Ireland. It comprises a 7.2 kW Solar PV Array connected via two separate single-phase inverters. The evaluation of the effect of the VC algorithm required the comparison of network performance in two operating scenarios, namely with and without the VC algorithm. The comparison of power injected by the upstream secondary substation is presented in Figure 6. As can be seen, the introduction of the VC algorithm successfully reduced the active power demand at the point of connection. This reduction is mostly attributable to reductions in energy loss due to the optimisation of the inverters’ reactive power injections. The results obtained for this trial site along with those obtained in the other trial sites of the project [8] indicate the necessity of updating the RCSs in the modern structures of electric energy systems.

5. Conclusions

The success of the energy transition in Europe depends on the sustainable replacement of conventional generation with renewable production. The variability of renewable energy indicates the necessity of the integration of energy citizens, as the new sources of flexibility, into the energy systems. This paper focuses on the barriers to the successful and sustainable engagement of European citizens in the energy transition that root in the European and national RCSs and hinders the effective realization of efficiency measures, DR, and EMS, as well as efficient LEMs in the member states as the enablers of citizens’ engagement. Dealing with the gaps in RCSs pave the way for the effective engagement of energy citizens in the energy transition by assuring service quality through developing effective standards.

The barriers with regards to the other enablers and roots have been left to be analysed in future studies. Many of the barriers that mattered in the past have been removed and new gaps are emerging as the energy transition progresses. This study identified that the low quality of renovation (mostly limited to cosmetic fixes), institutional and legal frameworks that slow down renovation, and the lack of building class-oriented standards delineating the minimum level of renovation, hinder putting the energy efficiency measures into action. The implementation of domestic DR is hindered by insufficient wholesale price variation, energy and a network tariff structure that does not support demand shift in time and switching to e-mobility and electrical heating, DSO remuneration approach that incentivises non-wire solutions over DR, ambiguous rights for direct control of citizen’s loads, and regulation interaction barriers. Finally, the complexity of prosumers’ remuneration, data confidentiality/transparency, technical responsibilities for aggregators and fairness in allocating such responsibilities, and recognition of user characteristics for market-oriented DR comprise a subset of the barriers to the efficient integration of citizens into LEMs. In addition, the outdated wholesale market mechanisms, separate power exchange and flexibility market, technical problems, lack of standardization on smart metering, inconsistency of market instruments for incentivizing renewables, DSOs regulations motivating investment in only wired solutions, and long administrative procedures for energy community projects are also hindering the efficient implementation of LEMs.