Quantification of Demand-Supply Balancing Capacity among Prosumers and Consumers: Community Self-Sufficiency Assessment for Energy Trading

Abstract

With the increased adoption of distributed energy resources (DERs) and renewables, such as solar panels at the building level, consumers turn into prosumers with generation capability to supply their on-site demand. The temporal complementarity between supply and demand at the building level provides opportunities for energy exchange between prosumers and consumers towards community-level self-sufficiency. Investigating different aspects of community-level energy exchange in cyber and physical layers has received attention in recent years with the increase in renewables adoption. In this study, we have presented an in-depth investigation into the impact of energy exchange through the quantification of temporal energy deficit–surplus complementarity and its associated self-sufficiency capacities by considering the impact of variations in community infrastructure configurations, variations in household energy use patterns, and the potential for user adaptation for load flexibility. To this end, we have adopted a data-driven simulation using real-world data from a case-study neighborhood consisting of ~250 residential buildings in Austin, TX with a mix of prosumers and consumers and detailed data on decentralized DERs. By accounting for the uncertainties in energy consumption patterns across households, different levels of PV and energy storage integration, and different modalities of user adaptation, various scenarios of operations were simulated. The analysis showed that with PV integration of more than 75%, energy exchange could result in self-sufficiency for the entire community during peak generation hours from 11 a.m. to 3 p.m. However, there are limited opportunities for energy exchange during later times with PV-standalone systems. As a potential solution, leveraging building-level storage or user adaptation for load shedding/shifting during the 2-h low-generation timeframe (i.e., 5–7 p.m.) was shown to increase community self-sufficiency during generation hours by 17% and 5–10%, respectively, to 83% and 71–76%.

1. Introduction

Managing distributed energy resources (DERs) in microgrids draws on opportunities that are provided by elements such as communication technologies, metering devices, controllable loads, renewable energy sources (e.g., solar panels), storage systems, and human–building interaction for the improved operation of a power system. These elements have paved the way for decentralized energy distribution and reduce the drawbacks of relying on centralized systems [1]. The integration of small-scale distributed energy resources at the household level has increased over recent years, enabling neighborhoods to operate as microgrids. Consumers who adopt renewables such as solar panels become prosumers, which can supply their own energy demand. Given the imbalanced and intermittent nature of renewable energy generation and the uncertainty in consumer behavior, efficient management of the prosumers’ energy surplus is an important factor in renewable integration. The surplus energy management could take different forms including supplying surplus to a central grid, using energy storage systems, load shifting, or energy exchange. As such, Peer-to-Peer (P2P) energy exchange is the next-generation approach for supply and demand balancing, in which the collective participation of prosumers and consumers at the community level could contribute to the success of DERs’ integration and decentralized distribution [2]. Different factors in cyber, physical, and behavioral layers could play a role in driving self-sufficiency (the ratio of self-consumed generated energy to the total demand) of the community through energy exchange. Therefore, physical and behavioral dynamics could affect the temporal balance between generation and consumption in the form of surplus–deficit complementarity.

Achieving self-consumption is desired as reflected in policies such as decreased feed-in-tariff rates [3,4,5]. As a general tendency, the match of on-site generation and demand is inherently promoted [6]. In community-level local energy exchange, the goal is to cover the demand of local consumers using the surplus from prosumers and small-scale DERs. However, balancing the surplus–deficit could be affected by the inherent temporal mismatch between solar generation and demands that are driven by uncertain and volatile energy consumption and generation patterns observed across different households [7,8,9,10]. The research efforts on community-level energy exchange have investigated different dimensions including energy-flow optimization [11], the impact of network constraints [12], and its financial aspects [13]. However, in previous studies, little research has been conducted on the impact of human factors in driving complementarity capacity. In this context, human factors are represented by realistic and volatile demand-side energy profiles, as well as load flexibility from user-adaptive behaviors that affect self-sufficiency capacities.

In this study, we have investigated how these human factors could affect the temporal complementarity under the constraints of infrastructure configurations, specifically the penetration level of small-scale DER assets—i.e., solar panels and storage systems at the prosumer level. To this end, we have examined how the reshaping of energy profiles through load flexibility/user adaptation impacts load-balancing capacities. Furthermore, we have investigated how diverse and realistic household load profiles (representing energy use behaviors) affect the temporal complementarity for self-sufficiency in communities with different mixes of prosumers and consumers. These investigations have been guided by defining varying levels of solar panels and energy storage penetration as distributed DERs to provide an insight into the infrastructure impact. In answering these questions, we have adopted a data-driven quantitative analysis of the load-balancing capacities among prosumers and consumers at a neighborhood scale. To account for realistic profiles of consumption and generation, data from a community of ~250 households in Austin, TX—i.e., the Pecan Street project community and the Dataport dataset—has been used as a case-study neighborhood to create simulated communities of different scales using statistical sampling. The impact of the study variables has been assessed by measuring the surplus–deficit complementarity and community self-sufficiency.

The contribution of this study includes: (1) Quantifying the temporal deficit–surplus balancing capacity for community self-sufficiency assessment between prosumers and consumers considering the variation of demand and supply using realistic load profiles, as well as varied infrastructure configurations, and (2) studying community self-sufficiency by considering human adaptation for load flexibility. In turn, the findings could help infrastructure planners evaluate the required share of DERs and the impact of consumer-focused programs in future networks that rely on energy trading for decentralized energy management.

2. Research Background

The energy exchange paradigm enables prosumers to have better control over energy trading and contribute to a sustainable energy market [2,14]. Energy exchange increases the utilization of renewable assets in a decentralized way and reduces power loss due to shorter transmission distances [15]. In recent years, several trials and real-world implementations have adopted energy exchange prototypes through P2P trading (see [16,17] for a comprehensive list of projects). Several recent studies have investigated the economic aspects of energy exchange through coordinating load balancing across households [18,19,20]. For instance, studies have focused on approaches that help participating households improve their electricity cost [19]. Furthermore, maximizing profit based on the energy consumption level of prosumers and consumers has been investigated [20].

In one direction of the research efforts, the impact of user preferences was studied to improve users’ willingness in contributing to the community-level load balancing [21,22]. For example, in one study [21], an online platform was designed in which households could engage in energy exchange and trading based on pricing incentives and households’ energy storage rate. It was shown that households could be classified as participants desiring economic benefits and participants desiring individual independence and state of storage. Their investigation of more than 300 households showed that 77% of the community was willing to participate in energy exchange.

In another class of studies, the impact of energy storage using batteries on load balancing has been investigated [19,23,24]. Zepter et al. [23] have shown that the effect of combining energy exchange with storage may result in a 60% cost reduction on electricity bills. Alam et al. [19] have simulated the microgrid energy and trading values during energy exchange. They have shown that combined PV–battery systems could result in up to 74% cost savings. However, with the increase in PV–battery adoption, the saving in the microgrid declined after a saturating point, since the extra generation could not be used or stored at a given moment. Nguyen et al. [24] estimated the cost-saving for energy exchange and showed that 28% of maximal saving can be achieved with PV–battery systems. However, their findings also showed that the benefit of adopting a battery is associated with the PV panel size, and the benefit is justified if the PV panel size is sufficiently large.

Using data-driven methods, the concept of local demand-supply balancing has been also studied (e.g., [25,26]). The study in [25] suggested the optimal number of PV systems and the capacity of combined heat and power (CHP) to maximize the local demand and supply balancing. In [26], a cooperative community was suggested, in which all the buildings shift the operating times of controllable appliances (dishwasher, washing machine, and dryer) to maximize solar energy utilization. A case study on five buildings showed a 15% increase in the on-site generation that is covered by buildings. In contrast to previous efforts, in this study we have investigated how the temporal load-balancing capacities are affected by human factors including diversity in electricity use patterns and flexibility in operating loads. In doing so, we have accounted for the impact of DERs’ integration levels. Furthermore, we have evaluated these capacities by considering community diversity through statistical sampling.

3. Methodology

Our methodology centered around simulating case-study communities by sampling from a large-scale community of residential buildings with real energy use (demand) and solar generation profiles. Surplus–deficit complementarity and load-balancing capacities were quantified using the data-driven analysis of operations for different community realizations. To form diverse communities, groups of prosumers and consumers were sampled using the bootstrapping technique to account for the uncertainty in daily energy use profiles. Furthermore, in cases that involved load flexibility by considering user adaptation, the impact of load deferral or partial load shedding was simulated for individual households, and daily load profiles were reconstructed through simulation to reflect user adaptation. Differences in infrastructure configurations were reflected in the number of prosumers and consumers with PV panels and storage systems. Upon quantifying the impact of each variable of interest on the response from individual households, we analyzed the aggregate impact of simulated energy exchange across the community. In characterizing the energy exchange capacity of a community, we assumed that all the instantaneous available surplus energy could be used to supply the community deficit. The following sub-sections provide more details on the variables, definitions, and methods of simulation and quantification.

3.1. Basic Definitions

Prosumers (H^P): Electricity consumers with at least one type of renewable source of energy are called prosumers—they can generate energy for their use. In this study, we considered PV panels as the renewable source of energy. A household in the prosumer group is shown as n ∈ H^P.

Consumers (H^C): Consumers are not equipped with local sources of energy and rely on either a central market or their peer prosumers to buy their surplus generated energy. A household in the consumer group is shown as n ∈ H^C.

Surplus energy: Surplus energy is available when PV generation at each time instance exceeds the instantaneous demand for prosumers. Surplus energy could be transferred in a trading market, stored in a battery for later consumption, or fed back to a central aggregator. Given the surplus energy of a prosumer as s_n, n ∈ H^P, the total surplus energy of the community is S^C = ∑s_n.

Deficit energy: Deficit energy is the demand that is supplied either by central aggregators or peers with surplus energy. Both consumers and prosumers could face a deficit during a day. For prosumers, the deficit happens when demand exceeds their generation or available storage. Given the unit deficit energy as d_n, n ∈ {H^C, H^P}, the community deficit is D^C = ∑d_n.

Net demand: The net demand is the power drawn from the central aggregator to cover the community deficit. The net demand for each household, L(t), at each time t is:

$$\mathrm{L}\left(\mathrm{t}\right)=\mathrm{P}\left(\mathrm{t}\right)-\mathrm{G}\left(\mathrm{t}\right)-\mathrm{B}\left(\mathrm{t}\right)$$

(1)

in which P(t) is the power demand, G(t) is the PV generation, and B(t) is the battery power. Here, P(t) > 0, G(t) > 0, and B(t) < 0 while charging and B(t) > 0 while discharging. Also, L(t) < 0 contributes to the accumulation of surplus energy, while L(t) > 0 contributes to the accumulation of deficit energy. The line losses were considered to be negligible and not considered in this equation.

Complementarity factor: The objective of energy exchange is to reduce the aggregate energy deficit by using community energy surplus. We defined the complementarity factor (CF) as the ratio of instantaneous energy deficit that can be covered by prosumers in the community:

$$\mathrm{CF}(\%)=100\ast \left|\frac{{\mathrm{S}}^{\mathrm{C}}}{{\mathrm{D}}^{\mathrm{C}}}\right|$$

(2)

in which S^C and D^C are the community energy surplus and deficit, respectively. S^C is provided by prosumers with a negative net demand, while D^C is needed by consumers and/or prosumers with a positive net demand. A CF = 100% is ideal because it indicates complete independence from a central aggregator and that demand is being met merely through energy exchange. A value of CF > 100% indicates surplus beyond community need, which can be stored or be fed to the central aggregator.

Self-sufficiency: We extend the definition of energy self-sufficiency for individual households [27] to a community to reflect solar energy utilization in the presence of energy exchange. Self-sufficiency is defined as:

$${\mathsf{\phi }}_{\mathrm{ss}}=\frac{{{\displaystyle \int }}_{{\mathrm{t}}_1}^{{\mathrm{t}}_2}{{\displaystyle \sum }}_{\mathrm{n}\in {\mathrm{H}}^{\mathrm{p}}}\mathrm{M}\left(\mathrm{t}\right)\mathrm{dt}}{{{\displaystyle \int }}_{{\mathrm{t}}_1}^{{\mathrm{t}}_2}{{\displaystyle \sum }}_{\mathrm{n}\in \{{\mathrm{H}}^{\mathrm{c}},{\mathrm{H}}^{\mathrm{p}}\}}\mathrm{P}\left(\mathrm{t}\right)\mathrm{dt}}$$

(3)

in which M(t) is the power generation utilized on-site as follows:

$$\mathrm{M}\left(\mathrm{t}\right)=\min \left\{\mathrm{P}\left(\mathrm{t}\right),\mathrm{G}\left(\mathrm{t}\right)+\mathrm{B}\left(\mathrm{t}\right)\right\}$$

(4)

[t₁, t₂] is the timeframe for measuring self-sufficiency. In this study, assuming energy exchange during PV output, [t₁, t₂] is considered to cover all hours of PV generation. The numerator in Equation (3) is the on-site generation by PV or storage saving and is measured for the prosumer set (H^P), and the denominator is the community energy demand that is measured for both prosumer (H^P) and consumer (H^C) sets. Simply put, community self-sufficiency is the share of PV generation that is directly consumed by the community during generation hours.

3.2. Dynamic Energy Use Behavior at the Household Level

Electricity daily load profiles for residential buildings are known to have high variations across households and different days [28,29]. Although certain pre-defined (a limited number) [30] or synthetic load profiles [31] can be employed to evaluate the impact of PV adoption in different capacities, it could limit the generalization of findings by overlooking the stochastic nature of household load profiles. Therefore, to present a reasonable estimation of community load-balancing potential, it is essential to account for such uncertainties and include diverse possibilities in load profiles, driven by occupants’ energy behavior. Here, for simulating energy exchange scenarios, we use the daily profiles of real demand and generation across two months for each household. Bootstrapping sampling was used to form communities by selecting different combinations of households from the case-study community. Therefore, to account for the variation of load profiles, we sampled a different number of households ranging from 20 to 100 to form different communities. Each community realization and its associated experiments were repeated 100 times to account for the stochastic nature of load profiles in quantifying the energy exchange capacities. In calculating energy from a daily load profile P(t), numerical integration in the timeframe of interest [t₁, t₂] was used:

$$\mathrm{E}=\underset{\mathrm{t}1}{\overset{\mathrm{t}2}{{\displaystyle \int }}}\mathrm{P}\left(\mathrm{t}\right)\mathrm{dt}$$

(5)

3.3. Dynamic Energy Use Behavior and Load Profile Change at the Household Level

User adaptation could play a significant role in creating complementarity capacity in communities for the energy exchange. In this context, users can change their load profiles and reschedule load operations to leverage the demand elasticity of flexible loads under incentives such as economic benefits, which in turn could improve the self-sufficiency of the community. The users’ control over energy trading in a local market provides incentives for them to adapt for energy management, specifically if this adaptation is coupled with automation. In Section 4.5, we have presented a discussion on how market design and economic behavior could motivate user participation in energy exchange. In the smart grid context, load flexibility can be achieved by user adaptation through shedding load or shifting it by using smart loads or appliances that rely on automated scheduling or predictive control capabilities. Therefore, flexibility through reshaping demand profiles could result in increased complementarity. Specific loads with high flexibility potentials [8,32,33] include air conditioning systems (ACs), electric vehicles (EVs), and wet appliances (i.e., washing machine, dryer, and dishwasher). AC flexibility can be achieved by leveraging the thermal storage capacity in buildings and changing the temperature setpoint to reduce demand, while in deferrable appliances (EVs and wet appliances) flexibility could be achieved by shifting the operation/charging time.

To account for the challenges of solar energy management during the off-peak generation period, we have investigated load flexibility and profile change in driving complementarity capacity in the timeframes in which solar generation is declining. In doing so, user adaptation and profile change was simulated using appliance-level energy data associated with each daily load profile in the dataset. The load profiles for EV charging (if any) and wet appliances (if any) were represented as hourly profiles. For AC systems, we simulated different strategies for load reduction when solar generation is declining. These strategies reflect different control and user preference configurations. To this end, we adopted an AC load peak reduction model based on the work by Hu et al. [34,35], in which the energy variations from demand response (DR) operations, associated with increasing thermostat setpoint by 1 °C, 1 °C (with pre-cooling), and 2 °C (with pre-cooling), were reflected on the actual AC load profiles during a 2 h timeframe when the solar generation was declining. This DR approach is based on a grey-box RC thermal model that leverages several parameters from building and the environment, including temperature variations, solar gain, thermal mass, and building envelope characteristics. The RC model is characterized through a system identification process that uses different analytical and data-driven optimization techniques for the objective of minimizing the error for indoor temperature based on example temperature measurements. A simplified empirical energy consumption model with an on/off control has been coupled with the thermal model to estimate AC performance [34]. Using this model, Hu et al. [34] showed that AC power reductions of 25%, 31%, and 68% could be achieved during the DR timeframe for an increase in setpoint of 1 °C, 1 °C (with pre-cooling), and 2 °C (with pre-cooling), respectively. We measured the aggregate variation of the complementarity using metrics described in Section 3.1 to account for varying levels of engagement by considering the participation of a subset of prosumers/consumers who were willing to take adaptive actions for flexible operations.

3.4. Battery Modeling

Given the temporal mismatch of generation and demand, storage systems for prosumers will create capacities for self-sufficiency and complementarity. Therefore, we also integrated storage systems in our simulations using the specifications of the commercial Tesla Powerwall [36] as the most widely installed system in the US, with a capacity of 13.5 kW h and 7 kW peak power. Taking into account the average daily solar energy generation per prosumer in our case-study community is around 15 kW h, we assumed one battery per house in our simulations. Considering the goal of capacity quantification, we adopted a battery charging schedule as shown in Figure 1 for each prosumer unit. The batteries are charged at the time of surplus energy according to their physical constraints and are discharged during the deficit. Prosumers’ surplus energy that exceeded the battery capacity or was stored at the battery (available at later times after covering the prosumer’s demand) was used in energy exchange. We have adopted this intuitive model for battery scheduling to show the impact of storage in surplus–deficit complementarity and to offer the extra surplus stored in the battery during peak demand when solar generation is declining. However, more sophisticated battery scheduling schemes under the constraint of dynamic pricing, cycle life, and losses could be used as well [37].

4. Results and Discussion

4.1. Case-Study Community Characteristics

We used the historical energy dataset of 244 residential households located in Austin, TX for our experiments. The dataset is available through the Pecan Street Project [38]. The community included 119 prosumers with PV systems and 125 consumers. Since our case study focused on the PV rooftop solar panels as the renewable source, we used the data for July and August of 2015, as the representative summer months. Similar to the resolution of smart meters, 15-min resolution data for both demand and generation was used. Among the generation time series, less than 0.001% of profiles (8 out of 7152) had missing information and were eliminated from the dataset, resulting in 7144 and 7492 daily profiles for prosumers and consumers, respectively. Each daily profile included 96 data points, which were annotated with a ‘household ID’ and ‘day of the year’ index for information retrieval. Figure 2 shows the daily load profiles of net demand for an example consumer (Figure 2a) and an example prosumer (Figure 2b) for 20 successive days as a case of complementarity potential in the case-study community. Furthermore, Figure 3 shows the characterization of solar generation for all the prosumers in the community—Figure 3a, all solar generation profiles; and Figure 3b, the energy generation distribution from 9 a.m. to 7 p.m., which is the timeframe of our study. The median PV peak power was 3.95 kW (5th and 95th quantile of 1.8 kW, 6.5 kW), with the highest PV peak of 11.0 kW.

To evaluate prosumers’ capacity to offer surplus for energy exchange, Figure 4 compares prosumers and consumers’ energy demand during the generation timeframe. In Figure 4a, the median net energy demand for prosumers is −0.6 kW h (5th, 25th, 75th, and 95th percentile of −18 kW h, −8 kW h, 8 kW h, and 27 kW h). This shows that prosumers in general have the capacity for surplus exchange. In Figure 4b, it is shown that the prosumers’ energy demand (without considering PV generation), is on average higher than that of consumers. The median energy demand is 25 kW h (5th, 25th, 75th, and 95th percentile of 8 kW h, 18 kW h, 33 kW h, and 52 kW h) and 14 kW h (5th, 25th, 75th, and 95th percentile of 3 kW h, 7 kW h, 24 kW h, and 42 kW h) for prosumers and consumers, respectively. The higher energy demand of prosumers could be associated with factors such as household size, plug-in EV ownership, or behavioral tendencies due to the availability of free energy sources. Figure 5 presents the variation of total daily energy consumption across households using two months of daily profiles. In Figure 5a, 53% of prosumers have a daily negative net energy demand (i.e., surplus to offer for exchange). Figure 5b shows net energy for consumers with considerable variations in net demand across households.

According to the U.S. Energy Information Administration, the daily average consumption for houses in Texas is 39 kW h with an average hourly power demand of ~1.6 kW, which is higher than the average demand in the United States (that is, 1.22 kW). The average power demand for the prosumers and consumers in the case-study community during the solar generation hours (9 a.m. to 7 p.m.) were 2.5 kW and 1.4 kW, respectively. In the absence of other representative metadata such as building size, occupancy information, or other socio-economic factors for the state of Texas or the sampled case-study community, the energy efficiency of the community appears to follow the trends in the state of Texas.

Energy use patterns’ variability and uncertainty in the case-study community: Individual households had varied energy use profiles on different days and these variations could affect the reliability of energy exchange. Using K-mean clustering, in Figure 6a,b we have presented the clusters of net energy profiles for prosumers and consumers. The power profiles were normalized based on their daily maximum values as follows:

$$\overline{\mathrm{p}\left(\mathrm{t}\right)}=\frac{\mathrm{p}\left(\mathrm{t}\right)}{\max \left(\mathrm{p}\left(\mathrm{t}\right)\right)}, \mathrm{t}\in \left\{1,\dots ,96\right\}$$

(6)

where p(t) is the power at timestamp t, max(p(t)) is the maximum power observed over a day, and $\overline{\mathrm{p}\left(\mathrm{t}\right)}$ is the normalized power.

As shown in Figure 6, except for cluster two with 20% frequency and no surplus energy, in the prosumer group all clusters showed some surplus energy with different peak levels. Cluster 3 had the highest potential for energy exchange due to its sharp surplus peak during PV generation in addition to its moderately low demand peak when PV generation diminished. On the consumer side (Figure 6b), cluster two was a suitable candidate for energy exchange due to its high demand during the peak of PV generation, while cluster three (with uniform consumption), and clusters one and five (with peak demand around 6:00 p.m.) offered some potential for energy exchange. Cluster four had a peak around 11 p.m. and fit better for storage-based energy exchange.

To quantify the uncertainty associated with consistent patterns of energy use for each household, the entropy of a household, E_n, was calculated as:

$${\mathrm{E}}_{\mathrm{n}}=-{\displaystyle \sum }_{\mathrm{i}=1}^{\mathrm{K}}\mathrm{P}\left({\mathrm{C}}_{\mathrm{i}}\right)\log \left(\mathrm{P}\left({\mathrm{C}}_{\mathrm{i}}\right)\right)$$

(7)

in which P(C_i) is the probability of observing cluster i, and K is the total number of clusters. A low value of E_n indicates the higher stability of households’ load profiles across different days, while a higher E_n denotes low predictability. The lowest value for E_n is zero when all daily load profiles of a household belong to one cluster. Figure 6c,d shows the distribution of entropy versus average daily net energy for prosumers and consumers, and each data point represents one household. The horizontal/vertical dashed lines reflect the 25th and 75th percentile of the values. For each of the nine areas, the case-study community included a variety of households with low to high predictability and energy demand, which further reflects the varied range of households with different energy use behaviors.

4.2. Baseline Surplus–Deficit Temporal Complementarity Quantification

As the baseline case, we investigated the temporal complementarity capacity in communities of different sizes and with varied energy use patterns without considering storage systems or load flexibility.

Equal prosumer/consumer size: We simulated sampled communities of N = {20, 40, 60, 80, 100} households with the same number of prosumers and consumers—N⁄2 for each set. Using the complementarity factor (Equation (2)), we measured the extent of temporal energy balancing capacity at the community level. For each community size and each hour of PV generation, 100 experiments, representing 100 simulated communities, were performed. Figure 7 presents the variation of prosumers’ net energy, consumers’ demand, and the complementarity factor for three successive hours (2–5 p.m.) as examples. This figure shows that increasing the community size resulted in a linear change in net energy and demand with R² > 0.98 for all cases. Therefore, the complementarity factor (CF) for various community sizes remains almost constant with an std < 1.5 for all three subplots. Moving from 2–3 p.m. (the highest PV generation timeframe) to 3–4 p.m. and 4–5 p.m., CF reduced from an average of ~ 50.0% to 31.8% and 15.5%, respectively. This reduction is associated with both the decline of PV generation and the considerable increase in demand from prosumers and consumers at the hours stretching into the evening. As illustrated, for 4–5 p.m., average prosumers’ net energy is positive. Therefore, unlike 2–4 p.m., the aggregate surplus energy offered by a subset of prosumers is not sufficient to cancel out the deficit of other households.

Table 1. Energy surplus, energy deficit, and complementarity factor for various community sizes with equal numbers of prosumers and consumers.

Time of Day	Community Size	Prosumer Net Energy (kW h) *, **	Consumer Net Energy (kW h) *	CF (%) *
9 a.m.–10 a.m.	20	3.9	10.1	17.2
	40	8.6	22.1	14.8
	60	12.0	33.0	15.3
	80	16.4	42.6	15.6
	100	20.0	54.3	15.1
10 a.m.–11 a.m.	20	−2.6	13.0	45.7
	40	−3.8	23.5	43.1
	60	−6.1	36.0	41.5
	80	−9.4	48.4	42.5
	100	−10.1	60.3	41.2
11 a.m.–12 p.m.	20	−6.0	13.7	62.2
	40	−13.6	27.5	63.9
	60	−21.5	41.5	64.6
	80	−28.2	54.8	63.7
	100	−34.1	71.6	59.8
12 p.m.–1 p.m.	20	−10.2	15.3	79.3
	40	−19.6	33.2	69.6
	60	−29.4	47.1	71.7
	80	−39.8	62.6	72.1
	100	−48.9	78.9	69.9
1 p.m.–2 p.m.	20	−8.5	18.8	61.4
	40	−17.3	35.2	61.8
	60	−25.0	55.2	57.8
	80	−32.6	72.1	57.3
	100	−42.3	89.4	58.4
2 p.m.–3 p.m.	20	−6.6	18.9	52.2
	40	−12.6	37.1	49.9
	60	−21.1	58.2	50.2
	80	−27.0	76.4	49.6
	100	−32.7	95.6	48.3
3 p.m.–4 p.m.	20	−1.5	20.2	32.7
	40	−3.8	41.5	31.9
	60	−5.3	63.2	30.8
	80	−7.6	81.2	32.1
	100	−10.0	103.2	31.6
4 p.m.–5 p.m.	20	5.6	20.7	16.2
	40	9.0	44.8	16.1
	60	17.5	64.8	15.0
	80	22.4	86.9	15.2
	100	27.4	109.1	15.2
5 p.m.–6 p.m.	20	14.3	23.3	4.9
	40	29.4	48.1	4.5
	60	42.9	70.7	4.6
	80	57.8	94.9	4.6
	100	71.1	117.2	4.8
6 p.m.–7 p.m.	20	23.8	25.2	0.8
	40	48.0	51.4	0.8
	60	70.0	75.6	0.9
	80	94.1	101.0	0.9
	100	117.9	125.3	0.9
Average				35.6

* Values for each cell in these columns reflect the average of 100 experiments. ** Values for each cell in these columns reflect the aggregation of negative net energy (the numerator of Equation (2)) and positive net energy (prosumers’ deficit contributor in Equation (2)) from individual prosumers.

shows the outcome of simulated load-balancing capacity for each hour of the entire solar energy generation timeframe. The highest CF happens at 12–1 p.m., in which, on average, prosumers’ surplus is 72.5% of the entire community’s demand. However, before 10 a.m. and after 4 p.m., net energy complementarity decreases considerably. For 5–6 p.m. and 6–7 p.m., the CF is less than 5% and 1%, respectively, indicating almost no baseline potential for energy exchange. Apart from these hours, the CF varies between 15.5% (3–4 p.m.) and 62.9% (11–12 p.m.) with a standard deviation of 24%. Therefore, in communities with an equal distribution of prosumers and consumers and without any storage or user adaptation capacities, during the maximum generation hours, ~70% of demand could be covered by community surplus. Energy exchange could be used to maximize community self-consumption and avoid trade-ins.

Varied prosumers’ ratios (PR): Given the linear dependence of load-balancing capacities on the community size (N), we investigated the impact of varied prosumers’ ratios (PRs) for a community of N = 20 with 100 repetitions. Figure 8 presents CF from 2 to 5 p.m. and for PRs (i.e., PV integration ratio) of {25%, 50%, 75%, 100%}. From 2 to 3 p.m., with 75% and 100% PRs, the CF exceeded 100% (109% and 274%, respectively), indicating full complementarity capacities plus excess surplus for storage or trade-in. For 25% and 50% PRs, the median of CF is 19% and 56%, respectively. From 3 to 4 p.m., the CF exceeds 100% (149%) only with 100% PR, while for 25%, 50%, and 75% PRs, 13%, 33%, and 74% of the community demand can be supplied by prosumers. However, from 4 to 5 p.m., the CF only reaches 50% with 100% PR, while lowering PV integration further limits the complementarity in the community (CF of only 7% with 25% PR).

Figure 9 shows a sample distribution of CF from 2 to 3 p.m. for a PR of 0.25—first case in Figure 8a. A two-sample Kolmogorov–Smirnov (KS) test indicated a normal distribution (p-value = 0.89). The KS test for all possible scenarios showed p-values of higher than 0.05 except for one case. Therefore, in presenting the results, we have used the average of the 100 repeated experiments.

Figure 10 compares the average CF values and their 95% confidence interval for all PRs during the hours of PV generation. Due to the high variation across different PRs and hours of the day, the y-axis is shown on a logarithmic scale. For all the scenarios, it was observed that from 10 a.m. to 4 p.m., the aggregate net energy of prosumers was negative (i.e., indicating surplus energy), with a high potential for energy exchange. For 25% and 50% PRs, the highest CF values were 27% and 75% from 12 to 1 p.m. The 75% and 100% PRs are the only conditions that provide capacities for community self-consumption with CF values of more than 100% from around 10 a.m. to 3 p.m., with the highest values of 165% and 420% from 12 to 1 p.m., respectively. Furthermore, from 11 a.m. to 3 p.m., with high PRs, energy storage could provide opportunities for increased complementarity and self-sufficiency after 3 p.m.—on average the remaining surplus was 30% (with 75% PR) and 217% (with 100% PR) of the community demand from 11 a.m. to 3 p.m. However, for PRs lower than 75%, the complementarity of the loads could ideally enable renewables’ self-consumption without the need for storage or supply to the grid. After 4 p.m., the aggregate prosumers’ net energy is positive, which considerably limits the complementarity. Therefore, even with the 100% PR, without energy storage only 12% and 2% of the community deficit could be supplied by PV generation. Figure 10 summarizes the community capacities for temporal self-sufficiency considering different PV penetration ratios. It must be noted that simulations were carried out by integrating real load profiles to report these values. Nonetheless, either the integration of less than 75% PR or expanding the time span of 11 a.m.–3 p.m. to get the same demand/supply matching could be achieved by leveraging demand elasticity for load profile change enabled by different economic or environmental incentives, such as higher revenue or the desire for clean energy use (more discussion have been presented in Section 4.5).

In what follows, we have investigated how energy storage and load flexibility due to user adaptation affect these capacities.

4.3. Energy Storage Integration

To evaluate the impact of energy storage on community load balancing, we considered different levels of battery storage adoption among prosumers in the sample communities of the baseline analyses. For each prosumers’ ratio, different battery integration ratios of {0.25, 0.5, 0.75, and 1} were used in simulating communities with N = 20 for 100 repetitions. For example, for the PR of 0.5 and the battery integration ratio of 0.5, ten prosumers and ten consumers form the community, out of which five prosumers have battery storage. The ultimate goal of quantifying complementarity is to evaluate the potential for improved self-sufficiency at the community level. Therefore, Figure 11 presents the community self-sufficiency for the solar generation timeframe (9 a.m. to 7 p.m.) under different prosumers and battery integration ratios. The battery integration ratio of 0 in Figure 11 represents the cases presented in Figure 10. Maximum improvements of 4.8%, 11.3%, 13.0%, and 17.0% in self-sufficiency are observed with full battery integration for 25%, 50%, 75%, and 100% of PRs, respectively. With 100% PV–battery adoption, the ideal community self-sufficiency reaches almost 83% by enabling the surplus energy stored during the 10 a.m. to 3 p.m. timeframe to be used for self-consumption and energy exchange later in the day.

4.4. User Adaptation and Load Profile Change as Complementarity Capacity

User adaptation through load flexibility and profile change has the potential to improve self-sufficiency, specifically for the timeframe when solar energy generation is declining (as shown in Figure 10). It is known that not all users are willing to adopt adaptive behavior for load flexibility because of different circumstances of convenience and comfort. However, a portion of users will participate in smart load scheduling for load profile change due to economic or environmental incentives. Therefore, we considered different levels of participation as {20%, 40%, 60%, 80%, 100%} for different combinations of adaptation modalities and load profile change {AC 1°, AC 1° + deferrable loads, AC 1°(with pre-cooling), AC 1°(with pre-cooling) + deferrable loads, AC 2°(pre-cooling), AC 2°(pre-cooling) + deferrable loads}. AC a° refers to an adaptation mode, in which the thermostat setpoint is increased (considering our summer season analysis) by a° to shed the load. Pre-cooling enables the energy management system to benefit from the thermal storage capacity of the building to use energy available earlier in the day. On the other hand, deferrable loads (EV and wet appliances) pertain to rescheduling of the appliances that have high demands and could potentially be operated at a different time. Given the low potential of energy exchange during the hours leading to evening as shown in Figure 10, the impact of load flexibility due to user adaptation was considered during this relatively short timeframe of need—i.e., 5–7 p.m.

Figure 12 presents community self-sufficiency for different PV integration levels and different levels of participation and load profile change from prosumers/consumers in taking adaptative actions. The flexibility participation ratio of 0 in Figure 12 represents the cases presented in Figure 10. Each line represents one adaptation modality, and each data point shows the mean value from 100 experiments in communities of 20 households. The user adaptation could provide additional complementarity capacity on both prosumers and consumers’ sides. For 25% PR, a maximum increase of 4.4% (from 22.1% to 26.5%), for 50% PR, an increase of 5.4% (from 39.3% to 44.7%), for 75% PR, an increase of 9.0% (from 53.2% to 62.2%), and for 100% PR, an increase of 10.4% (65.1% to 75.5%) were observed. Regarding the level of participation, regardless of PRs, on average improvements of 2.2%, 4.1%, 5.0%, and 7.5% with 25%, 50%, 75%, and 100% participation over the baseline were observed. User adaptation has the potential to boost self-sufficiency up to about 10% for a PR of 75% and higher. Considering a more conservative adaptation modality of AC 1°(with pre-cooling) + deferrable loads and a 50% participation, the self-sufficiency improvement is about 4 to 5%. This potential could be considerably increased by extending the user adaptation capacities throughout the day and expanding on smart home automation.

User adaptation versus battery storage: To compare the impact of user adaptation versus energy storage, Figure 13 shows the improvement of self-sufficiency by leveraging load flexibility and battery storage compared to the baseline as presented in Figure 10. It should be noted that the self-sufficiency improvement from energy storage stems from surplus throughout the day while user adaptation pertains to adaptation during a limited timeframe. With a low PR of 25%, the impact of user flexibility is comparable with the battery storage, while for high PV integration ratios battery storage shows a higher improvement compared to user flexibility. Nonetheless, with high PV ratios of 75% and 100% in the community, user flexibility could create capacities for self-sufficiency improvements that are 69% and 61% of that of battery integration, respectively. Therefore, in the case of low or no battery integration in a community, user adaptation and its resultant flexibility can be deemed an alternative approach for increasing complementarity and thus self-sufficiency at the community level. The potentials from user adaptation could be realized by using monitoring systems for user–building interactions [39,40], as well as automated demand-side Home Energy Management (HEM) systems that use the trade-off between community self-sufficiency and user convenience. Advanced technologies, such as smart appliances and smart thermostats, which allow for automation while accounting for user preferences, would be an integral part of these smart home eco-systems.

4.5. Market Design and User Behavior Dimensions’ Impact on Self-Sufficiency

Market design impact: In recent years, the economic dimension of market design has been the subject of research from theoretical [41,42] and analytical perspectives [43,44] with recent expansions to real-world implementations for energy exchange [45,46]. In a P2P energy trading market design, considerations are given to individual user preferences and different trading strategies [47,48]. Specifically, the P2P market relies on the auction mechanism [49,50], in which participants could choose their bid resulting in different options for price and quantity of energy to purchase and/or sell. Some approaches in market design include uniform double auction [46,51], combinatorial auction [43], and time discrete discriminatory double auction [45] that enable users to actively participate in pricing and energy allocation mechanisms. Previous field studies and real-world implementations have shown the increased added value of energy trading from an economic perspective [45,46] and the amount of energy traded among peers compared to using utility-set fixed rates [45].

In a field study in Switzerland, in a setting comparable to our case-study communities, a community of prosumers (with PV and PV–battery owners) and consumers participated in a P2P market [45]. The double auction mechanism was adopted as the market design for prosumers and consumers to have the option of purchasing or selling green energy under their preferred conditions. In this market, after the bids were collected, the market was cleared such that the highest purchase order was matched the lowest sell price, and this procedure was continued until the entire bid list was checked, resulting in the price for each exchange being the mean of the purchase/sell bid [45]. The remaining demand was supplied by the central utility provider with their standard tariff. Given this arbitrarily defined pricing market, the prosumers with excessive solar (already supplying their own demand or feeding their battery unit, if any) could set a price lower than purchasing from the utility provider and higher than the feed-in tariff, which would benefit both prosumers and consumers. For example, in the field study of Worner et al. [45], consumers were willing to pay lower rates to their peers for solar energy (0.19 CHF/kW h) compared to the utility price (0.21 CHF/kW h), whereas prosumers were willing to set even lower prices on average (0.13 CHF/kW h), resulting the trading process to be economically beneficial for both parties. On average, the increased revenue for prosumers was 32% while consumers had a 7% cost saving on their bills [45]. Moreover, the authors showed that the variations in price were compatible with the solar energy generation level assumed in our simulations. In other words, the price increased in hours that the solar energy generation was low, and ramped down as we approached high-generation hours. Thus, our simulations could represent the economic dimension of the market dynamics. Therefore, the P2P market could potentially provide incentives through added economic value for both sides in addition to utilizing a clean energy resource.

User-driven demand–supply balancing for DER integration: As presented, we have considered different ratios of infrastructure configurations and showed how they affect the self-sufficiency of the communities, as well as strategies that could be adopted to improve self-sufficiency when the solar energy generation declines moving towards the late afternoon hours. We considered the impact of human adaptive behaviors for demand elasticity to improve self-sufficiency in those hours (Section 4.4). However, the impact of adaptive behavior could be expanded into earlier hours to improve self-sufficiency for communities with less PV penetration. Economic incentives have been traditionally used for shifting energy use behavior so that the cost (both economic and environmental) of energy generation could be minimized. Adaptive behaviors could include load shifting to a different time of the day and load shedding for appliances/devices that could be used more efficiently. For example, AC systems could be operated using human-centered control strategies to reduce energy use. Therefore, users could either reduce their energy use or shift the operation to a different time of the day. Considering the energy source during the high solar generation hours is low-cost green energy, it is preferred that the energy consumption is shifted to those hours. However, there are strategies to reduce energy use even for those hours such as the efficient use of AC systems specifically for house types that could benefit from more refined thermal energy management compared to the conventional single thermostat controller system. As previous research has shown [14], the behavioral incentives for energy trading are categorized into three classes of ‘energy matching,’ which is the willingness to use green local energy; ‘preference satisfaction,’ which is setting preferences for what clean resources to use and what price to set for purchasing or selling energy; and ‘uncertainty reduction,’ which is associated with obtaining economic profit through energy exchange and increasing revenue. Accordingly, the DER integration level (for PV or battery adoption) could be reduced or self-sufficiency values increased by selecting one of these methods, depending on the user’s preferences, values/beliefs, and lifestyle [52].

4.6. Limitations

There are a number of limitations associated with this study: (1) Like any data-driven study, the findings presented here were extracted from a dataset. Therefore, the generalizability of findings is associated with similarity in the energy profiles of prosumers and consumers. Nonetheless, we used the data from the Pecan Street Project [38], which is currently one of the largest campaigns for energy studies, and selected ~250 households from the ERCOT grid. Furthermore, the majority of the analyses presented here include sub-hourly resolution demand and generation data. Demand data was extracted from smart meters, which are a ubiquitous and available metering infrastructure in most regions. Generation data, in the absence of real PV data, can also be estimated from weather and geographical location information with open-source solutions (e.g., PV Watts [53]). Therefore, similar types of analyses can be carried out on other datasets. (2) The findings for demand–supply balancing potential for the community were obtained through the aggregation of energy surplus and deficit of individual consumers, assuming energy trading without including transmission loss, network constraints, and other limiting factors in the economic operation of the market, such as exercise of market power in price-setting strategies [54]. Studying these factors could shed further light on the findings of this work. (3) For the scenarios with battery integration, since we were interested in quantifying the community capacity for energy exchange, we considered flat pricing rates. Therefore, batteries were discharged at the earliest time that demand exceeded generation. In the presence of dynamic pricing that reflects the dynamics of demand, optimization techniques could be explored for maximizing the benefit from energy exchange. (4) We relied on the presence of PV systems that were installed in prosumers’ households, with the same capacity as specified in the dataset. Therefore, the impact of different PV capacities or battery sizing was not taken into account. (5) Different climate conditions could be considered to expand the analyses for air conditioning load flexibility. (6) We presented the results of the case study for one summer season. Therefore, integrating the results over a yearly period could show the impact of seasonality.

5. Conclusions

A comprehensive analysis of surplus–deficit complementarity capacity in communities with different ratios of prosumers and consumers was conducted to quantify the load-balancing capacity of communities through self-consumption and energy exchange. In doing so, we quantified the temporal variation of surplus–deficit complementarity capacity by considering varied infrastructure configurations—i.e., the ratio of prosumers to consumers. Improved complementarity led to increased community-level self-sufficiency, which was quantified by considering the impact of varying levels of energy storage integrations compared to user adaptation for load flexibility. In communities with at least 75% prosumers, without any energy storage capacity, and no economical behavior adaptation for demand elasticity, community surplus–deficit balancing could be achieved through energy exchange during times of high solar generation (from 11 a.m. to 2 p.m.) and additional surplus energy was available for storage. With an equal distribution of prosumers and consumers, ~75% of deficit energy in the community could be canceled out by prosumers at the peak PV generation hours. Therefore, energy exchange could ideally be used to avoid the need for considerable energy storage infrastructure. With 100% PV integration, the community self-sufficiency could be extended to 3 p.m. However, for the timeframe leading to the evening (from 4 p.m. to 7 p.m.), when PV generation diminishes regardless of the PV integration ratio, the community could no longer be self-sufficient through energy exchange. As a result, the impact of alternative measures such as battery storage integration or creating flexibility potential from user adaptation was studied. It was observed that with 100% battery integration or 100% participation of users in adjusting flexible loads during critical timeframes, the self-sufficiency of the community could improve by 17% and 10.4% up to 83% or 76%, respectively. User flexibility was considered to be thermostat adjustment with pre-cooling, as well as rescheduling the operations of deferrable loads for the time that solar generation is declining—i.e., 5–7 p.m. A more moderate case of user adaptation which included a 1° increase in setpoint plus load rescheduling and for 50% user participation resulted in ~5% self-sufficiency improvement. Furthermore, in the best cases, it was observed that in the absence of energy storage systems, user adaptation for load flexibility showed an improvement in self-sufficiency by ~60% of what could be offered by commercial conventional storage systems.

Future directions of this research include: (1) the study of automated smart home technologies that optimize user adaptation under the constraints of comfort and convenience; (2) assessing the impact of seasonality and community diversity; (3) investigating the impact of different DER configurations, network constraints, and control algorithms for the realization of energy exchange using human adaptation; and (4) investigating different economic and social factors that drive and enhance community level demand/supply matching.