1. Introduction
Demand response (DR) is designed to change electricity consumption patterns, which includes shifting electricity load from on-peak to off-peak periods, shifting electricity consumption to when renewable energy is abundant, reducing demand when the system reliability is jeopardized, or responding to dynamic price signals. DR can be applied to (1) reduce greenhouse gas (GHG) emissions by integrating renewable energy [
1,
2], (2) provide ancillary services such as frequency control and operating reserves [
3,
4], and (3) balance power generation and demand in an electricity market [
2,
5,
6]. DR is considered as the most economical approach to these applications and has gained increasing attention from both academics and industry. To support DR applications, the U.S. Department of Energy lists three types of price-based DR options: time of use (TOU), real-time pricing (RTP), and critical peak pricing (CPP) [
7]. Other mechanisms include inclining block rate (IBR), direct load control (DLC), demand bidding, and capacity programs in the electricity market. A more recent review of state-of-the-art approaches to implement DR programs in a smart grid environment can be found in [
8].
Compared to DR implementation in the commercial/industry sector, residential DR implementation is more challenging because: (1) individual residential load is small in scale; (2) residential electricity consumption is random in nature; and (3) consumers’ acceptance of inconvenience from shifting loads and/or privacy issues must be considered.
Multi-agent systems (MAS), optimization models, and game theories are useful and suitable tools to tackle these challenges. In general, a MAS is defined as a group of autonomous agents that interact, cooperate, and negotiate with each other to reach both the design objectives of individual agents as well as the global objectives [
9,
10,
11]. The IEEE Power Engineering Society’s Multi-Agent Systems Working Group has identified two novel approaches of using MAS in power engineering applications, namely: (1) building flexible and extensible systems; (2) simulation and modeling [
12].
A number of examples of agent-based modeling and simulation for residential DR can be found in the literature. The automated DR problem under RTP was studied in a MAS, and a novel control mechanism was proposed to cope with disconnected communication among agents [
13]. A MAS-based multistep and multilevel optimization algorithm was developed to incorporate DR in a community energy management system (EMS) [
14]. However, the benefits of DR for householders were not evaluated in [
13,
14]. For example, the work in [
14] focuses on reducing system operational cost. The work in [
15] develops an agent-based multi-layered hierarchical control scheme for residential DR under RTP. Optimal stopping rule theory was applied to tackle price uncertainty. An hour-ahead DR algorithm for a home EMS was developed, in which the artificial neural network (ANN) technique was used to predict the electricity prices and multi-agent reinforcement learning was adopted to control household loads [
16]. Potential benefits from individual appliances were evaluated. However, DR benefits at a utility level were not studied in [
15,
16]. A load forecasting algorithm based on ANN was developed and further incorporated into a MAS model for virtual power plants in [
17]. Agent-based simulation platforms have been developed. The work in [
18] presents a smart grid co-simulation software platform—SimApi. It is an open source smart grid software infrastructure that provides a benchmark for building control algorithms. A Multi-Agent Smart Grid Simulation Platform (MASGriP) is presented in [
19].
A microgrid consisting of 1000 residential homes and various generators was simulated in a MAS [
20]. The flexible loads were electric heating appliances. The impact on GHG emissions and generation dispatch requirements were studied by incorporating wind power through DR. Energy management strategies to effectively reveal vehicle-to-grid (V2G) potential in grid-connected microgrids were proposed and evaluated in [
21]. This work focuses on managing electric vehicle (EV) charging/discharging and different strategies were suggested based on the accuracy of electricity price forecasts. A hierarchical MAS was developed to incorporate air conditioning (AC) loads to the day-ahead electricity market under DR [
22]. However, the studies in [
20,
21,
22] only consider one particular type of appliance as the controllable load. In our proposed MAS, we have modelled a variety of household loads, and an optimal control model under DR is incorporated in the MAS. We evaluated the DR benefits for both homes and utilities.
Various deterministic and stochastic optimization models can be seen in the literature. A mixed-integer linear programming (MILP) model was developed to study V2G technology and energy storage systems (ESS) responding to the combination of a RTP with a peak power limiting-based strategy, and a 65% reduction in the cost of electrical energy was reported [
23]. A bi-level optimization model was developed in a MAS framework in response to dynamic prices to flatten the load profile [
24]. A DLC strategy was evaluated in [
25]. An incentive-based strategy was investigated to find the trade-off between rewards and comfort levels [
26]. A decentralized DR framework was proposed for DR programs considering generation cost, the consumers’ discomfort cost, and transmission constraints [
27]. A transactive DR model was proposed with high EV penetration levels [
28].
An important aspect of residential loads is human intervention in load control. Additionally, renewable energy availability is stochastic in nature. Use of stochastic optimization approaches such as stochastic programming, robust optimization, and stochastic game theory have become a new trend to investigate influences of this unpredictability on residential DR applications. The impacts from human behaviors on DR have been investigated in [
29]. Two-stage stochastic and three-stage programming models were developed for applications in the electricity market [
2,
5,
6,
30]. A distributed random access framework was developed to mitigate bus congestion and voltage drops [
31]. Since individual residential load is small/trivial in DR applications, a load aggregator was introduced between utility operators and end-users in [
32]. Multi-layer optimization models were developed and solved through the intermediation by the aggregator. An incentive-based residential DR program was proposed in [
33] to aggregate the loads to meet a pre-defined load profile. A load aggregator was also introduced in a day-head market for various objectives [
34,
35]. Game theories were applied in demand response applications [
36,
37,
38]. A stochastic game approach was proposed to study the interactions among service providers, EV charging stations, and EV owners, and a Nash equilibrium provides a RTP scheme [
39].
In this study, we develop a MAS framework. We also develop a household load forecast model, an electricity price forecast model, and an optimization model. The optimization model incorporates the human impact on electricity consumption. These models are further incorporated into the proposed MAS. The software agents collectively optimize their own profits; meanwhile, the global optimal solution is achieved. This study is based on our earlier work [
40] but with significant enhancement. The difference and improvements are listed as follows.
We developed a co-operative MAS, in which the home agents (HAs) collectively optimize their objectives. The global optimal solution is achieved through integration among the HAs.
The HAs optimize the objective based on global information through the interaction among HAs.
Game theory is used to prove the existing and uniqueness of the optimal solution.
The impact of EV penetration levels and charging strategies are evaluated.
The main contributions of this paper are summarized as follows.
A MAS with incorporated load and price forecast models and an optimization model is developed. The proposed system is linearly scalable and can be implemented in parallel into large-scale applications.
The prediction model forecasts household load including EV charging as a benchmark for heterogeneous homes. Each home has a unique load profile. The benefits of DR applications and the effectiveness of the proposed approaches are clearly demonstrated.
The individual privacy load information is not exposed for any homes because the exchanged load profile is the sum of all the other homes.
The proposed multi-agent optimization approach can significantly reduce electricity payment/cost and improve the energy efficiency.
The rest of this paper is organized as follows. In
Section 2, we describe the multi-agent system.
Section 3 discusses the mathematical formulations and models. Case studies are shown in
Section 4. A discussion of this work is presented in
Section 5 followed by conclusions and future work.
3. Mathematical Formulation
In this section, we illustrate the mathematical models, namely, the load forecast model, the real-time pricing model, and the optimization model. The load forecast model and the real-time pricing model have been developed in our earlier work [
40]. However, for the notion purpose, they are concisely introduced. We then focus on describing the optimization model.
3.1. Load Forecast Model
The load forecast model is defined as follows.
where
represents the electricity load of an appliance
at a time
.
is defined as the rated power.
is the initial operating time and
is the operating period. The standby power is assumed to be zero. Detailed mechanisms of how to obtain the parameters of
,
, and
can be found in our earlier publication [
40].
The predicted load and energy consumption are defined as follows.
where
is the column vector of the predicted load profile.
and
is the predicted electrical energy of un-schedulable appliances and schedulable appliances. Note that
and
are column vectors (e.g.,
).
3.2. Real-Time Pricing Model
Multiple types of power generators, such as thermal, hydro, natural gas generators, etc., are required to meet the demand. These generators are economically dispatched based on their marginal generation cost, i.e., the generation cost increases with demand since cheapest generations must be used first followed by more expensive ones. In this study, we define the electricity price as piecewise linear functions of power (kW) as indicated as follows.
where
,
,
and
are constant parameters. The increase of electricity price based on load is accelerated once it exceeds the threshold
. This is indicated by
. The parameters are determined by linear regression using the data from PJM [
41].
In our study, we focus on a microgrid of 100 homes and assume the existence of a local electricity market. It is noted that the electricity price or locational marginal price (LMP) relatively depends on the load profile, rather than being based on the absolute magnitude of the power consumption. As can be seen from PJM data, the load consumption in the American Electric Power (AEP) zone is about ten times more than the Duquesne Lighting Company (DUQ) zone, but the LMP in two zones are similar [
41].
3.3. Optimization Model
HAs reschedule the load based on the proposed optimization model to minimize electrical energy cost as well as discomfort from rescheduling the load. We define this optimization problem as a convex program (CP) model.
The optimization model has a combined two objectives. The first objective is to minimize the electrical energy cost shown by the first term in Equation (11).
is the load of the HA and
is the aggregated load profile of the other HAs denoted as
. The aggregated load profile is obtained through information exchange. The electricity price
is defined in Equation (10). The HA predicts the electricity price based on the global load information. The second objective shown by the second term in Equation (11) is the consideration of discomfort of the rescheduling of the load.
is a scaler to reflect the magnitude of discomfort to reschedule load
.
is delayed time due to the load reschedule defined in Equation (16).
represents the amount of rescheduled load. The second term
performs as a penalty function to balance the two objectives.
Equations (12)–(15) show the constraints of the model. Equations (12) and (13) show that the appliance should operate in the period of . is the deadline for the operation to be completed. In this period, the appliance may run at any power between 0 kW and the rated power . However, since in the objective function is a monotonically increasing function, the optimal solution will lead to a sparse vector. In other words, the decision variables will be either 0 or . Equation (14) shows that the energy consumption will not change after the rescheduling of usage. Equation (15) is an operational constraint.
The HAs solve the optimization model locally but with exchanged information in order to achieve a global optimal solution. We have developed a heuristic algorithm (shown in Algorithm 1) to accomplish this task. The HAs schedule their load by solving the optimization model in a random order. Note that in the first round, the first HA only has local information. However, the HAs get more and more information from other HAs. They have the opportunity to reschedule the electricity consumption. This progress will continue till no HA reschedules its loads.
Algorithm 1. The algorithm for the multi-agent optimization model. |
Round repeat |
for in a random order |
1: Input: The forecasted local load profile and the exchanged information |
2: Solving the proposed optimization model |
3: Determine the electrical energy cost |
if this is the first round |
Store the electrical energy cost for the HA |
else |
if the cost < earlier cost |
Update the load profile for the HA |
Broadcast the load profile |
else |
Keep the load profile from the last round |
4: Output: Optimal solution |
Next round till no HA is willing to reschedule the schedulable loads |
Furthermore, it is now shown that the solution is unique for any particular order of HAs in the scheduling process.
Proof. The proposed model be formulated as a finite game as follows.
Players: The , where represents a set of HAs.
Strategies: Each minimizes the objective function in the proposed optimization model. In other words, each HA determines its load profile to maximize its payoff.
Payoffs: The payoffs can be seen as the maximum utility denoted as negative of the objective function:
Existence: Theorem (Nash, 1951). Every game with a finite number of players and strategy profiles has at least one Nash equilibrium [
42] (p. 71). In this case, no players/HAs are willing to reschedule their schedulable loads.
Unique: Since the objective function is strictly convex, the
is strictly concave. Therefore, the Nash equilibrium is unique [
43]. □
Therefore, the proposed algorithm can be converged, and the solution is unique for any particular order of HAs in the scheduling process.
5. Discussions
The power system prefers a load profile with low variations, i.e., low values for PAPR and standard deviation. In
Table 4, it can be seen that lower values for PAPR, standard deviation, and electrical energy cost are achieved using the proposed approach.
Table 5 shows the electrical energy cost for the homes individually. Because EV charging requires significant energy, the homes with an EV have much higher cost than those without EVs. However, all the homes enjoy a significant drop of the electrical energy cost by using the proposed approach. Specifically, the homes in the EV group have an average cost reduction of 18.8%. The cost reduction for the homes with no EVs was even more pronounced, with an average reduction of 20.2%. The maximum cost reduction is 24.5%. This clearly demonstrates the effectiveness of the proposed mechanism.
In this study, we focus on demand-side management in terms of improving power system stability and energy efficiency. However, this work can be readily incorporated into renewable energy sources (RES), e.g., solar and wind power. We could develop a roof-top solar power prediction model and consider it as a household appliance with negative load consumption. The utility level of RES will change the landscape of smart grids, which could be captured by a RES agent and included into the MAS. New objectives, e.g., to minimize CO2 emission can be incorporated in the CP model.