**1. Introduction**

As the development of the economy and society continues, energy demand is growing explosively in all walks of life. Consequently, the power grid has to face the serious challenge in balancing energy supply and demand. Especially in peak demand hours, the tense situation of supply and demand happens from time to time, which affects the stability of the grid. In order to relieve the pressure of energy supply and demand, the power grid can promote energy supply ability by building new power plants or reduce the energy demand of consumers. However, peak demand hours only take a tiny proportion in a whole year, and meanwhile, building new power plants needs a lot of manpower and material resources. Therefore, it is uneconomical to build new plants to solve the energy supply problem in such peak hours. For this reason, demand response (DR), which is one of the core technologies in the smart grid, is taking an increasingly important role in digging up demand-side resources and relieving the tension problem of supply and demand [1,2]. Generally, energy consumers include residential users, commercial users, and industrial users. In which, residential users have abundant flexible resources, hence, residential DR can effectively reduce energy demand in peak hours [3,4].

In recent years, there exists abundant research on the energy consumption scheduling or mechanism design of residential DR [5–7]. The authors in [8] proposed a reward mechanism for residential customers to shave peak loads, in which users' consumption characteristics were modeled by survey questionnaires. In order to aggregate a large number of households in the DR project, Mhanna et al. [9] designed a distributed algorithm from the perspective of the DR aggregator, through which households were aggregated and coordinated as a whole and then scheduled based on the objective of the aggregator. Moreover, Reference [10] studied residential DR with consideration of the power distribution network and the associated constraints, and proposed a distributed scheme where the load service entity and the households interactively communicate to compute an optimal demand schedule. However, the above research lacks consideration on the mutual effect of consumers' strategies and does not capture the dynamic property. To answer this issue, different game-theoretic frameworks have been proposed [11–15]. Authors in [16] formulated an energy consumption scheduling program with game theory, where players are residential users and their strategies are the daily schedules of household appliances. Authors in [17] proposed an event-triggered game-theoretic strategy for managing the power grid's demand side, capable of responding to changes in consumer preferences or the price parameters coming from the wholesale market. Reference [18] adopted a dynamic non-cooperative repeated game with Pareto-efficient pure strategies as the decentralized approach to optimize the energy consumption and energy trading amounts for the next day. Reference [19] focused on an hourly billing mechanism for DR management to solve several theoretical and practical questions, including the uniqueness of the consumption profile corresponding to the Nash equilibrium and the computational issue of the equilibrium profile. While in [20], the trading problem was formulated as a bargaining-based cooperative model, where DR aggregators and the generation company collaboratively decide the amounts of energy trade and the associated payments. Authors in [21] formulated a Stackelberg game among the DR aggregator and electricity generators, in which the DR aggregator plays as the leader to optimize the bidding strategy, and generators play as the followers to maximize their own profits.

By reviewing the above literature, it is found that the research hides a common assumption, that is, all DR participants are absolutely rational and their irrational behavior has been abandoned completely. However, in a real system, consumers on the demand side, residential users in particular, rarely have absolute rationality. Actually, a consumer's irrational behavior has a great influence on the decision-making process in the implementation of the DR project. One consumer may be affected to be in DR by its neighbors who have participated in the DR project. That is, the decision on whether consumers participate in DR is not only related with individual circumstances, but is also closely interrelated with the other group consumers. Although several papers have focused on the DR program considering consumers' irrationality [22,23], they mainly concentrated on the design of the DR mechanism to relieve the effect of irrationality and ignored the analysis of irrational behavior characteristics. For example, Reference [22] proposed a novel non-cooperative game among customers with prospect theory to incorporate the impact of customer irrational behavior, and Reference [23] put forward a dynamic pricing mechanism based on game theory, considering the existence of inexperienced or irrational users. Different from the existing research, this paper mainly concentrates on the analysis of irrational human behavior for residential users in the decision-making process of DR. In our proposed framework, a residential community is responsible for the load aggregation of internal users, and the DR project is divided into multiple stages. The residential community can independently decide whether it will participate in each stage of the DR project. In order to describe the irrationality of communities in the decision on whether to be in DR, a novel decision-making behavior model in each stage of the DR project is proposed based on the Markov chain. Accordingly, the population evolution result of being in DR can be obtained with the implementation of the DR project. Such an evolutionary process of the population is very significant for the grid to measure the feasibility of the designed DR mechanism. Furthermore, in each stage of the DR project, residential communities who are willing to be in DR have to participate in the day-ahead DR market to determine the bidding amount. To reduce the risk of participants' unreasonable bidding amount and price, a non-cooperative game approach is proposed to describe the competition behavior among communities in the day-ahead DR market. In brief, the contributions of this paper are as follows:

(1) A scenario is proposed for the multi-stage DR project to analyze the population evolution participating in DR considering the residential community's irrational behavior in the decision-making process, which can provide decision guidance in the design of the DR mechanism for dispatching the center of the smart grid.

(2) A novel decision-making behavior model is formulated with the Markov chain to forecast whether the residential community will participate in DR, which can provide a better understanding of the practical performance of the DR project.

(3) A non-cooperative game approach is formulated to search the bidding equilibrium among residential communities willing to participate in DR, which can contribute to the stability of the DR bidding market.

The rest of this paper is organized as follows. The system model is introduced in Section 2. In Section 3, we formulate the non-cooperative game approach and prove the existence of the Nash equilibrium. And then, the Markov model for the population evolution is given in Section 4. The case study and simulation results are presented in Section 5. Finally, this paper is concluded in Section 6.

#### **2. System Model**

A DR framework for residential community is proposed in Figure 1. Assume that there are total *I* residential communities with the set *I* = *N* ∪ *M*, in which group *N* contains *N* residential communities who are willing to participate in DR, while group *M* contains *M* communities who are unwilling to participate. Each community contains many residential users with a rich flexible load, such as air condition and an electrical vehicle. The dispatching center of the smart grid is mainly responsible for DR transaction with the residential communities in group *N* [24,25]. In the day-ahead DR market, since only the bidding price and amount are exchanged between the dispatching center and community, each community does not reveal the details about the energy consumption of native users' appliances. Therefore, privacy can be protected from the residential community level [26]. Furthermore, the members in each group do not always remain unchanged. After a period of time, each community obtains the opportunity to choose to participate in DR or not. In the paper, such a time slot is set to one week. Since the residential community in the scenario has bounded rationality, we assume that a community can be infected with probability β by each neighboring community in group *N*, while a community in group *N* will transfer to group *M* with probability α. That is, at the beginning of each week, each community will make a new choice with the corresponding probability. Accordingly, in the proposed framework, there exists two time scales: one is a short-time scale for daily energy consumption scheduling with the set *T* = [1, 2, ..., *T*]; the other is long-time scale for residential community decision-making with the set *H* = [1, 2, ..., *H*].

**Figure 1.** Framework for residential community participating in DR.

## *2.1. Energy Dispatching Model*

Assume that residential community *n* ∈ *N* chooses to participate in DR in week *h* ∈ *H*, then its bidding amount is *Lh*,*<sup>t</sup> <sup>n</sup>* in time slot *t* ∈ *T*. Considering the limitation of flexible DR resources on the residential side, the bidding amount has to satisfy the following constraint:

$$L\_n^{h, \text{min}} \le L\_n^{h, t} \le L\_n^{h, \text{max}} \tag{1}$$

where *Lh*,min *<sup>n</sup>* and *<sup>L</sup>h*,max *<sup>n</sup>* represent the minimal and maximal bidding amount of community *n* in week *h*, respectively. Therefore, the individual feasible bidding amount set of community *n* in week *h* can be expressed as follows:

$$\mathcal{L}\_n^h = \left\{ L\_n^h : L\_n^{h, \text{min}} \le L\_n^{h, t} \le L\_n^{h, \text{max}}, \forall t \in T \right\} \tag{2}$$

where *L<sup>h</sup> <sup>n</sup>* <sup>=</sup> *Lh*,1 *<sup>n</sup>* , *<sup>L</sup>h*,2 *<sup>n</sup>* , ··· , *<sup>L</sup>h*,*<sup>T</sup> n* is the bidding amount set of community *n* in all dispatching slots. Accordingly, the feasible bidding amount set of all residential communities in group *N* can be expressed as follows:

$$\mathcal{L}^{\rm h} = \mathcal{L}\_1^{\rm h} \times \mathcal{L}\_2^{\rm h} \times \cdots \times \mathcal{L}\_N^{\rm h} \tag{3}$$

Note that this paper mainly concentrates on peak load shaving, hence, bidding amount refers to the dispatching amount that will be cut down in real time.

#### *2.2. Bidding Price Model*

When a residential community agrees to be scheduled by the dispatching center of the smart grid, it can obtain an economic benefit from the grid, but it firstly has to take part in the day-ahead bidding market. In order to maintain bidding market stability, it is necessary for the dispatching center to design a reasonable bidding price model. In the paper, we assume that the bidding price mechanism must satisfy the following conditions:


Accordingly, a linear function is employed as the bidding price model. Since the bidding amount of community *n* in week *h* is *Lh*,*<sup>t</sup> <sup>n</sup>* , the total bidding amount of all communities in time slot *t* can be expressed as:

$$L\_h^t = \sum\_{n=1}^N L\_n^{h,t} \tag{4}$$

Therefore, the bidding price in the market can be expressed as follows:

$$p\_h^t = a\_h^t L\_h^t + b\_h^t \tag{5}$$

where *at <sup>h</sup>* <sup>&</sup>lt; 0 and *bt <sup>h</sup>* <sup>&</sup>gt; 0 are constants correlated with time slot *<sup>t</sup>* and week *<sup>h</sup>*. Parameter *<sup>a</sup><sup>t</sup> <sup>h</sup>* < 0 can guarantee the bidding price decreases with the increase of the bidding amount and, at the same time, can also effectively reduce the implementation cost of the DR project for the dispatching center.
