Bilateral Contracting and Price-Based Demand Response in Multi-Agent Electricity Markets: A Study on Time-of-Use Tariffs

Abstract

Electrical energy can be traded in liberalized organized markets or by negotiating private bilateral contracts. Competitive markets are central systems where market players can purchase and sell electrical energy. Bilateral contracting consists typically in a private negotiation of power over several months or years between two parties. Price-based demand response considers the active participation of consumers in electricity markets. Consumers adopt demand response programs when responding to market prices or tariffs, as they change over time. Those tariffs can be proposed by retailers by considering their load shape goals, influencing consumers to change their behavior. Consumers may adopt strategies from two different groups, namely by curtailing energy at times of high prices (e.g., peak and intermediate periods) and rescheduling energy away from those times to other times (shifting). This article considers bilateral contracting in electricity markets with demand response. It investigates how curtailment and shifting affect the energy quantity and energy cost of consumers that adopt a time-of-use tariff involving three block periods (i.e., base, intermediate and peak periods). The results indicate that consumers respond to changes in energy price according to their consumption flexibility, while retailers do not always change energy price in response to consumers’ efforts to change their consumption patterns. On average, by considering a 5% consumption reduction in the intermediate and peak periods by a consumer agent, a retailer agent reduces the energy price only by 1.5%.

1. Introduction

Liberalized electricity markets (EMs) are systems where competitive demand and supply players trade energy. Several electricity market models are often considered, including pools, derivatives and bilateral contracts [1,2]. Pools are marginal markets, which consist of auctions of supply and demand offers with the goal of maximizing the social welfare of participants. Derivative markets involve physical and financial tradable products that players can use to mitigate the price volatility of energy pools [3]. Bilateral contracting consists typically of a private negotiation of power over several months or years between two parties. The parties set the terms and conditions of agreements independently of market operators. They negotiate their own terms and only submit prices, quantities, periods and market zones to market operators.

Bilateral contracts are used to mitigate the risk associated with uncertain pool prices [4]. However, there is always some risk associated with energy price and quantity [5]. Price risk is related to wholesale market prices and is relevant in cases of poor agreements, when negotiated prices significantly differ from future prices of electricity [6]. Quantity risk is also relevant. Retailers usually sign non-standard forward contracts with consumers, consisting of fixed prices, but variable quantities [7]. So, there is the risk of acquiring energy that differs markedly from the energy requested by customers, paying substantial penalties for deviations [8]. A possible solution to mitigate this risk is to promote demand response (DR) programs and demand-side management (DSM) measures (see, e.g., [9,10]).

The ongoing transition to carbon-neutral societies involves a high penetration of variable renewable energy, such as wind and solar photovoltaic [11]. Variable renewable energy sources (vRESs) can be cost-effective when compared to traditional fossil fuel power plants, whose tendency is to increase their cost with the increased cost of carbon emissions [12]. However, renewable energy has a stochastic nature, being uncertain in the mid term and depending on weather conditions during real-time operation [13]. Scarcity periods may lead to high market prices and periods with an excess of energy to vRES curtailments. Replacing fossil fuel dispatchable technologies with vRES relies on significant investments in expensive energy storage solutions and/or fast-responsive dispatchable power plants, to balance supply and demand. Hence, to avoid the payment of high costs to achieve a carbon-neutral society, consumers can have an active participation in the power system. To this end, consumers can adopt demand response programs and demand-side management measures.

Demand response may be defined as changes in the usage of electric power by end-use customers from their typical consumption patterns in response to changes in the price of electricity over time [14]. DR is used to mitigate the need for additional power capacity and/or distribution facilities. DR also allows consumers to adapt their behavior to the uncertainty of renewable variable generation, avoiding investing in expensive storage solutions and/or fast-responsive dispatchable power plants. Price-based DR programs consider the active participation of consumers in electricity markets by responding to price changes over time. From the perspective of retailers, the emphasis of DR is on reducing the energy usage at peak periods and increasing it in base periods. Retailers can plan and encourage the participation of consumers in price-based DR programs, by offering time-of-use (TOU) tariffs with two, three and more periods of time involving significant differences between the price of energy in each period. This can influence consumers to change their consumption behavior, with the goal of reducing the cost of electricity. This work focuses on TOU tariffs.

In relation to demand-side management, customers may adopt one or more of the following load shape strategies [15,16]: peak clipping, valley filling, load shifting, strategic conservation, strategic load building and flexible load shape. These strategies may contribute to a reduction in the load and market price volatility, reducing the risk inherent to retailers. This reduction of risk, together with a potential increase in the profit of retailers, may contribute to a reduction in their tariffs. Such a reduction may, in turn, incentivize a change in the behavior of consumers, avoiding an increase in peak consumption and promoting an increase in off-peak consumption. Indirectly, this work considers that customers may adopt three load shape strategies: peak clipping, valley filling and load shifting strategies. While peak clipping may increase the return of retailers by decreasing peak prices, valley filling and load shifting may increase their profit by increasing the consumption of clients. The conservation strategy may decrease the profit of retailers and the flexible load shape strategy can be hard and costly to implement for retailers. All strategies can be favorable to a reduction in the costs of market players, except the load building strategy.

DR programs and DSM measures incentivize the investment and use of more efficient technologies, which may reduce the costs for customers. These technologies tend to last longer and have lower operation and maintenance costs. This is especially true for measures that incentivize the use of high efficiency heating, ventilation and air conditioning (HVAC) systems, energy efficient lighting and process technologies, such as fans and motors. Using high efficiency HVAC systems to define lower ambient temperature set-points is the main measure to reduce natural gas consumption, mainly during winter time. Another important measure consists in replacing natural gas with alternative sources [17]. Adapting the HVAC system set-points is one of the most easy and quick DSM measures for curtailing and/or shifting consumption (see, e.g., [9]). However, in some cases it may affect the comfort levels of consumers. In this case, utility functions can be used to evaluate the cost–benefit of DR and DSM measures.

A review of the main concepts behind DR and DSM can be found in [18,19,20,21]. In particular, a primer on demand-side management with a description of load-shape objectives and real examples of price-based DR programs applied in different states of the United States is presented in [18]. There is a special focus on describing the outcomes of time-of-use tariffs and the response of consumers to critical-peak pricing (CPP) and real-time pricing (RTP) tariffs. The real examples show the success of DR and DSM measures, with special emphasis on a reduction of 27% in the peak consumption of residential consumers with CPP tariffs, in the state of California.

The majority of work about DR and DSM uses load aggregation to achieve target reductions in the consumption of peak periods (see [22,23,24,25,26]). There is only a reduced number of articles that focus on the perspective of consumers. For example, Ogunjuyigbe et al. [27] developed a load-satisfaction algorithm that maximizes the residential satisfaction of consumers considering a target budget. The algorithm receives the power ratings of the devices, their time-of-use tariffs, their consumption and the satisfaction of users for each device. By considering a target budget, defined by users, the algorithm selects the devices that should be utilized to maximize the satisfaction of users. Arteconi et al. [28] simulated the control of the cooling loads of an industrial building in four different scenarios, with and without storage, by considering the following: (i) a typical dual Italian tariff (peak: 0.164 EUR/kWh, off peak: 0.149 EUR/kWh) without storage (baseline), (ii) a typical Italian tariff with storage, (iii) a tariff similar to the Chinese tariff (peak: 0.164 EUR/kWh, off peak: 0.082 EUR/kWh) with storage and (iv) PV and the Italian tariff with storage. In relation to the baseline, the authors concluded that only the fourth scenario can bring economic outcomes in a typical week, while in a hot week only the second scenario does not bring economic outcomes. Wang et al. [29] presented a bi-level demand response model to coordinate spatially distributed thermally controllable loads. The authors considered a demand response market by using a load aggregator of thermally controllable loads, which responds to price signals of distribution network operators that want to improve the efficiency of operations by reducing costs. Bernath et al. [30] analyzed the impact of the most relevant flexibility options of consumers on the market value of renewable energy. Consumers adopted the available options to shift consumption to cheaper hours with higher penetrations of variable generation, decreasing their costs with energy and increasing the renewable energy value. Tavakkoli et al. [31] proposed to use consumption flexibility for strategic bidding of consumers that participate in day-ahead and balancing markets, considering a bi-level optimization model. Simulation results showed that consumers can impact the price in their favor by bidding strategically in power markets.

Against this background, the purpose of this article is threefold:

• To model retailer agents able to consider DR programs and to propose strategic tariffs by considering the behavior of consumers;
• To model consumer agents able to respond to DR programs and to adopt DSM measures in order to deal with different tariffs (and minimize costs);
• To study the behavior of retailer and consumer agents. In particular, to experimentally analyze the response of a consumer to a time-of-use tariff proposed by a retailer, as well as the response of the retailer to the behavior changes of the consumer.

The work presented here is a natural extension of our previous work in the area of multi-agent electricity markets. Lopes et al. [32] presented a model of individual behavior for software agents operating in distributed environments (e.g., a supply chain or an energy market). The model enables agents to negotiate the terms and conditions of bilateral contracts. Specifically, the authors presented a bilateral negotiation model involving an interaction protocol and a set of negotiation strategies and tactics (e.g., concession and integrative strategies and tactics).

Following this, Lopes et al. [33] formalized several new strategies, including a “price management” strategy for producers/retailers and a “volume management” strategy for consumers. Both strategies are associated with the load response of consumers, particularly shifting, enabling market players to promote DR. The authors also presented a study on forward bilateral contracts.

Moreover, Lopes et al. [34] extended the previous work by formalizing several DR strategies involving hour-wise tariffs and enabling consumers to deal with both energy price and energy quantity variations, instead of energy price concessions only. Lopes et al. [35] paid special attention to the scoring preferences of the negotiating agents. Notably, they considered different utility functions to rate and compare offers. Algarvio et al. [9] described a real-world case-study involving a public library (real consumption and equipment specifications). The study considered specific DSM measures and a direct load control program to reduce costs. The results showed a significant reduction in total energy cost. Finally, Lopes et al. [10] described key DR programs and presented a study of the price-effect of DR on the Iberian day-ahead market for the period between January 2014 and January 2017. The study considered market prices above 80 EUR/MWh and reductions in demand of 1%, 3% and 5%. The results showed that the prices were reduced between 0.38% in 2017 and 37.35% in 2014. The benefit of adopting DR measures varied between 3.44% in cases of a reduction of 1% in 2017 and 49.40% in cases of a reduction of 5% in 2014.

The remainder of the paper is structured as follows. Section 2 is devoted to negotiations between software agents. It introduces a multi-agent electricity market and briefly describe a negotiation model for autonomous agents. Section 3 is devoted to bilateral contracting with DR. It describes several strategies for promoting demand response. Section 4 presents a study on bilateral contracting involving DR. Finally, concluding remarks are presented in Section 5.

2. Multi-Agent Electricity Markets and Bilateral Negotiation

2.1. Multi-Agent Electricity Market

The present work considers an agent-based system for both helping to manage the complexity of electricity markets towards long-term sustainability and bridging the opportunities between market models—notably bilateral contracting—and emerging technologies such as DR. The agents are equipped with models that enable them to participate in competitive electricity markets and adapt to different market structures and behaviors.

The naturally decentralized nature of multi-agent systems makes them the perfect suit to simulate the real-world behavior of liberalized competitive markets with multiple large and small players—or agents. Each agent can have its own architecture and can be equipped with its own models. Moreover, each agent can communicate with other agents using specific communication protocols. To this end, the system interface and market models were programmed in Java and agents used communication protocols of the JADE platform [36].

The agents are autonomous computer systems able to perform actions in pursuit of their design objectives. They are equipped with a generic conceptual model of individual behavior (see [32] for details). Moreover, they are pro-active—can exhibit goal-directed behavior—and social—can interact with other agents through message passing. Figure 1 shows the main agents operating in the electricity market and the interactions between them. For generality, the figure also shows other agents that can play an important role in the market (e.g., the market operator and the system operator). The following four types of agents are of particular importance to this paper (see Figure 1):

• Generator or producer agents: these represent utility companies and operate in a wholesale market; they sell electricity to a wholesale market;
• Retailer or supplier agents: these represent the business units that sell electricity to consumers; they buy energy from a wholesale market and sell energy to a retail market;
• Trader agents: similar to retailers, these promote liberalization and competition and simplify dealings between sellers and buyers;
• Consumer or customer agents: this includes commercial, industrial, residential and other electricity consumers.

The wholesale market shown in Figure 1 is typically composed by a day-ahead market (and several other markets, such as intra-day and balancing markets). The day-ahead market is the most important spot market of electricity, being used in this work for retailers obtaining the energy required by consumers. Retailers rely on expected future day-ahead prices to negotiate private bilateral contracts and promote DR programs for end-use consumers. Bilateral contracts are negotiated privately between retailers and consumers, meaning that retailers compete to attract consumers to their portfolios.

2.2. Bilateral Negotiation

This subsection gives a brief overview of bilateral negotiation between autonomous agents (e.g., agents operating in an energy market). Agents equipped with the model are able to negotiate the terms and conditions of bilateral contracts (see [32] for an in-depth description of the model). The section also describes some concession tactics.

Negotiation involves a set of issues, such as energy price, energy quantity, time of delivery and duration. Let $\mathcal{A}=\{a_s,a_b\}$ be the set of agents and denote the set of issues at stake by $\mathcal{I}$. For each issue X in $\mathcal{I}$, the range of acceptable values is represented by the interval $\mathcal{D}=[x_{min},x_{max}]$. Let $\mathbf{X}=(x_1,\cdots ,x_n)$ designate a vector of n issue values. The negotiating agenda is the set $\mathcal{I}$ of prices ($p_n$) and quantities ($q_n$) of energy to be deliberated during negotiation: This means that we consider in this paper only the most important issues, i.e., the prices and quantities of energy for different blocks of time.

Effective negotiators often define priorities and targets for each issue on the table. Prioritization involves assigning positive numbers to energy prices and quantities. The weight of an issue is a real number that represents the preference of an agent for the issue. The limit is the point where an agent decides to stop trading and walk away from the negotiating table, because any settlement beyond this point is not minimally acceptable.

The trading procedure involves basically an iterative exchange of offers and counter-offers. Let $\mathcal{T}=\{1,2,\cdots \}$ be the set of discrete time periods. The parties interact with one another according to the rules of an alternating offers protocol [38]. One party offers in odd periods $\{1,3,\cdots \}$ and the other party in even periods $\{2,4,\cdots \}$. Each party has a continuous utility function U to rate offers and counter-offers. Accordingly, when the utility for one party from one offer is greater than from another offer, the party prefers the first offer over the second. The issues are bundled and discussed jointly, i.e., an offer (or proposal) consists of price-quantity pairs. The trading procedure marks branching points at which the agents need to make decisions—that is, the procedure restricts the possible ways to trade energy, but often does not specify a unique and exclusive way. Strategies are thus necessary to fully determine the course of trading.

At every period of time, the agents follow their own strategies to choose among different possible actions to execute (e.g., offer acceptance, offer rejection or breaking off). Moreover, strategies state whether trading should continue or terminate (either party should break off unilaterally or continue trading). More formally, strategies are functions that specify the tactics to be used during the course of negotiation. The line between strategies and tactics often seems indistinct, but one key difference is that of scope. Tactics are structured and driven by strategic considerations—they are functions that define the short-term moves to enact or pursue high-level strategies.

Concession making or yielding strategies are probably the most common strategies for managing price—agents reduce their demands to accommodate the opponent. The formal definition of several price management strategies is presented elsewhere [39]. Concession tactics model concessions on single energy prices toward agreement. For a given time t, they compute new values for price either by directly or indirectly considering the opponent’s price and quantity. A description of several concession tactics follows.

A generic price concession tactic models concessions on energy prices by considering a positive concession factor. Conceptually, the concession factor is modeled as a real number in the range $[0,1]$. A large number of price concession tactics can be defined simply by assigning different values—or specifying a different range of values—for the concession factor ${\mathit{C}}_{\mathit{f}}$ (see [32] for further information).

Let $p\in \mathcal{I}$ and $q\in \mathcal{I}$ denote the price and energy to be traded in a specific time block, respectively. Let $p_n$ be the price for quantity $q_n$ of issue n. An energy-dependent price concession tactic (EDPCT) for a customer agent $a_b$ is formalized by a function $\mathcal{Q}$ with the following general form (see also [40]):

$$\mathcal{Q}(p_n,q_n)=p_n+{\mathit{e}}^{-\beta \frac{q_n}{{\mathit{q}}_{\mathit{total}}}}\left({\mathit{p}}_{\mathit{n},\mathit{lim}}-p_n\right)$$

(1)

where $\beta \in {\mathbb{R}}^{+}$ is a parameter, ${\mathit{q}}_{\mathit{total}}$ is the bilateral contract amount of energy and ${\mathit{p}}_{\mathit{n},\mathit{lim}}$ is the limit for $p_n$.

Although this tactic can be employed by both sellers and buyers of electrical energy, it is probably more appropriate for buyers (hence, it is formalized above for $a_b$ only). Strictly speaking, the function $\mathcal{Q}$ represents an infinite number of possible tactics, one for each value of $\beta$. This parameter determines the convexity degree of the curve ${\mathit{C}}_{\mathit{f}}={\mathit{e}}^{-\beta q_n/{\mathit{q}}_{\mathit{total}}}$. The downward-sloping shape of this curve represents the typical willingness to concede more when energy usage is lower.

An optimal-profit price concession tactic (OPPC) for a supplier agent $a_s$ is formalized by a function ${\mathcal{P}}_{\mathcal{R}}$, which takes the prices $P=(p_1^s,\cdots ,p_n^s)$ offered by $a_s$ at time t and the adjusted quantities $Q^{\prime }=(q_1^{\prime },\cdots ,q_n^{\prime })$ received from $a_b$ at time $t+1$ as input and computes new (optimal) prices by solving the following optimization problem:

$$\underset{p_n^s}{max}{\mathcal{P}}_{\mathcal{R}}=\underset{p_n^s}{max}\left\{\sum _{n=1}^n\left(p_n^s-c_n^s\right)\times q_n^{\prime }\right\}$$

(2)

subject to:

$$\begin{array}{c}{p}_n^s\ge c_n^s\mathrm{and}{p^{\prime }}_n^s\le p_n^s\mathrm{for}N=1,\cdots ,n\hfill \end{array}$$

(3)

where ${\mathcal{P}}_{\mathcal{R}}$ is the expected profit of the contract, $c_n^s$ the expected cost of quantity $q_n^{\prime }$ and ${p^{\prime }}_n^s$ the new selling price. Constraint (3) assures that the cost of production, for producers or the average expected cost of energy, for retailers, do not exceed the price of energy considered by $a_s$ in order to have some profit. This tactic reacts to adjustments in energy quantities by changing prices. Specifically, it models concessions on the various prices for the different periods of the day by maximizing the expected profit of the contract.

3. Strategies and Tactics for Promoting Demand Response

This section focuses on price-based demand response and presents several tactics for promoting DR (see also [41] for details). Broadly speaking, customers respond to high prices by adopting load response strategies from the two fundamental groups [14,42]:

• Foregoing: this involves reducing electricity usage at times of high prices without making it up later (e.g., a commercial facility may turn off office equipment or adjust the air-conditioning set-points); typically, reduction in usage occurs during peak hours and consumption pattern is often not changed during other periods;
• Shifting: this involves rescheduling energy usage away from times of high prices to other times (e.g., a larger industrial customer may shift some peak demand operations to medium and/or off-peak hours).

Let $a_b\in \mathcal{A}$ be a customer agent, $Q\in \mathcal{I}$ the energy to be traded in a given block of time and $\mathcal{D}=[q_{min},q_{max}]$ the range of acceptable values for Q. The customer $a_b$ responds to high prices either by curtailing usage or by shifting usage to lower-priced hours. Accordingly, $a_b$ can adjust the value q of Q within a feasible DR range, represented by $q_{lb}\le q\le q_{ub}$, where $q_{lb}$ and $q_{ub}$ are the lower and upper DR bounds, respectively.

Keeping the amount of energy within a feasible range is partially justified by the potential costs for customers. In particular, maintaining the quantity at or above a lower DR bound helps to control expected ongoing costs (e.g., to control the expected costs of rescheduling activities). The upper DR bound is a little more arbitrary. If the price differentials between blocks of hours are significant, customers react to high peak prices by adjusting the timing of their electricity usage either to take advantage of lower-priced periods or to avoid consuming during high-priced peak periods [14].

Consider the specific case of the customer $a_b$ and let $p_{1_{init}},\cdots ,p_{n_{init}}$ and $q_{1_{init}},\cdots ,q_{n_{init}}$ be its initial prices and quantities, respectively. For peak-load periods, $a_b$ may adjust electricity usage by reducing the value q of Q only. Accordingly, for these periods, the upper DR bound $q_{ub}$ is set either at the initial value ${\mathit{q}}_{\mathit{init}}$ of Q or at a lower value. For other periods, $a_b$ may adjust electricity usage either by reducing or by increasing Q during the course of trading.

The load response strategies in the two aforementioned fundamental groups are implemented through a variety of tactics. Let $o_{s\to b}^t=\langle (p_1,q_1),\cdots ,(p_n,q_n)\rangle$ be the offer received by $a_b$ at time t and $o_{b\to s}^{t+1}=\langle (p_1^{\prime },q_1^{\prime }),\cdots ,(p_n^{\prime },q_n^{\prime })\rangle$ the offer that $a_b$ is ready to send at time $t+1$. For a given time t, DR tactics compute new values $q_1^{\prime },\cdots ,q_n^{\prime }$ for energy quantities by considering the selling prices $p_1,\cdots ,p_n$. Four representative DR tactics, namely two foregoing tactics and two shifting tactics, follow.

A price threshold foregoing tactic (PTFT) reduces energy usage in high-priced peak periods when the price rises above a predetermined electricity price threshold. That is, it considers a specific price threshold at which customers will reduce peak loads. Let $p_n^s$ be the selling price for quantity $q_n$ of Q at time t. A PTFT tactic for a customer agent $a_b$ is a function with the following general form:

$$\mathcal{F}(q_n,p_n^s)=\left\{\begin{array}{cc}{q}_n-\mathit{C}\times (q_n-q_{n,lb})\hfill & \mathrm{if}{p}_n^s>p_{n,thr}\hfill \\ q_n\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(4)

such that:

$$\begin{array}{c}{q}_n\ge q_{n,lb}\mathrm{and}{q}_n^{\prime }\le q_n\hfill \end{array}$$

(5)

where $\mathit{C}\in [0,1]$ is a curtailment factor that defines the magnitude of the reduction in $q_n$, $q_{n,lb}$ is the lower DR bound, $p_{n,thr}$ is the price threshold and $q_n^{\prime }$ is the new value of $q_n$.

Constraint (5) limits how much Q can decrease and also states that Q cannot increase. This permits, for instance, the modeling of reductions in quantity for peak-load periods only—that is, the consumption pattern is not changed during low-demand periods. Now, it is worth noting that the following three DR tactics will be modeled as functions of an n-vector of quantities (and an n-vector of prices).

An optimal-load foregoing tactic (OLFT) reduces the total amount of electricity to be delivered by a specific percentage and then adjusts energy usage in the different periods of the day by minimizing the contract cost. Let $o_{s\to b}^t$ be the offer received by the customer agent $a_b$ at time t. For $a_b$ this tactic is formalized by a function ${\mathcal{F}}_{\mathcal{O}}$, which takes the energy quantities $Q=(q_1,\cdots ,q_n)$ and the prices $P=(p_1^s,\cdots ,p_n^s)$ as arguments, reduces the total amount ${\mathit{q}}_{\mathit{total}}$ by considering a curtailment response factor $C_r$ and then computes new energy quantities by solving the following optimization problem:

$$\underset{q_n}{min}c=\underset{q_n}{min}\left(\sum _{n=1}^n{p}_n^s\times q_n\right)$$

(6)

subject to:

$$\begin{array}{c}\sum _{n=1}^n{q}_n={\mathit{q}}_{\mathit{total}}^{\prime }\hfill \end{array}$$

(7)

$$\begin{array}{c}{q}_n\ge q_{n_{lb}}\mathrm{and}{q}_n^{\prime }\le q_n\mathrm{for}N=1,\cdots ,n\hfill \end{array}$$

(8)

where c is the bilateral contract cost, ${\mathit{q}}_{\mathit{total}}^{\prime }=(1-C_r){\mathit{q}}_{\mathit{total}}$ is the reduced bilateral contract amount of energy (the magnitude of reduction is defined by $C_r\in [0,1]$), $q_{n_{lb}}$ is the lower DR bound, ${\mathit{q}}_{\mathit{n}}^{\prime }$ is the new (single) quantity value and $p_n^s$ is the price for quantity ${\mathit{q}}_{\mathit{n}}$. Constraint (7) ensures that the contract amount stays at the level defined by $C_r$ and constraint (8) keeps the value of $q_n$ at or above the lower bound and also prevents $q_n$ from increasing.

An optimal-load shifting tactic (OLST) adjusts the timing of electricity usage to take advantage of lower-priced periods. Specifically, it reschedules usage away from high-priced peak periods to lower-priced periods by minimizing the contract cost. For a customer agent $a_b$, this tactic is formalized by a function ${\mathcal{S}}_{\mathcal{O}}$, which takes Q and P as input and shifts electricity usage to lower-priced periods by computing new energy quantities, i.e., by solving the following optimization problem:

$$\underset{q_n}{min}c=\underset{q_n}{min}\left(\sum _{n=1}^n{p}_n^s\times q_n\right)$$

(9)

subject to:

$$\begin{array}{c}\sum _{n=1}^n{q}_n={\mathit{q}}_{\mathit{total}}\hfill \end{array}$$

(10)

$$\begin{array}{c}{q}_{n_{lb}}\le q_n\le {\mathit{q}}_{{\mathit{n}}_{\mathit{ub}}}\mathrm{for}N=1,\cdots ,n\hfill \end{array}$$

(11)

where c is the bilateral contract cost, ${\mathit{q}}_{\mathit{total}}$ is the bilateral contract amount of energy, $q_{n_{lb}}$ is the lower DR bound and $q_{n_{ub}}$ is the upper DR bound. Note that constraint (10) keeps the bilateral contract amount at ${\mathit{q}}_{\mathit{total}}$ and constraint (11) gives upper and lower bounds for $q_n$.

A price threshold optimal shifting tactic (PTOST) manages the timing of electricity usage by considering price thresholds. Specifically, it reschedules usage away from time periods when selling prices rise above predetermined price thresholds to other time periods. For a customer agent $a_b$, this tactic is formalized by a function ${\mathcal{S}}_{\mathcal{T}}$, which takes Q and P as input and shifts usage from hours when selling prices rise above price thresholds to lower-priced hours, by solving the following optimization problem:

$$\underset{q_n}{min}c=\underset{q_n}{min}\left(\sum _{n=1}^n{p}_n^s\times q_n\right)$$

(12)

subject to:

$$\begin{array}{c}\sum _{n=1}^n{q}_n={\mathit{q}}_{\mathit{total}}\hfill \end{array}$$

(13)

$$\begin{array}{c}{q}_n\ge q_{n_{lb}}\mathrm{and}{q}_n^{\prime }\le q_n\mathrm{if}{p}_n^s>p_{n_{thr}},\mathrm{for}N=1,\cdots ,n\hfill \end{array}$$

(14)

$$\begin{array}{c}{q}_{n_{lb}}\le q_n\le {\mathit{q}}_{{\mathit{n}}_{\mathit{ub}}}\mathrm{if}{p}_n^s\le p_{n_{thr}},\mathrm{for}N=1,\cdots ,n\hfill \end{array}$$

(15)

Constraints (13)–(15) are essentially identical to the previous constraints and details are therefore omitted.

4. A Study on Time-of-Use Tariffs

4.1. The Experimental System and Hypotheses

The experimental system consists of two software agents and a simulated environment. Let $A_g=\{a_s,a_b\}$ be the set of agents. The agent $a_s$ plays the role of seller (or retailer) and the agent $a_b$ plays the role of a buyer (or customer). The agents negotiate a three-rate tariff, i.e., negotiation involves six major issues: price-1 to price-3 and volume-1 to volume-3. This tariff follows the usual three-rate tariff, by taking into account the following:

• Peak: from 11 a.m. to 3 p.m. and from 6 p.m. to 10 p.m.;
• Intermediate: from 7 a.m. to 11 a.m., from 3 p.m. to 6 p.m. and from 10 p.m. to 12 p.m.;
• Base: from 12 p.m. to 7 a.m.

The retailer agent sets the hourly rates in accordance with the wholesale price, i.e., more expensive in periods where the price is higher and cheaper when it is lower. On the one hand, the retailer agent “advises” the customer agent when to place consumption. From the point of view of the customer agent, the increment of price at certain hours constitutes an incentive to move consumption into cheaper hours.

Every agent $a_i$ is equipped with the aforementioned negotiation model. Negotiation involves an iterative exchange of offers and counter-offers, where:

• The first agent to submit a proposal is the retailer agent;
• The agents are allowed to propose only strictly monotonic offers—the customer’s offers increase monotonically and the retailer’s offers decrease monotonically;
• The acceptability of a proposal is determined by a negotiation threshold—an agent $a_i\in \mathcal{A}$ accepts a proposal $o_{j\to i}^{t-1}$, submitted by $a_j\in \mathcal{A}$ at period $t-1$, when the difference between the benefit provided by the proposal $o_{i\to j}^t$ that $a_i$ is ready to send in the next time period t is lower than or equal to a negotiation threshold;
• The agents are allowed to exchange only a maximum number of proposals, denoted by ${\mathit{max}}_{\mathit{prop}}$.

The agents and the environment have a built-in set of parameters that govern their behavior and facilitate experimentation. The relevant parameters for the experiment and their values are shown in

Table 1. Experimental parameter values.

Experimental Parameter	Value
Percentage for computing consumer’s market perceived price	1%
Percentage for computing retailer’s market perceived price	1%
Percentage for computing consumer’s limit	10%
Percentage for computing retailer’s limit	10%
Percentage for computing consumer’s initial level of aspiration	5%
Percentage for computing retailer’s initial level of aspiration	5%
Maximum number of proposals	20
Utility threshold	0

. Most values are based on data and results of case studies published in the negotiation literature (see, e.g., [43]).

Table 2. Reference price and energy for the negotiating parties (customer and retailer).

Day Period	Reference Price (EUR/MWh)	Energy (MWh)
Base	43.96	552.62
Intermediate	51.62	969.67
Peak	54.79	897.62

shows the reference prices and energy quantities for both agents (the retailer and the customer). Some values were selected by looking up to real trading prices associated with a pool market in an attempt to approximate this experience to the real world. Specifically, the minimum retailer price (i.e., the limit) per period is equal to the market reference price per period. The market reference price was obtained through the analysis of the Iberian market day-ahead electricity prices. Moreover, the energy volumes are based on a large commercial load profile provided by the New Hampshire Electric Co-op. The perceived market price is computed from the reference market value. The limit and the initial level of prices are then computed from the perceived market price.

The independent variable is the customer (or buyer) pricing strategy (and associated tactics). This variable has five levels, namely the control strategy/tactic (EDPCT) and the four DR strategies/tactics presented in Section 3. The value of this variable is under the control of the experimenter. The dependent variables are the negotiated prices and volumes, the customer’s utility, the joint utility, the time spent in negotiation and the outcome of the negotiation. The values of these variables are observed by the experimenter as measurements. The first and second dependent variables are the negotiated prices and volumes. These are the most important variables. The volumes can show the impact of DR and the prices may demonstrate if the customer’s effort to reduce peak consumption could result in better prices. The third dependent variable is the customer’s utility. This is a dependent variable because a major purpose of the research consists of examining the effect of concession strategies on the bargainer who uses these strategies (the customer in this study) and not the opponent.

Consider that $a_s$ and $a_b$ agreed with a proposal containing the prices $P=\{p_1,\cdots ,p_n\}$ and the volumes $Q=\{q_1,\cdots ,q_n\}$. The utility $U_b(p_1,\cdots ,p_n,q_1,\cdots ,q_n)$ of $a_b$ is given by the following function:

$$U_b=\sum _{n=1}^n{\mathit{w}}_{\mathit{B},\mathit{n}}{\mathit{B}}_{\mathit{n}}\left(p_n\right)+{\mathit{w}}_{\mathit{V},\mathit{n}}{\mathit{V}}_{\mathit{n}}\left(q_n\right)+{\mathit{w}}_{\mathit{n}}{\mathit{B}}_{\mathit{n}}\left(p_n\right){\mathit{V}}_{\mathit{n}}\left(q_n\right)$$

(16)

where ${\mathit{w}}_{\mathit{B},\mathit{n}}$ is the weight of $a_i$ for an issue $p_n\in \mathcal{I}$, ${\mathit{w}}_{\mathit{V},\mathit{n}}$ is the weight of $a_i$ for an issue $q_n\in \mathcal{I}$, ${\mathit{w}}_{\mathit{n}}$ is a constant, ${\mathit{B}}_{\mathit{n}}\left(p_n\right)$ is the (marginal) utility function of $a_b$ for $p_n$ and ${\mathit{V}}_{\mathit{n}}\left(q_n\right)$ is the (marginal) utility function of $a_b$ for $q_n$. The (marginal) utility function ${\mathit{B}}_{\mathit{n}}\left(p_n\right)$ is defined as follows:

$$B_n=\frac{li{m}_n-p_n}{li{m}_n-p_{n,init}}$$

(17)

where $li{m}_n$ is the limit of $a_b$ for the price $p_n$ and $p_{n,init}$ is the customer’s initial price in the first proposal. Similarly, ${\mathit{V}}_{\mathit{n}}\left(q_n\right)$ is defined as follows:

$$V_n=1-\frac{|q_n-q_{n,init}|}{q_{n,init}}$$

(18)

where $q_{n,init}$ is the customer standard profile at period n.

The fourth dependent variable is the joint utility, measured in terms of the sum of both agents’ utilities. A high joint utility is normally obtained through the use of win–win or integrative negotiation strategies and typically means that both agents achieve a mutual beneficial agreement, so all are satisfied. The fifth dependent variable is the total number of offers exchanged by the agents until either they found an agreement or reached the maximum number of proposals, set to $ma{x}_{prop}$. The last dependent variable is the outcome of negotiation (agreement or not). This variable is used to compute the percentage of deals made in a number of negotiations.

The customer agent adopts the energy dependent concession making a strategy/tactic for changing the energy prices during the course of negotiation. The retailer agent negotiates electricity prices with the customer agent and may adopt different strategies, such as the well known tit-for-tat strategy, which imitates the price concessions of the opponent throughout negotiation (see, e.g., [44]).

The experimental hypotheses are as follows:

• H1: The DR strategies lead, on average, to a peak volume reduction (in relation to the control strategy);
• H2: The curtailment strategies (i.e., PTFT and OLFT) lead, on average, to a higher peak volume reduction than the shifting strategies (i.e., OLST and PTOST);
• H3: The DR strategies lead, on average, to a peak price reduction (in relation to the control strategy);
• H4: The curtailment strategies lead, on average, to a higher peak price reduction than the shifting strategies;
• H5: The DR strategies lead, on average, to a lower total cost (in relation to the control strategy);
• H6: The curtailment strategies lead, on average, to a lower total cost than the shifting strategies;
• H7: The DR strategies lead, on average, to a decrease in the consumption (in relation to the control strategy);
• H8: The curtailment strategies lead, on average, to a decrease in the consumption (in relation to all the other strategies);

4.2. Experimental Results and Discussion

The experiment was conducted on a personal computer using the MATREM system [45]. For each of the strategies, we conducted 30 trials, in a total of 360 trials. The results are shown in

Table 3. Experimental results (mean values of the variables under evaluation).

Experimental Evaluation Variables		Consumer’s Strategies
			Curtailment		Shifting
	Periods	EDPCT	PTFT	OLFT	OLST	PTOST
	Base	552.62	552.62	552.62	607.88	607.88
Volume (kWh)	Intermediate	969.67	914.83	938.43	1004.17	970.05
	Peak	897.62	844.86	807.86	807.86	841.98
	Base	42.04	42.07	41.84	41.75	41.98
Price (EUR/MWh)	Intermediate	51.16	50.67	51.06	51.69	51.18
	Peak	54.77	54.36	53.97	53.99	54.28
Levelized		50.42	49.97	49.87	49.96	49.95
Price (EUR/MWh)
Daily Cost (EUR)		122.00	115.54	114.64	120.91	120.86

. We note that most results are statistically significant.

The main response measures are the negotiated volumes and prices. In relation to the energy volumes, it was predicted that the DR strategies yielded a peak reduction when compared with the control strategy. Table 3 reports the mean volumes obtained in negotiation. The results indicate that the DR strategies lead to a peak volume reduction and therefore hypothesis 1 is supported. Moreover, considering the curtailment strategies, it was predicted that they yielded a reduction in high peak volumes. The results in Table 3 indicate that the OLFT and OLST strategies lead to higher reductions in peak volumes when compared with all other strategies. However, we found no statistical differences between the results and therefore hypothesis 2 is not supported.

Another key response measure is the energy price. The prediction was that the DR strategies decreased the peak volume, which then resulted in a peak price reduction. The results indicate that this prediction is confirmed (see Table 3). The DR strategies lead to a decrease in the peak price and therefore hypothesis 3 is supported. Analyzing, now, the curtailment strategies, it was predicted that they yielded a lower peak price. The results indicate that the OLFT and OLST strategies lead to lower peak prices when compared with all the other strategies (see Table 3). However, we found no statistical differences between these results. Hence, hypothesis 4 is not supported.

Another important measure is the total cost. It was predicted that the DR strategies yielded lower costs due to a reduction in the peak volume. The results indicate that this prediction is confirmed (see Table 3) and therefore hypothesis 5 is supported. It was also predicted that the curtailment strategies yielded a lower cost due to a reduction in the peak and intermediate volumes, while keeping the base volume, which would result in a reduction in the total volume (when compared with all the other strategies) and thus would lead to a decrease in the total cost. The results indicate that this prediction is confirmed. Thus, hypothesis 6 is supported. The OLFT strategy leads to a lower cost when compared with all the other strategies (see Table 3).

The last measure involves the adoption of different strategies and the subsequent decrease in consumption. While the use of the shifting strategies do not bring any changes to consumption, the curtailment strategies can reduce consumption significantly (see Table 3). Thus, hypothesis 7 is not supported, while hypothesis 8 is supported. The OLFT strategy leads to the highest reduction in consumption when comparing with all the other strategies.

In the next paragraphs, we analyze the results obtained with the different trials to get some preliminary conclusions about the behavior of both the customer and the retailer. Although preliminary, the conclusions can shed some light on the typical behavior of the customer and the retailer, regarding the adoption of the three-rate tariff (by the customer) and the price-based response (of the retailer).

Customer response to the three-rate tariff. Typically, the customer agent responds to the three-rate tariff by reducing consumption in the peak period. This agent also reduces consumption in the intermediate period, but not always (e.g., when the consumer keeps or increases the consumption in that period). In the base period, the consumer agent only maintains or increases consumption.

Figure 2 depicts the behavior of the customer agent in relation to the energy prices. Broadly, we can conclude that the customer responds to the magnitude of the prices almost independently of the block periods. For high prices, the tendency is to decrease large amounts of energy and for low prices to increase or maintain consumption. Statistically, we found no differences between the results of the control strategy (EDPCT) and the curtailment strategies, meaning that the consumer tends to maintain the base consumption. However, we note that the shifting strategies led to an increase in the base consumption.

For intermediate prices, the results of the various strategies are statistically significant. The exception is the PTOST strategy, a shifting strategy, which led the agent to maintain the intermediate consumption in the majority of the trials. The curtailment strategies led to reductions in consumption of different magnitudes.

The peak prices are critical for the customer agent. The OLFT and OLST strategies resulted in large reductions in this agent (see Table 3). In particular, the OLFT strategy led to the highest reduction in the daily cost of electricity. This can also be verified by the levelized price of the corresponding tariff.

Price-response of the retailer. Generally, consumers respond directly to tariffs, while retailers respond to both consumers’ consumption and market prices and thus the analysis of their behavior is more complex. Furthermore, the day-ahead market considers hourly prices while time-of-use tariffs consider two, three or more block periods (with several hours).

For the base period, we found no significant differences in the price. However, for the sake of completeness, we note that the retailer agent tends to make higher discounts for increases in consumption (see Figure 3). For the intermediate period, the results are statically significant. To some extent, this suggests that the intermediate period is almost as important as the peak period for the retailer agent. Moreover, we can conclude that the retailer only decreases the price in cases of a consumption reduction (see Figure 4). For the peak period, the behavior of the retailer is similar to the intermediate period, but makes smaller price reductions in order to not cause the customer to increase consumption (see Figure 5).

5. Conclusions

This paper described retailer agents able to consider DR programs and to propose strategic tariffs by considering the behavior of consumers. Moreover, it described customer agents able to respond to DR programs and to adopt DSM measures in order to deal with different tariffs and minimize costs. To analyze the behavior of the retailer and the consumer agents, the paper presented a study involving a three-rate tariff (time-of-use tariff) and two agents: a retailer and a customer.

The experimental results showed that the customer agent responds directly to the time-of-use tariff, independently of the block period. The response is dependent on the willingness of the agent to reduce or shift consumption, i.e., the flexibility of the customer agent. Moreover, the response of the retailer agent to the customer variation of consumption is as follows. For the base period, the retailer gives small discounts (<1%) for reasonable increases in consumption (>8%). For the intermediate and peak periods, the retailer increases the price (>1%) for small increases in consumption (<5%) and slightly decreases the price (>1%) for significant decreases in consumption (>5%). This behavior incentivizes the customer agent to keep changing their consumption behavior.

To conclude, a DR program in the form of a time-of-use tariff can lead to a reduction in market price, since the base and peak prices seem to get closer. The reduction of market price can, in turn, lead retailers to increase their return and consequently, to reduce the prices of their tariffs. For consumers, there are also advantages, since they can benefit from lower tariffs and by reducing peak consumption, from lower global costs.

In future work, we intend to study the behavior of several consumers adopting DR programs and the impact of retail competition, by taking into consideration several retailers that try to attract consumers to their portfolios (we note that the present study only considers the behavior of one consumer and one retailer).