^{1}

^{*}

^{2}

^{3}

^{2}

A real-time pricing system of electricity is a system that charges different electricity prices for different hours of the day and for different days, and is effective for reducing the peak and flattening the load curve. In this paper, using a Markov decision process (MDP), we propose a modeling method and an optimal control method for real-time pricing systems. First, the outline of real-time pricing systems is explained. Next, a model of a set of customers is derived as a multi-agent MDP. Furthermore, the optimal control problem is formulated, and is reduced to a quadratic programming problem. Finally, a numerical simulation is presented.

In recent years, there has been growing interest in energy and the environment. For problems on energy and the environment such as energy saving, several approaches have been studied (see, e.g., [

In this paper, using a Markov decision process (MDP), we propose a mathematical model of real-time pricing systems. Since in many cases, the status of electricity conservation of customers is discrete and stochastic, it is appropriate to use an MDP. Then, a set of electricity customers is modeled by a multi-agent MDP. Furthermore, we consider the finite-time optimal control problem. By appropriately setting the cost function, it is achieved that customers conserve electricity actively. This problem can be used for the model predictive control method, which is a control method that the finite-time optimal control problem is solved at each time. In addition, the finite-time optimal control problem can be reduced to a quadratic programming problem. The proposed approa- ch provides us with a basic of real-time pricing systems.

This paper is organized as follows. In Section 2, the outline of real-time pricing systems is explained. In Section 3, a model of electricity customers is derived. In Section 4, the optimal control problem is formulated, and its solution method is derived. In Section 5, a numerical simulation is shown. In Section 6, we conclude this paper.

Notation: Let

symbol

under the event

In this section, we explain the outline of real-time pricing systems studied in this paper.

In this paper, the status of electricity conservation of each customer is modeled by a Markov decision process (MDP). Then a set of customers is modeled by a multi-agent MDP (MA-MDP). Furthermore, by using the obtained MA-MDP model, we consider the optimal control problem and its solution method.

Illustration of real-time pricing systems

First, consider modeling the dynamics of each customer by a one-dimensional MDP. The value of the state

is randomly chosen among the finite set

electricity conservation, and “

where

the probability that the state is

The transition probability matrix

The control input is determined under the condition for each element:

and the condition for each column:

Next, consider modeling the dynamics of a set of customers by an MA-MDP. The number of customers is

given by

given by

Then, we suppose that the MA-MDP model expressing the dynamics of a set of customers is given by

where

condition:

For simplicity of discussion, coupling terms are given by

some condition corresponding to (5).

Consider the following problem.

Problem 1. Suppose that for the MA-MDP model (4) expressing the dynamics of customers, the initial state

input sequence

subject to the following constraint:

where

Hereafter, for simplicity of notation, the condition

By using the constraint (7), the input constraint such as

adjusting

We derive a solution method for Problem 1. First, consider the MDP model (1). The MDP model is a class of nonlinear systems. However, in this case, it can be transformed into a linear system. The MDP model (1) can be rewritten as

where

By the property of the probability distribution, the relation

where

Next, by using the linear system (8), consider representing the MA-MDP model (4) as a linear system. The linear system for the customer

Then, the MA-MDP model (4) can be equivalently transformed into the following linear system:

where

Finally, consider the cost function (6). Define

Then we can obtain

Therefore, the cost function (6) can be rewritten as

From the above discussion, Problem 1 is equivalent to the following problem.

Problem 2 is reduced to a quadratic programming (QP) problem, and can be solved by a suitable solver such as MATLAB and IBM ILOG CPLEX. In addition, if

ming (LP) problem (we remark that

Since it is difficult to use data in real systems, we present an artificial example. The state is chosen among the

finite set

system for the consumer

The parameters

The parameters

From

addition, the input constraint

In this numerical example, we consider the following two cases:

• The price for each customer is the same (i.e.,

• The price for each customer is different.

Case (i) is the conventional case in real-time pricing systems. In Case (ii), we suppose that the difference in the price is covered by using local concurrencies such as the Eco-point point system [

Next, we present the computational results. First, the computational result in Case (i) is explained. Figures 2-6 show the probability distribution for each customer. From these figures, we see that

electricity maximally, with a certain probability. Furthermore, the optimal value of the cost function is

π^{1}(t) in Case (i)

π^{2}(t) in Case (i)

π^{3}(t) in Case (i)

π^{4}(t) in Case (i)

Next, the computational result in Case (ii) is explained. Figures 7-11 show the probability distribution for

each customer. Comparing Figures 2-6 with Figures 7-11, we see that transient responses of

improved in Case (ii). In particular, for the customer

π^{5}(t) in Case (i)

π^{1}(t) in Case (ii)

π^{2}(t) in Case (ii)

optimal value of the cost function is improved. The optimal control input

is derived as

π^{3}(t) in Case (ii)

π^{4}(t) in Case (ii)

From these values, we see that in the steady state,

In this paper, we have proposed a modeling method and an optimal control method of real-time pricing systems using the MDP-based approach. In many cases, the status of electricity conservation of customers is discrete and

π^{5}(t) in Case (ii)

stochastic, and the use of the MDP model is effective. A real-time pricing system is modeled by multi-agent MDPs, and the optimal control problem is reduced to a QP problem. Furthermore, a numerical simulation has been shown. The proposed method provides us with a new method in real-time pricing of electricity.

There are several open problems. First, it is important to develop the identification method of the MA-MDP model based on the existing result (see, e.g., [

This research was partly supported by JST, CREST.