ABSTRACT

Contract Net Protocol (CNP) has been widely used in solving the problems of multi-Agent cooperates and reservoir flood control optimization dispatch. This paper designs an Agent functional module based on the multi-Agent coordinated the construction of reservoir flood control optimization dispatch and the corresponding Agent to solve the problem of classical CNP in the Agent communication aspect, to enhance the efficiency of reservoir optimization dispatch and to improve the insufficiency of the classical CNP in the application of reservoir flood control dispatcher. Then, the capacity factor and the cooperation level are introduced in the module. Experiments are conducted on the Agentbuilder simulation platform to simulate reservoir flood control optimization dispatching with the improved CNP. The simulation results show the communication interactive efficiency and the performance of new protocol is superior to those of the classical CNP.

1. Introduction

Multi-Agent technology has been widely used in complicated system, which plays a role in solving problems coordinately [1]. The reservoir is used as an effective measure to dispatch the water resources, not only can satisfy irrigation, water supply, and shipping requests in the society, but also can satisfy the flood prevention function [2]. The system of reservoir flood control optimal operation process is built based on the multi-Agent technology, and uses the Agent cooperation during the operation to solve water resources conflicts among the various departments. It has the significance meaning to raise the reservoir optimization dispatch efficiency.

The multi-Agent cooperation strategy is the foundation of Agent interaction and implementation of the function module in the multi-Agent system [3]. Therefore, the multi-Agent cooperation strategy has determined the overall performance of multi-Agent system to a great extent. Because of the multi-objective, constraints and other characteristics of reservoirs dispatch, there are many deficiencies and limitations in the traditional method for the reservoirs optimal operation. It is very essential to design more feasible and effective multiAgent cooperation strategy for the characteristics of the reservoir system [4].

At present, the domestic and foreign researchers already have proposed many kinds of reservoir optimization dispatch algorithm, which may mainly divide into three categories: 1) system programming algorithm. These algorithms contain fuzzy programming [5], dynamic programming [6], etc. 2) conventional algorithm, which includes calendar algorithm [7], and algorithm of average [8], etc. 3) intelligent optimization algorithm, which includes multi-Agent [9,10], artificial neural networks [11], and genetic algorithm [12]. Compared with conventional algorithm and system programming algorithm, the intelligent optimization algorithm has the obvious superiority in solving the reservoir dispatch and been an active research topic. Contract Net Protocol (CNP) in the multi-Agent method proposed by Smith comes from the use of the people in the commercial process to manage the commodity and the service contract mechanism [13]. Multi-Agent Optimization dispatch method with CNP can overcome the shortcomings of other methods, which has huge model and complex operation in the reservoir, and it can be a good description of the actual situation during the operation. Because of the characteristic of fully complex, changeable and the dynamic in the reservoir dispatch, the dispatch result may be satisfied for users. In addition, there is much solution to the reservoir optimization dispatch with CNP, which has a good application prospect.

Because CNP does not limit the scope of the issuing bidding documents or evaluate the complete situation of Agent, the efficiency of interactive communication is not high in the applications of reservoir operation optimization. In order to overcome this problem, this paper firstly designs an Agent function module based on multi-Agent coordination. Moreover, the capacity factor and the cooperation rank have been introduced in the improvement CNP, and the simulation results show the interaction of the system efficiency and performance of communication than the classical CNP.

2. Reservoir Optimal Dispatch Framework Based on Multi-Agent

Taking into account the structure of internal and external constraints in the reservoir dispatch, we propose a reservoir dispatch framework based on multi-Agent in this work, which is shown in Figure 1. The whole framework can be divided into five parts: 1) Government Agent. 2) Reservoir dispatch Agent. 3) Water use Agent. 4) Constrained Agents group. 5) Information Agent.

2.1. Government Agent

The role of Agent in the system corresponds to the actual management in the river basin. For example, the government departments in the river basin management coordinate the process of the reservoir dispatch and make the goal of macro-control. The role of government Agent in the Multi Agent systems is also same as above. The main duty of government Agent is to allocate the tasks for Reservoir dispatch Agent in the river basin. When an Agent can not work independently for a certain task, government Agent can make it collaborate with the other Agents. Meanwhile, government Agent can assist to complete the communication task between Agent and Agent group so as to reduce communication costs. In addition, government Agent can solve the conflicts between the Agents.

From Figure 1, it can be seen that government Agent mainly plays the role in coordinating and managing the entire structure.

2.2. Reservoir Dispatch Agent

This is the main body of reservoir dispatch, which can be conceptualized to departmental management, such as hydropower development department, shipping management, gate control, and water supply departments. Reservoir dispatch Agent maximizes the benefits of their own goals by the coordination and control of government Agent. Moreover, it also reflects information exchange between departments.

From Figure 1, we can see that reservoir dispatch Agent is the main body of the whole structure, which is responsible for reservoir water dispatching. When a reservoir dispatch Agent can’t complete the task, it will be with the collaboration collaborative Agent of the Constrained group Agents to complete the local task. Besides, the data of reservoir dispatch Agent can be updated by information Agents and databases.

2.3. Water Use Agent

This is the terminal Agent in the established model, but in the real world, there are no terminal of the dispatching and utilization of water resources. In the proposed framework, the whole water use Agents are divided into four Agents, namely: industrial water use Agent, agricultural water use Agent, domestic water use Agent and ecological water use Agent.

From Figure 1, it can be seen that the terminal of the entire structure consists of four water use Agent groups. The management of the reservoir is not involved, which will feed back the information to the reservoir dispatch Agent and the government Agent when it achieves its goals.

2.4. Constrained Agent Group

According to the basic principles of sustainable development of reservoir dispatch, it can not ignore the surrounding natural resources and cultural resource protection when Agents are used to meet water users and management to maximize the benefits in the process of reservoir operation. Therefore, the constrained Agent groups can be extracted from multi-Agent framework and include land resources Agent, forest resources Agent, wildlife Agent, scenic spot and historic resort Agent, watershed regulations Agent, channel protection Agent and environmental regulations Agent.

Constrained Agent groups are not involved in the dispatch of the reservoir. However, it is responsible for the management of various natural and human resources in the catchments under the jurisdiction of reservoir. It can bound indirectly reservoir Agent, and change the dispatching way, so that the dispatch of water resources is more reasonable while meeting the principles of sustainable development.

2.5. Information Agent

Information Agent mainly plays the role of classifying, filtering and screening information in the multi-Agent framework, stores data in the database according to a certain format.

From Figure 1, we can see that the proposed framework has the following features. In the reservoir dispatching process, the different components of each level can be abstracted to a certain intelligence Agent. Each Agent has its own organizational structure, behavior and management strategies, and interacts with other Agents in the system and their environment accordance with its individual goals and rules. Meanwhile, a number of Agent consist of Agent group which has similar rules and consistent goal. The group also has some interaction so as to come into being a large complex organization structure.

3. Strategic Collaboration among Multi-Agents

CNP is the most classic one among multi-agents collaboration strategic, which brings “bid invitation, bid tendering, bid winning” mechanism of market economy into the process of task allocation and can effectively solve the conflict on resource and knowledge among different agents. As a result, it is also an effective solution to problems of interactive and cooperative which exist in reservoir scheduling architecture established in this work. However, the classical CNP has the following two shortcomings:

1) No limit in bid document In classical CNPs, Agents which initiate task need to broadcast all the bidding information to all other Agents of the global system or the local system. These Agents can participate in the tender, but many of them do not have the ability to complete the tasks even the subtasks. This case leads to the waste of bid document, frequent communication and large consumption of system resources, so it can be seen that the release of bid document without selection results in communication congestion and nodes overload.

2) Lack of quality evaluation on the implementation of tasks for Agents which has won the bids In classical CNPs, when the two sides have signed the contract and completed the task, Agent which has won the bids will inform the Agent which initiates task about the task results. At this time the two sides terminate the contract, so the collaborative process is completed. However, during the whole process Agent which initiates task doesn’t give the evaluation of task quality. In this case, for Agent which initiates task each Agent only has difference on division of tasks instead of the level of capacity. As a result, occurrence of task finished with low quality will arise in an intelligent system. If this case occurs frequently, the activity and capacity of solving problem will be reduced.

3.1. The Improved CNP

To overcome the shortcomings of classical CNP existing in the application of reservoir scheduling system, two improved methods are presented in this paper:

1) Evaluate the ability of bidders In order to distinguish the capacity level of completing task and improve the activity of multi-Agent system, initiated task Agent gives evaluation on task situation after the completion of each task, and stores information about its ability. Moreover, the information should update continuously and dynamically adjusted by the task situation in order to provide ability information for all Agents which are going to invite the bids. As a result, the quality of task completion can be improved.

In order to reflect the ability of Agent and improve the quality of task completion, this paper introduces capacity coefficient and collaboration rank in the bidding process of CNPs.

a) Capacity coefficient In multi-Agent system, evaluation on the completion of task T for Agent a is called the capacity coefficient of Agent a on T, which is denoted by CoLevel (a, T). It is measured by the effect of task completion. Capacity coefficient of each Agent arises during the process of cooperation, and the higher the capacity factor, the greater the likelihood of the tender side to entrust the task to it. If Agent a can successfully complete the task entrusted by the task initiators which are usually the management Agent in the system, and can finish the task and reach a satisfied standard of initiator, then capacity coefficient of Agent a will increase. With the increase of capacity coefficient, Agent a can be considered with higher priority in the bidding process. On the contrary, if Agent a can not successfully complete the task or can not come to satisfied standard, then capacity coefficient of Agent a will decrease and be avoided when possible to take similar tasks in the future.

The calculation of capacity coefficient is showed as follows: the initial value of capacity coefficient on task T for each Agent is set before collaboration. Then, its capacity coefficient is increased by after the completion of task, which can be computed by

(1)

Otherwise, its capacity coefficient is decreased by, which can be computed by 

(2)

In the formula (1) and (2), is set to be smaller than. This means that severe punishment is given to Agent which has not finished task or can not come to the standard by decreasing its capacity coefficient substantially in order to avoid selecting such Agents which are lack of capacity.

b) Cooperation level Besides capacity coefficient, cooperation level is introduced to the Agent, which is denoted as CoLevel (a, T). Cooperation level indicates the ability of to collaborate with other Agents. The main factor affecting CoLevel (a, T) is the number of successful completion of the task, and the larger the number is, the bigger the value of CoLevel (a, T) is. So CoLevel (a, T) is the key indicator of selection for the bidding Agent in the first time.

The computation of cooperation level is similar to capacity coefficient. Cooperation level increases in database when in cooperation, and decreases in independence. Besides, Cooperation level decreases when the task has not been finished. These three cases can be depicted by the follow formulas,

(3)

(4)

(5)

In the above formula, is set to be smaller than, that is, severe punishment is given to Agent which has not finished task by decreasing its cooperation level substantially and the cooperation level is slightly decreased for Agent which has finished task without corporation. In this situation, when the initiators of task come to the same problem, they will not choose the Agent with high failure rate of collaboration.

2) Limitation of the tender issued If the task initiating Agent before the release can roughly grasp the information related to Agent in the system, then the tender can establish a list of the candidate Agents before the start of a bid. Then the information of tender can be only transmitted with broadcasting to the Agents in the list. Such primary screening, not only may reduce the traffic load in the network, but can also reduce the tender processing for sponsor Agent.

For the selection of bids Agent, we can divide tender Agent into three categories by the latest information of ability: Agent I, Agent II, and Agent III. They satisfy the following conditions:

Agent I, Agent II, and Agent III represent having ability to complete the task, and are in the list of Agent in duty, having ability to complete the task but not in the list as well as having no ability to complete the task Agent, respectively. Three bids Agent can be defined as:

(6)

(7)

(8)

where,

expresses that Agent can complete task,

, expresses that Agent can not complete task, in such case, the collaboration coefficient are not considered no whatever it can meet the standardsL is a constant, which is established by manager Agent according to system’s specific case and is the standard to evaluate the Agent ability. If L is 0, the cooperation rank is not considered. In such case, only the value of the capacity factor is judged to be True or false, CoLevel (a, T) ≥ L expresses cooperation rank satisfying the requests of the duty Agent, otherwise the cooperation rank is smaller than L, which does not give the consideration in primary screening.

Therefore, at the beginning of the CNP establishment in the multi-Agent system, management Agents are given the appropriate value of L based on the specific circumstances of the task. Then, bids Agent can complete a preliminary screening before the release of tenders according to the information of the Agent database. Moreover, because some Agents are unable to obtain the contract in this process, the cooperation coefficient may be unable to obtain the promotion and lead to the collaboration only existing between the Agents with very high collaboration coefficients. This can also cause the system less active. In order to avoid this case, average collaboration coefficient will be considered in the preliminary screening process and few Agents will also be selected in Agent II.

3.2. Design of Improved CNP Algorithm

Because individual Agent is unable to complete the task independently in the construction of reservoir flood control optimization dispatch, the task is decomposed to tender. This Agent is the task initiated Agent. The specific process is as follows:

1) Preliminary screening of bidder’s: The preliminary screening is carried on to all Agents through the existing screening strategy and the existing Agent information.

2) Tender: The task initiated Agent through the broadcast way transmits bidding documents to the Agent which was selected in the first step, namely bids Agent. To make the process of communication effective and smooth, it needs to unify the format of the bidding documents. Its structure is as follows:

(9)

whereTask ID: Mission number, the only marking for task;

Manager ID: Task tender side ID;

Task Content: The task definition explanation, this is one of evidences that the bids Agent decide whether to bid;

Task Constraint: The task tender side stipulates the condition the bid side should satisfy;

Expire Time: The maximum time of response bid.

3) Bid: After receiving the bidding documents, bid Agent must first check two items of the tender “Task Content” and “Task Constraint”, then according to their own capacity coefficient to determine whether to complete the task. If the capacity coefficient can satisfy the conditions, it should determine whether to tender upon their current load, the acceptance of the constraints on the task. If it decides to tender, the tender documents should give the corresponding response according to the constraints.

4) Evaluation of Bid: The task sponsor will receive many bids application by the bidder. After assessment of bidders, one of the most appropriate bidders will be selected to assign the task. This paper introduces the comprehensive index as the selection criteria.

The comprehensive index formula is as follows:

(10)

where, is weighted factor of ith index, is the function of the ith index (). The weighted factor may be changed dynamically according to the needs of different dispatch objectives.

5) Win the Biding: The most suitable bidder is selected for the execution of tasks by Agent. Then the task initiator sends the tender notice to the selected Agent and establishes a formal contract with the selected Agent.

6) Termination of the contract: The selected Agent sends the task executed result to the task initiator Agent and the result is evaluated by the task initiator. If the task completion is satisfied, then the selected Agent capacity coefficient and the collaboration level will be updated and the task will end. If it is unsatisfied, then the result will be given up, and the capacity coefficient and the collaboration level will be also updated. The task will be redistributed, or given to the high-level management Agent for processing.

4. Simulation of Multi-Agent Cooperation Process

The simulation experiment is based on the construction of reservoir flood control optimization dispatch, which is shown in Figure 1. The analog simulation is carried on between the Agent cooperation processes. The process result with improved contract net’s cooperation will be compared to the process result with the classical contract net’s cooperation.

In the simulation process, the water used household Agent involves in domestic water Agent, cooperated Agent, reservoir dispatcher Agent, information Agent and government Agent. The life and the agriculture water used household have different demand for the water quantity. Cooperated Agent is responsible for the water used household Agent to accept the task and to select the contract. There are three Agents which are responsible to supply the water to the water used household in the reservoir dispatcher aspect. The information Agent is primary responsible for the data interaction, including providing the screening of the information and the read of capacity factor and the cooperation level initially. Government Agent is responsible for the administrative intervention on task when cooperated Agent is unable to complete the task. In addition, because in the reservoir dispatching process’s multi-objective characteristic, scenic spot and historic resort Agent and wild animal Agent in the constraints Agent are introduced, which are responsible to limit the lowest water level and the lowest water temperature in the water supply process, respectively.

4.1. Selecting Index of Reservoir Dispatcher Agent

We must first set the correspondence value of reservoir dispatcher Agent’s own coefficient and the constraint condition in the database; In Table 1, we set three reservoir dispatcher Agent targets.

After cooperated Agent receives the bid parameter of reservoir dispatcher Agent, because there are the different dimension parameters for the lowest water level and the lowest water temperature in the water supply process, appropriate reservoir dispatcher Agent cannot be chosen directly according to these two indexes, which are used to calculate a comprehensive index to instruct the choice of cooperated Agent. A quite practical method is to carry on non-dimension standardization processing to the lowest water level and the lowest water temperature in the water supply process. Non-dimension standardization processing is one kind way of eliminating the influences of original variable through the mathematical manipulation. This article carries on the non-dimension standardized processing according to the formula (11).

(11)

where and represent the target value in the row the column in Tables 1 and 2.

The weighting factors are determined for the lowest water level and the lowest water temperature according to their different requirements, respectively. Multi-objective’s optimized questions are transformed to single target’s optimized question according to the formula (12).

(12)

where is the index scaling coefficient, is

the overall index of the scheme.

4.2. Analysis of Simulation Results

The experiments are carried on the Agent Builder 1.4 platform, which is an integrated development tool for intelligence Agent developed by American Reticular Corporation. It is composed of two parts: Toolkit and Run-Time System. The Toolkit includes the complete tools which are used during the Agent software development process, such as the tool analyzing the Agent operation domain, the tool designing and developing the communicated Agent network, the tool defining individual Agent behavior, the testing and the debugging aids. In the run-time system the main part is an Agent engine, which provides a specific environment to carry out the Agent software.

The experiments are carried out by two strategies: 1) The classical CNP without capacity coefficient and collaboration level, issuing bidding documents to all Agents by broadcast [14]; 2) Improvement CNP by setting the constraint conditions such as capacity coefficient and collaboration rank, only to tender the eligible reservoir dispatcher Agent.

The water demand and water supply of domestic water Agent are simulated based on the improved CNP and the classical CNP. The constraints Agent for domestic water Agent requires the lowest water level and the lowest water temperature to be 11 m and 7˚C, respectively. The simulation results are shown in Figure 2 and Figure 3.

In order to display the entire reservoir flood control dispatcher process of the Agent cooperation, the communication process is continuously captured. Each subimage of Figure 2 mainly contains two parts. The upper part stands for the optimal process, where each icon in the upper part represents one Agent. There are 8 groups of Agent, which contains three reservoir flood control dispatcher Agent (, ,), one cooperates Agent (), one information Agent (), one domestic water Agent (), and two constraint group Agents: wild animal Agent ()and scenic spot and historic resort Agent (). The bottom part expresses the program execution.

It can be seen from Figure 2 that cooperated Agent sends message to information Agent to preliminary screen the existing reservoir dispatcher Agent (Figures 2(a) and (b)) after receiving the water task and the water supply constraints of the water used household. Then, the tender notice is issued to Agent 1 and Agent 3, which have the dispatch water ability (Figure 2(c)). According the computation result of formula (12), Agent 3 is selected as the final choice of the task holder, and the capacity factor of Agent 3 is updated through information Agent (Figure 2(d)).

Figure 3 shows the steps of reservoir dispatching process of classic CNP. We can see from the Figure 3(b) that cooperated Agent transmits the tender notice to all Agents including scenic spot and historic resort Agent and wild animal Agent which have inability to complete the task Agent after receiving the water used household duty and the constraints. In the Figure 3(c), Agent 3 which is the final choice of the task holder does not update the capacity coefficient of the task holder to information Agent.

In order to increase the persuasiveness of simulation, the process using the same way to agricultural water Agent and industrial water Agent of the water needs and the water supply process are simulated and compared. When constraint conditions are changed and water is supplied to the agricultural water Agent, the lowest water level and the lowest water temperature are set to be 11.9 m and 7.5˚C for constraints Agent. When industrial water Agent needs the water, the lowest water level and the lowest water temperature will be set to be 9.9 m and 11˚C, and two groups of simulation experiments will be established.

Simulation 1: In the Agent Builder, the number of communication among the Agents is recorded by adding a counting class. Table 3 reflects the interactive times of two kinds of different cooperation strategy Agents. From Table 3, it can be seen that the interactive number of water need main body with the improved contract net is obviously lower than that of the classical contract net.

Simulation 2: In the experiment the class of timing function is be added, which is used to record the communication time between the Agents. Before the beginning of each tender, Agent will call this kind of update Start Time () function to record the time. At the end of the bidding, Agent will automatic call the update End Time () function and record closure time. Finally Agent calls the get Time Interval () function to obtain the total time of this task. Table 4 is the comparison of mutual time between the improved CNP and the classical CNP. From Table 4, it can be seen that the interactive time of water main body with the improved contract net is obviously lower than that of the classical contract net.

From the Tables 3 and 4, we can see that the communication load and interactive time are improved in the task mutual number of times. The improved CNP selects the bid strategy based on the capacity factor, in which bidding documents are only transmitted to the high capacity factor Agent, and lightens each Agent module burden and reduces the correspondence number of times. Based on the introduction of updated capacity factors, the high or low ability of each Agent can be distinguished

and the efficiency of reservoir operation in the Agent is improved.

5. Conclusions

This paper designs a reservoir flood control scheduling framework based on Agent collaboration, and the correlative Agent function module is built according to properties of one reservoir in reality. In order to solve the problem of collaboration between function modules in the process of the reservoir’s optimization and scheduling by the improved CNP, its communication interactive efficiency is advanced and the reservoir’s scheduling efficiency is also increased. The simulated experiment in the above process is implemented, and the result shows that improved CNP is reduced in terms of the information interaction frequency compared with classical CNP, and interactive time, and communication interactive efficiency is higher than classical CNP.

REFERENCES