ABSTRACT

Honey Bees Mating Optimization (HBMO) is a novel developed method used in different engineering areas. Optimization process in this algorithm is inspired of natural mating behavior between bees. In this paper, we have attempted to createa new collaborative learning automata based honey bees mating optimization (C-LA-DHBMO).In previous model presented by very authors, the same learning automata parameters for all drones were used. However in the presented method, learning automatas with different reward and penalty parameters have been used which enhance reliability of algorithm and also has high convergence speed compared to previous proposed algorithm (LA-DHBMO). Simulation and comparisons based on several well-studied benchmarks demonstrate the effectiveness, efficiency and robustness of the proposed algorithms.

1. Introduction

Hybridization of intelligent techniques, coming from different computational intelligence areas, has become popular because of the growing awareness that such combinations frequently perform better than the individual techniques coming from computational intelligence [1]. Therefore, in this paper a new collaborative algorithm based on honey bees mating optimization and Learning Automaton has been developed.

A very interesting swarm in nature is honey bee swarm that allocates the tasks dynamically and adapts itself in response to changes in the environment in a collective intelligent manner. Honey Bees Mating Optimization (HBMO) algorithm is a typical swarm-based approach to optimization, in which the search algorithm is inspired by the honey bees mating process. The Honey Bees Mating Optimization Algorithm was first presented in [2,3], and since then it was used on a number of different applications [4-6]. HBMO is an evolutionary computation algorithm which simulates the mating process of the queen of the hive. The mating process of the queen begins when the queen flights away from the nest performing the mating flight which the drones follow the queen and mate with her in the air.

Learning Automaton (LA) is a general-purpose stochastic optimization tool, which has been developed as a model for learning systems. They are typically used as the basis oflearning systems, which through interactions with a stochastic unknown environment learn the optimal action for that environment. The learning automaton tries to determine, iteratively, the optimal action to apply to environment from a finite number of actions that are available to it. The environment returns a reinforcement signal that shows the relative quality of action of the learning automaton. This signal is given to learning automaton and learning automaton adjusts itself by a learning algorithm [7,8]. In previous model presented by very authors [9], learning automata parameters were considered constantly which were not suitable enough and it was possible that the final answer might depended on learning automatas parameters value. In new proposed algorithm, different parameters have been attributed to each automata which can produce better Answers by algorithm.

The results of computer sihe rest of this paper is organized as follows: In the next section a short review honey bees mating optimization is presented, Section 3 describes the Learning Automata. mulations show that the proposed algorithm attains better solutions in a faster way for most of the problems. T in Section 6, a proposed algorithm is described in detail. Section 5, indicates experimental setting to show proficiency of suggested algorithm, Section 6 presents results on the benchmark optimization functions. Section 7 contains the concluding remarks and the future work.

2. Honey Bees Mating Optimization

A very interesting swarm in nature is honey bee swarm that allocates the tasks dynamically and adapts itself in response to changes in the environment in a collective intelligent manner [10]. Before presenting the algorithm describing the use of honey bee mating behaviors, behavior of the colony is explained below: Bees are social insects living as colonies. The mating between honey bees includes three different bee categories: queen bee, drones and workers [2,3,10].

Queen bee is the only egg-laying female who is the mother of all the members of the colony. The queen usually mates only once in her life and she stores sperms in spermatheca and use them to produce broods. Drones are the fathers of the colony. Drones die after they mate with the queen. This process has been simulated in the model as a search of global optima. Workers are responsible for conserving the produced bees during mating process between queen and drones. Workers search on produced answers locally in simulated model. The main process in honey bees mating is mating flight. It begins with a dance of queen. During the flight the drones follow the queen and mate with her in the air. Sperm of the drones will be deposited and accumulated in the queen’s spermatheca to form the genetic pool of the potential broods to be produced by the queen.

2.1. Honey-Bees Modeling

The mating-flight may be considered as a set of transitions in possible solutions where the queen moves between the different states in some speed and mates with the drone encountered at each state probabilistically. At the beginning of flight, each queen is initialized by an amount of energy and if this amount reaches a threshold or Zero, or even spermatheca has been filled, the queens will return to the nest. In this algorithm, workers’ task is watching broods. In developed algorithm, workers are implemented as heuristic functions which cause fitness of broods to be increased. A drone mates with a queen probabilistically using below function as [2,3]:

where Prob(Q, D) is the probability of adding the sperm of drone D to the spermatheca of queen Q (that is, the probability of a successful mating); ∆(f) is the absolute difference between the fitness of D (i.e. f(D)) and the fitness of Q (i.e. f(Q)); and S(t) is the speed of the queen at time t. It is apparent that this function acts as an annealing function, where the probability of mating is high when either the queen is still in the start of her mating-flight and therefore her speed is high, or when the fitness of the drone is as good as the queen’s. After each transition in space, the queen’s speed, S(t), and energy, E(t), decay using the following equations:

where α is a factor [0, 1] and γ is the amount of energy reduction after each transition. Thus, Honey Bees Mating Optimization (HBMO) algorithm may be constructed with the following five main stages [4]:

1) The algorithm starts with the mating-flight, where a queen (best solution) selects drones probabilistically to form the spermatheca (list of drones). A drone is then selected from the list at random for the creation of broods.

2) Creation of new broods (trial solutions) by crossoverring the drones’ genotypes with the queen’s.

3) Use of workers (heuristics) to conduct local search on broods (trial solutions).

4) Adaptation of workers’ fitness based on the amount of improvement achieved on broods.

5) Replacement of weaker queens by fitter broods.

The algorithm starts with three user-defined parameters and one predefined parameter. The predefined parameter is the number of workers, representing the number of heuristics encoded in the program. The three user-defined parameters are the number of queens, the queen’s spermatheca size and the number of broods that will be born by all queens.

3. Learning Automata

Learning Automata are adaptive decision-making devices operating on unknown random environments. The Learning Automaton has a finite set of actions and each action has a certain probability of getting rewarded by the environment of the automaton. The aim is to learn to choose the optimal action (i.e. the action with the highest probability of being rewarded) through repeated interaction on the system. If the learning algorithm is chosen properly, then the iterative process of interacting on the environment can be made to result in selection of the optimal action. Figure 1 illustrates how a stochastic automaton works in feedback connection with a random environment. Learning Automata can be classified into two main families: fixed structure learning automata and variable structure learning automata (VSLA) [7,8].

In the following, the variable structure learning automata is described. A VSLA is a quintuple <α, β, p, T(α, β, p)>, where α, β, pare an action set with s actions, an environment response set and the probability set p containing s probabilities, each being the probability of performing every action in the current internal automaton state, respectively. The function of T is the reinforcement algorithm, which modifies the action probability vector p with respect to the performed action and received response. Let a VSLA operate in an environment with β = {0, l}. Let be the set of nonnegative integers. A general linear schema for updating action probabilities can be represented as follows. Let action i be performed. If β(t) = 0 (Reward),

If β (t) = 1 (Penalty),

where a and b are reward and penalty parameters. When a = b, automaton is called L_RP. If b = 0 the automaton is called L_RI and if the automaton is called L_REP.

4. Collaborative Honey Bees Mating Optimization Based on Learning Automata

In previous presented paper [9] we introduced a new algorithm based on learning automata is suggested. In this model, each drone has been attributed to a set of learning automata which are responsible for producing better drones. As a result, producing better drones and their mating with queen can produce better broods and it causes to converge faster close to global optima. In this algorithm each drone consists of two parts, Model genome and string genome. Model genome is a set of learning automata. The set of actions selected by the set of learning automata determines the second component of the genome called string genome which a reinforcement signal is produced after mating flight and replacing queens with better broods for every automaton. Each learning automaton based on the received signalupdate its internal structure according to a Learning algorithm. Then each drone generates a new string genome and compares its fitness with the fitness of the string genome of the drone. If the fitness of the generated genome is better than the quality of the sting genome of the drone, the generated string genome becomes the string genome of that drone. This process of generating string genome by the drones is iterated until a termination condition is satisfied. There is a snapshot of Honey Bees Mating Optimization based on Learning Automata presented in Figure 2.

We are going to introduce a new algorithm called Collaborative Honey Bees Mating Optimization based on Learning Automata (C-LA-DHBMO) in this paper. Unlike previous model, in this model learning automatas parameters are not the same and various parameters for learning automatas are considered. The algorithm routine can be defined as this: high parameters are chosen for half of drones and for the rest of them, low parameters will be chosen. In characterized repetition, string genome of each drone with high and low parameter will be changed with each other. This action causes learning automatas not to converge to a specific action and produce better result compared to the first proposed algorithm. The collaboration between drones is presented in Figure 3.

Proposed algorithms as the one described algorithm in this paper can be used in any arbitrary finite discrete search space. To simplify the algorithm, we assume that sight search space is a binary finite search space. So the optimization problem can be presented as follows. Assume be a real function that is to be minimized.

In order to use the algorithm for the optimization function f first a set of learning automata is associated to each drone. The number of learning automata associated to a drone is the number bits in the string genome representing points of the search space of f. Each automaton has two actions called action 0 and 1. After each mating flight, these set of steps are done:

1) Every automata in a drone_i chooses one of its actions using its action probability vector.

2) drone_i generates a new string genome, newⁱ, by combining the actions chosen by the learning automata of drone_i. The newly generated string genome is obtained by concatenating the actions of the automata (0 or 1) associated to that drone. This section of algorithm is equivalent to learning from previous self-experiences.

3) Every drone_i computes the fitness value of string genome newⁱ; if the fitness of this string genome is better than the one in the drone then the new string genome newⁱ becomes the string genome of that cell. That is.

4) Se queens of the hive are selected. This Selection is based on the fitness value of the queens in hive.

5) Based on selected queens a reinforcement vector is generated. This vector becomes the input for the set of learning automata associated to the drones. This section of algorithm is equivalent to learning from experiences of others. Let Q_s be set selected queens:

where

β^i,j, the reinforcement signal given to learning automaton j of drone i, is computed as follows:

where u(·) is a step function. The proposed algorithm is summarized in the Pseudo code of Figure 4.

5. Experimental Setting

In this section, the efficiency of proposed algorithm is presented. For this purpose, Standard functions borrowed from reference [11] are used to show proficiency of suggested algorithm. In the Table 1, functions with their characteristics are described (All experiments have been done on 10 dimension functions).

The parameters used for C-LA-DHBMO, LA-DHBMO and DHBMO consist of the number of queens set to 5, the spermatheca capacity set to 5, the maximum number of produced broods set to 30, and also the number of workers suggested in paper [2,3] set to 4. Learning automata L_RP has been used for algorithms LA-DHBMO and C-LA-DHBMO and they were performed separately with values R = P = 0.01 and R = P = 0.001 for LA-DHBMO and they are shown in results as LA-DHBMO_{RP (0.01, 0.01)} and LA-DHBMO_{RP (0.001, 0.001)}. Automatas which owns reward and high penalty parameters with values of R_H = P_H = 0.01 and also the ones which owns reward and low penalty parameters with values R_L = P_L = 0.001 are used for C-LA-DHBMO. They are shown as C-LA-DHBMO_{RP (0.01, 0.01, 0.001, 0.001)} in results.

6. Results

To demonstrate reliability of algorithm, the results of algorithm performance on functions of table I have been shown in Tables 2-5. The results indicate better and closer answers to global optima.

To demonstrate high speed of convergence and also escaping from local optimas, acomparison between previous methods and proposed method on functions of Table 1 has been shown in Figures 5-8.

7. Conclusion

In this paper we have strived to present a new model to

improve effectiveness and convergence speed in optimization problems. The new model presented by using learning automata and the concept of collaboration between automatas. Unlike former model, proposed model used learning automatas with different parameters such as reward and penalty. As it is shown in experiments, the presented algorithm has got excellence compared to previous ones. In future researches we will attempt to provide contact possibilities between searching agents to use them better.

REFERENCES