The Differential Evolution (DE) algorithm is arguably one of the most powerful stochastic optimization algorithms, which has been widely applied in various fields. Global numerical optimization is a very important and extremely dif-ficult task in optimization domain, and it is also a great need for many practical applications. This paper proposes an opposition-based DE algorithm for global numerical optimization, which is called GNO2DE. In GNO2DE, firstly, the opposite point method is employed to utilize the existing search space to improve the convergence speed. Secondly, two candidate DE strategies “DE/rand/1/bin” and “DE/current to best/2/bin” are randomly chosen to make the most of their respective advantages to enhance the search ability. In order to reduce the number of control parameters, this algorithm uses an adaptive crossover rate dynamically tuned during the evolutionary process. Finally, it is validated on a set of benchmark test functions for global numerical optimization. Compared with several existing algorithms, the performance of GNO2DE is superior to or not worse than that of these algorithms in terms of final accuracy, convergence speed, and robustness. In addition, we also especially compare the opposition-based DE algorithm with the DE algorithm without using the opposite point method, and the DE algorithm using “DE/rand/1/bin” or “DE/current to best/2/bin”, respectively.
Neural networks are widely used in the characterization of nonlinear systems [1-7], time-varying time-delay nonlinear systems [
The system may be with invariant parameters or timevarying parameters. The variation of some system may be such as; system with slow time-varying parametric uncertainties [13,14], with arbitrarily rapid time-varying parameters in a known compact set [
Using MLP architecture depends on various parameters, for instance the number of hidden layers, the number of neurons in each hidden layers, the activation function and the learning rate. These parameters present a difficulty to find the suitable architecture of the MLP.
A renewed interest in Radial Basis Function (RBF) neural network has been found in recent years in various application areas such as modeling and control [1-2], pattern recognition [
Using an RBF leads to a general model structure is less complex than that produced by an MLP network. The computational complexity induced by their learning is less than that induced by learning the MLP networks. The RBF network performance depends, to a choice of activation function [
Several methods such as iterative methods [28-32] with the gradient descent method and evolutionary algorithms [
This paper focuses on the optimization of radial basis functions architecture, and compares it to the MLP architecture. The proposed algorithms are applied to timevarying nonlinear systems. The RBF using genetic algorithms gave the best results.
This paper is organized as follows. Nonlinear system modeling by MLP and RBF network is presented at the second and the third section. A comparative study between the MLP and RBF model, applied to two examples of nonlinear systems is presented in the forth section. Conclusions are given in the fifth section.
Modeling a nonlinear system from its input-output can be for several models. Among these models, the NARMA (Nonlinear Auto-Regressive Moving Average) [
where is a nonlinear mapping, and are the input and output vector, and are the maximum input and output lags, respectively. In this paper, the coefficients of the model (1) depend on time.
The used MLP in this paper is to describe the nonlinear system (1). The objective of the modeling is to obtain an MLP that its output follows the output of the system.
The multilayer perceptron network consists of an input layer, an output layer and usually one or more hidden layers.
For the nonlinear system (1), if no knowledge about the structure of the nonlinearity of the system is available such system is considered as a “black box” system modeling.
The output of MLP model is given by the following equation:
where is a sigmoid activation function, is the output vector of MLP., , and are the synaptic weights of MLP. is the input vector of MLP.
Among the optimization methods of MLP, the gradient descent method is used in this paper. Optimization of the MLP is to minimize the mean square error E.
where is a function cost.
In this section, 2 hidden layers are taken into account with a single input layer and one output layer, the result of optimization is given by equations (4) to (10):
with:
As we did with the MLP model, the RBF is used to describe the nonlinear system (1).
The RBF consists of only three layers; an input layer, an output layer and usually one hidden layers contains a hidden radial basis function. The RBF model calculates a linear combination of radial basis functions as is given by the following equation:
where is the output vector of RBF. is the synaptic weights of RBF and is a Gaussian activation function:
Compared to the MLP, the RBF contains a very small number of parameters. The purpose of optimizing RBF is to determine, and by minimizing the function cost.
In order to find the minimum of, and two strategies are proposed in the literature for finding the minimum of. The first is based on supervised methods or algorithms using direct time-consuming calculation to determine the minimum of. The second adopts a hybrid scheme (less costly in computation time) to determine the minimum of. Solving these problems can be by various methods such as iterative methods (the Gradient Descent method) and evolutionary algorithms (Genetic Algorithm).
The principle of the GD method is applied to optimize the parameters of the RBF model. It uses the rules of delta:
Hence the calculation of partial derivatives introduced by the following equations:
finally, we obtain
The learning rate satisfies the following condition:
is nonlinear in the parameters, which calls for finding the minimum, the use of an iterative algorithm that requires an arbitrary initialization of RBF network parameters and a suitable choice of. To maximize the chances of finding the global minimum of, several initialization parameters and therefore more training is needed, which increases the computing time.
Another method is to optimize separately the parameters of the hidden layer (the centers and the widths) by genetic algorithms and the synaptic weights between the hidden layer and output layer by the gradient descent method.
The genetic algorithm is an optimization algorithm based on techniques derived from genetics and natural evolution: crossover, mutation, selection. The GA is often used for optimization of RBF [34-38].
In this paper, the GA is used in order to optimize separately the parameters of the hidden layer (the centers and the widths) of the RBF model.
To find suitable parameters, five elements of GA are called:
• A population is generated randomly. The population size is chosen to achieve a compromise between computation time and solution quality.
• The evaluation of each individual is performed by an evaluation function called fitness function. This function represents the only link between the physical problem and GA. In this paper, the used fitness function is given by the following equation.
with and are respectively the and the which are used also in the following equation:
• Once the evaluation of generation is realized, it makes a selection from the fitness function. In this paper, the tournament selection is used.
• The crossover operator is designed to enrich the diversity of the population by manipulating the genes of individuals existing in the population. In the other hand, the mutation operator involves the inversion of a bit in a chromosome. The mutation that mathematically guarantees the global optimum can be reached.
• The stopping criterion indicates that the solution is sufficiently approximate the optimum. In this paper, the maximum number of generations is chosen as stopping criterion.
The obtained parameters (,) by GA are used also in the following equation:
and the synaptic weights are calculated using the gradient descend method:
The effectiveness of the suggested methods applied to
the identification of behavior of two nonlinear timevarying systems are demonstrated by simulation experiments.
The performance of MLP and RBF models are evaluated by Normalized root Mean Square Error between the system output and the model output, denoted.
and
We consider the nonlinear time-varying system described by input-output model:
with:
The trajectory of and are given in figure 3.
The input is sinusoidal signal and it is defined by the following equation:
In figure 4, the time-varying system responses, the MLP model and the optimized RBF model by the GD method are presented. In this simulation figure, the MLP parameters are, , and . The obtained is. However, the RBF parameters are, and. The obtained is .
In figure 5, the time-varying system responses, the MLP model and the optimized RBF model by GA are illustrated. In this simulation the same MLP parameters are taken, while the RBF parameters are, , and. The obtained is.
In these two figures (4 and 5), it is clear that the responses of MLP and RBF models follow the system response although the variation of parameters.
In one hand, the obtained MLP model is found by several tests of parameters and of learning. The large number of MLP parameter increases the difficulty of its use. However, the simplicity of RBF makes modeling is simple and takes much less training time.
In figure 4, the optimized RBF model by the gradient descent method depends on an expensive time of training and depends on different learning rate (, and) while, in figure 5, the parameter of RBF model (and) are finding separately by the GA and the synaptic weights () by the DG method, hence the model is faster than the previous.
To validate the quality of the proposed algorithm, an added white noise is used. The influence of the noise of modeling, the Signal Noise Ratio (SNR) is used. The figures 6(a)-(c) present the evolution of different SNR for the both models.
where and are respectively the output average value and noise average value.
In these three figures 6(a)-(c), we remark firstly the error goes down when the SNR value goes high, then the lowest MSE is obtained when the GA is used (figure 6(c)). Finally, in these all figures we see that the responses of MLP and RBF models follow the time-varying system response despite of the variation of parameters and an added noise.
To test the effectiveness of the MLP and RBF models we test them on a Continuous Stirred Tank Reactor, CSTR, which is a type of slowly time-varying nonlinear system used for the conduct of the chemical reactions [39-41]. However, the input-output are used in discrete time. A diagram of the reactor is given in the figure 6. The physical equations describing the process are (34) and (35):
where is the height of the mixture in the reactor, (respectively) is the feed of reactant 1 (respectively reactant 2) and (respectively) is the concentration of reactant 1(respectively reactant 2). is the feed product of reaction and its concentration is., , and are consumption reactant rate. They are assumed to be constant. The temperature in the reactor is assumed constant and equal to the ambient temperature. The feed of reactant and the concentration are the input of the process however represents its output. A diagram of the reactor is given in the figure 7.
For the purpose of the simulations the CSTR model of the reactor provided with Simulink-Matlab is used.
In figure 8, the responses of the chemical reactor, the model that produced by MLP and optimized RBF model by the gradient descent method are presented. In figure 9, the responses of the chemical reactor, the MLP model and the optimized RBF model by genetic algorithms are illustrated.
In figures 8 and 9, the responses of the optimized RBF model by the GD method and MLP model follow the response of chemical reactor. Indeed, in figure 8, the MLP model is carried out with 5 neurons in input layer, 25 neurons in first hidden layer, 22 neurons in hidden layer and the learning rate equal to 0.4. However, the RBF model depends only 3 layers, the second and the only hidden layer contains 11 neurons. The used parameters in the GD method are, and. In contrary, in
RBF model are optimized separately (and) using GA and () using gradient descent. These parameters are, , and.
The NMSE is given in the table below:
From this table, the NMSE computed in the RBF model optimized by the genetic algorithms method is lower than that found by applying the gradient descent method which proves that the evolutionary algorithms give good accuracy for modeling methods of dynamical systems.
This paper has dealt with the study and the comparison of two systems modeling techniques the multilayer network model and the radial basis function neural network model. These two approaches are applied in a class of nonlinear system with time-varying parameters. It has been shown that the MLP architecture depends on various parameters and of course a much training time. However, the RBF model depends on the synaptic weights, center and width of its function. In this paper, the RBF model is optimized by gradient descent method and genetic algorithms. Each optimized RBF models are compared with multilayer perceptron. Mean square error is carried out to evaluate performance of both models and the influence of an additive noise on the identification qualities. These models have been tested for modeling of chemical reactor and results are successful. The RBF model optimized by genetic algorithms showed good performance compared to that optimized by gradient descent method.
: process output,
: process input,
: unknown function,
: output delay,
: input delay, ,
: output of MLP,
: input vector of (MLP or RBF),
: number of hidden layer,
: number of nodes of input layer,
: number of nodes of hidden layer,
: number of nodes of output layer,
: synaptic weights of MLP,
: activation function,
: learning rate,
: regularization coefficient,
: output of RBF,
: hidden radial basis function,
: hidden center,
: hidden width,
: synaptic weights of RBF,
: crossover probability,
: mutation probability,
: size generation,
: white noise,
: number of observationsSNR: Signal Noise Ratio.