Open Journal of Optimization
Vol.2 No.1(2013), Article ID:29457,10 pages DOI:10.4236/ojop.2013.21003

The Solution Classical Feedback Optimal Control Problem for m-Persons Differential Game with Imperfect Information

Jaykov Foukzon1, Elena Men’kova2, Alex Potapov3

1Department of Mathematics, Israel Institute of Technologies, Haifa, Israel

2All-Russian Research Institute for Opto-Physical Measurements, Moscow, Russia

3V.A. Kotel’nikov Institute of Radio Engineering and Electronics, Russian Academy of Sciences, Moscow, Russia

Email: jaykovfoukzon@list.ru, E_Menkova@mail.ru, potapov@cplire.ru

Received January 12, 2013; revised February 11, 2013; accepted March 5, 2013

Keywords: Optimal Control; Bellman Equation

ABSTRACT

The paper presents a new approach to construct the Bellman function and optimal control directly by way of using strong large deviations principle for the solutions Colombeau-Ito’s SDE. The generic imperfect dynamic models of air-to-surface missiles are given in addition to the related simple guidance law. A four examples have been illustrated, corresponding numerical simulations have been illustrated and analyzed.

1. Mathematical Challenge: Creating a Game Theory That Scales

What new scalable mathematics is needed to replace the traditional Partial Differential Equations (PDE) approach to differential games?

Let be a probably space. Any stochastic process on is a measurable mapping. Many stochastic optimal control problems essentially come down to constructing a function that has the properties 1)2)

 

, where is the termination payoff functional, is a control and is some Markov process governed by some stochastic Ito’s equation driven by a Brownian motion of the form 3)where is the Brownian motion. Traditionally the function has been computed by way of solving the associated Bellman equation, for which various numerical techniques mostly variations of the finite difference scheme have been developed. Another approach, which takes advantage of the recent developments in computing technology and allows one to construct the function by way of backward induction governed by Bellman’s principle such that described in [1]. In paper [1] Equation (3) is approximated by an equation with affine coefficients which admits an explicit solution in terms of integrals of the exponential Brownian motion. In approach proposed in paper [2,3] we have replaced Equation (3) by Colombeau-Ito’s Equation (4)

, where is the white noise on, i.e., almost surely in, and is the smoothed white noise on i.e.,

and is a model delta net [2,4]. Fortunately in contrast with Equation (3) one can solve Equation (4) without any approximation using strong large deviations principle [4]. In this paper we considered only quasi stochastic case, i.e.. General case will be considered in forthcoming papers.

Statement of the novelty and uniqueness of the proposed idea: A new approach, which is proposed in this paper allows one to construct the Bellman function and optimal control directly, i.e., without any reference to the Bellman equation, by way of using strong large deviations principle for the solutions Colombeau-Ito’s SDE (CISDE).

2. Proposed Approach

Let us consider an m-persons Colombeau-Ito’s differential game with a stochastic nonlinear dynamics:

;

(1)

and m-persons Colombeau-Ito’s differential game with imperfect information about the system [5-8]:

;

(2)

Here is the algebra of Colombeau generalized functions [9], is the ring of Colombeau’s generalized numbers [10-12],; is the control chosen by the i-th player, within a set of admissible control values, and the playoff for the i-th player is:

. (3)

where is the trajectory of the Equation (1). Optimal control problem for the i-th player is:

. (4)

Let us consider now a family of the solutions Colombeau-Ito’s SDE:

(5)

where is n-dimensional Brownian motion,

is a polynomial, i.e.

Definition 1. CISDE (5) is -dissipative if exist Lyapunov candidate function and Colombeau constants, such that:

1)

  

2).

Theorem 1. Main result (strong large deviations principle) [5,13]. For any solution of dissipative CISDE (5) and valued parameters, there exist Colombeau constant

such that:

. (6)

where a function is the solution of the master equation:

(7)

where the Jacobian, i.e. is a - matrix:

.

Remark.1. We note that

.

Example 1.

.

From a general master Equation (7) one obtain the next linear master equation:

. (8)

From the differential master Equation (8) one obtain transcendental master equation

. (9)

Numerical simulation: Figures 1 and 2.

. (10)

Here Let us consider now an m-persons Colombeau stochastic differential game with nonlinear dynamics

(11)

Here, is the control chosen by the i-th player, within a set of admissible control values, and the playoff of the i-th player is

(12)

where and is the trajectory of the Equation (11).

Theorem 2. For any solution

of the dissipative and val-

Figure 1. The solution of the Equation (8) in a comparison with a corresponding solution of the ODE (10).

Figure 2. versus R.

ued parameters, there exists Colombeau constant such that:

. (13)

where the trajectory of the corresponding master game

(14)

Example 2.

1)

optimal control problem for the first player:

and optimal control problem for the second player:

From Equation (14) we obtain corresponding master game:

2)

optimal control problem for the first player is:

and optimal control problem for the second player is:

Having solved by standard way [14,15] linear master game (2) one obtain optimal feedback control of the first player:

and optimal feedback control of the second player [5]:

Here

where is a part-whole of a number. Thus, for numerical simulation we obtain ODE:

Numerical simulation: Figures 3-6

Theorem 3. For any solution

Figure 3. Optimal trajectory:.

Figure 4. Optimal velocity:.

Figure 5. Optimal control of the first player.

Figure 6. Control of the second player.

of the dissipative and

valued parameters, there exists Colombeau constant such that:

. (15)

where the trajectory of the corresponding master game

(16)

Example 3. Game with imperfect measurements.

1)

From Equation (16) one obtain corresponding master game:

2)

Having solved by standard way linear master game (2) one obtain local optimal feedback control of the first player [5]:

and local optimal feedback control of the second player:

Thus, finally we obtain global optimal control of the next form [5]:

Here

where is a part-whole of a number. Thus, for numerical simulation we obtain ODE:

Numerical simulation: Figures 7-12. Game with imperfect measurements: red curves. Classical game: blue curves .

3. Homing Missile Guidance with Imperfect Measurements Capable to Defeat in Conditions of Hostile Active Radio-Electronic Jamming

Homing missile guidance strategies (guidance laws) dictate the manner in which the missile will guide to intercept, or rendezvous with, the target. The feedback nature of homing guidance allows the guided missile (or, more generally, the pursuer) to tolerate some level of (sensor) measurement uncertainties, errors in the assumptions used to model the engagement (e.g., unanticipated target maneuver), and errors in modeling missile capability (e.g., deviation of actual missile speed of response to guidance commands from the design assumptions). Nevertheless, the selection of a guidance strategy and its subsequent mechanization are crucial design factors that can have substantial impact on guided missile performance. Key drivers to guidance law design include the type of targeting sensor to be used (passive IR, active or semi-active RF, etc.), accuracy of the targeting and inertial measurement unit (IMU) sensors, missile maneuverability, and, finally yet important, the types of targets to be engaged and their associated maneuverability levels.

Figure 13 shows the intercept geometry of a missile in planar pursuit of a target. Taking the origin of the reference frame to be the instantaneous position of the missile, the equation of motion in polar form are [16]:

(17)

1) The variable denotes a true target-tomissile range.

2) The variable denotes the it is real measured target-to-missile range.

3) The variable denotes a true line-of-sight angle (LOST) i.e., the it is true angle between the constant reference direction and target-to-missile direction.

4) The variable denotes the it is real measured line-of-sight angle (LOSM) i.e., the it is true angle between the constant reference direction and target-tomissile direction.

5) The variable denotes the missiles acceleration along direction which perpendicularly to line-of-sight direction.

6) The variable denotes the missile acceleration along target-to-missile direction.

7) The variable denotes the target acceleration along direction which perpendicularly to line-of-sight direction.

8) The variable denotes the target acceleration along target-to-missile direction.

Using replacement into Equation (17) one obtain:

(18)

Figure 7. Uncertainty of speed measurements.

Figure 8. Cutting function.

Figure 9. Optimal trajectory.

Figure 10. Optimal velocity.

Figure 11. Optimal control of the first player.

Figure 12. Optimal control of the second player.

Figure 13. Planar intercept geometry.

(18)

Suppose that:

Therefore

(20)

Let us consider antagonistic Colombeau differential game

with non-linear dynamics and imperfect measurements [6]:

(21)

Optimal control problem of the first player is:

(22)

Optimal control problem of the second player is:

(23)

From Equations (21)-(23) one obtain corresponding linear master game:

(24)

From Equation (24) we obtain quasi optimal solution for the antagonistic differential game

given by Equations (21)(23). Quasi optimal control of the first player and quasi optimal control of the second player are:

(25)

Thus, for numerical simulation we obtain ODE:

Example 4: Figures 14-24.

Figure 14. Cutting function:.

Figure 15. Uncertainty of measurements of a variable.

Figure 16. Target-to-missile range..

4. Conclusions

Supporting Technical Analysis: Let us consider optimal control problem from Example 1, corresponding Bellman type equation is:

(27)

Figure 17. Speed of rapprochement missile-to-target:.

Figure 18. Variable.

Figure 19. Variable.

Figure 20. Variable.

Figure 21. Missile acceleration along target-to-missile direction:.

Figure 22. Missile acceleration along direction which perpendicularly to line-of-sight direction:.

Figure 23. Target acceleration along target-to-missile direction:.

Figure 24. Target acceleration along direction which perpendicularly to line-of-sight direction:.

Complete constructing the exact analytical solution for PDE (27) is a complicated unresolved classical problem, because PDE (27) is not amenable to analytical treatments. Even the theorem of existence classical solution for boundary Problems such (27) is not proved. Thus, even for simple cases a problem of construction feedback optimal control by the associated Bellman equation complicated numerical technology or principal simplification is needed [17]. However as one can see complete constructing feedback optimal control from Theorems 1-2 is simple. In study [6], the generic imperfect dynamic models of air-to-surface missiles are given in addition to the related simple guidance law.

REFERENCES

  1. A. Lyasoff, “Path Integral Methods for Parabolic Partial Differential Equations with Examples from Computational Finance,” Mathematical Journal, Vol. 9, No. 2, 2004, pp. 399-422.
  2. D. Rajter-Ciric, “A Note on Fractional Derivatives of Colombeau Generalized Stochastic Processes,” Novi Sad Journal of Mathematics, Vol. 40, No. 1, 2010, pp. 111- 121.
  3. C. Martiasa, “Stochastic Integration on Generalized Function Spaces and Its Applications,” Stochastics and Stochastic Reports, Vol. 57, No. 3-4, 1996, pp. 289-301. doi:10.1080/17442509608834064
  4. M. Oberguggenberger and D. Rajter-Ciric, “Stochastic Differential Equations Driven by Generalized Positive Noise,” Publications de l’Institut Mathématique, Nouvelle Série, Vol. 77, No. 91, 2005, pp. 7-19.
  5. J. Foukzon, “The Solution Classical and Quantum Feedback Optimal Control Problem without the Bellman Equation,” 2009. http://arxiv.org/abs/0811.2170v4
  6. J. Foukzon and A. A. Potapov, “Homing Missile Guidance Law with Imperfect Measurements and Imperfect Information about the System,” 2012. http://arxiv.org/abs/1210.2933
  7. P. Bernhard and A.-L. Colomb, “Saddle Point Conditions for a Class of Stochastic Dynamical Games with Imperfect Information,” IEEE Transactions on Automatic Control, Vol. 33, No. 1, 1988, pp. 98-101. http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=367
  8. A. V. Kryazhimskii, “Differential Games of Approach in Conditions of Imperfect Information about the System,” Ukrainian Mathematical Journal, Vol. 27, No. 4, 1975, pp. 425-429. doi:10.1007/BF01085592
  9. J. F. Colombeau, “Elementary Introduction to New Generalized Functions,” North-Holland, Amsterdam, 1985.
  10. J. F. Colombeau, “New Generalized Functions and Multiplication of Distributions,” North-Holland, Amsterdam, 1984.
  11. H. Vernaeve, “Ideals in the Ring of Colombeau Generalized Numbers,” 2007. http://arxiv.org/abs/0707.0698
  12. E. Mayerhofer, “Spherical Completeness of the NonArchimedian Ring of Colombeau Generalized Numbers,” Bulletin of the Institute of Mathematics Academia Sinica (New Series), Vol. 2, No. 3, 2007, pp. 769-783.
  13. J. Foukzon, “Large Deviations Principles of Non-FreidlinWentzell Type,” 2008. http://arxiv.org/abs/0803.2072
  14. S. Gutman, “On Optimal Guidance for Homing Missiles,” Journal of Guidance and Control, Vol. 2, No. 4, 1979, pp. 296-300. doi:10.2514/3.55878
  15. V. Glizer and V. Turetsky, “Complete Solution of a Differential Game with Linear Dynamics and Bounded Controls,” Applied Mathematics Research Express, Vol. 2008, 2008, p. 49. doi:10.1093/amrx/abm012
  16. M. Idan and T. Shima, “Integrated Sliding Model Autopilot-Guidance for Dual-Control Missiles,” Journal of Guidance, Control and Dynamics, Vol. 30, No. 4, 2007, pp. 1081-1089. doi:10.2514/1.24953
  17. W. Cai and J. Z. Wang, “Adaptive Wavelet Collocation Methods for Initial Value Boundary Problems of Nonlinear PDE’s,” Pentagon Reports, 1993. http://www.stormingmedia.us/44/4422/A442272.html10.1093/amrx/abm012