Advances in Pure Mathematics
Vol.07 No.04(2017), Article ID:75837,10 pages
10.4236/apm.2017.74018

An Effective Algorithm for Quadratic Optimization with Non-Convex Inhomogeneous Quadratic Constraints

Kaiyao Lou

Department of Mathematics, College of Science, Shanghai University, Shanghai, China

Copyright © 2017 by author and Scientific Research Publishing Inc.

This work is licensed under the Creative Commons Attribution International License (CC BY 4.0).

http://creativecommons.org/licenses/by/4.0/

Received: April 5, 2017; Accepted: April 27, 2017; Published: April 30, 2017

ABSTRACT

This paper considers the NP (Non-deterministic Polynomial)-hard problem of finding a minimum value of a quadratic program (QP), subject to m non- convex inhomogeneous quadratic constraints. One effective algorithm is proposed to get a feasible solution based on the optimal solution of its semidefinite programming (SDP) relaxation problem.

Keywords:

Nonconvex Inhomogeneous Quadratic Constrained Quadratic Optimization, Semidefinite Programming Relaxation, Np-Hard

1. Introduction

Consider the following quadratic optimization problem with non-convex inhomogeneous quadratic constraints:

m i n x R n f ( x ) = x T A x s .t . x T A k x + ( b k ) T x 1, k = 1, , m , (1)

where A , A k R n × n ( k = 1 , , m ) are symmetric positive semidefinite matrices and b k R n . Note that if n = 1 , then the problem (1) is easily solved, so assume n 2. Generally, this problem is NP-hard [1] [2] [3] . It has a lot of applications in telecommunications, robust control, portfolio and so on. So it’s helpful to solve it.

By means of the work of Lovász and Schrijver [4] , Shor [5] , and others that certain NP-hard combinatorial optimization problems can be approximated by semidefinite programming (SDP) problems, for which efficient solution methods exist [6] [7] . So many mathematics workers with this motive put forward a lot of algorithms to solving quadratic optimization problem based on the semidefinite programming (SDP) relaxation. Recently, there were several results on solving different forms of quadratic problems. The results can be summarized as follows.

•  If all A k are symmetric positive semidefinite n × n matrices with positive definite sum and A is an arbitrary symmetric n × n matrix. A. Nemirov- ki [8] produce a feasible solution x ˜ such that, with constant probability,

f ( x ˜ ) 1 2 ln ( 2 m 2 ) v ( S D P ) . (2)

•  If A , A k 0, A k = ( A k ) T , b k = 0, Luo et al. [9] showed that the ratio between the original optimal value and the SDP relaxation optimal value is bounded by O ( m 2 ) , have

v ( P ) v ( S D P ) 27 m 2 π . (3)

•  If all A k , A are symmetric matrices, and two or more of them are indefinite. S. He et al. [10] compute a feasible solution x ˜ such that,

f ( x ˜ ) 10 6 m 2 π v ( S D P ) . (4)

•  Of special interest is the case of ellipsoid constraints

A k = ( F k ) T F k , b k = 2 ( F k ) T , c k = ( g k ) 2 h k , k = 1 , , m , (5)

where F k R n , g k R n , h k 0 , 1, denotes the Euclidean norm, so x T A k x + b k x + c k = F k x + g k 2 h k , k = 1 , , m . Nemirovski [8] show that if all b k = 0 and A k are positive semidefinite. Then a feasible solution x ˜ can be generated from (SDP) satisfying

f ( x ˜ ) 1 2 ln ( 2 ( m + 1 ) μ ) v ( S D P ) , μ = min { m + 1 , max k = 1 , , m r a n k ( A k ) } . (6)

•  In particular, if (1) has a ball constraints, μ = min { m + 1 , n } . Ye and Zhang(Corollary 2.6 in [11] ) showed that a feasible x satisfying

f ( x ) 1 min { m 1 , n } v ( S D P ) , (7)

can be found in polynomial time.

•  Ye. [12] extended the above result to allow the ellipsoids not to have a common center but assuming A 0 . Ye showed that a feasible solution x ˜ can be randomly generated such that

E ( x ˜ A T x ˜ ) ( 1 max k = 1 , , m g k ) 2 4 ln ( 4 m n max k = 1 , , m ( r a n k ( A k ) ) ) v ( S D P ) . (8)

However, the existing algorithms are just for the problem of discrete problems or continuous problems, which is mostly based on homogeneous or inhomogeneous convex constraint problems. For this kind of quadratic optimization problem with non-convex inhomogeneous quadratic constraints, cannot find a very effective algorithm. This paper will propose a new effective algorithm to solve this problem.

This paper is organized as follows. In Section 2, we present a semidefinite pro- gramming (SDP) relaxation of (1). In Section 3, we propose a new effective algorithm to get the feasible solution of quadratic optimization problem (1) with non-convex inhomogeneous quadratic constraints. At last, some conclusions and the future works are given in Section 4.

Notations. Throughout this paper, we denote by R n and S + n the n-dimen- sional real vector space and n × n positive semidefinite symmetric matrices space. A 0 denotes that A is semidefinite. T r ( ) represents the trace of a matrix. The inner product of two matrices A and B is denoted by

A B = T r ( A B T ) = i = 1 n j = 1 n a i j b i j P r ( ) stands for the probability.

2. Semidefinite Programming (SDP) Relaxation

In this section, we present a semidefinite programming (SDP) relaxation of (1). To avoid trivial cases, we first make the following assumption.

Assumption Let b k = 2 A k y k , has a solution, y k R n .

Assume t is a constant, and satisfy t 2 = 1 . (1) is equivalent to:

m i n ( x , t ) R n × R f ( x , t ) = x T A x s .t . 1 1 + ( y k ) T A k y k [ x T A k x + ( b k ) T x t + ( y k ) T A k y k t 2 ] 1, k = 1, , m t 2 = 1. (9)

Let x * be the global optimal solution of the above problem, the objective value is v ( x * ) . Assume X 0 , it’s block structure like this:

X = [ X ( 1 ) X ( 3 ) ( X ( 3 ) ) H X ( 2 ) ] S + ( n + 1 ) × ( n + 1 ) . (10)

where

B = [ A 0 0 0 ] 0, B k = 1 1 + ( y k ) T A k y k [ A k 1 2 b k 1 2 ( b k ) T ( y k ) T A k y k ] 0, k = 1, , m . (11)

By letting X = x x T and dropping the rank one constraint, the semidefinite programming relaxation of (9) can be drawn up as follows.

m i n B X s .t . B k X 1, k = 1, , m X n + 1, n + 1 = 1, X 0, X R ( n + 1 ) × ( n + 1 ) . (SDP)

An optimal solution of SDP relaxation (SDP) can be computed efficiently using, say, interior-point mathods; see [13] and references therein.

3. Algorithm

In this section, we bring an effective algorithm for solving (1). The algorithm is divided into two parts. The main idea as follows: the first stage produces a solution which satisfies the first constraint of problem (9). Making a small change to the solution which obtained in the first stage, we can get the solution of (9) in the second stage. We will set the randomization algorithm as follows.

3.1. The First Stage

The first stage of the algorithm uses the randomization algorithm, which is proposed by Luo et al. [3] . At first, we need obtain an optimal solution X * of (SDP), then construct a feasible solution for the first constraint of problem (9) using the following randomization procedure:

First, it can be easily verified that x ˜ = ( x T , x ˜ n + 1 ) T satisfy the first constraint of problem (9).

1 1 + ( y k ) T A k y k [ x T A k x + ( b k ) T x x ˜ n + 1 + ( y k ) T A k y k x ˜ n + 1 2 ] = ξ T B k ξ m i n k ξ T B k ξ 1, (12)

(12) is equivalent to:

1 1 + ( y k ) T A k y k [ ( x x ˜ n + 1 ) T A k x x ˜ n + 1 + b k x x ˜ n + 1 + ( y k ) T A k y k ] 1 x ˜ n + 1 2 , k = 1, , m . (13)

Lemma 1 For x ˜ generated in step 2, we have that

p r ( 1 x ˜ n + 1 2 4 10 4 m 2 ) 1 100 . (14)

Proof. By the step 2, we first have

x ˜ n + 1 = e n + 1 T ξ m i n k ξ T B k ξ , (15)

where e n + 1 R ( n + 1 ) × 1 is a vector with the ( n + 1 ) th element being 1 and all the other elements being 0. By denoting Q = e n + 1 e n + 1 T , we obtain that

p r ( 1 x ˜ n + 1 2 M ) = p r ( 1 M m i n k ξ T B k ξ ξ T e n + 1 e n + 1 T ξ ) = p r ( 1 M m i n k ξ T B k ξ ξ T Q ξ ) . (16)

By using the total probability formula for the last term in (16), we have

p r ( 1 M M ξ T Q ξ ) p r ( M m i n k ξ T B k ξ ) + p r ( M > m i n k ξ T B k ξ ) 1 p r ( 1 ξ T Q ξ ) + p r ( M > m i n k ξ T B k ξ ) . (17)

By Lemma 3.1 and Lemma 3.2 in [10] ,

p r ( 1 ξ T Q ξ ) = p r ( T r ( Q X * ) ξ T Q ξ ) = p r ( E ( ξ T Q ξ ) ξ T Q ξ ) < 1 3 100 . (18)

Since X * is feasible for (SDP), it follows that T r ( B k X * ) 1 for all k = 1 , , m . Since E ( ξ T B k ξ ) = T r ( B k X * ) 1, so

p r ( M > min k ξ T B k ξ ) p r ( M T r ( B k X * ) > min k ξ T B k ξ ) = p r ( M E ( ξ T B k ξ ) > min k ξ T B k ξ ) k = 1 m p r ( M E ( ξ T B k ξ ) > ξ T B k ξ ) . (19)

According to Lemma 1 in [9] , we have

p r ( M E ( ξ T B k ξ ) > ξ T B k ξ ) max { M , 2 ( 2 ( m + 1 ) 1 ) M π 2 } . (20)

Thus, it follows from (17), (18), (19) and (20) that:

p r ( 1 x ˜ n + 1 2 M ) 1 3 100 + m max { M , 2 ( 2 ( m + 1 ) 1 ) M π 2 } . (21)

The proof is completed by setting M = 4 10 4 m 2 .

Note that by Lemma 1 and (13), it can be concluded that

p r { 1 1 + ( y k ) T A k y k [ ( x x ˜ n + 1 ) T A k x x ˜ n + 1 + b k x x ˜ n + 1 + ( y k ) T A k y k ] 4 10 4 m 2 } 1 100 . (22)

So there is a x ¯ , for any k = 1 , , m satisfies:

1 1 + ( y k ) T A k y k [ ( x ¯ ) T A k x ¯ + b k x ¯ + ( y k ) T A k y k ] 4 10 4 m 2 . (23)

3.2. The Second Stage

In this part, we make a change to the solution which constructed in the first stage in order to satisfy the problem (9). In this stage, we will by ways of the algorithm in [14] .

The procedure as follows:

Let f ( τ ) = τ 2 x ^ T A k x ^ + τ ( b k ) T x ^ , so f ( τ ) can be seen as a quadratic function for τ . The symmetry axis of f ( τ ) is:

x = ( b k ) T x ^ 2 x ^ T A k x ^ . (24)

Because the x ^ can’t make ( x ^ ) T A k ( x ^ ) + ( b k ) T ( x ^ ) 1 for all k = 1 , , m set up. We introduce a parameter τ ^ , and construct a new solution τ ^ x ^ . It’s the feasible solution of (1).

When x ^ T A k x ^ + ( b k ) T x ^ > 1 , the symmetry axis of f ( τ ) satisfies:

x = ( b k ) T x ^ 2 x ^ T A k x ^ < x ^ T A k x ^ 1 2 x ^ T A k x ^ = 1 2 1 2 x ^ T A k x ^ < 1 2 . (25)

So for all τ > 1 , can make ( τ x ^ ) T A k ( τ x ^ ) + ( b k ) T ( τ x ^ ) > 1 set up. It’s helpful for us to solve the problem, because we only need to find τ satisfying ( τ x ^ ) T A k ( τ x ^ ) + ( b k ) T ( τ x ^ ) > 1 in the situation of x ^ T A k x ^ + ( b k ) T x ^ 1

When x ^ T A k x ^ + ( b k ) T x ^ 1 , because A k 0 ( k = 1, , m ) are symmetric, b k = 2 A k y k . To simplify the writing, we introduce the following notations:

x k = ( A k ) 1 2 x ^ , y k = ( A k ) 1 2 y k , z k = ( A k ) 1 2 x ^ + ( A k ) 1 2 y k , (26)

So

f ( τ ) = τ 2 x ^ T A k x ^ + τ ( b k ) T x ^ = ( τ 2 τ ) ( x k ) 2 + τ [ ( x k ) 2 + ( b k ) T x ^ + ( y k ) 2 ] τ ( y k ) 2 = ( τ 2 τ ) ( x k ) 2 + τ ( z k ) 2 τ ( y k ) 2 . (27)

According to the norm inequality:

x y | x y | . (28)

we have

( x k ) 2 = ( ( A k ) 1 2 x ^ ) 2 ( | ( A k ) 1 2 x ^ + ( A k ) 1 2 y k ( A k ) 1 2 y k | ) 2 = ( z k y k ) 2 . (29)

For τ [ 1, + ) , using (28), (29). The last term in (27) can be simplified to be

( τ 2 τ ) ( z k y k ) 2 + τ ( z k ) 2 τ ( y k ) 2 = τ 2 ( z k y k ) 2 + 2 τ ( ( y k ) ( z k ) ( y k ) 2 ) = τ 2 ( z k y k ) 2 + 2 τ ( y k ) ( z k y k ) , (30)

Whenever τ ( y k ) ( z k y k ) + ( z k y k ) 2 ( 1 + ( y k ) 2 ) ( z k y k ) 2 , it can be easily checked that ( τ x ^ ) T A k ( τ x ^ ) + ( b k ) T ( τ x ^ ) 1.

From (23), we can get

1 1 + ( y k ) T A k y k [ ( x ^ ) T A k x ^ + b k x ^ + ( y k ) T A k y k ] 4 10 4 m 2 z k 1 50 m ( 1 + ( y k ) 2 ) , (31)

and we also have

x ^ T A k x ^ + ( b k ) T x ^ 1 z k 1 + ( y k ) 2 . (32)

Thus

τ ^ max k = 1 , , m ( y k ) ( z k y k ) + ( z k y k ) 2 ( 1 + ( y k ) 2 ) ( z k y k ) 2 max k = 1 , , m ( y k ) 2 + 1 + ( y k ) | z k y k | . (33)

Using (31) and (32), the last term in (33) can be simplified as

max k = 1 , , m { 1 + ( y k ) 2 + ( y k ) | 1 50 m ( 1 + ( y k ) 2 ) y k | , 1 + ( y k ) 2 + ( y k ) 1 + ( y k ) 2 ( y k ) } . (34)

We will give the analysis of (34) as follows.

First, let

f ( y k ) = 1 + ( y k ) 2 + ( y k ) 1 + ( y k ) 2 ( y k ) , g ( y k ) = 1 + ( y k ) 2 + ( y k ) | 1 50 m ( 1 + ( y k ) 2 ) y k | . (35)

we can simplified f ( y k )

f ( y k ) = 1 + ( y k ) 2 + ( y k ) 1 + ( y k ) 2 ( y k ) = ( 1 + ( y k ) 2 + ( y k ) ) 2 . (36)

Since y k 0 , we know that f ( y k ) is an increasing function about y k .

However, g ( y k ) is a function depend on y k and the number of the constraints. According to simple calculations, we find, when satisfies y k < 1 50 m 1 , g ( y k ) is an increasing function about y k . In this situation, we also have g ( y k ) > f ( y k ) . When y k > 1 50 m 1 , g ( y k ) becomes smaller with the increase of y k .

where

γ 1 = max k = 1 , , m ( y k ) , γ 2 = min k = 1 , , m ( y k ) . (37)

we can write (34) as a piecewise function about γ 1 and γ 2

f ( γ 1 , γ 2 ) = { ( γ 1 ) 2 + 1 + γ 1 | 1 50 m ( 1 + ( γ 1 ) 2 ) γ 1 | γ 1 , γ 2 < 1 50 m 1 + γ 1 > 1 50 m 1 , γ 2 < 1 50 m 1 max { ( γ 2 ) 2 + 1 + γ 2 | 1 50 m ( 1 + ( γ 2 ) 2 ) γ 2 | , ( γ 1 ) 2 + 1 + γ 1 ( γ 1 ) 2 + 1 γ 1 } γ 1 , γ 2 > 1 50 m 1 . (38)

So it can be easily verified that

x = f ( γ 1 , γ 2 ) x ^ , with γ 1 = max k = 1 , , m ( y k ) , γ 2 = min k = 1 , , m ( y k ) , (39)

is a feasible solution of the original problem.

4. Conclusions

For the quadratic optimization problem with non-convex inhomogeneous quadratic constraints, it’s NP-hard. We can’t find an effective algorithm solving it. In this paper, we put forward an effective algorithm. According to it, many problems in life can be solved. Through the algorithm, we can get the feasible solution of (1). Transforming the original problem into (SDP) is a very important step in solving the problem. So we give the semidefinite programming (SDP) relaxation of (1) in Section 2, then propose an effective algorithm which given in Section 3 to construct the feasible solution of (1).

In the future, I will do the following work: discusses the quality of the feasible solution about (1), and gives some numerical experiments to verify it, we will consider the problem with inhomogeneous objective function. To this problem, we want to find an algorithm solve it by ways of the effective algorithm which put forward in this paper.

Cite this paper

Lou, K.Y. (2017) An Effective Algorithm for Quadratic Optimization with Non-Convex Inhomogeneous Quadratic Constraints. Advances in Pure Mathematics, 7, 314-323. https://doi.org/10.4236/apm.2017.74018

References

  1. 1. Pardalos, P.M. and Schnitger, G. (1988) Checking Glocal Optimality in Constrained Quadratic Programming is NP-hard. Operations Research Letters, 7, 33-35.https://doi.org/10.1016/0167-6377(88)90049-1

  2. 2. Pardalos, P.M. and Vavasis, S.A. (1991) Quadratic Programming with One Negative Eigenvalue Is NP-Hard. The Journal of Global Optimization, 1, 15-22.https://doi.org/10.1007/BF00120662

  3. 3. Luo, Z.-Q., Sidiropoulos, N., Tseng, P. and Zhang, S. (2007) Approximation Bounds for Quadratic Optimization with Homogeneous Quadratic Constraints. SIAM Journal on Optimization, 18, 1-28. https://doi.org/10.1137/050642691

  4. 4. Lovász, L. and Schrijver, A. (1991) Cones of Matrices and Set-functions and 0-1 Optimization. SIAM Journal on Optimization, 1, 166-190. https://doi.org/10.1137/0801013

  5. 5. Shor, N.Z. (1987) Quadratic Optimization Problems. Soviet Journal of Computer and Systems Sciences, 25, 1-11.

  6. 6. Alizadeh, F. (1995) Interior Point Methods in Semidefinite Programming with Applications to Combinatorial Optimization. SIAM Journal on Optimization, 5, 13-51.https://doi.org/10.1137/0805002

  7. 7. Nesterov, Y. and Nemirovskii, A. (1994) Interior-Point Polynomial Algorithms in Convex Programming. Studies in Applied and Numerical Mathematics, Philadelphia, PA. https://doi.org/10.1137/1.9781611970791

  8. 8. Nemirovsi, A. and Roos, C. and Terlaky, T. (1999) On Maximization of Quadratic form over Intersection of Ellopsoids with Common Center. Math Program, 86, 463-473. https://doi.org/10.1007/s101070050100

  9. 9. Luo, Z.Q., Sidiropoulos, N.D., Tseng, P. and Zhang, S. (2007) Approximation Bounds for Quadratic Optimization with Homogeneous Quadratic Constraints. SIAM Journal on Optimization, 18, 1-28. https://doi.org/10.1137/050642691

  10. 10. He, S., Luo, Z.Q., Nie, J. and Zhang, S. (2008) Semidefinite Relaxation Bounds for Indefinite Homogeneous Quadratic Optimization. SIAM Journal on Optimization, 19, 503-523. https://doi.org/10.1137/070679041

  11. 11. Ye, Y. and Zhang, S. (2003) New Results on Quadratic Minimization. SIAM Journal on Optimization, 14, 245-267. https://doi.org/10.1137/S105262340139001X

  12. 12. Ye, Y. (1999) Approximating Global Quadratic Optimization with Convex Quadratic Constraints. The Journal of Global Optimization, 15, 1-17. https://doi.org/10.1023/A:1008370723217

  13. 13. Pataki, G. (2003) Computational Semidefinite and Second Order Cone Programming. The State of the Art, Math, Program, 95, 3-51.

  14. 14. Hsia, Y., Wang, S. and Xu, Z. (2015) Improved Semidefinite Approximation Bounds for Nonconvex Nonhomogeneous Quadratic Optimization with Ellipsoid Constraints. Operations Research Letters, 43, 378-383.