This paper presents an approach that directly utilizes the Hessian matrix to investigate the existence and uniqueness of global solutions for the ECQP problem. The novel features of this proposed algorithm are its uniqueness and faster rate of convergence to the solution. The merit of this algorithm is base on cost, accuracy and number of operations.
Usually the general quadratic programming problem has a structure of the form
where is a symmetric matrix, and are finite sets of indices. In quadratic programming problems, the matrix is called the Hessian matrix. The vectors and are column vectors in. To make computational life easier, we consider only the equality constraints and formulate the equality constrained quadratic programming problem as follows
where is a jacobian matrix of constraints (with) [
In sequential quadratic programming algorithms, an phase that employs second derivative information (Hessian matrix) is usually added to enhance rapid convergence to the solution [4-7] algorithms [
In 1985, Gould investigates the conditions under which the problem can be said to have a finite solution. Gould’s analysis of the problem is based on the concepts of the reduced Hessian matrix and signs of the eigenvalues of the Karush-Kuhn Tucker matrix [
[
Our goal in this paper is to present a new method that utilizes a necessary and sufficient condition for the existence and uniqueness of the solutions of the problem. In this paper, we show that for the problem to have a global solution, its Hessian matrix must possess a Cholesky factor. As we shall see in Section 2, this paper focuses only on the condition(s) under which the problem is said to have a global solution [
This paper is organized as follows. In Section 2, we discuss our method. Gould’s method is reviewed in Section 3. The analysis follow in Section 4 and some concluding remarks are made in Section 5.
In this section, we introduce our new method of analyzing the solution of the problem. It is based on the fact that the Cholesky decomposition is unique for positive definite matrices.
Cholesky DecompositionLet be a matrix that can undergo Cholesky decomposition with a Cholesky factor (Lower triangular matrix) then we can write
where is the transpose of. We let
Substituting Equation (2.2) into Equation (2.1) gives
From Equation (2.3), we see that the conditions for to be positive definite are satisfied. Therefore,our conditions for positive definiteness are; the matrix must be a square matrix and possesses a Cholesky factor.
We let to be a column vector say
and we write
From Equation (2.3), it is clear that the first and second terms are always positive, which implies their sum is also always positive and greater than the third term if. When the matrix always equals zero. Therefore, the matrix is always positive if and only if the column vector has entries and (such that) and the matrix has a Cholesky factor.
In the above demonstration, which means that a matrix and a column vector produces Equation (2.5). Analogously, any matrix and any column vector (where) shall produce an equation similar in properties to Equation (2.5). If and only if.
Corollary 2.1: Let be any non-singular matrix and the Hessian matrix being Cholesky factorizable. Then the matrix
is nonsingular and has a unique solution.
Corollary 2.2: Let be the Karush-Kuhn-Tucker matrix
and assume is any matrix. Then the problem has a global minimum if and only if the Hessian matrix has a Cholesky factor.
In this section, we review Gould’s method. The method consists of three approaches: Null-space methods, Lagrangian methods and Schur complement methods [
Null-space methods: For to be a solution of the problem, a vector (i.e. Lagrange multipliers) must exist such that the system of equations below is satisfied
We let
with being some estimate of the solution and the desired step. By expressing as in Equation (3.2), Equation (3.1) can be written in a form that is more useful for computational purposes as given below
where
This method finds and first, by partitioning the vector into two components as follows
where and have orthonormal columns and can be obtain from the factorization of. An interesting property of this approach is that [1,3], which makes the calculation of and possible by solving the four equations below
This method has a wider application than the Rangespace methods because; it doesn’t require being nonsingular. According to this paper, the condition, that must undergo Cholesky decomposition is the only requirement for the problem to have a global minimum. A knowledge of the null space basis matrix is not important at all.
Lagrangian methods: This method calculates the values of and directly from Equation (3.3), i.e. the Karush-Kuhn-Tucker equations for the problem.
In this paper, the problem can only have a global minimum if possesses a Cholesky factor.
Schur complement methods: Here we assume that has a Cholesky factor and derive two equations from Equation (3.3) for the solutions of and. These equations are as follows
It is easy to see that both and are positive definite. In this paper, we show that and have Cholesky factors and hence is always positive definite, which indicate the existence of a global solution for the problem Section 2.
In this section we will solve a numerical example from [
We will write the above problem in the standard form described in the introduction by defining
For Gould’s algorithm we need to find from the factorization of matrix i.e..
We can obtain from the column space of matrix and the matrix must satisfies the constrain. Hence we have
Therefore, and according to Gould’s algorithm the problem has a global minimum.
For our algorithm we only need to show that the matrix has a Cholesky factor. Let be the Cholesky factor of.
According to our algorithm,this implies the matrix is positive definite and therefore the problem has a minimum solution. To show this fact we select any matrix that is a subset of the set of matrices described in subsection (2.1) and suppose we have that matrix to be
then.
Let us consider another matrix
We will have result
Finally, we consider a matrix with all negative entries as follows
This gives the result
From the above example, we observed the following result:
1) Multiplying a matrix that has a Cholesky factor with any other matrix except the zero matrix, doesn’t alter the positive definite property of matrix and hence the existence of global minimum.
2) Decimals are encountered in Gould’s approach which may lead to rounding off errors and hence inaccuracy. Decimals have no effects on our method as long as the Hessian matrix has a Cholesky factor.
3) The number of matrix operations that are involved in Gould’s approach are far more than those that are involved in our algorithm which implies that our method is faster than that of Gould.
Gould’s approach uses the notion of the reduced Hessian matrix and the signs of the eigenvalues of the Karush-Kuhn-Tucker matrix to analyze the conditions under which the problem shall have a global solution [
Finally, this proposed method has fewer iterations than Gould’s algorithm, inexpensive and naturally faster (Cholesky factorization) than Gould’s approach (with more iterations).
In 1985, Gould investigates the practical conditions for the existence and uniqueness of solutions of the problem based on and inertia of the matrix. In this piece of work, we present a new method that directly works with to analyze global solutions of the problem.
The advantages of our method lie in its accuracy, cost and number of operations. It is true that this noble algorithm is unique and computationally faster (i.e. Cholesky decomposition) than Gould’s method. Our method also revealed that if the Hessian matrix has a Cholesky factor then, the Hadamard inequality [
We finally conclude that the existence and uniqueness of solutions of the problem is independent of its constraints but depend wholly and solely on the Hessian matrix.
We would like to thank Khalid O. Elaalim for his useful contributions during the early stages of this work. We are also grateful to anonymous referees for their valuable comments on this paper. Finally, we would like to extend special thanks to the Chinese Scholarship Council which funded this research.