_{1}

^{*}

In this paper we present a new type of Restarted Krylov methods for calculating peripheral eigenvalues of symmetric matrices. The new framework avoids the Lanczos tridiagonalization process, and the use of polynomial filtering. This simplifies the restarting mechanism and allows the introduction of several modifications. Convergence is assured by a monotonicity property that pushes the eigenvalues toward their limits. The Krylov matrices that we use lead to fast rate of convergence. Numerical experiments illustrate the usefulness of the proposed approach.

In this paper we present a new type of Restarted Krylov methods. Given a symmetric matrix^{2} is considerably smaller than n. The need for computing a few extreme eigenvalues of such a matrix arises in many applications, see [

The traditional restarted Krylov methods are classified into three types of restarts: “Explicit restart” [

Let the eigenvalues of G be ordered to satisfy

or

Then the new algorithm is built to compute one of the following four types of target clusters that contain k extreme eigenvalues.

A dominant cluster

A right-side cluster

A left-side cluster

A two-sides cluster is a union of a right-side cluster and a left-side cluster. For example,

Note that although the above definitions refer to clusters of eigenvalues, the algorithm is carried out by computing the corresponding k eigenvectors of G. The subspace that is spanned by these eigenvectors is called the target space.

The basic iteration

The qth iteration,

Step 1: Eigenvalues extraction. First compute the Rayleigh quotient matrix

Then compute k eigenpairs of

which is used to compute the related matrix of Ritz vectors,

Note that both

Step 2: Collecting new information. Compute a Krylov matrix

Step 3: Orthogonalize the columns of

Step 4: Build an orthonormal basis of Range (Z_{q}). Compute a matrix,

whose columns form an orthonormal basis of Range (Z_{q}). This can be done by a QR factorization of_{q}) is smaller than_{q}).)

Step 5: Define

which ensures that

At this point we are not concerned with efficiency issues, and the above description is mainly aimed to clarify the purpose of each step. Hence there might be better ways to carry out the basic iteration.

The plan of the paper is as follows. The monotonicity property that motivates the new method is established in the next section. Let

The monotonicity property is an important feature of the new iteration, whose proof is given in [

Theorem 1 (Cauchy interlace theorem) Let

Let the symmetric matrix

denote the eigenvalues of H. Then

and

In particular, for

Corollary 2 (Poincarà separation theorem) Let the matrix

Let us turn now to consider the qth iteration of the new method,

and let the eigenvalues of the matrix

be denoted as

Then the Ritz values which are computed at Step 1 are

and these values are the eigenvalues of the matrix

Similarly,

are the eigenvalues of the matrix

Therefore, since the columns of

On the other hand from Corollary 2 we obtain that

Hence by combining these relations we see that

for

The treatment of a left-side cluster is done in a similar way. Assume that the algorithm is aimed at computing a cluster of k left-side eigenvalues of G,

Then similar arguments show that

for

Recall that a two-sides cluster is the union of a right-side cluster and a left- side one. In this case the eigenvalues of

The basic Krylov information matrix has the form

where the sequence

where

The proof of the Kaniel-Paige-Saad bounds relies on the properties of Chebyshev polynomials, while the building of these polynomials is carried out by using a three term recurrence relation, e.g. [

Let

The preparations part

a) Compute the starting vector:

b) Compute

Orthogonalize

Normalize

c) Compute

Orthogonalize

Orthogonalize

Normalize

The iterative part

For

a) Set

b) Orthogonalize

c) Orthogonalize

d) Reorthogonalization: For

e) Normalize

The reorthogonalization step is aimed to ensure that the numerical rank of

Assume for a moment that the algorithm is aimed at calculating a cluster of k dominant eigenvalues. Then the Kaniel-Paige-Saad bounds suggest that slow rate of convergence is expected when these eigenvalues are poorly separated from the other eigenvalues. Indeed, this difficulty is seen in

The implementation of this idea is carried out by introducing a small modification in the construction of

(In our experiments

The usefulness of the Power-Krylov approach depends on two factors: The cost of a matrix-vector product and the distribution of the eigenvalues. As noted above, it is expected to reduce the number of iterations when the k largest eigenvalues of G are poorly separated from the rest of the spectrum. See

To start the algorithm we need to supply an “initial” orthonormal matrix,

be generated as in Section 3, using some arbitrary starting vector_{0}). A similar procedure is used in the Power-Krylov method.

In our experiments

In this section we describe some experiments with the proposed methods. The test matrices have the form

where

Recall that in Krylov methods there is no loss of generality in experimenting with diagonal matrices, e.g., ( [

Thus, for example, from

The ability of the basic Krylov matrix to achieve accurate computation of the eigenvalues is illustrated in

Matrix type | Matrix eigenvalues |
---|---|

Harmonic squares | |

Harmonic | |

Harmonic roots | |

Very fast geometric decay | |

Fast geometric decay | |

Geometric decay | |

Moderate geometric decay | |

Slow geometric decay | |

Very slow geometric decay | |

Dense equispaced | |

Equispaced | |

Low-Rank-100 | |

Low-Rank-50 | |

Low-Rank-10 | |

Harmonic Triples | |

Multiple-Harmonic | |

Multiple-Geometric | |

Equispaced-Geometric Gap |

rank, like “Harmonic” or “Geometric”, and matrices with gap in the spectrum. In such matrices the initial orthonormal matrix,

As expected, a slower rate of convergence occurs when the dominant eigen- alues that we seek are poorly separated from the other eigenvalues. This situation is demonstrated in matrices like “Dense equispaced” or “Very slow geometric”. Yet, as

The new type of Restarted Krylov methods avoids the use of Lanczos algorithm. This simplifies the basic iteration, and clarifies the main ideas behind the

Matrix type | Number of iterations | |||||
---|---|---|---|---|---|---|

Harmonic squares | 0 | 0 | 0 | 0 | 0 | 0 |

Harmonic | 0 | 0 | 0 | 0 | 0 | 0 |

Harmonic roots | 0 | 0 | 0 | 1 | 2 | 3 |

Very fast Geometric | 0 | 0 | 0 | 0 | 0 | 0 |

Fast Geometric | 0 | 0 | 0 | 0 | 0 | 0 |

Geometric | 0 | 0 | 0 | 0 | 0 | 0 |

Moderate Geometric | 1 | 1 | 2 | 1 | 0 | 0 |

Slow Geometric | 7 | 7 | 8 | 9 | 10 | 7 |

Very slow Geometric | 28 | 27 | 28 | 23 | 26 | 23 |

Dense Equispaced | 43 | 43 | 39 | 33 | 31 | 32 |

Equispaced | 6 | 7 | 7 | 9 | 7 | 4 |

Low-Rank-100 | 1 | 1 | 1 | 0 | 0 | 0 |

Low-Rank-50 | 0 | 0 | 0 | 0 | 0 | 0 |

Low-Rank-10 | 0 | 0 | 0 | 0 | 0 | 0 |

Harmonic triples | 2 | 2 | 2 | 3 | 3 | 4 |

Multiple-Harmonic | 3 | 2 | 2 | 2 | 2 | 1 |

Multiple-Geometric | 3 | 4 | 3 | 2 | 2 | 1 |

Equispaced-Geometric Gap | 1 | 1 | 1 | 0 | 0 | 0 |

Matrix type | Number of iterations | |||||
---|---|---|---|---|---|---|

Harmonic squares | 0 | 0 | 0 | 0 | 0 | 1 |

Harmonic | 0 | 0 | 0 | 0 | 0 | 0 |

Harmonic roots | 0 | 0 | 0 | 0 | 0 | 0 |

Very fast Geometric | 0 | 0 | 0 | 0 | 0 | 0 |

Fast Geometric | 0 | 0 | 0 | 0 | 0 | 0 |

Geometric | 0 | 0 | 0 | 0 | 0 | 1 |

Moderate Geometric | 0 | 0 | 0 | 0 | 0 | 0 |

Slow Geometric | 3 | 4 | 4 | 4 | 4 | 2 |

Very Slow Geometric | 14 | 17 | 18 | 15 | 15 | 12 |

Dense Equispaced | 16 | 17 | 19 | 18 | 16 | 15 |

Equispaced-1000 | 3 | 4 | 3 | 4 | 3 | 2 |

Low-Rank-100 | 0 | 0 | 0 | 0 | 0 | 0 |

Low-Rank-50 | 0 | 0 | 0 | 0 | 0 | 0 |

Low-Rank-10 | 0 | 0 | 0 | 0 | 0 | 0 |

Harmonic triples | 1 | 2 | 2 | 2 | 2 | 2 |

Multiple-Harmonic | 2 | 2 | 2 | 1 | 1 | 1 |

Multiple-Geometric | 1 | 3 | 1 | 2 | 0 | 0 |

Equispaced-Geometric Gap | 0 | 0 | 0 | 0 | 0 | 0 |

method. The driving force that ensures convergence is the monotonicity proper- ty, which is easily concluded from the Cauchy-Poincaré interlacing theorems. The proof indicates too important points. First, there is a lot of freedom in choosing the information matrix, and that monotonicity is guaranteed as long as we achieve proper orthogonalizations. Second, the rate of convergence depends on the “quality” of the information matrix. This raises the question of how to define this matrix. Since the algorithm is aimed at computing a cluster of exterior eigenvalues, a Krylov information matrix is a good choice. In [

Indeed, the results of our experiments are quite encouraging. We see that the algorithm requires a remarkably small number of iterations. In particular, it efficiently handles various kinds of low-rank matrices. In these matrices the initial orthonormal matrix is often sufficient for accurate computation of the desired eigenpairs. The algorithm is also successful in computing eigenvalues of “difficult” matrices like “Dense equispaced” or “Very slow geometric decay”.

Dax, A. (2017) A New Type of Restarted Krylov Methods. Advances in Linear Algebra & Matrix Theory, 7, 18-28. https://doi.org/10.4236/alamt.2017.71003