Kybernetika 49 no. 2, 224-235, 2013

Recursive form of general limited memory variable metric methods

Ladislav Lukšan and Jan Vlček

Abstract:

In this report we propose a new recursive matrix formulation of limited memory variable metric methods. This approach can be used for an arbitrary update from the Broyden class (and some other updates) and also for the approximation of both the Hessian matrix and its inverse. The new recursive formulation requires approximately $4 m n$ multiplications and additions per iteration, so it is comparable with other efficient limited memory variable metric methods. Numerical experiments concerning Algorithm 1, proposed in this report, confirm its practical efficiency.

Keywords:

algorithms, unconstrained optimization, large scale optimization, variable metric updates, recursive matrix formulation, limited memory methods

Classification:

49K35, 90C06, 90C47, 90C51

References:

  1. I. Bongartz, A. R. Conn, N. Gould and P. L. Toint: CUTE: constrained and unconstrained testing environment. ACM Trans. Math. Software 21 (1995), 123-160.   CrossRef
  2. R. H. Byrd, J. Nocedal and R. B. Schnabel: Representation of quasi-Newton matrices and their use in limited memory methods. Math. Programming 63 (1994), 129-156.   CrossRef
  3. W. C. Davidon: Optimally conditioned optimization algorithms without line searches. Math. Programming 9 (1975), 1-30.   CrossRef
  4. E. D. Dolan and J. J. Moré: Benchmarking optimization software with performance profiles. Math. Programming 91 (2002), 201-213.   CrossRef
  5. S. Hoshino: A formulation of variable metric methods. J. Institute of Mathematics and its Applications 10 (1972), 394-403.   CrossRef
  6. L. Lukšan: Quasi-Newton methods without projections for unconstrained minimization. Kybernetika 18 (1982), 290-306.   CrossRef
  7. L. Lukšan, C. Matonoha and J. Vlček: Sparse Test Problems for Unconstrained Optimization. Report V-1064, Institute of Computer Science AS CR, Prague 2010.   CrossRef
  8. L. Lukšan, C. Matonoha and J. Vlček: Modified CUTE Problems for Sparse Unconstrained Optimization. Report V-1081, Institute of Computer Science AS CR, Prague 2010.   CrossRef
  9. L. Lukšan amd E. Spedicato: Variable metric methods for unconstrained optimization and nonlinear least squares. J. Comput. Appl. Math. 124 (2000), 61-95.   CrossRef
  10. H. Matthies and G.Strang: The solution of nonlinear finite element equations. Internat. J. Numer. Methods Engrg. 14 (1979), 1613-1623.   CrossRef
  11. J. Nocedal: Updating quasi-Newton matrices with limited storage. Math. Comput. 35 (1980), 773-782.   CrossRef
  12. J. Vlček and L. Lukšan: Generalizations of the Limited-memory BFGS Method Based on Quasi-product Form of Update. Report V-1060, Institute of Computer Science AS CR, Prague 2009.   CrossRef