A new multi-update conjugate gradient method for unconstrained optimization

<p>Issam A R Moghrabi</p>

A new multi-update conjugate gradient method for unconstrained optimization

Issam A R Moghrabi

Gulf University for Science and Technology, Kuwait

: J Phys Res Appl

Abstract

In this work the emphasis is placed on methods which combine the merits of both the CG and Quasi-Newton methods. The paper presents a new Variable-Metric quasi-Newton-like CG algorithm that implicitly employs a self-scaling update formula in the computation of the search direction. More specifically, the classical Conjugate Gradient method is revisited and combined with the multi-step Quasi-Newton methods [] to derive a hybrid CG algorithm that utilizes data available from recent iterations. The computation of the search directions, at each iteration, is done in two steps. The aim of developing such self-scaling Variable Metric CG methods is to improve the quality of the generated search direction vectors. Such vectors are built at minimal extra storage and are efficiently computed. The computation involves a number of inner products and requires just extra O(n) storage requirements. Thus, the resulting method is a memoryless Variable-Metric algorithm. The new algorithm is numerically compared to well-known similar memoryless quasi-Newton methods in order to assess the merits of the new algorithm. The methods are tested on a collection of 300 varied dimensionality test problems with dimensions ranging 2 ≤ n ≤ 10000. A cubic fitting technique line search strategy is used for all methods. The algorithms terminate when the gradient vector magnitude is less than 10-5. All methods are restarted every n iterations or whenever a certain criteria is met.