NS  Vol.3 No.1 , January 2011
A hybrid conjugate gradient method for optimization problems
A hybrid method of the Polak-Ribière-Polyak (PRP) method and the Wei-Yao-Liu (WYL) method is proposed for unconstrained optimization pro- blems, which possesses the following properties: i) This method inherits an important property of the well known PRP method: the tendency to turn towards the steepest descent direction if a small step is generated away from the solution, preventing a sequence of tiny steps from happening; ii) The scalar holds automatically; iii) The global convergence with some line search rule is established for nonconvex functions. Numerical results show that the method is effective for the test problems.

Cite this paper
Li, X. and Zhao, X. (2011) A hybrid conjugate gradient method for optimization problems. Natural Science, 3, 85-90. doi: 10.4236/ns.2011.31012.
[1]   Ahmed, T. and Storey, D. (1990) Efficient hybrid conjugate gradient techniques. Journal of Optimization Theory and Applications, 64, 379-394. doi:10.1007/BF00939455

[2]   Al-Baali, A. (1985) Descent property and global convergence of the Flecher-Reeves method with inexact line search. IMA Journal of Numerical Analysis, 5, 121-124. doi:10.1093/imanum/5.1.121

[3]   Bongartz, K.E., Conn, A.R., Gould, N.I.M. and Toint, P.L. (1995) CUTE: Constrained and unconstrained testing environments. ACM Transactions on Mathematical Software, 21, 123-160. doi:10.1145/200979.201043

[4]   Dai, Y. (2002) A nonmonotone conjugate gradient algorithm for unconstrained optimization. Journal of Systems Science and Complexity, 15, 139-145.

[5]   Dai, Y. and Yuan, Y. (2000) A nonlinear conjugate gradient with a strong global convergence properties. SIAM Journal on Optimization, 10, 177-182. doi:10.1137/S1052623497318992

[6]   Dai, Y. and Yuan, Y. (1998) Nonlinear conjugate gradient methods. Shanghai Scientific and Technical Publishers, Shanghai.

[7]   Dolan, E.D. and Moré, J.J. (2002) Benchmarking optimization software with performance profiles. Mathematical Programming, 91, 201-213. doi:10.1007/s101070100263

[8]   Fletcher, R. (1997) Practical Method of Optimization, Vol I: Unconstrained Optimization. 2nd Edition, Wiley, New York.

[9]   Fletcher, R. and Reeves, C. (1964) Function minimization bu conjugate gradients. Computer Journal, 7, 149-154. doi:10.1093/comjnl/7.2.149

[10]   Gibert, J.C. and Nocedal, J. (1992) Global convergence properties of conugate gradient methods for optimization. SIAM Journal on Optimization, 2, 21-42. doi:10.1137/0802003

[11]   Grippo, L. and Lucidi, S. (1997) A globally convergent version of the Polak-Ribière gradient method. Mathematical Programming, 78, 375-391. doi:10.1007/BF02614362

[12]   Hager, W.W. and Zhang, H.C. (2005) A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM Journal on Optimization, 16, 170-192. doi:10.1137/030601880

[13]   Hestenes, M.R. and Stiefel, E. (1952) Method of conjugate gradient for solving linear equations. Journal of Research National Bureau of Standards, 49, 409-436.

[14]   Hu, Y.F. and Storey, C. (1991) Global convergence result for conjugate method. Journal of Optimization Theory and Applications, 71, 399-405. doi:10.1007/BF00939927

[15]   Li, G., Tang, C. and Wei, Z. (2007) New conjugacy condition and related new conjugate gradient methods for unconstrained optimization. Journal of Computational and Applied Mathematics, 2, 523-539. doi:10.1016/j.cam.2006.03.005

[16]   Liu, Y. and Storey, C. (1992) Effcient generalized conjugate gradient algorithms, part 1: theory. Journal of Optimization Theory and Applications, 69, 17-41.

[17]   Moré, J.J., Garbow, B.S. and Hillstrome, K.E. (1981) Testing unconstrained optimization software. ACM Transactions on Mathematical Software, 7, 17-41. doi:10.1145/355934.355936

[18]   Polak, E. and Ribiere, G. (1969) Note sur la xonvergence de directions conjugees. Rev. Francaise informat Recherche Operatinelle, 3e Annee, 16, 35-43.

[19]   Powell, M.J.D. (1984) Nonconvex minimization calculations and the conjugate gradient method. Lecture Notes in Mathematics, 1066, 122-141.

[20]   Wei, Z., Li, G. and Qi, L. (2006) New nonlinear conjugate gradient formulas for large-scale unconstrained optimization problems. Applied Mathematics and Computation, 179, 407-430. doi:10.1016/j.amc.2005.11.150

[21]   Wei, Z., Yao, S. and Liu, L. (2006) The convergence properties of some new conjugate gradient methods. Applied Mathematics and Computation, 183, 1341-1350. doi:10.1016/j.amc.2006.05.150

[22]   Yu, G.H. (2007) Nonlinear self-scaling conjugate gradient methods for large-scale optimization problems. Thesis of Doctor’s Degree, Sun Yat-Sen University.

[23]   Yuan, G.L. (2009) Modified nonlinear conjugate gradient methods with sufficient descent property for large-scale optimization problems. Optimization Letters, 3, 11-21. doi:10.1007/s11590-008-0086-5

[24]   Yuan, G.L. and Lu, X.W. (2008) A new line search method with trust region for unconstrained optimization. Communications on Applied Nonlinear Analysis, 15, 35-49.

[25]   Yuan, G.L. and Lu, X.W. (2009) A modified PRP conjugate gradient method. Annals of Operations Research, 166, 73-90. doi:10.1007/s10479-008-0420-4

[26]   Yuan, G.L., Lu, X.W. and Wei, Z.X. (2007) New two-point stepsize gradient methods for solving unconstrained optimization problems. Natural Science Journal of Xiangtan University, 29, 13-15.

[27]   Yuan, Y. and Sun, W. (1999) Theory and Methods of Optimization. Science Press of China, Beijing.

[28]   Yuan, G.L. and Wei, Z.X. (2009) New line search methods for unconstrained optimization. Journal of the Korean Statistical Society, 38, 29-39. doi:10.1016/j.jkss.2008.05.004

[29]   Yuan, G.L. and Wei, Z.X. (2008) The superlinear convergence analysis of a nonmonotone BFGS algorithm on convex objective functions. Acta Mathematica Sinica, 24, 35-42. doi:10.1007/s10114-007-1012-y

[30]   Yuan, G.L. and Wei, Z.X. (2010) Convergence analysis of a modified BFGS method on convex minimizations. Computational Optimization and Applications, 47, 237-255. doi:10.1007/s10589-008-9219-0

[31]   Yuan, G.L. and Wei, Z.X. (2009) A Rank-One fitting method for unconstrained optimization problems. Mathematica Applicata, 22, 118-122.

[32]   Zhang, H.C. and Hager, W.W. (2004) A nonmonotone line search technique and its application to unconstrained optimization. SIAM Journal on Optimization, 14, 1043-1056. doi:10.1137/S1052623403428208

[33]   Zhang, L., Zhou, W. and Li, D. (2006) A descent modified Polak-Ribiére-Polyak conjugate method and its global convergence. IMA Journal on Numerical Analysis, 26, 629-649. doi:10.1093/imanum/drl016

[34]   Zoutendijk, G. (1970) Nonlinear programming computational methods. In: Abadie, J. Ed., Integer and Nonlinear Programming, NorthHolllad, Amsterdam, 37-86.