A Nonmonotone Line Search Method for Regression Analysis
Affiliation(s)
College of Mathematics and Information Science, Guangxi University, Nanning, Guangxi, China.
College of Mathematics and Information Science, Guangxi University, Nanning, Guangxi, China.
ABSTRACT
In this paper, we propose a nonmonotone line search combining with the search direction (G. L. Yuan and Z. X.Wei, New Line Search Methods for Unconstrained Optimization, Journal of the Korean Statistical Society, 38(2009), pp. 29-39.) for regression problems. The global convergence of the given method will be established under suitable conditions. Numerical results show that the presented algorithm is more competitive than the normal methods.
In this paper, we propose a nonmonotone line search combining with the search direction (G. L. Yuan and Z. X.Wei, New Line Search Methods for Unconstrained Optimization, Journal of the Korean Statistical Society, 38(2009), pp. 29-39.) for regression problems. The global convergence of the given method will be established under suitable conditions. Numerical results show that the presented algorithm is more competitive than the normal methods.
Cite this paper
nullG. Yuan and Z. Wei, "A Nonmonotone Line Search Method for Regression Analysis," Journal of Service Science and Management, Vol. 2 No. 1, 2009, pp. 36-42. doi: 10.4236/jssm.2009.21005.
nullG. Yuan and Z. Wei, "A Nonmonotone Line Search Method for Regression Analysis," Journal of Service Science and Management, Vol. 2 No. 1, 2009, pp. 36-42. doi: 10.4236/jssm.2009.21005.
References
[1] D. M. Bates and D. G. Watts, “Nonlinear regression analysis and its applications,” New York: John Wiley & Sons, 1988. [2] S. Chatterjee and M. Machler, “Robust regression: A weighted least squares approach, communications in sta-tistics,” Theorey and Methods, 26, pp. 1381-1394, 1997. [3] R. Christensen, “Analysis of variance, design and regres-sion: Applied statistical methods,” New York: Chapman and Hall, 1996. [4] N. R. Draper and H. Smith, “Applied regression analysis,” 3rd ed., New York: John Wiley & Sons, 1998. [5] F. A. Graybill and H. K. Iyer, “Regression analysis: Con-cepts and applications, Belmont,” CA: Duxbury Press, 1994. [6] R. F. Gunst and R. L. Mason, “Regression analysis and its application: A data-Oriented approach,” New York: Mar-cel Dekker, 1980. [7] R. H. Myers, “Classical and modern regression with ap-plications,” 2nd edition, Boston: PWS-KENT Publishing Company, 1990. [8] R. C. Rao, “Linear statistical inference and its applica-tions,”New York: John Wiley & Sons, 1973. [9] D. A. Ratkowsky, “Nonlinear regression modeling: A uni?ed practical approach,” New York: Marcel Dekker, 1983. [10] D. A. Ratkowsky, “Handbook of nonlinear regression modeling,” New York: Marcel Dekker, 1990. [11] A. C. Rencher, “Methods of multivariate analysis,” New York: John Wiley & Sons, 1995. [12] G. A. F. Seber and C. J. Wild, “Nonlinear regression,” New York: John Wiley & Sons, 1989. [13] A. Sen and M. Srivastava, “Regression analysis: Theory, methods, and applications,” New York: Springer-Verlag, 1990. [14] J. Fox, “Linear statistical models and related methods,” New York: John Wiley & Sons, 1984. [15] S. Haberman and A. E. Renshaw, “Generalized linear models and actuarial science,” The Statistician, 45, pp. 407-436, 1996. [16] S. Haberman and A. E. Renshaw, “Generalized linear models and excess mortality from peptic ulcers,” Insur-ance: Mathematics and Economics, 9, pp. 147-154, 1990. [17] R. R. Hocking, “The analysis and selection of variables in linear regression,” Biometrics, 32, pp. 1-49, 1976. [18] P. McCullagh and J. A. Nelder, “Generalized linear mod-els,” London: Chapman and Hall, 1989. [19] J. A. Nelder and R. J. Verral, “Credibility theory and gener-alized linear models,” ASTIN Bulletin, 27, pp. 71-82, 1997. [20] M. Raydan, “The Barzilai and Borwein gradient method for the large scale unconstrained minimization prob-lem,”SIAM Journal of Optimization, 7, pp. 26-33, 1997. [21] J. Schropp, “A note on minimization problems and multistep methods,” Numerical Mathematics, 78, pp. 87-101, 1997. [22] J. Schropp, “One-step and multistep procedures for con-strained minimization problems,” IMA Journal of Nu-merical Analysis, 20, pp. 135-152, 2000. [23] D. J. Van. Wyk, “Di?erential optimization techniques,” Appl. Math. Model, 8, pp. 419-424, 1984. [24] M. N. Vrahatis, G. S. Androulakis, J. N. Lambrinos, and G. D. Magolas, “A class of gradient unconstrained minimiza-tion algorithms with adaptive stepsize,” Journal of Compu-tational and Applied Mathematics, 114, pp. 367-386, 2000. [25] G. L. Yuan and X. W. Lu, “A new line search method with trust region for unconstrained optimization,” Com-munications on Applied Nonlinear Analysis, Vol. 15, No. 1, pp. 35-49, 2008. [26] G. Yuan, X. Lu, and Z. Wei, “New two-point stepsize gradient methods for solving unconstrained optimization problems,” Natural Science Journal of Xiangtan Univer-sity, (1)29, pp. 13-15, 2007. [27] G. L. Yuan and Z. X. Wei, “New line search methods for unconstrained optimization,” Journal of the Korean Statis-tical Society, 38, pp. 29-39, 2009. [28] Y. Dai, “A nonmonotone conjugate gradient algorithm for unconstrained optimization,” Journal of Systems Science and Complexity, 15, pp. 139-145, 2002. [29] Y. Dai and Y. Yuan, “A nonlinear conjugate gradient with a strong global convergence properties,” SIAM Journal of Optimization, 10, pp. 177-182, 2000. [30] R. Fletcher, “Practical Method of Optimization,” Vol 1: Unconstrained Optimization, 2nd edition, Wiley, New York, 1997. [31] R. Fletcher and C. Reeves, “Function minimization by conjugate gradients,” The Computer Journal, 7, pp, 149-154, 1964. [32] Y. Liu and C. Storey, “E?cient generalized conjugate gradient algorithms, part 1: theory,” Journal of Optimiza-tion Theory and Application, 69, pp. 17-41, 1992. [33] E. Polak and G. Ribiere, “Note sur la convergence de directions conjugees,” Rev. Francaise informat Recherche Operatinelle, 3e Annee, 16, pp. 35-43, 1969. [34] Z. Wei, G. Li, and L. Qi, “New nonlinear conjugate gra-dient formulas for large-scale unconstrained optimization problems,” Applied Mathematics and Computation, 179, pp. 407-430, 2006. [35] Z. Wei, S. Yao, and L. Liu, “The convergence properties of some new conjugate gradient methods,” Applied Mathematics and Computation, 183, pp. 1341-1350, 2006. [36] G. L. Yuan, “Modi?ed nonlinear conjugate gradient meth-ods with su?cient descent property for large-scale opti-mization problems,” Optimization Letters, DOI: 10. 1007/s11590-008-0086-5, 2008. [37] G. L. Yuan and X. W. Lu, “A modi?ed PRP conjugate gradient method,” Annals of Operations Research, 166, pp. 73-90, 2009. [38] J. C. Gibert and J. Nocedal, “Global convergence proper-ties of conjugate gradient methods for optimization,” SIAM Journal of Optimization, 2, pp. 21-42, 1992. [39] J. J. Mor′e, B. S. Garbow, and K. E. Hillstrome, “Testing unconstrained optimization software,” ACM Transactions Math. Software, 7, pp. 17-41, 1981. [40] L. Grippo, F. Lamparillo, and S. Lucidi, “A nonmonotone line search technique for Newton’s method,” SIAM Jour-nal of Numerical Analysis, 23, pp. 707-716, 1986. [41] L. Grippo, F. Lamparillo, and S. Lucidi, “A truncate New-ton method with nonmonotone line search for uncon-strained optimization,” Journal of Optimization Theory and Applications, 60, pp. 401-419, 1989. [42] L. Grippo, F. Lamparillo, and S. Lucidi, “A class of non-monotone stabilization methods in unconstrained optimi-zation,” Numerical Mathematics, 59, pp. 779-805, 1991. [43] G. H. Liu, J. Y. Han, and D. F. Sun, “Global convergence analysis of the BFGS algorithm with nonmonotone line-search,” Optimization, Vol. 34, pp. 147-159, 1995. [44] G. H. Liu, J. M. Peng, The convergence properties of a nonmonotonic algorithm,” Journal of Computational Mathematics, 1, pp. 65-71, 1992. [45] J. Y. Han and G. H. Liu, “Global convergence analysis of a new nonmonotone BFGS algorithm on convex objective functions,” Computational Optimization and Applications 7, pp. 277-289, 1997. [46] G. L. Yuan and Z. X. Wei, “The superlinear convergence analysis of a nonmonotone BFGS algorithm on convex objective functions,” Acta Mathematica Sinica, English Series, Vol. 24, No. 1, pp. 35-42, 2008. [47] H. C. Zhang and W. W. Hager, “A nonmonotone line search technique and its application to unconstrained op-timization,” SIAM Journal of Optimization, Vol. 14, No. 4, pp. 1043-1056, 2004. [48] M. R. Hestenes and E. Stiefel, “Method of conjugate gra-dient for solving linear equations,” J, Res. Nat. Bur. Stand., 49, pp. 409-436, 1952. [49] S. Chatterjee, A. S. Hadi, and B. Price, “Regression analy-sis by example,” 3rd Edition, John Wiley & Sons, 2000.
[1] D. M. Bates and D. G. Watts, “Nonlinear regression analysis and its applications,” New York: John Wiley & Sons, 1988. [2] S. Chatterjee and M. Machler, “Robust regression: A weighted least squares approach, communications in sta-tistics,” Theorey and Methods, 26, pp. 1381-1394, 1997. [3] R. Christensen, “Analysis of variance, design and regres-sion: Applied statistical methods,” New York: Chapman and Hall, 1996. [4] N. R. Draper and H. Smith, “Applied regression analysis,” 3rd ed., New York: John Wiley & Sons, 1998. [5] F. A. Graybill and H. K. Iyer, “Regression analysis: Con-cepts and applications, Belmont,” CA: Duxbury Press, 1994. [6] R. F. Gunst and R. L. Mason, “Regression analysis and its application: A data-Oriented approach,” New York: Mar-cel Dekker, 1980. [7] R. H. Myers, “Classical and modern regression with ap-plications,” 2nd edition, Boston: PWS-KENT Publishing Company, 1990. [8] R. C. Rao, “Linear statistical inference and its applica-tions,”New York: John Wiley & Sons, 1973. [9] D. A. Ratkowsky, “Nonlinear regression modeling: A uni?ed practical approach,” New York: Marcel Dekker, 1983. [10] D. A. Ratkowsky, “Handbook of nonlinear regression modeling,” New York: Marcel Dekker, 1990. [11] A. C. Rencher, “Methods of multivariate analysis,” New York: John Wiley & Sons, 1995. [12] G. A. F. Seber and C. J. Wild, “Nonlinear regression,” New York: John Wiley & Sons, 1989. [13] A. Sen and M. Srivastava, “Regression analysis: Theory, methods, and applications,” New York: Springer-Verlag, 1990. [14] J. Fox, “Linear statistical models and related methods,” New York: John Wiley & Sons, 1984. [15] S. Haberman and A. E. Renshaw, “Generalized linear models and actuarial science,” The Statistician, 45, pp. 407-436, 1996. [16] S. Haberman and A. E. Renshaw, “Generalized linear models and excess mortality from peptic ulcers,” Insur-ance: Mathematics and Economics, 9, pp. 147-154, 1990. [17] R. R. Hocking, “The analysis and selection of variables in linear regression,” Biometrics, 32, pp. 1-49, 1976. [18] P. McCullagh and J. A. Nelder, “Generalized linear mod-els,” London: Chapman and Hall, 1989. [19] J. A. Nelder and R. J. Verral, “Credibility theory and gener-alized linear models,” ASTIN Bulletin, 27, pp. 71-82, 1997. [20] M. Raydan, “The Barzilai and Borwein gradient method for the large scale unconstrained minimization prob-lem,”SIAM Journal of Optimization, 7, pp. 26-33, 1997. [21] J. Schropp, “A note on minimization problems and multistep methods,” Numerical Mathematics, 78, pp. 87-101, 1997. [22] J. Schropp, “One-step and multistep procedures for con-strained minimization problems,” IMA Journal of Nu-merical Analysis, 20, pp. 135-152, 2000. [23] D. J. Van. Wyk, “Di?erential optimization techniques,” Appl. Math. Model, 8, pp. 419-424, 1984. [24] M. N. Vrahatis, G. S. Androulakis, J. N. Lambrinos, and G. D. Magolas, “A class of gradient unconstrained minimiza-tion algorithms with adaptive stepsize,” Journal of Compu-tational and Applied Mathematics, 114, pp. 367-386, 2000. [25] G. L. Yuan and X. W. Lu, “A new line search method with trust region for unconstrained optimization,” Com-munications on Applied Nonlinear Analysis, Vol. 15, No. 1, pp. 35-49, 2008. [26] G. Yuan, X. Lu, and Z. Wei, “New two-point stepsize gradient methods for solving unconstrained optimization problems,” Natural Science Journal of Xiangtan Univer-sity, (1)29, pp. 13-15, 2007. [27] G. L. Yuan and Z. X. Wei, “New line search methods for unconstrained optimization,” Journal of the Korean Statis-tical Society, 38, pp. 29-39, 2009. [28] Y. Dai, “A nonmonotone conjugate gradient algorithm for unconstrained optimization,” Journal of Systems Science and Complexity, 15, pp. 139-145, 2002. [29] Y. Dai and Y. Yuan, “A nonlinear conjugate gradient with a strong global convergence properties,” SIAM Journal of Optimization, 10, pp. 177-182, 2000. [30] R. Fletcher, “Practical Method of Optimization,” Vol 1: Unconstrained Optimization, 2nd edition, Wiley, New York, 1997. [31] R. Fletcher and C. Reeves, “Function minimization by conjugate gradients,” The Computer Journal, 7, pp, 149-154, 1964. [32] Y. Liu and C. Storey, “E?cient generalized conjugate gradient algorithms, part 1: theory,” Journal of Optimiza-tion Theory and Application, 69, pp. 17-41, 1992. [33] E. Polak and G. Ribiere, “Note sur la convergence de directions conjugees,” Rev. Francaise informat Recherche Operatinelle, 3e Annee, 16, pp. 35-43, 1969. [34] Z. Wei, G. Li, and L. Qi, “New nonlinear conjugate gra-dient formulas for large-scale unconstrained optimization problems,” Applied Mathematics and Computation, 179, pp. 407-430, 2006. [35] Z. Wei, S. Yao, and L. Liu, “The convergence properties of some new conjugate gradient methods,” Applied Mathematics and Computation, 183, pp. 1341-1350, 2006. [36] G. L. Yuan, “Modi?ed nonlinear conjugate gradient meth-ods with su?cient descent property for large-scale opti-mization problems,” Optimization Letters, DOI: 10. 1007/s11590-008-0086-5, 2008. [37] G. L. Yuan and X. W. Lu, “A modi?ed PRP conjugate gradient method,” Annals of Operations Research, 166, pp. 73-90, 2009. [38] J. C. Gibert and J. Nocedal, “Global convergence proper-ties of conjugate gradient methods for optimization,” SIAM Journal of Optimization, 2, pp. 21-42, 1992. [39] J. J. Mor′e, B. S. Garbow, and K. E. Hillstrome, “Testing unconstrained optimization software,” ACM Transactions Math. Software, 7, pp. 17-41, 1981. [40] L. Grippo, F. Lamparillo, and S. Lucidi, “A nonmonotone line search technique for Newton’s method,” SIAM Jour-nal of Numerical Analysis, 23, pp. 707-716, 1986. [41] L. Grippo, F. Lamparillo, and S. Lucidi, “A truncate New-ton method with nonmonotone line search for uncon-strained optimization,” Journal of Optimization Theory and Applications, 60, pp. 401-419, 1989. [42] L. Grippo, F. Lamparillo, and S. Lucidi, “A class of non-monotone stabilization methods in unconstrained optimi-zation,” Numerical Mathematics, 59, pp. 779-805, 1991. [43] G. H. Liu, J. Y. Han, and D. F. Sun, “Global convergence analysis of the BFGS algorithm with nonmonotone line-search,” Optimization, Vol. 34, pp. 147-159, 1995. [44] G. H. Liu, J. M. Peng, The convergence properties of a nonmonotonic algorithm,” Journal of Computational Mathematics, 1, pp. 65-71, 1992. [45] J. Y. Han and G. H. Liu, “Global convergence analysis of a new nonmonotone BFGS algorithm on convex objective functions,” Computational Optimization and Applications 7, pp. 277-289, 1997. [46] G. L. Yuan and Z. X. Wei, “The superlinear convergence analysis of a nonmonotone BFGS algorithm on convex objective functions,” Acta Mathematica Sinica, English Series, Vol. 24, No. 1, pp. 35-42, 2008. [47] H. C. Zhang and W. W. Hager, “A nonmonotone line search technique and its application to unconstrained op-timization,” SIAM Journal of Optimization, Vol. 14, No. 4, pp. 1043-1056, 2004. [48] M. R. Hestenes and E. Stiefel, “Method of conjugate gra-dient for solving linear equations,” J, Res. Nat. Bur. Stand., 49, pp. 409-436, 1952. [49] S. Chatterjee, A. S. Hadi, and B. Price, “Regression analy-sis by example,” 3rd Edition, John Wiley & Sons, 2000.