JILSA  Vol.5 No.1 , February 2013
LP-SVR Model Selection Using an Inexact Globalized Quasi-Newton Strategy
Abstract: In this paper we study the problem of model selection for a linear programming-based support vector machine for regression. We propose generalized method that is based on a quasi-Newton method that uses a globalization strategy and an inexact computation of first order information. We explore the case of two-class, multi-class, and regression problems. Simulation results among standard datasets suggest that the algorithm achieves insignificant variability when measuring residual statistical properties.
Cite this paper: P. Rivas-Perea, J. Cota-Ruiz, J. Venzor, D. Chaparro and J. Rosiles, "LP-SVR Model Selection Using an Inexact Globalized Quasi-Newton Strategy," Journal of Intelligent Learning Systems and Applications, Vol. 5 No. 1, 2013, pp. 19-28. doi: 10.4236/jilsa.2013.51003.

[1]   A. J. Smola and B. Scholkopf, “A Tutorial on Support Vector Regression,” Statistics and Computing, Vol. 14, No. 3, 2004, pp. 199-222. doi:10.1023/B:STCO.0000035301.49549.88

[2]   D. Anguita, A. Boni, S. Ridella, F. Rivieccio and D. Sterpi, “Theoretical and Practical Model Selection Methods for Support Vector Classifiers,” Support Vector Machines: Theory and Applications, Vol. 177, 2005, pp. 159-179. doi:10.1007/10984697_7

[3]   K. Duan, S. Keerthi and A. Poo, “Evaluation of Simple Performance Measures for Tuning SVM Hyperparameters,” Neurocomputing, Vol. 51, 2003, pp. 41-59. doi:10.1016/S0925-2312(02)00601-X

[4]   Z. Hui-ren and P. Zheng, “Method for Selecting Parameters of Least Squares Support Vector Machines Based on GA and Bootstrap,” Journal of System Simulation, Vol. 12, 2008.

[5]   D. Anguita, S. Ridella, F. Rivieccio and R. Zunino, “Hyperparameter Design Criteria for Support Vector Classifiers,” Neurocomputing, Vol. 55, No. 1-2, 2003, pp. 109-134. doi:10.1016/S0925-2312(03)00430-2

[6]   L. Wang and S. O. Service, “Support Vector Machines: Theory and Applications,” Studies in Fuzziness and Soft Computing, Springer-Verlag, Berlin, 2005.

[7]   G. Cawley, “Leave-One-Out Cross-Validation Based Model Selection Criteria for Weighted Ls-Svms,” IEEE International Conference on Neural Networks, 16-21 July 2006. doi:10.1109/IJCNN.2006.246634

[8]   P. R. Perea, “Algorithms for Training Large-Scale Linear Programming Support Vector Regression and Classification,” Ph.D. Thesis, The University of Texas, El Paso, 2011.

[9]   J. Dennis and R. Schnabel, “Numerical Methods for Unconstrained Optimization and Nonlinear Equations,” Society for Industrial Mathematics, 1996. doi:10.1137/1.9781611971200

[10]   M. Argaez and L. Velazquez, “A New Infeasible InteriorPoint Algorithm for Linear Programming,” Proceedings of the 2003 Conference on Diversity in Computing, ACM, New York, 2003, pp. 12-14.

[11]   J. Mercer, “Functions of Positive and Negative Type, and Their Connection with the Theory of Integral Equations,” Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character, Vol. 209, No. 441-458, 1909, pp. 415-446. doi:10.1098/rsta.1909.0016

[12]   R. Courant and D. Hilbert, “Methods of Mathematical Physics,” Interscience, New York, 1966.

[13]   Z. Lu, J. Sun and K. R. Butts, “Linear Programming Support Vector Regression with Wavelet Kernel: A New Approach to Nonlinear Dynamical Systems Identification,” Mathematics and Computers in Simulation, Vol. 79, No. 7, 2009, pp. 2051-2063. doi:10.1016/j.matcom.2008.10.011

[14]   Y. Torii and S. Abe, “Decomposition Techniques for Training Linear Programming Support Vector Machines,” Neurocomputing, Vol. 72, No. 4-6, 2009, pp. 973-984. doi:10.1016/j.neucom.2008.04.008

[15]   L. Zhang and W. Zhou, “On the Sparseness of 1-Norm Support Vector Machines,” Neural Networks, Vol. 23, No. 3, 2010, pp. 373-385.

[16]   T. Fawcett, “Roc Graphs: Notes and Practical Considerations for Researchers,” Machine Learning, Vol. 31, 2004, pp. 1-38.

[17]   J. Nocedal and S. Wright, “Numerical Optimization,” Springer Verlag, New York, 1999. doi:10.1007/b98874

[18]   M. Hestenes, “Pseudoinversus and Conjugate Gradients,” Communications of the ACM, Vol. 18, No. 1, 1975, pp. 40-43. doi:10.1145/360569.360658