A New Algorithm for Generalized Least Squares Factor Analysis with a Majorization Technique
Author(s)
Kohei Adachi
ABSTRACT
Factor analysis (FA) is a time-honored multivariate analysis procedure for exploring the factors underlying observed variables. In this paper, we propose a new algorithm for the generalized least squares (GLS) estimation in FA. In the algorithm, a majorization step and diagonal steps are alternately iterated until convergence is reached, where Kiers and ten Berge’s (1992) majorization technique is used for the former step, and the latter ones are formulated as minimizing simple quadratic functions of diagonal matrices. This procedure is named a majorizing-diagonal (MD) algorithm. In contrast to the existing gradient approaches, differential calculus is not used and only elmentary matrix computations are required in the MD algorithm. A simuation study shows that the proposed MD algorithm recovers parameters better than the existing algorithms.
Factor analysis (FA) is a time-honored multivariate analysis procedure for exploring the factors underlying observed variables. In this paper, we propose a new algorithm for the generalized least squares (GLS) estimation in FA. In the algorithm, a majorization step and diagonal steps are alternately iterated until convergence is reached, where Kiers and ten Berge’s (1992) majorization technique is used for the former step, and the latter ones are formulated as minimizing simple quadratic functions of diagonal matrices. This procedure is named a majorizing-diagonal (MD) algorithm. In contrast to the existing gradient approaches, differential calculus is not used and only elmentary matrix computations are required in the MD algorithm. A simuation study shows that the proposed MD algorithm recovers parameters better than the existing algorithms.
KEYWORDS
Exploratory Factor Analysis, Generalized Least Squares Estimation, Matrix Computations, Majorization
Exploratory Factor Analysis, Generalized Least Squares Estimation, Matrix Computations, Majorization
Cite this paper
Adachi, K. (2015) A New Algorithm for Generalized Least Squares Factor Analysis with a Majorization Technique. Open Journal of Statistics, 5, 165-172. doi: 10.4236/ojs.2015.53020.
Adachi, K. (2015) A New Algorithm for Generalized Least Squares Factor Analysis with a Majorization Technique. Open Journal of Statistics, 5, 165-172. doi: 10.4236/ojs.2015.53020.
References
[1] Harman, H.H. (1976) Modern Factor Analysis. 3rd Edition, The University of Chicago Press, Chicago. [2] Mulaik, S.A. (2010) Foundations of Factor Analysis. 2nd Edition, CRC Press, Boca Raton. [3] Yanai, H. and Ichikawa, M. (2007) Factor Analysis. In: Rao, C.R. and Sinharay, S., Eds., Handbook of Statistics, Vol. 26: Psychometrics, Elsevier, Amsterdam, 257-296. [4] Anderson, T.W. and Rubin, H. (1956) Statistical Inference in Factor Analysis. In: Neyman, J., Ed., Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, Vol. 5, University of California Press, Berkeley, 111-150. [5] Lange, K. (2010) Numerical Analysis for Statisticians. 2nd Edition, Springer, New York. [6] ten Berge, J.M.F. (1993) Least Squares Optimization in Multivariate Analysis. DSWO Press, Leiden. [7] Jöreskog, K.G. (1967) Some Contributions to Maximum Likelihood Factor Analysis. Psychometrika, 32, 443-482.
http://dx.doi.org/10.1007/BF02289658 [8] Jennrich, R.I. and Robinson, S.M. (1969) A Newton-Raphson Algorithm for Maximum Likelihood Factor Analysis. Psychometrika, 34, 111-123.
http://dx.doi.org/10.1007/BF02290176 [9] Jöreskog, K.G. and Goldberger, A.S. (1972) Factor Analysis by Generalized Least Squares. Psychometrika, 37, 243-250.
http://dx.doi.org/10.1007/BF02306782. [10] Lee, S.Y. (1978) The Gauss-Newton Algorithm for the Weighted Least Squares Factor Analysis. Journal of the Royal Statistical Society: Series D (The Statistician), 27, 103-114.
http://dx.doi.org/10.2307/2987906 [11] Harman, H.H. and Jones, W.H. (1966) Factor Analysis by Minimizing Residuals (Minres). Psychomerika, 31, 351-369.
http://dx.doi.org/10.1007/BF02289468 [12] Rubin, D.B. and Thayer, D.T. (1982) EM Algorithms for ML Factor Analysis. Psychometrika, 47, 69-76.
http://dx.doi.org/10.1007/BF02293851 [13] Dempster, A.P., Laird, N.M. and Rubin, D.B. (1977) Maximum Likelihood from Incomplete Data via the EM Algorithm. Journal of the Royal Statistical Society, Series B, 39, 1-38. [14] Groenen, P.J.F. (1993) The Majorization Approach to Multidimensional Scaling: Some Problems and Extensions. DSWO Press, Leiden. [15] Unkel, S. and Trendafilov, N.T. (2010) A Majorization Algorithm for Simultaneous Parameter Estimation in Robust Exploratory Factor Analysis. Computational Statistics and Data Analysis, 54, 3348-3358.
http://dx.doi.org/10.1016/j.csda.2010.02.003 [16] Unkel, S. and Trendafilov, N.T. (2010) Simultaneous Parameter Estimation in Exploratory Factor Analysis: An Expository Review. International Statistical Review, 78, 363-382.
http://dx.doi.org/10.1111/j.1751-5823.2010.00120.x [17] Adachi, K. (2012) Some Contributions to Data-Fitting Factor Analysis with Empirical Comparisons to Covariance-Fitting Factor Analysis. Journal of the Japanese Society of Computational Statistics, 25, 25-38.
http://dx.doi.org/10.5183/jjscs.1106001_197 [18] Kiers, H.A.L. and ten Berge, J.M.F. (1992) Minimization of a Class of Matrix Trace Functions by Means of Refined Majorization. Psychometrika, 57, 371-382.
http://dx.doi.org/10.1007/BF02295425 [19] Kiers, H.A.L. (1990) Majorization as a Tool for Optimizing a Class of Matrix Functions. Psychometrika, 55, 417-428.
http://dx.doi.org/10.1007/BF02294758 [20] Costa, P.T. and McCrae, R.R. (1992) NEO PI-R Professional Manual: Revised NEO Personality Inventory (NEO PI-R) and NEO Five-Factor Inventory (NEO-FFI). Psychological Assessment Resources, Odessa, FL. [21] Kaiser, H.F. (1958) The Varimax Criterion for Analytic Rotation in Factor Analysis. Psychometrika, 23, 187-200.
http://dx.doi.org/10.1007/BF02289233 [22] Gower, J.C. and Dijksterhuis, G.B. (2004) Procrustes Problems. Oxford University Press, Oxford.
http://dx.doi.org/10.1093/acprof:oso/9780198510581.001.0001 [23] Adachi, K. (2013) Factor Analysis with EM Algorithm Never Gives Improper Solutions When Sample Covariance and Initial Parameter Matrices Are Proper. Psychometrika, 78, 380-394.
http://dx.doi.org/10.1007/s11336-012-9299-8
[1] Harman, H.H. (1976) Modern Factor Analysis. 3rd Edition, The University of Chicago Press, Chicago. [2] Mulaik, S.A. (2010) Foundations of Factor Analysis. 2nd Edition, CRC Press, Boca Raton. [3] Yanai, H. and Ichikawa, M. (2007) Factor Analysis. In: Rao, C.R. and Sinharay, S., Eds., Handbook of Statistics, Vol. 26: Psychometrics, Elsevier, Amsterdam, 257-296. [4] Anderson, T.W. and Rubin, H. (1956) Statistical Inference in Factor Analysis. In: Neyman, J., Ed., Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, Vol. 5, University of California Press, Berkeley, 111-150. [5] Lange, K. (2010) Numerical Analysis for Statisticians. 2nd Edition, Springer, New York. [6] ten Berge, J.M.F. (1993) Least Squares Optimization in Multivariate Analysis. DSWO Press, Leiden. [7] Jöreskog, K.G. (1967) Some Contributions to Maximum Likelihood Factor Analysis. Psychometrika, 32, 443-482.
http://dx.doi.org/10.1007/BF02289658 [8] Jennrich, R.I. and Robinson, S.M. (1969) A Newton-Raphson Algorithm for Maximum Likelihood Factor Analysis. Psychometrika, 34, 111-123.
http://dx.doi.org/10.1007/BF02290176 [9] Jöreskog, K.G. and Goldberger, A.S. (1972) Factor Analysis by Generalized Least Squares. Psychometrika, 37, 243-250.
http://dx.doi.org/10.1007/BF02306782. [10] Lee, S.Y. (1978) The Gauss-Newton Algorithm for the Weighted Least Squares Factor Analysis. Journal of the Royal Statistical Society: Series D (The Statistician), 27, 103-114.
http://dx.doi.org/10.2307/2987906 [11] Harman, H.H. and Jones, W.H. (1966) Factor Analysis by Minimizing Residuals (Minres). Psychomerika, 31, 351-369.
http://dx.doi.org/10.1007/BF02289468 [12] Rubin, D.B. and Thayer, D.T. (1982) EM Algorithms for ML Factor Analysis. Psychometrika, 47, 69-76.
http://dx.doi.org/10.1007/BF02293851 [13] Dempster, A.P., Laird, N.M. and Rubin, D.B. (1977) Maximum Likelihood from Incomplete Data via the EM Algorithm. Journal of the Royal Statistical Society, Series B, 39, 1-38. [14] Groenen, P.J.F. (1993) The Majorization Approach to Multidimensional Scaling: Some Problems and Extensions. DSWO Press, Leiden. [15] Unkel, S. and Trendafilov, N.T. (2010) A Majorization Algorithm for Simultaneous Parameter Estimation in Robust Exploratory Factor Analysis. Computational Statistics and Data Analysis, 54, 3348-3358.
http://dx.doi.org/10.1016/j.csda.2010.02.003 [16] Unkel, S. and Trendafilov, N.T. (2010) Simultaneous Parameter Estimation in Exploratory Factor Analysis: An Expository Review. International Statistical Review, 78, 363-382.
http://dx.doi.org/10.1111/j.1751-5823.2010.00120.x [17] Adachi, K. (2012) Some Contributions to Data-Fitting Factor Analysis with Empirical Comparisons to Covariance-Fitting Factor Analysis. Journal of the Japanese Society of Computational Statistics, 25, 25-38.
http://dx.doi.org/10.5183/jjscs.1106001_197 [18] Kiers, H.A.L. and ten Berge, J.M.F. (1992) Minimization of a Class of Matrix Trace Functions by Means of Refined Majorization. Psychometrika, 57, 371-382.
http://dx.doi.org/10.1007/BF02295425 [19] Kiers, H.A.L. (1990) Majorization as a Tool for Optimizing a Class of Matrix Functions. Psychometrika, 55, 417-428.
http://dx.doi.org/10.1007/BF02294758 [20] Costa, P.T. and McCrae, R.R. (1992) NEO PI-R Professional Manual: Revised NEO Personality Inventory (NEO PI-R) and NEO Five-Factor Inventory (NEO-FFI). Psychological Assessment Resources, Odessa, FL. [21] Kaiser, H.F. (1958) The Varimax Criterion for Analytic Rotation in Factor Analysis. Psychometrika, 23, 187-200.
http://dx.doi.org/10.1007/BF02289233 [22] Gower, J.C. and Dijksterhuis, G.B. (2004) Procrustes Problems. Oxford University Press, Oxford.
http://dx.doi.org/10.1093/acprof:oso/9780198510581.001.0001 [23] Adachi, K. (2013) Factor Analysis with EM Algorithm Never Gives Improper Solutions When Sample Covariance and Initial Parameter Matrices Are Proper. Psychometrika, 78, 380-394.
http://dx.doi.org/10.1007/s11336-012-9299-8