Comparison of the Sampling Efficiency in Spatial Autoregressive Model
Affiliation(s)
1 Department of Economics, University of Nagasaki, Nagasaki, Japan. 2 Faculty of Law, Politics and Economics, Chiba University, Chiba, Japan.
1 Department of Economics, University of Nagasaki, Nagasaki, Japan. 2 Faculty of Law, Politics and Economics, Chiba University, Chiba, Japan.
ABSTRACT
A random walk Metropolis-Hastings algorithm has been widely used in sampling the parameter of spatial interaction in spatial autoregressive model from a Bayesian point of view. In addition, as an alternative approach, the griddy Gibbs sampler is proposed by [1] and utilized by [2]. This paper proposes an acceptance-rejection Metropolis-Hastings algorithm as a third approach, and compares these three algorithms through Monte Carlo experiments. The experimental results show that the griddy Gibbs sampler is the most efficient algorithm among the algorithms whether the number of observations is small or not in terms of the computation time and the inefficiency factors. Moreover, it seems to work well when the size of grid is 100.
A random walk Metropolis-Hastings algorithm has been widely used in sampling the parameter of spatial interaction in spatial autoregressive model from a Bayesian point of view. In addition, as an alternative approach, the griddy Gibbs sampler is proposed by [1] and utilized by [2]. This paper proposes an acceptance-rejection Metropolis-Hastings algorithm as a third approach, and compares these three algorithms through Monte Carlo experiments. The experimental results show that the griddy Gibbs sampler is the most efficient algorithm among the algorithms whether the number of observations is small or not in terms of the computation time and the inefficiency factors. Moreover, it seems to work well when the size of grid is 100.
KEYWORDS
Acceptance-Rejection Metropolis-Hastings Algorithm, Griddy Gibbs Sampler, Markov Chain Monte Carlo (MCMC), Random Walk Metropolis-Hastings Algorithm, Spatial Autoregressive Model
Acceptance-Rejection Metropolis-Hastings Algorithm, Griddy Gibbs Sampler, Markov Chain Monte Carlo (MCMC), Random Walk Metropolis-Hastings Algorithm, Spatial Autoregressive Model
Cite this paper
Ohtsuka, Y. and Kakamu, K. (2015) Comparison of the Sampling Efficiency in Spatial Autoregressive Model. Open Journal of Statistics, 5, 10-20. doi: 10.4236/ojs.2015.51002.
Ohtsuka, Y. and Kakamu, K. (2015) Comparison of the Sampling Efficiency in Spatial Autoregressive Model. Open Journal of Statistics, 5, 10-20. doi: 10.4236/ojs.2015.51002.
References
[1] Ritter, C. and Tanner, M. (1992) Facilitating the Gibbs Sampler: The Gibbs Stopper and the Griddy-Gibbs Sampler. Journal of the American Statistical Association, 87, 861-868.
http://dx.doi.org/10.1080/01621459.1992.10475289 [2] Ohtsuka, Y. and Kakamu, K. (2009) Estimation of Electric Demand in Japan: A Bayesian Spatial Autoregressive AR(p) Approach. In: Schwartz, L.V., Ed., Inflation: Causes and Effects, Nova Science Publisher, New York, 156-178. [3] Anselin, L. (2003) Spatial Externalities, Spatial Multipliers, and Spatial Econometrics. International Regional Science Review, 26, 153-166.
http://dx.doi.org/10.1177/0160017602250972 [4] Gelfand, A.E., Banerjee, S., Sirmans, C.F., Tu, Y. and Ong, S.E. (2007) Multilevel Modeling Using Spatial Processes: Application to the Singapore Housing Market. Computational Statistics and Data Analysis, 51, 3567-3579.
http://dx.doi.org/10.1080/01621459.1990.10476213 [5] Anselin, L. (2010) Thirty Years of Spatial Econometrics. Papers in Regional Science, 89, 3-25.
http://dx.doi.org/10.1111/j.1435-5957.2010.00279.x [6] Ord, K. (1975) Estimation Methods for Models for Spatial Interaction. Journal of the American Statistical Association, 70, 120-126.
http://dx.doi.org/10.1080/01621459.1975.10480272 [7] Lee, L.F. (2004) Asymptotic Distributions of Quasi-Maximum Likelihood Estimators for Spatial Autoregressive Models. Econometrica, 72, 1899-1925.
http://dx.doi.org/10.1111/j.1468-0262.2004.00558.x [8] Conley, T.G. (1999) GMM Estimation with Cross Sectional Dependence. Journal of Econometrics, 92, 1-45.
http://dx.doi.org/0.1016/S0304-4076(98)00084-0 [9] Kelejian, H.H. and Prucha, I.R. (1999) A Generalized Moments Estimator for the Autoregressive Parameter in a Spatial Model. International Economic Review, 40, 509-533.
http://dx.doi.org/10.1111/1468-2354.00027 [10] Anselin, L. (1980) A Note on Small Sample Properties of Estimators in A First-order Spatial Autoregressive Model. Environment and Planning A, 14, 1023-1030.
http://dx.doi.org/10.1068/a141023 [11] LeSage, J.P. (1997) Regression Analysis of Spatial Data. The Journal of Regional Analysis and Policy, 27, 83-94. [12] Kakamu, K. and Wago, H. (2008) Small-Sample Properties of Panel Spatial Autoregressive Models: Comparison of the Bayesian and Maximum Likelihood Methods. Spatial Economic Analysis, 3, 305-319.
http://dx.doi.org/10.1080/17421770802353725 [13] Holloway, G., Shankar, B. and Rahman, S. (2002) Bayesian Spatial Probit Estimation: A Primer and an Application to HYV Rice Adoption. Agricultural Economics, 27, 383-402.
http://dx.doi.org/10.1111/j.1574-0862.2002.tb00127.x [14] Ohtsuka, Y., Oga, T. and Kakamu, K. (2010) Forecasting Electricity Demand in Japan: A Bayesian Spatial Autoregressive ARMA Approach. Computational Statistics & Data Analysis, 54, 2721-2735.
http://dx.doi.org/10.1016/j.csda.2009.06.002 [15] Tierney, L. (1994) Markov Chains for Exploring Posterior Distributions (with Discussion). Annals of Statistics, 22, 1701-1728.
http://dx.doi.org/10.1214/aos/1176325750 [16] Chib, S. and Greenberg, E. (1994) Bayes Inference in Regression Models with ARMA( ) Errors. Journal of Econometrics, 64, 183-206.
http://dx.doi.org/10.1016/0304-4076(94)90063-9 [17] Watanabe, T. (2001) On Sampling the Degree-of-Freedom of Student’s-t Disturbances. Statistics & Probability Letters, 52, 177-181.
http://dx.doi.org/10.1016/S0167-7152(00)00221-2 [18] Mitsui, H. and Watanabe, T. (2003) Bayesian Analysis of GARCH Option Pricing Models. Journal of the Japan Statistical Society (Japanese Issue), 33, 307-324. [19] LeSage, J.P. and Pace, R.K. (2008) Introduction to Spatial Econometrics (Statistics: A Series of Textbooks and Monographs). Chapman and Hall/CRC, London. [20] Stakhovych, S. and Bijmolt, T.H.A. (2009) Specification of Spatial Models: A Simulation Study on Weights Matrices. Papers in Regional Science, 88, 389-408.
http://dx.doi.org/10.1111/j.1435-5957.2008.00213.x [21] Sun, D., Tsutakawa, R.K. and Speckman, P.L. (1999) Posterior Distribution of Hierarchical Models Using CAR(1) Distributions. Biometrika, 86, 341-350.
http://dx.doi.org/10.1093/biomet/86.2.341 [22] Bauwens, L. and Lubrano, M. (1998) Bayesian Inference on GARCH Models Using the Gibbs Sampler. The Econometrics Journal, 1, 23-46.
http://dx.doi.org/10.1111/1368-423X.11003 [23] Chib, S. and Greenberg, E. (1998) Analysis of Multivariate Probit Models. Biometrika, 85, 347-361.
http://dx.doi.org/10.1093/biomet/85.2.347 [24] Chib, S. and Greenberg, E. (1995) Understanding the Metropolis-Hastings Algorithm. The American Statistician, 49, 327-335. [25] Gelfand, A.E. and Smith, A.F.M. (1990) Sampling-Based Approaches to Calculating Marginal Densities. Journal of the American Statistical Association, 85, 398-409.
http://dx.doi.org/10.1080/01621459.1990.10476213 [26] Asai, M. (2005) Comparison of MCMC Methods for Estimating Stochastic Volatility Models. Computational Economics, 25, 281-301.
http://dx.doi.org/10.1007/s10614-005-2974-4 [27] Asai, M. (2006) Comparison of MCMC Methods for Estimating GARCH Models. Journal of the Japan Statistical Society, 36, 199-212.
http://dx.doi.org/10.14490/jjss.36.199 [28] Chib, S. (2001) Markov Chain Monte Carlo Methods: Computation and Inference. In: Heckman, J.J. and Leamer, E., Eds., Handbook of Econometrics, Elsevier, Amsterdam, 3569-3649. [29] Doornik, J.A. (2006) Ox: An Object Oriented Matrix Programming Language. Timberlake Consultants Press, London.
[1] Ritter, C. and Tanner, M. (1992) Facilitating the Gibbs Sampler: The Gibbs Stopper and the Griddy-Gibbs Sampler. Journal of the American Statistical Association, 87, 861-868.
http://dx.doi.org/10.1080/01621459.1992.10475289 [2] Ohtsuka, Y. and Kakamu, K. (2009) Estimation of Electric Demand in Japan: A Bayesian Spatial Autoregressive AR(p) Approach. In: Schwartz, L.V., Ed., Inflation: Causes and Effects, Nova Science Publisher, New York, 156-178. [3] Anselin, L. (2003) Spatial Externalities, Spatial Multipliers, and Spatial Econometrics. International Regional Science Review, 26, 153-166.
http://dx.doi.org/10.1177/0160017602250972 [4] Gelfand, A.E., Banerjee, S., Sirmans, C.F., Tu, Y. and Ong, S.E. (2007) Multilevel Modeling Using Spatial Processes: Application to the Singapore Housing Market. Computational Statistics and Data Analysis, 51, 3567-3579.
http://dx.doi.org/10.1080/01621459.1990.10476213 [5] Anselin, L. (2010) Thirty Years of Spatial Econometrics. Papers in Regional Science, 89, 3-25.
http://dx.doi.org/10.1111/j.1435-5957.2010.00279.x [6] Ord, K. (1975) Estimation Methods for Models for Spatial Interaction. Journal of the American Statistical Association, 70, 120-126.
http://dx.doi.org/10.1080/01621459.1975.10480272 [7] Lee, L.F. (2004) Asymptotic Distributions of Quasi-Maximum Likelihood Estimators for Spatial Autoregressive Models. Econometrica, 72, 1899-1925.
http://dx.doi.org/10.1111/j.1468-0262.2004.00558.x [8] Conley, T.G. (1999) GMM Estimation with Cross Sectional Dependence. Journal of Econometrics, 92, 1-45.
http://dx.doi.org/0.1016/S0304-4076(98)00084-0 [9] Kelejian, H.H. and Prucha, I.R. (1999) A Generalized Moments Estimator for the Autoregressive Parameter in a Spatial Model. International Economic Review, 40, 509-533.
http://dx.doi.org/10.1111/1468-2354.00027 [10] Anselin, L. (1980) A Note on Small Sample Properties of Estimators in A First-order Spatial Autoregressive Model. Environment and Planning A, 14, 1023-1030.
http://dx.doi.org/10.1068/a141023 [11] LeSage, J.P. (1997) Regression Analysis of Spatial Data. The Journal of Regional Analysis and Policy, 27, 83-94. [12] Kakamu, K. and Wago, H. (2008) Small-Sample Properties of Panel Spatial Autoregressive Models: Comparison of the Bayesian and Maximum Likelihood Methods. Spatial Economic Analysis, 3, 305-319.
http://dx.doi.org/10.1080/17421770802353725 [13] Holloway, G., Shankar, B. and Rahman, S. (2002) Bayesian Spatial Probit Estimation: A Primer and an Application to HYV Rice Adoption. Agricultural Economics, 27, 383-402.
http://dx.doi.org/10.1111/j.1574-0862.2002.tb00127.x [14] Ohtsuka, Y., Oga, T. and Kakamu, K. (2010) Forecasting Electricity Demand in Japan: A Bayesian Spatial Autoregressive ARMA Approach. Computational Statistics & Data Analysis, 54, 2721-2735.
http://dx.doi.org/10.1016/j.csda.2009.06.002 [15] Tierney, L. (1994) Markov Chains for Exploring Posterior Distributions (with Discussion). Annals of Statistics, 22, 1701-1728.
http://dx.doi.org/10.1214/aos/1176325750 [16] Chib, S. and Greenberg, E. (1994) Bayes Inference in Regression Models with ARMA( ) Errors. Journal of Econometrics, 64, 183-206.
http://dx.doi.org/10.1016/0304-4076(94)90063-9 [17] Watanabe, T. (2001) On Sampling the Degree-of-Freedom of Student’s-t Disturbances. Statistics & Probability Letters, 52, 177-181.
http://dx.doi.org/10.1016/S0167-7152(00)00221-2 [18] Mitsui, H. and Watanabe, T. (2003) Bayesian Analysis of GARCH Option Pricing Models. Journal of the Japan Statistical Society (Japanese Issue), 33, 307-324. [19] LeSage, J.P. and Pace, R.K. (2008) Introduction to Spatial Econometrics (Statistics: A Series of Textbooks and Monographs). Chapman and Hall/CRC, London. [20] Stakhovych, S. and Bijmolt, T.H.A. (2009) Specification of Spatial Models: A Simulation Study on Weights Matrices. Papers in Regional Science, 88, 389-408.
http://dx.doi.org/10.1111/j.1435-5957.2008.00213.x [21] Sun, D., Tsutakawa, R.K. and Speckman, P.L. (1999) Posterior Distribution of Hierarchical Models Using CAR(1) Distributions. Biometrika, 86, 341-350.
http://dx.doi.org/10.1093/biomet/86.2.341 [22] Bauwens, L. and Lubrano, M. (1998) Bayesian Inference on GARCH Models Using the Gibbs Sampler. The Econometrics Journal, 1, 23-46.
http://dx.doi.org/10.1111/1368-423X.11003 [23] Chib, S. and Greenberg, E. (1998) Analysis of Multivariate Probit Models. Biometrika, 85, 347-361.
http://dx.doi.org/10.1093/biomet/85.2.347 [24] Chib, S. and Greenberg, E. (1995) Understanding the Metropolis-Hastings Algorithm. The American Statistician, 49, 327-335. [25] Gelfand, A.E. and Smith, A.F.M. (1990) Sampling-Based Approaches to Calculating Marginal Densities. Journal of the American Statistical Association, 85, 398-409.
http://dx.doi.org/10.1080/01621459.1990.10476213 [26] Asai, M. (2005) Comparison of MCMC Methods for Estimating Stochastic Volatility Models. Computational Economics, 25, 281-301.
http://dx.doi.org/10.1007/s10614-005-2974-4 [27] Asai, M. (2006) Comparison of MCMC Methods for Estimating GARCH Models. Journal of the Japan Statistical Society, 36, 199-212.
http://dx.doi.org/10.14490/jjss.36.199 [28] Chib, S. (2001) Markov Chain Monte Carlo Methods: Computation and Inference. In: Heckman, J.J. and Leamer, E., Eds., Handbook of Econometrics, Elsevier, Amsterdam, 3569-3649. [29] Doornik, J.A. (2006) Ox: An Object Oriented Matrix Programming Language. Timberlake Consultants Press, London.