Journal of the Iranian Statistical Society

Testing Several Rival Models Using the Extension of Vuong's Test and Quasi Clustering

Author

Department of Computer Sciences and Statistics, K. N. Toosi University of Technology, Tehran-Iran.

10.52547/jirss.20.2.43

Abstract

The two main goals in model selection are firstly introducing an approach to test homogeneity of several rival models and secondly selecting a set of reasonable models or estimating the best rival model to the true one. In this paper we extend Vuong's method for several models to cluster them. Based on the working paper of Katayama $(2008)$, we propose an approach to test whether rival models have expected relations. The multivariate extension of Vuong's test gives the opportunity to examine some hypotheses about the rival models and their relations with respect to the unknown true model. On the other hand, the standard method of model selection provides an implementation of Occam's razor, in which parsimony or simplicity is balanced against goodness of fit. Therefore, we are interested in clustering the rival models based on their divergence from the true model to select a suitable set of rival models. In this paper we have introduced two approaches to select suitable sets of rival models based on the multivariate extension of Vuong's test and quasi clustering approach.

Keywords

  1. ‎Akaike‎, ‎H‎. ‎(1973)‎. ‎textit{Information Theory and an Extension of‎ ‎Maximum Likelihood Principle}‎. ‎Second International Symposium on‎ ‎Information Theory‎, ‎Akademia Kiado‎, ‎$267-281$.‎ ‎Atkinson‎, ‎A.C‎. ‎(1970)‎. ‎textit{A Method for Discriminating Between‎ ‎Models}‎. ‎Journal of the Royal Statistical Society B‎, ‎textbf{32}‎, ‎$323-344$.‎ ‎ Barmalzan, G. and Sayyareh, A. ‎$‎‎(2011)$. textit{The Choice of an Admissible Sete of Rival Models}. ‎Journal ‎of ‎Statistical ‎Sciences, ‎textbf{4},‎ 2, ‎‎$‎‎149-165‎$‎.‎ ‎‎ Clarke, K., A. and Signorino, C. S. ‎$‎‎(2010)$‎. textit{Discriminating Methods: Tests for Non-Nested Discrete Choice Models}. Political Studies, ‎textbf{58}, ‎$‎‎368-388$.‎ ‎ ‎Commenges‎, ‎D.‎, ‎Sayyareh‎, ‎A.‎, ‎Letenneur‎, ‎L.‎, ‎Guedj‎, ‎J‎. ‎and Bar-Hen‎, ‎A‎. ‎(2008)‎. ‎textit{Estimating a Difference of Kullback-Leibler‎ ‎Risks Using a Normalized Difference of AIC}‎. ‎The Annal of Applied‎ ‎Statistics‎, ‎textbf{2}‎, ‎3‎, ‎$1123-1142$.‎ ‎Cox‎, ‎D.R‎. ‎(1961)‎. ‎textit{Test of Separate Families of Hypothesis}‎. ‎Proceeding of the Fourth Berkeley Symposium on Mathematical‎ ‎Statistics and Probability‎, ‎textbf{1}‎, ‎$105-123$.‎ ‎Cox‎, ‎D.R‎. ‎(1962)‎. ‎textit{Further Result on Tests of Separate‎ ‎Families of Hypothesis}‎. ‎Journal of the Royal Statistical Society‎ ‎B‎, ‎textbf{24}‎, ‎$406-424$.‎ ‎Katayama‎, ‎N.‎, ‎(2008)‎, ‎textit{Portmanteau Likelihood Ratio Tests for Model Selection}.(http:/‎/ ‎www.economics.smu.edu.sg/femes/2008/CS Info/papers/169.pdf)‎ ‎Kullback, S.‎, ‎Leibler,R.‎, ‎(1951)‎. ‎textit{On information and sufficiency}‎. ‎Annals of Mathematical Statistics‎. ‎textbf{22}‎, ‎$79-86$.‎ ‎ Liao, Z. and Shi, ‎X.‎, (2016). ‎textit{A Uniform Vuong Test for Semi/Nonparametric Models}. $‎https://www.econ.cuhk.edu.hk/econ/images/content/news_event/seminars/2015-2016-2nd-semester/NPVuong223.pdf‎$‎.‎ Lorestan, H. and Sayyareh, A (2017). textit{Model selection using :union:-intersection principle for non nested models}. Communications in Statistics-Theory and Methods. ‎textbf{46-4}‎, ‎$1636-1649‎‎$‎.‎ ‎Pesaran‎, ‎M. H‎. ‎(1974)‎. ‎textit{On the General Test of Model‎ ‎Selection}‎. ‎Review of Economic Studies‎, ‎textbf{41}‎, ‎$153-171$.‎ ‎Pesaran‎, ‎M. H‎. ‎and Deaton‎, ‎A.S‎. ‎(1978)‎. ‎textit{Testing Non-Nested‎ ‎Nonlinear Regression Models}‎. ‎Econometrica‎, ‎textbf{46}‎, ‎$667-694$.‎ ‎ Pho, K. H., Ly, S. Ly, S. and Lukusa, T. M. (2019). ‎textit{Comparison among Akaike Information Criterion, Bayesian Information Criterion and Vuong's test in Model Selection: A Case Study of Violated Speed Regulation in Taiwan}. Journal of Advanced Engineering and Computation. textbf{3}, 1, 293-303.‎ ‎Sayyareh‎, ‎A‎. ‎Obeidi‎, ‎R‎. ‎and Bar-Hen‎, ‎A.‎, ‎(2011)‎. ‎textit{Empirical Comparison of Some Model Selection Criteria}‎. ‎Communication in Statistics-Simulation and Computation‎. ‎textbf{40}‎, ‎$72-86$.‎ ‎Sayyareh‎, ‎A‎. ‎(2012)‎. ‎textit{Inference After Separated Hypotheses Testing‎: ‎An Investigation for Linear Models}‎. ‎Journal of Statistical Computation and Simulation‎. ‎textbf{82}‎, 9, ‎$1275-1286‎‎$‎.‎‎ ‎ ‎Sayyareh‎, ‎A‎. ‎(2017)‎. ‎textit{Non parametric multiple comparisons of non nested rival models}‎. textit{Communications in Statistics-Theory and Methods}. textbf{46}, 17, ‎$8369-8386‎‎$.‎‎‎ ‎Shimodiara‎, ‎H‎. ‎(1998)‎. ‎textit{An Application of Multiple‎ ‎Comparison Techniques to Model Selection}‎. ‎Annals of Institute‎ ‎Statistical Mathematics‎, ‎textbf{50}‎, ‎$1$‎, ‎$1-13$.‎ ‎Shimodaira‎, ‎H‎. ‎(2001)‎. ‎textit{Multiple Comparisons of‎ ‎log-likelihoods and Combining Non-Nested Models with Application‎ ‎to Phylogenetic tree Selection}‎. ‎Communication in Statistics‎, ‎textbf{30}‎, ‎$1751-1772$.‎ ‎Vuong‎, ‎Q. H‎. ‎(1989)‎. ‎textit{Likelihood Ratio Tests for Model‎ ‎Selection and Non-Nested Hypotheses}‎. ‎Econometrica‎, ‎textbf{57}‎, ‎2‎, ‎$307-333$.‎ ‎Yanagihara‎, ‎H‎. ‎and Ohomoto‎, ‎C‎. ‎(2005)‎. ‎textit{On‎ ‎Distribution of AIC in Linear Regression Models}‎. ‎Journal of‎ ‎Statistical Planning and Inference‎, ‎textbf{133}‎, ‎$417-433$.‎ ‎White‎, ‎H‎. ‎(1982a)‎. ‎textit{Maximum Likelihood Estimation‎ ‎of Misspecified Models}‎. ‎Econometrica‎, ‎textbf{50}‎, ‎$1-26$.‎ ‎White‎, ‎H‎. ‎(1982b)‎. ‎textit{Regularity Conditions for Cox's Test of‎ ‎Non-Nested Hypotheses}‎. ‎Journal of Econometrics‎, ‎textbf{19}‎, ‎$301-318$.‎ ‎Zucchini‎, ‎W‎. ‎(2000)‎. ‎textit{An introduction to model selection‎ ‎}‎. ‎Journal of Mathematical Psychology‎, ‎textbf{44}‎, ‎$41-61$‎
  2. Akaike, H. (1973), Information theory and an extension of maximum likelihood principle. Second International Symposium on Information Theory, Akademia Kiado, 267-281.
  3. Atkinson, A.C. (1970), A method for discriminating between models. Journal of the Royal Statistical Society B, 32, 323-344. [DOI:10.1111/j.2517-6161.1970.tb00845.x]
  4. Barmalzan, G. and Sayyareh, A. (2011), The choice of an admissible sete of rival models. Journal of Statistical Sciences, 4(2), 149-165.
  5. Clarke, K., A. and Signorino, C. S. (2010), Discriminating methods: Tests for non-nested discrete choice models. Political Studies, 58, 368-388. [DOI:10.1111/j.1467-9248.2009.00813.x]
  6. Commenges, D., Sayyareh, A., Letenneur, L., Guedj, J. and Bar-Hen, A. (2008), Estimating a difference of Kullback-Leibler risks Using a normalized difference of AIC. The Annals of Applied Statistics, 2(3), 1123-1142. [DOI:10.1214/08-AOAS176]
  7. Cox, D.R. (1961), Test of separate families of hypothesis. proceeding of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, 1, 105-123.
  8. Katayama, N. (2008), Portmanteau likelihood ratio tests for model selection, (http://
  9. www.economics.smu.edu.sg/femes/2008/169.pdf).
  10. Kullback, S., Leibler, R. (1951), On information and sufficiency. Annals of Mathematical Statistics, 22, 79-86. [DOI:10.1214/aoms/1177729694]
  11. bibitem[Al-qaness et al. (2020)]{Al-qaness Lorestan, H. and Sayyareh, A. (2017), Model selection using :union:-intersection principle for non nested models. Communications in Statistics-Theory and Methods, 46(4), 1636-1649. [DOI:10.1080/03610926.2015.1024863]
  12. Pesaran, M. H. (1974), On the general test of model selection. Review of Economic Studies, 41, 153-171. [DOI:10.2307/2296710]
  13. Pesaran, M. H., and Deaton, A.S. (1978), Testing non-nested nonlinear regression models. Econometrica, 46, 667-694. [DOI:10.2307/1914240]
  14. Pho, K. H., Ly, S. Ly, S., and Lukusa, T. M. (2019), Comparison among Akaike information criterion, Bayesian information criterion and Vuong's test in model selection: A case study of violated speed regulation in Taiwan. Journal of Advanced Engineering and Computation, 3(1), 293-303. [DOI:10.25073/jaec.201931.220]
  15. Sayyareh, A. Obeidi, R., and Bar-Hen, A. (2011), Empirical comparison of some model selection criteria. Communication in Statistics-Simulation and Computation, 40, 72-86. [DOI:10.1080/03610918.2010.530367]
  16. Sayyareh, A. (2012), Inference after separated hypotheses testing: An investigation for linear models. Journal of Statistical Computation and Simulation. 82(9), 1275-1286. [DOI:10.1080/00949655.2011.575783]
  17. Sayyareh, A. (2017), Non parametric multiple comparisons of non nested rival models. Communications in Statistics-Theory and Methods, 46(17), 8369-8386. [DOI:10.1080/03610926.2016.1179759]
  18. Shimodiara, H. (1998), An application of multiple comparison techniques to model selection. Annals of Institute Statistical Mathematics, 50(1), 1-13. [DOI:10.1023/A:1003483128844]
  19. Shimodaira, H. (2001), Multiple comparisons of log-likelihoods and combining non-nested models with application to phylogenetic tree selection. Communication in Statistics-Theory and methods, 30, 1751-1772. [DOI:10.1081/STA-100105696]
  20. Vuong, Q. H. (1989), Likelihood ratio tests for model selection and non-nested hHypotheses. Econometrica, 57(2), 307-333. [DOI:10.2307/1912557]
  21. Yanagihara, H., and Ohomoto, C. (2005), On distribution of AIC in linear regression models. Journal of Statistical Planning and Inference, 133, 417-433. [DOI:10.1016/j.jspi.2004.03.016]
  22. White, H. (1982a). Maximum likelihood estimation of misspecified models. Econometrica, 50, 1-26. [DOI:10.2307/1912526]
  23. White, H. (1982b), Regularity conditions for Cox's test of non-nested hypotheses. Journal of Econometrics, 19, 301-318. [DOI:10.1016/0304-4076(82)90007-0]
  24. Zucchini, W. (2000), An introduction to model selection. Journal of Mathematical Psychology, 44, 41-61. [DOI:10.1006/jmps.1999.1276]
Journal of the Iranian Statistical Society
Volume 20, Issue 2
December 2021
Pages 43-63
  • Receive Date: 23 July 2022
  • Revise Date: 14 May 2024
  • Accept Date: 23 July 2022