Journal of the Iranian Statistical Society

Modified Maximum Likelihood Estimation in First-Order Autoregressive Moving Average Models with some Non-Normal Residuals

Authors

1 Science and Research branch, Islamic Azad University,Tehran, Iran

2 K.N. Toosi University of thechnology

10.52547/jirss.19.2.33

Abstract

When modeling time series data using autoregressive-moving average processes, it is a common practice to presume that the residuals are normally distributed. However, sometimes we encounter non-normal residuals and asymmetry of data marginal distribution. Despite widespread use of pure autoregressive processes for modeling non-normal time series, the autoregressive-moving average models have less been used. The main reason is the difficulty in estimating the autoregressive-moving average model parameters. The purpose of this study is to address this intricacy by approximating maximum likelihood estimators, which is particularly important from model selection perspective. Accordingly, the coefficients and residual distribution parameters of the first-order stationary autoregressive-moving average model with residuals that follow exponential and Weibull families, were estimated. Then based on the simulation study, the obtained theoretical results were investigated and it was shown that the modified maximum likelihood estimators were suitable estimators to estimate the first-order autoregressive-moving average model parameters in non-normal mode. In a numerical example positive skewness of obtained residuals from fitting the first-order autoregressive-moving average model was shown. Following that, the parameters of candidate residual distributions estimated by modified maxim-um likelihood estimators and one of the estimated models for modeling the data was selected.

Keywords

  1. Box‎, ‎G‎. ‎E‎. ‎P.‎, ‎and Jenkins‎, ‎G‎. ‎M‎. ‎(1976)‎. ‎Time series analysis forecasting and control (Revised ed.)‎. ‎Retrieved from https://trove.nla.gov.au/version/208005862 ‎Braga‎, ‎D.‎, and Calmon‎, ‎W‎. ‎(2017)‎. ‎Periodic Gamma Autoregressive Model‎: ‎An application to the Brazilian hydroelectric system‎. ‎RAIRO‎ - ‎Operations Research‎, ‎51(2)‎, ‎469–483‎. ‎Casella‎, ‎G., and Berger‎, ‎R‎. ‎L‎. ‎(2001)‎. ‎Statistical Inference (2nd edition)‎. ‎Cengage Learning‎. ‎Cramér‎, ‎H‎. ‎(1946)‎. ‎Mathematical methods of statistics‎. ‎Princeton University Press‎. ‎Fama‎, ‎E‎. ‎F‎. ‎(1965)‎. ‎The behavior of stock-market prices‎. ‎The Journal of Business‎, ‎38(1)‎, ‎34–105‎. ‎Fernandez‎, ‎B.‎, and Salas‎, ‎J‎. ‎D‎. ‎(1986)‎. ‎Periodic gamma autoregressive processes for operational hydrology‎. ‎Water Resources Research‎, ‎22(10)‎, ‎1385–1396‎. ‎Gaver‎, ‎D‎. ‎P.‎, and Lewis‎, ‎P‎. ‎A‎. ‎W‎. ‎(1980)‎. ‎First-order autoregressive gamma sequences and point processes‎. ‎Advances in Applied Probability‎, ‎12(3)‎, ‎727–745‎. ‎Grunwald‎, ‎G‎. ‎K.‎, ‎Hyndman‎, ‎R‎. ‎J.‎, ‎Tedesco‎, ‎L.‎, ‎and Tweedie‎, ‎R‎. ‎L‎. ‎(2000)‎. ‎Theory and Methods‎: ‎Non-Gaussian Conditional Linear AR (1) Models‎. ‎Australian and New Zealand Journal of Statistics‎, ‎42(4)‎, ‎479–495‎. ‎Huber‎, ‎P‎. ‎J‎. ‎(1967)‎. ‎The behavior of maximum likelihood estimates under nonstandard conditions‎. ‎Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability‎, ‎Volume 1‎: ‎Statistics‎. ‎https://projecteuclid.org/euclid.bsmsp/1200512988‎ ‎Kendall‎, ‎M‎. ‎G.‎, ‎and Hill‎, ‎A‎. ‎B‎. ‎(1953)‎. ‎The analysis of economic time-series-part i‎: ‎Prices‎. ‎Journal of the Royal Statistical Society‎. ‎Series A (General)‎, ‎116(1)‎, ‎11–34‎. ‎Li‎, ‎W‎. ‎K., and McLeod‎, ‎A‎. ‎I‎. ‎(1988)‎. ‎ARMA modelling with non-Gaussian innovations‎. ‎Journal of Time Series Analysis‎, ‎9(2)‎, ‎155–168‎. ‎Mandelbrot‎, ‎B‎. ‎(1967)‎. ‎The variation of some other speculative prices‎. ‎The Journal of Business‎, ‎40(4)‎, ‎393–413‎. ‎Mandelbrot‎, ‎B‎. ‎B‎. ‎(1997)‎. ‎The variation of certain speculative prices‎. ‎In Fractals and scaling in finance ‎371–418‎. ‎Springer‎. ‎Mandelbrot‎, ‎B.‎, ‎and Taylor‎, ‎H‎. ‎M‎. ‎(1967)‎. ‎On the distribution of stock price differences‎. ‎Operations Research‎, ‎15(6)‎, ‎1057–1062‎. ‎Tiku‎, ‎M‎. ‎L.‎, ‎Wong‎, ‎W‎. ‎K.‎, ‎and Bian‎, ‎G‎. ‎(1999)‎. ‎Time series models with asymmetric innovations‎. ‎Communications in Statistics‎ - ‎Theory and Methods‎, ‎28(6)‎, ‎1331–1360‎. ‎Trindade‎, ‎A‎. ‎A.‎, ‎Zhu‎, ‎Y.‎, and Andrews‎, ‎B‎. ‎(2010)‎. ‎Time series models with asymmetric Laplace innovations‎. ‎Journal of Statistical Computation and Simulation‎, ‎80(12)‎, ‎1317–1333‎. ‎Wei‎, ‎W‎. ‎W‎. ‎(2006)‎. ‎Time series analysis‎. ‎In The Oxford Handbook of Quantitative Methods in Psychology‎: ‎Vol‎. ‎2‎. ‎Weiss‎, ‎G‎. ‎(1975)‎. ‎Time-Reversibility of linear stochastic processes‎. ‎https://doi.org/10.1017/s0021900200048804‎ ‎Weiss‎, ‎G‎. ‎(1977)‎. ‎Shot noise models for the generation of synthetic stream flow data‎. ‎Water Resources Research‎, ‎13(1)‎, ‎101–108‎. ‎https://doi.org/10.1029/WR013i001p00101‎ ‎Yakowitz‎, ‎S‎. ‎J‎. ‎(1973)‎. ‎A stochastic model for daily river flows in an arid region‎. ‎Water Resources Research‎, ‎9(5)‎, ‎1271–1285‎. ‎Zamani Mehreyan‎, ‎S.‎, ‎and Sayyareh‎, ‎A‎. ‎(2017)‎. ‎Separated hypotheses testing for autoregressive models with non-negative residuals‎. ‎Journal of Statistical Computation and Simulation‎, ‎87(4)‎, ‎689–711‎.
  2. Akaike, H., Petrov, B. N., and Csaki, F. (1973). Second international symposium on information theory. Akadémiai Kiadó, Budapest.
  3. Bayer, F. M., Cintra, R. J., and Cribari-Neto, F. (2018). Beta seasonal autoregressive moving average models. ArXiv E-Prints, 1806, arXiv:1806.07921. [DOI:10.1080/00949655.2018.1491974]
  4. Box, G. E., and Cox, D. R. (1964). An analysis of transformations. Journal of the Royal Statistical Society: Series B (Methodological), 26(2), 211-243. [DOI:10.1111/j.2517-6161.1964.tb00553.x]
  5. Box, G. E. P., and Jenkins, G. M. (1976). Time series analysis forecasting and control (Revised ed).
  6. Braga, D., and Calmon, W. (2017). Periodic Gamma Autoregressive Model: An application to the Brazilian hydroelectric system. RAIRO - Operations Research, 51(2), 469-483. [DOI:10.1051/ro/2016035]
  7. Casella, G., and Berger, R. L. (2001). Statistical Inference (2nd edition). Cengage Learning.
  8. Cramér, H. (1946). Mathematical methods of statistics. Princeton University Press. [DOI:10.1515/9781400883868]
  9. Cryer, J. D., and Chan, K.-S. (2008). Time Series Analysis: With Applications in R (2nd ed.). Springer-Verlag. [DOI:10.1007/978-0-387-75959-3]
  10. Duca, V. E. L. de A., Souza, R. C., Ferreira, P. G. C., and Oliveira, F. L. C. (2019). Simulation of time series using periodic gamma autoregressive models. International Transactions in Operational Research, 26(4), 1315-1338. [DOI:10.1111/itor.12593]
  11. Fama, E. F. (1965). The behavior of stock-market prices. The Journal of Business, 38(1), 34-105. [DOI:10.1086/294743]
  12. Fernandez, B., and Salas, J. D. (1986). Periodic gamma autoregressive processes for operational hydrology. Water Resources Research, 22(10), 1385-1396. [DOI:10.1029/WR022i010p01385]
  13. Gaver, D. P., and Lewis, P. A. W. (1980). First-order autoregressive gamma sequences and point processes. Advances in Applied Probability, 12(3), 727-745. [DOI:10.2307/1426429]
  14. Huber, P. J. (1967). The behavior of maximum likelihood estimates under nonstandard conditions. Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Statistics.
  15. Hurvich, C. M., and Tsai, C.-L. (1989). Regression and time series model selection in small samples. Biometrika, 76(2), 297-307. [DOI:10.1093/biomet/76.2.297]
  16. Kendall, M. G., and Hill, A. B. (1953). The analysis of economic time-series-part i: Prices. Journal of the Royal Statistical Society. Series A (General), 116(1), 11-34. [DOI:10.2307/2980947]
  17. Kullback, S., and Leibler, R. A. (1951). On information and sufficiency, Annals Maths. Statist, 22, 79-86. [DOI:10.1214/aoms/1177729694]
  18. Kwiatkowski, D., Phillips, P., Schmidt, P., and Shin, Y. (1992). Testing the null hypothesis of stationarity against the alternative of a unit root: How sure are we that economic time series have a unit root?, Journal of Econometrics, 54(1-3), 159-178. [DOI:10.1016/0304-4076(92)90104-Y]
  19. Li, W. K., and McLeod, A. I. (1988). ARMA modelling with non-Gaussian innovations. Journal of Time Series Analysis, 9(2), 155-168. [DOI:10.1111/j.1467-9892.1988.tb00461.x]
  20. Mandelbrot, B. (1967). The variation of some other speculative prices. The Journal of Business, 40(4), 393-413. [DOI:10.1086/295006]
  21. Mandelbrot, B. B. (1997). The variation of certain speculative prices. In Fractals and scaling in finance (pp. 371-418). Springer. [DOI:10.1007/978-1-4757-2763-0_14]
  22. Mandelbrot, B., and Taylor, H. M. (1967). On the distribution of stock price differences. Operations Research, 15(6), 1057-1062. [DOI:10.1287/opre.15.6.1057]
  23. Sarlak, N., and Sorman, A. (2007). Gamma autoregressive models and application on Kizilirmak Basin.
  24. Schwarz, G. (1978). Estimating the Dimension of a Model. Annals of Statistics, 6(2), 461-464. [DOI:10.1214/aos/1176344136]
  25. Shapiro, S. S., and Wilk, M. B. (1965). An analysis of variance test for normality (complete samples). Biometrika, 52(3-4), 591-611. [DOI:10.1093/biomet/52.3-4.591]
  26. Spierdijk, L. (2016). Confidence intervals for ARMA-GARCH Value-at-Risk: The case of heavy tails and skewness. Computational Statistics and Data Analysis, 100, 545-559. [DOI:10.1016/j.csda.2014.08.011]
  27. Tiku, M. L., Wong, W. K., and Bian, G. (1999). Time series models with asymmetric innovations. Communications in Statistics - Theory and Methods, 28(6), 1331-1360. [DOI:10.1080/03610929908832360]
  28. Trindade, A. A., Zhu, Y., and Andrews, B. (2010). Time series models with asymmetric Laplace innovations. Journal of Statistical Computation and Simulation, 80(12), 1317-1333. [DOI:10.1080/00949650903071088]
  29. Organization for Economic Co-operation and Development. (2020, January 1). Unemployment
  30. Rate: Aged 15-64: All Persons for the United States. FRED, Federal Reserve Bank of
  31. St. Louis. https://fred.stlouisfed.org/series.
  32. Wei, W. W. (2006). Time series analysis. In The Oxford Handbook of Quantitative Methods in Psychology: Vol. 2.
  33. Weiss, G. (1975). Time-Reversibility of linear stochastic processes. [DOI:10.1017/S0021900200048804]
  34. Weiss, G. (1977). Shot noise models for the generation of synthetic streamflow data. Water Resources Research, 13(1), 101-108. [DOI:10.1029/WR013i001p00101]
  35. Yakowitz, S. J. (1973). A stochastic model for daily river flows in an arid region. Water Resources Research, 9(5), 1271-1285. [DOI:10.1029/WR009i005p01271]
  36. Zamani Mehreyan, S., and Sayyareh, A. (2017). Separated hypotheses testing for autoregressive models with non-negative residuals. Journal of Statistical Computation and Simulation, 87(4), 689-711. [DOI:10.1080/00949655.2016.1222613]
  37. Zhang, Z., and Li, W. K. (2019). An Experiment on Autoregressive and Threshold Autoregressive Models with Non-Gaussian Error with Application to Realized Volatility. Economies, 7(2), 58. [DOI:10.3390/economies7020058]
Journal of the Iranian Statistical Society
Volume 19, Issue 2
December 2020
Pages 33-66
  • Receive Date: 23 July 2022
  • Revise Date: 10 May 2024
  • Accept Date: 23 July 2022