Journal of the Iranian Statistical Society

Mixture of Forward-Directed and Backward-Directed Autoregressive Hidden Markov Models for Time Series Modeling

Authors

1 Department of Statistics‎, ‎Faculty of mathematics and Computer Sciences‎, ‎Allameh Tabataba'i University‎, ‎Tehran‎, ‎Iran‎.

2 Financial Mathematics Group‎, ‎Faculty of Financial Sciences‎, ‎Kharazmi University‎, ‎Iran.

3 Structural Bioinformatics and High Performance Computing Research Group (BIO-HPC) Universidad Católica de Murcia (UCAM), Spain

4 Laboratory of Functional and Structural Genomics, Center of New Technologies, University of Warsaw, Warsaw, Poland

10.29252/jirss.18.1.89

Abstract

Hidden Markov models (HMM) are a ubiquitous tool for modeling time series data. The HMM can be poor at capturing dependency between observations because of the statistical assumptions it makes. Therefore, the extension of the HMM called forward-directed Autoregressive HMM (ARHMM) is considered to handle the dependencies between observations. It is also  more appropriate to use an Autoregressive Hidden Markov Model directed backward in time. In this paper, we present a sequence-level mixture of these two forms of ARHMM (called MARHMM), effectively allowing the model to choose for itself whether a forward-directed or backward-directed model or a soft combination of the two models are most appropriate for a given data set. For this purpose, we use the conditional independence relations in the context of a Bayesian network which is a probabilistic graphical model. The performance of the MARHMM is discussed by applying it to the simulated and real data sets. We show that the proposed model has greater modeling power than the conventional forward-directed ARHMM.

Keywords

[1] Antezana, Jorge, Pedro Massey, and Demetrio Stojano ."Jensen's inequality for spectral order and submajorization. "Journal of mathematical analysis and applications 331.1 (2007): 297-307. [2] Bartolucci, Francesco, and FulviaPennoni."A class of latent Markov models for capturerecapture data allowing for time, heterogeneity, and behavior effects." Biometrics 63.2 (2007): 568-578. [3] Bartolucci, Francesco, Silvia Bacci, and FulviaPennoni."Longitudinal analysis of self-reported health status by mixture latent auto-regressive models." Journal of the Royal Statistical Society: Series C (Applied Statistics) 63.2 (2014): 267-288. [4] Baum, Leonard E., and Ted Petrie."Statistical inference for probabilistic functions of finite state Markov chains." The annals of mathematical statistics (1966): 1554- 1563. [5] Bilmes, Jeff A. "A gentle tutorial of the EM algorithm and its application to parameter estimation for Gaussian mixture and hidden Markov models." International Computer Science Institute 4.510 (1998): 126. [6] Bishop, Christopher M. "Mixture Models and the EM Algorithm." Microsoft Research, Cambridge (2006). [7] Borodovsky, Mark, and Svetlana Ekisheva. Problems and solutions in biological sequence analysis. Cambridge: Cambridge University Press, (2006). [8] Dempster, Arthur P., Nan M. Laird, and Donald B. Rubin."Maximum likelihood from incomplete data via the EM algorithm."Journal of the Royal Statistical Society. Series B (Methodological) (1977): 1-38. [9] Do, Chuong B., and Sera mBatzoglou. "What is the expectation maximization algorithm?." Nature biotechnology 26.8 (2008): 897-900. [10] Durbin, Richard, et al. Biological sequence analysis: probabilistic models of proteins and nucleic acids. Cambridge university press, (1998). [11] Dymarski, Przemyslaw. "Hidden Markov Models, Theory and Applications."InTech Open Access Publishers (2011). [12] Eddy, Sean R. "Pro le hidden Markov models." Bioinformatics 14.9 (1998): 755-763. [13] Ghahramani, Zoubin. "An introduction to hidden Markov models and Bayesian networks."International Journal of Pattern Recognition and Arti ficial Intelligence 15.01 (2001): 9-42. [14] Jebara, Tony. Machine learning: discriminative and generative. Springer, (2004). [15] Kawamoto, Ryouhei, et al. "Hidden Markov model for analyzing time-series health checkup data." MedInfo. (2013). [16] Letunic, Ivica, et al. "Recent improvements to the SMART domain-based sequence annotation resource." Nucleic acids research 30.1 (2002): 242-244. [17] MacDonald, Iain L., and Walter Zucchini. Hidden Markov and other models for discrete-valued time series.Vol. 110.CRC Press, (1997). [18] McLachlan, Geo rey, and Thriyambakam Krishnan.The EM algorithm and extensions.Vol. 382.John Wiley Sons, (2007). [19] Movellan, Javier R. "Tutorial on hidden Markov models." Machine perception laboratory online tutorials (2003). [20] Murphy, Kevin, and SairaMian. Modelling gene expression data using dynamic Bayesian networks.Vol. 104.Technical report, Computer Science Division, University of California, Berkeley, CA, (1999). [21] Pearl, Judea. "Probabilistic reasoning in intelligent systems: Networks of plausible reasoning." (1988). [22] Petrushin, Valery A. "Hidden Markov models: Fundamentals and applications."Online Symposium for Electronics Engineer. (2000). [23] Rabiner, Lawrence. "A tutorial on hidden Markov models and selected applications in speech recognition."Proceedings of the IEEE 77.2 (1989): 257-286. [24] Reichenbach, Hans, and Maria Reichenbach. The direction of time.Vol. 65.Univof California Press, (1991). [25] Satish, L., and B. I. Gururaj."Use of hidden Markov models for partial discharge pattern classi cation." Electrical Insulation, IEEE Transactions on28.2 (1993): 172-182. [26] Shannon, Matt, and William Byrne. "Autoregressive HMMs for speech synthesis."in proceeding of inter speech (2009). [27] Shannon, Matt, Heiga Zen, and William Byrne. "Autoregressive models for statistical parametric speech synthesis." Audio, Speech, and Language Processing, IEEE Transactions on 21.3 (2013): 587-597. [28] Siepel, Adam, and David Haussler."Combining phylogenetic and hidden Markov models in biosequence analysis."Journal of Computational Biology 11.2-3 (2004): 413-428. [29] Shirley, Kenneth E., et al. "Hidden Markov models for alcoholism treatment trial data." The Annals of Applied Statistics 4.1 (2010): 366-395. [30] Stanculescu, Ioan, Christopher KI Williams, and Yvonne Freer."Autoregressive Hidden Markov Models for the Early Detection of Neonatal Sepsis." Biomedical and Health Informatics, IEEE Journal of 18.5 (2014): 1560-1570. [31] Takiguchi, Tetsuya, et al. "Model adaptation based on HMM decomposition for reverberant speech recognition." Acoustics, Speech, and Signal Processing, 1997.ICASSP-97., 1997 IEEE International Conference on.Vol. 2.IEEE, (1997). [32] Zucchini, Walter, and Iain L. MacDonald. Hidden Markov models for time series:an introduction using R. CRC Press, (2009).
Journal of the Iranian Statistical Society
Volume 18, Issue 1
June 2019
Pages 89-112
  • Receive Date: 23 July 2022
  • Revise Date: 20 May 2024
  • Accept Date: 23 July 2022