Practical Learning Directed Acyclic Graphs with General Noise Assumptions

Al Sarray, Russul; Rezaeitabar, Vahid

doi:10.22034/jirss.2024.2002863.1022

Practical Learning Directed Acyclic Graphs with General Noise Assumptions

Document Type : Original Article

Authors

Russul Al Sarray

Vahid Rezaeitabar

Department of statistics, Faculty of Statistics, Mathematics and Computer, Allameh Tabataba'i University, Tehran, Iran.

10.22034/jirss.2024.2002863.1022

Abstract

Directed Acyclic Graphs are investigated focusing on learning the coefficient matrix via continuous optimization. We have provided three learning strategies and their corresponding improvements in comparison with former algorithms using some numerical illustrations. Each method is widely introduced and its corresponding concepts are also studied. We have extended, the learning assumptions for each strategy. Lots of preliminary assumptions including normality of noises, and independent and identically distributed errors are removed and with these general considerations, the learning methods are even improved than some existing methods. Furthermore, the number of new criteria that can evaluate the learning processes are given and throughout simulation studies are analyzed. Their sensitivity analysis is also presented which can be useful due to the learning power of any presented strategy. Finally, some further discussions are introduced as guidance for future works.

Keywords

Directed Acyclic Graph

Learning

Linear

Not Independent and Not Identically

Structural Equation Model

Subjects

Applied Probability

Aragam, B., Amini, A. A., and Zhou, Q. (2015), Learning directed acyclic graphs with penalized neighbourhood regression. arXiv preprint arXiv:1511.08963.

Beinlich, I. A., Suermondt, H. J., Chavez, R. M., and Cooper, G.F. (1989), The ALARM monitoring system: A case study with two probabilistic inference techniques for belief networks. Lecture Notes in Medical Informatics, 89, 247–256. doi:10.1007/978-3-642-93437-7-28.

Casella, G., and Berger, R. L. (2021), Statistical inference. Cengage Learning.

Chickering, D.M. (1996), Learning Bayesian networks is NP-complete. In Learning from data: Artificial intelligence and statisticsV (Vol. 12, pp. 121-130). doi:10.1007/978-1-4612-2404-4-12.

Cooper, G. F., and Herskovits, E. (1991), A Bayesian method for constructing Bayesian belief networks from databases. Uncertainty Proceedings, 1, 86-94. doi:10.1016/B978-1-55860-203-8.50015-2.

Darmois, G. (1953), Analyse generale des liaisons stochastiques: etude particuliere de l’analyse factorielle lineaire. Revue de l’Institut international de statistique, 2(1), 2-8. doi:10.2307/1401511.

Giudici, P., and Castelo, R. (2003), Improving Markov chain Monte Carlo model search for data mining. Machine learning, 50(1), 127-158. doi:10.1023/A:1020202028934.

Goudie, R., and Mukherjee, S. (2016), A Gibbs Sampler for Learning DAGs. Microtome Publishing.

Gu, J., Fu, F., and Zhou, Q. (2019), Penalized estimation of directed acyclic graphs from discrete data. Statistics and Computing, 29(1), 161-176. doi:10.1007/s11222-018-9801-y.

Hyvarinen, A. (2000), Independent component analysis: algorithms and applications. Neural networks, 13(5), 411-430. doi:10.1016/S0893-6080(00)00026-5.

Kratzer, G., and Furrer, R. (2019), mcmcabn: a structural MCMC sampler for DAGs learned from observed systemic datasets. R package version 3(1): https://cran.microsoft.com/snapshot/2020-0408/web/packages/mcmcabn/vignettes/mcmcabn.html.

Kazempoor, J., Habibirad, A., and Okhli, K. (2020), Bounds for CDFs of order statistics arising from INID random variables. Journal of the Iranian Statistical Society, 19(1), 39-57.

Robinson, R. W. (1977), Counting unlabeled acyclic digraphs. Springer Berlin Heidelberg. Rohatgi, V. K., and Saleh, A.K.M.E.S. (2015), An introduction to probability and statistics. John Wiley & Sons.

Shimizu, S., Hoyer, P. O., Hyvarinen, A., Kerminen, A., and Jordan, M.I. (2006), A linear non-Gaussian acyclic model for causal discovery. Journal of Machine Learning Research, 7, 2003-2030.

Skitovitch, V. P. (1953), On a property of the normal distribution. DAN SSSR, 89(1), 217-219.

Spiegelhalter, D. J., Dawid, A. P., Lauritzen, S. L., and Cowell, R. G. (1993), Bayesian analysis in expert systems. Statistical science, 8(3), 219-247.

Spirtes, P., and Glymour, C. (1991), An algorithm for fast recovery of sparse causal graphs. Social science computer review, 9(1), 62-72.

Stone, J. V. (2004), Independent component analysis: a tutorial introduction. MIT press.

Wright, S. (1920a), Principles of livestock breeding. US Department of Agriculture.

Wright, S. (1920b), The relative importance of heredity and environment in determining the piebald pattern of guinea-pigs. US Department of Agriculture.

Zareifard, H., Rezaei Tabar,V., and Plewczynski, D. (2021),AGibbs sampler for learning DAG: a unification for discrete and Gaussian domains. Journal of Statistical Computation and Simulation, 91(14), 2833-2853.

Zheng, X. (2020), Learning DAGs with Continuous Optimization [PhD diss., University of Pittsburgh Medical Center].