Bayesian Neural Networks for Nonlinear Regression: Posterior Inference, Uncertainty Quantification and Scalability

Articles in Press

Document Type : Original Article

Authors
Fatemeh Hosseini
Omid Karimi
Semnan University, Semnan, Iran
10.22034/jirss.2026.2085011.1182
Abstract
Bayesian neural networks provide a probabilistic framework for nonlinear regression by combining the expressive flexibility of neural networks with principled uncertainty quantification. However, practical implementation remains challenging because posterior inference is computationally demanding and often requires approximate methods. This study presents a comparative analysis of several inference approaches for Bayesian neural-network regression, including Hamiltonian Monte Carlo, the No- U-Turn Sampler and variational inference, with emphasis on predictive uncertainty, posterior calibration and computational efficiency. The results show that different inference strategies often achieve comparable predictive accuracy, whereas substantially larger differences emerge in uncertainty quantification and scalability. Sampling-based approaches provide more reliable posterior characterization and better-calibrated predictive uncertainty, particularly under complex noise structures, but incur substantially higher computational cost. In contrast, variational inference offers competitive predictive performance together with markedly improved computational efficiency, although it may underestimate posterior uncertainty in more challenging settings. Overall, the findings suggest that the primary practical benefit of Bayesian neural networks lies in reliable and interpretable uncertainty quantification rather than solely improved point prediction. The choice of inference strategy should therefore balance posterior fidelity, uncertainty calibration, and computational scalability according to
Keywords
Bayesian neural networks
Hamiltonian Monte Carlo
Variational inference
Uncertainty calibration
Nonlinear regression
Subjects
Bishop CM. Pattern Recognition and Machine Learning. Springer; 2006.
Blei DM, Kucukelbir A, McAuliffe JD. Variational inference: A review for statisticians. Journal of the American Statistical Association. 2017;112(518):859–877. https://doi.org/10.1080/01621459.2017.1285773.
Blundell C, Cornebise J, Kavukcuoglu K, Wierstra D.: Weight Uncertainty in Neural Networks; 2015. https://arxiv.org/abs/1505.05424.
Breiman L. Random forests. Machine Learning. 2001;45(1):5–32. https://doi.org/10.1023/A:1010933404324.
Gal Y, Ghahramani Z.: Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning; 2016. https://arxiv.org/abs/1506.02142.
Gawlikowski J, Tassi CRN, Ali M, Lee J, Humt M, Feng J, et al. A survey of uncertainty in deep learning. Machine Learning. 2023;112:745–811. https://doi.org/10.1007/s10994-021-05946-3.
Ghahramani Z. Probabilistic machine learning and artificial intelligence. Nature.2015;521(7553):452–459. https://doi.org/10.1038/nature14541.
Goodfellow I, Bengio Y, Courville A. Deep Learning. MIT Press; 2016.
Hastie T, Tibshirani R. Generalized Additive Models. Chapman and Hall/CRC; 1990.
Hoffman MD, Gelman A. The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo. Journal of Machine Learning Research. 2014;15(1):1593–1623. https://doi.org/10.5555/2627435.2638586.
Hornik K. Approximation capabilities of multilayer feedforward networks. Neural Networks. 1991;4(2):251–257. https://doi.org/10.1016/0893-6080(91)90009-T.
Kendall A, Gal Y.: What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?; 2017. https://arxiv.org/abs/1703.04977.
Kingma DP, Ba J.: Adam: A Method for Stochastic Optimization; 2014. https://arxiv.org/abs/1412.6980.
McCullagh P, Nelder JA. Generalized Linear Models. 2 ed. Chapman and Hall; 1989.
Neal RM. Bayesian Learning for Neural Networks. Lecture Notes in Statistics, Springer; 1996.
Rasmussen CE, Williams CKI. Gaussian Processes for Machine Learning. MIT Press; 2006.
Wang H, Yeung DY. Towards Bayesian deep learning: A framework and some existing methods. IEEE Transactions on Knowledge and Data Engineering. 2016;28(12):3395–3408. https://doi.org/10.1109/TKDE.2016.2606428.
Zhang C, Butepage J, Kjellstrom H, Mandt S. Advances in variational inference. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2018;41(8):2008–2026. https://doi.org/10.1109/TPAMI.2018.2889774.
Journal of the Iranian Statistical Society

Articles in Press, Accepted Manuscript
Available Online from 30 June 2026

  • Receive Date 09 February 2026
  • Revise Date 15 June 2026
  • Accept Date 15 June 2026