Workload-aware Materialization for Efficient Variable Elimination on Bayesian Networks
2021 (English)In: 2021 IEEE 37TH INTERNATIONAL CONFERENCE ON DATA ENGINEERING (ICDE 2021), Institute of Electrical and Electronics Engineers (IEEE) , 2021, p. 1152-1163Conference paper, Published paper (Refereed)
Abstract [en]
Bayesian networks are general, well-studied probabilistic models that capture dependencies among a set of variables. Variable Elimination is a fundamental algorithm for probabilistic inference over Bayesian networks. In this paper, we propose a novel materialization method, which can lead to significant efficiency gains when processing inference queries using the Variable Elimination algorithm. In particular, we address the problem of choosing a set of intermediate results to precompute and materialize, so as to maximize the expected efficiency gain over a given query workload. For the problem we consider, we provide an optimal polynomial-time algorithm and discuss alternative methods. We validate our technique using real-world Bayesian networks. Our experimental results confirm that a modest amount of materialization can lead to significant improvements in the running time of queries, with an average gain of 70%, and reaching up to a gain of 99%, for a uniform workload of queries. Moreover, in comparison with existing junction tree methods that also rely on materialization, our approach achieves competitive efficiency during inference using significantly lighter materialization.
Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE) , 2021. p. 1152-1163
Series
IEEE International Conference on Data Engineering, ISSN 1084-4627
Keywords [en]
probabilistic inference, materialization
National Category
Probability Theory and Statistics Computer Sciences
Identifiers
URN: urn:nbn:se:kth:diva-302588DOI: 10.1109/ICDE51399.2021.00104ISI: 000687830800097Scopus ID: 2-s2.0-85112865571OAI: oai:DiVA.org:kth-302588DiVA, id: diva2:1606561
Conference
37th IEEE International Conference on Data Engineering (IEEE ICDE), APR 19-22, 2021, ELECTR NETWORK
Note
Part of proceedings: ISBN 978-1-7281-9184-3, QC 20230117
2021-10-272021-10-272025-01-27Bibliographically approved