Many adaptive IS (and some VI) methods are based on matching the moments of a target distributions. When the target has heavy tails, these moments can be undefined or their estimation can have high variance. We propose an AIS method that overcomes this by matching the moments of a (lighter tailed) modified target, which is exponentiated to a power alpha. Despite this, the procedure actually minimizes the alpha-divergence between the proposal and the true target. Note: many previous works propose AIS methods with heavy-tailed proposals, but not necessarily suitable for heavy-tailed targets.
Publications, preprints & working papers
- Adaptive importance sampling for heavy-tailed distributions via α-divergence minimization; Branchini, Nicola♦ and Guilmeau, Thomas♦ and Chouzenoux, Emilie and Elvira, Víctor. To appear: 27th Conference on Artificial Intelligence and Statistics (AISTATS), Proceedings of Machine Learning Research, 2024. (♦ arbitrary orderings, equal contribution.)
- Variational Resampling; Kviman, Oskar and Branchini, Nicola and Elvira, Víctor and Lagergen, Jens. To appear: 27th Conference on Artificial Intelligence and Statistics (AISTATS), Proceedings of Machine Learning Research, 2024.
Coming !
Causal optimal transport of abstractions; Felekis, Yorgos and Zennaro, Fabio and Branchini, Nicola and Damoulas, Theodoros. In 3rd Conference on Causal Learning and Reasoning (CLeaR 2024).
The task of causal abstraction involves finding a mapping (a measurable transport map) between structural causal models (SCMs) and their corresponding “abstracted versions”, which can be simplified or coarser SCMs (fewer variables or different functional relationships). We consider the problem of learning causal abstractions from data. We propose a framework that does so without specifying parametric relationships for the SCM functions. The method involves a multimarginal OT problem (as many marginals as there are considered interventions (not really, but roughly to get the idea)) with soft constraints and a cost function econding knowledge of the underlying causal DAGs. Nicely, the soft constraints have a do-calculus interpretation.
An adaptive mixture view of particle filters; Branchini, Nicola and Elvira, Víctor. To appear in: FoDS (Foundations of Data Science).
Coming !
- Generalized self-normalized importance sampling; Branchini, Nicola and Elvira, Víctor. (In revision; presented at MCM 2023; link to BayesComp2023 poster).
Coming !!
- Causal Entropy Optimization; Branchini, Nicola and Aglietti, Virginia and Dhir, Neil and Damoulas, Theodoros. In 26th Conference on Artificial Intelligence and Statistics (AISTATS), Proceedings of Machine Learning Research, 2023.
In this paper, we studied the problem of “causal global optimization”: finding the optimum intervention that is the minimizer of several causal effects (that is, we consider possibly intervening on many different subset of variables). When the underlying causal graph is not known, the first step is studying what happens if we assume any one of the possible graphs is the true one, and run “CBO”- causal Bayesian optimization - as normal. We studied what the effect of this kind of incorrect causal assumption is for optimization purposes. Further, since in many cases the underlying function can be optimized efficiently even if the graph is not fully known, we designed an acquisition function that automatically trades-off optimization of the effect and structure learning.
- Optimized Auxiliary Particle Filters: adapting mixture proposals via convex optimization, Branchini, Nicola and Elvira, Víctor. In 37th Conference on Uncertainty in Artificial Intelligence (UAI), Proceedings of Machine Learning Research, 2021.
In this paper we wanted to improve on the Auxiliary Particle Filter (APF), which is thought for estimating the likelihood in sequential latent variable models with very informative observations. This algorithm however still has severe drawbacks; among some, the resampling weights are chosen independently, i.e. each particle chooses its own without “knowing” what the others are doing. We devise a new way to optimize these resampling weights by viewing them as mixture weights of an importance sampling mixture proposal. It turns out that choosing mixture weights in order to minimize the resulting empirical variance of the importance weights leads to a convex optimization problem.
Video and slides from UAI