Computing Entropies with Nested Sampling
Date: 03 June 2022, Friday
Time: 4pm AEDT
Speaker: Dr Brendon Brewer (University of Auckland)
Abstract:
The Nested Sampling algorithm, invented in the mid-2000s by John Skilling,
represented a major advance in Bayesian computation. Whereas Markov Chain
Monte Carlo (MCMC) methods are usually effective for sampling posterior
distributions, Nested Sampling also calculates the marginal likelihood
integral used for model comparison, which is a computationally demanding
task. However, there are other kinds of integrals that we might want to
compute. Specifically, the entropy, relative entropy, and mutual
information, which quantify uncertainty and relevance, are all integrals
whose form is inconvenient in most practical applications. I will present
my technique, based on Nested Sampling, for estimating these quantities for
probability distributions that are only accessible via MCMC sampling. This
includes posterior distributions, marginal distributions, and distributions
of derived quantities. I will present an example from experimental design,
where one wants to optimise the relevance of the data for inference of a
parameter.
Link: https://unsw.zoom.us/j/82299381917?pwd=ZDRLeVZveFdDSHlOSGkxYWRMK0JXZz09
Password: 017349
Date: 03 June 2022, Friday
Time: 4pm AEDT
Speaker: Dr Brendon Brewer (University of Auckland)
Abstract:
The Nested Sampling algorithm, invented in the mid-2000s by John Skilling,
represented a major advance in Bayesian computation. Whereas Markov Chain
Monte Carlo (MCMC) methods are usually effective for sampling posterior
distributions, Nested Sampling also calculates the marginal likelihood
integral used for model comparison, which is a computationally demanding
task. However, there are other kinds of integrals that we might want to
compute. Specifically, the entropy, relative entropy, and mutual
information, which quantify uncertainty and relevance, are all integrals
whose form is inconvenient in most practical applications. I will present
my technique, based on Nested Sampling, for estimating these quantities for
probability distributions that are only accessible via MCMC sampling. This
includes posterior distributions, marginal distributions, and distributions
of derived quantities. I will present an example from experimental design,
where one wants to optimise the relevance of the data for inference of a
parameter.
Link: https://unsw.zoom.us/j/82299381917?pwd=ZDRLeVZveFdDSHlOSGkxYWRMK0JXZz09
Password: 017349