How proper are Bayesian models in the astronomical literature?
-
by
JeanTate
That's the title of an astro-ph preprint, Tak+ (2017), dated 10 December 2017. Here's the abstract:
The well-known Bayes theorem assumes that a posterior distribution is a probability distribution. However, the posterior distribution may no longer be a probability distribution if an improper prior distribution (non-probability measure) such as an unbounded uniform prior is used. Improper priors are often used in the astronomical literature to reflect on a lack of prior knowledge, but checking whether the resulting posterior is a probability distribution is sometimes neglected. It turns out that 24 articles out of 75 articles (32%) published online in two renowned astronomy journals (ApJ and MNRAS) between Jan 1, 2017 and Oct 15, 2017 make use of Bayesian analyses without rigorously establishing posterior propriety. A disturbing aspect is that a Gibbs-type Markov chain Monte Carlo (MCMC) method can produce a seemingly reasonable posterior sample even when the posterior is not a probability distribution (Hobert and Casella, 1996). In such cases, researchers may erroneously make probabilistic inferences without noticing that the MCMC sample is from a non-existent probability distribution. We review why checking posterior propriety is fundamental in Bayesian analyses when improper priors are used and discuss how we can set up scientifically motivated proper priors to avoid the pitfalls of using improper priors.
Curiously, it sparked another, "The paper "How proper are Bayesian models in the astronomical literature?" [arXiv:1712.03549] by Tak, Ghosh and Ellis is improper", Sereno (2017). Here's the abstract:
In their "How proper are Bayesian models in the astronomical literature?" [arXiv:1712.03549], Hyungsuk Tak, Sujit K. Ghosh and Justin A. Ellis criticised my work with false statements. This is an infamous case of straw man fallacy. They give the impression of refuting an opponent's argument, while they refute an argument that was not presented.
Whether or not Sereno is correct/has a point, the general thrust of Tak+ (2017) seems apt, don't you think?
Posted
-
by
mlpeck
Jean:
Without having done more than skim the paper by Tak+ it's important to point out that "proper" in this context has a technical meaning that's much more restricted than the usual English language usage. The title is not synonymous with asking, for example, "Is it appropriate to use Bayesian models in astronomical literature?".
Proper, in this context, just means that a certain integral (the one in the denominator of their equation 1) is finite.
Posted
-
by
JeanTate
in response to mlpeck's comment.
Yes indeed.
Although I noticed this paper in astro-ph, it did not sufficiently pique my interest until I read telescoper's blog post on it, Have you got a proper posterior?
An "aha!" light went off when I read about the distinction between prior (and posterior) distributions and probability distributions ... I had somehow, intuitively, not realized that this is (can be) an important distinction, and that it should at least be considered.
Posted
-
by
mlpeck
in response to JeanTate's comment.
Jean:
The title is pretty clickbait-y. I think they are right to criticize authors for failing to specify priors clearly or for using improper uniform priors without explanation, and there's some useful (and some not so useful) advice in their discussion. But the issues they raise are more technical than philosophical.
As for the work by Sereno & collaborators that got such an indignant response, their underlying simulation engine is a package called JAGS that actually enforces a requirement for proper priors, so they can't be guilty of that crime at least. They've also published their code, which is at least a good start towards doing reproducible science.
Posted
-
by
JeanTate
in response to mlpeck's comment.
The title is pretty clickbait-y.
Yes, it surely is. As are a rather too large number (for my comfort) of titles in recent times, at least in astro-ph.
They've also published their code, which is at least a good start towards doing reproducible science.
For which they deserve much praise. The "publish your code" part of Open Science has been rather slow to take off, as I see it, certainly much slower than the "publish your data" part. 😦
I think they are right to criticize authors for failing to specify priors clearly or for using improper uniform priors without explanation
I particularly like that they have done so. In my own research, I have come across rather too many instances of, shall I say, sloppy work being published in respected peer-reviewed journals. I'm pretty sure many of those very familiar with the niche are well aware of such shortcomings, yet it's rare that anyone goes on record as pointing them out (this is meant to be general, certainly far broader than improper priors).
Posted