Bayesian Inference with Bayes Factors

Motivation

Mr. Micawber’s rule of scientific success

We really do need to move away from \(p\)-values as a gold-standard of truth in experimental science. The ruinous role of the \(p\)-value in modern science can not be overstated. This one value is responsible for happiness and despair in equal measure. They say money is the root of all unhappiness and Dicken’s Mr. Micawber had this to say about the role of money in life:

‘Annual income 20 pounds, annual expenditure 19 [pounds] 19 shillings and six pence, result happiness. Annual income 20 pounds, annual expenditure 20 pounds ought and six, result misery.’

in science a corollary exists:

‘\(p\) below 0.05, result success, papers, grants, and tenure. \(p\) above 0.05 result failure, misery, ignominy, and rejection.’

The truth is that \(p < 0.05\) is an entirely arbitrary cut-off and is not in itself a helpful or meaningful value. Various scientific communities, led by publishing requirements, have accepted \(p < 0.05\) as a gold standard of truth against sense and often against rigorousness. With Bayesian tests we will be able to completely do away with \(p\)-values and confidence intervals and in their place use a more evidence based approach to making inferences.

Learning to select hypotheses using Bayesian approaches

The sort of statistics that most experimental science students are taught are called ‘Frequentist Statistics’. They include the \(t\)-tests, ANOVA and \(\chi^2\)-tests and the linear models that we have studied already.

The inferential approach (how we make decisions about data) in the Frequentist paradigm is often criticised for being weak and is often abused. Although the abuse is as much a consequence of convention in the scientific literature and in scientific publishing, the misinterpretation of \(p\)-values by generations of scientists as it is the philosophical weakness of the methods themselves, the weaknesses persist and over time other paradigms have emerged.

We have seen an alternative in Estimation Statistics, in this course we will look at another - Bayesian Inference. We will use Bayes Factors to compare levels of evidence for one hypothesis over another, rather than just accepting or rejecting a simplistic null hypothesis.

The advantage of this will be that we can much more directly select between specific hypotheses that might describe our data. This will give us a much clearer idea about a question that we instinctively want to answer when we do statistics - ‘Which hypothesis is most likely true?’, we will see that we can formulate this in lots of ways, but in general the hypotheses we want to compare will be something along the lines of some measured quantity being different in different samples. With Frequentist Inference we can only ask the roundabout question, ‘How often does the difference we observe occur by chance?’ and if it isn’t likely, say so. With Bayes Factors we will be able to compare directly competing hypotheses and reject the least likely absolutely.