Bayesian inference

Bayesian inference is a statistical inference in which probabilities are interpreted not as frequencies or proportions or the like, but rather as degrees of belief. The name comes from the frequent use of Bayes' theorem in this discipline.

More mathematical examples

Naive Bayes classifier

See: naive Bayes classifier.

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

Posterior distribution of the binomial parameter

In this example we consider the computation of the posterior distribution for the binomial parameter.

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

This is the same problem considered by Bayes in Proposition 9 of his essay.

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

We are given m observed successes and n observed failures in a binomial experiment.

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

The experiment may be tossing a coin, drawing a ball from an urn, or asking someone their opinion, among many other possibilities.

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

What we know about the parameter (let's call it a) is stated as the prior distribution, p(a).

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

For a given value of a,

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

the probability of m successes in m+n trials is

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

: p(m,n|a) = egin{pmatrix} n+m \ m end{pmatrix} a^m (1-a)^n.

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

Since m and n are fixed, and a is unknown,

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

this is a likelihood function for a.

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

From the continuous form of the law of total probability we have

~ ~ ~ ~ ~ ~ ~ ~ ~ ~

: p(a|m,n) = rac{p(m,n|a),p(a)}{int_0^1 p(m,n|a),p(a),da}

~ ~ ~ ~ ~ ~ ~ ~ ~ ~