Conjugate distributions

In [83]:
import numpy as np
from numpy.random import beta
import matplotlib.pyplot as plt

Beta-Bernoulli

Let's look at the shape of the Beta distribution

In [84]:
a, b = 1, 1
densities = plt.hist([beta(a,b) for _ in range(5000)], bins=10)

Suppose we have observed a sequence of $n$ coin flips with $x$. Then the posterior -- what we know believe about $p$, given our prior and the new evidence -- also follows a Beta distribution.

Here is the derivation (from the Wiki page), where $s$ denotes "successes" (heads, here) and $f$ failures (so $n-x$).

alt text

This is a Beta distribution with parameters $\alpha + s, \beta + f$.

In [85]:
def posterior(a, b, x, n):
    samples = [beta(a + x, b + n - x) for _ in range(10000)]
    densities = plt.hist(samples, bins=10)
    return np.array(samples)
In [88]:
samples = posterior(.01, 1, 1, 1)
In [77]:
np.mean(samples)
Out[77]:
0.6677448353537242
In [78]:
samples = posterior_under_prior(1, .01, 1, 2)
In [79]:
samples = posterior_under_prior(.01, 1, 1, 2)
In [91]:
samples = posterior_under_prior(1, .001, 100, 200)

Normal-Normal

There is a nice interactive demo here: https://stephens999.github.io/fiveMinuteStats/shiny_normal_example.html

In [ ]: