5.1 Proof that a mixture of conjugate priors is indeed conjugate¶
$$p(\theta) =\sum_k p(z = k)p(\theta|z = k)$$\begin{align}
p(\theta|D)&=\frac{p(\theta, D)}{p(D)}\\
&=\frac{\sum_k p(\theta, D,z=k)}{p(D)}\\
&=\frac{\sum_k p(D,z=k)p(\theta|D,z=k)}{p(D)}\\
&=\sum_k p(z=k|D)p(\theta|D,z=k)\\
p(z=k|D)&=\frac{p(D,z=k)}{p(D)}\\
&=\frac{p(D,z=k)}{\sum_i p(D,z=i)}\\
&=\frac{p(z=k)p(D|z=k)}{\sum_i p(z=i)p(D|z=i)}
\end{align}