### 一.指数族分布的形式¶

$$p(x\mid\eta)=h(x)g(\eta)exp[\eta^T\mu(x)]$$

#### 伯努利分布¶

$$p(x\mid\mu)=\mu^x(1-\mu)^{1-x}\\ =exp[xlog\mu+(1-x)log(1-mu)]\\ =(1-\mu)exp[log(\frac{\mu}{1-\mu})x]$$

$$\mu=\sigma(\eta)=\frac{1}{1+exp(-\eta)}$$

$$h(x)=1,g(\eta)=1-\mu=1-\sigma(\eta)=\sigma(-\eta),\mu(x)=x$$

#### 单一观测的多项式分布¶

$$p(x\mid\mu)=\prod_{k=1}^M\mu_k^{x_k}=exp[\sum_{k=1}^Mx_klog\mu_k]$$

$$h(x)=1\\ g(\eta)=1\\ \mu(x)=(x_1,...,x_M)^T=x\\ \eta=(log\mu_1,...,log\mu_M)^T$$

#### 一元高斯分布¶

$$p(x\mid\mu,\sigma^2)=\frac{1}{(2\pi\sigma^2)^{\frac{1}{2}}}exp[-\frac{1}{2\sigma^2}(x-\mu)^2]\\ =\frac{1}{(2\pi\sigma^2)^{\frac{1}{2}}}exp[-\frac{1}{2\sigma^2}x^2+\frac{\mu}{\sigma^2}x-\frac{1}{2\sigma^2}\mu^2]\\ =\frac{1}{(2\pi\sigma^2)^{\frac{1}{2}}}exp[-\frac{1}{2\sigma^2}\mu^2]exp[-\frac{1}{2\sigma^2}x^2+\frac{\mu}{\sigma^2}x]$$

$$\eta=(\frac{\mu}{\sigma^2},\frac{-1}{2\sigma^2})^T\\ \mu(x)=(x,x^2)^T\\$$

$$h(x)=(2\pi)^{2\frac{1}{2}}\\ g(\eta)=(-2\eta_2)^{\frac{1}{2}}exp(\frac{\eta_1^2}{4\eta_2})$$

### 二.极大似然估计¶

$$g(\eta)\int h(x)exp[\eta^T\mu(x)]dx=1$$

$$\nabla g(\eta)\int h(x)exp[\eta^T\mu(x)]dx+g(\eta)\int h(x)exp[\eta^T\mu(x)]u(x)dx=0\\ \Leftrightarrow -\nabla g(\eta)\frac{1}{g(\eta)}=g(\eta)\int h(x)exp[\eta^T\mu(x)]u(x)dx=E[\mu(x)]\\ \Leftrightarrow -\nabla ln[g(\eta)]=E[\mu(x)]$$

$$-\nabla ln[g(\eta_{ML})]=\frac{1}{N}\sum_{n=1}^N\mu(x_n)$$

### 三.共轭先验¶

$$p(\eta\mid \chi,\nu)=f(\chi,\nu)g(\eta)^\nu exp[\nu\eta^T\chi]$$

$$p(X\mid\eta)=(\prod_{n=1}^Nh(x_n))g(\eta)^Nexp[\eta^T\sum_{n=1}^N\mu(x_n)]$$

$$p(\eta\mid x,\chi,\nu)\propto g(\eta)^{\nu+N}exp[\eta^T(\sum_{n=1}^N\mu(x_n)+\nu\chi)]$$

### 四.小结一下¶

In [ ]: