### 一.原理推导¶

$$q(Z)\rightarrow p(Z\mid X)$$

$$KL(q\mid\mid p)=\int q(Z)ln\{\frac{q(Z)}{p(Z\mid X)}\}dZ$$

$$q^*(Z)=arg\min_{q(Z)}KL(q\mid\mid p)$$

$$p(X)=\frac{p(X,Z)}{p(Z\mid X)}$$

$$ln\ p(X)=ln\ p(X,Z)-ln\ p(Z\mid X)$$

$$ln\ p(X)=ln\ p(X,Z)-ln\ p(Z\mid X)\\ =ln\ \frac{p(X,Z)}{q(Z)}-ln\ \frac{p(Z\mid X)}{q(Z)}$$

$$ln\ p(X)=\int q(Z)ln\{\frac{p(X,Z)}{q(Z)}\}dZ-\int q(Z)ln\{\frac{p(Z\mid X)}{q(Z)}\}dZ\\ =\int q(Z)ln\{\frac{p(X,Z)}{q(Z)}\}dZ+\int q(Z)ln\{\frac{p(q(Z)}{Z\mid X)}\}dZ\\ =\mathcal{L}(q)+KL(q\mid\mid p)$$

$$q^*(Z)=arg\max_{q(Z)}\int q(Z)ln\{\frac{p(X,Z)}{q(Z)}\}dZ$$

### 二.对$q(Z)$进行简化¶

$$q(Z)=\prod_{i=1}^Mq_i(Z_i)$$

$$\mathcal{L}(q)=\int\prod_i q_i(Z_i)[ln\ p(X,Z)-\sum_i ln\ q_i(Z_i)]dZ\\ =\int q_j(Z_j)[\int ln\ p(X,Z)\prod_{i\neq j}q_i(Z_i)dZ_i]dZ_j-\int q_j(Z_j)ln\ q_j(Z_j)dZ_j+const\\ =\int q_j(Z_j)ln\ \tilde{p}(X,Z_j)dZ_j-\int q_j(Z_j)ln\ q_j(Z_j)dZ_j\\ =-KL(q_j(Z_j)\mid\mid\tilde{p}(X,Z_j))$$

$$ln\ \tilde{p}(X,Z_j)=\int ln\ p(X,Z)\prod_{i\neq j}q_i(Z_i)dZ_i+const=E_{i\neq j}[ln\ p(X,Z)]+const$$

$$q_j^*(Z_j)=\tilde{p}(X,Z_j)\\$$

$$\tilde{p}(X,Z_j)=\frac{exp(E_{i\neq j}[ln\ p(X,Z)])}{\int exp(E_{i\neq j}[ln\ p(X,Z)])dZ_j}（const即是分母部分取负对数）$$

$$ln\ q_j^*(Z_j)=E_{i\neq j}[ln\ p(X,Z)]+const$$

In [ ]: