ddpm损失函数

ddpm损失函数

从本文开始正式介绍ddpm损失函数。在扩散模型推导前置中我们首次介绍了最大化似然的目标，通过逆向过程$p$计算$x_0$概率最大化就可以生成图片

$$\mathbb{E}[-\log p_{\theta }(x_0)]\le {\mathbb{E}}_q[-\log \frac{p_{\theta }(x_{0:T})}{q(x_{1:T}|x_0)}]$$

$$p_{\theta }(x_{0:T}):=p(x_T)\prod _{t=1}^T{p}_{\theta }(x_{t-1}|x_t)$$

$$q(x_{1:T}|x_0):=\prod _{t=1}^{T}q(x_t|x_{t-1})$$

正文

我们将$p_{\theta },q$带入公式中：

$$\begin{array}{}\text{(Eq.1)}& \begin{array}{rl}-\log \frac{p_{\theta }(x_{0:T})}{q(x_{1:T}|x_0)}& =-\log \frac{p(x_T)\prod _{t=1}^T{p}_{\theta }(x_{t-1}|x_t)}{\prod _{t=1}^{T}q(x_t|x_{t-1})}\\ & =-\log p(x_T)-\log \prod _{t-1}^T{p}_{\theta }(x_{t-1}|x_t)+\log \prod _{t=1}^{T}q(x_t|x_{t-1})\\ & =-\log p(x_T)-\sum _{t=1}^T\log p_{\theta }(x_{t-1}|x_t)+\sum _{t=1}^T\log q(x_t|x_{t-1})\\ & =-\log p(x_T)-\sum _{t=1}^T\log p_{\theta }(x_{t-1}|x_t)+\sum _{t=1}^T\log \frac{q(x_{t-1}|x_t)q(x_t)}{q(x_{t-1})}\\ & =-\log p(x_T)-\sum _{t=1}^T\log p_{\theta }(x_{t-1}|x_t)+\sum _{t=1}^T\log q(x_{t-1}|x_t)+\log \frac{q(x_T)}{q(x_0)}\\ & =-\log p(x_T)-\sum _{t=1}^T\log \frac{p_{\theta }(x_{t-1}|x_t)}{q(x_{t-1}|x_t)}+\log \frac{q(x_T)}{q(x_0)}\end{array}\end{array}$$

Eq.1中$\frac{q(x_T)}{q(x_0)}$是常量，最大似然估计前两项即可，和附件中公式(3)的计算结果一致：

$$\begin{array}{}\text{(Eq.2)}& {\mathbb{E}}_q[-\log \frac{p_{\theta }(x_{0:T})}{q(x_{1:T}|x_0)}]={\mathbb{E}}_q[-\log p(x_T)-\sum _{t=1}^T\log \frac{p_{\theta }(x_{t-1}|x_t)}{q(x_{t-1}|x_t)}]=:L\end{array}$$

损失函数引入条件$x_0$

我们在前文逆向过程推导中对$q$引入了条件$x_0$完成计算，我们针对损失也可以同样引入。我们将Eq.1的最终结果进一步整理，将变量相同的放在一起：

$$\begin{array}{}\text{(Eq.3)}& \begin{array}{rl}-\log \frac{p_{\theta }(x_{0:T})}{q(x_{1:T}|x_0)}& =-\log p(x_T)-\sum _{t=1}^T\log \frac{p_{\theta }(x_{t-1}|x_t)}{q(x_{t-1}|x_t)}+\log \frac{q(x_T)}{q(x_0)}\\ & =-\log \frac{p(x_T)}{q(x_T)}-\sum _{t=1}^T\log \frac{p_{\theta }(x_{t-1}|x_t)}{q(x_{t-1}|x_t)}-\log (q(x_0))\\ & \Rightarrow -\log \frac{p(x_T)}{q(x_T|x_0)}-\sum _{t=1}^T\log \frac{p_{\theta }(x_{t-1}|x_t)}{q(x_{t-1}|x_t,x_0)}-\log (q(x_0|x_0))& & (\mathrm{引}\mathrm{入}\mathrm{条}\mathrm{件}{x}_0)\\ & =-\log \frac{p(x_T)}{q(x_T|x_0)}-\sum _{t=1}^T\log \frac{p_{\theta }(x_{t-1}|x_t)}{q(x_{t-1}|x_t,x_0)}-\log (q(x_0|x_0))\\ & =-\log \frac{p(x_T)}{q(x_T|x_0)}-\sum _{t=1}^T\log \frac{p_{\theta }(x_{t-1}|x_t)}{q(x_{t-1}|x_t,x_0)}& & (q(x_0|x_0)=1)\\ & =-\log \frac{p(x_T)}{q(x_T|x_0)}-\sum _{t>1}^T\log \frac{p_{\theta }(x_{t-1}|x_t)}{q(x_{t-1}|x_t,x_0)}-\log \frac{p_{\theta }(x_0|x_1)}{q(x_0|x_t,x_0)}& & (\mathrm{拆}\mathrm{分}t=1)\\ & =-\log \frac{p(x_T)}{q(x_T|x_0)}-\sum _{t>1}^T\log \frac{p_{\theta }(x_{t-1}|x_t)}{q(x_{t-1}|x_t,x_0)}-\log p_{\theta }(x_0|x_1)\end{array}\end{array}$$

我们将Eq.3加上期望，最终结果如下：

$$\begin{array}{}\text{(Eq.4)}& \begin{array}{rl}{\mathbb{E}}_q[-\log \frac{p_{\theta }(x_{0:T})}{q(x_{1:T}|x_0)}]& ={\mathbb{E}}_q[-\log \frac{p(x_T)}{q(x_T|x_0)}-\sum _{t>1}^T\log \frac{p_{\theta }(x_{t-1}|x_t)}{q(x_{t-1}|x_t,x_0)}-\log p_{\theta }(x_0|x_1)]\\ & ={\mathbb{E}}_q[-\int q(x_T|x_0)\log \frac{p(x_T)}{q(x_T|x_0)}dx-\sum _{t>1}^T\int q(x_{t-1}|x_t,x_0)\log \frac{p_{\theta }(x_{t-1}|x_t)}{q(x_{t-1}|x_t,x_0)}dx-\log p_{\theta }(x_0|x_1)\int q(x)dx]\\ & ={\mathbb{E}}_q[D_{KL}[q(x_T|x_0)\Vert p(x_T)]+D_{KL}[q(x_{t-1}|x_t,x_0)\Vert p_{\theta }(x_{t-1}|x_t)]-\log p_{\theta }(x_0|x_1)]& & (KL\mathrm{散}\mathrm{度}\mathrm{定}\mathrm{义})\end{array}\end{array}$$

Eq.4和附件中的公式(5)结果一致。综上我们通过推导最大似然估计的损失直到使用KL散度判断$q,p$的相似度完成训练目标。在这里我们额外提一下第二项中$q(x_{t-1}|x_t,x_0)$，根据马尔科夫链的性质，这里的$x_0$应该是可以去除的，但是论文保留了，为了结果一致性我们也保留了$x_0$

至此我们完成了background所有内容的介绍，对ddpm论文阅读基本没有太多障碍了。在扩散模型推导前置文章中我们介绍了附件公式(1)(2)(3)左边的内容，在前向过程推导文章中我们介绍了附件公式(2)右(4)的由来,在逆向过程推导文章中我们介绍了附件公式(6)(7)。最终在本文介绍完了附件公式(3)和(5)。

附件（原文background）