Deep Learning5: Linear Decoders with Autoencoders

对于

\begin{align} z^{(3)} &= W^{(2)} a^{(2)} + b^{(2)} \\ a^{(3)} &= f(z^{(3)}) \end{align}

Linear Decoders设定，a(3) = z(3)则称之为线性编码

sigmoid激活函数要求输入范围在[0,1]之间，某些数据集很难满足，则采用线性编码

此时，误差项更新为

\begin{align} \delta_i^{(3)} = \frac{\partial}{\partial z_i} \;\; \frac{1}{2} \left\|y - \hat{x}\right\|^2 = - (y_i - \hat{x}_i) \cdot f'(z_i^{(3)}) \end{align}

\begin{align} \delta_i^{(3)} = - (y_i - \hat{x}_i) \end{align}

\begin{align} \delta^{(2)} &= \left( (W^{(2)})^T\delta^{(3)}\right) \bullet f'(z^{(2)}) \end{align}

posted on 2016-10-13 17:36  Beginnerpatienceless  阅读(194)  评论(0)  编辑  收藏  举报