DNN前向传播与反向传播

${a^l} = \sigma \left( {{z^l}} \right) = \sigma \left( {{W^l}{a^{l - 1}} + {b^l}} \right)$，${W^l}$是${\rm{n}} \times m$矩阵，$n$是第$l$层神经元个数，$m$是第$l{\rm{ - 1}}$层神经元个数

 

DNN反向传播

${z^{l + 1}} = {W^{l + 1}}\sigma \left( {{z^l}} \right) + {b^{l + 1}},{\rm{   }}{a^{l + 1}} = \sigma \left( {{W^{l + 1}}{a^l} + {b^{l + 1}}} \right)$

 

$\frac{{\partial J\left( {W,b,x,y} \right)}}{{\partial {W^l}}} = {\delta ^l}{\left( {{a^{l - 1}}} \right)^T}$

$\frac{{\partial J\left( {W,b,x,y} \right)}}{{\partial {b^l}}} = {\delta ^l}$

 

参考博客

https://www.cnblogs.com/pinard/p/6422831.html