Intuition behind back-propagation

没整明白forward-mode和reverse-mode，在这里可以找到答案

　　http://colah.github.io/posts/2015-08-Backprop/#fn1

 

记下让我印象比较深刻的三句话：

　　“Forward-mode differentiation gave us the derivative of our output with respect to a single input, but reverse-mode differentiation gives us all of them.”

　　“Where the reverse-mode gives the derivatives of one output with respect to all inputs, the forward-mode gives us the derivatives of all outputs with respect to one input.”

　　“Derivatives are cheaper than you think.”

 

再补充一个讲Auto Differentiation的链接

　　http://www.ams.org/publicoutreach/feature-column/fc-2017-12