机器学习相关规定

machine learning ≈ looking for function

this course focuses on deep learning

deep learning's function is 类神经网络

Step1:function with unknown

Step2:define loss from training data

Step3:optimization

弹性化function之后得$y=b+\sum \limits_{i} c_{i} \text { sigmoid }\left(b_{i}+\sum \limits_{j} w_{i j} x_{j}\right)$

其中$\alpha_1 = sigmoid(r_1)=\frac{1}{1+e^{-r_1}}$

$\theta$代表所有未知参数，$L(\theta)$为Loss

$\boldsymbol{\theta}^{*}=\arg \min \limits_{\theta} L$

$\boldsymbol{\theta}=\left[\begin{array}{c}
\theta_{1} \\
\theta_{2} \\
\theta_{3} \\
\vdots
\end{array}\right]$

optimization过程：

(Randomly)Pick initial values $\theta^0$

Compute gradient $\boldsymbol{g}=\nabla L\left(\boldsymbol{\theta}^{0}\right) \cdots$

g即梯度(gradient)

$\boldsymbol{g}=\nabla L\left(\boldsymbol{\theta}^{0}\right)$

接下来update一下

$\left[\begin{array}{c}
\theta_{1}^{1} \\
\theta_{2}^{1} \\
\vdots
\end{array}\right] \leftarrow\left[\begin{array}{c}
\theta_{1}^{0} \\
\theta_{2}^{0} \\
\vdots
\end{array}\right]-\left[\begin{array}{c}
\left.\eta \frac{\partial L}{\partial \theta_{1}}\right|_{\boldsymbol{\theta}=\boldsymbol{\theta}^{0}} \\
\left.\eta \frac{\partial L}{\partial \theta_{2}}\right|_{\boldsymbol{\theta}=\boldsymbol{\theta}^{0}} \\
\vdots
\end{array}\right]$

简写为$\boldsymbol{\theta}^{1} \leftarrow \boldsymbol{\theta}^{0}-\eta \boldsymbol{g}$

$\eta$即为学习率

但实际上我们使用batch来update，所有的batch看过一遍叫一次epoch