无约束梯度算法

梯度方向

梯度方向的定义

为什么选梯度方向

<img src="https://img2018.cnblogs.com/blog/1414369/201905/1414369-20190504125519759-953385073.jpg"width="440"height="200" align=center/>

<img src="https://img2018.cnblogs.com/blog/1414369/201905/1414369-20190504125530059-1378762556.jpg"width="440"height="200" align=center/>

沿梯度方向存在的问题

<img src="https://img2018.cnblogs.com/blog/1414369/201905/1414369-20190504125618818-1412551098.jpg"width="440"height="200" align=center/>

注：其实就是“沿梯度方向，函数不能再有限步达到最优！”

梯度算法

梯度算法的定义

<img src="https://img2018.cnblogs.com/blog/1414369/201905/1414369-20190504125832328-1000740396.jpg"width="440"height="200" align=center/>

<img src="https://img2018.cnblogs.com/blog/1414369/201905/1414369-20190504125842894-1609553471.jpg"width="440"height="200" align=center/>

<img src="https://img2018.cnblogs.com/blog/1414369/201905/1414369-20190504125851445-1058170853.jpg"width=440"height="200" align=center/>

梯度算法例题

<img src="https://img2018.cnblogs.com/blog/1414369/201905/1414369-20190504130102073-2002014241.png"width="440"height="200" align=center/>

最优梯度

最优梯度的定义

<img src="https://img2018.cnblogs.com/blog/1414369/201905/1414369-20190504130148066-1765828659.jpg"width="440"height="200" align=center/>

<img src="https://img2018.cnblogs.com/blog/1414369/201905/1414369-20190504130157956-1869652286.jpg"width="440"height="200" align=center/>

最优梯度的例题

<img src="https://img2018.cnblogs.com/blog/1414369/201905/1414369-20190504130315585-543518543.png"width="440"height="200" align=center/>

最优梯度的收敛特性

<img src="https://img2018.cnblogs.com/blog/1414369/201905/1414369-20190504130349193-1506040087.jpg"width="440"height="200" align=center/>