机器学习-线性回归

机器学习-线性回归

前言

线性回归是机器学习中基础的模型之一，是有监督模型。

定义：线性回归是确定两种或两种以上变量间相互依赖的定量关系的一种统计分析方法

表达式：$y=wx+b$

参数：${\theta }_0,{\theta }_1$

预测：$h={\theta }_0+{\theta }_{1}x$ 可得 $\left[\begin{array}{cc}1& x\end{array}\right]$$\left[\begin{array}{c}{\theta }_0\\ {\theta }_1\end{array}\right]$ = $X\theta$

h = np.dot(X,theta)

代价：$J=\frac{1}{2m}\sum (h-y{)}^2$

J = 0.5 * np.mean((h - y) ** 2)

梯度下降：$\mathrm{\Delta }{\theta }_j=\frac{1}{m}{X}^T(h-y)$

deltatheta = (1.0 / m) * X.T.dot(h - y)

更新参数：${\theta }_j={\theta }_j-\alpha \mathrm{\Delta }{\theta }_j$

theta = theta - alpha * deltatheta

正则化

正则化代价： $J=\frac{1}{2m}\sum (h-y{)}^2+\frac{\lambda }{2m}{\theta }^2$

正则化梯度下降：$$\mathrm{\Delta }{\theta }_j=\frac{1}{m}{X}^T(h-y)+\frac{\lambda }{2m}\theta$$

单变量线性回归

import numpy as np
import matplotlib.pyplot as plt
 
x = [4, 3, 3, 4, 2, 2, 0, 1, 2, 5, 1, 2, 5, 1, 3]
y = [8, 6, 6, 7, 4, 4, 2, 4, 5, 9, 3, 4, 8, 3, 6]
 
 
X = np.c_[np.ones(len(x)),x]
 
y = np.c_[y]
 
#预测
def mov(theta):
    h = np.dot(X,theta)
    return h
 
#代价
def cos(h):
    j = 0.5*np.mean((h-y)**2)
    return j
 
#梯度下降
def grad(sums=10000,alph=0.1):
    m,n = X.shape
    theta = np.zeros((n,1))
    j = np.zeros(sums)
    for i in range(sums):
        h = mov(theta)
        j[i] = cos(h)
        te = (1/m)*X.T.dot(h-y)
        theta -= alph * te
    return h,j,theta
 
if __name__ == '__main__':
    h,j,theta = grad()

github地址