逻辑回归
性质
- 判别模型
- 分类模型
模型
\[P(Y=1|x) = \frac{e^{w \cdot x}}{1+e^{w \cdot x}}\\
P(Y=0|x)=\frac{1}{1+e^{w \cdot x}}
\]
损失函数
最大似然估计
\[\begin{aligned}
L(w) & = \prod_{i=1}^N P(y_i=1|x_i)^{y_i}P(y_i=0|x_i)^{(1-y_i)}\\
& = \prod_{i=1}^N P(y_i=1|x_i)^{y_i}[1-P(y_i=1|x_i)]^{(1-y_i)}\\
\end{aligned}
\]
取负对数
\[\begin{aligned}
J(w) & = -lnL(w)\\
& = - \sum_{i=1}^N [y_ilogP(y_i=1|x_i)+(1-y_i)log(1-P(y_i=1|x_i))]\\
& = - \sum_{i=1}^N [y_ilog\frac{P(y_i=1|x_i)}{1-P(y_i=1|x_i)}+log(1-P(y_i=1|x_i))]\\
& = - \sum_{i=1}^N [y_i(wx_i)-log(1+e^{wx_i})]
\end{aligned}
\]
训练算法
梯度下降
