Explanation of logistic regression cost function

\begin{matrix} \hat{y} = σ (w^{T} x + b) w h e r e σ (z) = \frac{1}{1 + e^{- z}} \\ i n t e r p r e t \hat{y} = P (y = 1 ∣ x) \\ i f y = 1 : P (y ∣ x) = \hat{y} \\ i f y = 0 : P (y ∣ x) = 1 - \hat{y} \\ y = 0, 1 b e c a u s e o f b i n a r y c o s t e q u a t i o n . \\ a n d w e c a n s u c h e q u a t i o n t o m a i n t a i n i t s c o n t u n i t y \\ P (y ∣ x) = {\hat{y}}^{y} \cdot (1 - \hat{y})^{1 - y} \\ l o g (P (y ∣ x)) = y \cdot l o g (\hat{y}) + (1 - y) \cdot l o g (1 - \hat{y}) \\ a n d o u r s i n g l e l o s s f u n c t i o n = - l o g (P (y ∣ x)) \\ b e c a u s e m i n i m i z e l o s s i s e q u i v a l e n t t o m a x i m i z e P (y ∣ x) \end{matrix}

$\begin{array}{c} \hat{y} = \sigma(w^Tx+b)\quad where\;\sigma(z) = \frac{1}{1+e^{-z}}\\ interpret \quad\hat{y} =P(y=1\mid x)\\ if\quad y=1:P(y\mid x)=\hat{y}\\ if\quad y=0:P(y\mid x)=1-\hat{y}\\ y=0,1\;because\;of\;binary\;cost\;equation.\\ and\;we\;can\;such\;equation\;to\;maintain\;its\;contunity\\ P(y\mid x)=\hat{y}^y\cdot(1-\hat{y})^{1-y}\\ log(P(y\mid x))=y\cdot log(\hat{y})+(1-y)\cdot log(1-\hat{y})\\ and\;our\;single\;loss\;function=-log(P(y\mid x))\\ because\;minimize\;loss\;is\;equivalent\;to\;maximize\;P(y\mid x) \end{array}$

对于m个独立样例的数据集时

P (l a b e l s i n t r a i n i n g s e t) = \prod_{i = 1}^{m} P (y^{(i)} ∣ x^{(i)})

$P(labels\;in\;training\;set)=\prod^{m}_{i=1}P(y^{(i)}\mid x^{(i)})$

我们希望通过寻找一组参数使得上式概率最大(maximun likelihood estimation)，则

\begin{aligned} (1) & l o g P (\dots) & = \sum_{i = 1}^{m} l o g (P (y^{(i)} ∣ x^{(i)})) \\ (2) & = \sum_{i = 1}^{m} (- L ({\hat{y}}^{(i)}, y^{i})) \\ (3) & = - \sum_{i = 1}^{m} (L ({\hat{y}}^{(i)}, y^{i})) \end{aligned}

$\begin{align} log\;P(\cdots)&=\sum^{m}_{i=1}log(P(y^{(i)}\mid x^{(i)}))\\ &=\sum^{m}_{i=1}(- \mathcal{L}(\hat{y}^{(i)},y^{i}))\\ &=-\sum^{m}_{i=1}( \mathcal{L}(\hat{y}^{(i)},y^{i})) \end{align}$

所以我们定义了成本函数