2.9 Model Selection and the Bias–Variance Tradeoff

结论

  • 模型复杂度↑Bias↓Variance↓

例子

  • $y_i=f(x_i)+\epsilon_i,E(\epsilon_i)=0,Var(\epsilon_i)=\sigma^2$
    使用knn做预测,在点$x_0$处的Excepted prediction error:
    $EPE(x_0)=E\left[\left(y_0-\hat{f}(x_0)\right)^2|x_0\right]\\ \ \ =E\left[\left(y_0-E(y_0)\right)^2|x_0\right]+\left[E(\hat{f}(x_0))-E(y_0)|x_0\right]^2+E\left[\hat{f}(x_0)-E(\hat{f}(x_0))\right]^2\\ \ \ =\sigma^2+{Bias}^2(\hat{f}(x_0))+Var(\hat{f}(x_0))\\ \ \ =\sigma^2+\left[f(x_0)-\frac{1}{k}\sum_l^k y_l\right]^2+\frac{\sigma^2}{k}$
    k↑在训练集上的表现(考虑k=1,k=2)↓模型复杂度(模型越复杂,在训练集上的表现越好)↓

posted @ 2015-08-13 23:55  porco  阅读(293)  评论(0编辑  收藏  举报