2.9 Model Selection and the Bias–Variance Tradeoff
结论
-
模型复杂度↑Bias↓Variance↓
例子
-
$y_i=f(x_i)+\epsilon_i,E(\epsilon_i)=0,Var(\epsilon_i)=\sigma^2$
使用knn做预测,在点$x_0$处的Excepted prediction error:
$EPE(x_0)=E\left[\left(y_0-\hat{f}(x_0)\right)^2|x_0\right]\\ \ \ =E\left[\left(y_0-E(y_0)\right)^2|x_0\right]+\left[E(\hat{f}(x_0))-E(y_0)|x_0\right]^2+E\left[\hat{f}(x_0)-E(\hat{f}(x_0))\right]^2\\ \ \ =\sigma^2+{Bias}^2(\hat{f}(x_0))+Var(\hat{f}(x_0))\\ \ \ =\sigma^2+\left[f(x_0)-\frac{1}{k}\sum_l^k y_l\right]^2+\frac{\sigma^2}{k}$
k↑在训练集上的表现(考虑k=1,k=2)↓模型复杂度(模型越复杂,在训练集上的表现越好)↓