Informally, we dene the bias of a model to be the expected generalization error even if we were to fit it to a very (say, innitely) large training set. Thus, for the problem above, the linear model suers from large bias, and may undert (i.e., fail to capture structure exhibited by) the data.
Apart from bias, there's a second component to the generalization error, consisting of the variance of a model tting procedure. Specically, when
tting a 5th order polynomial as in the rightmost gure, there is a large risk that we're tting patterns in the data that happened to be present in our
small, nite training set, but that do not re ect the wider pattern of the relationship between x and y. This could be, say, because in the training set
we just happened by chance to get a slightly more-expensive-than-average house here, and a slightly less-expensive-than-average house there, and so
on. By tting these \spurious" patterns in the training set, we might again obtain a model with large generalization error. In this case, we say the model
has large variance.
估计是一个随机变量。 bias是估计的期望和ground truth的差别。 variance是估计本身的方差。
模型是指随机数据的概率分布，通过训练集学出来的也是一个模型，grouth trurh也是一个模型。
bias描述的是两个模型之间的差距，因为都是分布，所以才比期望。
