Machine Learning No.4: Regularization
1. Underfit = High bias
Overfit = High varience
2. Addressing overfitting:
(1) reduce number of features.
Manually select which features to keep.
Model selection algorithm
disadvantage: throw out some useful information
(2) Regularization
Keep all the features, but reduce magnitude/values of parameters θj
works well when we have a lot of features, each of which contributλes a bit to predicting y.
3. Regularization
if λ is extremely large, , then J(θ) will be underfitting
4. Gradient desent
Repeat {
(j = 1, 2 ... n)
}
5. Normal equation
if λ > 0
if m <= n
is non-invertible/singular
but using regularization will avoid this problem