正则化（尚未完成）

正则化策略

Since because of the function of being a potential tool for ensuring the generalization of the algorithm, studies on regularization of the algorithm becomes the main research topic in machine learning. Moreover, the regularization becomes very crucial step in the deep learning model that has more parameters than the training data sets. Regularization is a technique to avoids the overfitting of the algorithm and to avoids the overfitting of coefficients to fit so perfectly as model complexity increases. Overfitting often occurs when the algorithm learns the input data along with noises. Over the past few years, verity of methods are proposed and developed for the machine learning algorithm to regularize such as data argumentation, L2regularization or weight decay, L1regularization, dropout, drop connect, stochastic pooling and early stopping.

1. 数据增强

2. \(L_1\) 和 \(L_2\) 正则化

3. Dropout

4. DropConnect

5. Early stopping

---

参考文献：
【1】 机器学习中的范数规则化之（一）L0、L1与L2范数
【2】机器学习中的范数规则化之（二）核范数与规则项参数选择
【3】论文：Regularization and Optimization strategies in Deep Convolutional Neural Network