Boosting理解
俗话说的好，三个臭皮匠顶个诸葛亮。Boosting大概就是做这种事情，也可以理解是一个很牛的企业管理者，充分的利用底下员工的各种优点，合理发挥每个人的能量。
简单的理解boosting就是把很多个不同的classifiers使用不同的权重组合起来成一个新的strong classifier进行分类工作。
把我的学习笔记记录一下吧，以供以后使用这个算法的时候参考。
How to obtain the rough rule of thumb?
How to combine this rule of thumbs to get a good thumb?
What is the procedure to choose examples in each of rounds?
a)What is the procedure to choose examples in each of rounds?
We'll focus on the hardest examples so that we will choose the examples that the previous thumb misclassified.
b)How to combine this rule of thumbs to get a good thumb?
To take a majority vote or weighted majority vote.
Change a subset of examples to construct distribution D_t
That's the same thing.
The weight of D_t stands for how we concentrate on a particular round of boosting.
Combine all of h_t into a H_final classifier.
Q:
1.how do you construct the distribution D_t
2.how do we combine all of h_t into a H_final classifier?
AdaBoosting can solve this two questions.
A:
1.At the very first round, we don't have any information. We use uniform distribution weight.
At the following rounds, we try to focus on incorrectly classified examples. We cut the weight of the examples that are correctly classified.
2.
This formular is the weight vote of weak thumbs.
There's ten samples and three samples are misclassified.
So epsilon_1 = 3 / 10 = 0.30.
The alpha_t may bigger if one classifier has less error rate.
The classifier is a black box, so that you can choose whatever method that sitting around.
最后附上我自己使用Python实现的adaboosting的一个package或者说class供参考。如有错误请多不吝指正。
https://gitcafe.com/NeighborhoodGuo/Ada_boosting.git
Reference:
2.Boosting resources collected by Lyon
