Online ML那点事>-<!

一：译自wiki：

KeyWord：标签反馈;

Survey：

online machine learning is a model of induction that learns one instance at a time. The goal in on-line learning is to predict labels for instances. For example, the instances could describe the current conditions of thestock market, and an online algorithm predicts tomorrow's value of a particular stock. The key defining characteristic of on-line learning is that soon after the prediction is made, the true label of the instance is discovered. This information can then be used to refine the prediction hypothesis used by the algorithm. The goal of the algorithm is to make predictions that are close to the true labels.

在线机器学习一次学习一个模型。其目标是预测实例的标签。比如：实例可以形容为股票市场的现状，在线学习算法预测特定条件下股票明天的值；在线学习的关键特征是在预测被确定不就之后，实例被贴上标签。这种信息可以用来更行算法的参数假设。算法的目标是尽可能的使预测贴近真实标签；（用标签实例来更新模型？只更新模型的少量参数，而不必重新训练整个模型）。

More formally, an online algorithm proceeds in a sequence of trials. Each trial can be decomposed into three steps. First the algorithm receives an instance. Second the algorithm predicts the label of the instance. Third the algorithm receives the true label of the instance.^[1] The third stage is the most crucial as the algorithm can use this label feedback to update its hypothesis for future trials. The goal of the algorithm is to minimize some performance criteria. For example, with stock market prediction the algorithm may attempt to minimize sum of the square distances between the predicted and true value of a stock. Another popular performance criterion is to minimize the number of mistakes when dealing with classification problems. In addition to applications of a sequential nature, online learning algorithms are also relevant in applications with huge amounts of data such that traditional learning approaches that use the entire data set in aggregate are computationally infeasible.

更一般化的说，在线学习算法有一些列方法，每种方法都可分解为以下几步：首先，算法接受一个实例；接着算法预测实例的标签；第三算法接受实例的真实标签（有正确和错误之分，根据结果来调整算法）。第三步比较重要，因为算法根据标签反馈来更新算法参数，~~来更新未来试验预测的假设~~。算法的目的是最小化某些性能标准（？）。例如，在股票市场，算法尝试最小化股票预测和现实股票真实值的偏差（整个模型是动态的）。可以用来处理那些数据量太大而计算能力不能一次性处理整个训练集的情况。（有没有觉得像人的学习过程，一个一个，而不是简单背规则，然后错了就错了）

Because on-line learning algorithms continually receive label feedback, the algorithms are able to adapt and learn in difficult situations. Many online algorithms can give strong guarantees on performance even when the instances are not generated by a distribution. As long as a reasonably good classifier exists, the online algorithm will learn to predict correct labels. This good classifier must come from a previously determined set that depends on the algorithm. For example, two popular on-line algorithmsperceptron and winnow can perform well when a hyperplane exists that splits the data into two categories. These algorithms can even be modified to do provably well even if the hyperplane is allowed to infrequently change during the on-line learning trials.

因为算法不断的接受标签反馈,算法可以适应困难条件下的学习。许多在线学习算法可以在实例由于干扰而没有被生成的时也有良好表现。一旦良好的分类器被建立，算法感知器可以预测正确标签。算法必须依赖于已存的良好算法。比如现存的两个流行算法感知器和winnow在一个二分类超平面存在时可以表现良好。这些算法甚至可以在分裂反馈之后被修改。

Unfortunately, the main difficulty of on-line learning is also a result of the requirement for continual label feedback. For many problems it is not possible to guarantee that accurate label feedback will be available in the near future. For example, when designing a system that learns how to do optical character recognition, typically some expert must label previous instances to help train the algorithm. In actual use of the OCR application, the expert is no longer available and no inexpensive outside source of accurate labels is available. Fortunately, there is a large class of problems where label feedback is always available. For any problem that consists of predicting the future, an on-line learning algorithm just needs to wait for the label to become available. This is true in our previous example of stock market prediction and many other problems.

在线学习的主要困难是不断的接受标签反馈。对于很多问题，它不能保证精确的标签反馈在不就之后被获得，当设计一个光学字符识别系统，一些专家必须保提前标签一些实例训练算法。标签的获得是困难的，也许在将来，我们的在线学习算法只需要等待足量的标签。

一个典型的在线监督学习算法：A prototypical online supervised learning algorithm

In the setting of supervised learning, or learning from examples, we are interested in learning a function $f : X \to Y$ , where $X$ is thought of as a space of inputs and $Y$ as a space of outputs, that predicts well on instances that are drawn from a joint probability distribution $p(x,y)$ on $X \times Y$ . In this setting, we are given aloss function $V : Y \times Y \to \mathbb{R}$ , such that $V(f(x), y)$ measures the difference between the predicted value $f(x)$ and the true value $y$ . The ideal goal is to select a function $f \in \mathcal{H}$ , where $\mathcal{H}$ is a space of functions called a hypothesis space, so as to minimize the expected risk:

$I[f] = \mathbb{E}[V(f(x), y)] = \int V(f(x), y)\,dp(x, y) \ .$

In reality, the learner never knows the true distribution $p(x,y)$ over instances. Instead, the learner usually has access to a training set of examples $(x_1, y_1), \ldots, (x_n, y_n)$ that are assumed to have been drawni.i.d. from the true distribution $p(x,y)$ . A common paradigm in this situation is to estimate a function $\hat{f}$ throughempirical risk minimization or regularized empirical risk minimization (usuallyTikhonov regularization). The choice of loss function here gives rise to several well-known learning algorithms such as regularizedleast squares andsupport vector machines.

The above paradigm is not well-suited to the online learning setting though, as it requires complete a priori knowledge of the entire training set. In the pure online learning approach, the learning algorithm should update a sequence of functions $f_1, f_2, \ldots$ in a way such that the function $f_{t+1}$ depends only on the previous function $f_t$ and the next data point $(x_t, y_t)$ . This approach has low memory requirements in the sense that it only requires storage of a representation of the current function $f_t$ and the next data point $(x_t, y_t)$ . A related approach that has larger memory requirements allows $f_{t+1}$ to depend on $f_t$ and all previous data points $(x_1, y_1), \ldots, (x_t, y_t)$ . We focus solely on the former approach here, and we consider both the case where the data is coming from an infinite stream $(x_1, y_1), (x_2, y_2), \ldots$ and the case where the data is coming from a finite training set $(x_1, y_1), \ldots, (x_n, y_n)$ , in which case the online learning algorithm may make multiple passes through the data.