Law of large numbers and Central limit theorem
大数定律 Law of large numbers (LLN)
虽然名字是 Law，但其实是严格证明过的 Theorem
- weak law of large number (Khinchin's law)
The weak law of large numbers: the sample average converges in probability to the expected value
$\bar{X_n}=\frac{1}{n}(X_1+ \cdots +X_n) \overset{p}{\to} E\{X\} $
- strong law of large number (proved by Kolmogorov in 1930)
The strong law of large numbers: the sample average converges almost surely to the expected value
$\bar{X_n}=\frac{1}{n}(X_1+ \cdots +X_n) \overset{a.s.}{\to} E\{X\} $
https://en.wikipedia.org/wiki/Law_of_large_numbers
https://terrytao.wordpress.com/2008/06/18/the-strong-law-of-large-numbers/
中心极限定理 Central Limit Theorem (CLT)
https://en.wikipedia.org/wiki/Central_limit_theorem
切比雪夫不等式 (Chebyshev's Inequality)
Let $X$ be a random variable with finite expected value $\mu$ and finit non-zero variance $\sigma^2$, then for any real number $k>0$,
$ \mathrm{Pr} \left( \left|X-\mu\right| \geq k \right) \leq \frac{\sigma^2}{k^2}$
马尔科夫不等式 (Markov's inequality)
If X is a nonnegative random variable and a > 0, then the probability that X is at least a is at most the expectation of X divided by a
$ \mathrm{Pr} \left( X \geq a \right) \leq \frac{\mu}{a} $
切尔诺夫限 (Chernoff bound)
The generic Chernoff bound for a random variable X is attained by applying Markov's inequality to etX. For every t > 0:
$ \mathrm{Pr} \left( X \geq a \right)=\mathrm{Pr} \left( e^{tX} \geq e^{ta} \right) \leq \frac{E[e^{tX}]}{e^{ta}} $
