Probability Concepts

Unconditional probability and Conditional Probability

Unconditional Probability (a.k.a. marginal probability): refer to the probability off an event regardless of the past or future occurrence of other events.
Conditional Probability: refer to one where the occurrence of one event affects the probability of the occurrence of another event.

Probability Rules

P(AB) = P(A|B)*P(B)
P(A or B) = P(A) + P(B) - P(AB)
P(A) = P(A|B1)P(B1) + P(A|B2)P(B2) + ... + P(A|Bn)P(Bn) (B1..Bn) is a mutually exclusive and exhaustive set of outcomes.

Expected Value

The expected value is the weighted average of the possible outcomes of a random variable, where the weights are the probabilities that the outcomes will occur.

The degree of disperation of outcomes around the expected value off a random variable is measured using the variance and standard deviation. When pairs of random variables are being observed, the covariance and correlation are used to measure the extent of the relationship between the observed values for the two variables from one observation to the next.

Variance

The variance is calculated as the probability-weighted sum of the squared differences between each possible outcome and expected value.

Var(R)=E([R-E(R)]^2) = w1(R1-E1)^2 + w2(R2-E2)^2 + ... + wn(Rn-En)^2

Note: 普通统计中的方差是平方和直接除以N，这里面假设每个值出现的概率（权重）是一样的，所以除以N就可以了。而且平方和中是每个值减去平均值，而不是期望。同样是就假设每个值的权重是一样的。

Covariance and Correlation

The variance and standard deviation measure the disperation, or valatility, of only one variable. In many finance situations, however, we are interested in how two random variables move in relation to each other.

Covariance is a measure of how two assets move together. It is the expected value of the product of the deviations of two random variables from their respective expected values.

Cov(Ri,Rj) = E{[Ri-E(Ri)][Rj-E(Rj)]}

Note 1: The variance measures how a random variable moves with itself, and the covariance measures how one random variable move with another random variable.

Note 2: The covariance may range from negative infinity to positive infinity

Correlation Coefficient, or simply, correlation is calculate by the covariance of two random variables divided by the product of the random variable's standard deviations.

Corr(Ri, Rj)=Cov(Ri, Rj)/[σ(Ri)σ(Rj)]

Note 1: Correlation measures the strength of the linear relationship between two random variables.
Note 2: Correlation has no units.
Note 3: The correlation ranges from -1 to 1.
Note 4: If Corr(Ri, Rj)=1.0, the random variables have perfect positive correlation.
Note 5: If Corr(Ri, Rj)=-1.0, the random variables have perfect negative correlation.
Note 5: If Corr(Ri, Rj)=0, there is no linear relationship between the variables, indicating that prediction of Ri cannot be made on the basis of Rj using linear method.