Forward Algorithm
The goal of the forward algorithm is to compute the joint probability ,
where for notational convenience we have abbreviated
as
and
as
.
Computing
directly would require marginalizing over
all possible state sequences
,
the number of which grows exponentially with
.
Instead, the forward algorithm takes advantage of the conditional independence rules
of the hidden Markov model (HMM) to perform the calculation recursively.
To demonstrate the recursion, let
-
-
.
-
Using the chain rule to expand ,
we can then write
-
-
.
-
Because is conditionally independent of
everything but
, and
is
conditionally independent of everything but
,
this simplifies to
-
-
.
-
Thus, since and
are
given by the model's emission distributions and transition
probabilities, one can quickly calculate
from
and
avoid incurring exponential computation time.
The forward algorithm is easily modified to account for observations from variants of the hidden Markov model as well, such as the Markov jump linear system.