Simple Error Code Generation

The need of error code generation is growing in the field of communication. Both the imitation of real situation and the feasibility of the implementation are in the consideration.

The simple Markov model is just one step from the plain one (uniform distribution noise).

The model can be described as:
1. If the last bit is correct, then the probability that the current bit is error is p01;
2. If the last bit is error, then the probability that the current bit is error is p11.
3. For the first bit,  its proability of error is p0.

Apparently, it is a Markov model with transit matrix:
A = [1-p01 1-p11]
    [  p01   p11]

Assume the probability of error for each specific index point k is:
p[k] = [p0[k], p1[k]]'

We have:
[p0[k+1]] = [1-p01 1-p11][p0[k]]
[p0[k+1]]   [  p01   p11][p1[k]]

i.e.

p[k+1] = A * p[k], and p[1] = [p00, 1-p00]'

Here, we are interested in the error rate (overall probability of error) this generation model leads to, that is the total expected occurrence of error against the total number of bits.

We assume the mathematical expectation of number of error bits at each specific index k is:
G[k]

Then the overall error rate for k is E[k] = G[k]/k

We use 'v(k)' to denote the existing bit error pattern of length k, which is a k-dimensional vector of binary value, each component of which is a bit (0 or 1) used to determine whether to make an error on the bit of the corresponding index.

Then v(k+1) is either [v(k),0] or [v(k),1].

So the probability of each case is:
P{v(k+1) = [v(k),0]} = P{v(k)} * ( P{v(k)[k]==0 | v(k)} * (1 - p01) + P{v(k)[k]==1 | v(k)} * (1 - p11) )

Hence,
G[k] = sum(all v(k), nnz(v(k)) * P{v(k)}), where nnz(v(k)) gives the number of ones in v(k)
We can deduce G[k+1] from G[k]
G[k+1] = sum(all v(k), n([v(k),0])*P{[v(k),0]} + n([v(k),1])*P{[v(k),1]})
= sum(all v(k), n(v(k))*P{[v(k),0]} + (n(v(k))+1)*P{[v(k),1]})
= sum(all v(k), n(v(k))*P{v(k)} + P{[v(k),1]})
= G[k] + sum(all v(k), P{[v(k),1]})

sum(all v(k), P{[v(k),1]}) = sum( all v(k), P{v(k)[k]==0}*p01 + P{v(k)[k]==1}*p11 )
= sum(all v(k), P{v(k)[k]==0}) * p01 + sum(all v(k), P{v(k)[k]==1}) * p11
= p[k][0] * p01 + p[k][1] * p11

According to p[k+1] = A * p[k], and p[1] = [p00, 1-p00]'
p[k] = A^n-1*p[1]

Decompose A into:
A = S*C*S^-1, where C is the matrix with eigenvalues of A on its diagonal,  S is composed of eigenvectors.

Therefore:
C = [1   0]
    [0 b-a]

S = [b-1  1]
    [ -a -1]

S^-1
= [-1  -1] / (a-b+1)
  [ a b-1]

So
p[k]
= [ a*(b-a)^k+1-b] / (a-b+1)
  [-a*(b-a)^k+a  ]

and
          a            (b-a) * (1 - (b-a)^k)
G[k] = ------- * (k - -----------------------)
        a-b+1                1 - (b-a)

           G[k]      a            (b-a) * (1 - (b-a)^k)
err_rate = ---- = ------- * (1 - -----------------------)
            k      a-b+1              k * (1 - (b-a))

                      G[k]      a
inf_err_rate =   lim  ---- = -------
               k->Inf  k      a-b+1

This result can also be given by y(1) when y = A * y.

From this simple example, we can see two of the most important branches of applied mathematics for communication and information theory are linear algebra and probability theory.