Poisson Process

1. Counting process

​ We say that \(\{N(t),t\ge0\}\) is a counting process if \(N(t)\) represents the total number of "events" occur by time \(t\) and satisfies the following:

1. \(N(t)\ge0\)
2. \(N(t)\in\N^+\)
3. \(N(s)\le N(t)\) if \(s\le t\)
4. for \(s<t\), \(N(t)-N(s)\) equals the number of events that occur in the interval \((s,t]\).

​ A counting process is said to possess independent increments if the numbers of events that occur in disjoint time intervals are independent.

2. Poisson process

1st definition

​ The counting process \(\{N(t), t\ge0\}\) is said to be a Poisson process having rate \(\lambda,\lambda>0\), if

1. \(N(0)=0\)

2. The process has independent increments

3. The number of events in any interval of length \(t\) is Poisson distributed with mean \(\lambda t\). Which means, for any \(s, t\ge0\)

   \[P\left\{N(s+t)-N(s)=n\right\} = \displaystyle{\frac{(\lambda t)^ne^{-\lambda t}}{n!}},\quad n=0,1,... \]

2nd definition

​ The counting process \(\{N(t),t\ge0\}\) is said to be a Poisson process having rate \(\lambda,\lambda >0\) if

1. \(N(0)=0\)
2. The process has stationary and independent increments
3. \(P\{N(h)=1\}=\lambda h+o(h)\)
4. \(P\{N(h)\ge2\}=o(h)\)

Proof of the equivalence of above two definitions

​ First we prove 2nd definition \(\Longrightarrow\) 1st definition. To start, fix \(u\ge0\) and let

\[g(t) = E[\exp\{-uN(t)\}] \]

​ We derive a differential equation of \(g(t)\) as follows:

\[\begin{align} g(t+h) &= E\left[\exp\{-uN(t+h)\}\right]\\ &=E[\exp\{-u(N(t+h)-N(t))\}\cdot \exp\{-uN(t)\}]\\ &=g(t)E(\exp\{-u(N(t+h)-N(t))\})\\ &=g(t)E(\exp\{N(h)\}) \end{align} \]

By the 3rd and 4th condition of the 2nd definition, we can yield

\[E[\exp\{-uN(h)\}]=1-\lambda h+e^{-u}\lambda h+o(h)\\ g(t+h)=g(t)(1-\lambda h+e^{-u}\lambda h) \]

which implies that

\[\frac{g(t+h)-g(t)}{h} = \lambda(e^{-u}-1)g(t)+\frac{o(h)}{h}\\ \lim_{h\rarr0}\frac{g(t+h)-g(t)}{h}=\lambda(e^{-u}-1)g(t)\\ g'(t) = \lambda(e^{-u}-1)g(t) \]

Solve this differential equation we can get that

\[g(t) = C\exp\{\lambda(e^{-u}-1)t\} \]

That is, the Laplace transform of \(N(t)\) evaluated at \(u\) is \(e^{\lambda t(e^{-u}-1)}\). Since that is also the Laplace transform of a Poisson random variable with mean \(\lambda t\), the result follows from the fact that the distribution of a nonnegative random variable is uniquely determined by its Laplace transform.

3. Interarrival and Waiting Time Distributions

​ Denote the time of the first event by \(T_1\), denote the elapsed time between the \((n-1)\)st event and the \(n\)th event by \(T_n\) for \(n>1\).

​ Now we determine the distribution of \(T_n\), note that

\[\begin{align} P(T_n\le t) &= 1-P(T_n>t)\\ &= 1-P\{N(t)\le n-1\} \end{align} \]

The event \(\{T_1>t\}\) takes place if and only if no events of the Poisson process occur in the interval \([0,t]\) and thus,

\[P(T_1>t) = P(N(t) =0)=e^{-\lambda t} \]

Hence, \(T_1\) has an exponential distribution with mean \(1/\lambda\). Now,

\[P(T_2>t) = E[P(T_2>t\ |\ T_1)] \]

However,

\[\begin{align} P(T_2>t\ | T_1=s) &= P\{N(s+t)-N(t)=0\ |\ T_1=s\}\\ &=P\{N(s+t)-N(t)=0\}\\ &=e^{-\lambda t} \end{align} \]

where the last two equations followed from independent and stationary increments. Therefore, we conclude that \(T_2\) is also an exponential random variable with mean \(1/\lambda\) and, furthermore, that \(T_2\) is independent of \(T_1\). Repeating the same argument yields the following.

Proposition 3.1 \(T_n=1,2,...\) are independent identically distributed exponential random variables having mean \(1/\lambda\).

​ The arrival time of the \(n\)th event is called the waiting time until the \(n\)th event.

\[S_n = \sum_{i=1}^nT_i,\quad n\ge1 \]

\(S_n\) has a gamma distribution with parameters \(n\) and \(\lambda\), the probability of \(S_n\) is given by

\[fs_n(t)=\lambda e^{-\lambda t}\frac{(\lambda t)^{n-1}}{(n-1)!},\quad t\ge0 \]

4. Further Properties of Poisson Process

​ Suppose each event occur in a Poisson process is classified as either type I or type II. Suppose further that each event is classified as a type I event with probability \(p\) and as a type II event with probability \(1-p\).

​ Let \(N_1(t)\) and \(N_2(t)\) denote respectively the number of type I and type II event occurring in \([0,t]\).

Proposition 4.1 \(\{N_1(t),t\ge0\}\) and \(\{N_2(t),t\ge0\}\) are both Poisson processes having respective rate \(\lambda p\) and \(\lambda(1-p)\). Furthermore, two processes are independent.

​ Now we prove \(N_1(t)\) is Poisson process:

• \[\begin{align} P\{N_1(h)=1\} &= P\{N_1(h)=1\ |\ N(h)=1 \}P\{N(h)=1\}\\ &+P\{N_1(h)=1\ |\ N(h)\ge 2 \}P\{N(h)\ge2\}\\ &= p\lambda h+o(h) \end{align} \]

• \(P\{N_1(h)\ge2\}\le P\{N(h)\ge2\}\)

Thus we can see that \(N_1(t)\) satisfies the 2nd definition of Poisson Process with rate \(\lambda p\).

---

The Coupon Collecting Problem

​ Problem There are \(m\) different types coupons. Each time a person collects a coupon it is, independently of ones previously obtained, a type \(j\) coupon with probability \(p_j\), \(\sum_{j=1}^mp_j=1\). Let \(N\) denote the number of coupons one needs to collect in order to have a complete collection of at least of one of each type. Find \(E[N]\).

​ Solution Let \(N_j\) be the number we must collect to obtain a type \(j\) coupon, then we can express \(N\) as

\[N = \max_{1\le j\le m}N_j \]

let \(X=\max_{1\le j\le m}X_j\) denote the time at which a complete collection is amassed. The waiting time of each type is independent and is exponentially distributed with parameter \(p_j\)

\[P(X<t) = \prod_{j=1}^m(1-e^{-p_jt}) \]

Therefore,

\[\begin{align} E[X] &= \int_0^\infin P(X>t)\ \mathrm dt\\ &=\int_0^\infin\left\{1-\prod_{j=1}^m(1-e^{-p_jt}) \right\}\ \mathrm dt \end{align} \]

Let \(T_i\) denote the \(i\)th interarrival time of the Poisson process that counts the number of coupons obtained.

\[X=\sum_{i=1}^NT_i\\ E[X|N] = NE[T_i] = N\\ \]

Therefore

\[E[X] = E[N] \]

---

5. Conditional Distribution of the Arrival Times

​ Suppose we are told that exactly one event of a Poisson process has taken place by time \(t\), and we are asked to determine the distribution of the time at which the event occurred. For \(s\le t\)

\[\begin{align} P\{T_1<s|N(t)=1\} &= \frac{P\{T_1<s,N(t)=1\}}{P\{N(t)=1\}}\\ &=\frac{P\{N(s)=1,N(t)-N(s)=0\}}{P\{N(t)=1\}}\\ &=\frac{\lambda se^{-\lambda s}e^{-\lambda (t-s)}}{\lambda te^{-\lambda t}}\\ &=\frac st \end{align} \]

​ Theorem 5.1 Given that \(N(t)=n\), the \(n\) arrival times \(S_1,S_2,...,S_n\) have the same distribution as the order statistic corresponding to \(n\) independent random variables uniformly distributed on the interval \((0,t)\).

​ Definition 5.1 We say the \(Y_{(1)},...,Y_{(n)}\) are the order statistics corresponding to \(n\) random variables \(Y_1,...,Y_n\) if \(Y_{(k)}\) is \(k\)th smallest value among \(Y_1,...,Y_m\). If the \(Y_i,i=1,...,n\) are independent identically distributed continuous random variables with density function \(f\), the joint density of order statistics \(Y_{(1)},...,Y_{(m)}\) is given by

\[f(y_1,...,y_n) = n!\prod_{i=1}^nf(y_i), \quad y_1<y_2<\cdots<y_n \]

​ Proof of Theorem 5.1 Note that the event that \(S_1=s_1,S_2=s_2,...,S_n=s_n,N(t)=n\) is equivalent to the event that the first \(n+1\) interarrival times satisfy \(T_1=s_1,T_2=s_2-s_1,...,T_n=s_n-s_{n-1},T_{n+1}>t-s_n\). According to Proposition 3.1, we have the conditional join density of \(S_1,...,S_n\) given that \(N(t)=n\) is as follows:

\[\begin{align} f(s_1,...,s_n|n)&=\frac{f(s_1,...,s_n,n)}{P\{N(t)=n\}}\\ &=\frac {\lambda e^{-\lambda s_1}\cdots\lambda e^{-\lambda (s_n-s_{n-1})}\lambda e^{-\lambda(t-s_n)}} {e^{-\lambda t}(\lambda t)^n/n!}\\ &= \frac{n!}{t^n}\quad 0<s_1<\cdots<s_n<t \end{align} \]

​ Proposition 5.1 Suppose each time a Poisson event is classified into \(k\) types of events. Every type of event occurred at time \(t\), is independent of anything that has previously occurred, with probability \(P_i(t),i=1,...,k\). If \(N_i(t),i=1,...,k\) represents the number of type \(i\) events occurring by time \(t\) then \(N_i(t), i=1,...,k\) are independent Poisson random variables having means

\[E[N_i(t)]=\lambda\int_0^tP_i(s)\mathrm ds \]

​ Proof Now consider an arbitrary event that occurred in the interval \([0,t]\). If it had occurred at time \(s\), then the probability that it would be a type \(i\) event would be \(P_i(s)\). According to Theorem 5.1, the probability that this event will be a type \(i\) event is

\[P_i = \frac1t\int_0^tP_i(s)\mathrm ds \]

which is independent of the other events. Hence the joint probability

\[P\left\{N_i(t)=n_i,i=1,...,k\ \left|\ N(t)=\sum_{i=1}^kn_i \right.\right\}=\frac{(\sum_{i=1}^kn_i)!}{n_1!\cdots n_k!}P_1^{n_1}\cdots P_k^{n_k} \]

Consequently,

\[\begin{align} P\{N_i(t)=n_i,i=1,...,k\} &= \frac{(\sum_in_i)!}{n_1!\cdots n_k!}P_1^{n_1}\cdots P_k^{n_k} \left(e^{-\lambda t}\frac{(\lambda t)^{\sum_in_i}}{\left(\sum_in_i\right)!}\right)\\ &= \prod_{i=1}^k e^{-\lambda tP_i}\frac{(\lambda tP_i))^{n_i}}{n_i!} \end{align} \]

As said above, \(N_i(t), i=1,...,k\) are independent, so random variable \(N_i(t)\) with probability function

\[P\{N_i(t)=n_i\} =e^{-\lambda tP_i}\frac{(\lambda tP_i))^{n_i}}{n_i!} \]

satisfies the Poisson distribution with rate \(\lambda P_i\), therefore

\[E[N_i(t)] = \lambda tP_i = \lambda \int_0^tP_i(s)\mathrm ds \]

and the proof is complete.

​ Proposition 5.2 Given that \(S_n=t\), the set \(S_1,...,S_{n-1}\) has the distribution of a set of \(n-1\) independent uniform \((0,t)\) random variables.

6. Generalizations of the Poisson Process

6.1 Nonhomogeneous Poisson Process

​ Definition 6.1 The counting process \(\{N(t),t\ge0\}\) is said to be nonhomogeneous Poisson process with intensity function \(\lambda(t),t\ge0\). if

1. \(N(0)=0\)
2. \(\{N(t),t\ge0\}\) has independent increments
3. \(P\{N(t+h)-N(t)\ge2\}=o(h)\)
4. \(P\{N(t+h)-N(t)=1\}=\lambda h+o(h)\)

​ Proposition 6.1 Let \(\{N(t),t\ge0\}\) and \(\{M(t),t\ge0\}\), be independent nonhomogeneous Poisson processes, with respective intensity functions \(\lambda(t)\) and \(\mu(t)\), and let \(N^*(t)=N(t)+M(t)\). Then, the following are true.

1. \(\{N^*(t),t\ge0\}\) is a nonhomogeneous Poisson process with intensity function \(\lambda(t)+\mu(t)\).
2. Given that an event of \(\{N^*(t),t\ge0\}\) process occurs at time \(t\) then, independent of what occurred prior to \(t\), the event at \(t\) from the \(\{N(t)\}\) process with probability \(\displaystyle{\frac{\lambda(t)}{\lambda(t)+\mu(t)}}\).

6.2 Compound Poisson Process

​ Definition 6.2 A stochastic process \(\{X(t),t\ge0\}\) is said to be a compound Poisson process if it can be expressed as

\[X(t)=\sum_{i=1}^{N(t)}Y_i,\quad t\ge0 \]

where \(\{N(t),t\ge0\}\) is a Poisson process, and \(\{Y_i,i\ge1\}\) is a family of independent and identically distributed random variables that is also independent of \(\{N(t),t\ge0\}\).

​ We have

\[\begin{align} E[X(t)] &= E\left[\sum_{i=1}^{N(t)}Y_i\right]\\ &= \sum_{n=1}^\infin E\left[\sum_{i=1}^nY_i\left|N(t)=n\right.\right]P\{N(t)=n\}\\ &=\sum_{n=1}^\infin \left(E\left[\sum_{i=1}^nY_i\right]P\{N(t)=n\}\right)\\ &= \sum_{n=1}^\infin\left(nE[Y]P\{N(t)=n\}\right) \\ &= E[Y]\sum_{n=1}^\infin\left(nP\{N(t)=n\}\right) \\ &=\lambda tE[Y] \end{align} \]

Similarly,

\[\mathrm{Var}(X(t)) = \lambda tE[Y^2] \]

6.3 Conditional or Mixed Poisson Processes

​ Let \(\{N(t),t\ge0\}\) be a counting process whose probabilities are defined as follows. There is a positive random variable \(L\) such that, conditional on \(L=\lambda\), the counting process is a Poisson process with rate \(\lambda\). Such a counting process is called a conditional or a mixed Poisson process.

​ Suppose that \(L\) is continuous with density function \(g\). Then

\[\begin{align} P\{N(t+s)-N(s)=n\} &= \int_0^\infin P\left\{N(t+s)-N(s)=n\ |\ L=\lambda \right\}g(\lambda)\ \mathrm d\lambda\\ &=\int_0^\infin e^{-\lambda t}\frac{(\lambda t)^n}{n!}g(\lambda)\ \mathrm d\lambda \end{align} \]

we see that a conditional Poisson process has a stationary increments. However, knowing how many events occur in an interval gives information about the possible value of \(L\), which affects the distribution of the number of events in any other interval, it follows that a conditional Poisson process does not generally have independent increments. Consequently, a conditional Poisson is not generally a Poisson process.