A friendly introduction to compressive sensing
Compressive sensing or compressed sensing is a technique to recover sparse signal from its non-adaptive and insufficient linear measurements. ---------one sentence introduction
General Introduction
At the beginning of the introduction, I give some explanation of the signs used in this document. A signal ( or a discrete signal) is represented as a column vector with finite length. For instance, the following signal
$\boldsymbol{x}=[1,2,3,5,4,6]^{T}$
is a signal with length 6. Vectors are represented by bold lower-case letters and matrices are often represented by bold upper-case letters.
Compressive sensing is a signal recovery technique. The signal that is to be recovered is a sparse signal, where sparse signal is defined as the signal with most of its component being zero value. The measurement euqation is given as:
$\boldsymbol{y}=\boldsymbol{\Phi x}$ (measurement equation)
where $\boldsymbol{x}\in\boldsymbol{R}^{n\times 1}$ is the sparse signal, $\boldsymbol{\Phi}\in\boldsymbol{R}^{m\times n}$ is the measurement matrix and $\boldsymbol{y}\in\boldsymbol{R}^{m\times 1}$ is the measurement signal.
In compressive sensing, $m<n$, which means that the length of the measurement signal is fewer than that of the sparse signal. Therefore, the measurement equation is a indefinite linear equation. In mathematics, indefinite linear equation has infinite number of solutions. One of the commonly encountered solutions is $\hat{\boldsymbol{x}}=\boldsymbol{\Phi}^{\dagger}\boldsymbol{y}$, where $\boldsymbol{\Phi}^{\dagger}$ is the psudo-inverse of matrix $\boldsymbol{\Phi}$, which can be expressed as $\boldsymbol{\Phi}^{\dagger}=\boldsymbol{\Phi}^{T}(\boldsymbol{\Phi\Phi}^{T})^{-1}$.
However, with no prior information about $\boldsymbol{x}$, one can not be confident that the aforementioned solution is the true solution. In compressive sensing, an important assumption is that the signal $\boldsymbol{x}$ is sparse. With this assumption or prior information, signal recovery problem can be formulated as the following optimization problem
$min_{\boldsymbol{x}} \|\boldsymbol{x}\|_{0}$ s.t. $\boldsymbol{y}=\boldsymbol{\Phi x}$ ($l_{0}$ minimization)
Unfortunately, the $l_{0}$ minization problem is a NP hard problem. Directly solving this problem is unpractical.
In compressive sensing, various algorithms are proposed to recover the sparse signal. They can be categorized as follows:
- Heuristic method: greedy algorithms. They include Orthogonal Matching Pursuit (OMP), Subspace Pursuit (SP), etc.
- FOCUSS algorithm, which relax the $l_{0}$ norm in the problem to $l_{p}$ (0<p<1) norm.
- Basis pursuit algorithm, which relax the $l_{0}$ norm in the problem to $l_{1}$ norm.
- Sparse Bayesian learning and Bayesian compressive sensing, which employ the Bayesian inference technique, such as Expectation Minimization (EM) and variational Bayesian inference, to solve the sparse signal recovery problem.
Other problem in compressive sensing including dictionary learning, measurement matrix design, one bit compressive sensing, phase retrieval with sparse signal. The application of compressive sensing is to many to mention.
Generally, solving compressive sensing problem require the knowledge of convex/non-convex optimizaiton and probabbility theory.
Recommended Readings
Personally recommend papers are as follows:
-
- Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm. IEEE Trans. Signal Processing, vol. 45, no. 3, pp: 600-616, 1997.
- Sparse recovery from random measurements via orthogonal matching pursuit. IEEE Trans. Signal Processing, vol. 53, no. 12, pp: 4655-4666, 2007.
- Subspace pursuit for compressive sensing signal reconstruction. IEEE Trans. Information Theory, vol. 55, no. 5, pp: 2230-2248, 2009.
- Bayesian compessive sensing. IEEE Trans. Signal Processing, vol. 56, no. 6, pp:2346-2356, 2008.
- An introduction to compressive sampling. IEEE Signal Processing Magazine, vol.25, no. 2, pp: 21-30, 2008.
- Sparse Bayesian learning for basis selection. IEEE Trans. Signal Processing, vol. 52, no. 8, pp: 2153-2164, 2004
If you want to know about me, please visit my personal page. If anyone have question about compressive sensing, or suggestion about this article, please feel free to leave a message.
