SciTech-BigDataAIML-Python Time Series Handbook - Kalman filter:( Optimal Recursive Data Processing Algorithm): 卡尔曼滤波器算法(也称“最优的递归数据处理算法”)

Kalman Filter(卡尔曼滤波器算法)

SciTech-BigDataAIML-Python Time Series Handbook
Kalman filter is also known as:
Optimal Recursive Data Processing Algorithm.
最优的递归数据处理算法

网上文档:

• Python时间序列手册: 有ipynb和PDF文件:
  https://filippomb.github.io/python-time-series-handbook/notebooks/07/kalman-filter.html

• MIT PDF:An Introduction to the Kalman Filter - MIT

• Illinois University PDF: Understanding the Basis of the Kalman Filter Via a …

• PDF: Standford University: Lecture 8 The Kalman filter - Stanford University

• ArXiv:
  PDF: An Elementary Introduction to Kalman Filtering - arXiv.org
  PDF: A Step by Step Mathematical Derivation and Tutorial on …

Overview

This lecture will cover the following topics:

• Introduction to the Kalman Filter.
• Model components and assumptions.
• The Kalman Filter algorithm.
• Application to static and dynamic one-dimensional data.
• Application to higher-dimensional data.

What is a Kalman Filter?

• The Kalman Filter (KF) is an algorithm that uses a series of measurements observed over time, containing statistical noise and other inaccuracies.
• It produces estimates of unknown variables that tend to be more accurate than those based only on measurements.
• Developed by Rudolf E. Kálmán in the late 1950s.
• Applications:
  tracking objects (Auto EV, GPS, self driving cars).
  image processing.
  economic and financial modeling.

Theoretical foundations

• Dynamic systems are systems that change over time.
• They are described by a set of equations that predict the future state of the system based on its current state.
• The KF assumes the system is a linear dynamic system with Gaussian noise.
• Gaussian distributions are used because of their nice properties when dealing with averages and variances.

What are the ingredients?

State

• The true value of the variables we want to estimate.
• The state vector represents all the information needed to describe the current state of the system.

Observation model

*Relates the current state to the measurements or observations.

Noisy measurements

The process noise and measurement noise (assumed to be Gaussian) represent the uncertainty in our models and measurements.

Example:

• Car moving at a constant velocity.
• Model: Car position = velocity × time + system noise.
• Measurements: position and velocity (which are also noisy).

The role of noise

• The tradeoff between the influence of the model and the measurements is determined by noise.
• If the model has relatively large errors, more importance is given to the latest measurements in computing the current estimate.
• If the measurements have larger errors, more importance is given to the model in making the current estimate.
• Therefore, you need to estimate not only your state but also the errors

Update strategy

• KF is a recursive algorithm:

  • Uses information from previous time step to update the estimates.
  • Does not keep in memory all the data acquired so far.

• Has predictor-corrector structure:

• Make a prediction based on the model.

• Update the prediction with the measurements.

• Repeat.

Fundamental assumptions

The Kalman filter approach is based on three fundamental assumptions:

• The system can be described or approximated by a linear model.
• All noise (from both the system and the measurements) is white, i.e., the values are not correlated.

All noise is Gaussian.

Assumption 1: linearity

• Each variable at the current time is a linear function of the variables at previous times.
• Many systems can be approximated this way.
• Linear systems are easy to analyze.
  Nonlinear systems can often be approximated by linear models around a current estimate (extended KF).

Assumption 2: Whiteness

• The noise values are not correlated in time.
  If you know the noise at time, it doesn't help you to predict the noise at future times.
• White noise is a reasonable approximation of the real noise.
  The assumption makes the mathematics tractable.
  📝 Note White noise contains a mix of all the different frequencies at the same intensity blended together.
  This is similar to how white light contains all the colors of the rainbow combined.

Assumption 3: Gaussian noise

At any point in time, the probability density of the noise is a Gaussian.
System and measurement noise are often a combination of many small sources of noise.
The combination effect is approximately Gaussian.
If only mean and variance are known (typical case in engineering systems), Gaussian distribution is a good choice as these two quantities completely determine the Gaussian distribution.
Gaussian distribution have nice properties and are easy to treat mathematically.

Technical details

Combining two sources

Assume a car has a certain initial position and initial velocity.
If the speed is constant, we have: