6. Eigenvalues and Eigenvectors
Keys:
- What are Eigenvalues and Eigenvectors?
- How to find Eigenvalues and Eigenvectors?
- Applications of Egenvalues and Eigenvectors:
- Difference equation \(u_{k+1}=Au_k\)
- Solution of \(\frac{du}{dt}=Au\)
- Markov Matrices
- Projections and Fourier Series
- Special Matrix
- Symmetric Matrices
- Positive Definite Matrix
- Similar Matrices
- Jordan Theorem
6.1 Introduction to Eigenvalues and Eigenvectors
keys:
- If X lies along the same direction as AX : \(AX = \lambda X\)，then \(\lambda\) is eigenvalue and X is eigenvector.
- If \(AX=\lambda X\) then \(A^2X=\lambda^2 X\) and \(A^{-1}X=\lambda^{-1} X\) and \((A+cI)X=(\lambda + c) X\) : the same eigenvector X.
- If \(AX=\lambda X\) then \((A-\lambda I)X=0\) and \(A-\lambda I\) is singular and \(det(A-\lambda I)=0\) can find eigenvalues and eigenvectors.
- Check : \(\lambda_1 + \lambda_2 + \cdots + \lambda_n = a_{11} + a_{22} + \cdots + a_{nn}\)
- Projection Matrix : \(\lambda = 1 \ and \ 0\)；Reflections Matrix : \(\lambda = 1 \ and \ -1\)；Rotations Matrix : \(\lambda = e^{i \theta} \ and \ e^{-i \theta}\)。
The Equation for the Eigenvalues and Eigenvectors
- Compute the determinant of \(A-\lambda I\).
- Find the roots of the polynomial of the determinant of \(A-\lambda I\)，by solving det(\(A-\lambda I\)) = 0.
- For each eigenvalue \(\lambda\)，solve \((A-\lambda I)X = 0\) to find an eigenvector X.
example:
If \(AX=\lambda X\)，the \((A+nI)X = \lambda X + nIX = (\lambda + n)X\)；If eigenvectors of A is the same as eigenvectors of B, the \((A+B)X=(\lambda_{A} + \lambda_{B})X\).
Diagonalizing a Matrix
Eigenvectors of A for n different \(\lambda's\) are independent.Then we can diagonalize A.
The columns of X are eigenvectors.
So:
example:
When all \(|\lambda_i| < 0\)，the \(A^k \rightarrow 0\).
6.2 Applications of Eigenvalue and Eigenvector
Difference equation \(u_{k+1} = Au_k\)
Matrix Powers \(A^k\) : \(u_{k}=A^ku_0 = (X \Lambda X^{-1})(X \Lambda X^{-1})\cdots(X \Lambda X^{-1})u_0=X \Lambda^k X^{-1}u_0\)
step1 :
step2~3:
It solves \(u_{k+1} = Au_k\)
example:
Fibonacci Numbers: 0,1,1,2,3,5,8,13...
\(F_{k+2}=F_{k+1}+F_{k}\)
Let \(u_k = \left[ \begin{matrix} F_{k+1}\\F_k \end{matrix}\right]\)
Solution of du/dt = Au
key : \(e^{At}\)
Taylor Series : \(e^x = 1 + x + \frac{1}{2}x^2+\cdots+\frac{1}{n!}x^n\)
S is eigenvectors matrix of A.
Solve Steps:
-
Find eigenvalues and eigenvectors of A by solving \(det(A-\lambda I)=0\).
-
Write u(0) as a combination \(c_1x_1 + c_2x_2 + \cdots + c_nx_n\) of the eigenvectors of A.
-
Multiply each eigenvector \(x_i\) by its growth factor \(e^{\lambda_i t}\).
-
The solution is the combinations of those pure solutions \(e^{\lambda t}x\).
\[\frac{du}{dt} = Au \\ u(t) = c_1e^{\lambda_1 t}x_1 + c_2e^{\lambda_2 t}x_2 + \cdots + c_ne^{\lambda_n t}x_n \]
example:
State:
- Stabillity : \(u(t) -> 0 (e^{\lambda t}->0, real\ \ part\ \ \lambda < 0)\)
- Steady State : \(\lambda_1 = 0\) and other real part \(\lambda's < 0\)
- Blow up if any real part \(\lambda > 0\)
Markov Matrices
keys:
- All entries >=0.
- All columns add to 1.
- \(\lambda =1\) is one of eigenvalues.
- All other \(|\lambda_i|<1\).
- \(u_k = A^{k}u_0 = c_1\lambda_1^{k}x_1 + c_2\lambda_2^{k}x_2 + \cdots + c_n\lambda_n^{k}x_n \rightarrow c_1x_1 \ \ (steady \ \ state)\)
example: people movement model
\(u_{k+1} = Au_{k}\)，A is Markov Matrix.
if \(\left [ \begin{matrix} u_{col} \\ u_{mass} \end{matrix}\right]_{0} = \left [ \begin{matrix} 0 \\ 1000 \end{matrix}\right]\) , and \(c_1=1000/3, c_2=2000/3\)
\(u_k = c_1\lambda_1^{k}x_1+c_2\lambda_2^{k}x_2 = \frac{1000}{3}1^{k}\left [ \begin{matrix} 2 \\ 1 \end{matrix}\right] + \frac{2000}{3}0.7^{k}\left [ \begin{matrix} -1 \\ 1 \end{matrix}\right] \rightarrow \frac{1000}{3}\left [ \begin{matrix} 2 \\ 1 \end{matrix}\right]\) (steady state)
?Projections and Fourier Series
Projections with orthonormal basis:
Fourier series:
\(f(x) = a_0 + a_1cosx + b_1sinx + a_2cos2x + b_2sin2x + \cdots + b_nsinnx\)
(\(1,cosx,sinx,cos2x,sin2x...\)) are basis of f(x)
check: \(f(x) = f(x+ 2\pi)\)
\(f^Tg = \int_{0}^{2\pi}f(x)g(x)dx=0\) with f(x) = 1,cosx,sinx,cos2x,sin2x..., g(x) = 1,cosx,sinx,cos2x,sin2x..., \(f(x) \neq g(x)\)
example:
\(\int_{0}^{2\pi}f(x)cosxdx= \int_{0}^{2\pi}(a_0cosx + a_1(cosx)^2 + b_1cosxsinx...)dx= a_1\int_{0}^{2\pi} (cosx)^2 dx = a_1\pi\)
\(a_1 = \frac{1}{\pi}\int_{0}^{2\pi}f(x)cosxdx\)
6.3 Special Matrix
6.3.1 Symmetric Matrices
keys:
- A symmetric matrix S has n real eigenvalues \(\lambda_i\) and n orthonormal eigenvectors \(q_1,q_2,...,q_n\).
- Every real symmetric S can be diagonalized: \(S=Q \Lambda Q^{-1} = Q \Lambda Q^{T} =\left[ \begin{matrix} q_1&q_2&\cdots&q_n \end{matrix}\right] \left[ \begin{matrix} \lambda_1&& \\ &\lambda_2 \\ &&\ddots \\ &&&\lambda_n\end{matrix} \right] \left[ \begin{matrix} q_1^{T}\\q_2^{T}\\\vdots\\q_n^{T} \end{matrix}\right]\).
- The number of positive eigenvalues of S equals the number of positive pivots.
- Antisymmetric matrices \(A = A^{-T}\) have imaginary \(\lambda's\) and orthonormal (complex) q's.
example:
6.3.2 Positive Definite Matrix
keys:
-
Symmetric S : all eigenvalues > 0 \(\Leftrightarrow\) all pivots > 0 \(\Leftrightarrow\) all upper left determinants > 0
-
The Symmetric S is the postive definite : \(x^TSx > 0\) for all vectors \(x\neq0\).
-
\(A^TA\) is positive definite matrix.
proof: A is m by n
\[x^T(A^TA)x = (Ax)^T(Ax) = |Ax|^2 >= 0 \\ if \ \ A \ \ rank=n \\ |Ax|^2 >0 \]\(A^TA\) is positive definite matrix.
\(A^TA\) is invertible, that \(\widehat{x} = (A^TA)^{-1}A^Tb\) work fine.
example:
so A is positive definite matrix.
Minimum :
First derivatives : \(\frac{\partial f}{\partial x_1} = \frac{\partial f}{\partial x_2} = \frac{\partial f}{\partial x_3} =0\)
Second derivatives : \(\frac{\partial^2 f}{\partial x_1^2} = \frac{\partial^2 f}{\partial x_2^2} = \frac{\partial^2 f}{\partial x_3^2} >0\)
Maximum :
First derivatives : \(\frac{\partial f}{\partial x_1} = \frac{\partial f}{\partial x_2} = \frac{\partial f}{\partial x_3} =0\)
Second derivatives : \(\frac{\partial^2 f}{\partial x_1^2} = \frac{\partial^2 f}{\partial x_2^2} = \frac{\partial^2 f}{\partial x_3^2} <0\)
when \(f = x^TAx = 2x_1^2 + 2x_2^2 + 2x_3^2-2x_1x_2-2x_2x_3 = (x_1-x_2)^2 + (x_2-x_3)^2 + x_1^2 = 1\)
\(x^TAx=1\) describe an ellipse in 4D, with \(A=Q\Lambda Q^{T}\), Q are the directions of the principal axes, \(\Lambda\) are the lengths of those axes.
6.3.3 Similar Matrices
if \(B = M^{-1}AM\) for some matrix M, that A and B are similar.
example: \(A = \left [ \begin{matrix} 2&1 \\ 1&2 \end{matrix}\right]\)
-
Special example: A is similar to \(\Lambda\)，\(S^{-1}A S = \Lambda \ 或 \ A=S^{-1}\Lambda S \Rightarrow \Lambda = \left [ \begin{matrix} 3&0 \\ 0&1 \end{matrix}\right]\);
-
other :
\[B = M^{-1}AM =\left [ \begin{matrix} 1&-4 \\ 0&1 \end{matrix}\right] \left [ \begin{matrix} 2&1 \\ 1&2 \end{matrix}\right] \left [ \begin{matrix} 1&4 \\ 0&1 \end{matrix}\right] = \left [ \begin{matrix} -2&-15 \\ 1&6 \end{matrix}\right] \]\(A,\Lambda,B\) have the same \(\lambda's\).
- A and \(\Lambda\) with same eigenvalues and eigenvectors.
- A and B with same eigenvalues and numbers of eigenvectors, different eigenvectors.(\(X_B=M^{-1}X_A\))
?6.3.4 Jordan Theorem
Every square A is similar to a Jordan matrix：
Numbers of Jordan blocks is equal to numbers of eigenvectors.
Good : \(J=\Lambda\)，（d=n)
