幂迭代法求特征值和特征向量

幂迭代法求第k大的特征值和特征向量

数学表述

设矩阵A

\[A = \left[ \begin{matrix} X_{11}&\ldots&X_{1n} \newline X_{21} & \ldots& X_{2n} \newline &\vdots& \newline X_{n1}&\ldots &X_{nn} \newline \end{matrix} \right] \]

求其最大特征值对应的特征向量\(b_k = [v_1,v_2,\ldots,v_n]^T\):

\[b_{k+1} = \frac{Ab_k}{||Ab_k ||}..........(1) \]

证明：见英文wiki Power_iteration

代码：

import numpy as np
def power_iteration(A, num_simulations: int):
    # Ideally choose a random vector
    # To decrease the chance that our vector
    # Is orthogonal to the eigenvector
    b_k = np.random.rand(A.shape[1])
  
    for _ in range(num_simulations):
        # calculate the matrix-by-vector product Ab
        b_k1 = np.dot(A, b_k)
 
        # calculate the norm
        b_k1_norm = np.linalg.norm(b_k1)
 
        # re normalize the vector
        b_k = b_k1 / b_k1_norm
 
    return b_k

例子

\[A = \left[ \begin{matrix} 2&1\newline 1&2\newline \end{matrix} \right] \]

求其特征值：

\[|A-\lambda I| = \left| \begin{matrix} 2-\lambda &1\newline 1 &2-\lambda\newline \end{matrix} \right| = 3 - 4\lambda + \lambda^2 \]

求得最大特征值\(\lambda_2 = 3\)，另一特征值\(\lambda_1 = 1\)
\(\lambda_2 = 3\) 时：

\[|(A-\lambda I)V| = 0 \]

\[(A-3I)V = \left[ \begin{array} {cccc} -1 &1\newline 1 &-1\newline \end{array} \right] \left[ \begin{array} {cccc} v_1\newline v_2\newline \end{array} \right] = \left[ \begin{array} {cccc} 0\newline 0\newline \end{array} \right] \]

求得\(v_1 = v_2\),归一化后，\(V_{\lambda=3} = [0.707,0.707]^T\)

验证代码：

A = np.array([[2,1],[1,2]])
V = power_iteration(A, 100).reshape(1,-1)
V

array([[0.70710678, 0.70710678]])

求得特征向量后，特征值

\[\lambda = VAV^T........(2) \]

代码

lbd = np.dot(V,np.dot(A,V.T))
lbd 

array([[3.]])

扩展

求次大特征向量与特征值:

\[B = A - \frac{\lambda}{||V||^2}V^TV ........(3) \]

\[b_{k+1}' = \frac{Bb_k'}{||Bb_k' ||} ...........(4) \]

该理论借鉴于stackexchange,其中\(||V||^2\)是对特征向量归一化（好像求得的特征向量模长应该都是1，不需要归一化应该也没问题）。

代码

B = A - lbd / np.linalg.norm(V)**2 * np.dot(V.T,V)
V_2 = power_iteration(B, 100)
lbd1 = np.dot(V_2.T,np.dot(B,V_2))
lbd1

1.0000000000000002

以此类推，我们可以得到任意第k大特征值。这在很多机器学习方法中，取前k重要特征有着重要的作用。
欢迎访问个人博客。转载请注明出处。