线性代数

• independent .VS. dependent

给定向量u, v, w，如果x1*u+x2*v+x3*w = 0当且仅当x1=0,x2=0,x3=0时才成立，则称u,v,w是independent

Independent columns: Ax = 0 has one solution. A is an inveritlbe matrix

Depedent columns: Ax = 0 has many solutions. A is a singular matrix

• Gauss-Jordan Elimination

可以手动的求逆，[A I]通过初等变换得到[I B]，B就是A的逆

• LU factorization

L是下三角矩阵，U是上三角矩阵，U的对角元素是pivot

(E₃₂E₃₁E₂₁)A = U 变成 A = (E_21^-1 E_31^-1 E_32^-1 )U 即 A=LU

简单来说，L就是把原来初等矩阵对焦元素下方的符号变了

• LDU factorization

也就是对LU中的U进一步分解提出来个D={d_ii =d_i }，把U每一行除以d_i

• LU factorization to solve equation

把one square system变成two triangular systems

Solve Ax = b  -----> Solve Lc = b and then solve Ux = c 

• Transpose 

Ax combines the columns of A while x^TA^T combines the rows of A^T

• Symmetric matrix

如果对称矩阵(A=A^T)，对它做LDU分解，则U=L^T

• Permuation matrix

置换矩阵（使矩阵行交换），元素只有0和1，对于n阶矩阵，只有n!个置换矩阵。

1. 如果P是置换矩阵，则P^-1也是置换矩阵，同时P^-1=P^T

2. PA = LU

• Fundamental Theorem of Linear Algebra, Part I

In Rⁿ the row space and nullspace have dimensions r and n-r (total n).

In R^m the column space and left nullspace have dimensions r and m-r (total m).

• Fundamental Theorem of Linear Algebra, Part II

N(A) is the orthogonal complement of the row space C(A^T) (in Rⁿ)

N(A^T) is the orthogonal complement of the column space C(A) (in R^m)

• Projections

The projection of b onto the subspace is p: p = A x = A(A^TA)^-1A^Tb

Th n by n projection matrix that produces p = Pb: P = A(A^TA)^-1A^T

• Gram-Schmidt --- a way to create orthonormal vectors

Start with three independent vectors a, b, c

A = a, B = b - (A^Tb)/(A^TA) A, C = c - (A^Tc)/(A^TA) A - (B^Tc)/(B^TB)B

q₁ = A/||A||,  q₂ = B/||B||, q₃ = C//||C||

• QR factorization

[a b c] = [q1 q2 q3][q1^Ta q1^Tb q1^Tc; 0 q2^Tb q2^Tc; 0, 0, q3^Tc]

• The Properties of the Determinant

The determinant of the n by n identity matrix is 1.

The determinant changes sign when two rows are exchanged.

The determinant is a linear function of each row separately.

If two rows of A are equal, then det A = 0.

Substracting a multiple of one row from another row leaves det A unchanged.

A matrix with a row of zeros has det A = 0

If A is triangular then det A = a11a22...a_nn = product of diagonal entries.

If A is singular then det A = 0. If A is invertible then det A doesn't equal 0 .

The determinant of AB is det A times det B: |AB| = |A||B|.

The transpose A^T has the same determinant as A.

• Eigenvalues Ax = λx

Each column of A adds to 1, so λ = 1 is an eigenvalue.

A is singular, so λ = 0 is an eigenvalue.

A is symmetric, so its eigenvectors are perpendicular.

λ1+λ2+...+λn = trace = a11 + a22 + ... + ann.

λ1*λ2*...*λn = determinant(A).