Typesetting math: 100%

Linear Algebra 期末复习

Linear System

consistent/inconsistent, coefficient matrix, augmented matrix

Elementary row operations (ERO)

  • replacement: replace one row by the sum of itself and a multiple of another row. (AiAi+rAjAiAi+rAj)
  • interchange: interchange two rows. (AiAj,AjAiAiAj,AjAi)
  • scaling: multiple a nonzero scaler to a row. (AirAiAirAi)

echelon form, reduced echelon form, pivot position

Def. A matrix is in an echelon form if the following conditions hold.

  • all nonzero rows are above all zero rows;
  • each leading entry (the leftmost nonzero entry) of a row is in a column to the right of the leading entry of the row above it.
  • (a consequence) all entries in a column below leading entry are zeros.

Def. A matrix is in reduced row echelon form if the following additional conditions are satisfied.

  • the leading entry in each row is 11.
  • each leading 11 is the only nonzero entry in its column.

Def. A pivot position in a matrix AA is a location in AA that corresponds to a leading 11 in the reduced echelon form of AA (i.e. a leading entry of a row in the echelon form of AA). A pivot column is a column of AA containing a pivot position.

The row reduction algorithm to solve a linear system


Vector Spaces

field, vector space, subspace

FF is called a field if it's a set where ++ and are defined.

VV is called a vector space if it's a set where addition and scalar multiplication are defined.

If WVWV, and WW is still a vector space with addition and scalar multiplication from VV, then WW is called subspace of VV.

Thm. WW is a subspace of VV if and only if 0W0W, and WW is closed under addition and scalar multiplication.

linear dependent/linear independent

Thm. Suppose SS is linearly independent, then S{v}S{v} is linearly dependent if and only if vspan(S)vspan(S).

basis, dimension

Thm. Suppose WW is a subspace of VV (containing more than one vector). Then any linearly independent set in HH can be expanded, if necessary, to a basis for HH.

Thm.(The basis theorem) Let VV be a pp-dimensional vector space, then any linearly independent set of exactly pp elements in VV is automatically a basis for VV.

​ Any set of exactly pp elements that spans VV is automatically a basis for VV.


Linear Transformation

linear transformation, range/image, kernel/null space, rank, nullity

Thm.(Dimension Theorem) Suppose T:VWT:VW, then nullity(T)+rank(T)=dim(V)nullity(T)+rank(T)=dim(V).

one-to-one, onto

Thm. Suppose {v1,v2,,vn}{v1,v2,,vn} is a basis for VV. For w1,w2,,wnWw1,w2,,wnW, there exists exactly one linear transformation T:VWT:VW such that T(vi)=wiT(vi)=wi for i=1,2,,ni=1,2,,n.

coordinate vector of xx relative to ββ

Let β=\setu1,u2,,unβ=\setu1,u2,,un be an ordered basis for a finite-dimensional vector space VV. For xVxV, let a1,a2,,ana1,a2,,an be the unique scalars such that x=ni=1aiuix=ni=1aiui. We define the coordinate vector of xx relative to ββ(坐标向量), denoted [x]β[x]β, by [x]β=(a1a2an)T[x]β=(a1a2an)T.

the vector space of all linear transformations from VV to WW

We denote the vector space of all linear transformations from VV into WW by L(V,W)L(V,W). In the case that V=WV=W, we write L(V)L(V) instead of L(V,W)L(V,W).

matrix representation

Def. Suppose that VV and WW are finite-dimensional vector spaces with ordered bases β=\setv1,v2,,vnβ=\setv1,v2,,vn and γ=\setw1,w2,,wmγ=\setw1,w2,,wm, respectively. Let T:VWT:VW be linear. Then for each jj, 1jn1jn, there exist unique scalars aijF,1imaijF,1im, such that T(vj)=mi=1aijwiT(vj)=mi=1aijwi for 1jn1jn. We call the m×nm×n matrix AA defined by Aij=aijAij=aij the matrix representation of TT in the ordered bases ββ and γγ and write A=[T]γβA=[T]γβ.

Thm. Let T:VWT:VW be linear, and let β,γβ,γ be ordered bases for V,WV,W, respectively. Then we have [T(x)]γ=[T]γβ[x]β[T(x)]γ=[T]γβ[x]β.

Thm. Let T:VWT:VW and U:WZU:WZ be linear transformations, and let α,β,γα,β,γ be ordered bases for V,WV,W and ZZ, respectively. Then we have [UT]γα=[U]γβ[T]βα[UT]γα=[U]γβ[T]βα.

left-multiplication transformation

Def. Let AA be an m×nm×n matrix with entries from a field FF. We denote by LALA the mapping FnFmFnFm defined by LA(x)=AxLA(x)=Ax (the matrix product of AA and xx) for each column vector xFnxFn. We call LALA a left-multiplication transformation.

inverse, invertible

Thm. A linear transformation is invertible if and only if it’s both one-to-one and onto.

Thm. Let T:VWT:VW be a linear transformation. If dim(V)=dim(W)dim(V)=dim(W), then TT is invertible if and only if rank(T)=dim(V)rank(T)=dim(V).

isomorphic, isomorphism

Def. Let VV and WW be vector spaces. We say VV is isomorphic to WW if there exists a linear transformation T:VWT:VW that is invertible. Such a linear transformation is called an isomorphism from VV onto WW.

Thm. Let VV and WW be finite-dimensional vector spaces over the same field. Then VV is isomorphic to WW if and only if dim(V)=dim(W)dim(V)=dim(W).

standard representation of VV with respect to ββ

Def. Let ββ be an ordered basis for an nn-dimensional vector space VV over a field FF. The standard representation of VV with respect to ββ is the function ϕβ:VFnϕβ:VFn defined by ϕβ(x)=[x]βϕβ(x)=[x]β for each xVxV.

change of coordinate matrix

Def. Let ββ and ββ be two ordered bases for a finite-dimensional vector space VV. The matrix Q=[IV]ββQ=[IV]ββ is called a change of coordinate matrix, and we say that QQ changes ββ-coordinates into ββ-coordinates.

Thm. Let TT be a linear operator on a finite-dimensional vector space VV, and let ββ and ββ be ordered bases of VV. Suppose that QQ is the change of coordinate matrix that changes ββ-coordinates into ββ-coordinates, then [T]β=Q1[T]βQ[T]β=Q1[T]βQ.

similar

linear operator, linear functional

coordinate function, dual space, dual basis

Def. Let VV be a finite-dimensional vector space and let β=\setx1,,xnβ=\setx1,,xn be an ordered basis for VV. For each i=1,2,,ni=1,2,,n, define fi(x)=aifi(x)=ai, where [x]β=(a1a2an)T[x]β=(a1a2an)T is the coordinate vector of xx relative to ββ. Then fifi is a linear functional on VV called the ii-th coordinate function with respect to the basis ββ.

Def. For a vector space VV over FF, we define the dual space of VV to be the vector space L(V,F)L(V,F), denoted by VV. We also define the double dual space VV of VV to be the dual space of VV.

Thm. Suppose that VV is a finite-dimensional vector space with the ordered basis β=\setx1,,xnβ=\setx1,,xn. Let fi(1in)fi(1in) be the ii-th coordinate function with respect to ββ, and let β=\setf1,,fnβ=\setf1,,fn. Then ββ is an ordered basis for VV, and for any fVfV, we have f=ni=1f(xi)fif=ni=1f(xi)fi. The ordered basis ββ is called the dual basis of ββ.

Thm. Let VV be a finite-dimensional vector space. For a vector xVxV, we define ˆx:VF^x:VF by ˆx(f)=f(x)^x(f)=f(x) for every fVfV. Define ψ:VVψ:VV by ψ(x)=ˆxψ(x)=^x. Then ψψ is an isomorphism.


Matrix

rank

Def. If AMm×n(F)AMm×n(F), we define the rank of AA, denoted rank(A)rank(A), to be the rank of the linear transformation LA:FnFmLA:FnFm.

Thm. Elementary row and column operations on a matrix are rank-preserving.

所有的ERO可以用左乘一个矩阵描述,这些矩阵的秩都是 nn

交换 i,ji,j 两行:在单位矩阵的基础上,把 (i,i),(j,j)(i,i),(j,j)00,把 (i,j),(j,i)(i,j),(j,i)11

把第 ii 行的 kk 倍加到第 jj 行上:在单位矩阵的基础上,把 (j,i)(j,i)kk

把第 ii 行翻 kk 倍:在单位矩阵的基础上,把 (i,i)(i,i)kk

Thm. The rank of any matrix equals the maximum number of its linearly independent columns; that is, the rank of a matrix is the dimension of the subspace generated by its columns.

Corollary. The rank of a matrix equals the number of pivot columns.

Thm. Let AA be an m×nm×n matrix of rank rr. Then rm,rnrm,rn, and by means of a finite number of elementary row and column operations, AA can be transformed into the matrix D=(IrO1O2O3)D=(IrO1O2O3), where O1,O2,O3O1,O2,O3 are zero matrices. Thus Dii=1Dii=1 for irir and Dij=0Dij=0 otherwise.

Corollary. Let AA be an m×nm×n matrix of rank rr, then there exist invertible matrices BB and CC of sizes m×mm×m and n×nn×n, respectively, such that D=BACD=BAC and DD has the form above.

这个“对角化”的做法就是做行变换和列变换。

partitioned matrices

LU decomposition, inverse

For a linear system Ax=bAx=b, if we can decompose AA as LULU, we can divide the linear system into two linear systems Lc=bLc=b and Ux=cUx=c. Here, for an m×nm×n matrix AA, LL is an m×mm×m lower triangular matrix, and UU is in echelon form.

AA 做 ERO 把 AA 变成 echelon form 就得到了 UULijLij 是过程中第 jj 行加到第 ii 行的系数的相反数。

AA 可逆,对 (AI)(AI) 做 ERO 把 AA 变成单位矩阵,得到的矩阵就是 (IA1)(IA1)

homogeneous system

Def. A system Ax=bAx=b of mm linear equations in nn unknowns is said to be homogeneous(齐次的)if b=0b=0. Otherwise the system is said to be nonhomogeneous.

Thm. Let KK be the solution set of a system of linear equations Ax=bAx=b, and let KHKH be the solution set of the corresponding homogeneous system Ax=0Ax=0. Then for any solution ss to Ax=bAx=b, K=\sets+KH=\sets+kkKHK=\sets+KH=\sets+kkKH.

Thm. Let Ax=bAx=b be a system of nn linear equations in nn unknowns. If AA is invertible, then the system has exactly one solution, namely, A1bA1b. Conversely, if the system has exactly one solution, then AA is invertible.


Determinant

Def. Let AMn×n(F)AMn×n(F). If n=1n=1, so that A=(A11)A=(A11), we define det(A)=A11det(A)=A11. For n2n2, we define det(A)det(A) recursively as det(A)=nj=1(1)1+jA1jdet(˜A1j)det(A)=nj=1(1)1+jA1jdet(~A1j). Here, ˜Aij~Aij means the (n1)×(n1)(n1)×(n1) matrix obtained from AA by deleting row ii and column jj.

The scalar det(A)det(A) is called the determinant of AA and is also denoted by |A||A|. The scalar cij=(1)i+jdet(˜Aij)cij=(1)i+jdet(~Aij) is called the cofactor of the entry of AA in row ii, column jj. We can express the formula for the determinant of AA as det(A)=ni=1A1ic1idet(A)=ni=1A1ic1i, and this formula is called cofactor expansion along the first row of AA.

Thm. The determinant of a square matrix can be evaluated by cofactor expansion along any row. That is, if AMn×n(F)AMn×n(F), then for any integer i(1in)i(1in), det(A)=nj=1Aijcijdet(A)=nj=1Aijcij.

The following rules summarize the effect of an elementary row operation on the determinant of a matrix AMn×n(F)AMn×n(F).

  • (interchange) If BB is a matrix obtained by interchanging any two rows of AA, then det(B)=det(A)det(B)=det(A).
  • (scaling) If BB is a matrix obtained by multiplying a row of AA by a nonzero scalar kk, then det(B)=kdet(A)det(B)=kdet(A).
  • (replacement) If BB is a matrix obtained by adding a multiple of one row of AA to another row of AA, then det(B)=det(A)det(B)=det(A).

These facts can be used to simplify the evaluation of a determinant. By using elementary row operations of types 11 and 33 only, we can transform any square matrix into an upper triangular matrix, and so we can easily evaluate determinant of any square matrix since the determinant of an upper triangular matrix is the product of its diagonal entries.

Thm. For any A,BMn×n(F)A,BMn×n(F), det(AB)=det(A)det(B)det(AB)=det(A)det(B).

Corollary. A matrix AMn×n(F)AMn×n(F) is invertible iff det(A)0det(A)0. Furthermore, if AA is invertible, then det(A1)=1det(A)det(A1)=1det(A).

Thm.(Cramer's Rule) Let Ax=bAx=b be the matrix form of a system of nn linear equations in nn unknowns, where x=(x1,,xn)tx=(x1,,xn)t. If det(A)0det(A)0, then this system has a unique solution, and for each k=1,2,,nk=1,2,,n, xk=det(Mk)det(A)xk=det(Mk)det(A), where MkMk is the n×nn×n matrix obtained from AA by replacing column kk of AA by bb.


Diagonalization

diagonalizable

eigenvector, eigenvalue, eigenspace

Thm. A linear operator TT on a finite-dimensional vector space VV is a diagonalizable iff there exists an ordered basis ββ for VV consisting of eigenvectors of TT. Furthermore, if TT is diagonalizable, β={v1,,vn}β={v1,,vn} is an ordered basis of eigenvectors of TT, and D=[T]βD=[T]β, then DD is a diagonal matrix is DjjDjj is the eigenvalue corresponding to vjvj for 1jn1jn.

characteristic polynomial, split, (algebraic) multiplicity

计算矩阵 AA 的特征值,通过求关于 tt 的多项式 det(AtI)det(AtI) 的根。计算 λλ 对应的特征向量,只要解方程 (AλI)x=0(AλI)x=0.

计算线性算子 TT 的特征值和特征向量,先选一个基 ββ,求 A=[T]βA=[T]β 的特征值和特征向量。AA 的特征值就是 TT 的特征值,AA 的特征向量是 TT 的特征向量在基 ββ 下的坐标。

Thm. Let TT be a linear operator on a finite-dimensional vector space VV such that the characteristic polynomial of TT splits. Let λ1,,λkλ1,,λk be the distinct eigenvalues of TT. Then

  • TT is diagonalizable if and only if the multiplicity of λiλi is equal to dim(Eλi)dim(Eλi) for all ii.
  • If TT is diagonalizable and βiβi is an ordered basis for EλiEλi for each ii, then β=β1βkβ=β1βk is an ordered basis for VV consisting of eigenvectors of TT.

这个定理给出了对角化的方法。

微分方程组 ddtx=Axddtx=Ax

考虑线性常微分方程组 xi=nj=1aijxj,i=1,2,,nxi=nj=1aijxj,i=1,2,,n,其中 xi=xi(t)xi=xi(t) 是关于 tt 的函数。对 AA 做对角化 Q1AQ=DQ1AQ=D,设 y=Q1xy=Q1x,则 y=Dyy=Dy。根据 DD 是对角矩阵求 yy,进而求得 xx

sum, direct sum

TT-invariant, TT-cyclic subspace of VV generated by xx

Let TT be a linear operator on a vector space VV, and let xx be a nonzero vector in VV. The subspace W=span({x,T(x),T2(x),})W=span({x,T(x),T2(x),}) is called the TT-cyclic subspace of VV generated by xx.

Thm. Let TT be a linear operator on a finite-dimensional vector space VV, and let WW be a TT-invariant subspace of VV. Then the characteristic polynomial of TWTW divides the characteristic polynomial of TT.

Thm. Let TT be a linear operator on a finite-dimensional vector space VV, and let WW denote the TT-cyclic subspace of VV generated by a nonzero vector vVvV. Let k=dim(W)k=dim(W), then

  • {v,T(v),T2(v),,Tk1(v)}{v,T(v),T2(v),,Tk1(v)} is a basis for WW.
  • If a0v+a1T(v)++ak1Tk1(v)+Tk(v)=0a0v+a1T(v)++ak1Tk1(v)+Tk(v)=0, then the characteristic polynomial of TWTW is f(t)=(1)k(a0+a1t++ak1tk1+tk)f(t)=(1)k(a0+a1t++ak1tk1+tk).

Thm.(Cayley-Hamilton) Let TT be a linear operator on a finite-dimensional vector space VV, and let f(t)f(t) be the characteristic polynomial of TT. Then f(T)=T0f(T)=T0, the zero transformation. That is, TT "satisfies" its characteristic equation.

Corollary.(Cayley-Hamilton Theorem for Matrices) Let AA be an n×nn×n matrix and let f(t)f(t) be the characteristic polynomial of AA. Then f(A)=Of(A)=O, the n×nn×n zero matrix.

Thm. Let TT be a linear operator on a finite-dimensional vector space VV, and suppose that V=W1W2WkV=W1W2Wk, where WiWi is a TT-invariant subspace of VV for each i(1ik)i(1ik). Suppose that fi(t)fi(t) is the characteristic polynomial of TWi(1ik)TWi(1ik), then f1(t)f2(t)fk(t)f1(t)f2(t)fk(t) is the characteristic polynomial of TT.

Thm. Let TT be a linear operator on a finite-dimensional vector space VV, and let W1,W2,,WkW1,W2,,Wk be TT-invariant subspaces of VV such that V=W1W2WkV=W1W2Wk. For each ii, let βiβi be an ordered basis for WiWi, and let β=β1β2βkβ=β1β2βk. Let A=[T]βA=[T]β and Bi=[TWi]βiBi=[TWi]βi for i=1,2,,ki=1,2,,k, then A=B1B2BkA=B1B2Bk.


Inner Product Spaces

inner product

standard inner product, Frobenius inner product

complex/real inner product space

norm/length, unit vector

orthogonal/perpendicular, orthonormal

Def. Let VV be an inner product space. Vectors xx and yy in VV are orthogonal (perpendicular) if <x,y>=0<x,y>=0. A subset SS of VV is orthogonal if any two distinct vectors in SS are orthogonal. A subset SS of VV is orthonormal if SS is orthogonal and consists entirely of unit vectors.

这里 orthogonal 理解成“正交”而非垂直。在后面某些地方会有区别。

Def. Let VV be an inner product space. A subset of VV is an orthonormal basis for VV if it is an ordered basis that is orthonormal.

Thm. Let VV be an inner product space and S={v1,v2,,vk}S={v1,v2,,vk} be an orthogonal subset of VV consisting of nonzero vectors. If yspan(S)yspan(S), then y=ki=1<y,vi>vi2viy=ki=1<y,vi>vi2vi. If, in addition to the hypotheses of this theorem, SS is orthonormal and yspan(S)yspan(S), then y=ki=1<y,vi>viy=ki=1<y,vi>vi.

Gram-Schmidt process

Thm. Let VV be an inner product space and S={w1,,wn}S={w1,,wn} be a linearly independent subset of VV. Define S={v1,,vn}S={v1,,vn}, where v1=w1v1=w1 and vk=wkk1j=1<wk,vj>vj2vjvk=wkk1j=1<wk,vj>vj2vj for 2kn2kn. Then SS is an orthogonal set of nonzero vectors such that span(S)=span(S)span(S)=span(S). The construction of {v1,,vn}{v1,,vn} is called the Gram-Schmidt process.

Fourier coefficients

Def. Let ββ be an orthonormal subset (possibly infinite) of an inner product space VV, and let xVxV. We define the Fourier coefficients of xx relative to ββ to be the scalars <x,y><x,y>, where yβyβ.

orthogonal complement, orthogonal projection

一个集合 SS 的正交补集(orthogonal complement)是和其中所有向量都垂直的向量构成的集合。这个集合用 SS 表示,形式化地说,有 S={xVS={xV<x,y> = 0 \text{ for all } y\in S}<x,y> = 0 \text{ for all } y\in S}

Thm. Let WW be a finite-dimensional subspace of an inner product space VV, and let yVyV. Then there exist unique vectors uWuW and zWzW such that y=u+zy=u+z. Furthermore, if {v1,v2,,vk}{v1,v2,,vk} is an orthonormal basis for WW, then u=ki=1<y,vi>viu=ki=1<y,vi>vi.

Corollary. In the notation of this theorem, the vector uu is the unique vector in WW that is "closest" to yy; that is, for any xWxW, yxyuyxyu, and this inequality is an equality if and only if x=ux=u. The vector uu is called the orthogonal projection of yy on WW.

adjoint

一个算子的共轭算子利用它们的矩阵表示定义。如果 TTUU 在某个标准正交基下的矩阵表示互为共轭转置,那么 TTUU 互为共轭算子。下面的定理给出了共轭算子的一个等价定义。

Thm. Let VV be a finite-dimensional inner product space, and let TT be a linear operator on VV. Then there exists a unique function T:VVT:VV such that <T(x),y>=<x,T(y)><T(x),y>=<x,T(y)> for all x,yVx,yV.

least squares approximation (最小二乘法)

最小二乘法考虑的问题是用一条直线拟合若干个点。形式化地讲,给定 mm 个点 (t1,y1),(t2,y2),(t1,y1),(t2,y2),(tm,ym)(tm,ym),要寻找一条直线 y=ct+dy=ct+d,使得 E=mi=1(yictid)2E=mi=1(yictid)2 最小。这个问题可以进一步抽象化:设 A=(t11t21tm1),x=(cd),y=(y1y2ym)A=⎜ ⎜ ⎜ ⎜t11t21tm1⎟ ⎟ ⎟ ⎟,x=(cd),y=⎜ ⎜ ⎜ ⎜y1y2ym⎟ ⎟ ⎟ ⎟, 则 E=yAx2E=yAx2。我们将抛开这种固定的形式,对任意给定的 AAyy,求一个 xx 使得 yAxyAx 最小。

Thm. Let AMm×n(F)AMm×n(F) and yFmyFm. Then there exists x0Fnx0Fn such that (AA)x0=Ay(AA)x0=Ay and Ax0yAxyAx0yAxy for all xFnxFn. Furthermore, if rank(A)=nrank(A)=n, then x0=(AA)1Ayx0=(AA)1Ay.

Ax0Ax0R(LA)R(LA) 中最靠近 yy 的向量;如果 x0x0 满足这一条件,则 Ax0yR(LA)Ax0yR(LA),也就是 <x,A(Ax0y)>=0<x,A(Ax0y)>=0 对所有 xx 成立。再注意到如果 rank(A)=nrank(A)=n,则 rank(AA)=nrank(AA)=n,就可以直接推出全部结论。

minimal solution

下面的定理给出了求一个线性方程组模长最小的解(minimal solution)的方法。

Thm. Let AMm×n(F)AMm×n(F) and bFmbFm. Suppose that Ax=bAx=b is consistent, then the following statements are true.

  • There exists exactly one minimal solution ss of Ax=bAx=b, and sR(LA)sR(LA).
  • The vector ss is the only solution to Ax=bAx=b that lies in R(LA)R(LA). That is, if uu satisfies (AA)u=b(AA)u=b, then s=Aus=Au.

Thm.(Schur) Let TT be a linear operator on a finite-dimensional inner product space VV. Suppose that the characteristic polynomial of TT splits, then there exists an orthonormal basis ββ for VV such that the matrix [T]β[T]β is upper triangular.

normal

Def. Let VV be an inner product space, and let TT be a linear operator on VV. We say that TT is normal if TT=TTTT=TT. An n×nn×n real or complex matrix AA is normal if AA=AAAA=AA.

Thm. Let TT be a linear operator on a finite-dimensional complex inner product space VV, then TT is normal if and only if there exists an orthonormal basis for VV consisting of eigenvectors of TT.

self-adjoint(Hermitian)

Def. Let TT be a linear operator on an inner product space VV. We say that TT is self-adjoint(Hermitian) if T=TT=T. An n×nn×n real or complex matrix AA is self-adjoint(Hermitian) if A=AA=A.

Thm. Let TT be a linear operator on a finite-dimensional real inner product space VV. Then TT is self-adjoint if and only if there exists an orthonormal basis ββ for VV consisting of eigenvectors of TT.

unitary/orthogonal operator

Def. Let TT be a linear operator on a finite-dimensional inner product space VV (over FF). If T(x)=xT(x)=x for all xVxV, we call TT a unitary operator if F=CF=C and an orthogonal operator if F=RF=R.

Thm. Let TT be a linear operator on a finite-dimensional inner product space VV. Then the following statements are equivalent.

  • TT=TT=ITT=TT=I.
  • <T(x),T(y)>=<x,y><T(x),T(y)>=<x,y> for all x,yVx,yV.
  • If ββ is an orthonormal basis for VV, then T(β)T(β) is an orthonormal basis for VV.
  • There exists an orthonormal basis ββ for VV such that T(β)T(β) is an orthonormal basis for VV.
  • T(x)=xT(x)=x for all xVxV.

Def. A square matrix AA is called an orthogonal matrix if AtA=AAt=IAtA=AAt=I and unitary if AA=AA=IAA=AA=I.

Def. Two matrix AA and BB are unitary equivalent [orthogonal equivalent] if and only if there exists a unitary [orthogonal] matrix PP such that B=PAPB=PAP.

Thm. Let AA be a complex n×nn×n matrix. Then AA is normal if and only if AA if unitarily equivalent to a diagonal matrix.

Thm. Let AA be a real n×nn×n matrix. Then AA is symmetric if and only if AA if orthogonally equivalent to a diagonal matrix.

实数域和复数域上部分概念和结论是不一样的,具体来说有以下几处。

首先,对于一个算子,如果有一个由其特征向量组成的标准正交基,那么,在复数域上,可以推出这个算子是正规(normal)的;在实数域上,可以推出这个算子是自伴(self-adjoint)的。

其次是一个称呼的不同,如果一个算子保范数(norm/length),在复数域上,将其称为酉(unitary)算子;在实数域上,将其称为正交(orthogonal)算子。不过后者的称呼并不常见。

再有,如果一个矩阵在复数域上可以酉对角化,可以推出这个矩阵是正规的(normal);如果一个矩阵在实数域上可以正交(orthogonal)对角化,可以推出这个矩阵是对称的。

orthogonal projection

Recall that if V=W1W2V=W1W2, then a linear operator TT on VV is the projection on W1W1 along W2W2 if, whenever x=x1+x2x=x1+x2, with x1W1x1W1 and x2W2x2W2, we have T(x)=x1T(x)=x1.

Def. Let VV be an inner product space, and let T:VVT:VV be a projection. We say that TT is an orthogonal projection if R(T)=N(T)R(T)=N(T) and N(T)=R(T)N(T)=R(T).

Thm. Let VV be an inner product space, and let TT be a linear operator on VV. Then TT is an orthogonal projection if and only if TT has an adjoint TT and T2=T=TT2=T=T.

Spectral Theorem, spectrum, resolution of the identity operator, spectral decomposition

Thm. (The Spectral Theorem) Suppose that TT is a linear operator on a finite-dimensional inner product space VV over FF with the distinct eigenvalues λ1,λ2,,λkλ1,λ2,,λk. Assume that TT is normal if F=CF=C and that TT is self-adjoint if F=RF=R. For each i(1ik)i(1ik), let WiWi be the eigenspace of TT corresponding to the eigenvalue λiλi, and let TiTi be the orthogonal projection of VV on WiWi. Then the following statements are true.

  • V=W1W2WkV=W1W2Wk.
  • If WiWi denotes the direct sum of the subspaces WjWj for jiji, then Wi=WiWi=Wi.
  • TiTj=δijTiTiTj=δijTi for 1i,jk1i,jk.
  • I=T1+T2++TkI=T1+T2++Tk.
  • T=λ1T1+λ2T2++λkTkT=λ1T1+λ2T2++λkTk.

The set {λ1,λ2,,λk}{λ1,λ2,,λk} of eigenvalues of TT is called the spectrum of TT. The sum I=T1+T2++TkI=T1+T2++Tk in (d) is called the resolution of the identity operator, and the sum T=λ1T1+λ2T2++λkTkT=λ1T1+λ2T2++λkTk in (e) is called the spectral decomposition of TT.

Corollary 1. If F=CF=C, then TT is normal if and only if T=g(T)T=g(T) for some polynomial gg.

Corollary 2. If F=CF=C, then TT is unitary if and only if TT is normal and |λ|=1|λ|=1 for every eigenvalue of TT.

Corollary 3. If F=CF=C and TT is normal, then TT is self-adjoint if and only if every eigenvalue of TT is real.

Corollary 4. Let TT be as in the spectral theorem with spectral decomposition T=λ1T1+λ2T2++λkTkT=λ1T1+λ2T2++λkTk. Then each TjTj is a polynomial in TT.

singular value decomposition, singular value

奇异值分解(singular value decomposition,SVD)是把一个 m×nm×n 矩阵 AA 分解成 UΣVUΣV 的形式,其中 U,VU,V 分别是 m×m,n×nm×m,n×n 的矩阵,而 ΣΣ 是一个 m×nm×n 矩阵,并且存在一个 rr,使得 ΣΣ 中仅有 Σ11Σ22ΣrrΣ11Σ22Σrrrr 个元素是非零的。我们将从线性变换开始研究虽然可能只会考计算

Thm. (Singular Value Theorem for Linear Transformations) Let VV and WW be finite-dimensional inner product spaces, and let T:VWT:VW be a linear transformation of rank rr. Then there exists orthonormal bases {v1,v2,,vn}{v1,v2,,vn} for VV and {u1,u2,,um}{u1,u2,,um} for WW and positive scalars σ1σ2σrσ1σ2σr such that T(vi)={σiuiif ir0if i>rT(vi)={σiuiif ir0if i>r.

Furthermore, suppose the preceding conditions are satisfied. Then for 1in1in, vivi is an eigenvector of TTTT with corresponding eigenvalue σ2iσ2i if 1ir1ir and 00 if i>ri>r. Therefore the scalars σ1,σ2,,σrσ1,σ2,,σr are uniquely determined by TT.

Def. The unique scalars σ1,σ2,,σrσ1,σ2,,σr are called the singular values(奇异值)of TT. If r<mr<m and r<nr<n, then the term singular value is extended to include σr+1==σk=0σr+1==σk=0 where k=min(m,n)k=min(m,n).

Def. Let AA be a m×nm×n matrix. We define the singular values of AA to be the singular values of the linear transformation LALA.

Thm. (Singular Value Theorem for Matrices) Let AA be an m×nm×n matrix of rank rr with singular values σ1σ2σrσ1σ2σr, and let ΣΣ be the m×nm×n matrix defined by Σij={σiif i=jr0elseΣij={σiif i=jr0else. Then there exists an m×mm×m unitary matrix UU and an m×nm×n unitary matrix VV such that A=UΣVA=UΣV.

Def. Let AA be an m×nm×n matrix of rank rr with positive singular values σ1σ2σrσ1σ2σr. A factorization A=UΣVA=UΣV where UU and VV are unitary matrices and ΣΣ is the m×nm×n matrix is called a singular value decomposition of AA.

下面来叙述一下求 SVD 的方法。我们对 AAAA 做酉对角化,得到的结果就是 AA=VΣ2VAA=VΣ2V;相应的,对 AAAA 做酉对角化,得到的结果就是 AA=UΣ2UAA=UΣ2U

事实上,假设 A=UΣVA=UΣV,那么 AV=UΣAV=UΣ。注意 UU=VV=IUU=VV=IA=VΣUA=VΣU (注意 ΣΣ 的共轭转置就是 ΣΣ),那么 AA=VΣ2VAA=VΣ2VAA=UΣ2UAA=UΣ2U

bilinear form, matrix representation

双线性型(bilinear form)是指一类 V×VFV×VF 的映射,它对两个变量都线性。我们用 B(V)B(V) 表示这类映射的集合。可以自然地定义加法和数乘,然后 B(V)B(V) 就是一个线性空间。

一个双线性型 HH 相对于一个基 β={v1,v2,,vn}β={v1,v2,,vn} 的矩阵表示(matrix representation) A=ψβ(H)A=ψβ(H)Aij=H(vi,vj)Aij=H(vi,vj) 定义。不难发现 H(x,y)=[x]TβA[y]βH(x,y)=[x]TβA[y]β(这里的 TT 是转置)。特别地,如果 V=FnV=Fn,那么存在一个矩阵 AA,使得 H(x,y)=xTAyH(x,y)=xTAy

symmetric, diagonalizable

双线性型的对称和对角化可以直接由它的矩阵表示定义。

Thm. Let VV be a finite-dimensional vector space over a field FF not of characteristic two. Then every symmetric bilinear form on VV is diagonalizable. Here, FF is of characteristic two if 1+1=01+1=0 in FF.

这里给出了对角化一个对称矩阵的方法。只需要两个方向同时做初等变换,例如把第一行加到第二行上后,立即把第一列加到第二列上。

quadratic form

Def. Let VV be a vector space over FF. A function K:VFK:VF is called a quadratic form if and only if there exists a symmetric bilinear form HB(V)HB(V) such that K(x)=H(x,x)K(x)=H(x,x) for all xVxV.

If the field FF is not of characteristic two, there is a one-to-one correspondence between symmetric bilinear forms and quadratic forms.

Thm. Let VV be a finite-dimensional real inner product space, and let HH be a symmetric bilinear form on VV. Then there exists an orthonormal basis ββ for VV such that ψβ(H)ψβ(H) is a diagonal matrix.

Corollary. Let KK be a quadratic form on a finite-dimensional real inner product space VV. There exists an orthonormal basis β={v1,,vn}β={v1,,vn} for VV and scalars λ1,,λnλ1,,λn (not necessarily distinct) such that if xVxV and x=ni=1sivi,siRx=ni=1sivi,siR, then K(x)=ni=1λis2iK(x)=ni=1λis2i. In fact, if HH is the symmetric bilinear form determined by KK, then ββ can be chosen to be any orthonormal basis for VV such that ψβ(H)ψβ(H) is a diagonal matrix.

Jordan Canonical Form

首先说明 Jordan Canonical Form 只要会算就可以。不过这里还是给出核心的推理过程。

generalized eigenvector, generalized eigenspace

Def. Let TT be a linear operator on a vector space VV, and let λλ be a scalar. A nonzero vector xx in VV is called a generalized eigenvector of TT corresponding to λλ if (TλI)p(x)=0(TλI)p(x)=0 for some integer pp.

Def. Let TT be a linear operator on a vector space VV and let λλ be an eigenvalue of TT. The generalized eigenspace of TT corresponding to λλ, denoted KλKλ, is the subset of VV defined by Kλ={xV(TλI)p(x)=0 for some integer p}Kλ={xV(TλI)p(x)=0 for some integer p}.

Thm. Let TT be a linear operator on a finite-dimensional vector space VV such that the characteristic polynomial of TT splits, and let λ1,λ2,,λkλ1,λ2,,λk e the distinct eigenvalues of TT with corresponding multiplicities m1,m2,,mkm1,m2,,mk. For 1ik1ik, let βiβi be an ordered basis for KλiKλi. Then the following statements are true.

  • βiβj=βiβj= for ijij.
  • β=β1β2βkβ=β1β2βk is an ordered basis for VV.
  • dim(Kλi)=midim(Kλi)=mi for all ii.

这个结论可以直接表述为 V=Kλ1Kλ2KλkV=Kλ1Kλ2Kλk。结论的证明很复杂。

cycle of generalized eigenvectors

Def. Let TT be a linear operator on a finite-dimensional vector space VV, and let xx be a generalized eigenvector of TT corresponding to the eigenvalue λλ. Suppose that pp is the smallest positive integer for which (TλI)p=0(TλI)p=0. Then the ordered set {(TλI)p1(x),{(TλI)p1(x),(TλI)p2,,(TλI)(x),x}(TλI)p2,,(TλI)(x),x} is called a cycle of generalized eigenvectors of TT corresponding to λλ. The vectors (TλI)p1(x)(TλI)p1(x) and xx are called the initial vector and the end vector of the cycle, respectively. We say the length of the cycle is pp.

Thm. Let TT be a linear operator on a vector space VV, and let λλ be an eigenvalue of TT. Suppose that γ1,γ2,,γqγ1,γ2,,γq are cycles of generalized eigenvectors of TT corresponding to λλ such that the initial vectors of the γiγi's are distinct and form a linearly independent set. Then γiγi's are disjoint, and their union γ=qi=1γiγ=qi=1γi is linearly independent.

Thm. Let TT be a linear operator on a finite-dimensional vector space VV, and let λλ be an eigenvalue of TT. Then KλKλ has an ordered basis consisting of a union of disjoint cycles of generalized eigenvectors corresponding to λλ.

对每个广义特征向量空间,存在由若干个循环的并构成的基。

Thm. Let TT be a linear operator on a finite-dimensional vector space VV whose characteristic polynomial splits, and suppose that ββ is a basis for VV such that ββ is a disjoint union of cycles of generalized eigenvectors of TT. Then for each cycle γγ of generalized eigenvectors contained in ββ, W=span(γ)W=span(γ) is TT-invariant, and [TW]γ[TW]γ is a Jordan block. Furthermore, ββ is a Jordan canonical basis for VV.

每个循环给出一个 Jordan block;所有循环的并如果构成一个基,那么它就是一个 Jordan 基。

到这里就可以定论,如果一个算子的特征多项式可分解,那么它就有一个 Jordan canonical form。可以用 left-multiplication transformation 定义矩阵的 Jordan canonical form。

Jordan Canonical Form

首先说明 Jordan Canonical Form 只要会算就可以。不过这里还是给出核心的推理过程。

generalized eigenvector, generalized eigenspace

Def. Let TT be a linear operator on a vector space VV, and let λλ be a scalar. A nonzero vector xx in VV is called a generalized eigenvector of TT corresponding to λλ if (TλI)p(x)=0(TλI)p(x)=0 for some integer pp.

Def. Let TT be a linear operator on a vector space VV and let λλ be an eigenvalue of TT. The generalized eigenspace of TT corresponding to λλ, denoted KλKλ, is the subset of VV defined by Kλ={xV(TλI)p(x)=0 for some integer p}Kλ={xV(TλI)p(x)=0 for some integer p}.

Thm. Let TT be a linear operator on a finite-dimensional vector space VV such that the characteristic polynomial of TT splits, and let λ1,λ2,,λkλ1,λ2,,λk e the distinct eigenvalues of TT with corresponding multiplicities m1,m2,,mkm1,m2,,mk. For 1ik1ik, let βiβi be an ordered basis for KλiKλi. Then the following statements are true.

  • βiβj=βiβj= for ijij.
  • β=β1β2βkβ=β1β2βk is an ordered basis for VV.
  • dim(Kλi)=midim(Kλi)=mi for all ii.

这个结论可以直接表述为 V=Kλ1Kλ2KλkV=Kλ1Kλ2Kλk。结论的证明很复杂。

cycle of generalized eigenvectors

Def. Let TT be a linear operator on a finite-dimensional vector space VV, and let xx be a generalized eigenvector of TT corresponding to the eigenvalue λλ. Suppose that pp is the smallest positive integer for which (TλI)p=0(TλI)p=0. Then the ordered set {(TλI)p1(x),{(TλI)p1(x),(TλI)p2,,(TλI)(x),x}(TλI)p2,,(TλI)(x),x} is called a cycle of generalized eigenvectors of TT corresponding to λλ. The vectors (TλI)p1(x)(TλI)p1(x) and xx are called the initial vector and the end vector of the cycle, respectively. We say the length of the cycle is pp.

Thm. Let TT be a linear operator on a vector space VV, and let λλ be an eigenvalue of TT. Suppose that γ1,γ2,,γqγ1,γ2,,γq are cycles of generalized eigenvectors of TT corresponding to λλ such that the initial vectors of the γiγi's are distinct and form a linearly independent set. Then γiγi's are disjoint, and their union γ=qi=1γiγ=qi=1γi is linearly independent.

Thm. Let TT be a linear operator on a finite-dimensional vector space VV, and let λλ be an eigenvalue of TT. Then KλKλ has an ordered basis consisting of a union of disjoint cycles of generalized eigenvectors corresponding to λλ.

对每个广义特征向量空间,存在由若干个循环的并构成的基。

Thm. Let TT be a linear operator on a finite-dimensional vector space VV whose characteristic polynomial splits, and suppose that ββ is a basis for VV such that ββ is a disjoint union of cycles of generalized eigenvectors of TT. Then for each cycle γγ of generalized eigenvectors contained in ββ, W=span(γ)W=span(γ) is TT-invariant, and [TW]γ[TW]γ is a Jordan block. Furthermore, ββ is a Jordan canonical basis for VV.

每个循环给出一个 Jordan block;所有循环的并如果构成一个基,那么它就是一个 Jordan 基。

到这里就可以定论,如果一个算子的特征多项式可分解,那么它就有一个 Jordan canonical form。可以用 left-multiplication transformation 定义矩阵的 Jordan canonical form。

dot diagram

考虑一个固定的特征值 λλ 和相应的广义特征空间 KλKλ。根据上面的定理,我们可以找到若干个循环的并构成 KλKλ 的基。具体的,我们将它写成 {v1,(TλI)(v1),,(TλI)p11(v1)}{v1,(TλI)(v1),,(TλI)p11(v1)} {v2,(TλI)(v2),,(TλI)p21(v2)}{v2,(TλI)(v2),,(TλI)p21(v2)} {vk,(TλI)(vk),,(TλI)pk1(vk)}{vk,(TλI)(vk),,(TλI)pk1(vk)}。假设 p1p2pkp1p2pk,然后就可以用一个点状图(dot diagram)来表示(第 ii 列有 pipi 个点,从上到下依次是 (TλI)pi1(vi),,(TλI)(vi),vi(TλI)pi1(vi),,(TλI)(vi),vi)。

这个点状图直接给出了 Jordan canonical form。假设所有的特征值是 λ1,λ2,,λkλ1,λ2,,λk,那么 Jordan canonical form 是由 kk 个大块组成的。设第 ii 个特征值 λiλi 对应的基是由 v1,v2,,vniv1,v2,,vni 的循环的并组成的,然后 vjvj 的循环的长度是 pjpj,那么第 ii 个大块又是由 nini 个小块组成的。第 jj 个小块就是 pj×pjpj×pj 的矩阵,对角线上的值是 λiλi

能不能说点人话

下一个定理直接给出了点状图的形状。设特征多项式是 (tλ1)r1(tλ2)r2(tλk)rk(tλ1)r1(tλ2)r2(tλk)rk,那么第 ii 个点状图前 ll 行的点数就是 nullity((TλiI)l)nullity((TλiI)l)。这个定理是不难理解的:对于点状图作用一次 TλiITλiI,第一行都变成零了,后面的行向上移动;再作用一次,原本的第二行又变成零,后面的行再向上移动,依此类推。

有了点状图的形状,就有了 Jordan canonical form 这个矩阵,下一步就是求出 Jordan canonical basis。对每个特征值分别计算。先算点状图的最后一行。这些向量应当是 N((TλI)r)N((TλI)r) 的基的一部分,但不在 N((TλI)r1)N((TλI)r1) 内。然后对这些向量做一遍 TλITλI,得到倒数第二行的向量。倒数第二行可能还有一些点,我们要补全这些向量,使得最后两行的向量构成了 N((TλI)r)N((TλI)r) 的基不在 N((TλI)r2)N((TλI)r2) 的部分。以此类推就可以求出需要的基这玩意根本不是能手算的吧

posted @   by_chance  阅读(185)  评论(0编辑  收藏  举报
相关博文:
阅读排行:
· 在鹅厂做java开发是什么体验
· 百万级群聊的设计实践
· WPF到Web的无缝过渡:英雄联盟客户端的OpenSilver迁移实战
· 永远不要相信用户的输入:从 SQL 注入攻防看输入验证的重要性
· 浏览器原生「磁吸」效果!Anchor Positioning 锚点定位神器解析
点击右上角即可分享
微信分享提示