Orthogonal Complement
Orthogonal Complement in Linear Algebra
Definition. Let \(V\) be an inner product space and \(S\) a nonempty subset of \(V\). The orthogonal complement of \(S\) is
Example. For any matrix \(A\in M_{m\times n}(\mathbb{C})\), \(\text{im}(L_{A^*})^{\perp}=\ker(L_A)\).
Theorem. Let \(W\) be a finite-dimensional subspace of an inner product space \(V\). Then for any \(y\in V\), there exists a unique pair \((u,z)\in W\times W^{\perp}\) such that \(y=u+z\) . It follows that \(V=W\oplus W^{\perp}\), and hence if \(\dim V<\infty\), then \(\dim V=\dim W+\dim W^{\perp}\). Moreover, if \(\{v_1,v_2,\cdots,v_k\}\) is an orthonormal basis for \(W\), then
Here \(u\) is called the orthogonal projection of \(y\) on \(W\), and in \(W\) it is the closest vector to \(y\), i.e.,
Remark. By Replacement Theorem and Gram-Schmidt process, an orthogonal set \(S=\{v_1,v_2,\cdots,v_k\}\) in an \(n\)-dimensional inner product space \(V\) can be extended to an orthonormal basis \(\{v_1,v_2,\cdots,v_k,v_{k+1},\cdots,v_n\}\) for \(V\). Denote \(W=\text{span}(S)\), then \(S'=\{v_{k+1},\cdots,v_n\}\) is an orthonormal basis for \(W^{\perp}\).
Proposition. (1) Let \(W_1,W_2\) be subspaces of an inner product space \(V\). Then
Further, if \(V\) is finite-dimensional, then (apply (3) below)
(2) Let \(S,S_0\) be subsets of inner product space \(V\). Then
(3) Let \(W\) be a finite-dimensional subspace of an inner product space \(V\). Then for any \(x\notin W\), there exists \(y\in W^{\perp}\) such that \(\langle x,y\rangle\neq0\). Consequently, \((W^{\perp})^{\perp}=W\).
Without the assumption \(\dim W<\infty\) , the statement (3) is false.
Counterexample. Let \(F\) be any field. A sequence in \(F\) is a function
and is denoted by \(\{a_n\}\). Let \(V\) consist of all sequences \(\{a_n\}\) in \(F\) that have only a finite number of nonzero terms \(a_n\). If \(\{a_n\}\) and \(\{b_n\}\) are in \(V\) and \(t\in F\), define
With these operations \(V\) is a vector space.
Now let \(F=\mathbb{R}\) or \(\mathbb{C}\).
- Fisrt we define an inner product on \(V\) by
Note that since all but a finite number of terms of the series are \(0\), the series converges.
- Then we construct an orthonormal basis for \(V\).
Let \(e_n\) be a sequence s.t. \(e_n(k)=\delta_{n,k}\), where \(\delta_{n,k}\) is the Kronecker delta.
Clearly \(\{e_1,e_2,\cdots\}\) is an orthonormal basis for \(V\).
- Finally we construct an infinite-dimensional subspace of \(V\) that fails the statement.
Let \(\sigma(n)=e_1+e_n\) and \(W=\text{span}(\{\sigma_n:n\ge 2\})\).
Clearly \(e_1\notin W\), so \(W\neq V\). But \(W^{\perp}=\{0\}\) (as showed below), thus \((W^{\perp})^{\perp}\neq W\).
Remark. The statement (3) will hold whenever the orthogonal projection map of \(V\) on \(W\) exists, i.e., each vector in \(V\) has an orthogonal projection in \(W\). (https://math.stackexchange.com/questions/3567361/question-regarding-orthogonal-complement)
Orthogonal Complement in Functional Analysis
Problem-Solving
Problem. Let \(L\subset \mathbb{R}^{n+l}\ (n,l\in \mathbb{Z}_{\ge 1})\) be an \(n\)-dimensional subspace and let \(\{v_1,\cdots,v_{n+l}\}\) be a basis of \(\mathbb{R}^{n+l}\). Show that there exists \(1\le j_1<\cdots<j_n\le n+l\) such that the restriction \(P_{j_1,\cdots,j_n}|_L:L\to \mathbb{R}^{n+l}\) is injective (and hence induces an isomphism), where \(P_{j_1,\cdots,j_n}\) is the orthogonal projection of \(\mathbb{R}^{n+l}\) on \(\text{span}\{v_{j_1},\cdots,v_{j_n}\}\).
Proof I (Haosen). The orthogonal complement of the kernel of \(P_{j_1,\cdots,j_n}|_L\) is
Therefore \(P_{j_1,\cdots,j_n}|_L\) is injective iff \(\text{span}(\{v_{j_1},\cdots,v_{j_n}\})+L^{\perp}=\mathbb{R}^{n+l}\).
Let \(\{w_1,\cdots,w_l\}\) be a basis of the \(l\)-dimensional subspace \(L^{\perp}\). There exists \(A\in M_{(n+l)\times l}(\mathbb{R})\) of rank \(l\) such that
Now we note that there exists a sequence of elementary column operations that transforms \(A\) into a matrix with exactly \(n\) zero rows, in other words, there exists \(Q\in GL_{l}(\mathbb{R})\) such that \(AQ\) has exactly \(n\) zero rows, with the other \(l\) nonzero rows being linearly independent. We claim that the labels of the zero rows, denoted by \(1\le j_1<\cdots<j_n\le n+l\), will fulfill the proof. For clarity, let \(1\le i_1<\cdots<i_l\le n+l\) denote the labels of the nonzero rows.
In fact, \(L^{\perp}=\text{span}(\{w_1,\cdots,w_l\})=\) the column space of the matrix \((w_1\ \cdots\ w_l)=\) the column space of the matrix \((w_1\ \cdots\ w_l)Q=\) the column space of the matrix \((v_1\ \cdots\ v_{n+l})AQ=\text{span}(\{v_{i_1},\cdots,v_{i_l}\})\). Hence, \(\text{span}(\{v_{j_1},\cdots,v_{j_n}\})+L^{\perp}=\text{span}(\{v_1,\cdots,v_{n+l}\})=\mathbb{R}^{n+l}\). \(\blacksquare\)
Proof II (Shihe). This proof is geometrically intuitive. For each \(1\le k\le n+l\), take a nonzero vector \(w_k\in \mathbb{R}^{n+l}\) such that \(\text{span}(\{w_k\})=\text{span}(\{v_1,\cdots,v_{n+l}\}\setminus\{v_k\})^{\perp}\). We claim that \(w_1,\cdots,w_{n+l}\) are linearly independent.
In fact, if \(\sum_{k=1}^{n+l}\lambda_k w_k=0\) (\(\lambda_1,\cdots,\lambda_{n+l}\in \mathbb{R}\)), then by applying \(\langle \cdot,v_k\rangle\) to both sides we get \(\lambda_k\langle w_k,v_k\rangle=0\) (\(k=1,\cdots,n+l\)). Clearly \(\langle w_k,v_k\rangle\neq 0\), because otherwise \(w_k\) is orthogonal to every vector in the basis \(\{v_1,\cdots,v_{n+l}\}\) and hence must be zero, contradiction. Thus \(\lambda_k=0\ (k=1,\cdots,n+l)\) and hence \(w_1,\cdots,w_{n+l}\) are linearly independent.
Since \(\dim L=n<n+l\), there exists some \(w_{k_1}\) that is not in \(L\). The orthogonal projection of \(\mathbb{R}^{n+l}\) along \(\text{span}(\{w_{k_1}\})\), denoted by \(P_{w_{k_1}}:\mathbb{R}^{n+l}\to \mathbb{R}^{n+l}\), induces an isomorphism when restricted to \(L\), as \(\ker(P_{w_{k_1}}|_L)=\text{span}(\{w_{k_1}\})\cap L=0\). Moreover,
If \(l=1\), we're done. If \(l\ge 2\), then \(\dim P_{w_{k_1}}(L)=n<n+l-1\) and so there exists \(k_2\neq k_1\) such that \(w_{k_2}\notin P_{w_{k_1}}(L)\). The orthogonal projection of \(\text{im}(P_{k_1})\) along \(\text{span}(\{w_{k_2}\})\), denoted by \(P_{w_2}:\text{im}(P_{w_1})\to \text{im}(P_{w_1})\), induces an isomorphism when restricted to \(P_{w_{k_1}}(L)\), as \(\ker(P_{w_{k_2}}|_{P_{w_{k_1}}(L)})=\text{span}(\{w_{k_2}\})\cap P_{w_1}(L)=0\). Moreover,
Continue this process inductively until we get \(P_{w_l}\). Then
is a well-defined orthogonal projection of \(\mathbb{R}^{n+l}\) on \(\text{span}(\{v_1,\cdots,v_{n+l}\}\setminus\{v_{k_1},\cdots,v_{k_l}\})\), fulfilling the proof. \(\blacksquare\)
