3.1 公式3.8证明

关键点：用matrix notation对矩阵迹进行变换
$\hat{y}=X{(X^TX)}^{-1}X^Ty=Hy$

$\sum_i^n {(y_i-\hat{y}_i)}^2={(y-Hy)}^T(y-Hy)=y^Ty-y^THy-y^THy+y^TH^THy\\ \ \ =y^Ty-y^THy$ 其中$H^TH=H$

$E(y^Ty)=E(\sum_i^ny_i^2)=\sum_i^n (\sigma^2+C_i)=n\sigma^2+\sum_i C_i^2=n\sigma^2+C^TC$ 其中$E(y_i)=C_i$

$E(y^THy)=E(\sum_i\sum_jy_iy_jH_{ij})=E(\sum_i y_i^2H_{ii}+\sum_i\sum_{j\neq i}y_iy_jH_{ij})\\ \ \ =\sigma^2\sum_i H_{ii}+\sum_i C_i^2H_{ii}+\sum_i\sum_{j\neq i}C_iC_jH_{ij}\\ \ \ =\sigma^2\sum_i H_{ii}+\sum_i\sum_jC_iC_jH_{ij}\\ \ \ =\sigma^2\sum_i H_{ii}+C^THC$

$C=X\beta$
$C^THC=\beta^TX^TX{(X^TX)}^{-1}X^TX\beta=C^TC$

$\sum_iH_{ii}=\sum_i\left[X{(X^TX)}^{-1}X^T\right]_{ii}=\sum_i \sum_j \left[X{(X^TX)}^{-1}\right]_{ij}X_{ij}\\ \ \ =\sum_i \sum_j \sum_k X_{ik}{(X^TX)}_{kj}^{-1}X_{ij}\\ \ \ =\sum_j\sum_k \left[X^TX\right]_{kj}{(X^TX)}_{kj}^{-1}=\sum_j\left[X^TX{(X^TX)}^{-1}\right]_{jj}\\ \ \ =\sum_j I_{jj}=p+1$

所以$E(\sum_i^n {(y_i-\hat{y}_i)}^2)=(n-p-1)\sigma^2$