Matrix Calculus

1 Scalar Function

$\text{If }f(\mathbf{x})\in \mathbf{R},\mathrm{then}$

$$df=\frac{\partial f}{\partial x}dx+\frac{\partial f}{\partial y}dy+\frac{\partial f}{\partial z}dz=\left[\begin{array}{ccc}\frac{\partial f}{\partial x}& \frac{\partial f}{\partial y}& \frac{\partial f}{\partial z}\end{array}\right]\left[\begin{array}{c}dx\\ dy\\ dz\end{array}\right]=f(\mathbf{x}{)}^{\prime }d\mathbf{x}.$$

1.1 Derivative

So

$${\displaystyle \frac{\partial f}{\partial \mathbf{x}}}=\left[\begin{array}{ccc}{\displaystyle \frac{\partial f}{\partial x}}& {\displaystyle \frac{\partial f}{\partial y}}& {\displaystyle \frac{\partial f}{\partial z}}\end{array}\right]=f(\mathbf{x}{)}^{\prime }$$

这里用的是 Numerator layout.

1.2 Gradient

$${\mathrm{\nabla }f={\left(\frac{\partial f}{\partial \mathbf{x}}\right)}^{\mathsf{T}}=[\begin{array}{ccc}\frac{\partial f}{\partial x}& \frac{\partial f}{\partial y}& \frac{\partial f}{\partial z}\end{array}]}^{\mathsf{T}}=\left[\begin{array}{c}\frac{\partial f}{\partial x}\\ \frac{\partial f}{\partial y}\\ \frac{\partial f}{\partial z}\end{array}\right].$$

2 Vector Function

$\text{if }\mathbf{f}(\mathbf{x})=\left[\begin{array}{c}f(\mathbf{x})\\ g(\mathbf{x})\\ h(\mathbf{x})\end{array}\right]\in {\mathbf{R}}^3,\text{then:}$

2.1 Jacobian

$$\mathbf{J}(\mathbf{x})={\displaystyle \frac{\partial \mathbf{f}}{\partial \mathbf{x}}}=\left[\begin{array}{ccc}\frac{\partial f}{\partial x}& \frac{\partial f}{\partial y}& \frac{\partial f}{\partial z}\\ \frac{\partial g}{\partial x}& \frac{\partial g}{\partial y}& \frac{\partial g}{\partial z}\\ \frac{\partial h}{\partial x}& \frac{\partial h}{\partial y}& \frac{\partial h}{\partial z}\end{array}\right]$$

2.2 Divergence

$$\mathrm{\nabla }\cdot \mathbf{f}=\frac{\partial f}{\partial x}+\frac{\partial g}{\partial y}+\frac{\partial h}{\partial z}$$

2.3 Curl

$$\mathrm{\nabla }\times \mathbf{f}=\left[\begin{array}{c}\frac{\partial h}{\partial y}-\frac{\partial g}{\partial z}\\ \frac{\partial f}{\partial z}-\frac{\partial h}{\partial x}\\ \frac{\partial g}{\partial x}-\frac{\partial f}{\partial y}\end{array}\right]$$

2.4 Hessian

$$\mathbf{H}=\mathbf{J}(\mathrm{\nabla }f(\mathbf{x}))=\left[\begin{array}{ccc}\frac{{\partial }^{2}f}{\partial x^2}& \frac{{\partial }^{2}f}{\partial x\partial y}& \frac{{\partial }^{2}f}{\partial x\partial z}\\ \frac{{\partial }^{2}f}{\partial y\partial x}& \frac{{\partial }^{2}f}{\partial y^2}& \frac{{\partial }^{2}f}{\partial y\partial z}\\ \frac{{\partial }^{2}f}{\partial z\partial x}& \frac{{\partial }^{2}f}{\partial z\partial y}& \frac{{\partial }^{2}f}{\partial z^2}\end{array}\right]$$

https://www.value-at-risk.net/functions/

X ref

1. HKU : Vector calculus for engineers
2. MIT : s096-matrix-calculus-for-machine-learning
3. GAMES103 : Physics-Based Animation
4. Wikipedia : Matrix Calculus