SciTech-Mathmatics-Sinusoids正弦波/Harmonics(谐波) + FrequenciesSynthesis(频率合成) + Fourier Series(傅里叶级数):PeriodicalFunctions + (Discrete)FourierTransform:AllFunctions + Spectral Analysis

几何角度看，Fourier Series(傅立叶级数) 其实非常容易理解:

每一组 $(\cos (2 π n f t), \sin (2 π n f t))$ 三角函数都可看作决定一“谐波空间”的 $x$ 轴与 $y$ 轴;
Time-domain 的信号, 是用 Freq.-domain 的 N 个独立空间的谐波合成;
每一个“谐波空间”, 都是一对可配置 $(f r e q ., m a g n i t u d e, p h a s e)$ 的三角函数(cos/sin);
每一个“谐波空间”, 都有三个坐标轴: $(1, \cos (2 π n f t), \sin (2 π n f t))$ :
$\begin{array}{ll} x (t) & = \frac{a_{0}}{2} + \sum_{n = 1}^{\infty} [a_{n} \cos (2 π n f t) + b_{n} \sin (2 π n f t)] \\ \frac{a_{0}}{2} ⟶ 分解到基函数 1 (常数函数) 对应的坐标轴上的坐标 \\ a_{n} ⟶ 分解到基函数 \cos (2 π n f t) 对应的坐标轴上的坐标 \\ b_{n} ⟶ 分解到基函数 \sin (2 π n f t) 对应的坐标轴上的坐标 \end{array}$

$\begin{array}{ll} x (t) & = \int_{n = - \infty}^{+ \infty} S_{n} \cdot e^{i \frac{2 π n t}{T}} d t \\ S_{n} & = \frac{1}{T} \int_{t_{0}}^{t_{0} + T} x (t) \cdot e^{- i \frac{2 π n t}{T}} d t \\ S_{n} : 分解到正交基函数 (复函数) e^{i 2 π n f t} 坐标轴上的坐标 \\ w h e n & T \to \infty, 即频域谐波组频点无穷多, 就变成 “ 连续的频带 ” \\ F (ω) & = \frac{1}{2 π} \int_{- \infty}^{+ \infty} x (t) \cdot e^{- i ω t} d t \\ ω : 代表频率 \end{array}$
合成的 $X (t)$ 周期为 $T = \frac{1}{f}$ ,
是因为其 $k$ 个Frequency Components(频率成分) 的周期的最小公倍数为 T;
这些 Harmonics(Frequency Components) 谐波任意“加减”组合, 周期最小公倍数为 T.
下文 “Preliminaries.3. Frequency Components) for Synthesis” 有述;
内积(inner product of vectors) 与分解;
- 向量的内积, 在有限维向量空间, 两个向量的内积:
  $\begin{aligned} \vec{u} \cdot \vec{v} & = x_{1} x_{2} + y_{1} y_{2} \\ w h e r e & \vec{u} = (x_{1}, y_{1}), \vec{v} = (x_{2}, y_{2}) \end{aligned}$
- 函数的内积, 在无限维函数空间, 两个函数(连续)的内积(内积要重新定义,用到积分):
  $\begin{aligned} < f, g > & = \int_{t_{0}}^{t_{0} + T} f (t) g (t) d t \end{aligned}$
- 离散向量分解:
  将向量 $\vec{u}$ 分解到两个"正交向量" $\vec{v_{1}}$ 与 $\vec{v_{2}}$ 上:
  假设在( $\vec{v_{1}}$ , $\vec{v_{2}}$ )两坐标轴上坐标为( $c_{1}$ , $c_{2}$ ), 则有:
  $\begin{array}{ll} \vec{u} = c_{1} \vec{v_{1}} + c_{2} \vec{v_{2}} \\ 分解出的坐标 (c_{1}, c_{2}) 分别为 : \\ c_{1} = \frac{\vec{u} \cdot \vec{v_{1}}}{\vec{v_{1}} \cdot \vec{v_{1}}}, c_{2} = \frac{\vec{u} \cdot \vec{v_{2}}}{\vec{v_{2}} \cdot \vec{v_{2}}} \end{array}$
- 连续向量分解:
  将连续向量函数 $\vec{x (t)}$ 分解到两个"正交基函数" $\vec{\cos (2 π n f t)}$ 与 $\vec{\sin (2 π n f t)}$ 上:
  要将以上向量分解公式,由 有限维的向量空间 扩展到 无限维的函数空间;
  要将内积由 有限维的向量空间 扩展到 无限维的函数空间,
  并且, 要将每个三角函数(cos/sin)都看作一个“独立的坐标轴”,
  用到上文的 “函数的内积” 积分公式.
  但是为方便高效, 最好用 $E u l e r^{'} s E q u a t i o n$ 将$cos/sin转换为:
  $\begin{array}{ll} ∵ & s_{n} & = \frac{1}{T} \int_{t_{0}}^{t_{0} + T} x (t) \cdot e^{- i 2 π n f t} d t \\ T & = \infty \\ ∴ & F (ω) & = \frac{1}{2 π} \int_{- \infty}^{+ \infty} x (t) \cdot e^{- i ω t} d t \end{array}$
向量的正交与分解: 投影到正交的一堆向量为基的空间;
之所以选择分解到 $f (θ) = \cos θ$ 与 $f (θ) = \sin θ$ , 是因为这是“一对正交基函数”,
而且在任意时刻 $t_{0}$ 开始, 时长 $2 π$ 的 $\sin x \cos x$ 信号的积分都是 $0$ :
$\begin{aligned} < s i n, c o s > & = \int_{t_{0}}^{t_{0} + 2 π} \sin x \cos x d x = 0 \end{aligned}$

类比: 就像 $X O Y$ 平面的 $X 轴$ 与 $Y 轴$ 是正交的,
因此 $X O Y$ 平面上任何一点, 可用其分别在 $X 轴$ 与 $Y 轴$ 上的投影作为(x,y)坐标;

向量正交: 对于 $X O Y$ 平面上两个向量 $\vec{u} = (x_{1}, y_{1}), \vec{v} = (x_{2}, y_{2})$ ,
如果它们的内积为0, 夹角为 $90^{\circ}$ , 则称这两个向量点是“正交的”, 即 :
$\begin{aligned} \vec{u} \cdot \vec{v} = x_{1} x_{2} + y_{1} y_{2} = 0 \end{aligned}$

函数正交: 余弦函数 $\cos θ$ 与正弦函数 $\sin θ$ ,
可看作 无限维的 Hilbert 空间 的 无限维向量, 向量的各个“元”是连续区间的“函数值”;
无限维空间的向量内积需要用到积分重新定义为“两个连续函数的函数值先相乘后积分”:
$\begin{aligned} < f, g > & = \int_{x_{0}}^{x_{0} + T} f (x) g (x) d x \\ < s i n, c o s > & = \int_{t_{0}}^{t_{0} + 2 π} \sin t \cos t d t = 0 \end{aligned}$
分解 Time-domain 信号到 Frequency-domain,
总体思路上, 与“Taylor Series(泰勒级数)拟合任意N阶可导函数”类似:
- Taylor Series 是以无限项 $a_{j} (x - x_{0})^{j}$ (带可调参数的幂函数) 合成任意的 $f (x)$ ;
  确定系数 $a_{j}$ 与 $x_{0}$ 是用 $f^{'} (x), f^{″} (x), f^{3} (x), . . .,, f^{n} (x)$ 无限高阶导数进行拟合与误差估计;
- Fourier Series 是以无限项 $[a_{n} \cos (2 π n f t) + b_{n} \sin (2 π n f t)]$ (可配置系数的sin正弦信号发生器);
  确定系数 $a_{n}$ 与 $b_{n}$ 是用积分/复数运算进行拟合与误差估计;
实际上, 把 Time-domain 的 Signal 函数 $X (t)$ 分解/投影到 Frequency-domain 的多个“平面”(sin信号发生器):
- 每个“平面”有一对“正交基函数”(cos/sin), 实现上是一组( $N$ 个)“可配置系数”的sin信号发生器(谐波信号轮);
  每个“sin正弦信号轮”的用:
  - 半径(Radius)表示 $m a g n i t u d e$ 系数,
  - 用起始的距0弧度的bias量表示 $p h a s e$ 系数,
  - 以配置的常数的转动频率周期转动(产生信号)表示 Harmonic( $n f_{0}$ ) 固定的谐波频率;
  通过这一组( $N$ 个)“sin信号轮”以配置好的 $(m a g n i t u d e, p h a s e, h a r m o n i c)$ 同时转动(产生信号),
  可以合成任意的周期函数; 谐波信号轮个数越多( $N$ 越大)越精准;
- 参数 $(a_{j}, b_{j})$ 可以导出 $(m a g n i t u d e, p h a s e)$ , 因为 $f r e q u e n c y$ 这个系数,
  总是 Harmonic(谐波频点, “基频 $f$ ”的自然数倍), 所以可视为可变的常数;
  因此实际最需要确定的系数 $(m a g n i t u d e, p h a s e)$ 用 $(a_{j}, b_{j})$ 导出就足够;
  Sampling在任意时刻 $t_{0}$ , 最短采集( $T_{0} = \frac{1}{f_{0}}$ (the fundamental period)$时长就足够;
既可以选用 sin 系列的正弦函数, 也可以选用 cos系列的余弦函数;
之所以选用 cos 与 sin 两个函数作为“基函数”:
- 因为 cos 与 sin 这一对“基函数”正交.
- cos 与 sin 可以非常好的与 Complex Space 转换;
- cos 与 sin 可以用 Euler's Equation $e^{i θ} = \cos θ + i \sin θ$ 进行变换计算;
- cos 与 sin 在实现、转换、变换以及实际应用时非常方便实现.
  $c o s θ$ 可用 $s i n (θ - \frac{π}{2})$ 合成(即phase左移 $\frac{π}{2}$ );
  实现上, 只要做好一种可配置(freq., magnitude, phase)的 sin正弦信号发生器就可规模化量产;
- 总之优点众多.

Preliminaries

Periodic signals:

At least to begin, we’ll mainly be concerned with signals that are periodic.
Informally, a periodic signal is one that repeats, over and over, forever. To be more precise:
A signal $x (t)$ is said to be periodic if there exists some number $T$ ,
$\begin{array}{lll} such that \\ x (t) & = & x (t + T), for all t \\ T : the period of the signal \\ smallest T : the fundamental period \end{array}$ .
Sinusoids:

Precisely what do we mean by a sinusoid? The term “sinusoid” means a sine wave,
but we don’t just mean the standard $\sin (t)$ . To enable our analysis,
we want to be able to work sine waves of different $h e i g h t s$ , $w i d t h s$ and $p h a s e s$ .
So, to us, a single sinusoid means a function of the form
$\begin{array}{lll} x (t) & = A & \sin (2 π f t + ϕ) \\ for some \\ A & i t s a m p l i t u d e \\ f & i t s f r e q u e n c y, p e r i o d T = \frac{1}{f} \\ ϕ & i t s p h a s e \end{array}$ ,
Frequency Components for Synthesis:

$\begin{aligned} G I V I N G \\ x_{1} (t) & = & A_{1} \sin (1 \times (2 π f t + ϕ)), freq.: f, amplitude: A_{1}, phase: ϕ \\ x_{2} (t) & = & A_{2} \sin (2 \times (2 π f t + ϕ)), freq.: 2 f, amplitude: A_{2}, phase: 2 ϕ \\ x_{3} (t) & = & A_{3} \sin (3 \times (2 π f t + ϕ)), freq.: 3 f, amplitude: A_{3}, phase: 3 ϕ \\ . . . \\ x_{k} (t) & = & A_{k} \sin (k \times (2 π f t + ϕ)), freq.: n f, amplitude: A_{k}, phase: k ϕ \\ X (t) & = & \sum_{n = 1}^{k} x_{n} (t) \\ = & \sum_{n = 1}^{k} A_{n} \sin [n (2 π f t + ϕ)] \end{aligned}$
$THEN the period o f X (t) is T = \frac{1}{f}, SINCE$
$\begin{aligned} p e r i o d (x_{1}) & = T & = \frac{1}{f}, \\ p e r i o d (x_{2}) & = \frac{T}{2} & = \frac{1}{2 f}, \\ p e r i o d (x_{3}) & = \frac{T}{3} & = \frac{1}{3 f}, \\ . . . \\ p e r i o d (x_{k}) & = \frac{T}{k} & = \frac{1}{k f}, \end{aligned}$
Harmonics/Sinusoids:

The sinusoidal terms are often called $harmonics$ , a term borrowed from music.
The harmonics will have frequencies $f, 2 f, 3 f, 4 f$ and so on.
We also call each $h a r m o n i c$ , $A_{n} \sin (2 π n f t + ϕ n)$ , the $frequency component$ of $x (t)$ at frequency $n f$ .
For example, if $f = 10 H z$ , we call the $h a r m o n i c$ for which
$n = 3$ the “ $30 H z c o m p o n e n t$ ”, reflecting that
in this case $\sin (2 π n f t + ϕ n)$ is a sinusoid of frequency $30 H z$
Frequency-domain and Time-domain Representation:
- the frequency-domain representation of the periodic signal:
  represent a signal using the magnitudes and phases in its Fourier series.
  We often plot the magnitudes in the Fourier series,
  using a stem graph and labeling the frequency axis by frequency.
  In this sense, this representation is a function of frequency.
- the time-domain representation of the signal:
  represent a periodic signal as a function of time in its Fourier series.
- Example:

The Fourier Series:

Joseph Fourier’s idea was to express periodic signals as a sum of sinusoids.
Theorem.
- If $x (t)$ is a $well-behaved periodic signal$ with $period T$ ,
  
  $We call following sum t h e Fourier series of x (t)$
  $\begin{array}{ll} x (t) & = \begin{matrix} (1F) & A_{0} + \sum_{n = 1}^{\infty} A_{n} \sin (2 π n f t + n ϕ) \end{matrix} \\ frequency : f = 1 / T, \\ magnitudes : A_{i}, i \in [1, + \infty] \\ phases ϕ_{i}, i \in [1, + \infty] \end{array}$
  Another fact relating to Fourier series is that:
  the $m a g n i t u d e s A_{0}, A_{1}, A_{2}, . . .$ and $p h a s e s ϕ_{1}, ϕ_{2}, . . .$
  in above equation uniquely determine $x (t)$ . That is, if we can find these
  $m a g n i t u d e s$ and $p h a s e s$ corresponding to a periodic signal $x (t)$ ,
  then, in effect, we have another way of describing $x (t)$ .
- $Another useful form of Fourier series of x (t)$
  $\begin{array}{ll} x (t) & = \begin{matrix} (2F) & \frac{a_{0}}{2} + \sum_{n = 1}^{\infty} [a_{n} \cos (2 π n f t) + b_{n} \sin (2 π n f t)] \end{matrix} \\ a_{0} = 2 A_{0}, \\ a_{n} = A_{n} \cos (n ϕ), b_{n} = A_{n} \sin (n ϕ), \\ (a_{n})^{2} + (b_{n})^{2} = (A_{n})^{2} \\ above coefficients can then be found, \\ using the following integrals, where T = \frac{1}{f} : \\ a_{n} = \frac{2}{T} \int_{0}^{T} x (t) \cos (2 π n f t) d t \\ b_{n} = \frac{2}{T} \int_{0}^{T} x (t) \sin (2 π n f t) d t \\ and a_{0}, a_{1}, a_{2}, . . . and b_{1}, b_{2}, . . . \\ can be converted to m a g n i t u d e s and p h a s e s, \\ to fit above form (1F) . \end{array}$
  
  $Proof of form (1F) and form (2F) are the same$
  $\begin{array}{lll} ∵ & \sin (X + Y) = \sin X \cos Y + \cos X s i n Y \\ ∴ & \sin (2 π n f t + n ϕ) = \sin (2 π n f t) \cos (n ϕ) + \cos (2 π n f t) s i n (n ϕ) \\ x (t) & = A_{0} + \sum_{n = 1}^{\infty} A_{n} \sin (2 π n f t + n ϕ) \\ = A_{0} + \sum_{n = 1}^{\infty} (A_{n} \cos (n ϕ)) \sin (2 π n f t) + (A_{n} \sin (n ϕ)) \cos (2 π n f t)] \\ = \frac{a_{0}}{2} + \sum_{n = 1}^{\infty} [a_{n} \cos (2 π n f t) + b_{n} \sin (2 π n f t)] \end{array}$
- $Complex form of Fourier series of x (t)$
  $\begin{array}{lll} ∵ e^{i θ} & = \cos θ + i \sin θ, the Euler's Equation \\ ∴ e^{i 2 π n f t} & = \cos (2 π n f t) + i \sin (2 π n f t) \\ x (t) & = \frac{a_{0}}{2} + \sum_{n = 1}^{\infty} [a_{n} \cos (2 π n f t) + b_{n} \sin (2 π n f t)] \\ = \frac{1}{T} \int_{t_{0}}^{t_{0} + T} x (t) \cdot e^{- i 2 π n f t} d t \end{array}$
  
  $\begin{array}{ll} ∵ & S_{n} & = \frac{1}{T} \int_{t_{0}}^{t_{0} + T} x (t) \cdot e^{- i 2 π n f t} d t \\ T & = \infty \\ ∴ & F (ω) & = \frac{1}{2 π} \int_{- \infty}^{+ \infty} x (t) \cdot e^{- i ω t} d t \end{array}$

Time Domain + Frequency Domain

and Fourier Transforms @ Princeton University:

Fourier Series: Periodical Functions

Fourier Transform: All Functions
Frequency Domain and Fourier Transforms @ Princeton University:
https://www.princeton.edu/~cuff/ele201/kulkarni_text/frequency.pdf

Signals and the frequency domain

ENGR 40M lecture notes — July 31, 2017 Chuan-Zheng Lee, Stanford University
https://web.stanford.edu/class/archive/engr/engr40m.1178/slides/signals.pdf

https://www.mathworks.com/help/signal/ug/extract-regions-of-interest-from-whale-song.html

https://web.stanford.edu/class/stats253/lectures_2014/lect7.pdf
https://web.stanford.edu/class/stats253/lectures_2014/
https://resources.pcb.cadence.com/blog/2020-time-domain-analysis-vs-frequency-domain-analysis-a-guide-and-comparison

Time Domain and Frequency Domain:
- Illustrations:
  | | | |
  | ---- | ---- | ---- |
  | | | |

Application: The role of ECoG magnitude and phase in decoding position, velocity, and acceleration during continuous motor behavior

Frequency Domain and Fourier Transforms @ Princeton University:
https://www.princeton.edu/~cuff/ele201/kulkarni_text/frequency.pdf
Signals @ WEB.stanford.edu
https://web.stanford.edu/class/archive/engr/engr40m.1178/slides/signals.pdf
https://web.stanford.edu/class/stats253/lectures_2014/lect7.pdf

The Frequency Domain

(Discrete) Fourier Transform

Spectral Analysis

This is a new course.
• The material that we will be covering has not really been synthesized—because it is at the frontiers of statistics!
• We will be loosely following the books
• Shumway and Stoffer. Time Series Analysis and Applications (with R
Applications).
• Sherman. Spatial Statistics and Spatio-Temporal Data.
• You don’t have to purchase these books: they are available for free for Stanford students. (Link on course website.)
• Other useful references:
• Bivand et al. Applied Spatial Data Analysis with R. (also available free) • Cressie and Wikle. Statistics for Spatio-Temporal Data.