Lecture8 Neural NetWork:Representation
Lecture8 Neural NetWork:Representation
Non-linear hypotheses
Neural NetWork
\(a_i^{(j)}\) = "activation" of unit i in layer j
\(\Theta^{(j)}\) = matrix of weights controlling function mapping from layer \(j\) to layer \(j+1\)
\[\begin{align}
a_1^{(2)} & = g(\Theta_{10}^{(1)}x_0+\Theta_{11}^{(1)}x_1+\Theta_{12}^{(1)}x_2+\Theta_{13}^{(1)}x_3) \\
a_2^{(2)} & = g(\Theta_{20}^{(1)}x_0+\Theta_{21}^{(1)}x_1+\Theta_{22}^{(1)}x_2+\Theta_{23}^{(1)}x_3) \\
a_1^{(3)} & = g(\Theta_{30}^{(1)}x_0+\Theta_{31}^{(1)}x_1+\Theta_{32}^{(1)}x_2+\Theta_{33}^{(1)}x_3) \\
h_\Theta(x) & = a_1^{(3)}=g(\Theta_{10}^{(2)}a_0^{(2)}+\Theta_{11}^{(2)}a_1^{(2)}+\Theta_{12}^{(2)}a_2^{(2)}+\Theta_{13}^{(2)}a_3^{(2)})
\end{align}
\]
if network has \(s_j\) units in layer j,\(s_{j+1}\) units in layer j+1,then \(\Theta^{(j)}\) will be of dimension \(s_{j+1} \times (s_j+1)\)
Vectorized implementation
\[x = \left[\begin{matrix}
x_0 \\
x_1 \\
x_3 \\
x_4 \\
\end{matrix}\right]
\
z^{(2)} = \left[\begin{matrix}
z_1^{(2)} \\
z_2^{(2)} \\
z_3^{(2)}
\end{matrix}\right]
\\
z^{(2)} = \Theta^{(1)}a^{(1)} \\
a^{(2)} = g(z^{(2)}) \\
Add\ a_0^{(2)} = 1 \Rightarrow a^{(2)} \in \mathbb{R}^4\\
z^{(3)} = \Theta^{(2)})a^{(2)} \\
h_\Theta(x) = a^{(3)} = g(z^{(3)})
\]
Examples and intuitions
Non-linear classification example: XOR/XNOR
Simple example: AND
\(x_1,x_3 \in {0,1}\)
\(y = x1\ AND\ x_2\)
\[\Theta_{10} = -30 \\
\Theta_{11} = 20 \\
\Theta_{11} = 20 \\
h_\Theta(x) = g(-30 + 20x_1+20x_2)
\]
\(x_1\) | \(x_2\) | \(h_\Theta(x)\) |
---|---|---|
0 | 0 | \(g(-30)\approx0\) |
0 | 1 | \(g(-10)\approx0\) |
1 | 0 | \(g(-10)\approx0\) |
1 | 1 | \(g(10)\approx0\) |
Example: OR function
\[h_\Theta(x) = g(-10+20x_1+20x_2)
\]
\(x_1\) | \(x_2\) | \(h_\Theta(x)\) |
---|---|---|
0 | 0 | \(g(-10)\approx0\) |
0 | 1 | \(g(10)\approx1\) |
1 | 0 | \(g(10)\approx1\) |
1 | 1 | \(g(30)\approx1\) |
Negation
XOR
本文来自博客园,作者:Un-Defined,转载请保留本文署名Un-Defined,并在文章顶部注明原文链接:https://www.cnblogs.com/EIPsilly/p/15698248.html