Some Simple Models of Neurons
Linear neuron:
\[y=b+\sum\limits_i{x_i w_i}
\]
Binary threshold neuron:
\[z = \sum\limits_i{x_i w_i}
\]
\[y=\left\{\begin{aligned} 1,~~~~~~~z\gt \theta \\ 0,otherwise\end{aligned}\right.
\]
也可以写作：
\[z = \sum\limits_i{x_i w_i}
\]
\[y=\left\{\begin{aligned} 1,~~~~~~~z\gt 0 \\ 0, otherwise\end{aligned}\right.
\]
Rectified linear neuron:
\[z=b+\sum\limits_i{x_i w_i}
\]
\[y=\left\{\begin{aligned} 1,~~~~~~~z\gt 0 \\ 0, otherwise\end{aligned}\right.
\]
Sigmoid neuron:
\[z=b+\sum\limits_i{x_i w_i}
\]
\[y=\frac{1}{1+e^{-z}}
\]
Stochastic binary neuron:
we treat the output of the logistic as the probability of producing \(s=1\).
\[z = \sum\limits_i{x_i w_i}
\]
\[P(s=1)=\frac{1}{1+e^{-z}}
\]
Note：
- \(x\): neuron input.
- \(w\): input weight matrix.
- \(b\): bias term.
- \(y\): neuron output .
