LSTM&GRU原理及pytroch实现

1.LSTM&GRU的原理

https://blog.csdn.net/jerr__y/article/details/58598296

https://github.com/starflyyy/Gated-Recurrent-Unit-GRU

2.多层LSTM

pytorch里有一个num_layers,是指参数共享之后网络也有不同cell，即相当于隐含层的数目，是指cell串联和mlp很像，即为StackedRNN具体如下图

 

3.双向RNN

待继续了解https://blog.csdn.net/jojozhangju/article/details/51982254

4.lstm&rnn的实现和改造

源码分析https://zhuanlan.zhihu.com/p/63638656 

例子分析https://zhuanlan.zhihu.com/p/32103001，可以加上自己改写RNNcellhttps://github.com/huyingxi/new-LSTM-Cell

pytorch中rnn的输出有output和hidden，区别如下图（lstm）

其中：

Outputs: output, (h_n, c_n)

• output (seq_len, batch, hidden_size * num_directions): tensor containing the output features (h_t) from the last layer of the RNN, for each t. If a torch.nn.utils.rnn.PackedSequence has been given as the input, the output will also be a packed sequence.
• h_n (num_layers * num_directions, batch, hidden_size): tensor containing the hidden state for t=seq_len
• c_n (num_layer, batch, hidden_size): tensor containing the cell state for t=seq_len

一种实现，但其实现的多层lstm不太对https://github.com/emadRad/lstm-gru-pytorch/