【508】NLP实战系列（五）—— 通过 SimpleRNN 来做分类

参考：SimpleRNN层

1. 语法

keras.layers.GRU(units, activation='tanh', recurrent_activation='hard_sigmoid', use_bias=True, kernel_initializer='glorot_uniform', recurrent_initializer='orthogonal', bias_initializer='zeros', kernel_regularizer=None, recurrent_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, recurrent_constraint=None, bias_constraint=None, dropout=0.0, recurrent_dropout=0.0, implementation=1, return_sequences=False, return_state=False, go_backwards=False, stateful=False, unroll=False)

2. 参数

• units：输出维度

• activation：激活函数，为预定义的激活函数名（参考激活函数）

• use_bias: 布尔值，是否使用偏置项

• kernel_initializer：权值初始化方法，为预定义初始化方法名的字符串，或用于初始化权重的初始化器。参考initializers

• recurrent_initializer：循环核的初始化方法，为预定义初始化方法名的字符串，或用于初始化权重的初始化器。参考initializers

• bias_initializer：权值初始化方法，为预定义初始化方法名的字符串，或用于初始化权重的初始化器。参考initializers

• kernel_regularizer：施加在权重上的正则项，为Regularizer对象

• bias_regularizer：施加在偏置向量上的正则项，为Regularizer对象

• recurrent_regularizer：施加在循环核上的正则项，为Regularizer对象

• activity_regularizer：施加在输出上的正则项，为Regularizer对象

• kernel_constraints：施加在权重上的约束项，为Constraints对象

• recurrent_constraints：施加在循环核上的约束项，为Constraints对象

• bias_constraints：施加在偏置上的约束项，为Constraints对象

• dropout：0~1之间的浮点数，控制输入线性变换的神经元断开比例

• recurrent_dropout：0~1之间的浮点数，控制循环状态的线性变换的神经元断开比例

• 其他参数参考Recurrent的说明

3. 相关说明

• SimpleRNN takes inputs of shape (batch_size, timesteps, input_features).

• Like all recurrent layers in Keras, SimpleRNN can be run in two different modes: it can return either the full sequences of successive outputs for each timestep (a 3D tensor of shape (batch_size, timesteps, output_features)), or it can return only the last output for each input sequence (a 2D tensor of shape (batch_size, output_features)). These two modes are controlled by the return_sequences constructor argument. Let's take a look at an example:

 　　当将 SimpleRNN 叠加的时候，return_sequences 就有用了。

model = Sequential()
model.add(Embedding(10000, 32))
model.add(SimpleRNN(32, return_sequences=True))
model.add(SimpleRNN(32, return_sequences=True))
model.add(SimpleRNN(32, return_sequences=True))
model.add(SimpleRNN(32))  # This last layer only returns the last outputs.
model.summary()

　　outputs：

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding_3 (Embedding)      (None, None, 32)          320000    
_________________________________________________________________
simple_rnn_3 (SimpleRNN)     (None, None, 32)          2080      
_________________________________________________________________
simple_rnn_4 (SimpleRNN)     (None, None, 32)          2080      
_________________________________________________________________
simple_rnn_5 (SimpleRNN)     (None, None, 32)          2080      
_________________________________________________________________
simple_rnn_6 (SimpleRNN)     (None, 32)                2080      
=================================================================
Total params: 328,320
Trainable params: 328,320
Non-trainable params: 0
_________________________________________________________________

4. 具体实现 

4.1 加载数据

from keras.datasets import imdb
from keras.preprocessing import sequence

max_features = 10000  # number of words to consider as features
maxlen = 500  # cut texts after this number of words (among top max_features most common words)
batch_size = 32

print('Loading data...')
(input_train, y_train), (input_test, y_test) = imdb.load_data(num_words=max_features)
print(len(input_train), 'train sequences')
print(len(input_test), 'test sequences')

print('Pad sequences (samples x time)')
input_train = sequence.pad_sequences(input_train, maxlen=maxlen)
input_test = sequence.pad_sequences(input_test, maxlen=maxlen)
print('input_train shape:', input_train.shape)
print('input_test shape:', input_test.shape)

　　output：

Loading data...
25000 train sequences
25000 test sequences
Pad sequences (samples x time)
input_train shape: (25000, 500)
input_test shape: (25000, 500)

4.2 数据训练 

from keras.layers import Dense

model = Sequential()
model.add(Embedding(max_features, 32))
model.add(SimpleRNN(32))
model.add(Dense(1, activation='sigmoid'))

model.compile(optimizer='rmsprop', loss='binary_crossentropy', metrics=['acc'])
history = model.fit(input_train, y_train,
                    epochs=10,
                    batch_size=128,
                    validation_split=0.2)

　　outputs：

Train on 20000 samples, validate on 5000 samples
Epoch 1/10
20000/20000 [==============================] - 22s - loss: 0.6455 - acc: 0.6210 - val_loss: 0.5293 - val_acc: 0.7758
Epoch 2/10
20000/20000 [==============================] - 20s - loss: 0.4005 - acc: 0.8362 - val_loss: 0.4752 - val_acc: 0.7742
Epoch 3/10
20000/20000 [==============================] - 19s - loss: 0.2739 - acc: 0.8920 - val_loss: 0.4947 - val_acc: 0.8064
Epoch 4/10
20000/20000 [==============================] - 19s - loss: 0.1916 - acc: 0.9290 - val_loss: 0.3783 - val_acc: 0.8460
Epoch 5/10
20000/20000 [==============================] - 19s - loss: 0.1308 - acc: 0.9528 - val_loss: 0.5755 - val_acc: 0.7376
Epoch 6/10
20000/20000 [==============================] - 19s - loss: 0.0924 - acc: 0.9675 - val_loss: 0.5829 - val_acc: 0.7634
Epoch 7/10
20000/20000 [==============================] - 19s - loss: 0.0726 - acc: 0.9768 - val_loss: 0.5541 - val_acc: 0.7932
Epoch 8/10
20000/20000 [==============================] - 19s - loss: 0.0426 - acc: 0.9862 - val_loss: 0.5551 - val_acc: 0.8292
Epoch 9/10
20000/20000 [==============================] - 20s - loss: 0.0300 - acc: 0.9918 - val_loss: 0.5962 - val_acc: 0.8312
Epoch 10/10
20000/20000 [==============================] - 19s - loss: 0.0256 - acc: 0.9925 - val_loss: 0.6707 - val_acc: 0.8054