【489】高级的深度学习最佳实践
参考：Deep Learning with Python P196
一、不用 Sequential 模型的解决方案：Keras 函数式 API
- 一个多输入模型
- 一个多输出（或多头）模型
1.1 函数式 API 简介
都是按照输入输出的模式，以下两种模式是一致的。
from keras.models import Sequential, Model from keras import layers, Input seq_model = Sequential() seq_model.add(layers.Dense(32, activation='relu', input_shape=(64,))) seq_model.add(layers.Dense(32, activation='relu')) seq_model.add(layers.Dense(10, activation='softmax')) input_tensor = Input(shape=(64,)) h1 = layers.Dense(32, activation='relu')(input_tensor) h2 = layers.Dense(32, activation='relu')(h1) output_tensor = layers.Dense(10, activation='softmax')(h2) model = Model(input_tensor, output_tensor) model.summary()
output
_________________________________________________________________ Layer (type) Output Shape Param # ================================================================= input_1 (InputLayer) (None, 64) 0 _________________________________________________________________ dense_1 (Dense) (None, 32) 2080 _________________________________________________________________ dense_2 (Dense) (None, 32) 1056 _________________________________________________________________ dense_3 (Dense) (None, 10) 330 ================================================================= Total params: 3,466 Trainable params: 3,466 Non-trainable params: 0 _________________________________________________________________
对这种 Model 实例进行编译、训练或评估时，其 API 与 Sequential 模型相同。
1.2 多输入模型
典型的问答模型有两个输入：一个自然语言描述的问题和一个文本片段（比如新闻文章），后者提供用于回答问题的信息。然后模型要生成一个回答，在最简单的情况下，这个回答只包含一个词，可以通过对摸个预定义的词表做softmax得到。
输入：问题 + 文本片段
输出：回答（一个词）
from keras.models import Model
from keras import layers
from keras import Input
text_vocabulary_size = 10000
question_vocabulary_size = 10000
answer_vocabulary_size = 500
text_input = Input(shape=(None,),
dtype='int32',
name='text')
embeded_text = layers.Embedding(text_vocabulary_size,64)(text_input)
encoded_text = layers.LSTM(32)(embeded_text)
question_input = Input(shape=(None,),
dtype = 'int32',
name = 'question')
embeded_question = layers.Embedding(question_vocabulary_size,32)(question_input)
encoded_question = layers.LSTM(16)(embeded_question)
concatenated = layers.concatenate([encoded_text,encoded_question],axis=-1)
answer = layers.Dense(answer_vocabulary_size,activation='softmax')(concatenated)
model = Model([text_input,question_input],answer)
model.compile(optimizer='rmsprop',
loss = 'categorical_crossentropy',
metrics = ['acc'])
model.summary()
output:
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
text (InputLayer) (None, None) 0
__________________________________________________________________________________________________
question (InputLayer) (None, None) 0
__________________________________________________________________________________________________
embedding_5 (Embedding) (None, None, 64) 640000 text[0][0]
__________________________________________________________________________________________________
embedding_6 (Embedding) (None, None, 32) 320000 question[0][0]
__________________________________________________________________________________________________
lstm_5 (LSTM) (None, 32) 12416 embedding_5[0][0]
__________________________________________________________________________________________________
lstm_6 (LSTM) (None, 16) 3136 embedding_6[0][0]
__________________________________________________________________________________________________
concatenate_3 (Concatenate) (None, 48) 0 lstm_5[0][0]
lstm_6[0][0]
__________________________________________________________________________________________________
dense_6 (Dense) (None, 500) 24500 concatenate_3[0][0]
==================================================================================================
Total params: 1,000,052
Trainable params: 1,000,052
Non-trainable params: 0
__________________________________________________________________________________________________
训练这种模型需要能够对网络的各个头指定不同的损失函数，例如：年龄预测是标量回归任务,而性别预测是二分类任务，二者需要不同的损失过程。 但是，梯度下降要求将一个标量最小化,所以为了能够训练模型，我们必须将这些损失合并为单个标量。合并不同损失最简单的方法就是对所有损失求和。
#多输出模型的编译选项：多重损失
#方法一
model.compile(optimizer='rmsprop',
loss = ['mse','categorical_crossentropy','binary_crossentropy'])
#方法二
# model.compile(optimizer='rmsprop',
# loss={'age':'mse',
# 'income':'categorical_crossentropy',
# 'gender':'binary_crossentropy'})
#多输出模型的编译选项：损失加权
#方法一
model.compile(optimizer='rmsprop',
loss = ['mse','categorical_crossentropy',<br>'binary_crossentropy'],
loss_weights=[0.25,1.,10.])
#方法二
# model.compile(optimizer='rmsprop',
# loss={'age':'mse',
# 'income':'categorical_crossentropy',
# 'gender':'binary_crossentropy'},
# loss_weights={'age':0.25,
# 'income':1.,
# 'gender':10.})
不同的损失值具有不同的取值范围，为了平衡不同损失的贡献，应该对loss_weights进行设置
#将数据输入到多输出模型中
#方法一
model.fit(posts,[age_targets,income_targets,gender_targets],
epochs=10,batch_size=64)
#方法二
# model.fit(posts,{'age':age_targets,
# 'income':income_targets,
# 'gender':gender_targets},
# epochs=10,batch_size=64)
