tflearn 中文汉字识别模型试验汇总
def get_model(width, height, classes=40): # TODO, modify model # Building 'VGG Network' network = input_data(shape=[None, width, height, 1]) # if RGB, 224,224,3 network = conv_2d(network, 64, 3, activation='relu') #network = conv_2d(network, 64, 3, activation='relu') network = max_pool_2d(network, 2, strides=2) network = conv_2d(network, 128, 3, activation='relu') #network = conv_2d(network, 128, 3, activation='relu') network = max_pool_2d(network, 2, strides=2) network = conv_2d(network, 256, 3, activation='relu') #network = conv_2d(network, 256, 3, activation='relu') #network = conv_2d(network, 256, 3, activation='relu') network = max_pool_2d(network, 2, strides=2) network = conv_2d(network, 512, 3, 2, activation='relu') # network = conv_2d(network, 512, 3, activation='relu') # network = conv_2d(network, 512, 3, activation='relu') # network = max_pool_2d(network, 2, strides=2) # network = conv_2d(network, 512, 3, activation='relu') # network = conv_2d(network, 512, 3, activation='relu') # network = conv_2d(network, 512, 3, activation='relu') network = max_pool_2d(network, 2, strides=2) # network = fully_connected(network, 4096, activation='relu') # network = dropout(network, 0.5) #network = fully_connected(network, 1024, activation='relu') network = fully_connected(network, 2048, activation='relu') network = dropout(network, 0.8) network = fully_connected(network, classes, activation='softmax') network = regression(network, optimizer='rmsprop', loss='categorical_crossentropy', learning_rate=0.0001) model = tflearn.DNN(network, checkpoint_path='checkpoint', max_checkpoints=1, tensorboard_verbose=1) return model
if __name__ == "__main__": width, height = 32, 32 X, Y, org_labels = load_data(dirname="data", resize_pics=(width, height)) trainX, testX, trainY, testY = train_test_split(X, Y, test_size=0.2, random_state=666) print("sample data:") print(trainX[0]) print(trainY[0]) print(testX[-1]) print(testY[-1]) model = get_model(width, height, classes=100) filename = 'cnn_handwrite-acc0.8.tflearn' # try to load model and resume training #try: # model.load(filename) # print("Model loaded OK. Resume training!") #except: # pass # Initialize our callback with desired accuracy threshold. early_stopping_cb = EarlyStoppingCallback(val_acc_thresh=0.9) try: model.fit(trainX, trainY, validation_set=(testX, testY), n_epoch=500, shuffle=True, snapshot_epoch=True, # Snapshot (save & evaluate) model every epoch. show_metric=True, batch_size=32, callbacks=early_stopping_cb, run_id='cnn_handwrite') except StopIteration as e: print("OK, stop iterate!Good!") model.save(filename) # predict all data and calculate confusion_matrix model.load(filename) pro_arr =model.predict(X) predict_labels = np.argmax(pro_arr, axis=1) print(classification_report(org_labels, predict_labels)) print(confusion_matrix(org_labels, predict_labels))
上述模型效果:
---------------------------------
Training samples: 19094
Validation samples: 4774
--
Training Step: 597 | total loss: 3.60744 | time: 110.471s
| RMSProp | epoch: 001 | loss: 3.60744 - acc: 0.1455 | val_loss: 3.64326 - val_acc: 0.1257 -- iter: 19094/19094
--
Terminating training at the end of epoch 1
Training Step: 1194 | total loss: 1.74615 | time: 115.902s
| RMSProp | epoch: 002 | loss: 1.74615 - acc: 0.4955 | val_loss: 1.56680 - val_acc: 0.5840 -- iter: 19094/19094
--
Terminating training at the end of epoch 2
Training Step: 1791 | total loss: 1.06401 | time: 117.538s
| RMSProp | epoch: 003 | loss: 1.06401 - acc: 0.7183 | val_loss: 1.02607 - val_acc: 0.6986 -- iter: 19094/19094
。。。
试试mnist直接拿过来:
def get_model(width, height, classes=40): # TODO, modify model # Real-time data preprocessing img_prep = tflearn.ImagePreprocessing() img_prep.add_featurewise_zero_center(per_channel=True) network = input_data(shape=[None, width, height, 1]) #, data_preprocessing=img_prep) # if RGB, 224,224,3 network = conv_2d(network, 32, 3, activation='relu', regularizer="L2") network = max_pool_2d(network, 2) network = local_response_normalization(network) network = conv_2d(network, 64, 3, activation='relu', regularizer="L2") network = max_pool_2d(network, 2) network = local_response_normalization(network) network = fully_connected(network, 128, activation='tanh') network = dropout(network, 0.8) network = fully_connected(network, 256, activation='tanh') network = dropout(network, 0.8) network = fully_connected(network, classes, activation='softmax') network = regression(network, optimizer='adam', learning_rate=0.01, loss='categorical_crossentropy', name='target') # Training model = tflearn.DNN(network, tensorboard_verbose=0) return model
模型效果:很难收敛!!!
--
Training Step: 597 | total loss: 5.79258 | time: 26.039ss
| Adam | epoch: 001 | loss: 5.79258 - acc: 0.0064 | val_loss: 5.55333 - val_acc: 0.0107 -- iter: 19094/19094
--
Terminating training at the end of epoch 1
Training Step: 1194 | total loss: 5.87951 | time: 25.335s
| Adam | epoch: 002 | loss: 5.87951 - acc: 0.0084 | val_loss: 5.57970 - val_acc: 0.0105 -- iter: 19094/19094
--
Terminating training at the end of epoch 2
Training Step: 1791 | total loss: 5.93476 | time: 26.012s
| Adam | epoch: 003 | loss: 5.93476 - acc: 0.0124 | val_loss: 5.60627 - val_acc: 0.0107 -- iter: 19094/19094
--
Terminating training at the end of epoch 3
Training Step: 2388 | total loss: 5.76588 | time: 25.359s
| Adam | epoch: 004 | loss: 5.76588 - acc: 0.0116 | val_loss: 5.67958 - val_acc: 0.0119 -- iter: 19094/19094
--
Terminating training at the end of epoch 4
Training Step: 2985 | total loss: 5.87640 | time: 25.208s
| Adam | epoch: 005 | loss: 5.87640 - acc: 0.0111 | val_loss: 5.74356 - val_acc: 0.0101 -- iter: 19094/19094
--
Terminating training at the end of epoch 5
Training Step: 3582 | total loss: 6.01014 | time: 25.617ss
| Adam | epoch: 006 | loss: 6.01014 - acc: 0.0123 | val_loss: 5.68011 - val_acc: 0.0098 -- iter: 19094/19094
--
Terminating training at the end of epoch 6
Training Step: 4179 | total loss: 5.80083 | time: 25.633ss
| Adam | epoch: 007 | loss: 5.80083 - acc: 0.0067 | val_loss: 5.40268 - val_acc: 0.0088 -- iter: 19094/19094
--
Terminating training at the end of epoch 7
Training Step: 4776 | total loss: 5.90476 | time: 25.245ss
| Adam | epoch: 008 | loss: 5.90476 - acc: 0.0052 | val_loss: 5.69640 - val_acc: 0.0090 -- iter: 19094/19094
--
Terminating training at the end of epoch 8
Training Step: 5373 | total loss: 5.95897 | time: 25.667s
| Adam | epoch: 009 | loss: 5.95897 - acc: 0.0057 | val_loss: 5.58915 - val_acc: 0.0111 -- iter: 19094/19094
--
Terminating training at the end of epoch 9
Training Step: 5970 | total loss: 5.77673 | time: 25.025s
| Adam | epoch: 010 | loss: 5.77673 - acc: 0.0091 | val_loss: 5.52967 - val_acc: 0.0096 -- iter: 19094/19094
--
Terminating training at the end of epoch 10
Training Step: 6567 | total loss: 6.01010 | time: 25.004s
| Adam | epoch: 011 | loss: 6.01010 - acc: 0.0073 | val_loss: 5.84569 - val_acc: 0.0109 -- iter: 19094/19094
--
Terminating training at the end of epoch 11
Training Step: 7164 | total loss: 5.94524 | time: 25.614ss
| Adam | epoch: 012 | loss: 5.94524 - acc: 0.0120 | val_loss: 5.50813 - val_acc: 0.0101 -- iter: 19094/19094
--
Terminating training at the end of epoch 12
Training Step: 7761 | total loss: 5.75621 | time: 25.267ss
| Adam | epoch: 013 | loss: 5.75621 - acc: 0.0093 | val_loss: 5.52859 - val_acc: 0.0101 -- iter: 19094/19094
--
Terminating training at the end of epoch 13
Training Step: 8358 | total loss: 5.88941 | time: 25.958ss
| Adam | epoch: 014 | loss: 5.88941 - acc: 0.0082 | val_loss: 5.67036 - val_acc: 0.0067 -- iter: 19094/19094
--
Terminating training at the end of epoch 14
Training Step: 8955 | total loss: 5.80860 | time: 24.907s
| Adam | epoch: 015 | loss: 5.80860 - acc: 0.0101 | val_loss: 5.38732 - val_acc: 0.0107 -- iter: 19094/19094
--
Terminating training at the end of epoch 15
Training Step: 9552 | total loss: 5.93827 | time: 25.302s
| Adam | epoch: 016 | loss: 5.93827 - acc: 0.0163 | val_loss: 5.63285 - val_acc: 0.0101 -- iter: 19094/19094
--
接下来看看其他模型:
def get_model(width, height, classes=40): # TODO, modify model # Building 'VGG Network' network = input_data(shape=[None, width, height, 1]) # if RGB, 224,224,3 network = conv_2d(network, 64, 3, activation='relu') #network = conv_2d(network, 64, 3, activation='relu') network = max_pool_2d(network, 2, strides=2) network = conv_2d(network, 128, 3, activation='relu') #network = conv_2d(network, 128, 3, activation='relu') network = max_pool_2d(network, 2, strides=2) netword = tflearn.batch_normalization(network) network = fully_connected(network, 1024, activation='relu') network = dropout(network, 0.8) network = fully_connected(network, classes, activation='softmax') network = regression(network, optimizer='rmsprop', loss='categorical_crossentropy', learning_rate=0.0001) model = tflearn.DNN(network, checkpoint_path='checkpoint', max_checkpoints=1, tensorboard_verbose=1) return model
上述模型效果:
--
Training Step: 597 | total loss: 2.64693 | time: 280.077ss
| RMSProp | epoch: 001 | loss: 2.64693 - acc: 0.3916 | val_loss: 2.51221 - val_acc: 0.4246 -- iter: 19094/19094
--
Terminating training at the end of epoch 1
Training Step: 1194 | total loss: 1.30175 | time: 317.832ss
| RMSProp | epoch: 002 | loss: 1.30175 - acc: 0.6803 | val_loss: 1.17014 - val_acc: 0.6963 -- iter: 19094/19094
--
Terminating training at the end of epoch 2
Training Step: 1791 | total loss: 0.80158 | time: 330.904ss
| RMSProp | epoch: 003 | loss: 0.80158 - acc: 0.7837 | val_loss: 0.82845 - val_acc: 0.7713 -- iter: 19094/19094
Inception模型:
def get_model(width, height, classes=40): # TODO, modify model # Building 'VGG Network' network = input_data(shape=[None, width, height, 1]) # if RGB, 224,224,3 network = conv_2d(network, 64, 3, activation='relu') inception_3b_1_1 = conv_2d(network, 64, filter_size=1, activation='relu', name='inception_3b_1_1') inception_3b_3_3 = conv_2d(network, 64, filter_size=3, activation='relu', name='inception_3b_3_3') inception_3b_5_5 = conv_2d(network, 64, filter_size=5, activation='relu', name='inception_3b_5_5') inception_3b_output = merge([inception_3b_1_1, inception_3b_3_3, inception_3b_5_5], mode='concat', axis=3, name='inception_3b_output') network = max_pool_2d(inception_3b_output, kernel_size=3, strides=2, name='pool3_3_3') network = dropout(network, 0.4) network = fully_connected(network, classes, activation='softmax') network = regression(network, optimizer='momentum', loss='categorical_crossentropy', learning_rate=0.001) #network = regression(network, optimizer='rmsprop', # loss='categorical_crossentropy', # learning_rate=0.0001) model = tflearn.DNN(network, checkpoint_path='checkpoint', max_checkpoints=1, tensorboard_verbose=1) return model
上述模型效果:
-- Training Step: 597 | total loss: 4.36442 | time: 342.271ss | Momentum | epoch: 001 | loss: 4.36442 - acc: 0.0578 | val_loss: 4.30726 - val_acc: 0.1274 -- iter: 19094/19094 -- Terminating training at the end of epoch 1 Training Step: 1193 | total loss: 3.02893 | time: 322.366ss Training Step: 1194 | total loss: 3.00916 | time: 339.206ser: 19072/19094 | Momentum | epoch: 002 | loss: 3.00916 - acc: 0.2988 | val_loss: 2.71907 - val_acc: 0.4845 -- iter: 19094/19094 -- Terminating training at the end of epoch 2 Training Step: 1791 | total loss: 2.23406 | time: 347.633ss | Momentum | epoch: 003 | loss: 2.23406 - acc: 0.4559 | val_loss: 1.84004 - val_acc: 0.5888 -- iter: 19094/19094
换成avg pool跑起来很慢:
#network = max_pool_2d(inception_3b_output, kernel_size=3, strides=2, name='pool3_3_3') network = avg_pool_2d(inception_3b_output, kernel_size=7, strides=1) # acc: 0.0217 | val_loss: 4.50712 - val_acc: 0.0630 -- iter: 19094/19094
花费时间长,而且看不到什么效果:
--
Training Step: 597 | total loss: 4.53236 | time: 786.035s
| Momentum | epoch: 001 | loss: 4.53236 - acc: 0.0217 | val_loss: 4.50712 - val_acc: 0.0630 -- iter: 19094/19094
--
Terminating training at the end of epoch 1
^Caining Step: 692 | total loss: 4.49201 | time: 111.106ss
| Momentum | epoch: 002 | loss: 4.49201 - acc: 0.0247 -- iter: 03040/19094
Successfully left training! Final model accuracy: 0.0246666166931
resnet结构:
def get_model(width, height, classes=40): # TODO, modify model # Building 'VGG Network' network = input_data(shape=[None, width, height, 1]) # if RGB, 224,224,3 # Residual blocks # 32 layers: n=5, 56 layers: n=9, 110 layers: n=18 n = 2 net = tflearn.conv_2d(network, 16, 3, regularizer='L2', weight_decay=0.0001) net = tflearn.residual_block(net, n, 16) net = tflearn.residual_block(net, 1, 32, downsample=True) net = tflearn.residual_block(net, n-1, 32) net = tflearn.residual_block(net, 1, 64, downsample=True) net = tflearn.residual_block(net, n-1, 64) net = tflearn.batch_normalization(net) net = tflearn.activation(net, 'relu') net = tflearn.global_avg_pool(net) # Regression net = tflearn.fully_connected(net, classes, activation='softmax') mom = tflearn.Momentum(0.1, lr_decay=0.1, decay_step=32000, staircase=True) net = tflearn.regression(net, optimizer=mom, loss='categorical_crossentropy') # Training model = tflearn.DNN(net, checkpoint_path='model_resnet_cifar10', max_checkpoints=10, tensorboard_verbose=0, clip_gradients=0.) return model
--
Terminating training at the end of epoch 7
Training Step: 4776 | total loss: 0.13311 | time: 132.182ss
| Momentum | epoch: 008 | loss: 0.13311 - acc: 0.9561 | val_loss: 0.22734 - val_acc: 0.9370 -- iter: 19094/19094
--
Terminating training at the end of epoch 8
Successfully left training! Final model accuracy: 0.95614439249
OK, stop iterate!Good!
avg / total 0.97 0.96 0.96 23868
resnet加深结构:
def get_model(width, height, classes=40): # TODO, modify model # Building 'VGG Network' network = input_data(shape=[None, width, height, 1]) # if RGB, 224,224,3 # Building Residual Network net = tflearn.conv_2d(network, 64, 3, activation='relu', bias=False) # Residual blocks net = tflearn.residual_bottleneck(net, 3, 16, 64) net = tflearn.residual_bottleneck(net, 1, 32, 128, downsample=True) net = tflearn.residual_bottleneck(net, 2, 32, 128) net = tflearn.residual_bottleneck(net, 1, 64, 256, downsample=True) net = tflearn.residual_bottleneck(net, 2, 64, 256) net = tflearn.batch_normalization(net) net = tflearn.activation(net, 'relu') net = tflearn.global_avg_pool(net) # Regression net = tflearn.fully_connected(net, classes, activation='softmax') net = tflearn.regression(net, optimizer='momentum', loss='categorical_crossentropy', learning_rate=0.1) # Training model = tflearn.DNN(net, checkpoint_path='model_resnet_mnist', max_checkpoints=10, tensorboard_verbose=0) return model
结果是训练的时间更久了。
--
Terminating training at the end of epoch 5
Training Step: 3582 | total loss: 0.14701 | time: 313.084s
| Momentum | epoch: 006 | loss: 0.14701 - acc: 0.9516 | val_loss: 0.30464 - val_acc: 0.9103 -- iter: 19094/19094
--
Terminating training at the end of epoch 6
Successfully left training! Final model accuracy: 0.951571881771
OK, stop iterate!Good!
avg / total 0.94 0.93 0.93 23868
resnet加入预处理:
def get_model(width, height, classes=40): # TODO, modify model # Real-time data preprocessing img_prep = tflearn.ImagePreprocessing() img_prep.add_featurewise_zero_center(per_channel=True) network = input_data(shape=[None, width, height, 1], data_preprocessing=img_prep) # if RGB, 224,224,3 ...
效果:也还是很不错!
-- Training Step: 597 | total loss: 1.12591 | time: 312.814s | Momentum | epoch: 001 | loss: 1.12591 - acc: 0.6664 | val_loss: 1.86609 - val_acc: 0.5209 -- iter: 19094/19094 -- Terminating training at the end of epoch 1 Training Step: 1194 | total loss: 0.61108 | time: 312.415s | Momentum | epoch: 002 | loss: 0.61108 - acc: 0.8291 | val_loss: 0.56165 - val_acc: 0.8395 -- iter: 19094/19094
highway模型:又快又好!
def get_model(width, height, classes=40): # TODO, modify model network = input_data(shape=[None, width, height, 1]) # if RGB, 224,224,3 # Building convolutional network #highway convolutions with pooling and dropout for i in range(3): for j in [3, 2, 1]: network = highway_conv_2d(network, 16, j, activation='elu') network = max_pool_2d(network, 2) network = batch_normalization(network) network = fully_connected(network, 128, activation='elu') network = fully_connected(network, 256, activation='elu') network = fully_connected(network, classes, activation='softmax') network = regression(network, optimizer='adam', learning_rate=0.01, loss='categorical_crossentropy', name='target') model = tflearn.DNN(network, tensorboard_verbose=0) return model
--
Training Step: 597 | total loss: 0.95732 | time: 58.519ss
| Adam | epoch: 001 | loss: 0.95732 - acc: 0.7289 | val_loss: 1.46561 - val_acc: 0.6464 -- iter: 19094/19094
--
Terminating training at the end of epoch 1
Training Step: 1194 | total loss: 0.72415 | time: 57.346ss
| Adam | epoch: 002 | loss: 0.72415 - acc: 0.8067 | val_loss: 1.42666 - val_acc: 0.6919 -- iter: 19094/19094
--
Terminating training at the end of epoch 2
Training Step: 1791 | total loss: 0.78836 | time: 58.067ss
| Adam | epoch: 003 | loss: 0.78836 - acc: 0.8150 | val_loss: 1.05735 - val_acc: 0.7725 -- iter: 19094/19094
最后看看cifar10模型效果:
def get_model(width, height, classes=40): # TODO, modify model # Real-time data preprocessing img_prep = tflearn.ImagePreprocessing() img_prep.add_featurewise_zero_center(per_channel=True) img_prep.add_featurewise_stdnorm() network = input_data(shape=[None, width, height, 1], data_preprocessing=img_prep) # if RGB, 224,224,3 network = conv_2d(network, 32, 3, activation='relu') network = max_pool_2d(network, 2) network = conv_2d(network, 64, 3, activation='relu') network = conv_2d(network, 64, 3, activation='relu') network = max_pool_2d(network, 2) network = fully_connected(network, 512, activation='relu') network = dropout(network, 0.5) network = fully_connected(network, classes, activation='softmax') network = regression(network, optimizer='adam', loss='categorical_crossentropy', learning_rate=0.001) model = tflearn.DNN(network, tensorboard_verbose=0) return model
效果也很不错!又快又好:
--
Training Step: 597 | total loss: 0.70663 | time: 37.980ss
| Adam | epoch: 001 | loss: 0.70663 - acc: 0.7995 | val_loss: 0.55688 - val_acc: 0.8412 -- iter: 19094/19094
--
Terminating training at the end of epoch 1
Training Step: 1194 | total loss: 0.42443 | time: 37.595s
| Adam | epoch: 002 | loss: 0.42443 - acc: 0.8638 | val_loss: 0.43501 - val_acc: 0.8789 -- iter: 19094/19094
--
Terminating training at the end of epoch 2
Training Step: 1791 | total loss: 0.37865 | time: 37.516s
| Adam | epoch: 003 | loss: 0.37865 - acc: 0.9130 | val_loss: 0.30865 - val_acc: 0.9120 -- iter: 19094/19094
densenet效果
def get_model(width, height, classes=40): # Growth Rate (12, 16, 32, ...) k = 12 # Depth (40, 100, ...) L = 40 nb_layers = int((L - 4) / 3) network = input_data(shape=[None, width, height, 1]) net = tflearn.conv_2d(network, 16, 3, regularizer='L2', weight_decay=0.0001) net = tflearn.densenet_block(net, nb_layers, k) net = tflearn.densenet_block(net, nb_layers, k) net = tflearn.densenet_block(net, nb_layers, k) net = tflearn.global_avg_pool(net) # Regression net = tflearn.fully_connected(net, classes, activation='softmax') opt = tflearn.Nesterov(0.1, lr_decay=0.1, decay_step=32000, staircase=True) net = tflearn.regression(net, optimizer=opt, loss='categorical_crossentropy') # Training model = tflearn.DNN(net, checkpoint_path='model_densenet_cifar10', max_checkpoints=10, tensorboard_verbose=0, clip_gradients=0.) return model
---------------------------------
Training samples: 19094
Validation samples: 4774
--
Training Step: 597 | total loss: 1.25178 | time: 572.236s
| Nesterov | epoch: 001 | loss: 1.25178 - acc: 0.6439 | val_loss: 1.21137 - val_acc: 0.6200 -- iter: 19094/19094
--
Terminating training at the end of epoch 1
Training Step: 1194 | total loss: 0.79881 | time: 567.656s
| Nesterov | epoch: 002 | loss: 0.79881 - acc: 0.7648 | val_loss: 0.53844 - val_acc: 0.8349 -- iter: 19094/19094
--
Terminating training at the end of epoch 2
Training Step: 1791 | total loss: 0.53881 | time: 570.329s
| Nesterov | epoch: 003 | loss: 0.53881 - acc: 0.8397 | val_loss: 0.52145 - val_acc: 0.8458 -- iter: 19094/19094
--
Terminating training at the end of epoch 3
Training Step: 2388 | total loss: 0.40775 | time: 571.856s
| Nesterov | epoch: 004 | loss: 0.40775 - acc: 0.8794 | val_loss: 0.51006 - val_acc: 0.8486 -- iter: 19094/19094
--
Terminating training at the end of epoch 4
Training Step: 2985 | total loss: 0.47510 | time: 574.415s
| Nesterov | epoch: 005 | loss: 0.47510 - acc: 0.8796 | val_loss: 0.38588 - val_acc: 0.8814 -- iter: 19094/19094
--
Terminating training at the end of epoch 5
Training Step: 3582 | total loss: 0.45802 | time: 577.388s
| Nesterov | epoch: 006 | loss: 0.45802 - acc: 0.8881 | val_loss: 1.30243 - val_acc: 0.6341 -- iter: 19094/19094
--
Terminating training at the end of epoch 6
Training Step: 4179 | total loss: 0.26061 | time: 576.418s
| Nesterov | epoch: 007 | loss: 0.26061 - acc: 0.9182 | val_loss: 0.28861 - val_acc: 0.9166 -- iter: 19094/19094
--
Terminating training at the end of epoch 7
Successfully left training! Final model accuracy: 0.918215036392
resnext 模型:
def get_model(width, height, classes=40): # TODO, modify model # Real-time data preprocessing img_prep = tflearn.ImagePreprocessing() img_prep.add_featurewise_zero_center(per_channel=True) img_prep.add_featurewise_stdnorm() network = input_data(shape=[None, width, height, 1], data_preprocessing=img_prep) # if RGB, 224,224,3 n = 2 net = tflearn.conv_2d(network, 16, 3, regularizer='L2', weight_decay=0.0001) net = tflearn.resnext_block(net, n, 16, 32) net = tflearn.resnext_block(net, 1, 32, 32, downsample=True) net = tflearn.resnext_block(net, n-1, 32, 32) net = tflearn.resnext_block(net, 1, 64, 32, downsample=True) net = tflearn.resnext_block(net, n-1, 64, 32) net = tflearn.batch_normalization(net) net = tflearn.activation(net, 'relu') net = tflearn.global_avg_pool(net) # Regression net = tflearn.fully_connected(net, classes, activation='softmax') opt = tflearn.Momentum(0.1, lr_decay=0.1, decay_step=32000, staircase=True) net = tflearn.regression(net, optimizer=opt, loss='categorical_crossentropy') # Training model = tflearn.DNN(net, checkpoint_path='model_resnext_cifar10', max_checkpoints=10, tensorboard_verbose=0, clip_gradients=0.) return model
Training samples: 19094
Validation samples: 4774
--
Training Step: 597 | total loss: 1.28942 | time: 215.233ss
| Momentum | epoch: 001 | loss: 1.28942 - acc: 0.6302 | val_loss: 1.29879 - val_acc: 0.6148 -- iter: 19094/19094
--
Terminating training at the end of epoch 1
Training Step: 1194 | total loss: 0.64329 | time: 213.968s
| Momentum | epoch: 002 | loss: 0.64329 - acc: 0.8121 | val_loss: 0.60106 - val_acc: 0.8270 -- iter: 19094/19094
--
Terminating training at the end of epoch 2
Training Step: 1791 | total loss: 0.49910 | time: 214.042s
| Momentum | epoch: 003 | loss: 0.49910 - acc: 0.8448 | val_loss: 0.52757 - val_acc: 0.8458 -- iter: 19094/19094
--
Terminating training at the end of epoch 3
Training Step: 2388 | total loss: 0.51495 | time: 217.513ss
| Momentum | epoch: 004 | loss: 0.51495 - acc: 0.8590 | val_loss: 0.49154 - val_acc: 0.8582 -- iter: 19094/19094
--
Terminating training at the end of epoch 4
Training Step: 2985 | total loss: 0.43045 | time: 213.852ss
| Momentum | epoch: 005 | loss: 0.43045 - acc: 0.8896 | val_loss: 0.47342 - val_acc: 0.8607 -- iter: 19094/19094
--
Terminating training at the end of epoch 5
Training Step: 3582 | total loss: 0.24511 | time: 216.133ss
| Momentum | epoch: 006 | loss: 0.24511 - acc: 0.9249 | val_loss: 0.35686 - val_acc: 0.8957 -- iter: 19094/19094
--
Terminating training at the end of epoch 6
Training Step: 4179 | total loss: 0.46725 | time: 212.062s
| Momentum | epoch: 007 | loss: 0.46725 - acc: 0.8758 | val_loss: 0.46417 - val_acc: 0.8615 -- iter: 19094/19094
--
Terminating training at the end of epoch 7
Training Step: 4776 | total loss: 0.43170 | time: 214.533s
| Momentum | epoch: 008 | loss: 0.43170 - acc: 0.8885 | val_loss: 0.46593 - val_acc: 0.8676 -- iter: 19094/19094
--
Terminating training at the end of epoch 8
Training Step: 5373 | total loss: 0.19362 | time: 213.225s
| Momentum | epoch: 009 | loss: 0.19362 - acc: 0.9462 | val_loss: 0.28189 - val_acc: 0.9206 -- iter: 19094/19094
--
Terminating training at the end of epoch 9
Successfully left training! Final model accuracy: 0.946155309677