深度残差网络+自适应参数化ReLU激活函数（调参记录20）Cifar10~94.17%
在之前的调参记录18中，是将深度残差网络ResNet中的所有ReLU都替换成了自适应参数化ReLU（Adaptively Parametric ReLU，APReLU）。
由于APReLU的输入特征图与输出特征图的尺寸是完全一致的，所以APReLU可以被嵌入到神经网络的任意部分。
本文将APReLU放在每个残差模块的第二个卷积层之后。这种结构与Squeeze-and-Excitation Network是非常相似的，其区别在于APReLU额外地包含了非线性变换。
同时，迭代次数也从5000个epoch减少到了500个epoch。时间耗不起。
APReLU激活函数的原理如下图所示：
整体代码如下：
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Created on Tue Apr 14 04:17:45 2020
Implemented using TensorFlow 1.10.0 and Keras 2.2.1
Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Shaojiang Dong, Michael Pecht,
Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis,
IEEE Transactions on Industrial Electronics, 2020, DOI: 10.1109/TIE.2020.2972458
@author: Minghang Zhao
"""
from __future__ import print_function
import keras
import numpy as np
from keras.datasets import cifar10
from keras.layers import Dense, Conv2D, BatchNormalization, Activation, Minimum
from keras.layers import AveragePooling2D, Input, GlobalAveragePooling2D, Concatenate, Reshape
from keras.regularizers import l2
from keras import backend as K
from keras.models import Model
from keras import optimizers
from keras.preprocessing.image import ImageDataGenerator
from keras.callbacks import LearningRateScheduler
K.set_learning_phase(1)
# The data, split between train and test sets
(x_train, y_train), (x_test, y_test) = cifar10.load_data()
# Noised data
x_train = x_train.astype('float32') / 255.
x_test = x_test.astype('float32') / 255.
x_test = x_test-np.mean(x_train)
x_train = x_train-np.mean(x_train)
print('x_train shape:', x_train.shape)
print(x_train.shape[0], 'train samples')
print(x_test.shape[0], 'test samples')
# convert class vectors to binary class matrices
y_train = keras.utils.to_categorical(y_train, 10)
y_test = keras.utils.to_categorical(y_test, 10)
# Schedule the learning rate, multiply 0.1 every 150 epoches
def scheduler(epoch):
if epoch % 150 == 0 and epoch != 0:
lr = K.get_value(model.optimizer.lr)
K.set_value(model.optimizer.lr, lr * 0.1)
print("lr changed to {}".format(lr * 0.1))
return K.get_value(model.optimizer.lr)
# An adaptively parametric rectifier linear unit (APReLU)
def aprelu(inputs):
# get the number of channels
channels = inputs.get_shape().as_list()[-1]
# get a zero feature map
zeros_input = keras.layers.subtract([inputs, inputs])
# get a feature map with only positive features
pos_input = Activation('relu')(inputs)
# get a feature map with only negative features
neg_input = Minimum()([inputs,zeros_input])
# define a network to obtain the scaling coefficients
scales_p = GlobalAveragePooling2D()(pos_input)
scales_n = GlobalAveragePooling2D()(neg_input)
scales = Concatenate()([scales_n, scales_p])
scales = Dense(channels//16, activation='linear', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(scales)
scales = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(scales)
scales = Activation('relu')(scales)
scales = Dense(channels, activation='linear', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(scales)
scales = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(scales)
scales = Activation('sigmoid')(scales)
scales = Reshape((1,1,channels))(scales)
# apply a paramtetric relu
neg_part = keras.layers.multiply([scales, neg_input])
return keras.layers.add([pos_input, neg_part])
# Residual Block
def residual_block(incoming, nb_blocks, out_channels, downsample=False,
downsample_strides=2):
residual = incoming
in_channels = incoming.get_shape().as_list()[-1]
for i in range(nb_blocks):
identity = residual
if not downsample:
downsample_strides = 1
residual = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(residual)
residual = Activation('relu')(residual)
residual = Conv2D(out_channels, 3, strides=(downsample_strides, downsample_strides),
padding='same', kernel_initializer='he_normal',
kernel_regularizer=l2(1e-4))(residual)
residual = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(residual)
residual = Activation('relu')(residual)
residual = Conv2D(out_channels, 3, padding='same', kernel_initializer='he_normal',
kernel_regularizer=l2(1e-4))(residual)
residual = aprelu(residual)
# Downsampling
if downsample_strides > 1:
identity = AveragePooling2D(pool_size=(1,1), strides=(2,2))(identity)
# Zero_padding to match channels
if in_channels != out_channels:
zeros_identity = keras.layers.subtract([identity, identity])
identity = keras.layers.concatenate([identity, zeros_identity])
in_channels = out_channels
residual = keras.layers.add([residual, identity])
return residual
# define and train a model
inputs = Input(shape=(32, 32, 3))
net = Conv2D(16, 3, padding='same', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(inputs)
net = residual_block(net, 9, 32, downsample=False)
net = residual_block(net, 1, 32, downsample=True)
net = residual_block(net, 8, 32, downsample=False)
net = residual_block(net, 1, 64, downsample=True)
net = residual_block(net, 8, 64, downsample=False)
net = BatchNormalization(momentum=0.9, gamma_regularizer=l2(1e-4))(net)
net = Activation('relu')(net)
net = GlobalAveragePooling2D()(net)
outputs = Dense(10, activation='softmax', kernel_initializer='he_normal', kernel_regularizer=l2(1e-4))(net)
model = Model(inputs=inputs, outputs=outputs)
sgd = optimizers.SGD(lr=0.1, decay=0., momentum=0.9, nesterov=True)
model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])
# data augmentation
datagen = ImageDataGenerator(
# randomly rotate images in the range (deg 0 to 180)
rotation_range=30,
# Range for random zoom
zoom_range = 0.2,
# shear angle in counter-clockwise direction in degrees
shear_range = 30,
# randomly flip images
horizontal_flip=True,
# randomly shift images horizontally
width_shift_range=0.125,
# randomly shift images vertically
height_shift_range=0.125)
reduce_lr = LearningRateScheduler(scheduler)
# fit the model on the batches generated by datagen.flow().
model.fit_generator(datagen.flow(x_train, y_train, batch_size=100),
validation_data=(x_test, y_test), epochs=500,
verbose=1, callbacks=[reduce_lr], workers=4)
# get results
K.set_learning_phase(0)
DRSN_train_score = model.evaluate(x_train, y_train, batch_size=100, verbose=0)
print('Train loss:', DRSN_train_score[0])
print('Train accuracy:', DRSN_train_score[1])
DRSN_test_score = model.evaluate(x_test, y_test, batch_size=100, verbose=0)
print('Test loss:', DRSN_test_score[0])
print('Test accuracy:', DRSN_test_score[1])
实验结果如下：
Using TensorFlow backend. x_train shape: (50000, 32, 32, 3) 50000 train samples 10000 test samples Epoch 1/500 70s 140ms/step - loss: 2.5742 - acc: 0.4145 - val_loss: 2.1224 - val_acc: 0.5538 Epoch 2/500 52s 104ms/step - loss: 2.0566 - acc: 0.5631 - val_loss: 1.7678 - val_acc: 0.6502 Epoch 3/500 52s 104ms/step - loss: 1.7697 - acc: 0.6317 - val_loss: 1.5134 - val_acc: 0.7114 Epoch 4/500 52s 103ms/step - loss: 1.5790 - acc: 0.6694 - val_loss: 1.3269 - val_acc: 0.7508 Epoch 5/500 52s 104ms/step - loss: 1.4270 - acc: 0.6971 - val_loss: 1.2040 - val_acc: 0.7703 Epoch 6/500 52s 104ms/step - loss: 1.3109 - acc: 0.7165 - val_loss: 1.1187 - val_acc: 0.7809 Epoch 7/500 52s 104ms/step - loss: 1.2249 - acc: 0.7302 - val_loss: 1.0393 - val_acc: 0.7919 Epoch 8/500 52s 103ms/step - loss: 1.1457 - acc: 0.7482 - val_loss: 0.9639 - val_acc: 0.8084 Epoch 9/500 52s 104ms/step - loss: 1.0931 - acc: 0.7555 - val_loss: 0.9324 - val_acc: 0.8130 Epoch 10/500 52s 104ms/step - loss: 1.0418 - acc: 0.7693 - val_loss: 0.9043 - val_acc: 0.8138 Epoch 11/500 52s 104ms/step - loss: 1.0049 - acc: 0.7747 - val_loss: 0.8600 - val_acc: 0.8281 Epoch 12/500 51s 102ms/step - loss: 0.9835 - acc: 0.7796 - val_loss: 0.8364 - val_acc: 0.8288 Epoch 13/500 51s 103ms/step - loss: 0.9480 - acc: 0.7878 - val_loss: 0.7905 - val_acc: 0.8432 Epoch 14/500 52s 104ms/step - loss: 0.9246 - acc: 0.7906 - val_loss: 0.7895 - val_acc: 0.8400 Epoch 15/500 52s 103ms/step - loss: 0.9067 - acc: 0.7967 - val_loss: 0.7709 - val_acc: 0.8451 Epoch 16/500 52s 103ms/step - loss: 0.8928 - acc: 0.8000 - val_loss: 0.7728 - val_acc: 0.8447 Epoch 17/500 52s 104ms/step - loss: 0.8787 - acc: 0.8042 - val_loss: 0.7842 - val_acc: 0.8367 Epoch 18/500 52s 104ms/step - loss: 0.8644 - acc: 0.8095 - val_loss: 0.7588 - val_acc: 0.8489 Epoch 19/500 52s 103ms/step - loss: 0.8531 - acc: 0.8111 - val_loss: 0.7626 - val_acc: 0.8427 Epoch 20/500 52s 103ms/step - loss: 0.8463 - acc: 0.8140 - val_loss: 0.7256 - val_acc: 0.8620 Epoch 21/500 52s 103ms/step - loss: 0.8402 - acc: 0.8158 - val_loss: 0.7499 - val_acc: 0.8502 Epoch 22/500 52s 103ms/step - loss: 0.8347 - acc: 0.8170 - val_loss: 0.7154 - val_acc: 0.8629 Epoch 23/500 52s 103ms/step - loss: 0.8262 - acc: 0.8219 - val_loss: 0.7074 - val_acc: 0.8625 Epoch 24/500 51s 102ms/step - loss: 0.8251 - acc: 0.8211 - val_loss: 0.7165 - val_acc: 0.8601 Epoch 25/500 51s 102ms/step - loss: 0.8166 - acc: 0.8250 - val_loss: 0.7233 - val_acc: 0.8564 Epoch 26/500 51s 102ms/step - loss: 0.8090 - acc: 0.8266 - val_loss: 0.7401 - val_acc: 0.8481 Epoch 27/500 51s 102ms/step - loss: 0.8049 - acc: 0.8290 - val_loss: 0.6991 - val_acc: 0.8658 Epoch 28/500 51s 102ms/step - loss: 0.8037 - acc: 0.8302 - val_loss: 0.7159 - val_acc: 0.8630 Epoch 29/500 51s 102ms/step - loss: 0.7977 - acc: 0.8327 - val_loss: 0.7189 - val_acc: 0.8614 Epoch 30/500 51s 102ms/step - loss: 0.7968 - acc: 0.8334 - val_loss: 0.7030 - val_acc: 0.8706 Epoch 31/500 51s 102ms/step - loss: 0.7956 - acc: 0.8339 - val_loss: 0.6863 - val_acc: 0.8764 Epoch 32/500 51s 102ms/step - loss: 0.7875 - acc: 0.8377 - val_loss: 0.7160 - val_acc: 0.8647 Epoch 33/500 51s 102ms/step - loss: 0.7874 - acc: 0.8368 - val_loss: 0.7002 - val_acc: 0.8704 Epoch 34/500 51s 102ms/step - loss: 0.7917 - acc: 0.8357 - val_loss: 0.6829 - val_acc: 0.8783 Epoch 35/500 51s 102ms/step - loss: 0.7808 - acc: 0.8423 - val_loss: 0.7057 - val_acc: 0.8685 Epoch 36/500 51s 103ms/step - loss: 0.7795 - acc: 0.8413 - val_loss: 0.7044 - val_acc: 0.8710 Epoch 37/500 52s 103ms/step - loss: 0.7768 - acc: 0.8429 - val_loss: 0.6871 - val_acc: 0.8743 Epoch 38/500 52s 103ms/step - loss: 0.7763 - acc: 0.8418 - val_loss: 0.6995 - val_acc: 0.8687 Epoch 39/500 52s 103ms/step - loss: 0.7665 - acc: 0.8459 - val_loss: 0.6907 - val_acc: 0.8736 Epoch 40/500 52s 103ms/step - loss: 0.7751 - acc: 0.8428 - val_loss: 0.6947 - val_acc: 0.8717 Epoch 41/500 52s 103ms/step - loss: 0.7691 - acc: 0.8456 - val_loss: 0.6960 - val_acc: 0.8709 Epoch 42/500 52s 103ms/step - loss: 0.7704 - acc: 0.8460 - val_loss: 0.6690 - val_acc: 0.8851 Epoch 43/500 52s 103ms/step - loss: 0.7641 - acc: 0.8484 - val_loss: 0.6825 - val_acc: 0.8792 Epoch 44/500 52s 103ms/step - loss: 0.7630 - acc: 0.8499 - val_loss: 0.6765 - val_acc: 0.8781 Epoch 45/500 52s 103ms/step - loss: 0.7583 - acc: 0.8502 - val_loss: 0.6955 - val_acc: 0.8724 Epoch 46/500 52s 103ms/step - loss: 0.7599 - acc: 0.8493 - val_loss: 0.6750 - val_acc: 0.8773 Epoch 47/500 51s 102ms/step - loss: 0.7617 - acc: 0.8497 - val_loss: 0.6925 - val_acc: 0.8757 Epoch 48/500 51s 102ms/step - loss: 0.7589 - acc: 0.8501 - val_loss: 0.6848 - val_acc: 0.8775 Epoch 49/500 51s 102ms/step - loss: 0.7572 - acc: 0.8506 - val_loss: 0.6834 - val_acc: 0.8767 Epoch 50/500 51s 103ms/step - loss: 0.7512 - acc: 0.8523 - val_loss: 0.6908 - val_acc: 0.8742 Epoch 51/500 51s 102ms/step - loss: 0.7553 - acc: 0.8523 - val_loss: 0.6987 - val_acc: 0.8705 Epoch 52/500 51s 103ms/step - loss: 0.7501 - acc: 0.8523 - val_loss: 0.7237 - val_acc: 0.8665 Epoch 53/500 52s 103ms/step - loss: 0.7544 - acc: 0.8535 - val_loss: 0.6973 - val_acc: 0.8747 Epoch 54/500 52s 103ms/step - loss: 0.7467 - acc: 0.8561 - val_loss: 0.6836 - val_acc: 0.8814 Epoch 55/500 52s 103ms/step - loss: 0.7490 - acc: 0.8544 - val_loss: 0.6914 - val_acc: 0.8767 Epoch 56/500 52s 103ms/step - loss: 0.7451 - acc: 0.8568 - val_loss: 0.6881 - val_acc: 0.8803 Epoch 57/500 52s 103ms/step - loss: 0.7488 - acc: 0.8554 - val_loss: 0.6867 - val_acc: 0.8786 Epoch 58/500 52s 103ms/step - loss: 0.7465 - acc: 0.8576 - val_loss: 0.6845 - val_acc: 0.8759 Epoch 59/500 52s 103ms/step - loss: 0.7443 - acc: 0.8570 - val_loss: 0.6715 - val_acc: 0.8851 Epoch 60/500 52s 103ms/step - loss: 0.7472 - acc: 0.8562 - val_loss: 0.7045 - val_acc: 0.8751 Epoch 61/500 52s 103ms/step - loss: 0.7425 - acc: 0.8596 - val_loss: 0.6793 - val_acc: 0.8836 Epoch 62/500 52s 103ms/step - loss: 0.7433 - acc: 0.8587 - val_loss: 0.6963 - val_acc: 0.8752 Epoch 63/500 52s 103ms/step - loss: 0.7472 - acc: 0.8560 - val_loss: 0.6911 - val_acc: 0.8773 Epoch 64/500 52s 103ms/step - loss: 0.7423 - acc: 0.8577 - val_loss: 0.6808 - val_acc: 0.8809 Epoch 65/500 52s 103ms/step - loss: 0.7404 - acc: 0.8594 - val_loss: 0.7071 - val_acc: 0.8703 Epoch 66/500 52s 103ms/step - loss: 0.7418 - acc: 0.8580 - val_loss: 0.6881 - val_acc: 0.8776 Epoch 67/500 52s 103ms/step - loss: 0.7404 - acc: 0.8584 - val_loss: 0.6973 - val_acc: 0.8770 Epoch 68/500 52s 103ms/step - loss: 0.7458 - acc: 0.8569 - val_loss: 0.6871 - val_acc: 0.8783 Epoch 69/500 52s 103ms/step - loss: 0.7342 - acc: 0.8625 - val_loss: 0.6856 - val_acc: 0.8778 Epoch 70/500 52s 103ms/step - loss: 0.7358 - acc: 0.8603 - val_loss: 0.6999 - val_acc: 0.8742 Epoch 71/500 52s 103ms/step - loss: 0.7419 - acc: 0.8600 - val_loss: 0.6921 - val_acc: 0.8816 Epoch 72/500 52s 103ms/step - loss: 0.7368 - acc: 0.8607 - val_loss: 0.6825 - val_acc: 0.8813 Epoch 73/500 51s 103ms/step - loss: 0.7376 - acc: 0.8581 - val_loss: 0.6771 - val_acc: 0.8846 Epoch 74/500 51s 102ms/step - loss: 0.7371 - acc: 0.8598 - val_loss: 0.6963 - val_acc: 0.8787 Epoch 75/500 51s 102ms/step - loss: 0.7409 - acc: 0.8593 - val_loss: 0.6884 - val_acc: 0.8794 Epoch 76/500 51s 102ms/step - loss: 0.7348 - acc: 0.8633 - val_loss: 0.6655 - val_acc: 0.8904 Epoch 77/500 51s 102ms/step - loss: 0.7358 - acc: 0.8624 - val_loss: 0.6860 - val_acc: 0.8812 Epoch 78/500 51s 102ms/step - loss: 0.7392 - acc: 0.8616 - val_loss: 0.7251 - val_acc: 0.8686 Epoch 79/500 51s 102ms/step - loss: 0.7342 - acc: 0.8645 - val_loss: 0.6891 - val_acc: 0.8804 Epoch 80/500 51s 102ms/step - loss: 0.7341 - acc: 0.8635 - val_loss: 0.6761 - val_acc: 0.8847 Epoch 81/500 51s 102ms/step - loss: 0.7382 - acc: 0.8605 - val_loss: 0.7034 - val_acc: 0.8752ETA: 43s - loss: 0.6977 - acc: 0.8793 Epoch 82/500 51s 102ms/step - loss: 0.7377 - acc: 0.8617 - val_loss: 0.6670 - val_acc: 0.8867 Epoch 83/500 51s 102ms/step - loss: 0.7347 - acc: 0.8643 - val_loss: 0.6862 - val_acc: 0.8815 Epoch 84/500 51s 102ms/step - loss: 0.7301 - acc: 0.8660 - val_loss: 0.6837 - val_acc: 0.8818 Epoch 85/500 51s 102ms/step - loss: 0.7322 - acc: 0.8648 - val_loss: 0.6842 - val_acc: 0.8809 Epoch 86/500 51s 102ms/step - loss: 0.7303 - acc: 0.8640 - val_loss: 0.6906 - val_acc: 0.8823 Epoch 87/500 51s 102ms/step - loss: 0.7285 - acc: 0.8651 - val_loss: 0.6701 - val_acc: 0.8847 Epoch 88/500 51s 102ms/step - loss: 0.7313 - acc: 0.8645 - val_loss: 0.6774 - val_acc: 0.8832 Epoch 89/500 51s 102ms/step - loss: 0.7303 - acc: 0.8625 - val_loss: 0.6827 - val_acc: 0.8851 Epoch 90/500 51s 102ms/step - loss: 0.7283 - acc: 0.8647 - val_loss: 0.6886 - val_acc: 0.8821 Epoch 91/500 51s 103ms/step - loss: 0.7256 - acc: 0.8662 - val_loss: 0.6889 - val_acc: 0.8803 Epoch 92/500 52s 103ms/step - loss: 0.7313 - acc: 0.8634 - val_loss: 0.6747 - val_acc: 0.8865 Epoch 93/500 52s 103ms/step - loss: 0.7257 - acc: 0.8656 - val_loss: 0.6831 - val_acc: 0.8836 Epoch 94/500 52s 103ms/step - loss: 0.7303 - acc: 0.8645 - val_loss: 0.7008 - val_acc: 0.8772 Epoch 95/500 52s 103ms/step - loss: 0.7268 - acc: 0.8657 - val_loss: 0.6998 - val_acc: 0.8783 Epoch 96/500 51s 102ms/step - loss: 0.7245 - acc: 0.8659 - val_loss: 0.6927 - val_acc: 0.8789 Epoch 97/500 51s 102ms/step - loss: 0.7236 - acc: 0.8685 - val_loss: 0.6620 - val_acc: 0.8919 Epoch 98/500 51s 102ms/step - loss: 0.7249 - acc: 0.8669 - val_loss: 0.6741 - val_acc: 0.8857 Epoch 99/500 51s 102ms/step - loss: 0.7252 - acc: 0.8659 - val_loss: 0.6770 - val_acc: 0.8844 Epoch 100/500 51s 102ms/step - loss: 0.7239 - acc: 0.8672 - val_loss: 0.6815 - val_acc: 0.8851 Epoch 101/500 51s 102ms/step - loss: 0.7272 - acc: 0.8667 - val_loss: 0.6752 - val_acc: 0.8867 Epoch 102/500 51s 102ms/step - loss: 0.7268 - acc: 0.8651 - val_loss: 0.6985 - val_acc: 0.8766 Epoch 103/500 51s 102ms/step - loss: 0.7243 - acc: 0.8685 - val_loss: 0.7136 - val_acc: 0.8714 Epoch 104/500 51s 102ms/step - loss: 0.7293 - acc: 0.8646 - val_loss: 0.6930 - val_acc: 0.8813 Epoch 105/500 51s 102ms/step - loss: 0.7237 - acc: 0.8665 - val_loss: 0.6919 - val_acc: 0.8812 Epoch 106/500 51s 102ms/step - loss: 0.7275 - acc: 0.8669 - val_loss: 0.6722 - val_acc: 0.8857 Epoch 107/500 51s 102ms/step - loss: 0.7149 - acc: 0.8703 - val_loss: 0.6730 - val_acc: 0.8827 Epoch 108/500 51s 102ms/step - loss: 0.7214 - acc: 0.8689 - val_loss: 0.6530 - val_acc: 0.8930 Epoch 109/500 51s 102ms/step - loss: 0.7212 - acc: 0.8678 - val_loss: 0.7015 - val_acc: 0.8784 Epoch 110/500 51s 102ms/step - loss: 0.7238 - acc: 0.8676 - val_loss: 0.6730 - val_acc: 0.8833 Epoch 111/500 51s 102ms/step - loss: 0.7260 - acc: 0.8670 - val_loss: 0.6541 - val_acc: 0.8919 Epoch 112/500 51s 102ms/step - loss: 0.7209 - acc: 0.8688 - val_loss: 0.6577 - val_acc: 0.8926 Epoch 113/500 51s 102ms/step - loss: 0.7258 - acc: 0.8654 - val_loss: 0.6864 - val_acc: 0.8832 Epoch 114/500 51s 102ms/step - loss: 0.7211 - acc: 0.8695 - val_loss: 0.6749 - val_acc: 0.8873 Epoch 115/500 51s 102ms/step - loss: 0.7196 - acc: 0.8707 - val_loss: 0.6660 - val_acc: 0.8873 Epoch 116/500 51s 102ms/step - loss: 0.7189 - acc: 0.8684 - val_loss: 0.6945 - val_acc: 0.8809 Epoch 117/500 51s 102ms/step - loss: 0.7246 - acc: 0.8672 - val_loss: 0.7153 - val_acc: 0.8706 Epoch 118/500 51s 102ms/step - loss: 0.7189 - acc: 0.8719 - val_loss: 0.6718 - val_acc: 0.8849 Epoch 119/500 51s 102ms/step - loss: 0.7214 - acc: 0.8695 - val_loss: 0.6883 - val_acc: 0.8802 Epoch 120/500 51s 102ms/step - loss: 0.7154 - acc: 0.8706 - val_loss: 0.6846 - val_acc: 0.8844 Epoch 121/500 51s 102ms/step - loss: 0.7150 - acc: 0.8691 - val_loss: 0.6820 - val_acc: 0.8836 Epoch 122/500 51s 102ms/step - loss: 0.7190 - acc: 0.8696 - val_loss: 0.6737 - val_acc: 0.8857 Epoch 123/500 51s 102ms/step - loss: 0.7191 - acc: 0.8686 - val_loss: 0.6752 - val_acc: 0.8848 Epoch 124/500 51s 102ms/step - loss: 0.7185 - acc: 0.8701 - val_loss: 0.6841 - val_acc: 0.8828 Epoch 125/500 51s 103ms/step - loss: 0.7221 - acc: 0.8689 - val_loss: 0.6739 - val_acc: 0.8837 Epoch 126/500 51s 102ms/step - loss: 0.7202 - acc: 0.8699 - val_loss: 0.6787 - val_acc: 0.8888 Epoch 127/500 51s 102ms/step - loss: 0.7172 - acc: 0.8703 - val_loss: 0.6889 - val_acc: 0.8815 Epoch 128/500 51s 102ms/step - loss: 0.7157 - acc: 0.8719 - val_loss: 0.6832 - val_acc: 0.8852 Epoch 129/500 51s 102ms/step - loss: 0.7149 - acc: 0.8704 - val_loss: 0.6777 - val_acc: 0.8859 Epoch 130/500 51s 102ms/step - loss: 0.7205 - acc: 0.8698 - val_loss: 0.6675 - val_acc: 0.8908 Epoch 131/500 51s 102ms/step - loss: 0.7146 - acc: 0.8721 - val_loss: 0.6741 - val_acc: 0.8916 Epoch 132/500 51s 102ms/step - loss: 0.7140 - acc: 0.8720 - val_loss: 0.6649 - val_acc: 0.8891 Epoch 133/500 52s 103ms/step - loss: 0.7128 - acc: 0.8714 - val_loss: 0.6883 - val_acc: 0.8834 Epoch 134/500 52s 103ms/step - loss: 0.7203 - acc: 0.8699 - val_loss: 0.6899 - val_acc: 0.8854 Epoch 135/500 52s 103ms/step - loss: 0.7131 - acc: 0.8716 - val_loss: 0.6907 - val_acc: 0.8823 Epoch 136/500 52s 103ms/step - loss: 0.7127 - acc: 0.8706 - val_loss: 0.6957 - val_acc: 0.8786 Epoch 137/500 52s 103ms/step - loss: 0.7147 - acc: 0.8726 - val_loss: 0.7036 - val_acc: 0.8764 Epoch 138/500 52s 103ms/step - loss: 0.7125 - acc: 0.8735 - val_loss: 0.6704 - val_acc: 0.8896 Epoch 139/500 52s 103ms/step - loss: 0.7153 - acc: 0.8692 - val_loss: 0.6620 - val_acc: 0.8923 Epoch 140/500 52s 103ms/step - loss: 0.7125 - acc: 0.8707 - val_loss: 0.6862 - val_acc: 0.8839 Epoch 141/500 52s 103ms/step - loss: 0.7149 - acc: 0.8722 - val_loss: 0.6573 - val_acc: 0.8951 Epoch 142/500 51s 103ms/step - loss: 0.7154 - acc: 0.8726 - val_loss: 0.6658 - val_acc: 0.8898 Epoch 143/500 52s 103ms/step - loss: 0.7115 - acc: 0.8728 - val_loss: 0.6868 - val_acc: 0.8848 Epoch 144/500 52s 103ms/step - loss: 0.7116 - acc: 0.8733 - val_loss: 0.6679 - val_acc: 0.8894 Epoch 145/500 52s 103ms/step - loss: 0.7198 - acc: 0.8696 - val_loss: 0.6865 - val_acc: 0.8824 Epoch 146/500 52s 103ms/step - loss: 0.7146 - acc: 0.8726 - val_loss: 0.6906 - val_acc: 0.8826 Epoch 147/500 52s 103ms/step - loss: 0.7165 - acc: 0.8718 - val_loss: 0.6597 - val_acc: 0.8898 Epoch 148/500 52s 103ms/step - loss: 0.7130 - acc: 0.8725 - val_loss: 0.6596 - val_acc: 0.8926 Epoch 149/500 52s 103ms/step - loss: 0.7096 - acc: 0.8728 - val_loss: 0.6855 - val_acc: 0.8816 Epoch 150/500 52s 103ms/step - loss: 0.7131 - acc: 0.8730 - val_loss: 0.6872 - val_acc: 0.8838 Epoch 151/500 lr changed to 0.010000000149011612 52s 103ms/step - loss: 0.5994 - acc: 0.9120 - val_loss: 0.5846 - val_acc: 0.9190 Epoch 152/500 51s 103ms/step - loss: 0.5434 - acc: 0.9286 - val_loss: 0.5670 - val_acc: 0.9214 Epoch 153/500 52s 103ms/step - loss: 0.5249 - acc: 0.9328 - val_loss: 0.5552 - val_acc: 0.9216 Epoch 154/500 51s 103ms/step - loss: 0.5089 - acc: 0.9363 - val_loss: 0.5436 - val_acc: 0.9275 Epoch 155/500 51s 102ms/step - loss: 0.4992 - acc: 0.9393 - val_loss: 0.5400 - val_acc: 0.9263 Epoch 156/500 51s 102ms/step - loss: 0.4838 - acc: 0.9424 - val_loss: 0.5373 - val_acc: 0.9269 Epoch 157/500 51s 102ms/step - loss: 0.4733 - acc: 0.9442 - val_loss: 0.5283 - val_acc: 0.9264ETA: 24s - loss: 0.4741 - acc: 0.9435 Epoch 158/500 51s 103ms/step - loss: 0.4641 - acc: 0.9458 - val_loss: 0.5195 - val_acc: 0.9296 Epoch 159/500 51s 102ms/step - loss: 0.4579 - acc: 0.9462 - val_loss: 0.5159 - val_acc: 0.9316 Epoch 160/500 51s 102ms/step - loss: 0.4515 - acc: 0.9467 - val_loss: 0.5058 - val_acc: 0.9299 Epoch 161/500 51s 102ms/step - loss: 0.4420 - acc: 0.9485 - val_loss: 0.5039 - val_acc: 0.9296 Epoch 162/500 51s 102ms/step - loss: 0.4357 - acc: 0.9485 - val_loss: 0.4972 - val_acc: 0.9325 Epoch 163/500 51s 102ms/step - loss: 0.4238 - acc: 0.9514 - val_loss: 0.4934 - val_acc: 0.9320 Epoch 164/500 51s 102ms/step - loss: 0.4187 - acc: 0.9523 - val_loss: 0.4884 - val_acc: 0.9314 Epoch 165/500 51s 102ms/step - loss: 0.4139 - acc: 0.9523 - val_loss: 0.4825 - val_acc: 0.9311 Epoch 166/500 51s 102ms/step - loss: 0.4071 - acc: 0.9538 - val_loss: 0.4861 - val_acc: 0.9308 Epoch 167/500 51s 102ms/step - loss: 0.4004 - acc: 0.9539 - val_loss: 0.4780 - val_acc: 0.9292 Epoch 168/500 51s 102ms/step - loss: 0.3950 - acc: 0.9550 - val_loss: 0.4754 - val_acc: 0.9311 Epoch 169/500 51s 102ms/step - loss: 0.3948 - acc: 0.9541 - val_loss: 0.4717 - val_acc: 0.9328 Epoch 170/500 51s 102ms/step - loss: 0.3828 - acc: 0.9570 - val_loss: 0.4741 - val_acc: 0.9313 Epoch 171/500 51s 102ms/step - loss: 0.3831 - acc: 0.9554 - val_loss: 0.4666 - val_acc: 0.9302 Epoch 172/500 51s 102ms/step - loss: 0.3781 - acc: 0.9561 - val_loss: 0.4653 - val_acc: 0.9324 Epoch 173/500 51s 102ms/step - loss: 0.3707 - acc: 0.9575 - val_loss: 0.4633 - val_acc: 0.9300 Epoch 174/500 51s 102ms/step - loss: 0.3679 - acc: 0.9573 - val_loss: 0.4688 - val_acc: 0.9262 Epoch 175/500 51s 102ms/step - loss: 0.3621 - acc: 0.9585 - val_loss: 0.4521 - val_acc: 0.9307 Epoch 176/500 51s 102ms/step - loss: 0.3619 - acc: 0.9571 - val_loss: 0.4465 - val_acc: 0.9329ETA: 42s - loss: 0.3637 - acc: 0.9586 Epoch 177/500 51s 102ms/step - loss: 0.3586 - acc: 0.9569 - val_loss: 0.4481 - val_acc: 0.9315 Epoch 178/500 51s 102ms/step - loss: 0.3485 - acc: 0.9598 - val_loss: 0.4531 - val_acc: 0.9305 Epoch 179/500 51s 102ms/step - loss: 0.3469 - acc: 0.9590 - val_loss: 0.4452 - val_acc: 0.9324 Epoch 180/500 51s 102ms/step - loss: 0.3456 - acc: 0.9590 - val_loss: 0.4466 - val_acc: 0.9312 Epoch 181/500 51s 102ms/step - loss: 0.3440 - acc: 0.9584 - val_loss: 0.4395 - val_acc: 0.9324 Epoch 182/500 51s 102ms/step - loss: 0.3393 - acc: 0.9595 - val_loss: 0.4456 - val_acc: 0.9295 Epoch 183/500 51s 102ms/step - loss: 0.3372 - acc: 0.9591 - val_loss: 0.4441 - val_acc: 0.9268 Epoch 184/500 51s 102ms/step - loss: 0.3342 - acc: 0.9592 - val_loss: 0.4347 - val_acc: 0.9283 Epoch 185/500 51s 102ms/step - loss: 0.3313 - acc: 0.9604 - val_loss: 0.4392 - val_acc: 0.9277 Epoch 186/500 51s 102ms/step - loss: 0.3245 - acc: 0.9618 - val_loss: 0.4281 - val_acc: 0.9318 Epoch 187/500 51s 102ms/step - loss: 0.3279 - acc: 0.9596 - val_loss: 0.4449 - val_acc: 0.9266 Epoch 188/500 51s 102ms/step - loss: 0.3244 - acc: 0.9593 - val_loss: 0.4367 - val_acc: 0.9262 Epoch 189/500 51s 102ms/step - loss: 0.3223 - acc: 0.9601 - val_loss: 0.4250 - val_acc: 0.9316 Epoch 190/500 51s 102ms/step - loss: 0.3197 - acc: 0.9597 - val_loss: 0.4245 - val_acc: 0.9273 Epoch 191/500 51s 102ms/step - loss: 0.3179 - acc: 0.9591 - val_loss: 0.4279 - val_acc: 0.9299 Epoch 192/500 51s 102ms/step - loss: 0.3166 - acc: 0.9603 - val_loss: 0.4243 - val_acc: 0.9285 Epoch 193/500 51s 102ms/step - loss: 0.3156 - acc: 0.9600 - val_loss: 0.4209 - val_acc: 0.9299 Epoch 194/500 51s 103ms/step - loss: 0.3102 - acc: 0.9622 - val_loss: 0.4253 - val_acc: 0.9267 Epoch 195/500 51s 102ms/step - loss: 0.3044 - acc: 0.9627 - val_loss: 0.4237 - val_acc: 0.9283 Epoch 196/500 51s 102ms/step - loss: 0.3085 - acc: 0.9609 - val_loss: 0.4174 - val_acc: 0.9297 Epoch 197/500 51s 102ms/step - loss: 0.3044 - acc: 0.9617 - val_loss: 0.4135 - val_acc: 0.9283 Epoch 198/500 51s 102ms/step - loss: 0.2983 - acc: 0.9631 - val_loss: 0.4152 - val_acc: 0.9312 Epoch 199/500 51s 102ms/step - loss: 0.3054 - acc: 0.9601 - val_loss: 0.4156 - val_acc: 0.9296 Epoch 200/500 51s 102ms/step - loss: 0.3007 - acc: 0.9619 - val_loss: 0.4084 - val_acc: 0.9305 Epoch 201/500 51s 102ms/step - loss: 0.3018 - acc: 0.9608 - val_loss: 0.4119 - val_acc: 0.9285 Epoch 202/500 51s 102ms/step - loss: 0.3023 - acc: 0.9596 - val_loss: 0.4075 - val_acc: 0.9312 Epoch 203/500 51s 102ms/step - loss: 0.2984 - acc: 0.9608 - val_loss: 0.4125 - val_acc: 0.9315 Epoch 204/500 51s 102ms/step - loss: 0.2937 - acc: 0.9621 - val_loss: 0.4088 - val_acc: 0.9267 Epoch 205/500 51s 102ms/step - loss: 0.2918 - acc: 0.9620 - val_loss: 0.4138 - val_acc: 0.9258 Epoch 206/500 51s 102ms/step - loss: 0.2922 - acc: 0.9617 - val_loss: 0.4048 - val_acc: 0.9310 Epoch 207/500 51s 102ms/step - loss: 0.2914 - acc: 0.9615 - val_loss: 0.3929 - val_acc: 0.9343 Epoch 208/500 51s 102ms/step - loss: 0.2915 - acc: 0.9615 - val_loss: 0.4042 - val_acc: 0.9291 Epoch 209/500 51s 102ms/step - loss: 0.2880 - acc: 0.9619 - val_loss: 0.4065 - val_acc: 0.9257 Epoch 210/500 51s 102ms/step - loss: 0.2912 - acc: 0.9601 - val_loss: 0.4155 - val_acc: 0.9238 Epoch 211/500 51s 102ms/step - loss: 0.2896 - acc: 0.9616 - val_loss: 0.3948 - val_acc: 0.9304 Epoch 212/500 51s 102ms/step - loss: 0.2890 - acc: 0.9605 - val_loss: 0.4077 - val_acc: 0.9288 Epoch 213/500 51s 102ms/step - loss: 0.2913 - acc: 0.9596 - val_loss: 0.3965 - val_acc: 0.9282 Epoch 214/500 51s 102ms/step - loss: 0.2865 - acc: 0.9599 - val_loss: 0.4121 - val_acc: 0.9240 Epoch 215/500 51s 102ms/step - loss: 0.2841 - acc: 0.9610 - val_loss: 0.4025 - val_acc: 0.9302 Epoch 216/500 51s 102ms/step - loss: 0.2870 - acc: 0.9603 - val_loss: 0.4064 - val_acc: 0.9244 Epoch 217/500 51s 102ms/step - loss: 0.2811 - acc: 0.9610 - val_loss: 0.4086 - val_acc: 0.9276 Epoch 218/500 51s 102ms/step - loss: 0.2837 - acc: 0.9606 - val_loss: 0.3963 - val_acc: 0.9269 Epoch 219/500 51s 102ms/step - loss: 0.2801 - acc: 0.9623 - val_loss: 0.3981 - val_acc: 0.9290 Epoch 220/500 51s 102ms/step - loss: 0.2786 - acc: 0.9631 - val_loss: 0.4038 - val_acc: 0.9261 Epoch 221/500 51s 103ms/step - loss: 0.2792 - acc: 0.9617 - val_loss: 0.3992 - val_acc: 0.9275 Epoch 222/500 52s 104ms/step - loss: 0.2780 - acc: 0.9619 - val_loss: 0.3951 - val_acc: 0.9290 Epoch 223/500 51s 103ms/step - loss: 0.2816 - acc: 0.9602 - val_loss: 0.3909 - val_acc: 0.9290 Epoch 224/500 51s 102ms/step - loss: 0.2766 - acc: 0.9622 - val_loss: 0.4014 - val_acc: 0.9276 Epoch 225/500 51s 102ms/step - loss: 0.2801 - acc: 0.9612 - val_loss: 0.3954 - val_acc: 0.9264 Epoch 226/500 51s 102ms/step - loss: 0.2803 - acc: 0.9595 - val_loss: 0.3920 - val_acc: 0.9309 Epoch 227/500 51s 102ms/step - loss: 0.2759 - acc: 0.9608 - val_loss: 0.3968 - val_acc: 0.9281 Epoch 228/500 51s 102ms/step - loss: 0.2814 - acc: 0.9586 - val_loss: 0.3927 - val_acc: 0.9288 Epoch 229/500 51s 102ms/step - loss: 0.2732 - acc: 0.9636 - val_loss: 0.4051 - val_acc: 0.9249 Epoch 230/500 51s 102ms/step - loss: 0.2770 - acc: 0.9606 - val_loss: 0.3944 - val_acc: 0.9276 Epoch 231/500 51s 102ms/step - loss: 0.2765 - acc: 0.9616 - val_loss: 0.3943 - val_acc: 0.9279 Epoch 232/500 51s 102ms/step - loss: 0.2762 - acc: 0.9603 - val_loss: 0.3855 - val_acc: 0.9271 Epoch 233/500 51s 102ms/step - loss: 0.2747 - acc: 0.9609 - val_loss: 0.3948 - val_acc: 0.9272 Epoch 234/500 51s 102ms/step - loss: 0.2776 - acc: 0.9595 - val_loss: 0.3861 - val_acc: 0.9285 Epoch 235/500 51s 102ms/step - loss: 0.2763 - acc: 0.9606 - val_loss: 0.3802 - val_acc: 0.9301 Epoch 236/500 51s 102ms/step - loss: 0.2747 - acc: 0.9616 - val_loss: 0.3883 - val_acc: 0.9295 Epoch 237/500 51s 102ms/step - loss: 0.2763 - acc: 0.9604 - val_loss: 0.3882 - val_acc: 0.9288 Epoch 238/500 51s 103ms/step - loss: 0.2699 - acc: 0.9616 - val_loss: 0.3973 - val_acc: 0.9260 Epoch 239/500 52s 103ms/step - loss: 0.2720 - acc: 0.9612 - val_loss: 0.3893 - val_acc: 0.9255 Epoch 240/500 51s 102ms/step - loss: 0.2758 - acc: 0.9595 - val_loss: 0.3975 - val_acc: 0.9267 Epoch 241/500 51s 102ms/step - loss: 0.2724 - acc: 0.9619 - val_loss: 0.3955 - val_acc: 0.9242 Epoch 242/500 51s 102ms/step - loss: 0.2704 - acc: 0.9622 - val_loss: 0.3889 - val_acc: 0.9263 Epoch 243/500 51s 102ms/step - loss: 0.2699 - acc: 0.9612 - val_loss: 0.4002 - val_acc: 0.9267 Epoch 244/500 51s 102ms/step - loss: 0.2733 - acc: 0.9605 - val_loss: 0.3858 - val_acc: 0.9290 Epoch 245/500 51s 102ms/step - loss: 0.2733 - acc: 0.9603 - val_loss: 0.3944 - val_acc: 0.9272 Epoch 246/500 51s 102ms/step - loss: 0.2682 - acc: 0.9616 - val_loss: 0.3787 - val_acc: 0.9306 Epoch 247/500 51s 103ms/step - loss: 0.2679 - acc: 0.9623 - val_loss: 0.3862 - val_acc: 0.9281 Epoch 248/500 52s 103ms/step - loss: 0.2688 - acc: 0.9619 - val_loss: 0.3904 - val_acc: 0.9266 Epoch 249/500 52s 103ms/step - loss: 0.2683 - acc: 0.9626 - val_loss: 0.3898 - val_acc: 0.9269 Epoch 250/500 52s 103ms/step - loss: 0.2706 - acc: 0.9600 - val_loss: 0.3884 - val_acc: 0.9294 Epoch 251/500 52s 103ms/step - loss: 0.2681 - acc: 0.9612 - val_loss: 0.3848 - val_acc: 0.9289 Epoch 252/500 52s 103ms/step - loss: 0.2696 - acc: 0.9618 - val_loss: 0.3852 - val_acc: 0.9268 Epoch 253/500 52s 103ms/step - loss: 0.2717 - acc: 0.9603 - val_loss: 0.3837 - val_acc: 0.9270 Epoch 254/500 51s 102ms/step - loss: 0.2718 - acc: 0.9596 - val_loss: 0.3855 - val_acc: 0.9264 Epoch 255/500 51s 103ms/step - loss: 0.2658 - acc: 0.9626 - val_loss: 0.3890 - val_acc: 0.9286 Epoch 256/500 51s 102ms/step - loss: 0.2660 - acc: 0.9623 - val_loss: 0.3866 - val_acc: 0.9314 Epoch 257/500 51s 103ms/step - loss: 0.2713 - acc: 0.9605 - val_loss: 0.3863 - val_acc: 0.9271 Epoch 258/500 52s 103ms/step - loss: 0.2715 - acc: 0.9599 - val_loss: 0.3833 - val_acc: 0.9301 Epoch 259/500 52s 103ms/step - loss: 0.2705 - acc: 0.9596 - val_loss: 0.3897 - val_acc: 0.9257 Epoch 260/500 52s 103ms/step - loss: 0.2682 - acc: 0.9613 - val_loss: 0.3911 - val_acc: 0.9274 Epoch 261/500 52s 103ms/step - loss: 0.2692 - acc: 0.9601 - val_loss: 0.3798 - val_acc: 0.9292 Epoch 262/500 52s 103ms/step - loss: 0.2650 - acc: 0.9622 - val_loss: 0.3910 - val_acc: 0.9270 Epoch 263/500 52s 103ms/step - loss: 0.2639 - acc: 0.9620 - val_loss: 0.3870 - val_acc: 0.9285 Epoch 264/500 52s 103ms/step - loss: 0.2651 - acc: 0.9620 - val_loss: 0.3994 - val_acc: 0.9263 Epoch 265/500 52s 103ms/step - loss: 0.2694 - acc: 0.9601 - val_loss: 0.3807 - val_acc: 0.9314 Epoch 266/500 52s 103ms/step - loss: 0.2691 - acc: 0.9613 - val_loss: 0.3811 - val_acc: 0.9282 Epoch 267/500 52s 103ms/step - loss: 0.2618 - acc: 0.9631 - val_loss: 0.3789 - val_acc: 0.9334 Epoch 268/500 51s 102ms/step - loss: 0.2636 - acc: 0.9631 - val_loss: 0.3905 - val_acc: 0.9254 Epoch 269/500 51s 102ms/step - loss: 0.2695 - acc: 0.9609 - val_loss: 0.4035 - val_acc: 0.9235 Epoch 270/500 51s 102ms/step - loss: 0.2661 - acc: 0.9615 - val_loss: 0.3942 - val_acc: 0.9264 Epoch 271/500 51s 102ms/step - loss: 0.2640 - acc: 0.9623 - val_loss: 0.3878 - val_acc: 0.9294 Epoch 272/500 52s 103ms/step - loss: 0.2658 - acc: 0.9620 - val_loss: 0.3952 - val_acc: 0.9260 Epoch 273/500 52s 103ms/step - loss: 0.2654 - acc: 0.9623 - val_loss: 0.4001 - val_acc: 0.9225 Epoch 274/500 51s 102ms/step - loss: 0.2648 - acc: 0.9621 - val_loss: 0.3885 - val_acc: 0.9285 Epoch 275/500 51s 102ms/step - loss: 0.2618 - acc: 0.9625 - val_loss: 0.3925 - val_acc: 0.9260 Epoch 276/500 51s 103ms/step - loss: 0.2615 - acc: 0.9628 - val_loss: 0.3860 - val_acc: 0.9279 Epoch 277/500 51s 103ms/step - loss: 0.2740 - acc: 0.9584 - val_loss: 0.3831 - val_acc: 0.9279 Epoch 278/500 51s 102ms/step - loss: 0.2653 - acc: 0.9616 - val_loss: 0.3898 - val_acc: 0.9257 Epoch 279/500 51s 102ms/step - loss: 0.2623 - acc: 0.9630 - val_loss: 0.3902 - val_acc: 0.9271 Epoch 280/500 52s 103ms/step - loss: 0.2652 - acc: 0.9624 - val_loss: 0.3899 - val_acc: 0.9274 Epoch 281/500 51s 103ms/step - loss: 0.2674 - acc: 0.9614 - val_loss: 0.3948 - val_acc: 0.9232 Epoch 282/500 51s 102ms/step - loss: 0.2641 - acc: 0.9622 - val_loss: 0.3880 - val_acc: 0.9284 Epoch 283/500 51s 103ms/step - loss: 0.2689 - acc: 0.9600 - val_loss: 0.3856 - val_acc: 0.9281 Epoch 284/500 51s 102ms/step - loss: 0.2616 - acc: 0.9618 - val_loss: 0.3795 - val_acc: 0.9294 Epoch 285/500 51s 102ms/step - loss: 0.2645 - acc: 0.9618 - val_loss: 0.3904 - val_acc: 0.9270 Epoch 286/500 51s 102ms/step - loss: 0.2656 - acc: 0.9612 - val_loss: 0.3848 - val_acc: 0.9264 Epoch 287/500 51s 102ms/step - loss: 0.2645 - acc: 0.9621 - val_loss: 0.3850 - val_acc: 0.9273 Epoch 288/500 51s 103ms/step - loss: 0.2607 - acc: 0.9628 - val_loss: 0.3806 - val_acc: 0.9295 Epoch 289/500 51s 102ms/step - loss: 0.2627 - acc: 0.9619 - val_loss: 0.3875 - val_acc: 0.9279 Epoch 290/500 51s 103ms/step - loss: 0.2641 - acc: 0.9618 - val_loss: 0.3967 - val_acc: 0.9284 Epoch 291/500 52s 103ms/step - loss: 0.2624 - acc: 0.9633 - val_loss: 0.3906 - val_acc: 0.9253 Epoch 292/500 51s 103ms/step - loss: 0.2651 - acc: 0.9623 - val_loss: 0.3843 - val_acc: 0.9273 Epoch 293/500 52s 103ms/step - loss: 0.2604 - acc: 0.9632 - val_loss: 0.3887 - val_acc: 0.9264 Epoch 294/500 51s 103ms/step - loss: 0.2605 - acc: 0.9633 - val_loss: 0.3969 - val_acc: 0.9244 Epoch 295/500 51s 102ms/step - loss: 0.2636 - acc: 0.9615 - val_loss: 0.3853 - val_acc: 0.9295 Epoch 296/500 51s 103ms/step - loss: 0.2620 - acc: 0.9625 - val_loss: 0.3993 - val_acc: 0.9248 Epoch 297/500 52s 103ms/step - loss: 0.2657 - acc: 0.9616 - val_loss: 0.3840 - val_acc: 0.9283 Epoch 298/500 52s 103ms/step - loss: 0.2630 - acc: 0.9627 - val_loss: 0.3899 - val_acc: 0.9257 Epoch 299/500 52s 103ms/step - loss: 0.2623 - acc: 0.9619 - val_loss: 0.3894 - val_acc: 0.9249 Epoch 300/500 52s 103ms/step - loss: 0.2709 - acc: 0.9589 - val_loss: 0.3775 - val_acc: 0.9304 Epoch 301/500 lr changed to 0.0009999999776482583 52s 103ms/step - loss: 0.2329 - acc: 0.9743 - val_loss: 0.3555 - val_acc: 0.9362 Epoch 302/500 52s 103ms/step - loss: 0.2176 - acc: 0.9795 - val_loss: 0.3499 - val_acc: 0.9380 Epoch 303/500 52s 103ms/step - loss: 0.2121 - acc: 0.9810 - val_loss: 0.3486 - val_acc: 0.9394 Epoch 304/500 52s 103ms/step - loss: 0.2088 - acc: 0.9825 - val_loss: 0.3498 - val_acc: 0.9401 Epoch 305/500 52s 103ms/step - loss: 0.2045 - acc: 0.9837 - val_loss: 0.3475 - val_acc: 0.9406 Epoch 306/500 52s 103ms/step - loss: 0.2032 - acc: 0.9846 - val_loss: 0.3479 - val_acc: 0.9404 Epoch 307/500 52s 103ms/step - loss: 0.2013 - acc: 0.9845 - val_loss: 0.3481 - val_acc: 0.9407 Epoch 308/500 51s 103ms/step - loss: 0.2013 - acc: 0.9846 - val_loss: 0.3519 - val_acc: 0.9397 Epoch 309/500 51s 103ms/step - loss: 0.1980 - acc: 0.9865 - val_loss: 0.3485 - val_acc: 0.9411 Epoch 310/500 52s 103ms/step - loss: 0.1949 - acc: 0.9868 - val_loss: 0.3500 - val_acc: 0.9407 Epoch 311/500 51s 102ms/step - loss: 0.1946 - acc: 0.9866 - val_loss: 0.3517 - val_acc: 0.9394 Epoch 312/500 51s 102ms/step - loss: 0.1950 - acc: 0.9864 - val_loss: 0.3522 - val_acc: 0.9395 Epoch 313/500 51s 103ms/step - loss: 0.1925 - acc: 0.9874 - val_loss: 0.3497 - val_acc: 0.9406 Epoch 314/500 52s 103ms/step - loss: 0.1917 - acc: 0.9873 - val_loss: 0.3515 - val_acc: 0.9400 Epoch 315/500 51s 102ms/step - loss: 0.1919 - acc: 0.9876 - val_loss: 0.3528 - val_acc: 0.9391 Epoch 316/500 51s 102ms/step - loss: 0.1914 - acc: 0.9872 - val_loss: 0.3521 - val_acc: 0.9391 Epoch 317/500 51s 103ms/step - loss: 0.1891 - acc: 0.9883 - val_loss: 0.3520 - val_acc: 0.9387 Epoch 318/500 51s 102ms/step - loss: 0.1891 - acc: 0.9883 - val_loss: 0.3529 - val_acc: 0.9389 Epoch 319/500 51s 102ms/step - loss: 0.1885 - acc: 0.9881 - val_loss: 0.3531 - val_acc: 0.9402 Epoch 320/500 51s 102ms/step - loss: 0.1866 - acc: 0.9890 - val_loss: 0.3549 - val_acc: 0.9397 Epoch 321/500 51s 102ms/step - loss: 0.1869 - acc: 0.9885 - val_loss: 0.3530 - val_acc: 0.9401 Epoch 322/500 51s 103ms/step - loss: 0.1871 - acc: 0.9873 - val_loss: 0.3572 - val_acc: 0.9394 Epoch 323/500 52s 103ms/step - loss: 0.1862 - acc: 0.9881 - val_loss: 0.3538 - val_acc: 0.9403 Epoch 324/500 51s 103ms/step - loss: 0.1861 - acc: 0.9883 - val_loss: 0.3545 - val_acc: 0.9394 Epoch 325/500 51s 102ms/step - loss: 0.1832 - acc: 0.9894 - val_loss: 0.3555 - val_acc: 0.9394 Epoch 326/500 51s 102ms/step - loss: 0.1813 - acc: 0.9904 - val_loss: 0.3542 - val_acc: 0.9399 Epoch 327/500 51s 102ms/step - loss: 0.1842 - acc: 0.9887 - val_loss: 0.3552 - val_acc: 0.9382 Epoch 328/500 51s 102ms/step - loss: 0.1825 - acc: 0.9890 - val_loss: 0.3541 - val_acc: 0.9402 Epoch 329/500 51s 102ms/step - loss: 0.1804 - acc: 0.9899 - val_loss: 0.3554 - val_acc: 0.9397 Epoch 330/500 51s 103ms/step - loss: 0.1816 - acc: 0.9896 - val_loss: 0.3549 - val_acc: 0.9403 Epoch 331/500 51s 102ms/step - loss: 0.1804 - acc: 0.9899 - val_loss: 0.3518 - val_acc: 0.9407 Epoch 332/500 51s 102ms/step - loss: 0.1806 - acc: 0.9900 - val_loss: 0.3515 - val_acc: 0.9419 Epoch 333/500 51s 102ms/step - loss: 0.1786 - acc: 0.9903 - val_loss: 0.3517 - val_acc: 0.9401 Epoch 334/500 51s 102ms/step - loss: 0.1812 - acc: 0.9894 - val_loss: 0.3529 - val_acc: 0.9407 Epoch 335/500 51s 102ms/step - loss: 0.1796 - acc: 0.9894 - val_loss: 0.3528 - val_acc: 0.9424 Epoch 336/500 51s 102ms/step - loss: 0.1792 - acc: 0.9896 - val_loss: 0.3534 - val_acc: 0.9400 Epoch 337/500 51s 102ms/step - loss: 0.1777 - acc: 0.9902 - val_loss: 0.3531 - val_acc: 0.9401 Epoch 338/500 51s 102ms/step - loss: 0.1774 - acc: 0.9904 - val_loss: 0.3525 - val_acc: 0.9392 Epoch 339/500 52s 103ms/step - loss: 0.1768 - acc: 0.9905 - val_loss: 0.3517 - val_acc: 0.9414 Epoch 340/500 51s 103ms/step - loss: 0.1781 - acc: 0.9899 - val_loss: 0.3540 - val_acc: 0.9417 Epoch 341/500 51s 102ms/step - loss: 0.1750 - acc: 0.9908 - val_loss: 0.3574 - val_acc: 0.9396 Epoch 342/500 51s 102ms/step - loss: 0.1756 - acc: 0.9906 - val_loss: 0.3560 - val_acc: 0.9407 Epoch 343/500 51s 102ms/step - loss: 0.1758 - acc: 0.9908 - val_loss: 0.3568 - val_acc: 0.9401 Epoch 344/500 51s 102ms/step - loss: 0.1767 - acc: 0.9904 - val_loss: 0.3518 - val_acc: 0.9418 Epoch 345/500 51s 102ms/step - loss: 0.1748 - acc: 0.9911 - val_loss: 0.3542 - val_acc: 0.9399 Epoch 346/500 51s 102ms/step - loss: 0.1737 - acc: 0.9909 - val_loss: 0.3557 - val_acc: 0.9399 Epoch 347/500 51s 102ms/step - loss: 0.1727 - acc: 0.9915 - val_loss: 0.3572 - val_acc: 0.9395 Epoch 348/500 51s 102ms/step - loss: 0.1722 - acc: 0.9914 - val_loss: 0.3557 - val_acc: 0.9395 Epoch 349/500 51s 102ms/step - loss: 0.1743 - acc: 0.9904 - val_loss: 0.3515 - val_acc: 0.9407 Epoch 350/500 51s 102ms/step - loss: 0.1730 - acc: 0.9910 - val_loss: 0.3529 - val_acc: 0.9395 Epoch 351/500 51s 102ms/step - loss: 0.1732 - acc: 0.9906 - val_loss: 0.3518 - val_acc: 0.9390 Epoch 352/500 51s 102ms/step - loss: 0.1724 - acc: 0.9911 - val_loss: 0.3526 - val_acc: 0.9409 Epoch 353/500 51s 102ms/step - loss: 0.1705 - acc: 0.9913 - val_loss: 0.3529 - val_acc: 0.9404 Epoch 354/500 51s 103ms/step - loss: 0.1712 - acc: 0.9912 - val_loss: 0.3517 - val_acc: 0.9399 Epoch 355/500 51s 102ms/step - loss: 0.1697 - acc: 0.9915 - val_loss: 0.3525 - val_acc: 0.9416 Epoch 356/500 51s 102ms/step - loss: 0.1709 - acc: 0.9906 - val_loss: 0.3489 - val_acc: 0.9410 Epoch 357/500 51s 102ms/step - loss: 0.1689 - acc: 0.9921 - val_loss: 0.3503 - val_acc: 0.9412 Epoch 358/500 51s 102ms/step - loss: 0.1692 - acc: 0.9914 - val_loss: 0.3512 - val_acc: 0.9410 Epoch 359/500 51s 102ms/step - loss: 0.1703 - acc: 0.9914 - val_loss: 0.3504 - val_acc: 0.9407 Epoch 360/500 51s 102ms/step - loss: 0.1705 - acc: 0.9910 - val_loss: 0.3535 - val_acc: 0.9412 Epoch 361/500 51s 102ms/step - loss: 0.1684 - acc: 0.9922 - val_loss: 0.3494 - val_acc: 0.9413 Epoch 362/500 51s 102ms/step - loss: 0.1682 - acc: 0.9922 - val_loss: 0.3516 - val_acc: 0.9401 Epoch 363/500 51s 102ms/step - loss: 0.1687 - acc: 0.9910 - val_loss: 0.3511 - val_acc: 0.9403 Epoch 364/500 51s 102ms/step - loss: 0.1679 - acc: 0.9917 - val_loss: 0.3539 - val_acc: 0.9394 Epoch 365/500 51s 102ms/step - loss: 0.1685 - acc: 0.9909 - val_loss: 0.3545 - val_acc: 0.9389 Epoch 366/500 51s 102ms/step - loss: 0.1672 - acc: 0.9914 - val_loss: 0.3546 - val_acc: 0.9385 Epoch 367/500 51s 102ms/step - loss: 0.1663 - acc: 0.9921 - val_loss: 0.3563 - val_acc: 0.9391 Epoch 368/500 51s 102ms/step - loss: 0.1658 - acc: 0.9920 - val_loss: 0.3546 - val_acc: 0.9398 Epoch 369/500 51s 102ms/step - loss: 0.1675 - acc: 0.9908 - val_loss: 0.3553 - val_acc: 0.9393 Epoch 370/500 51s 102ms/step - loss: 0.1659 - acc: 0.9916 - val_loss: 0.3556 - val_acc: 0.9387 Epoch 371/500 51s 102ms/step - loss: 0.1656 - acc: 0.9915 - val_loss: 0.3538 - val_acc: 0.9386 Epoch 372/500 51s 102ms/step - loss: 0.1651 - acc: 0.9918 - val_loss: 0.3547 - val_acc: 0.9400 Epoch 373/500 51s 102ms/step - loss: 0.1656 - acc: 0.9916 - val_loss: 0.3567 - val_acc: 0.9399 Epoch 374/500 51s 102ms/step - loss: 0.1625 - acc: 0.9931 - val_loss: 0.3539 - val_acc: 0.9399 Epoch 375/500 51s 102ms/step - loss: 0.1644 - acc: 0.9920 - val_loss: 0.3547 - val_acc: 0.9399 Epoch 376/500 51s 102ms/step - loss: 0.1632 - acc: 0.9920 - val_loss: 0.3592 - val_acc: 0.9378 Epoch 377/500 51s 102ms/step - loss: 0.1620 - acc: 0.9927 - val_loss: 0.3564 - val_acc: 0.9392 Epoch 378/500 51s 102ms/step - loss: 0.1633 - acc: 0.9926 - val_loss: 0.3542 - val_acc: 0.9378 Epoch 379/500 51s 102ms/step - loss: 0.1629 - acc: 0.9925 - val_loss: 0.3539 - val_acc: 0.9391 Epoch 380/500 51s 102ms/step - loss: 0.1614 - acc: 0.9924 - val_loss: 0.3527 - val_acc: 0.9389 Epoch 381/500 51s 102ms/step - loss: 0.1632 - acc: 0.9917 - val_loss: 0.3532 - val_acc: 0.9400 Epoch 382/500 51s 102ms/step - loss: 0.1627 - acc: 0.9922 - val_loss: 0.3520 - val_acc: 0.9411 Epoch 383/500 51s 102ms/step - loss: 0.1615 - acc: 0.9922 - val_loss: 0.3541 - val_acc: 0.9399 Epoch 384/500 51s 102ms/step - loss: 0.1611 - acc: 0.9928 - val_loss: 0.3528 - val_acc: 0.9395 Epoch 385/500 51s 102ms/step - loss: 0.1613 - acc: 0.9921 - val_loss: 0.3521 - val_acc: 0.9406 Epoch 386/500 51s 102ms/step - loss: 0.1612 - acc: 0.9918 - val_loss: 0.3535 - val_acc: 0.9400 Epoch 387/500 51s 102ms/step - loss: 0.1606 - acc: 0.9927 - val_loss: 0.3505 - val_acc: 0.9412 Epoch 388/500 51s 102ms/step - loss: 0.1607 - acc: 0.9919 - val_loss: 0.3497 - val_acc: 0.9419 Epoch 389/500 51s 102ms/step - loss: 0.1589 - acc: 0.9925 - val_loss: 0.3491 - val_acc: 0.9431 Epoch 390/500 51s 102ms/step - loss: 0.1604 - acc: 0.9923 - val_loss: 0.3523 - val_acc: 0.9429 Epoch 391/500 51s 102ms/step - loss: 0.1588 - acc: 0.9927 - val_loss: 0.3509 - val_acc: 0.9417 Epoch 392/500 51s 102ms/step - loss: 0.1596 - acc: 0.9924 - val_loss: 0.3496 - val_acc: 0.9404 Epoch 393/500 51s 102ms/step - loss: 0.1584 - acc: 0.9928 - val_loss: 0.3477 - val_acc: 0.9425 Epoch 394/500 51s 102ms/step - loss: 0.1588 - acc: 0.9921 - val_loss: 0.3503 - val_acc: 0.9397 Epoch 395/500 51s 103ms/step - loss: 0.1582 - acc: 0.9925 - val_loss: 0.3512 - val_acc: 0.9399 Epoch 396/500 51s 102ms/step - loss: 0.1570 - acc: 0.9931 - val_loss: 0.3508 - val_acc: 0.9406 Epoch 397/500 51s 102ms/step - loss: 0.1573 - acc: 0.9927 - val_loss: 0.3472 - val_acc: 0.9397 Epoch 398/500 51s 102ms/step - loss: 0.1562 - acc: 0.9928 - val_loss: 0.3480 - val_acc: 0.9408ETA: 0s - loss: 0.1562 - acc: 0.9928 Epoch 399/500 51s 102ms/step - loss: 0.1569 - acc: 0.9930 - val_loss: 0.3514 - val_acc: 0.9402 Epoch 400/500 51s 102ms/step - loss: 0.1557 - acc: 0.9932 - val_loss: 0.3516 - val_acc: 0.9390 Epoch 401/500 51s 102ms/step - loss: 0.1567 - acc: 0.9929 - val_loss: 0.3554 - val_acc: 0.9403 Epoch 402/500 51s 102ms/step - loss: 0.1572 - acc: 0.9922 - val_loss: 0.3525 - val_acc: 0.9385 Epoch 403/500 51s 102ms/step - loss: 0.1560 - acc: 0.9929 - val_loss: 0.3521 - val_acc: 0.9395 Epoch 404/500 51s 102ms/step - loss: 0.1564 - acc: 0.9927 - val_loss: 0.3491 - val_acc: 0.9402 Epoch 405/500 51s 103ms/step - loss: 0.1543 - acc: 0.9932 - val_loss: 0.3494 - val_acc: 0.9417 Epoch 406/500 51s 102ms/step - loss: 0.1553 - acc: 0.9926 - val_loss: 0.3483 - val_acc: 0.9408 Epoch 407/500 51s 102ms/step - loss: 0.1548 - acc: 0.9930 - val_loss: 0.3520 - val_acc: 0.9399 Epoch 408/500 51s 102ms/step - loss: 0.1548 - acc: 0.9929 - val_loss: 0.3523 - val_acc: 0.9411 Epoch 409/500 51s 102ms/step - loss: 0.1552 - acc: 0.9926 - val_loss: 0.3506 - val_acc: 0.9399 Epoch 410/500 51s 102ms/step - loss: 0.1530 - acc: 0.9939 - val_loss: 0.3460 - val_acc: 0.9406 Epoch 411/500 51s 102ms/step - loss: 0.1549 - acc: 0.9924 - val_loss: 0.3445 - val_acc: 0.9397 Epoch 412/500 51s 102ms/step - loss: 0.1533 - acc: 0.9931 - val_loss: 0.3445 - val_acc: 0.9410 Epoch 413/500 51s 102ms/step - loss: 0.1534 - acc: 0.9928 - val_loss: 0.3471 - val_acc: 0.9406 Epoch 414/500 51s 102ms/step - loss: 0.1542 - acc: 0.9922 - val_loss: 0.3478 - val_acc: 0.9417 Epoch 415/500 51s 102ms/step - loss: 0.1518 - acc: 0.9934 - val_loss: 0.3527 - val_acc: 0.9414 Epoch 416/500 51s 102ms/step - loss: 0.1529 - acc: 0.9926 - val_loss: 0.3484 - val_acc: 0.9423 Epoch 417/500 51s 102ms/step - loss: 0.1533 - acc: 0.9928 - val_loss: 0.3468 - val_acc: 0.9421 Epoch 418/500 51s 103ms/step - loss: 0.1531 - acc: 0.9927 - val_loss: 0.3497 - val_acc: 0.9418 Epoch 419/500 51s 102ms/step - loss: 0.1519 - acc: 0.9931 - val_loss: 0.3500 - val_acc: 0.9413 Epoch 420/500 51s 102ms/step - loss: 0.1512 - acc: 0.9932 - val_loss: 0.3480 - val_acc: 0.9418 Epoch 421/500 51s 102ms/step - loss: 0.1512 - acc: 0.9931 - val_loss: 0.3491 - val_acc: 0.9414 Epoch 422/500 51s 102ms/step - loss: 0.1518 - acc: 0.9927 - val_loss: 0.3464 - val_acc: 0.9412 Epoch 423/500 51s 102ms/step - loss: 0.1505 - acc: 0.9935 - val_loss: 0.3495 - val_acc: 0.9400 Epoch 424/500 51s 102ms/step - loss: 0.1502 - acc: 0.9930 - val_loss: 0.3467 - val_acc: 0.9418 Epoch 425/500 51s 102ms/step - loss: 0.1513 - acc: 0.9928 - val_loss: 0.3472 - val_acc: 0.9391 Epoch 426/500 51s 102ms/step - loss: 0.1496 - acc: 0.9933 - val_loss: 0.3430 - val_acc: 0.9400 Epoch 427/500 51s 102ms/step - loss: 0.1490 - acc: 0.9935 - val_loss: 0.3452 - val_acc: 0.9398 Epoch 428/500 51s 102ms/step - loss: 0.1499 - acc: 0.9931 - val_loss: 0.3456 - val_acc: 0.9394 Epoch 429/500 51s 102ms/step - loss: 0.1496 - acc: 0.9933 - val_loss: 0.3405 - val_acc: 0.9426 Epoch 430/500 51s 102ms/step - loss: 0.1486 - acc: 0.9933 - val_loss: 0.3467 - val_acc: 0.9404 Epoch 431/500 51s 103ms/step - loss: 0.1481 - acc: 0.9933 - val_loss: 0.3448 - val_acc: 0.9408 Epoch 432/500 51s 102ms/step - loss: 0.1479 - acc: 0.9936 - val_loss: 0.3479 - val_acc: 0.9390 Epoch 433/500 51s 102ms/step - loss: 0.1485 - acc: 0.9934 - val_loss: 0.3484 - val_acc: 0.9380 Epoch 434/500 51s 102ms/step - loss: 0.1479 - acc: 0.9935 - val_loss: 0.3495 - val_acc: 0.9384 Epoch 435/500 51s 102ms/step - loss: 0.1468 - acc: 0.9940 - val_loss: 0.3490 - val_acc: 0.9425 Epoch 436/500 51s 103ms/step - loss: 0.1482 - acc: 0.9929 - val_loss: 0.3538 - val_acc: 0.9386 Epoch 437/500 51s 102ms/step - loss: 0.1478 - acc: 0.9932 - val_loss: 0.3494 - val_acc: 0.9400 Epoch 438/500 51s 102ms/step - loss: 0.1476 - acc: 0.9937 - val_loss: 0.3508 - val_acc: 0.9386 Epoch 439/500 51s 102ms/step - loss: 0.1483 - acc: 0.9932 - val_loss: 0.3483 - val_acc: 0.9386 Epoch 440/500 51s 103ms/step - loss: 0.1475 - acc: 0.9932 - val_loss: 0.3466 - val_acc: 0.9402 Epoch 441/500 51s 102ms/step - loss: 0.1473 - acc: 0.9932 - val_loss: 0.3477 - val_acc: 0.9406 Epoch 442/500 51s 102ms/step - loss: 0.1469 - acc: 0.9931 - val_loss: 0.3461 - val_acc: 0.9402 Epoch 443/500 51s 102ms/step - loss: 0.1460 - acc: 0.9936 - val_loss: 0.3483 - val_acc: 0.9404 Epoch 444/500 51s 102ms/step - loss: 0.1463 - acc: 0.9936 - val_loss: 0.3438 - val_acc: 0.9407 Epoch 445/500 51s 102ms/step - loss: 0.1454 - acc: 0.9936 - val_loss: 0.3488 - val_acc: 0.9401 Epoch 446/500 51s 102ms/step - loss: 0.1452 - acc: 0.9935 - val_loss: 0.3450 - val_acc: 0.9402 Epoch 447/500 51s 102ms/step - loss: 0.1452 - acc: 0.9933 - val_loss: 0.3494 - val_acc: 0.9401 Epoch 448/500 51s 102ms/step - loss: 0.1447 - acc: 0.9936 - val_loss: 0.3448 - val_acc: 0.9418 Epoch 449/500 51s 102ms/step - loss: 0.1446 - acc: 0.9936 - val_loss: 0.3507 - val_acc: 0.9403 Epoch 450/500 51s 103ms/step - loss: 0.1442 - acc: 0.9935 - val_loss: 0.3464 - val_acc: 0.9401 Epoch 451/500 lr changed to 9.999999310821295e-05 51s 102ms/step - loss: 0.1437 - acc: 0.9936 - val_loss: 0.3447 - val_acc: 0.9410 Epoch 452/500 51s 102ms/step - loss: 0.1435 - acc: 0.9938 - val_loss: 0.3450 - val_acc: 0.9408ETA: 19s - loss: 0.1439 - acc: 0.9936 Epoch 453/500 51s 102ms/step - loss: 0.1421 - acc: 0.9943 - val_loss: 0.3444 - val_acc: 0.9402 Epoch 454/500 51s 102ms/step - loss: 0.1419 - acc: 0.9945 - val_loss: 0.3443 - val_acc: 0.9407 Epoch 455/500 51s 102ms/step - loss: 0.1424 - acc: 0.9943 - val_loss: 0.3445 - val_acc: 0.9407 Epoch 456/500 51s 102ms/step - loss: 0.1425 - acc: 0.9942 - val_loss: 0.3442 - val_acc: 0.9406 Epoch 457/500 51s 102ms/step - loss: 0.1420 - acc: 0.9946 - val_loss: 0.3440 - val_acc: 0.9408 Epoch 458/500 51s 102ms/step - loss: 0.1422 - acc: 0.9938 - val_loss: 0.3443 - val_acc: 0.9407 Epoch 459/500 51s 102ms/step - loss: 0.1399 - acc: 0.9953 - val_loss: 0.3442 - val_acc: 0.9411 Epoch 460/500 51s 102ms/step - loss: 0.1417 - acc: 0.9944 - val_loss: 0.3446 - val_acc: 0.9415 Epoch 461/500 51s 102ms/step - loss: 0.1420 - acc: 0.9942 - val_loss: 0.3451 - val_acc: 0.9415 Epoch 462/500 51s 102ms/step - loss: 0.1402 - acc: 0.9949 - val_loss: 0.3443 - val_acc: 0.9413 Epoch 463/500 51s 102ms/step - loss: 0.1414 - acc: 0.9944 - val_loss: 0.3437 - val_acc: 0.9407 Epoch 464/500 51s 102ms/step - loss: 0.1411 - acc: 0.9947 - val_loss: 0.3442 - val_acc: 0.9408 Epoch 465/500 51s 102ms/step - loss: 0.1403 - acc: 0.9949 - val_loss: 0.3440 - val_acc: 0.9414 Epoch 466/500 51s 103ms/step - loss: 0.1406 - acc: 0.9948 - val_loss: 0.3439 - val_acc: 0.9408 Epoch 467/500 51s 102ms/step - loss: 0.1407 - acc: 0.9947 - val_loss: 0.3438 - val_acc: 0.9412 Epoch 468/500 51s 102ms/step - loss: 0.1419 - acc: 0.9942 - val_loss: 0.3438 - val_acc: 0.9413 Epoch 469/500 51s 102ms/step - loss: 0.1417 - acc: 0.9945 - val_loss: 0.3443 - val_acc: 0.9406 Epoch 470/500 51s 102ms/step - loss: 0.1409 - acc: 0.9945 - val_loss: 0.3439 - val_acc: 0.9407 Epoch 471/500 51s 102ms/step - loss: 0.1418 - acc: 0.9939 - val_loss: 0.3439 - val_acc: 0.9410 Epoch 472/500 51s 102ms/step - loss: 0.1407 - acc: 0.9948 - val_loss: 0.3434 - val_acc: 0.9408 Epoch 473/500 51s 102ms/step - loss: 0.1413 - acc: 0.9942 - val_loss: 0.3438 - val_acc: 0.9410 Epoch 474/500 51s 102ms/step - loss: 0.1394 - acc: 0.9950 - val_loss: 0.3439 - val_acc: 0.9413 Epoch 475/500 51s 102ms/step - loss: 0.1405 - acc: 0.9946 - val_loss: 0.3438 - val_acc: 0.9414 Epoch 476/500 51s 102ms/step - loss: 0.1396 - acc: 0.9949 - val_loss: 0.3438 - val_acc: 0.9411 Epoch 477/500 51s 102ms/step - loss: 0.1401 - acc: 0.9950 - val_loss: 0.3439 - val_acc: 0.9411 Epoch 478/500 51s 102ms/step - loss: 0.1406 - acc: 0.9947 - val_loss: 0.3442 - val_acc: 0.9408 Epoch 479/500 51s 102ms/step - loss: 0.1394 - acc: 0.9952 - val_loss: 0.3446 - val_acc: 0.9408 Epoch 480/500 51s 102ms/step - loss: 0.1399 - acc: 0.9950 - val_loss: 0.3447 - val_acc: 0.9402 Epoch 481/500 51s 102ms/step - loss: 0.1409 - acc: 0.9945 - val_loss: 0.3451 - val_acc: 0.9405 Epoch 482/500 51s 102ms/step - loss: 0.1392 - acc: 0.9955 - val_loss: 0.3446 - val_acc: 0.9406 Epoch 483/500 51s 102ms/step - loss: 0.1403 - acc: 0.9948 - val_loss: 0.3450 - val_acc: 0.9402 Epoch 484/500 51s 102ms/step - loss: 0.1396 - acc: 0.9950 - val_loss: 0.3450 - val_acc: 0.9403 Epoch 485/500 51s 102ms/step - loss: 0.1395 - acc: 0.9949 - val_loss: 0.3449 - val_acc: 0.9401 Epoch 486/500 51s 102ms/step - loss: 0.1413 - acc: 0.9944 - val_loss: 0.3450 - val_acc: 0.9402 Epoch 487/500 51s 103ms/step - loss: 0.1406 - acc: 0.9945 - val_loss: 0.3447 - val_acc: 0.9403 Epoch 488/500 51s 102ms/step - loss: 0.1403 - acc: 0.9943 - val_loss: 0.3448 - val_acc: 0.9409 Epoch 489/500 51s 102ms/step - loss: 0.1408 - acc: 0.9948 - val_loss: 0.3442 - val_acc: 0.9413 Epoch 490/500 51s 102ms/step - loss: 0.1398 - acc: 0.9951 - val_loss: 0.3438 - val_acc: 0.9416 Epoch 491/500 51s 102ms/step - loss: 0.1398 - acc: 0.9947 - val_loss: 0.3438 - val_acc: 0.9409 Epoch 492/500 51s 102ms/step - loss: 0.1396 - acc: 0.9951 - val_loss: 0.3437 - val_acc: 0.9414 Epoch 493/500 51s 102ms/step - loss: 0.1402 - acc: 0.9947 - val_loss: 0.3434 - val_acc: 0.9416 Epoch 494/500 51s 102ms/step - loss: 0.1400 - acc: 0.9951 - val_loss: 0.3441 - val_acc: 0.9413 Epoch 495/500 51s 102ms/step - loss: 0.1395 - acc: 0.9952 - val_loss: 0.3442 - val_acc: 0.9409 Epoch 496/500 51s 102ms/step - loss: 0.1395 - acc: 0.9948 - val_loss: 0.3446 - val_acc: 0.9413 Epoch 497/500 51s 102ms/step - loss: 0.1386 - acc: 0.9953 - val_loss: 0.3444 - val_acc: 0.9415 Epoch 498/500 51s 102ms/step - loss: 0.1396 - acc: 0.9951 - val_loss: 0.3440 - val_acc: 0.9415 Epoch 499/500 51s 102ms/step - loss: 0.1396 - acc: 0.9950 - val_loss: 0.3444 - val_acc: 0.9414 Epoch 500/500 51s 102ms/step - loss: 0.1386 - acc: 0.9951 - val_loss: 0.3449 - val_acc: 0.9417 Train loss: 0.128365815192461 Train accuracy: 0.9987200012207031 Test loss: 0.3449158281087875 Test accuracy: 0.941700000166893
相较于调参记录18的94.28%，这次的测试准确率低了一点。
但是，值得指出的是，这次只训练了500个epoch，而调参记录18训练了5000个epoch。通过观察loss可以发现，其实这次的loss下降得更快。
Minghang Zhao, Shisheng Zhong, Xuyun Fu, Baoping Tang, Shaojiang Dong, Michael Pecht, Deep Residual Networks with Adaptively Parametric Rectifier Linear Units for Fault Diagnosis, IEEE Transactions on Industrial Electronics, 2020, DOI: 10.1109/TIE.2020.2972458
