机器学习——快餐类型识别

 1 选题背景

　　随着信息网络的运用，部分餐厅已经实现点餐结账无人化，解放了收款人员。一些互联网公司与更大的连锁餐饮企业联手“智慧餐厅”，打造出更加极致的“无人概念餐厅”，如北京海底捞无人餐厅，店里全采用机器人服务，这种运营服务也让体力老店铺那个得到解脱，相比以往的服务这种服务更加高效。对于实现“无人餐厅”，计算机视觉技术是机器人服务员辨别对应的菜品的重要手段。

本项目将利用kaggle平台的快餐图像数据构建神经网络模型，训练出一个能够有效辨别快餐类型的模型。

2. 数据来源

Kaggle，网址：https://www.kaggle.com/datasets/utkarshsaxenadn/fast-food-classification-dataset

3 机器学习的实现步骤

3.1 环境准备

导入pandas，numpy，matplotlib，os，pathlib，sklearn库，导入keras，tensorflow框架。

import pandas as pd
import numpy as np
import tensorflow as tf
from sklearn.model_selection import train_test_split
import os
from keras.models import Sequential, load_model
from keras.layers import Activation,Dense, Dropout, Flatten, Conv2D, MaxPool2D
from keras.applications.resnet import preprocess_input
from keras.preprocessing.image import ImageDataGenerator
from keras.preprocessing import image
from keras import optimizers
from pathlib import Path
import matplotlib.pyplot as plt

3.2 图像数据读取

读取Baked Potato，Burger，Fries快餐的JPEG格式文件名，存储至列表filepaths。

# 读取Baked Potato，Burger,Fries,Pizza,Sandwich,Taquito快餐的JPEG格式文件名
dir_BakedPotato = Path('数据集/Baked Potato')
filepaths_BakedPotato = list(dir_BakedPotato.glob('**/*.JPEG'))

dir_Burger = Path('数据集/Burger')
filepaths_Burger = list(dir_Burger.glob('**/*.JPEG'))

dir_Fries = Path('数据集/Fries')
filepaths_Fries = list(dir_Fries.glob('**/*.JPEG'))

3.3 图像数据处理

# 构建列表，存储图片路径
filepaths = [filepaths_BakedPotato,filepaths_Burger,filepaths_Fries,]
# 把二维列表降一维
filepaths = sum(filepaths, [])

# 构建列表，存储图片对应类型
list_BakedPotato = list(map(lambda x:'BakedPotato', [i for i in range(len(filepaths_BakedPotato))]))
list_Burger = list(map(lambda x:'Burger', [i for i in range(len(filepaths_Burger))]))
list_Fries = list(map(lambda x:'Fries', [i for i in range(len(filepaths_Fries))]))
label = [list_BakedPotato,list_Burger,list_Fries,]
print(len(label))

# 把二维列表降一维
label = sum(label, [])
print(len(label))
# 合并两个列表为数据框
filepaths_S =  pd.Series(filepaths,name='FilePaths')
label_S = pd.Series(label,name='labels')
data = pd.merge(filepaths_S,label_S,right_index=True,left_index=True)
print(data.head())
print(data.info())

 

3.3.2 使用数据框中图片路径查看图片。

# 查看图像
pic = plt.figure(figsize=(12,7))
l1 = [3,1502,3002]
for i in range(1,4,1):
    ax = pic.add_subplot(2, 3, i)
    plt.imshow(plt.imread(data['FilePaths'][l1[i-1]]))
    plt.title(data['labels'][l1[i-1]])
plt.savefig('图像.jpg')
plt.show()

3.4 划分图像数据集

利用sklearn中train_test_split函数划分数据集，划分后，查看训练集，测试集，验证集的形状以及各类型图片的张数。

# 划分数据集,85:15划分x_train,x_test
data['FilePaths'] = data['FilePaths'].astype(str)
X_train, X_test = train_test_split(data, test_size=0.15,stratify=data['labels'])
print('训练集形状', X_train.shape)
print('测试集形状', X_test.shape)

# 划分数据集,4:1划分x_train,x_val
X_train, X_val = train_test_split(X_train, test_size=0.2,stratify=X_train['labels'])
print('训练集形状', X_train.shape)
print('验证集形状', X_val.shape)

# 查看各类型的图片张数
print(X_train['labels'].value_counts())
print(X_train['FilePaths'].shape)

3.5 图像预处理

使用tensorflow框架里面的ImageDataGenerator函数对图像预处理,批量数据的尺寸batch_size=32，随机种子seed=30。

# 图像预处理
img_preprocessing = ImageDataGenerator(rescale=1./255)
x_train = img_preprocessing.flow_from_dataframe(dataframe=X_train,
                                                x_col='FilePaths',
                                                y_col='labels',
                                                target_size=(112, 112),
                                                color_mode='rgb',
                                                class_mode='categorical',
                                                batch_size=32,
                                                seed=30)

x_test = img_preprocessing.flow_from_dataframe(dataframe=X_test,
                                               x_col='FilePaths',
                                               y_col='labels',
                                               target_size=(112, 112),
                                               color_mode='rgb',
                                               class_mode='categorical',
                                               batch_size=32,
                                               seed=30)

x_val = img_preprocessing.flow_from_dataframe(dataframe=X_val,
                                              x_col='FilePaths',
                                              y_col='labels',
                                              target_size=(112, 112),
                                              color_mode='rgb',
                                              class_mode='categorical',
                                              batch_size=32,
                                              seed=30)

3.6 构建神经网络模型

# 构建神经网络模型并训练
model = Sequential()

# Conv2D层，32个滤波器
model.add(Conv2D(filters=32, kernel_size=(3,3), padding='same'))
model.add(Activation('relu'))
model.add(MaxPool2D(pool_size=(2,2), strides=2, padding='valid'))

# Conv2D层，64个滤波器
model.add(Conv2D(filters=64, kernel_size=(3,3), padding='same'))
model.add(Activation('relu'))
model.add(MaxPool2D(pool_size=(2,2), strides=2, padding='valid'))

# Conv2D层，128个滤波器
model.add(Conv2D(filters=128, kernel_size=(3,3), padding='same'))
model.add(Activation('relu'))
model.add(MaxPool2D(pool_size=(2,2), strides=2, padding='valid'))

# 展平数据，降维
model.add(Flatten())

# 全连接层
model.add(Dense(256))
model.add(Activation('relu'))

# 减少过拟合
model.add(Dropout(0.5))

# 全连接层
model.add(Dense(3)) # 识别6种类
model.add(Activation('softmax')) # #使用softmax进行分类


# 模型编译
model.compile(optimizer=optimizers.RMSprop(learning_rate=1e-4), loss="categorical_crossentropy",
              metrics=["accuracy"]) # metrics指定衡量模型的指标

3.7 模型训练

3.7.1 设置参数epichs训练80轮，并打印验证集的最终的损失与准确率

# h1模型训练
print('training-----')
h1 = model.fit(x_train,validation_data=x_val,epochs=80)
model.summary()

print('\validating-----')
loss,accuracy=model.evaluate(x_val)

print('\nvalidation loss: ',loss)
print('\nvalidation accuracy:　',accuracy)

# 保存模型
model.save('h21')

 

3.7.2 绘制h1模型的损失变化曲线与准确率变化曲线，观察它们的变化

# 绘制h1模型损失变化曲线
loss = h1.history["loss"]
epochs = range(1, len(loss)+1)
val_loss = h1.history["val_loss"]
plt.plot(epochs, loss, "bo-", label="Training Loss")
plt.plot(epochs, val_loss, "ro--", label="Validation Loss")
plt.title("Training and Validation Loss")
plt.xlabel("Epochs")
plt.ylabel("Loss")
plt.legend()
plt.savefig('h1模型损失变化曲线图.jpg')
plt.show()

# 绘制h1模型准确率变化曲线
acc = h1.history["accuracy"]
epochs = range(1, len(acc)+1)
val_acc = h1.history["val_accuracy"]
plt.plot(epochs, acc, "bo-", label="Training Acc")
plt.plot(epochs, val_acc, "ro--", label="Validation Acc")
plt.title("Training and Validation Accuracy")
plt.xlabel("Epochs")
plt.ylabel("Accuracy")
plt.legend()
plt.savefig('h1模型准确率变化曲线图.jpg')
plt.show()

3.8 模型优化

3.8.1 从h1模型的损失变化曲线与准确率变化曲线，可以得出training loss不断降低，但validation loss不断上升，说明模型过拟合，因此我们需要降低过拟合，下面通过数据增强来降低过拟合。

# 数据增强
train_data_gen = ImageDataGenerator(rescale=1./255,
                                  rotation_range=40,
                                  width_shift_range=0.2,
                                  height_shift_range=0.2,
                                  shear_range=0.2,
                                  zoom_range=0.2,
                                  horizontal_flip=True,
                                  fill_mode='nearest')

val_data_gen = ImageDataGenerator(rescale=1./255)

x_train1 = train_data_gen.flow_from_dataframe(dataframe=X_train,
                                            x_col='FilePaths',
                                            y_col='labels',
                                            target_size=(112, 112),
                                            color_mode='rgb',
                                            class_mode='categorical',
                                            batch_size=32,
                                            seed=30)


x_val1 = val_data_gen.flow_from_dataframe(dataframe=X_val,
                                          x_col='FilePaths',
                                          y_col='labels',
                                          target_size=(112, 112),
                                          color_mode='rgb',
                                          class_mode='categorical',
                                          batch_size=32,
                                          seed=30)

3.8.2 激活特征图和打印特征图

 

img_path = "训练/Burger/Burger-Train (1470).jpeg"
img = image_utils.load_img(img_path, target_size=(150,150))
img_tensor = image_utils.img_to_array(img)
img_tensor = np.expand_dims(img_tensor, axis=0)
img_tensor /= 255.
print(img_tensor.shape)
#显示样本
import matplotlib.pyplot as plt
plt.imshow(img_tensor[0])
plt.show()

layer_outputs = [layer.output for layer in model.layers[:8]]
activation_model = tf.keras.models.Model(inputs=model.input, outputs=layer_outputs)
#获得改样本的特征图
activations = activation_model.predict(img_tensor)
#第一层激活输出特的第一个滤波器的特征图
first_layer_activation = activations[0]

# 存储层的名称
layer_names = []
for layer in model.layers[:4]:
    layer_names.append(layer.name)
# 每行显示16个特征图
images_pre_row = 16  # 每行显示的特征图数
# 循环8次显示8层的全部特征图
for layer_name, layer_activation in zip(layer_names, activations):
    n_features = layer_activation.shape[-1]  # 保存当前层的特征图个数
    size = layer_activation.shape[1]  # 保存当前层特征图的宽高
    n_col = n_features // images_pre_row  # 计算当前层显示多少行
    # 生成显示图像的矩阵
    display_grid = np.zeros((size * n_col, images_pre_row * size))
    # 遍历将每个特张图的数据写入到显示图像的矩阵中
    for col in range(n_col):
        for row in range(images_pre_row):
            # 保存该张特征图的矩阵(size,size,1)
            channel_image = layer_activation[0, :, :, col * images_pre_row + row]
            # 为使图像显示更鲜明，作一些特征处理
            channel_image -= channel_image.mean()
            channel_image /= channel_image.std()
            channel_image *= 64
            channel_image += 128
            # 把该特征图矩阵中不在0-255的元素值修改至0-255
            channel_image = np.clip(channel_image, 0, 255).astype("uint8")
            # 该特征图矩阵填充至显示图像的矩阵中
            display_grid[col * size:(col + 1) * size, row * size:(row + 1) * size] = channel_image

    scale = 1. / size
    # 设置该层显示图像的宽高
    plt.figure(figsize=(scale * display_grid.shape[1], scale * display_grid.shape[0]))
    plt.title(layer_name)
    plt.grid(False)
    # 显示图像
    plt.imshow(display_grid, aspect="auto", cmap="viridis")
plt.show()

3.8.3 再次训练模型，并打印验证集的最终的损失与准确率

# h2模型训练
print('training-----')
h2 = model.fit(x_train1,validation_data=x_val1,epochs=50)

print('\validating-----')
loss,accuracy=model.evaluate(x_val1)

print('\nvalidation loss: ',loss)
print('\nvalidation accuracy:　',accuracy)

# 保存模型
model.save('h22')

3.8.3 绘制h2模型损失变化曲线与准确率变化曲线

# 绘制h2模型损失变化曲线
loss = h2.history["loss"]
epochs = range(1, len(loss)+1)
val_loss = h2.history["val_loss"]
plt.plot(epochs, loss, "bo-", label="Training Loss")
plt.plot(epochs, val_loss, "ro--", label="Validation Loss")
plt.title("Training and Validation Loss")
plt.xlabel("Epochs")
plt.ylabel("Loss")
plt.legend()
plt.savefig('h2模型损失变化曲线图.jpg')
plt.show()

# 绘制h2模型准确率变化曲线
acc = h2.history["accuracy"]
epochs = range(1, len(acc)+1)
val_acc = h2.history["val_accuracy"]
plt.plot(epochs, acc, "bo-", label="Training Acc")
plt.plot(epochs, val_acc, "ro--", label="Validation Acc")
plt.title("Training and Validation Accuracy")
plt.xlabel("Epochs")
plt.ylabel("Accuracy")
plt.legend()
plt.savefig('h2模型准确率变化曲线图.jpg')
plt.show()

 

准确率基本是问题提高，说明模型拟合程度不错。

3.9 模型预测

# h1模型预测
# 3类型
l1 = ['BakedPotato','Burger','Fries',]
# 原图
i = 10
p1 = plt.figure(figsize=(12,7))
img1,label1 = x_test.next()
plt.rcParams["font.sans-serif"]=["FangSong"] # 解决中文显示异常
ax1 = p1.add_subplot(2, 1, 1)
plt.imshow((img1[i]*255).astype('uint8'))
plt.title('实际类型为：' + l1[np.argmax(label1[i])])

# 将图片转换成 4D 张量
x_test_img1 = img1[i].reshape(1, 112, 112, 3).astype("float32")

# 预测结果的概率柱状图
pr1 = h1.predict(x_test_img1)
ax2 = p1.add_subplot(2, 1, 2)
plt.title("预测结果的概率柱状图")
plt.bar(np.arange(3), pr1.reshape(3), align="center")
plt.xticks(np.arange(3),l1)
plt.show()

# 实际类型与h1模型预测结果对比
print('实际类型：' + l1[np.argmax(label1[i])])
print('h1模型预测的结果是：' + l1[np.argmax(pr1)])

# 加载h2模型
h2 = load_model('h22')

# h2模型预测
# 3类型
l2 = ['BakedPotato','Burger','Fries']
# 原图
i = 10
p2 = plt.figure(figsize=(12,7))
img2,label2 = x_test.next()
plt.rcParams["font.sans-serif"]=["FangSong"] # 解决中文显示异常
ax3 = p2.add_subplot(2, 1, 1)
plt.imshow((img2[i]*255).astype('uint8'))
plt.title('实际类型为：' + l2[np.argmax(label2[i])])

# 将图片转换成 4D 张量
x_test_img2 = img2[i].reshape(1, 112, 112, 3).astype("float32")

# 预测结果的概率柱状图
pr2 = h2.predict(x_test_img2)
ax4 = p2.add_subplot(2, 1, 2)
plt.title("预测结果的概率柱状图")
plt.bar(np.arange(3), pr2.reshape(3), align="center")
plt.xticks(np.arange(3),l2)
plt.show()

# 实际类型与h2模型预测结果对比
print('实际类型：' + l2[np.argmax(label2[i])])
print('h2模型预测的结果是：' + l2[np.argmax(pr2)])

最终我选择绘制概率柱状图来显示数据，方便查看和识别。

 

 

4 总结

　　结论：从最终效果可以得出，我们最开始出现过拟合的情况得到了比较有效的解决，快餐食品的识别也是可以正常的运行，从我们的损失变化曲线和准确率变化曲线可以看出，Training loss跟Validation loss都在下降,Training acc和Validation acc在不断的上升。并且我们的validation acc几乎始终比training acc高，我们的模型的拟合程度好,最终通过长时间训练，准确度得到了良好的提升。

　　个人收获：本次机器学习我查阅了大量资料，在最开始选题的时候就出现了问题，有很多的想法但是不切实际，代码实力不对等，最终选择类别识别的项目来做参考文件比较多，在开发过程中也遇到了大量的问题，查阅资料，询问一些相关行业朋友，通过大量的时间最终开发完成，开发中遇到了过拟合问题，根据课上的讲解，我先尝试Dropout,尝试了数据增强，最终大概解决了问题，然后记录下来，是一段宝贵的经历。学习了不少的代码经验，虽然没有系统化学习，但是我对此展开了较大的兴趣，希望在未来空闲时间去进行系统化的学习，自我的研究。

5 本文全部代码

import pandas as pd
import numpy as np
import tensorflow as tf
from keras.utils import image_utils
from sklearn.model_selection import train_test_split
import os
from keras.models import Sequential, load_model
from keras.layers import Activation,Dense, Dropout, Flatten, Conv2D, MaxPool2D
from keras.applications.resnet import preprocess_input
from keras.preprocessing.image import ImageDataGenerator
from keras.preprocessing import image
from keras import optimizers, activations
from pathlib import Path
import matplotlib.pyplot as plt

# 读取Baked Potato，Burger,Fries,Pizza,Sandwich,Taquito快餐的JPEG格式文件名
dir_BakedPotato = Path('数据集/Baked Potato')
filepaths_BakedPotato = list(dir_BakedPotato.glob('**/*.JPEG'))

dir_Burger = Path('数据集/Burger')
filepaths_Burger = list(dir_Burger.glob('**/*.JPEG'))

dir_Fries = Path('数据集/Fries')
filepaths_Fries = list(dir_Fries.glob('**/*.JPEG'))

# 构建列表，存储图片路径
filepaths = [filepaths_BakedPotato,filepaths_Burger,filepaths_Fries,]
# 把二维列表降一维
filepaths = sum(filepaths, [])

# 构建列表，存储图片对应类型
list_BakedPotato = list(map(lambda x:'BakedPotato', [i for i in range(len(filepaths_BakedPotato))]))
list_Burger = list(map(lambda x:'Burger', [i for i in range(len(filepaths_Burger))]))
list_Fries = list(map(lambda x:'Fries', [i for i in range(len(filepaths_Fries))]))
label = [list_BakedPotato,list_Burger,list_Fries,]
print(len(label))

# 把二维列表降一维
label = sum(label, [])
print(len(label))
# 合并两个列表为数据框
filepaths_S =  pd.Series(filepaths,name='FilePaths')
label_S = pd.Series(label,name='labels')
data = pd.merge(filepaths_S,label_S,right_index=True,left_index=True)
print(data.head())
print(data.info())

# 查看图像
pic = plt.figure(figsize=(12,7))
l1 = [3,1502,3002]
for i in range(1,4,1):
    ax = pic.add_subplot(2, 3, i)
    plt.imshow(plt.imread(data['FilePaths'][l1[i-1]]))
    plt.title(data['labels'][l1[i-1]])
plt.savefig('图像.jpg')
plt.show()

# 划分数据集,85:15划分x_train,x_test
data['FilePaths'] = data['FilePaths'].astype(str)
X_train, X_test = train_test_split(data, test_size=0.15,stratify=data['labels'])
print('训练集形状', X_train.shape)
print('测试集形状', X_test.shape)

# 划分数据集,4:1划分x_train,x_val
X_train, X_val = train_test_split(X_train, test_size=0.2,stratify=X_train['labels'])
print('训练集形状', X_train.shape)
print('验证集形状', X_val.shape)

# 查看各类型的图片张数
print(X_train['labels'].value_counts())
print(X_train['FilePaths'].shape)

# 图像预处理
img_preprocessing = ImageDataGenerator(rescale=1./255)
x_train = img_preprocessing.flow_from_dataframe(dataframe=X_train,
                                                x_col='FilePaths',
                                                y_col='labels',
                                                target_size=(112, 112),
                                                color_mode='rgb',
                                                class_mode='categorical',
                                                batch_size=32,
                                                seed=30)

x_test = img_preprocessing.flow_from_dataframe(dataframe=X_test,
                                               x_col='FilePaths',
                                               y_col='labels',
                                               target_size=(112, 112),
                                               color_mode='rgb',
                                               class_mode='categorical',
                                               batch_size=32,
                                               seed=30)

x_val = img_preprocessing.flow_from_dataframe(dataframe=X_val,
                                              x_col='FilePaths',
                                              y_col='labels',
                                              target_size=(112, 112),
                                              color_mode='rgb',
                                              class_mode='categorical',
                                              batch_size=32,
                                              seed=30)

# 构建神经网络模型并训练
model = Sequential()

# Conv2D层，32个滤波器
model.add(Conv2D(filters=32, kernel_size=(3,3), padding='same'))
model.add(Activation('relu'))
model.add(MaxPool2D(pool_size=(2,2), strides=2, padding='valid'))

# Conv2D层，64个滤波器
model.add(Conv2D(filters=64, kernel_size=(3,3), padding='same'))
model.add(Activation('relu'))
model.add(MaxPool2D(pool_size=(2,2), strides=2, padding='valid'))

# Conv2D层，128个滤波器
model.add(Conv2D(filters=128, kernel_size=(3,3), padding='same'))
model.add(Activation('relu'))
model.add(MaxPool2D(pool_size=(2,2), strides=2, padding='valid'))

# 展平数据，降维
model.add(Flatten())

# 全连接层
model.add(Dense(256))
model.add(Activation('relu'))

# 减少过拟合
model.add(Dropout(0.5))

# 全连接层
model.add(Dense(3)) # 识别6种类
model.add(Activation('softmax')) # #使用softmax进行分类


# 模型编译
model.compile(optimizer=optimizers.RMSprop(learning_rate=1e-4), loss="categorical_crossentropy",
              metrics=["accuracy"]) # metrics指定衡量模型的指标

# h1模型训练
print('training-----')
h1 = model.fit(x_train,validation_data=x_val,epochs=80)
model.summary()

print('\validating-----')
loss,accuracy=model.evaluate(x_val)

print('\nvalidation loss: ',loss)
print('\nvalidation accuracy:　',accuracy)

# 保存模型
model.save('h21')

# 绘制h1模型损失变化曲线
loss = h1.history["loss"]
epochs = range(1, len(loss)+1)
val_loss = h1.history["val_loss"]
plt.plot(epochs, loss, "bo-", label="Training Loss")
plt.plot(epochs, val_loss, "ro--", label="Validation Loss")
plt.title("Training and Validation Loss")
plt.xlabel("Epochs")
plt.ylabel("Loss")
plt.legend()
plt.savefig('h1模型损失变化曲线图.jpg')
plt.show()

# 绘制h1模型准确率变化曲线
acc = h1.history["accuracy"]
epochs = range(1, len(acc)+1)
val_acc = h1.history["val_accuracy"]
plt.plot(epochs, acc, "bo-", label="Training Acc")
plt.plot(epochs, val_acc, "ro--", label="Validation Acc")
plt.title("Training and Validation Accuracy")
plt.xlabel("Epochs")
plt.ylabel("Accuracy")
plt.legend()
plt.savefig('h1模型准确率变化曲线图.jpg')
plt.show()

# 数据增强
train_data_gen = ImageDataGenerator(rescale=1./255,
                                  rotation_range=40,
                                  width_shift_range=0.2,
                                  height_shift_range=0.2,
                                  shear_range=0.2,
                                  zoom_range=0.2,
                                  horizontal_flip=True,
                                  fill_mode='nearest')

val_data_gen = ImageDataGenerator(rescale=1./255)

x_train1 = train_data_gen.flow_from_dataframe(dataframe=X_train,
                                            x_col='FilePaths',
                                            y_col='labels',
                                            target_size=(112, 112),
                                            color_mode='rgb',
                                            class_mode='categorical',
                                            batch_size=32,
                                            seed=30)


x_val1 = val_data_gen.flow_from_dataframe(dataframe=X_val,
                                          x_col='FilePaths',
                                          y_col='labels',
                                          target_size=(112, 112),
                                          color_mode='rgb',
                                          class_mode='categorical',
                                          batch_size=32,
                                          seed=30)

img_path = "训练/Burger/Burger-Train (1470).jpeg"
img = image_utils.load_img(img_path, target_size=(150,150))
img_tensor = image_utils.img_to_array(img)
img_tensor = np.expand_dims(img_tensor, axis=0)
img_tensor /= 255.
print(img_tensor.shape)
#显示样本
import matplotlib.pyplot as plt
plt.imshow(img_tensor[0])
plt.show()

layer_outputs = [layer.output for layer in model.layers[:8]]
activation_model = tf.keras.models.Model(inputs=model.input, outputs=layer_outputs)
#获得改样本的特征图
activations = activation_model.predict(img_tensor)
#第一层激活输出特的第一个滤波器的特征图
first_layer_activation = activations[0]

# 存储层的名称
layer_names = []
for layer in model.layers[:4]:
    layer_names.append(layer.name)
# 每行显示16个特征图
images_pre_row = 16  # 每行显示的特征图数
# 循环8次显示8层的全部特征图
for layer_name, layer_activation in zip(layer_names, activations):
    n_features = layer_activation.shape[-1]  # 保存当前层的特征图个数
    size = layer_activation.shape[1]  # 保存当前层特征图的宽高
    n_col = n_features // images_pre_row  # 计算当前层显示多少行
    # 生成显示图像的矩阵
    display_grid = np.zeros((size * n_col, images_pre_row * size))
    # 遍历将每个特张图的数据写入到显示图像的矩阵中
    for col in range(n_col):
        for row in range(images_pre_row):
            # 保存该张特征图的矩阵(size,size,1)
            channel_image = layer_activation[0, :, :, col * images_pre_row + row]
            # 为使图像显示更鲜明，作一些特征处理
            channel_image -= channel_image.mean()
            channel_image /= channel_image.std()
            channel_image *= 64
            channel_image += 128
            # 把该特征图矩阵中不在0-255的元素值修改至0-255
            channel_image = np.clip(channel_image, 0, 255).astype("uint8")
            # 该特征图矩阵填充至显示图像的矩阵中
            display_grid[col * size:(col + 1) * size, row * size:(row + 1) * size] = channel_image

    scale = 1. / size
    # 设置该层显示图像的宽高
    plt.figure(figsize=(scale * display_grid.shape[1], scale * display_grid.shape[0]))
    plt.title(layer_name)
    plt.grid(False)
    # 显示图像
    plt.imshow(display_grid, aspect="auto", cmap="viridis")
plt.show()

# h2模型训练
print('training-----')
h2 = model.fit(x_train1,validation_data=x_val1,epochs=80)

print('\validating-----')
loss,accuracy=model.evaluate(x_val1)

print('\nvalidation loss: ',loss)
print('\nvalidation accuracy:　',accuracy)

# 保存模型
model.save('h22')

# 绘制h2模型损失变化曲线
loss = h2.history["loss"]
epochs = range(1, len(loss)+1)
val_loss = h2.history["val_loss"]
plt.plot(epochs, loss, "bo-", label="Training Loss")
plt.plot(epochs, val_loss, "ro--", label="Validation Loss")
plt.title("Training and Validation Loss")
plt.xlabel("Epochs")
plt.ylabel("Loss")
plt.legend()
plt.savefig('h2模型损失变化曲线图.jpg')
plt.show()

# 绘制h2模型准确率变化曲线
acc = h2.history["accuracy"]
epochs = range(1, len(acc)+1)
val_acc = h2.history["val_accuracy"]
plt.plot(epochs, acc, "bo-", label="Training Acc")
plt.plot(epochs, val_acc, "ro--", label="Validation Acc")
plt.title("Training and Validation Accuracy")
plt.xlabel("Epochs")
plt.ylabel("Accuracy")
plt.legend()
plt.savefig('h2模型准确率变化曲线图.jpg')
plt.show()

# 加载h1模型
h1 = load_model('h21')

# h1模型预测
# 3类型
l1 = ['BakedPotato','Burger','Fries',]
# 原图
i = 10
p1 = plt.figure(figsize=(12,7))
img1,label1 = x_test.next()
plt.rcParams["font.sans-serif"]=["FangSong"] # 解决中文显示异常
ax1 = p1.add_subplot(2, 1, 1)
plt.imshow((img1[i]*255).astype('uint8'))
plt.title('实际类型为：' + l1[np.argmax(label1[i])])

# 将图片转换成 4D 张量
x_test_img1 = img1[i].reshape(1, 112, 112, 3).astype("float32")

# 预测结果的概率柱状图
pr1 = h1.predict(x_test_img1)
ax2 = p1.add_subplot(2, 1, 2)
plt.title("预测结果的概率柱状图")
plt.bar(np.arange(3), pr1.reshape(3), align="center")
plt.xticks(np.arange(3),l1)
plt.show()

# 实际类型与h1模型预测结果对比
print('实际类型：' + l1[np.argmax(label1[i])])
print('h1模型预测的结果是：' + l1[np.argmax(pr1)])

# 加载h2模型
h2 = load_model('h22')

# h2模型预测
# 3类型
l2 = ['BakedPotato','Burger','Fries']
# 原图
i = 10
p2 = plt.figure(figsize=(12,7))
img2,label2 = x_test.next()
plt.rcParams["font.sans-serif"]=["FangSong"] # 解决中文显示异常
ax3 = p2.add_subplot(2, 1, 1)
plt.imshow((img2[i]*255).astype('uint8'))
plt.title('实际类型为：' + l2[np.argmax(label2[i])])

# 将图片转换成 4D 张量
x_test_img2 = img2[i].reshape(1, 112, 112, 3).astype("float32")

# 预测结果的概率柱状图
pr2 = h2.predict(x_test_img2)
ax4 = p2.add_subplot(2, 1, 2)
plt.title("预测结果的概率柱状图")
plt.bar(np.arange(3), pr2.reshape(3), align="center")
plt.xticks(np.arange(3),l2)
plt.show()

# 实际类型与h2模型预测结果对比
print('实际类型：' + l2[np.argmax(label2[i])])
print('h2模型预测的结果是：' + l2[np.argmax(pr2)])