【706】Keras官网语义分割例子解读

参考：Image segmentation with a U-Net-like architecture

目录：

1. 准备输入数据和目标分割掩膜的路径
2. 通过 Sequence class 来加载和向量化数据
3. Keras构建模型
4. 设置验证集 
5. 模型训练
6. 预测结果可视化

---

 

1. 准备输入数据和目标分割掩膜的路径

• 设置参数值：输入数据尺寸、分类数、batch size
• 输入数据路径list和目标图像路径list做到一一匹配

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27

import os   input_dir = "images/" target_dir = "annotations/trimaps/" img_size = (160, 160) num_classes = 3 batch_size = 32   input_img_paths = sorted(     [         os.path.join(input_dir, fname)         for fname in os.listdir(input_dir)         if fname.endswith(".jpg")     ] ) target_img_paths = sorted(     [         os.path.join(target_dir, fname)         for fname in os.listdir(target_dir)         if fname.endswith(".png") and not fname.startswith(".")     ] )   print("Number of samples:", len(input_img_paths))   for input_path, target_path in zip(input_img_paths[:10], target_img_paths[:10]):     print(input_path, "|", target_path)

2. 通过 Sequence class 来加载和向量化数据

• OxfordPets 类继承于 Sequence
• __init__：相关输入信息
• __len__：数据长度
• __getitem__：按照索引获取数据，每个 batch size
  • 获取每个 batch size 对应的数据路径 list
  • 构建 x 对应的 numpy.array
  • 构建 y 对应的 numpy.array
  • 返回 x, y 一一对应的数据

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33

from tensorflow import keras import numpy as np from tensorflow.keras.preprocessing.image import load_img     class OxfordPets(keras.utils.Sequence):     """Helper to iterate over the data (as Numpy arrays)."""       def __init__(self, batch_size, img_size, input_img_paths, target_img_paths):         self.batch_size = batch_size         self.img_size = img_size         self.input_img_paths = input_img_paths         self.target_img_paths = target_img_paths       def __len__(self):         return len(self.target_img_paths) // self.batch_size       def __getitem__(self, idx):         """Returns tuple (input, target) correspond to batch #idx."""         i = idx * self.batch_size         batch_input_img_paths = self.input_img_paths[i : i + self.batch_size]         batch_target_img_paths = self.target_img_paths[i : i + self.batch_size]         x = np.zeros((self.batch_size,) + self.img_size + (3,), dtype="float32")         for j, path in enumerate(batch_input_img_paths):             img = load_img(path, target_size=self.img_size)             x[j] = img         y = np.zeros((self.batch_size,) + self.img_size + (1,), dtype="uint8")         for j, path in enumerate(batch_target_img_paths):             img = load_img(path, target_size=self.img_size, color_mode="grayscale")             y[j] = np.expand_dims(img, 2)             # Ground truth labels are 1, 2, 3. Subtract one to make them 0, 1, 2:             y[j] -= 1         return x, y

3. Keras构建模型

• encoder和decoder部分
• inputs和outputs

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67

from tensorflow.keras import layers     def get_model(img_size, num_classes):     inputs = keras.Input(shape=img_size + (3,))       ### [First half of the network: downsampling inputs] ###       # Entry block     x = layers.Conv2D(32, 3, strides=2, padding="same")(inputs)     x = layers.BatchNormalization()(x)     x = layers.Activation("relu")(x)       previous_block_activation = x  # Set aside residual       # Blocks 1, 2, 3 are identical apart from the feature depth.     for filters in [64, 128, 256]:         x = layers.Activation("relu")(x)         x = layers.SeparableConv2D(filters, 3, padding="same")(x)         x = layers.BatchNormalization()(x)           x = layers.Activation("relu")(x)         x = layers.SeparableConv2D(filters, 3, padding="same")(x)         x = layers.BatchNormalization()(x)           x = layers.MaxPooling2D(3, strides=2, padding="same")(x)           # Project residual         residual = layers.Conv2D(filters, 1, strides=2, padding="same")(             previous_block_activation         )         x = layers.add([x, residual])  # Add back residual         previous_block_activation = x  # Set aside next residual       ### [Second half of the network: upsampling inputs] ###       for filters in [256, 128, 64, 32]:         x = layers.Activation("relu")(x)         x = layers.Conv2DTranspose(filters, 3, padding="same")(x)         x = layers.BatchNormalization()(x)           x = layers.Activation("relu")(x)         x = layers.Conv2DTranspose(filters, 3, padding="same")(x)         x = layers.BatchNormalization()(x)           x = layers.UpSampling2D(2)(x)           # Project residual         residual = layers.UpSampling2D(2)(previous_block_activation)         residual = layers.Conv2D(filters, 1, padding="same")(residual)         x = layers.add([x, residual])  # Add back residual         previous_block_activation = x  # Set aside next residual       # Add a per-pixel classification layer     outputs = layers.Conv2D(num_classes, 3, activation="softmax", padding="same")(x)       # Define the model     model = keras.Model(inputs, outputs)     return model     # Free up RAM in case the model definition cells were run multiple times keras.backend.clear_session()   # Build model model = get_model(img_size, num_classes) model.summary()

4. 设置验证集 

• 将数据分成训练集和验证集
• 并分别根据 OxfordPets 类来生成数据集 

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

import random   # Split our img paths into a training and a validation set val_samples = 1000 random.Random(1337).shuffle(input_img_paths) random.Random(1337).shuffle(target_img_paths) train_input_img_paths = input_img_paths[:-val_samples] train_target_img_paths = target_img_paths[:-val_samples] val_input_img_paths = input_img_paths[-val_samples:] val_target_img_paths = target_img_paths[-val_samples:]   # Instantiate data Sequences for each split train_gen = OxfordPets(     batch_size, img_size, train_input_img_paths, train_target_img_paths ) val_gen = OxfordPets(batch_size, img_size, val_input_img_paths, val_target_img_paths)

5. 模型训练

• 设置对应的训练参数
• 存储需要的结果

?

1 2 3 4 5 6 7 8 9 10 11 12

# Configure the model for training. # We use the "sparse" version of categorical_crossentropy # because our target data is integers. model.compile(optimizer="rmsprop", loss="sparse_categorical_crossentropy")   callbacks = [     keras.callbacks.ModelCheckpoint("oxford_segmentation.h5", save_best_only=True) ]   # Train the model, doing validation at the end of each epoch. epochs = 15 model.fit(train_gen, epochs=epochs, validation_data=val_gen, callbacks=callbacks)

6. 预测结果可视化

• 直接通过模型预测
• 通过PIL显示对应的结果

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

# Generate predictions for all images in the validation set   val_gen = OxfordPets(batch_size, img_size, val_input_img_paths, val_target_img_paths) val_preds = model.predict(val_gen)     def display_mask(i):     """Quick utility to display a model's prediction."""     mask = np.argmax(val_preds[i], axis=-1)     mask = np.expand_dims(mask, axis=-1)     img = PIL.ImageOps.autocontrast(keras.preprocessing.image.array_to_img(mask))     display(img)     # Display results for validation image #10 i = 10   # Display input image display(Image(filename=val_input_img_paths[i]))   # Display ground-truth target mask img = PIL.ImageOps.autocontrast(load_img(val_target_img_paths[i])) display(img)   # Display mask predicted by our model display_mask(i)  # Note that the model only sees inputs at 150x150.