[caffe]深度学习之图像分类模型AlexNet解读
在imagenet上的图像分类challenge上Alex提出的alexnet网络结构模型赢得了2012届的冠军。要研究CNN类型DL网络模型在图像分类上的应用,就逃不开研究alexnet。这是CNN在图像分类上的经典模型(DL火起来之后)。
在DL开源实现caffe的model例子中。它也给出了alexnet的复现。详细网络配置文件例如以下https://github.com/BVLC/caffe/blob/master/models/bvlc_reference_caffenet/train_val.prototxt:
接下来本文将一步步对该网络配置结构中各个层进行具体的解读(训练阶段):1. conv1阶段DFD(data flow diagram):
2. conv2阶段DFD(data flow diagram):
3. conv3阶段DFD(data flow diagram):
4. conv4阶段DFD(data flow diagram):
5. conv5阶段DFD(data flow diagram):
6. fc6阶段DFD(data flow diagram):
7. fc7阶段DFD(data flow diagram):
8. fc8阶段DFD(data flow diagram):
各种layer的operation很多其它解释能够參考http://caffe.berkeleyvision.org/tutorial/layers.html
从计算该模型的数据流过程中。该模型參数大概5kw+。
caffe的输出中也有包括这块的内容日志,详情例如以下:
I0721 10:38:15.326920 4692 net.cpp:125] Top shape: 256 3 227 227 (39574272) I0721 10:38:15.326971 4692 net.cpp:125] Top shape: 256 1 1 1 (256) I0721 10:38:15.326982 4692 net.cpp:156] data does not need backward computation. I0721 10:38:15.327003 4692 net.cpp:74] Creating Layer conv1 I0721 10:38:15.327011 4692 net.cpp:84] conv1 <- data I0721 10:38:15.327033 4692 net.cpp:110] conv1 -> conv1 I0721 10:38:16.721956 4692 net.cpp:125] Top shape: 256 96 55 55 (74342400) I0721 10:38:16.722030 4692 net.cpp:151] conv1 needs backward computation. I0721 10:38:16.722059 4692 net.cpp:74] Creating Layer relu1 I0721 10:38:16.722070 4692 net.cpp:84] relu1 <- conv1 I0721 10:38:16.722082 4692 net.cpp:98] relu1 -> conv1 (in-place) I0721 10:38:16.722096 4692 net.cpp:125] Top shape: 256 96 55 55 (74342400) I0721 10:38:16.722105 4692 net.cpp:151] relu1 needs backward computation. I0721 10:38:16.722116 4692 net.cpp:74] Creating Layer pool1 I0721 10:38:16.722125 4692 net.cpp:84] pool1 <- conv1 I0721 10:38:16.722133 4692 net.cpp:110] pool1 -> pool1 I0721 10:38:16.722167 4692 net.cpp:125] Top shape: 256 96 27 27 (17915904) I0721 10:38:16.722187 4692 net.cpp:151] pool1 needs backward computation. I0721 10:38:16.722205 4692 net.cpp:74] Creating Layer norm1 I0721 10:38:16.722221 4692 net.cpp:84] norm1 <- pool1 I0721 10:38:16.722234 4692 net.cpp:110] norm1 -> norm1 I0721 10:38:16.722251 4692 net.cpp:125] Top shape: 256 96 27 27 (17915904) I0721 10:38:16.722260 4692 net.cpp:151] norm1 needs backward computation. I0721 10:38:16.722272 4692 net.cpp:74] Creating Layer conv2 I0721 10:38:16.722280 4692 net.cpp:84] conv2 <- norm1 I0721 10:38:16.722290 4692 net.cpp:110] conv2 -> conv2 I0721 10:38:16.725225 4692 net.cpp:125] Top shape: 256 256 27 27 (47775744) I0721 10:38:16.725242 4692 net.cpp:151] conv2 needs backward computation. I0721 10:38:16.725253 4692 net.cpp:74] Creating Layer relu2 I0721 10:38:16.725261 4692 net.cpp:84] relu2 <- conv2 I0721 10:38:16.725270 4692 net.cpp:98] relu2 -> conv2 (in-place) I0721 10:38:16.725280 4692 net.cpp:125] Top shape: 256 256 27 27 (47775744) I0721 10:38:16.725288 4692 net.cpp:151] relu2 needs backward computation. I0721 10:38:16.725298 4692 net.cpp:74] Creating Layer pool2 I0721 10:38:16.725307 4692 net.cpp:84] pool2 <- conv2 I0721 10:38:16.725317 4692 net.cpp:110] pool2 -> pool2 I0721 10:38:16.725329 4692 net.cpp:125] Top shape: 256 256 13 13 (11075584) I0721 10:38:16.725338 4692 net.cpp:151] pool2 needs backward computation. I0721 10:38:16.725358 4692 net.cpp:74] Creating Layer norm2 I0721 10:38:16.725368 4692 net.cpp:84] norm2 <- pool2 I0721 10:38:16.725378 4692 net.cpp:110] norm2 -> norm2 I0721 10:38:16.725389 4692 net.cpp:125] Top shape: 256 256 13 13 (11075584) I0721 10:38:16.725399 4692 net.cpp:151] norm2 needs backward computation. I0721 10:38:16.725409 4692 net.cpp:74] Creating Layer conv3 I0721 10:38:16.725419 4692 net.cpp:84] conv3 <- norm2 I0721 10:38:16.725427 4692 net.cpp:110] conv3 -> conv3 I0721 10:38:16.735193 4692 net.cpp:125] Top shape: 256 384 13 13 (16613376) I0721 10:38:16.735213 4692 net.cpp:151] conv3 needs backward computation. I0721 10:38:16.735224 4692 net.cpp:74] Creating Layer relu3 I0721 10:38:16.735234 4692 net.cpp:84] relu3 <- conv3 I0721 10:38:16.735242 4692 net.cpp:98] relu3 -> conv3 (in-place) I0721 10:38:16.735250 4692 net.cpp:125] Top shape: 256 384 13 13 (16613376) I0721 10:38:16.735258 4692 net.cpp:151] relu3 needs backward computation. I0721 10:38:16.735302 4692 net.cpp:74] Creating Layer conv4 I0721 10:38:16.735312 4692 net.cpp:84] conv4 <- conv3 I0721 10:38:16.735321 4692 net.cpp:110] conv4 -> conv4 I0721 10:38:16.743952 4692 net.cpp:125] Top shape: 256 384 13 13 (16613376) I0721 10:38:16.743988 4692 net.cpp:151] conv4 needs backward computation. I0721 10:38:16.744000 4692 net.cpp:74] Creating Layer relu4 I0721 10:38:16.744010 4692 net.cpp:84] relu4 <- conv4 I0721 10:38:16.744020 4692 net.cpp:98] relu4 -> conv4 (in-place) I0721 10:38:16.744030 4692 net.cpp:125] Top shape: 256 384 13 13 (16613376) I0721 10:38:16.744038 4692 net.cpp:151] relu4 needs backward computation. I0721 10:38:16.744050 4692 net.cpp:74] Creating Layer conv5 I0721 10:38:16.744057 4692 net.cpp:84] conv5 <- conv4 I0721 10:38:16.744067 4692 net.cpp:110] conv5 -> conv5 I0721 10:38:16.748935 4692 net.cpp:125] Top shape: 256 256 13 13 (11075584) I0721 10:38:16.748955 4692 net.cpp:151] conv5 needs backward computation. I0721 10:38:16.748965 4692 net.cpp:74] Creating Layer relu5 I0721 10:38:16.748975 4692 net.cpp:84] relu5 <- conv5 I0721 10:38:16.748983 4692 net.cpp:98] relu5 -> conv5 (in-place) I0721 10:38:16.748998 4692 net.cpp:125] Top shape: 256 256 13 13 (11075584) I0721 10:38:16.749011 4692 net.cpp:151] relu5 needs backward computation. I0721 10:38:16.749022 4692 net.cpp:74] Creating Layer pool5 I0721 10:38:16.749030 4692 net.cpp:84] pool5 <- conv5 I0721 10:38:16.749039 4692 net.cpp:110] pool5 -> pool5 I0721 10:38:16.749050 4692 net.cpp:125] Top shape: 256 256 6 6 (2359296) I0721 10:38:16.749058 4692 net.cpp:151] pool5 needs backward computation. I0721 10:38:16.749074 4692 net.cpp:74] Creating Layer fc6 I0721 10:38:16.749083 4692 net.cpp:84] fc6 <- pool5 I0721 10:38:16.749091 4692 net.cpp:110] fc6 -> fc6 I0721 10:38:17.160079 4692 net.cpp:125] Top shape: 256 4096 1 1 (1048576) I0721 10:38:17.160148 4692 net.cpp:151] fc6 needs backward computation. I0721 10:38:17.160166 4692 net.cpp:74] Creating Layer relu6 I0721 10:38:17.160177 4692 net.cpp:84] relu6 <- fc6 I0721 10:38:17.160190 4692 net.cpp:98] relu6 -> fc6 (in-place) I0721 10:38:17.160202 4692 net.cpp:125] Top shape: 256 4096 1 1 (1048576) I0721 10:38:17.160212 4692 net.cpp:151] relu6 needs backward computation. I0721 10:38:17.160222 4692 net.cpp:74] Creating Layer drop6 I0721 10:38:17.160230 4692 net.cpp:84] drop6 <- fc6 I0721 10:38:17.160238 4692 net.cpp:98] drop6 -> fc6 (in-place) I0721 10:38:17.160258 4692 net.cpp:125] Top shape: 256 4096 1 1 (1048576) I0721 10:38:17.160265 4692 net.cpp:151] drop6 needs backward computation. I0721 10:38:17.160277 4692 net.cpp:74] Creating Layer fc7 I0721 10:38:17.160286 4692 net.cpp:84] fc7 <- fc6 I0721 10:38:17.160295 4692 net.cpp:110] fc7 -> fc7 I0721 10:38:17.342094 4692 net.cpp:125] Top shape: 256 4096 1 1 (1048576) I0721 10:38:17.342157 4692 net.cpp:151] fc7 needs backward computation. I0721 10:38:17.342175 4692 net.cpp:74] Creating Layer relu7 I0721 10:38:17.342185 4692 net.cpp:84] relu7 <- fc7 I0721 10:38:17.342198 4692 net.cpp:98] relu7 -> fc7 (in-place) I0721 10:38:17.342208 4692 net.cpp:125] Top shape: 256 4096 1 1 (1048576) I0721 10:38:17.342217 4692 net.cpp:151] relu7 needs backward computation. I0721 10:38:17.342228 4692 net.cpp:74] Creating Layer drop7 I0721 10:38:17.342236 4692 net.cpp:84] drop7 <- fc7 I0721 10:38:17.342245 4692 net.cpp:98] drop7 -> fc7 (in-place) I0721 10:38:17.342254 4692 net.cpp:125] Top shape: 256 4096 1 1 (1048576) I0721 10:38:17.342262 4692 net.cpp:151] drop7 needs backward computation. I0721 10:38:17.342274 4692 net.cpp:74] Creating Layer fc8 I0721 10:38:17.342283 4692 net.cpp:84] fc8 <- fc7 I0721 10:38:17.342291 4692 net.cpp:110] fc8 -> fc8 I0721 10:38:17.343199 4692 net.cpp:125] Top shape: 256 22 1 1 (5632) I0721 10:38:17.343214 4692 net.cpp:151] fc8 needs backward computation. I0721 10:38:17.343231 4692 net.cpp:74] Creating Layer loss I0721 10:38:17.343240 4692 net.cpp:84] loss <- fc8 I0721 10:38:17.343250 4692 net.cpp:84] loss <- label I0721 10:38:17.343264 4692 net.cpp:151] loss needs backward computation. I0721 10:38:17.343305 4692 net.cpp:173] Collecting Learning Rate and Weight Decay. I0721 10:38:17.343327 4692 net.cpp:166] Network initialization done. I0721 10:38:17.343335 4692 net.cpp:167] Memory required for Data 1073760256