002-ImageNetClassificationDeep2017

ImageNet classification with deep convolutional neural networks #paper

1. paper-info

1.1 Metadata

  • Author:: * Authors: [[Alex Krizhevsky]], [[Ilya Sutskever]], [[Geoffrey E. Hinton]]
  • 作者机构::
  • Keywords::
  • Journal:: [[Communications of the ACM]]
  • Date:: [[2017-05-24]]
  • 状态:: #待读

1.2 Abstract

We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0% which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation. To reduce overfitting in the fully-connected layers we employed a recently-developed regularization method called “dropout” that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry.

2. The Architecture


图 2.1 原始结构表示
源paper网络结构如图

图 2.2 AlexNet 结构图
5个卷积层加上3个全连接层，激活函数采用RuLU，输入为227x227x3的图片，输出为1000（1000类的图片标签），在第一个全连接层和第二个全连接层之间使用了dropout。

3. Analysis of some innovative points

  • [[ReLU]]
    防止梯度衰减过快。

  • [[dropout]]

  • [[Data expension]]

    • Translational transformation (crop): 移动裁剪，由（256x256）-> （224x224）4个角和中间各一张，共5张。
    • Reflective transformation (flip):
    • Illumination and color shift:

  • [[Overlap pooling]]
    汇聚层窗口大小s, 步幅z， 如果s=z就是传统的汇聚层，如果s>z，就得到了重叠汇聚层。在该算法中，使用overlap pooling可以减少过拟合。

  • LRN local response normalization

5. 参考文献

1. 网络结构
2. https://www.intefrankly.com/articles/Deep-Learning-Paper-Notes-I--The-AlexNet-Model-Explained/eda8465893b7
3. AlexNet 中的 LRN（Local Response Normalization） 是什么
4. 深度学习中Dropout原理解析

posted @ 2022-09-09 09:56  GuiXu40  阅读(18)  评论(0)  编辑  收藏  举报