docker[caffe&&pycaffe]
0 引言
今天花了一天，完成了整个caffe的dockerfile编写,其支持python3.6.6，这里主要的注意点是protobuf的版本（在3.6.0之后，只支持c11），还有在制作镜像的时候注意，尽可能少的创建镜像层，并且及时在当前层删除不要的数据，以减少镜像本身大小。
FROM nvidia/cuda:9.0-cudnn7-devel-centos7
COPY ./caffe /caffe # 将官网github项目下下来，并只修改里面的Makefile.config（下面列出）
COPY ./Centos-7.repo /etc/yum.repos.d # 下载163或者阿里云的yum源，以方便加速
ENV LANG=en_US.UTF-8
ARG PYINSTALL=/usr/local/python3
ENV PATH=$PYINSTALL/bin:$PATH
ARG http_proxy=http://xxxxxxxx:xxxx # 记得修改成你的代理，我们机器需要代理才能上网
ARG https_proxy=https://xxxxxxx:xxxx
RUN rm -f /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-CR.repo /etc/yum.repos.d/CentOS-Debuginfo.repo /etc/yum.repos.d/CentOS-Media.repo /etc/yum.repos.d/CentOS-Sources.repo /etc/yum.repos.d/CentOS-Vault.repo /etc/yum.repos.d/CentOS-fasttrack.repo && \
yum clean all && yum makecache && \
yum -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm && \
# 安装python numpy
yum -y install make zlib-devel openssl-devel bzip2-devel expat-devel gdbm-devel readline-devel sqlite-devel && \
yum -y install libSM libXrender libXext wget && \
wget https://www.python.org/ftp/python/3.6.6/Python-3.6.6.tgz -O /home/Python-3.6.6.tgz && \
tar -xvf /home/Python-3.6.6.tgz -C /home && \
cd /home/Python-3.6.6 && \
./configure --prefix=$PYINSTALL && \
make -j32 && make install && \
ln -s $PYINSTALL/bin/python3 $PYINSTALL/bin/python && \
/usr/local/python3/bin/pip3 install -i http://pypi.douban.com/simple/ --trusted-host=pypi.douban.com numpy && \
/usr/local/python3/bin/pip3 install -i http://pypi.douban.com/simple/ --trusted-host=pypi.douban.com scikit-image && \
# 准备caffe依赖
yum -y install leveldb-devel snappy-devel opencv-devel boost-devel hdf5-devel gflags-devel glog-devel lmdb-devel && \
yum -y install gflags-devel glog-devel lmdb-devel && \
yum -y install openblas-devel python36-devel && \
yum -y groupinstall "Development Tools" "Development Libraries" && \
# 编译boost 修复libboost_python3.so找不到的问题
wget https://dl.bintray.com/boostorg/release/1.67.0/source/boost_1_67_0.tar.gz -O /home/boost_1_67_0.tar.gz && \
tar -xvf /home/boost_1_67_0.tar.gz -C /home && \
cd /home/boost_1_67_0 && ./bootstrap.sh --with-libraries=python --with-toolset=gcc && \
./b2 cflags='-fPIC' cxxflags='-fPIC' --with-python include=/usr/include/python3.6m && \
./b2 install && \
ln -s /usr/local/lib/libboost_python36.so /usr/lib64/libboost_python3.so && \
echo /usr/local/lib >> /etc/ld.so.conf.d/caffe.conf && ldconfig && \
# 安装protobuf
wget https://github.com/protocolbuffers/protobuf/releases/download/v3.5.1/protobuf-cpp-3.5.1.zip -O /home/protobuf-cpp-3.5.1.zip && \
cd /home && unzip protobuf-cpp-3.5.1.zip && \
cd /home/protobuf-3.5.1 && \
./configure && make -j32 && make install && ldconfig && \
wget https://github.com/protocolbuffers/protobuf/releases/download/v3.5.1/protobuf-python-3.5.1.zip -O /home/protobuf-python-3.5.1.zip && \
cd /home && rm -rf protobuf-3.5.1 && \
unzip protobuf-python-3.5.1.zip && \
cd /home/protobuf-3.5.1 && \
cd python && /usr/local/python3/bin/python3 setup.py build && /usr/local/python3/bin/python3 setup.py install && \
# 安装caffe
cd /caffe && \
make -j32 && make pycaffe -j32 && \
# 清空缓存和无用数据
yum clean all &&＼
rm -rf /home/*
# docker run --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --runtime=nvidia -tid --name zzc_caffe_demo ImageName /bin/bash
其中caffe中的Makefile.config为：
## Refer to http://caffe.berkeleyvision.org/installation.html
# Contributions simplifying and improving our build system are welcome!
# cuDNN acceleration switch (uncomment to build with cuDNN).
USE_CUDNN := 1 # 支持cudnn
# CPU-only switch (uncomment to build without GPU support).
# CPU_ONLY := 1
# uncomment to disable IO dependencies and corresponding data layers
USE_OPENCV := 1 # 使用opencv
# USE_LEVELDB := 0
# USE_LMDB := 0
# This code is taken from https://github.com/sh1r0/caffe-android-lib
# USE_HDF5 := 0
# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)
# You should not set this flag if you will be reading LMDBs with any
# possibility of simultaneous read and write
# ALLOW_LMDB_NOLOCK := 1
# Uncomment if you're using OpenCV 3
# OPENCV_VERSION := 3
# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++
# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr
# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the *_50 through *_61 lines for compatibility.
# For CUDA < 8.0, comment the *_60 and *_61 lines for compatibility.
# For CUDA >= 9.0, comment the *_20 and *_21 lines for compatibility.
# 我基于CUDA 9.0，所以删除前面2个，让其从30开始
CUDA_ARCH := -gencode arch=compute_30,code=sm_30 \
-gencode arch=compute_35,code=sm_35 \
-gencode arch=compute_50,code=sm_50 \
-gencode arch=compute_52,code=sm_52 \
-gencode arch=compute_60,code=sm_60 \
-gencode arch=compute_61,code=sm_61 \
-gencode arch=compute_61,code=compute_61
# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
BLAS := open
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
# Leave commented to accept the defaults for your choice of BLAS
# (which should work)!
BLAS_INCLUDE := /usr/include/openblas
BLAS_LIB := /usr/lib64
# Homebrew puts openblas in a directory that is not on the standard search path
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
# BLAS_LIB := $(shell brew --prefix openblas)/lib
# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
# MATLAB_DIR := /usr/local
# MATLAB_DIR := /Applications/MATLAB_R2012b.app
# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
# PYTHON_INCLUDE := /usr/include/python2.7 \
# /usr/lib/python2.7/dist-packages/numpy/core/include
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
# ANACONDA_HOME := $(HOME)/anaconda
# PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
# $(ANACONDA_HOME)/include/python2.7 \
# $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include
# Uncomment to use Python 3 (default is Python 2)
# 下面要注意你安装python的路径
PYTHON_LIBRARIES := boost_python3 python3.6m
PYTHON_INCLUDE := /usr/include/python3.6m \
/usr/local/python3/lib/python3.6/site-packages/numpy/core/include
# We need to be able to find libpythonX.X.so or .dylib.
PYTHON_LIB := /usr/lib64
# PYTHON_LIB := $(ANACONDA_HOME)/lib
# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib
# Uncomment to support layers written in Python (will link against Python libs)
WITH_PYTHON_LAYER := 1
# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib
# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS += $(shell brew --prefix)/include
# LIBRARY_DIRS += $(shell brew --prefix)/lib
# NCCL acceleration switch (uncomment to build with NCCL)
# https://github.com/NVIDIA/nccl (last tested version: v1.2.3-1+cuda8.0)
# USE_NCCL := 1
# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
# USE_PKG_CONFIG := 1
# N.B. both build and distribute dirs are cleared on `make clean`
BUILD_DIR := build
DISTRIBUTE_DIR := distribute
# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1
# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0
# enable pretty build (comment to see full commands)
Q ?= @
renference:
