[深度学习] centos7上搭建基于Anaconda3的caffe+pycaffe环境(python3.6)
本文记录从零开始在CentOS7.x系统上搭建Caffe深度学习平台,并配置pycaffe环境。(由于在虚拟机上搭建,所以为CPU_ONLY模式)
1.选择CentOS7 mini版镜像安装虚拟机
镜像版本:CentOS-7-x86_64-Minimal-1511.iso
安装好虚拟机后,配置网卡,本文采用桥接模式接入局域网,能够正常访问互联网。
2.更新yum源
安装wget、net-tools等常用工具:
yum install wget net-tools -y
更换国内yum源:
wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
yum clean all yum makecache yum update -y
安装epel-release源:
yum install epel-release -y
3.安装依赖
按照caffe官方文档,安装依赖包:
yum install protobuf-devel leveldb-devel snappy-devel opencv-devel boost-devel hdf5-devel -y yum install gflags-devel glog-devel lmdb-devel -y yum install openblas-devel atlas-devel -y
安装其他需要的工具包:
yum install git gcc gcc-c++ bzip2 unzip -y
4.安装Anaconda3
从Anaconda Install Archive中下载:Anaconda3-5.2.0-Linux-x86_64.sh(对应python3.6)
安装Anaconda,安装目录在/usr/local/anaconda3
bash Anaconda3-5.2.0-Linux-x86_64.sh
根据提示,依次Enter、yes、输入安装位置,等待安装完成。结束不选择不添加.bashrc,不安装VSCode。
使用/usr/local/anaconda3/bin/pip安装protobuf==3.6.0和protobuf-py3
cd /usr/local/anaconda3/bin ./pip install protobuf==3.6.0 ./pip install protobuf-py3
5.下载caffe
选择从github上下载caffe,本文下载的是caffe的zip格式包,名为master,解压:
unzip ~/master
进入caffe-master目录:
cd ~/caffe-master
将配置文件复制一份:
cp Makefile.config.example Makefile.config
修改配置文件Makefile.config:
CPU_ONLY := 1 BLAS := open
BLAS_INCLUDE := /usr/include/openblas ANACONDA_HOME := /usr/local/anaconda3 PYTHON_INCLUDE := $(ANACONDA_HOME)/include \ $(ANACONDA_HOME)/include/python3.6m \ $(ANACONDA_HOME)/lib/python3.6/site-packages/numpy/core/include PYTHON_LIBRARIES := boost_python36 python3.6m PYTHON_LIB := $(ANACONDA_HOME)/lib
6.添加环境变量
将各个动态库加入环境变量:
vi /etc/profile
在末尾添加:
LD_LIBRARY_PATH=/usr/lib:/usr/lib64:/lib:/lib64:/usr/local/lib
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/anaconda3/lib
生效:
source /etc/profile
7.编译caffe
进入~/caffe-master目录:
make all -j8 make test make runtest
8.编译libboost_python36.so(编译pycaffe时依赖)
下载 boost_1_70_0.tar.gz载/root下。
要编译libboost_python36版本,需要将/usr/bin下的python软连接指向Anaconda3中的python
cd /usr/bin mv python python.bak ln -s /usr/local/anaconda3/bin/python3.6 python
注意,此时尽量不要使用yum,因为yum默认使用python2.7,修改了python软连接会导致报错,解决方法如下:
vi /usr/bin/yum
将第一行的#!/usr/bin/python 修改为:
#!/usr/bin/python2.7
同样修改/usr/libexec/urlgrabber-ext-down文件。
解压boost压缩包:
tar zxvf ~/boost_1_70_0.tar.gz
进入解压后的boost_1_70_0目录:
./bootstrap.sh --with-python=/usr/local/anaconda3/bin/python3 --with-python-version=3.6 --with-python-root=/usr/local/anaconda3/lib/python3.6
./b2 cflags='-fPIC' cxxflags='-fPIC' --with-python include="/usr/local/anaconda3/include/python3.6m/"
./b2 install
9.编译pycaffe
进入~/caffe-master目录
make pycaffe
10.添加环境变量
vi /etc/profile
添加:
export PYTHONPATH=/root/caffe-master/python:$PYTHONPATH
export PYTHONPATH=/usr/local/anaconda3/lib/python3.6/site-packages:$PYTHONPATH
11.python导入caffe
运行python3.6.5
import caffe
此时会报错:
>>> import caffe Failed to include caffe_pb2, things might go wrong! Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/root/caffe-master/python/caffe/__init__.py", line 4, in <module> from .proto.caffe_pb2 import TRAIN, TEST File "/root/caffe-master/python/caffe/proto/caffe_pb2.py", line 4, in <module> from google.protobuf.internal import enum_type_wrapper ModuleNotFoundError: No module named 'google'
这是需要更新protobuf的版本,从2.5.0版本更新为2.6.1。
12.升级protobuf
下载protobuf-2.6.1.tar.gz在/root下
卸载protobuf-2.5.0版本:
yum remove protobuf -y
这里的yum可能会报错:
[root@leocaffe lib]# yum remove protobuf File "/usr/bin/yum", line 30 except KeyboardInterrupt, e: ^ SyntaxError: invalid syntax
这是因为我们将python软连接指向了python3.6,我们可以使用前面所讲的解决错误的方法,修改/usr/bin/yum和/usr/libexec/urlgrabber-ext-down文件(参照8.编译libboost_python36.so)
解压:
cd ~
tar zxvf ~/protobuf-2.6.1.tar.gz
执行:
cd /usr/local mkdir protobuf cd ~/protobuf-2.6.1 ./configure --prefix=/usr/local/protobuf
然后编译:
cd ~/protobuf-2.6.1
make
make install
13.添加protobuf环境变量
vi /etc/profile
添加以下内容:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/protobuf/lib/ export LIBRARY_PATH=$LIBRARY_PATH:/usr/local/protobuf/lib/ export PATH=$PATH:/usr/local/protobuf/bin/ export C_INCLUDE_PATH=$C_INCLUDE_PATH:/usr/local/protobuf/include/ export CPLUS_INCLUDE_PATH=$CPLUS_INCLUDE_PATH:/usr/local/protobuf/include/ export PKG_CONFIG_PATH=/usr/local/protobuf/lib/pkgconfig/
检查以下安装是否成功:
protoc --version
[root@leocaffe lib]# protoc --version libprotoc 2.6.1
打印版本号说明安装成功。
14.python再次导入caffe
为保险,可以重新对caffe进行编译:
cd ~/caffe-master make clean make all -j8 make pycaffe
然后导入caffe:
import caffe
成功:
[root@leocaffe bin]# ./python Python 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:14:56) [GCC 7.2.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import caffe >>>
总结:
从本文可以看出,部署caffe平台坑还是挺多,主要集中在所依赖的动态库,环境变量以及软件版本问题。特别注意boost的编译安装、protobuf的版本更新、以及profile环境变量的修改。