[深度学习] centos7上搭建基于Anaconda3的caffe+pycaffe环境(python3.6)

本文记录从零开始在CentOS7.x系统上搭建Caffe深度学习平台,并配置pycaffe环境。(由于在虚拟机上搭建,所以为CPU_ONLY模式)

 

1.选择CentOS7 mini版镜像安装虚拟机

镜像版本:CentOS-7-x86_64-Minimal-1511.iso

安装好虚拟机后,配置网卡,本文采用桥接模式接入局域网,能够正常访问互联网。

2.更新yum源

安装wget、net-tools等常用工具:

yum install wget net-tools -y

更换国内yum源:

wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
yum clean all
yum makecache
yum update -y

安装epel-release源:

yum install epel-release -y

3.安装依赖

按照caffe官方文档,安装依赖包:

yum install protobuf-devel leveldb-devel snappy-devel opencv-devel boost-devel hdf5-devel -y
yum install gflags-devel glog-devel lmdb-devel -y
yum install openblas-devel atlas-devel -y

安装其他需要的工具包:

yum install git gcc gcc-c++ bzip2 unzip -y

4.安装Anaconda3

从Anaconda Install Archive中下载:Anaconda3-5.2.0-Linux-x86_64.sh(对应python3.6)

安装Anaconda,安装目录在/usr/local/anaconda3

bash Anaconda3-5.2.0-Linux-x86_64.sh

根据提示,依次Enter、yes、输入安装位置,等待安装完成。结束不选择不添加.bashrc,不安装VSCode。

使用/usr/local/anaconda3/bin/pip安装protobuf==3.6.0和protobuf-py3

cd /usr/local/anaconda3/bin
./pip install protobuf==3.6.0
./pip install protobuf-py3

 

5.下载caffe

选择从github上下载caffe,本文下载的是caffe的zip格式包,名为master,解压:

unzip ~/master

进入caffe-master目录:

cd ~/caffe-master

将配置文件复制一份:

cp Makefile.config.example Makefile.config

修改配置文件Makefile.config:

CPU_ONLY := 1
BLAS := open
BLAS_INCLUDE := /usr/include/openblas ANACONDA_HOME := /usr/local/anaconda3 PYTHON_INCLUDE := $(ANACONDA_HOME)/include \ $(ANACONDA_HOME)/include/python3.6m \ $(ANACONDA_HOME)/lib/python3.6/site-packages/numpy/core/include PYTHON_LIBRARIES := boost_python36 python3.6m PYTHON_LIB := $(ANACONDA_HOME)/lib

6.添加环境变量

将各个动态库加入环境变量:

vi /etc/profile

在末尾添加:

LD_LIBRARY_PATH=/usr/lib:/usr/lib64:/lib:/lib64:/usr/local/lib
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/anaconda3/lib

生效:

source /etc/profile

7.编译caffe

进入~/caffe-master目录:

make all -j8
make test
make runtest

8.编译libboost_python36.so(编译pycaffe时依赖)

下载 boost_1_70_0.tar.gz载/root下。

要编译libboost_python36版本,需要将/usr/bin下的python软连接指向Anaconda3中的python

cd /usr/bin
mv python python.bak
ln -s /usr/local/anaconda3/bin/python3.6 python

注意,此时尽量不要使用yum,因为yum默认使用python2.7,修改了python软连接会导致报错,解决方法如下:

vi /usr/bin/yum

将第一行的#!/usr/bin/python 修改为:

#!/usr/bin/python2.7

同样修改/usr/libexec/urlgrabber-ext-down文件。

 

解压boost压缩包:

tar zxvf ~/boost_1_70_0.tar.gz

进入解压后的boost_1_70_0目录:

./bootstrap.sh --with-python=/usr/local/anaconda3/bin/python3 --with-python-version=3.6 --with-python-root=/usr/local/anaconda3/lib/python3.6
./b2 cflags='-fPIC' cxxflags='-fPIC' --with-python include="/usr/local/anaconda3/include/python3.6m/"
./b2 install

9.编译pycaffe

进入~/caffe-master目录

make pycaffe

10.添加环境变量

vi /etc/profile

添加:

export PYTHONPATH=/root/caffe-master/python:$PYTHONPATH
export PYTHONPATH=/usr/local/anaconda3/lib/python3.6/site-packages:$PYTHONPATH

11.python导入caffe

运行python3.6.5

import caffe

此时会报错:

>>> import caffe
Failed to include caffe_pb2, things might go wrong!
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/caffe-master/python/caffe/__init__.py", line 4, in <module>
    from .proto.caffe_pb2 import TRAIN, TEST
  File "/root/caffe-master/python/caffe/proto/caffe_pb2.py", line 4, in <module>
    from google.protobuf.internal import enum_type_wrapper
ModuleNotFoundError: No module named 'google'

这是需要更新protobuf的版本,从2.5.0版本更新为2.6.1。

12.升级protobuf

下载protobuf-2.6.1.tar.gz在/root下

卸载protobuf-2.5.0版本:

yum remove protobuf -y

这里的yum可能会报错:

[root@leocaffe lib]# yum remove protobuf
  File "/usr/bin/yum", line 30
    except KeyboardInterrupt, e:
                            ^
SyntaxError: invalid syntax

这是因为我们将python软连接指向了python3.6,我们可以使用前面所讲的解决错误的方法,修改/usr/bin/yum和/usr/libexec/urlgrabber-ext-down文件(参照8.编译libboost_python36.so

解压:

cd ~
tar zxvf ~/protobuf-2.6.1.tar.gz

执行:

cd /usr/local
mkdir protobuf
cd ~/protobuf-2.6.1
./configure --prefix=/usr/local/protobuf

然后编译:

cd ~/protobuf-2.6.1
make
make install

13.添加protobuf环境变量

vi /etc/profile

添加以下内容:

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/protobuf/lib/
export LIBRARY_PATH=$LIBRARY_PATH:/usr/local/protobuf/lib/
export PATH=$PATH:/usr/local/protobuf/bin/
export C_INCLUDE_PATH=$C_INCLUDE_PATH:/usr/local/protobuf/include/
export CPLUS_INCLUDE_PATH=$CPLUS_INCLUDE_PATH:/usr/local/protobuf/include/
export PKG_CONFIG_PATH=/usr/local/protobuf/lib/pkgconfig/

检查以下安装是否成功:

protoc --version
[root@leocaffe lib]# protoc --version
libprotoc 2.6.1

打印版本号说明安装成功。

14.python再次导入caffe

 为保险,可以重新对caffe进行编译:

cd ~/caffe-master
make clean
make all -j8
make pycaffe

然后导入caffe:

import caffe

成功:

[root@leocaffe bin]# ./python
Python 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:14:56) 
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import caffe
>>> 

 

 

总结:

从本文可以看出,部署caffe平台坑还是挺多,主要集中在所依赖的动态库,环境变量以及软件版本问题。特别注意boost的编译安装、protobuf的版本更新、以及profile环境变量的修改。

posted @ 2019-10-09 19:22  风间悠香  阅读(1271)  评论(0编辑  收藏  举报