caffe2+cuda+Ubuntu16.04（u盘安装）

安装caffe2

预先准备、安装gflags及autoconf及GLOG

https://github.com/caffe2/caffe2/issues/1810

一、下载源代码通过网盘

https://blog.csdn.net/Gpwner/article/details/80068251?tdsourcetag=s_pctim_aiomsg

二、解压文件，二次解压。

$xz -d pytorch.tar.xz

$tar -xvf pytorch.tar

或者通过以下命令解压：

tar xvJf pytorch.tar.xz

三、安装依赖

sudo apt-get update
sudo apt-get install -y --no-install-recommends \
      build-essential \
      cmake \
      git \
      libgoogle-glog-dev \
      libgtest-dev \
      libiomp-dev \
      libleveldb-dev \
      liblmdb-dev \
      libopencv-dev \
      libopenmpi-dev \
      libsnappy-dev \
      libprotobuf-dev \
      openmpi-bin \
      openmpi-doc \
      protobuf-compiler \
      python-dev \
      python-pip  

# for Ubuntu 14.04
sudo apt-get install -y --no-install-recommends libgflags2
# for Ubuntu 16.04
sudo apt-get install -y --no-install-recommends libgflags-dev 

sudo pip install \
      future \
      numpy \
      protobuf

四、

构建与安装

获取源码

从这里（或终端输入命令 git clone --recursive https://github.com/caffe2/caffe2.git && cd caffe2 ）下载 caffe2 源码并解压进入 caffe2 文件夹。

修改配置文件

caffe需要自己配置，对于caffe2，使用 cmake 配置，可参考下面的构建安装部分。

添加环境变量

终端输入 sudo gedit ~/.bashrc 打开 “.bashrc” 文件，在文件末尾加入如下代码并保存：

#caffe2
#echo $PYTHONPATH
export PYTHONPATH=/usr/local:$PYTHONPATH
export PYTHONPATH=$PYTHONPATH:/home/cow/pytorch/build
#echo $LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH

输入 source ~/.bashrc 加载新的环境变量.

五、

构建安装

进入caffe2的根目录，终端执行如下命令，不报错误，代表安装成功！( -j$(nproc) 代表使用最大的线程编译，当然也可以手动指定，如 make -j4 )

# This will build Caffe2 in an isolated directory so that Caffe2 source is
# unaffected
mkdir build && cd build

# This configures the build and finds which libraries it will include in the
# Caffe2 installation. The output of this command is very helpful in debugging
cmake .. -DBUILD_TEST

# This actually builds and installs Caffe2 from makefiles generated from the
# above configuration step
sudo make install -j$(nproc)

执行完 cmake .. 终端输出：

$ echo $PYTHONPATH
/usr/local:/home/cow/pytorch/build
$ mkdir build && cd build
$ cmake ..
-- The CXX compiler identification is GNU 5.4.0
-- The C compiler identification is GNU 5.4.0
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Performing Test CAFFE2_LONG_IS_INT32_OR_64
-- Performing Test CAFFE2_LONG_IS_INT32_OR_64 - Success
-- Does not need to define long separately.
-- Performing Test CAFFE2_EXCEPTION_PTR_SUPPORTED
-- Performing Test CAFFE2_EXCEPTION_PTR_SUPPORTED - Success
-- std::exception_ptr is supported.
-- Performing Test CAFFE2_IS_NUMA_AVAILABLE
-- Performing Test CAFFE2_IS_NUMA_AVAILABLE - Success
-- NUMA is available
-- Performing Test CAFFE2_NEED_TO_TURN_OFF_DEPRECATION_WARNING
-- Performing Test CAFFE2_NEED_TO_TURN_OFF_DEPRECATION_WARNING - Success
-- Performing Test CAFFE2_COMPILER_SUPPORTS_AVX2_EXTENSIONS
-- Performing Test CAFFE2_COMPILER_SUPPORTS_AVX2_EXTENSIONS - Success
-- Current compiler supports avx2 extention. Will build perfkernels.
-- Build type not set - defaulting to Release
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Caffe2: Found protobuf with old-style protobuf targets.
-- Caffe2 protobuf include directory: 
-- Found Git: /usr/bin/git (found version "2.7.4") 
-- The BLAS backend of choice:Eigen
-- Could NOT find NNPACK (missing:  NNPACK_INCLUDE_DIR NNPACK_LIBRARY PTHREADPOOL_LIBRARY CPUINFO_LIBRARY) 
-- Brace yourself, we are building NNPACK
-- The ASM compiler identification is GNU
-- Found assembler: /usr/bin/cc
-- Found PythonInterp: /usr/bin/python (found version "2.7.12") 
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Caffe2: Cannot find gflags automatically. Using legacy find.
-- Found gflags: /usr/include  
-- Caffe2: Found gflags  (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libgflags.so)
-- Caffe2: Cannot find glog automatically. Using legacy find.
-- Found glog: /usr/include  
-- Caffe2: Found glog (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libglog.so)
-- git Version: v0.0.0
-- Version: 0.0.0
-- Performing Test HAVE_CXX_FLAG_STD_CXX11
-- Performing Test HAVE_CXX_FLAG_STD_CXX11 - Success
-- Performing Test HAVE_CXX_FLAG_WALL
-- Performing Test HAVE_CXX_FLAG_WALL - Success
-- Performing Test HAVE_CXX_FLAG_WEXTRA
-- Performing Test HAVE_CXX_FLAG_WEXTRA - Success
-- Performing Test HAVE_CXX_FLAG_WSHADOW
-- Performing Test HAVE_CXX_FLAG_WSHADOW - Success
-- Performing Test HAVE_CXX_FLAG_WERROR
-- Performing Test HAVE_CXX_FLAG_WERROR - Success
-- Performing Test HAVE_CXX_FLAG_PEDANTIC
-- Performing Test HAVE_CXX_FLAG_PEDANTIC - Success
-- Performing Test HAVE_CXX_FLAG_PEDANTIC_ERRORS
-- Performing Test HAVE_CXX_FLAG_PEDANTIC_ERRORS - Success
-- Performing Test HAVE_CXX_FLAG_WSHORTEN_64_TO_32
-- Performing Test HAVE_CXX_FLAG_WSHORTEN_64_TO_32 - Failed
-- Performing Test HAVE_CXX_FLAG_WFLOAT_EQUAL
-- Performing Test HAVE_CXX_FLAG_WFLOAT_EQUAL - Success
-- Performing Test HAVE_CXX_FLAG_FSTRICT_ALIASING
-- Performing Test HAVE_CXX_FLAG_FSTRICT_ALIASING - Success
-- Performing Test HAVE_CXX_FLAG_WZERO_AS_NULL_POINTER_CONSTANT
-- Performing Test HAVE_CXX_FLAG_WZERO_AS_NULL_POINTER_CONSTANT - Success
-- Performing Test HAVE_CXX_FLAG_WSTRICT_ALIASING
-- Performing Test HAVE_CXX_FLAG_WSTRICT_ALIASING - Success
-- Performing Test HAVE_CXX_FLAG_WD654
-- Performing Test HAVE_CXX_FLAG_WD654 - Failed
-- Performing Test HAVE_CXX_FLAG_WTHREAD_SAFETY
-- Performing Test HAVE_CXX_FLAG_WTHREAD_SAFETY - Failed
-- Performing Test HAVE_CXX_FLAG_COVERAGE
-- Performing Test HAVE_CXX_FLAG_COVERAGE - Success
-- Performing Test HAVE_STD_REGEX
-- Performing Test HAVE_STD_REGEX
-- Performing Test HAVE_STD_REGEX -- success
-- Performing Test HAVE_GNU_POSIX_REGEX
-- Performing Test HAVE_GNU_POSIX_REGEX
-- Performing Test HAVE_GNU_POSIX_REGEX -- failed to compile
-- Performing Test HAVE_POSIX_REGEX
-- Performing Test HAVE_POSIX_REGEX
-- Performing Test HAVE_POSIX_REGEX -- success
-- Performing Test HAVE_STEADY_CLOCK
-- Performing Test HAVE_STEADY_CLOCK
-- Performing Test HAVE_STEADY_CLOCK -- success
-- Found LMDB: /usr/include  
-- Found lmdb    (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/liblmdb.so)
-- Found LevelDB: /usr/include  
-- Found LevelDB (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libleveldb.so)
-- Found Snappy: /usr/include  
-- Found Snappy  (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libsnappy.so)
-- Found Numa: /usr/include  
-- Found Numa  (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libnuma.so)
-- OpenCV found (/usr/share/OpenCV)
CMake Warning at cmake/Dependencies.cmake:270 (find_package):
  By not providing "FindEigen3.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "Eigen3", but
  CMake did not find one.

  Could not find a package configuration file provided by "Eigen3" with any
  of the following names:

    Eigen3Config.cmake
    eigen3-config.cmake

  Add the installation prefix of "Eigen3" to CMAKE_PREFIX_PATH or set
  "Eigen3_DIR" to a directory containing one of the above files.  If "Eigen3"
  provides a separate development package or SDK, be sure it has been
  installed.
Call Stack (most recent call first):
  CMakeLists.txt:101 (include)


-- Did not find system Eigen. Using third party subdirectory.
-- Found PythonInterp: /usr/bin/python (found suitable version "2.7.12", minimum required is "2.7") 
-- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython2.7.so (found suitable version "2.7.12", minimum required is "2.7") 
-- Found NumPy: /usr/local/lib/python2.7/dist-packages/numpy/core/include (found version "1.13.3") 
-- NumPy ver. 1.13.3 found (include: /usr/local/lib/python2.7/dist-packages/numpy/core/include)
-- Could NOT find pybind11 (missing:  pybind11_INCLUDE_DIR) 
-- Found MPI_C: /usr/lib/openmpi/lib/libmpi.so  
-- Found MPI_CXX: /usr/lib/openmpi/lib/libmpi_cxx.so;/usr/lib/openmpi/lib/libmpi.so  
-- MPI support found
-- MPI compile flags: 
-- MPI include path: /usr/lib/openmpi/include/openmpi/opal/mca/event/libevent2021/libevent/usr/lib/openmpi/include/openmpi/opal/mca/event/libevent2021/libevent/include/usr/lib/openmpi/include/usr/lib/openmpi/include/openmpi
-- MPI LINK flags path:  -Wl,-rpath  -Wl,/usr/lib/openmpi/lib  -Wl,--enable-new-dtags
-- MPI libraries: /usr/lib/openmpi/lib/libmpi_cxx.so/usr/lib/openmpi/lib/libmpi.so
CMake Warning at cmake/Dependencies.cmake:324 (message):
  OpenMPI found, but it is not built with CUDA support.
Call Stack (most recent call first):
  CMakeLists.txt:101 (include)


-- Found CUDA: /usr/local/cuda-8.0 (found suitable version "8.0", minimum required is "7.0") 
-- Found CUDNN: /usr/local/cuda-8.0/include  
-- Caffe2: CUDA detected: 8.0
-- Found cuDNN: v6.0.21  (include: /usr/local/cuda-8.0/include, library: /usr/local/cuda-8.0/lib64/libcudnn.so)
-- Automatic GPU detection failed. Building for all known architectures.
-- Added CUDA NVCC flags for: sm_20 sm_21 sm_30 sm_35 sm_50 sm_52 sm_60 sm_61
-- Could NOT find NCCL (missing:  NCCL_INCLUDE_DIRS NCCL_LIBRARIES) 
-- Could NOT find CUB (missing:  CUB_INCLUDE_DIR) 
-- Could NOT find Gloo (missing:  Gloo_INCLUDE_DIR Gloo_LIBRARY) 
-- MPI include path: /usr/lib/openmpi/include/openmpi/opal/mca/event/libevent2021/libevent/usr/lib/openmpi/include/openmpi/opal/mca/event/libevent2021/libevent/include/usr/lib/openmpi/include/usr/lib/openmpi/include/openmpi
-- MPI libraries: /usr/lib/openmpi/lib/libmpi_cxx.so/usr/lib/openmpi/lib/libmpi.so
-- CUDA detected: 8.0
-- Found libcuda: /usr/local/cuda-8.0/lib64/stubs/libcuda.so
-- Found libnvrtc: /usr/local/cuda-8.0/lib64/libnvrtc.so
-- Found nccl: /home/liu/sfw/dlapp/caffe2/third_party/nccl/build/include  
CMake Warning at cmake/Dependencies.cmake:457 (message):
  mobile opengl is only used in android or ios builds.
Call Stack (most recent call first):
  CMakeLists.txt:101 (include)


CMake Warning at cmake/Dependencies.cmake:533 (message):
  Metal is only used in ios builds.
Call Stack (most recent call first):
  CMakeLists.txt:101 (include)


-- GCC 5.4.0: Adding gcc and gcc_s libs to link line
-- Include NCCL operators
-- Including image processing operators
-- Excluding video processing operators due to no opencv
-- Excluding mkl operators as we are not using mkl
-- Include Observer library
-- Using lib/python2.7/dist-packages as python relative installation path
-- Automatically generating missing __init__.py files.
-- 
-- ******** Summary ********
-- General:
--   CMake version         : 3.5.1
--   CMake command         : /usr/bin/cmake
--   Git version           : v0.8.1-1314-gd8770f8
--   System                : Linux
--   C++ compiler          : /usr/bin/c++
--   C++ compiler version  : 5.4.0
--   Protobuf compiler     : /usr/bin/protoc
--   Protobuf include path : /usr/include
--   Protobuf libraries    : optimized;/usr/lib/x86_64-linux-gnu/libprotobuf.so;debug;/usr/lib/x86_64-linux-gnu/libprotobuf.so;-lpthread
--   BLAS                  : Eigen
--   CXX flags             :  -O2 -fPIC -Wno-narrowing -Wno-invalid-partial-specialization
--   Build type            : Release
--   Compile definitions   : 
-- 
--   BUILD_BINARY          : ON
--   BUILD_DOCS            : OFF
--   BUILD_PYTHON          : ON
--     Python version      : 2.7.12
--     Python includes     : /usr/include/python2.7
--   BUILD_SHARED_LIBS     : ON
--   BUILD_TEST            : ON
--   USE_ATEN              : OFF
--   USE_ASAN              : OFF
--   USE_CUDA              : ON
--     CUDA version        : 8.0
--     CuDNN version       : 6.0.21
--     CUDA root directory : /usr/local/cuda-8.0
--     CUDA library        : /usr/local/cuda-8.0/lib64/stubs/libcuda.so
--     CUDA NVRTC library  : /usr/local/cuda-8.0/lib64/libnvrtc.so
--     CUDA runtime library: /usr/local/cuda-8.0/lib64/libcudart.so
--     CUDA include path   : /usr/local/cuda-8.0/include
--     NVCC executable     : /usr/local/cuda-8.0/bin/nvcc
--     CUDA host compiler  : /usr/bin/cc
--   USE_EIGEN_FOR_BLAS    : 1
--   USE_FFMPEG            : OFF
--   USE_GFLAGS            : ON
--   USE_GLOG              : ON
--   USE_GLOO              : ON
--   USE_LEVELDB           : ON
--     LevelDB version     : 1.18
--     Snappy version      : 1.1.3
--   USE_LITE_PROTO        : OFF
--   USE_LMDB              : ON
--     LMDB version        : 0.9.17
--   USE_METAL             : OFF
--   USE_MKL               : 
--   USE_MOBILE_OPENGL     : OFF
--   USE_MPI               : ON
--   USE_NCCL              : ON
--   USE_NERVANA_GPU       : OFF
--   USE_NNPACK            : ON
--   USE_OBSERVERS         : ON
--   USE_OPENCV            : ON
--     OpenCV version      : 2.4.9.1
--   USE_OPENMP            : OFF
--   USE_PROF              : OFF
--   USE_REDIS             : OFF
--   USE_ROCKSDB           : OFF
--   USE_THREADS           : ON
--   USE_ZMQ               : OFF
-- Configuring done
-- Generating done
-- Build files have been written to: /home/cow/pytorch/build

caffe2默认被安装在根目录下/usr/local/，当然你也可以在配置时修改安装目录。

六、

6.1. Verify that the Caffe2 python module can be properly invoked
$ python -c 'from caffe2.python import core' 2>/dev/null && echo "Success" || echo "Failure"

6.2 Verify that Caffe2 can run with GPU support
$ python2 -c 'from caffe2.python import workspace; print(workspace.NumCudaDevices())'

The installation is now complete. We can now start using Caffe2 for deep learning modeling and implementation. If you have problem with your Caffe2 installation, simply write it in the comment section.

6.3测试caffe2是否安装成功

cd ~ && python -c 'from caffe2.python import core' 2>/dev/null && echo "Success" || echo "Failure"

如果是failure，试着cd到.../pytorch/build的文件夹里，然后执行

python -c 'from caffe2.python import core' 2>/dev/null

如果successful，说明是环境变量的设置问题，如果还是失败，则会有具体的提示。

配置环境变量，编辑~/.bashrc

sudo gedit ~/.bashrc

添加以下内容：

export PYTHONPATH=/usr/local:/usr/local/lib:$PYTHONPATH
export PYTHONPATH=$PYTHONPATH:/home/....../pytorch/build (后面路径为caffe2的编译路径，在pytorch/build中，命令行输入pwd可以得到这个路径)
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH

七、sudo make install 时报错:

problem1：

mpi_test.cc.o: undefined reference to symbol '_ZN3MPI8Datatype4FreeEv

Solved：

You should also change 'option(USE_MPI "Use MPI" ON)' to'option(USE_MPI "Use MPI" OFF)' in the file 'CMakeLists.txt'。And run the commend:

cmake .. -DBUILD_TEST=OFF

Reference：

安装caffe2的文件

https://blog.csdn.net/enjoyyl/article/details/79521685

官网：

https://caffe2.ai/docs/getting-started.html?platform=ubuntu&configuration=compile#install-with-gpu-support

https://caffe2.ai/docs/faq.html

https://tech.amikelive.com/node-706/comprehensive-guide-installing-caffe2-with-gpu-support-by-building-from-source-on-ubuntu-16-04/

首先安装依赖及GLOG

https://github.com/caffe2/caffe2/issues/1810

测试caffe2

https://blog.csdn.net/cym1990/article/details/79409476

错误解决

https://blog.csdn.net/xiangxianghehe/article/details/70171342

python2.7换python3.5

https://blog.csdn.net/u013842516/article/details/80604409

libcudnn问题

https://askubuntu.com/questions/1025928/why-do-i-get-sbin-ldconfig-real-usr-local-cuda-lib64-libcudnn-so-7-is-not-a

https://devtalk.nvidia.com/default/topic/1032114/tensorrt-4-installation-libcudnn-so-7-is-not-a-symbolic-link/

install cuda TK8 and tensorflow in clean Ubuntu16.04

http://queirozf.com/entries/installing-cuda-tk-and-tensorflow-on-a-clean-ubuntu-16-04-install#-sbin-ldconfig-real-usr-local-cuda-lib64-libcudnn-so-5-is-not-a-symbolic-link

u盘安装ubuntu16.04

https://blog.csdn.net/chy555chy/article/details/61191771

posted @ 2018-09-06 22:06 Sailon 阅读(1748) 评论(0) 编辑收藏举报

刷新页面返回顶部

Sailon