[ERROR] 安装完Detectron后出现 cython_nms.so: undefined symbol: PyFPE_jbuf

上个月Install Detectron后,

检查caffe2是否安装成功:

# To check if Caffe2 GPU build was successful
# This must print a number > 0 in order to use Detectron
python2 -c 'from caffe2.python import workspace; print(workspace.NumCudaDevices())'

输出1 ,没问题

检查Detectron是否编译成功:

python2 $DETECTRON/tests/test_spatial_narrow_as_op.py

依旧通过。执行nference tutorial时也一切正常。

但是今天在执行inference tutorial时,却报错了:

python2 tools/infer_simple.py \
    --cfg configs/12_2017_baselines/e2e_mask_rcnn_R-101-FPN_2x.yaml \
    --output-dir /tmp/detectron-visualizations \
    --image-ext jpg \
    --wts https://s3-us-west-2.amazonaws.com/detectron/35861858/12_2017_baselines/e2e_mask_rcnn_R-101-FPN_2x.yaml.02_32_51.SgT4y1cO/output/train/coco_2014_train:coco_2014_valminusminival/generalized_rcnn/model_final.pkl \
    demo

提示:

ImportError: /home/cc/Detectron-master/lib/utils/cython_nms.so: undefined symbol: PyFPE_jbuf

于是我google了一下,很多提到出现这个问题的原因是系统上存在多个python环境导致的。我想起来,一开始由于重装系统,所以在安装Detectron时,使用的是系统自带的python2.7;而后来我因为要用tensorflow,就又重新装了anaconda2,并默认使用anaconda中的python了,我想这可能就是导致我失败的最终原因,在网上找了几种解决方法,都是坑!

第一个坑:

>>> pip uninstall numpy

  把numpy给卸载了,然后就出现 “ no module named numpy” ,我想重新给它装回去,pip install numpy却提示我系统已经安装了numpy ,可是我却无法import !!??

  后来还是通过 pip installl --ignore-installed numpy 重新给装回去了。

第二个坑:

>>> conda remove numpy

  我也没注意,出提示时直接输入了yes ,结果...把装好的tensorflow和其他一些相关的包都给卸载了!!??? WTF

好吧,没了就没 了,一会给它重新装回去就是了。但是执行inference tutorial依旧报相同的错误!!于是就进入了下面的一个坑。

第三个坑:

  1.删除Detecron、caffe2文件夹

  2.

>>> conda install -c caffe2 caffe2-cuda9.0-cudnn7-gcc4.8

 这个更改完犊子,安装某个包时,出现一个和Permision Denied有关的Error,然后安装结束了。我进入python可以import caffe2 但是,执行inference tutorial时,出现其他的错误。

然后我就想删了,conda remove  caffe2 caffe2-cuda9.0-cudnn7-gcc4.8 居然删不了,每次运行完指令,还是可以import caffe2,而且错误依旧存在。

无奈,最后我使出了杀手锏:删除Anaconda2 一切重新来过。。。

后续。。



 

 

Step1:安装Anaconda2,我事先下载了安装包,所以直接安装就行

 

bash ./Anaconda2-4.4.0-Linux-x86_64.sh

这里没什么好说的,正常装完后检查 python -V 显示是Anaconda2的。

Step 2: 从源码编译caffe2

 有一点要注意的时,这次在编译caffe2的时候,提示cmake版本最低要求 3.2,而本机自带的是2.8,所以需要升级:

>>> cd ~

>>> wget https://cmake.org/files/v3.2/cmake-3.2.2.tar.gz

>>> tar xvf cmake-3.2.2.tar.gz

>>> cd cmake-3.2.2

>>> ./bootstrap --prefix=/usr         #这一步很关键,如果没有指定prefix,后面使用时会报错Could not find CMAKE_ROOT

>>> make

>>> sudo make install

#check:

>>> cmake --version
cmake version 3.2.2

然后执行 再下载和编译caffe2

 

 

 

 

 

 

 

Step2: 安装Caffe2,要小心了。

>>> conda install -c caffe2 caffe2-cuda9.0-cudnn7-gcc4.8

下面是Terminal里的输出:

Fetching package metadata ...........
Solving package specifications: .

Package plan for installation in environment /home/cc/anaconda2:

The following NEW packages will be INSTALLED:

    caffe2-cuda9.0-cudnn7-gcc4.8: 0.8.dev-py27h67f0068_0 caffe2
    future:                       0.16.0-py27_1                
    gflags:                       2.2.0-1                      
    glog:                         0.3.5-0                      
    leveldb:                      1.20-0                       
    libprotobuf:                  3.4.0-0                      
    lmdb:                         0.9.21-0                     
    opencv:                       3.1.0-np112py27_1            
    protobuf:                     3.4.0-py27_0                 
    snappy:                       1.1.6-0                      

The following packages will be UPDATED:

    anaconda:                     4.4.0-np112py27_0             --> custom-py27_0        
    conda:                        4.3.21-py27_0                 --> 4.3.30-py27h6ae6dc7_0

The following packages will be DOWNGRADED:

    jpeg:                         9b-0                          --> 8d-2                 
    libtiff:                      4.0.6-3                       --> 4.0.6-2              
    pillow:                       4.1.1-py27_0                  --> 3.4.2-py27_0         
    qt:                           5.6.2-4                       --> 5.6.2-2              

Proceed ([y]/n)? 

没的说,我这里只能选y吧 ,等了几个小时后,都下载好了。

Proceed ([y]/n)? y

gflags-2.2.0-1 100% |################################| Time: 0:00:01  99.19 kB/s
glog-0.3.5-0.t 100% |################################| Time: 0:00:35  18.59 kB/s
jpeg-8d-2.tar. 100% |################################| Time: 0:00:47  17.43 kB/s
lmdb-0.9.21-0. 100% |################################| Time: 0:00:34  17.11 kB/s
snappy-1.1.6-0 100% |################################| Time: 0:00:02  19.58 kB/s
leveldb-1.20-0 100% |################################| Time: 0:00:22  15.62 kB/s
libprotobuf-3. 100% |################################| Time: 0:04:55  15.33 kB/s
libtiff-4.0.6- 100% |################################| Time: 0:01:38  15.48 kB/s
anaconda-custo 100% |################################| Time: 0:00:00  20.59 kB/s
future-0.16.0- 100% |################################| Time: 0:00:40  16.27 kB/s
opencv-3.1.0-n 100% |################################| Time: 0:38:52  16.46 kB/s
pillow-3.4.2-p 100% |################################| Time: 0:00:59  14.74 kB/s
protobuf-3.4.0 100% |################################| Time: 0:00:24  19.81 kB/s
caffe2-cuda9.0 100% |################################| Time: 0:22:54  20.96 kB/s
qt-5.6.2-2.tar 100% |################################| Time: 0:26:06  29.61 kB/s
qt-5.6.2-2.tar 100% |################################| Time: 0:40:15  19.21 kB/s
conda-4.3.30-p 100% |################################| Time: 0:00:26  19.93 kB/s

装完后,测试下看:

>>> python2 -c 'from caffe2.python import core' 2>/dev/null && echo "Success" || echo "Failure"
Segmentation fault (core dumped)
Failure

>>> python2 -c 'from caffe2.python import workspace; print(workspace.NumCudaDevices())'
WARNING:root:This caffe2 python run does not have GPU support. Will run in CPU only mode.
WARNING:root:Debug message: libnccl.so.2: cannot open shared object file: No such file or directory
Segmentation fault (core dumped)

重新打开安装页面,有如下两段话:

This does NOT include libraries that are necessary to run the tutorials, such as jupyter. See the tutorials page for the list of required packages needed to run the tutorials.
NOTE: This will install Caffe2 and all of its required dependencies into the current conda environment. 
We strongly suggest that you create a new conda environment and install Caffe2 into that.
A conda environment is like a separate python installation and so won’t have problems with your other conda environments. You can learn more about conda environments here.

我决定进一步尝试 tutorials page里的 list of required packages needed to run the tutorials.

First download the tutorials source.

>>> git clone --recursive https://github.com/caffe2/tutorials caffe2_tutorials

Cloning into 'caffe2_tutorials'...
remote: Counting objects: 280, done.
remote: Total 280 (delta 0), reused 0 (delta 0), pack-reused 280
Receiving objects: 100% (280/280), 38.42 MiB | 71.00 KiB/s, done.
Resolving deltas: 100% (113/113), done.
Checking connectivity... done.

To run the tutorials you will need some third-party libraries, including ipython-notebooks and matplotlib. You can install everything you’ll need with the following command.

(Anaconda users: If you’re using Anaconda, use conda install instead of pip install.) Anaconda user?好家伙,那不就是我吗 ?

原本是这样的:

pip install -U pip setuptools
pip install \
    graphviz \
    hypothesis \
    ipython \
    jupyter \
    matplotlib \
    notebook \
    pydot \
    python-nvd3 \
    pyyaml \
    requests \
    scikit-image \
    scipy

我应该输入成:

>>> conda install setuptools
Fetching package metadata ...........
Solving package specifications: .

Package plan for installation in environment /home/cc/anaconda2:

The following NEW packages will be INSTALLED:

    certifi:    2018.1.18-py27_0     
    libgcc-ng:  7.2.0-hdf63c60_3     

The following packages will be UPDATED:

    conda:      4.3.30-py27h6ae6dc7_0 --> 4.5.0-py27_0        
    conda-env:  2.6.0-0               --> 2.6.0-h36134e3_1    
    pycosat:    0.6.2-py27_0          --> 0.6.3-py27ha4109ae_0
    setuptools: 27.2.0-py27_0         --> 39.0.1-py27_0       

Proceed ([y]/n)? 
conda-env-2.6. 100% |################################| Time: 0:00:00   8.48 kB/s
libgcc-ng-7.2. 100% |################################| Time: 0:07:00  15.20 kB/s
certifi-2018.1 100% |################################| Time: 0:00:06  21.14 kB/s
pycosat-0.6.3- 100% |################################| Time: 0:00:05  19.45 kB/s
setuptools-39. 100% |################################| Time: 0:00:40  14.60 kB/s
conda-4.5.0-py 100% |################################| Time: 0:01:12  14.74 kB/s
>>> conda install graphviz hypothesis ipython jupyter matplotlib notebook pydot python-nvd3 pyyaml requests scikit-image scipy

Solving environment: failed

PackagesNotFoundError: The following packages are not available from current channels:

- python-nvd3

Current channels:

- https://repo.anaconda.com/pkgs/main/linux-64
- https://repo.anaconda.com/pkgs/main/noarch
- https://repo.anaconda.com/pkgs/free/linux-64
- https://repo.anaconda.com/pkgs/free/noarch
- https://repo.anaconda.com/pkgs/r/linux-64
- https://repo.anaconda.com/pkgs/r/noarch
- https://repo.anaconda.com/pkgs/pro/linux-64
- https://repo.anaconda.com/pkgs/pro/noarch

我就把python-nvd3从安装里去掉了,接着搞:

>>> conda install graphviz hypothesis ipython jupyter matplotlib notebook pydot pyyaml requests scikit-image scipy
Solving environment: done

## Package Plan ##

  environment location: /home/cc/anaconda2

  added / updated specs: 
    - graphviz
    - hypothesis
    - ipython
    - jupyter
    - matplotlib
    - notebook
    - pydot
    - pyyaml
    - requests
    - scikit-image
    - scipy


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    numpy-1.12.1               |   py27h9378851_1         3.7 MB
    terminado-0.8.1            |           py27_1          20 KB
    ca-certificates-2018.03.07 |                0         124 KB
    pyyaml-3.12                |   py27h2d70dd7_1         159 KB
    pydot-1.2.4                |           py27_0          37 KB
    attrs-17.4.0               |           py27_0          40 KB
    graphviz-2.38.0            |                2        12.0 MB
    hypothesis-3.53.0          |           py27_0         251 KB
    requests-2.18.4            |   py27hc5b0589_1          90 KB
    scikit-image-0.13.1        |   py27h14c3975_1        23.2 MB
    urllib3-1.22               |   py27ha55213b_0         153 KB
    backports.shutil_get_terminal_size-1.0.0|   py27h5bc021e_2           8 KB
    notebook-5.4.1             |           py27_0         6.6 MB
    libgfortran-ng-7.2.0       |       hdf63c60_3         1.2 MB
    jupyter-1.0.0              |           py27_4           5 KB
    intel-openmp-2018.0.0      |                8         620 KB
    ipython-5.6.0              |           py27_0        1020 KB
    jupyter_core-4.4.0         |   py27h345911c_0          60 KB
    send2trash-1.5.0           |           py27_0          16 KB
    openssl-1.0.2o             |       h20670df_0         3.4 MB
    scikit-learn-0.19.1        |   py27h445a80a_0         5.3 MB
    pysocks-1.6.8              |           py27_0          22 KB
    libstdcxx-ng-7.2.0         |       hdf63c60_3         2.5 MB
    scipy-1.0.1                |   py27hfc37229_0        17.7 MB
    numexpr-2.6.4              |   py27hd318778_0         180 KB
    jupyter_client-5.2.3       |           py27_0         122 KB
    mkl-2018.0.2               |                1       205.2 MB
    coverage-4.5.1             |   py27h14c3975_0         208 KB
    imageio-2.3.0              |           py27_0         3.3 MB
    ------------------------------------------------------------
                                           Total:       287.4 MB

The following NEW packages will be INSTALLED:

    attrs:                              17.4.0-py27_0       
    backports.shutil_get_terminal_size: 1.0.0-py27h5bc021e_2
    ca-certificates:                    2018.03.07-0        
    coverage:                           4.5.1-py27h14c3975_0
    graphviz:                           2.38.0-2            
    hypothesis:                         3.53.0-py27_0       
    imageio:                            2.3.0-py27_0        
    intel-openmp:                       2018.0.0-8          
    libgfortran-ng:                     7.2.0-hdf63c60_3    
    libstdcxx-ng:                       7.2.0-hdf63c60_3    
    pydot:                              1.2.4-py27_0        
    pysocks:                            1.6.8-py27_0        
    send2trash:                         1.5.0-py27_0        
    urllib3:                            1.22-py27ha55213b_0 

The following packages will be UPDATED:

    ipython:                            5.3.0-py27_0         --> 5.6.0-py27_0         
    jupyter:                            1.0.0-py27_3         --> 1.0.0-py27_4         
    jupyter_client:                     5.0.1-py27_0         --> 5.2.3-py27_0         
    jupyter_core:                       4.3.0-py27_0         --> 4.4.0-py27h345911c_0 
    mkl:                                2017.0.1-0           --> 2018.0.2-1           
    notebook:                           5.0.0-py27_0         --> 5.4.1-py27_0         
    numexpr:                            2.6.2-np112py27_0    --> 2.6.4-py27hd318778_0 
    numpy:                              1.12.1-py27_0        --> 1.12.1-py27h9378851_1
    openssl:                            1.0.2l-0             --> 1.0.2o-h20670df_0    
    pyyaml:                             3.12-py27_0          --> 3.12-py27h2d70dd7_1  
    requests:                           2.14.2-py27_0        --> 2.18.4-py27hc5b0589_1
    scikit-image:                       0.13.0-np112py27_0   --> 0.13.1-py27h14c3975_1
    scikit-learn:                       0.18.1-np112py27_1   --> 0.19.1-py27h445a80a_0
    scipy:                              0.19.0-np112py27_0   --> 1.0.1-py27hfc37229_0 
    terminado:                          0.6-py27_0           --> 0.8.1-py27_1         

Proceed ([y]/n)? 

 

posted on 2018-04-12 22:13  caffeauto  阅读(5424)  评论(1编辑  收藏  举报

导航