第7次实践作业 25组

一、在树莓派中安装opencv库

参考资料：
Raspbian Stretch: Install OpenCV 3 + Python on your Raspberry Pi

（1）扩展文件系统

sudo raspi-config

1.选择“高级选项”菜单项

2.接下来选择“扩展文件系统”

3.按Enter键，选择Finish，然后重新启动Pi

4.验证磁盘是否已扩展，并检查输出

df -h

可以看重启后，Raspbian文件系统已扩展为包括所有16GB的micro-SD卡，已经使用43％

5.删除LibreOffice和Wolfram引擎以释放Pi上的一些空间

sudo apt-get purge wolfram-engine
sudo apt-get purge libreoffice *
sudo apt-get clean
sudo apt-get autoremove

可以看到回收了将近1.6G

（2）安装依赖

1.换源

软件更新源

sudo nano /etc/apt/sources.list 
deb http://mirrors.ustc.edu.cn/raspbian/raspbian/ buster main contrib non-free rpi

系统更新源

sudo nano /etc/apt/sources.list.d/raspi.list  
deb http://mirrors.ustc.edu.cn/archive.raspberrypi.org/debian/ buster main ui

2.换完源以后就不会出现下列软件包有未满足的依赖关系的问题了

# 更新软件源，更新软件
sudo apt-get update && sudo apt-get upgrade

# Cmake等开发者工具
sudo apt-get install build-essential cmake pkg-config

# 图片I/O包
sudo apt-get install libjpeg-dev libtiff5-dev libjasper-dev libpng12-dev

# 视频I/O包
sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev
sudo apt-get install libxvidcore-dev libx264-dev

# OpenCV用于显示图片的子模块需要GTK
sudo apt-get install libgtk2.0-dev libgtk-3-dev

# 性能优化包
sudo apt-get install libatlas-base-dev gfortran

# 安装 Python2.7 & Python3
sudo apt-get install python2.7-dev python3-dev

（3）下载OpenCV源代码

1.下载源代码

这边我们尝试在树莓派内和windows都下载源代码，树莓派内会出现无法解压的问题，而在windows会提示网络错误无法下载
从同学那边获得了下载好的两个zip包，通过在上一次实践中配置好的ftp传入传入树莓派

2.解压

cd ~
unzip opencv-4.3.0.zip
unzip opencv_contrib-4.3.0.zip

注意：确保 opencv和 opencv_contrib版本相同
unzip命令解析文件出现问题的话，可使用jar命令来解析该文件

3.准备编译环境

不知道出于什么原因，我们使用python3安装虚拟机却在安装信息中检测不到python3，所以这边我们使用的python2安装虚拟机

# 安装pip
wget https://bootstrap.pypa.io/get-pip.py
sudo python get-pip.py
sudo python3 get-pip.py

# 安装虚拟环境，防止依赖冲突
sudo pip install virtualenv virtualenvwrapper
sudo rm -rf ~/.cache/pip

# 更新〜/ .profile文件
nano ~/.profile
export WORKON_HOME=$HOME/.virtualenvs
export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3
export VIRTUALENVWRAPPER_VIRTUALENV=/usr/local/bin/virtualenv
source /usr/local/bin/virtualenvwrapper.sh
export VIRTUALENVWRAPPER_ENV_BIN_DIR=bin

# 每次新开终端，需要虚拟环境时都要运行
source ~/.profile

# 创建虚拟环境cv，这里我们用的是python2
mkvirtualenv cv -p python2

# 进入虚拟环境
workon cv

# 安装numpy，我们使用的是1.16.4版本
pip install numpy==1.16.4

4.编译OpenCV

要确保已经进入了cv虚拟环境，命令提示符开头有(cv)

# 这里我们用的是4.3.0版本

cd ~/opencv-4.3.0/
mkdir build
cd build

# 设置CMake构建选项
cmake -D CMAKE_BUILD_TYPE=RELEASE \
    -D CMAKE_INSTALL_PREFIX=/usr/local \
    -D INSTALL_PYTHON_EXAMPLES=ON \
    -D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib-4.3.0/modules \
    -D BUILD_EXAMPLES=ON ..

为了避免编译时内存不足导致的CPU挂起，调整swap交换文件大小：

# CONF_SWAPSIZE由100改为1024，编译完成后改回来
sudo nano /etc/dphys-swapfile
# 重启swap服务
sudo /etc/init.d/dphys-swapfile stop
sudo /etc/init.d/dphys-swapfile start

到这里就完成了大部分准备工作，开始编译：

# 开始编译，很耗时
make -j4

终于成功啦！！！这边遇到了一些问题

5.安装OpenCV

sudo make install
sudo ldconfig

检查OpenCV的安装位置

ls -l /usr/local/lib/python2.7/site-packages/
cd ~/.virtualenvs/cv/lib/python2.7/site-packages/
ln -s /usr/local/lib/python2.7/site-packages/cv2.so cv2.so

6.测试OpenCV

source ~/.profile
workon cv
python
import cv2
cv2.__version__

二、使用opencv和python控制树莓派的摄像头

（1）测试树莓派相机

raspistill -o output.jpg

（2）安装picreame

source ~/.profile 
workon cv 
pip install "picamera[array]"

（3）拍照测试

nano takephoto.py
python takephoto.py

takephoto.py

# import the necessary packages
from picamera.array import PiRGBArray
from picamera import PiCamera
import time
import cv2
 
# initialize the camera and grab a reference to the raw camera capture
camera = PiCamera()
rawCapture = PiRGBArray(camera)
 
# allow the camera to warmup
time.sleep(5)
 
# grab an image from the camera
camera.capture(rawCapture, format="bgr")
image = rawCapture.array
 
# display the image on screen and wait for a keypress
cv2.imshow("Image", image)
cv2.waitKey(0)

三、利用树莓派的摄像头实现人脸识别

（1）安装所需依赖库

source ~/.profile 
workon cv 
pip install dlib
pip install face_recognition

dlib下载成功

但是face_recognition一直超时

从网上下载了两个文件

通过ftp传入树莓派后进行安装，这边还是有点慢，我们使用豆瓣源进行加速

python -m pip install face_recognition_models-0.3.0-py2.py3-none-any.whl -i http://pypi.douban.com/simple --trusted-host pypi.douban.com

python -m pip install face_recognition-1.3.0-py2.py3-none-any.whl -i http://pypi.douban.com/simple --trusted-host pypi.douban.com

测试安装

python
import face_recognition

安装成功啦

（2）切换到代码和图片所在文件夹运行代码

1.将代码和图片传入树莓派

2.运行代码

facerec_on_raspberry_pi.py

# This is a demo of running face recognition on a Raspberry Pi.
# This program will print out the names of anyone it recognizes to the console.
# To run this, you need a Raspberry Pi 2 (or greater) with face_recognition and
# the picamera[array] module installed.
# You can follow this installation instructions to get your RPi set up:
# https://gist.github.com/ageitgey/1ac8dbe8572f3f533df6269dab35df65

import face_recognition
import picamera
import numpy as np

# Get a reference to the Raspberry Pi camera.
# If this fails, make sure you have a camera connected to the RPi and that you
# enabled your camera in raspi-config and rebooted first.
camera = picamera.PiCamera()
camera.resolution = (320, 240)
output = np.empty((240, 320, 3), dtype=np.uint8)

# Load a sample picture and learn how to recognize it.
print("Loading known face image(s)")
image = face_recognition.load_image_file("Einstein_origin.jpg")
face_encoding = face_recognition.face_encodings(image)[0]

# Initialize some variables
face_locations = []
face_encodings = []

while True:

    print("Capturing image.")
    # Grab a single frame of video from the RPi camera as a numpy array
    camera.capture(output, format="rgb")

    # Find all the faces and face encodings in the current frame of video
    face_locations = face_recognition.face_locations(output)

    print("Found {} faces in image.".format(len(face_locations)))
    face_encodings = face_recognition.face_encodings(output, face_locations)

    # Loop over each face found in the frame to see if it's someone we know.
    for face_encoding in face_encodings:

        # See if the face is a match for the known face(s)
        match = face_recognition.compare_faces([face_encoding], face_encoding)
        name = "<Unknown Person>"

        if match[0]:
            name = "Einstein"
        print("I see someone named {}!".format(name))

facerec_from_webcam_faster.py

import face_recognition
import cv2
import numpy as np

# This is a demo of running face recognition on live video from your webcam. It's a little more complicated than the
# other example, but it includes some basic performance tweaks to make things run a lot faster:
#   1. Process each video frame at 1/4 resolution (though still display it at full resolution)
#   2. Only detect faces in every other frame of video.

# PLEASE NOTE: This example requires OpenCV (the `cv2` library) to be installed only to read from your webcam.
# OpenCV is *not* required to use the face_recognition library. It's only required if you want to run this
# specific demo. If you have trouble installing it, try any of the other demos that don't require it instead.

# Get a reference to webcam #0 (the default one)
video_capture = cv2.VideoCapture(0)

# Load a sample picture and learn how to recognize it.
Jay_Chou_image = face_recognition.load_image_file("Jay_Chou.jpg")
Jay_Chou_face_encoding = face_recognition.face_encodings(Jay_Chou_image)[0]

# Load a second sample picture and learn how to recognize it.
JJ_Lin_image = face_recognition.load_image_file("JJ_Lin.jpg")
JJ_Lin_face_encoding = face_recognition.face_encodings(JJ_Lin_image)[0]

# Create arrays of known face encodings and their names
known_face_encodings = [
    Jay_Chou_encoding,
    JJ_Lin_face_encoding
]

known_face_names = [
    "Jay_Chou",
    "JJ_Lin"
]



# Initialize some variables
face_locations = []
face_encodings = []
face_names = []

process_this_frame = True

while True:

    # Grab a single frame of video
    ret, frame = video_capture.read()

    # Resize frame of video to 1/4 size for faster face recognition processing
    small_frame = cv2.resize(frame, (0, 0), fx=0.25, fy=0.25)

    # Convert the image from BGR color (which OpenCV uses) to RGB color (which face_recognition uses)
    rgb_small_frame = small_frame[:, :, ::-1]

    # Only process every other frame of video to save time
    if process_this_frame:

        # Find all the faces and face encodings in the current frame of video
        face_locations = face_recognition.face_locations(rgb_small_frame)
        face_encodings = face_recognition.face_encodings(rgb_small_frame, face_locations)

        face_names = []
        for face_encoding in face_encodings:
            # See if the face is a match for the known face(s)
            matches = face_recognition.compare_faces(known_face_encodings, face_encoding)
            name = "Unknown"

            # # If a match was found in known_face_encodings, just use the first one.
            # if True in matches:
            #     first_match_index = matches.index(True)
            #     name = known_face_names[first_match_index]
            # Or instead, use the known face with the smallest distance to the new face
            face_distances = face_recognition.face_distance(known_face_encodings, face_encoding)
            best_match_index = np.argmin(face_distances)
            if matches[best_match_index]:
                name = known_face_names[best_match_index]

            face_names.append(name)

    process_this_frame = not process_this_frame

    # Display the results
    for (top, right, bottom, left), name in zip(face_locations, face_names):
        # Scale back up face locations since the frame we detected in was scaled to 1/4 size
        top *= 4
        right *= 4
        bottom *= 4
        left *= 4

        # Draw a box around the face
        cv2.rectangle(frame, (left, top), (right, bottom), (0, 0, 255), 2)

        # Draw a label with a name below the face
        cv2.rectangle(frame, (left, bottom - 35), (right, bottom), (0, 0, 255), cv2.FILLED)
        font = cv2.FONT_HERSHEY_DUPLEX
        cv2.putText(frame, name, (left + 6, bottom - 6), font, 1.0, (255, 255, 255), 1)

    # Display the resulting image
    cv2.imshow('Video', frame)

    # Hit 'q' on the keyboard to quit!
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Release handle to the webcam
video_capture.release()
cv2.destroyAllWindows()

四、结合微服务的进阶任务

（1）Docker的安装和配置

1.安装

sudo curl -sSL https://get.docker.com | sh

2.添加用户到docker组

sudo usermod pi -aG docker

3.配置

# config cgroup for Docker
echo Adding " cgroup_enable=cpuset cgroup_enable=memory" to /boot/cmdline.txt
sudo cp /boot/cmdline.txt /boot/cmdline_backup.txt
# if you encounter problems, try changing cgroup_memory=1 to cgroup_enable=memory.
orig="$(head -n1 /boot/cmdline.txt) cgroup_enable=cpuset cgroup_memory=1"
echo $orig | sudo tee /boot/cmdline.txt

sudo reboot

（2）在opencv的docker容器中运行示例代码facerec_on_raspberry_pi.py

1.下载示例镜像并开启容器

docker run -it \
--name face_recognition \
--device /dev/vchiq \
registry.cn-hangzhou.aliyuncs.com/denverdino/face_recognition \
bash

2.查看并进入代码存放目录

ls
cd face_recognition/examples/

3.示例代码facerec_on_raspberry_pi.py

# This is a demo of running face recognition on a Raspberry Pi.
# This program will print out the names of anyone it recognizes to the console.

# To run this, you need a Raspberry Pi 2 (or greater) with face_recognition and
# the picamera[array] module installed.
# You can follow this installation instructions to get your RPi set up:
# https://gist.github.com/ageitgey/1ac8dbe8572f3f533df6269dab35df65

import face_recognition
import picamera
import numpy as np

# Get a reference to the Raspberry Pi camera.
# If this fails, make sure you have a camera connected to the RPi and that you
# enabled your camera in raspi-config and rebooted first.
camera = picamera.PiCamera()
camera.resolution = (320, 240)
output = np.empty((240, 320, 3), dtype=np.uint8)

# Load a sample picture and learn how to recognize it.
print("Loading known face image(s)")
obama_image = face_recognition.load_image_file("obama_small.jpg")
obama_face_encoding = face_recognition.face_encodings(obama_image)[0]

# Initialize some variables
face_locations = []
face_encodings = []

while True:
    print("Capturing image.")
    # Grab a single frame of video from the RPi camera as a numpy array
    camera.capture(output, format="rgb")

    # Find all the faces and face encodings in the current frame of video
    face_locations = face_recognition.face_locations(output)
    print("Found {} faces in image.".format(len(face_locations)))
    face_encodings = face_recognition.face_encodings(output, face_locations)

    # Loop over each face found in the frame to see if it's someone we know.
    for face_encoding in face_encodings:
        # See if the face is a match for the known face(s)
        match = face_recognition.compare_faces([obama_face_encoding], face_encoding)
        name = "<Unknown Person>"

        if match[0]:
            name = "Barack Obama"

        print("I see someone named {}!".format(name))

4.运行代码

python3 facerec_on_raspberry_pi.py

5.结果如下

（3）在opencv的docker容器中运行示例代码facerec_from_webcam_faster.py

这边使用docker hub上的opencv的镜像

1.权限分配

# 允许所有用户访问显示接口
xhost +

2.创建容器

docker run -it --rm \ 
 --device=/dev/video0 \ 
 -e DISPLAY=unix$DISPLAY \ 
 -v /tmp/.X11-unix:/tmp/.X11-unix \ 
 -v /home/pi/Desktop/face-recognition:/face-recognition \ 
 demosense/raspberrypi3-opencv \
 bash

3.安装依赖

pip install picamera dlib face_recognition numpy
apt install x11-xserver-utils

4.运行代码facerec_from_webcam_faster.py

python facerec_from_webcam_faster.py

5.结果如下

五、遇到的问题及其解决方法

（1）安装时提示下列软件包有未满足的依赖关系

解决方法：
一个方法是：

sudo apt-get install aptitude
sudo aptitude install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev

但是在cmake或make -j4两个步骤可能会出问题，所以还是不建议用这种方法
我们就是在cmake那一步出了问题

另外一个方法是换源：
软件更新源

sudo nano /etc/apt/sources.list 
deb http://mirrors.ustc.edu.cn/raspbian/raspbian/ buster main contrib non-free rpi

系统更新源

sudo nano /etc/apt/sources.list.d/raspi.list  
deb http://mirrors.ustc.edu.cn/archive.raspberrypi.org/debian/ buster main ui

换完源以后就不会出现软件包有未满足的依赖关系的问题啦

（2）解压zip包的时候遇到如下问题

这边应该是两个zip包下载被中断导致文件不完整，不知道是什么原因，在树莓派和windows下载都不成功
解决方法：
同学分享了下载好的zip包

（3）编译OpenCV遇到的问题

• 问题1：

解决方法：
参考博客：
安装OpenCV时提示缺少boostdesc_bgm.i文件的问题解决方案
把博客中给的缩包解压到目录opencv_contrib/modules/xfeatures2d/src/下即可

• 问题2：

解决方法：

从/home/pi/opencv-4.3.0/modules/features2d/test 拷贝至/home/pi/opencv_contrib-4.3.0/modules/xfeatures2d/test

将/home/pi/opencv_contrib-4.3.0/modules/xfeatures2d/test下的test_features2d.cpp中的

这两行，改成

#include "test_detectors_regression.impl.hpp"
#include "test_descriptors_regression.impl.hpp"

在test_rotation_and_scale_invariance.cpp下也做如下修改

（4）虽然使用Python3安装，但是显示出来的只有Python2，在编译安装之后发现根本没有指定的site-packages目录

解决方法：
一开始认为可能是某一步没做好，重装了系统，进行了虚拟机环境配置等检查，但是还是一样的结果，最后决定使用Python2.7

（5）验证安装时输入import cv2，提示ImportError: numpy.core.multiarray failed to import

原因：决定使用Python2安装之后，安装了比较新版本的numpy，版本不匹配
解决方法：

# 卸载已安装的numpy
pip unstall numpy

# 下载1.16.4版本的numpy
pip install numpy==1.16.4

但是又出现了一个小问题：ERROR: THESE PACKAGES DO NOT MATCH THE HASHES FROM THE REQUIREMENTS FILE.
解决方法：

# 使用如下命令下载
pip install --upgrade numpy==1.16.4

再编译安装，就成功啦

（6）利用树莓派的摄像头实现人脸识别facerec_from_webcam_faster.py遇到IndexError: list index out of range

原因：
应该是图片的原因，我们本来选择使用这张图，可能的原因是侧脸或者图片不清晰

解决方法：
修改了代码换成了清晰的正脸图，就成功啦

（7）在结合微服务的进阶任务部分无法下载face_recognition

直接使用pip3 install face_recognition会超时报错
解决方法：
尝试使用离线下载的方式
但是在容器中使用离线下载的方式会出现无法识别命令的问题
尝试更新并下载命令（这边耗时3小时）还是不行
就决定换一种方式做第四部分了

六、小组成员名单以及在线协作的图片

（1）小组成员名单及分工

第25组

学号	姓名	分工
021700201	蔡峰	实际操作和博客撰写
171709030	吴珂雨	实际操作和博客撰写
111700233	郑木平	实验步骤整理和提供代码及测试图片

（2）分工协作

采用多次语音电话或者屏幕分分享的方式共同探讨解决问题

实际操作

（3）小结

  太太太太感谢可爱的晓楠了，分享了OpenCV的两个zip包给我们并且给了我们几个很有用的建议（这个都写在博客里啦），完美的避开了重做之前我遇到的几个大坑。但是还有更多大坑等着我们..做第一部分的时候我觉得我们简直是奇葩中的战斗机，遇到了很多匪夷所思的问题，无法下载和解压OpenCV的两个zip包，用了python3下载虚拟机，cmake结束后没有显示有python3，python3.7的文件夹中也没有site-packages这个文件夹，import cv2又显示版本不匹配等等。感觉就我们遇到了这些奇怪的问题，有的问题甚至不能够被解决，只能换个方式。虽然最后使用python2.7版本安装虚拟机，也使用了比较低版本的numpy，最后好歹是成功了，但我还是很想知道为什么会有这些奇怪的问题。
  团队协作大概耗费了3天的时间，确实遇到了很多很多问题，也寻找了很多解决方法，进行了各种尝试，有时候确实没办法解决，换一种方式，就容易了许多，当然我们也有换了好几种方式的情况。过程太艰辛啦，总之最后是做完了，成就感满满。
  这次实践学会了使用opencv和python控制树莓派的摄像头，利用树莓派的摄像头实现人脸识别，部署opencv的docker容器，并在opencv的docker容器中运行示例代码，希望下一次的实践能够碰到的问题能够友善一些。