Python 3 利用 Dlib 实现摄像头实时人脸识别
0. 引言
利用 Python 开发，对于输入的摄像头视频流，借助 Dlib 提供的检测识别模型来进行人脸识别；
首先，从摄像头中录入（裁剪）人脸图片存储到本地，然后提取特征，构建预设人脸特征；
对于 person_x，根据录入的多张人脸图片提取 128D 特征值，然后计算该人的 128D 特征均值；
然后和摄像头中实时获取到的人脸提取出的特征值，计算欧氏距离，判定是否为同一张人脸；
识别模型：基于 Dlib 的 ResNet 预训练模型（dlib_face_recognition_resnet_model_v1.dat）
识别算法：ResNet 神经网络（This model is a ResNet network with 29 conv layers. It's essentially a version of the ResNet-34 network from the paper Deep Residual Learning for Image Recognition by He, Zhang, Ren, and Sun with a few layers removed and the number of filters per layer reduced by half）
调用的模型以及大概的耗时：
博客中代码以 GitHub 为准，博客中可能没有及时更新；
# Blog : http://www.cnblogs.com/AdaminXie
# GitHub : https://github.com/coneypo/Dlib_face_recognition_from_camera
支持的功能：
- 支持人脸数据采集，自行建立人脸数据库
- 支持录入人脸的时候设置姓名
- 调用摄像头实时人脸检测和识别
- 支持同时识别多张人脸
- 支持中文姓名显示
- 利用 OT 来加速识别
人脸识别 / Face Recognition 的说明：
Wikipedia 上关于人脸识别系统 / Face Recognition System 的描述：they work by comparing selected facial features from given image with faces within a database.
本项目中就是比较 预设的人脸的特征 和 摄像头实时获取到的人脸的特征 ；
核心就是 提取 128D 人脸特征，然后计算 摄像头人脸特征 和 预设的特征脸的欧式距离，进行比对；
效果如下：
图 1 多人脸实时识别效果
1. 总体流程
先说下 人脸检测 ( Face detection ) 和 人脸识别 ( Face Recognition ) ，前者是达到检测出场景中人脸的目的就可以了，而后者不仅需要检测出人脸，还要和已有人脸数据进行比对，识别出是否在数据库中，或者进行身份标注之类处理，人脸检测和人脸识别两者有时候可能会被理解混淆；
利用“dlib_face_recognition_resnet_model_v1.dat” 这个训练好的 resnet-34 模型，提取 人脸图像的 128D 特征，然后比对不同人脸图片的 128D 特征的欧式距离，设定一个 阈值 来判断是否为同一张脸；
1 # face recognition model, the object maps human faces into 128D vectors
2 facerec = dlib.face_recognition_model_v1("dlib_face_recognition_resnet_model_v1.dat")
3
4 shape = predictor(img, dets[0])
5 face_descriptor = facerec.compute_face_descriptor(img, shape)
图 2 总体设计流程
2. 源码介绍
有三块代码，分别是人脸录入，特征提取和人脸识别：
- get_faces_from_camera.py
- features_extraction_to_csv.py
- face_reco_from_camera.py
2.1 get_faces_from_camera.py / 人脸注册录入
人脸识别需要将 提取到的图像数据 和 已有图像数据 进行比对分析，所以这部分代码实现的功能就是 人脸录入；
程序会生成一个窗口，显示调用的摄像头实时获取的图像；
（关于摄像头的调用方式可以参考这里： Python 3 利用 Dlib 19.7 实现摄像头人脸检测特征点标定）；
2.1.1 GUI with OpenCV
利用 OpenCV 生成的窗口界面，用户通过键盘输入进行人脸捕获：
- “N” 新录入人脸，新建文件夹 person_X/ 用来存储某人的人脸图像
- "S" 开始捕获人脸，将捕获到的人脸放到 person_X/ 路径下
- “Q” 退出窗口
摄像头的调用是利用 opencv 库的 cv2.VideoCapture(0), 此处参数为 0 代表调用的是笔记本的默认摄像头，你也可以让它调用传入已有视频文件；
可以参考 https://github.com/coneypo/Dlib_face_recognition_from_camera/blob/master/how_to_use_camera.py 如何通过 OpenCV 调用摄像头；
图 3 get_face_from_camera.py 的界面
“N”+“S”之后捕获到的一组人脸示例；
图 4 捕获到的一组人脸
1 # Copyright (C) 2020 coneypo
2 # SPDX-License-Identifier: MIT
3
4 # Author: coneypo
5 # Blog: http://www.cnblogs.com/AdaminXie
6 # GitHub: https://github.com/coneypo/Dlib_face_recognition_from_camera
7 # Mail: coneypo@foxmail.com
8
9 # 进行人脸录入 / Face register
10
11 import dlib
12 import numpy as np
13 import cv2
14 import os
15 import shutil
16 import time
17
18 # Dlib 正向人脸检测器 / Use frontal face detector of Dlib
19 detector = dlib.get_frontal_face_detector()
20
21
22 class Face_Register:
23 def __init__(self):
24 self.path_photos_from_camera = "data/data_faces_from_camera/"
25 self.font = cv2.FONT_ITALIC
26
27 self.existing_faces_cnt = 0 # 已录入的人脸计数器 / cnt for counting saved faces
28 self.ss_cnt = 0 # 录入 personX 人脸时图片计数器 / cnt for screen shots
29 self.current_frame_faces_cnt = 0 # 录入人脸计数器 / cnt for counting faces in current frame
30
31 self.save_flag = 1 # 之后用来控制是否保存图像的 flag / The flag to control if save
32 self.press_n_flag = 0 # 之后用来检查是否先按 'n' 再按 's' / The flag to check if press 'n' before 's'
33
34 # FPS
35 self.frame_time = 0
36 self.frame_start_time = 0
37 self.fps = 0
38
39 # 新建保存人脸图像文件和数据CSV文件夹 / Make dir for saving photos and csv
40 def pre_work_mkdir(self):
41 # 新建文件夹 / Create folders to save faces images and csv
42 if os.path.isdir(self.path_photos_from_camera):
43 pass
44 else:
45 os.mkdir(self.path_photos_from_camera)
46
47 # 删除之前存的人脸数据文件夹 / Delete the old data of faces
48 def pre_work_del_old_face_folders(self):
49 # 删除之前存的人脸数据文件夹, 删除 "/data_faces_from_camera/person_x/"...
50 folders_rd = os.listdir(self.path_photos_from_camera)
51 for i in range(len(folders_rd)):
52 shutil.rmtree(self.path_photos_from_camera+folders_rd[i])
53 if os.path.isfile("data/features_all.csv"):
54 os.remove("data/features_all.csv")
55
56 # 如果有之前录入的人脸, 在之前 person_x 的序号按照 person_x+1 开始录入 / Start from person_x+1
57 def check_existing_faces_cnt(self):
58 if os.listdir("data/data_faces_from_camera/"):
59 # 获取已录入的最后一个人脸序号 / Get the order of latest person
60 person_list = os.listdir("data/data_faces_from_camera/")
61 person_num_list = []
62 for person in person_list:
63 person_num_list.append(int(person.split('_')[-1]))
64 self.existing_faces_cnt = max(person_num_list)
65
66 # 如果第一次存储或者没有之前录入的人脸, 按照 person_1 开始录入 / Start from person_1
67 else:
68 self.existing_faces_cnt = 0
69
70 # 获取处理之后 stream 的帧数 / Update FPS of video stream
71 def update_fps(self):
72 now = time.time()
73 self.frame_time = now - self.frame_start_time
74 self.fps = 1.0 / self.frame_time
75 self.frame_start_time = now
76
77 # 生成的 cv2 window 上面添加说明文字 / PutText on cv2 window
78 def draw_note(self, img_rd):
79 # 添加说明 / Add some notes
80 cv2.putText(img_rd, "Face Register", (20, 40), self.font, 1, (255, 255, 255), 1, cv2.LINE_AA)
81 cv2.putText(img_rd, "FPS: " + str(self.fps.__round__(2)), (20, 100), self.font, 0.8, (0, 255, 0), 1,
82 cv2.LINE_AA)
83 cv2.putText(img_rd, "Faces: " + str(self.current_frame_faces_cnt), (20, 140), self.font, 0.8, (0, 255, 0), 1, cv2.LINE_AA)
84 cv2.putText(img_rd, "N: Create face folder", (20, 350), self.font, 0.8, (255, 255, 255), 1, cv2.LINE_AA)
85 cv2.putText(img_rd, "S: Save current face", (20, 400), self.font, 0.8, (255, 255, 255), 1, cv2.LINE_AA)
86 cv2.putText(img_rd, "Q: Quit", (20, 450), self.font, 0.8, (255, 255, 255), 1, cv2.LINE_AA)
87
88 # 获取人脸 / Main process of face detection and saving
89 def process(self, stream):
90 # 1. 新建储存人脸图像文件目录 / Create folders to save photos
91 self.pre_work_mkdir()
92
93 # 2. 删除 "/data/data_faces_from_camera" 中已有人脸图像文件 / Uncomment if want to delete the saved faces and start from person_1
94 if os.path.isdir(self.path_photos_from_camera):
95 self.pre_work_del_old_face_folders()
96
97 # 3. 检查 "/data/data_faces_from_camera" 中已有人脸文件
98 self.check_existing_faces_cnt()
99
100 while stream.isOpened():
101 flag, img_rd = stream.read() # Get camera video stream
102 kk = cv2.waitKey(1)
103 faces = detector(img_rd, 0) # Use Dlib face detector
104
105 # 4. 按下 'n' 新建存储人脸的文件夹 / Press 'n' to create the folders for saving faces
106 if kk == ord('n'):
107 self.existing_faces_cnt += 1
108 current_face_dir = self.path_photos_from_camera + "person_" + str(self.existing_faces_cnt)
109 os.makedirs(current_face_dir)
110 print('\n')
111 print("新建的人脸文件夹 / Create folders: ", current_face_dir)
112
113 self.ss_cnt = 0 # 将人脸计数器清零 / Clear the cnt of screen shots
114 self.press_n_flag = 1 # 已经按下 'n' / Pressed 'n' already
115
116 # 5. 检测到人脸 / Face detected
117 if len(faces) != 0:
118 # 矩形框 / Show the ROI of faces
119 for k, d in enumerate(faces):
120 # 计算矩形框大小 / Compute the size of rectangle box
121 height = (d.bottom() - d.top())
122 width = (d.right() - d.left())
123 hh = int(height/2)
124 ww = int(width/2)
125
126 # 6. 判断人脸矩形框是否超出 480x640 / If the size of ROI > 480x640
127 if (d.right()+ww) > 640 or (d.bottom()+hh > 480) or (d.left()-ww < 0) or (d.top()-hh < 0):
128 cv2.putText(img_rd, "OUT OF RANGE", (20, 300), self.font, 0.8, (0, 0, 255), 1, cv2.LINE_AA)
129 color_rectangle = (0, 0, 255)
130 save_flag = 0
131 if kk == ord('s'):
132 print("请调整位置 / Please adjust your position")
133 else:
134 color_rectangle = (255, 255, 255)
135 save_flag = 1
136
137 cv2.rectangle(img_rd,
138 tuple([d.left() - ww, d.top() - hh]),
139 tuple([d.right() + ww, d.bottom() + hh]),
140 color_rectangle, 2)
141
142 # 7. 根据人脸大小生成空的图像 / Create blank image according to the size of face detected
143 img_blank = np.zeros((int(height*2), width*2, 3), np.uint8)
144
145 if save_flag:
146 # 8. 按下 's' 保存摄像头中的人脸到本地 / Press 's' to save faces into local images
147 if kk == ord('s'):
148 # 检查有没有先按'n'新建文件夹 / Check if you have pressed 'n'
149 if self.press_n_flag:
150 self.ss_cnt += 1
151 for ii in range(height*2):
152 for jj in range(width*2):
153 img_blank[ii][jj] = img_rd[d.top()-hh + ii][d.left()-ww + jj]
154 cv2.imwrite(current_face_dir + "/img_face_" + str(self.ss_cnt) + ".jpg", img_blank)
155 print("写入本地 / Save into：", str(current_face_dir) + "/img_face_" + str(self.ss_cnt) + ".jpg")
156 else:
157 print("请先按 'N' 来建文件夹, 按 'S' / Please press 'N' and press 'S'")
158
159 self.current_frame_faces_cnt = len(faces)
160
161 # 9. 生成的窗口添加说明文字 / Add note on cv2 window
162 self.draw_note(img_rd)
163
164 # 10. 按下 'q' 键退出 / Press 'q' to exit
165 if kk == ord('q'):
166 break
167
168 # 11. Update FPS
169 self.update_fps()
170
171 cv2.namedWindow("camera", 1)
172 cv2.imshow("camera", img_rd)
173
174 def run(self):
175 cap = cv2.VideoCapture(0)
176 self.process(cap)
177
178 cap.release()
179 cv2.destroyAllWindows()
180
181
182 def main():
183 Face_Register_con = Face_Register()
184 Face_Register_con.run()
185
186
187 if __name__ == '__main__':
188 main()
考虑到有可能需要保存的矩形框超出摄像头范围，对于这种异常，如果矩形框超出范围，矩形框会从白变红，然后提示 "OUT OF RANGE"，这时候不允许用户保存人脸图片;
图 5 人脸录入异常（Out of range）处理
get_face_from_camera.py 的输出 log：
新建的人脸文件夹 / Create folders: data/data_faces_from_camera/person_1
写入本地 / Save into： data/data_faces_from_camera/person_1/img_face_1.jpg
写入本地 / Save into： data/data_faces_from_camera/person_1/img_face_2.jpg
写入本地 / Save into： data/data_faces_from_camera/person_1/img_face_3.jpg
写入本地 / Save into： data/data_faces_from_camera/person_1/img_face_4.jpg
新建的人脸文件夹 / Create folders: data/data_faces_from_camera/person_2
写入本地 / Save into： data/data_faces_from_camera/person_2/img_face_1.jpg
写入本地 / Save into： data/data_faces_from_camera/person_2/img_face_2.jpg
新建的人脸文件夹 / Create folders: data/data_faces_from_camera/person_3
写入本地 / Save into： data/data_faces_from_camera/person_3/img_face_1.jpg
写入本地 / Save into： data/data_faces_from_camera/person_3/img_face_2.jpg
2.1.2 GUI with Tkinter
利用 Tkinter 生成的 GUI，用户界面更加丰富，也可以在该界面，录入人脸的时候设置姓名；
- 如果需要，点击 “Clear” 来删除 /data/data_faces_from_camera/" 下的所有图片和 "features_all.csv"
- 直接点击 "Input"，会在人脸文件夹，相应的生成 "person_1/" ...
- 输入姓名，比如“Tom”，然后再点击 'Input"，会生成 "person_1_Tom/" ..
- 如果 "/data/data_faces_from_camera/" 目录下已有 person_x，会自动从 person_x+1 开始新建文件夹；
点击 Step 3 中的按钮“Save current face”，将当前帧中检测到的人脸保存下来；
2.2 features_extraction_to_csv.py
这部分代码实现的功能是将之前捕获到的人脸图像文件，提取出 128D 特征，然后计算出某人人脸数据的特征均值存入 CSV 中，方便之后识别时候进行比对；
利用 numpy.mean() 计算特征均值，生成一个存储所有录入人脸数据 database 的 "features_all.csv"；
features_extraction_to_csv.py 源码:
1 # Copyright (C) 2018-2021 coneypo 2 # SPDX-License-Identifier: MIT 3 4 # Author: coneypo 5 # Blog: http://www.cnblogs.com/AdaminXie 6 # GitHub: https://github.com/coneypo/Dlib_face_recognition_from_camera 7 # Mail: coneypo@foxmail.com 8 9 # 从人脸图像文件中提取人脸特征存入 "features_all.csv" / Extract features from images and save into "features_all.csv" 10 11 import os 12 import dlib 13 import csv 14 import numpy as np 15 import logging 16 import cv2 17 18 # 要读取人脸图像文件的路径 / Path of cropped faces 19 path_images_from_camera = "data/data_faces_from_camera/" 20 21 # Dlib 正向人脸检测器 / Use frontal face detector of Dlib 22 detector = dlib.get_frontal_face_detector() 23 24 # Dlib 人脸 landmark 特征点检测器 / Get face landmarks 25 predictor = dlib.shape_predictor('data/data_dlib/shape_predictor_68_face_landmarks.dat') 26 27 # Dlib Resnet 人脸识别模型，提取 128D 的特征矢量 / Use Dlib resnet50 model to get 128D face descriptor 28 face_reco_model = dlib.face_recognition_model_v1("data/data_dlib/dlib_face_recognition_resnet_model_v1.dat") 29 30 31 # 返回单张图像的 128D 特征 / Return 128D features for single image 32 # Input: path_img <class 'str'> 33 # Output: face_descriptor <class 'dlib.vector'> 34 def return_128d_features(path_img): 35 img_rd = cv2.imread(path_img) 36 faces = detector(img_rd, 1) 37 38 logging.info("%-40s %-20s", "检测到人脸的图像 / Image with faces detected:", path_img) 39 40 # 因为有可能截下来的人脸再去检测，检测不出来人脸了, 所以要确保是 检测到人脸的人脸图像拿去算特征 41 # For photos of faces saved, we need to make sure that we can detect faces from the cropped images 42 if len(faces) != 0: 43 shape = predictor(img_rd, faces[0]) 44 face_descriptor = face_reco_model.compute_face_descriptor(img_rd, shape) 45 else: 46 face_descriptor = 0 47 logging.warning("no face") 48 return face_descriptor 49 50 51 # 返回 personX 的 128D 特征均值 / Return the mean value of 128D face descriptor for person X 52 # Input: path_face_personX <class 'str'> 53 # Output: features_mean_personX <class 'numpy.ndarray'> 54 def return_features_mean_personX(path_face_personX): 55 features_list_personX = [] 56 photos_list = os.listdir(path_face_personX) 57 if photos_list: 58 for i in range(len(photos_list)): 59 # 调用 return_128d_features() 得到 128D 特征 / Get 128D features for single image of personX 60 logging.info("%-40s %-20s", "正在读的人脸图像 / Reading image:", path_face_personX + "/" + photos_list[i]) 61 features_128d = return_128d_features(path_face_personX + "/" + photos_list[i]) 62 # 遇到没有检测出人脸的图片跳过 / Jump if no face detected from image 63 if features_128d == 0: 64 i += 1 65 else: 66 features_list_personX.append(features_128d) 67 else: 68 logging.warning("文件夹内图像文件为空 / Warning: No images in%s/", path_face_personX) 69 70 # 计算 128D 特征的均值 / Compute the mean 71 # personX 的 N 张图像 x 128D -> 1 x 128D 72 if features_list_personX: 73 features_mean_personX = np.array(features_list_personX, dtype=object).mean(axis=0) 74 else: 75 features_mean_personX = np.zeros(128, dtype=int, order='C') 76 return features_mean_personX 77 78 79 def main(): 80 logging.basicConfig(level=logging.INFO) 81 # 获取已录入的最后一个人脸序号 / Get the order of latest person 82 person_list = os.listdir("data/data_faces_from_camera/") 83 person_list.sort() 84 85 with open("data/features_all.csv", "w", newline="") as csvfile: 86 writer = csv.writer(csvfile) 87 for person in person_list: 88 # Get the mean/average features of face/personX, it will be a list with a length of 128D 89 logging.info("%sperson_%s", path_images_from_camera, person) 90 features_mean_personX = return_features_mean_personX(path_images_from_camera + person) 91 92 if len(person.split('_', 2)) == 2: 93 # "person_x" 94 person_name = person 95 else: 96 # "person_x_tom" 97 person_name = person.split('_', 2)[-1] 98 features_mean_personX = np.insert(features_mean_personX, 0, person_name, axis=0) 99 # features_mean_personX will be 129D, person name + 128 features 100 writer.writerow(features_mean_personX) 101 logging.info('\n') 102 logging.info("所有录入人脸数据存入 / Save all the features of faces registered into: data/features_all.csv") 103 104 105 if __name__ == '__main__': 106 main()
我们可以看下对于某张图片，face_descriptor 这个 128D vectors 的输出结果：
绿色框内是我们的返回 128D 特征的函数；
在红色框内调用该函数来计算 img_face_13.jpg；
可以看到黄色框中的输出为 128D 的向量；
图 6 返回单张图像的 128D 特征的计算结果
之后就需要人脸图像进行批量化操作，提取出 128D 的特征，然后计算特征均值，存入 features_all.csv；
features_all.csv 是一个 n 行 129 列的 CSV， n 是录入的人脸数，129 列是某人的名字加上 128D 特征（如果没有设置名字，那么就是 person_1, person_2 之类）；
这存储的就是 录入的人脸特征数据，之后 摄像头捕获的人脸 将要拿过来和 这些特征值 进行比对，如果欧式距离比较近的话，就可以认为是同一张人脸；
get_features_into_CSV.py 的输出 log：
##### person_1 #####
data/data_csvs_from_camera/person_1.csv
正在读的人脸图像 / image to read: data/data_faces_from_camera/person_1/img_face_1.jpg
检测到人脸的图像 / image with faces detected: data/data_faces_from_camera/person_1/img_face_1.jpg
正在读的人脸图像 / image to read: data/data_faces_from_camera/person_1/img_face_2.jpg
检测到人脸的图像 / image with faces detected: data/data_faces_from_camera/person_1/img_face_2.jpg
正在读的人脸图像 / image to read: data/data_faces_from_camera/person_1/img_face_3.jpg
检测到人脸的图像 / image with faces detected: data/data_faces_from_camera/person_1/img_face_3.jpg
正在读的人脸图像 / image to read: data/data_faces_from_camera/person_1/img_face_4.jpg
检测到人脸的图像 / image with faces detected: data/data_faces_from_camera/person_1/img_face_4.jpg
##### person_2 #####
data/data_csvs_from_camera/person_2.csv
正在读的人脸图像 / image to read: data/data_faces_from_camera/person_2/img_face_1.jpg
检测到人脸的图像 / image with faces detected: data/data_faces_from_camera/person_2/img_face_1.jpg
正在读的人脸图像 / image to read: data/data_faces_from_camera/person_2/img_face_2.jpg
检测到人脸的图像 / image with faces detected: data/data_faces_from_camera/person_2/img_face_2.jpg
##### person_3 #####
data/data_csvs_from_camera/person_3.csv
正在读的人脸图像 / image to read: data/data_faces_from_camera/person_3/img_face_1.jpg
检测到人脸的图像 / image with faces detected: data/data_faces_from_camera/person_3/img_face_1.jpg
正在读的人脸图像 / image to read: data/data_faces_from_camera/person_3/img_face_2.jpg
检测到人脸的图像 / image with faces detected: data/data_faces_from_camera/person_3/img_face_2.jpg
...
2.3 face_reco_from_camera.py / 实时人脸识别对比分析
这部分源码实现的功能：调用摄像头，捕获摄像头中的人脸，然后如果检测到人脸，将 摄像头中的人脸提取出 128D 的特征，然后和 之前录入人脸的 128D 特征 进行计算欧式距离，如果比较小，可以判定为一个人，否则不是一个人；
图 7 face_reco_from_camera.py 实现逻辑
所以设计的伪代码如下：
# 人脸检测器/预测器/识别模型
detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor('data/data_dlib/shape_predictor_68_face_landmarks.dat')
facerec = dlib.face_recognition_model_v1("data/data_dlib/dlib_face_recognition_resnet_model_v1.dat")
faces = detector(img_gray, 0)
# 1. 如果检测到人脸
if len(faces) != 0:
# 遍历所有检测到的人脸
for i in range(len(faces)):
# 2. 提取当前帧人脸的特征描述子
shape = predictor(img_rd, faces[i])
facerec.compute_face_descriptor(img_rd, shape)
# 3. 将当前帧人脸特征描述子和数据库的特征描述子进行对比
for i in range(len(self.features_known_list)):
e_distance_tmp = self.return_euclidean_distance(self.features_camera_list[k], self.features_known_list[i])
关于 https://github.com/coneypo/Dlib_face_recognition_from_camera/blob/master/face_reco_from_camera.py 里面变量的定义：
| 变量 | 说明 |
| self.feature_known_list | 存储所有录入人脸特征的数组 / Save the features of faces in the database |
| self.name_known_list | 存储已录入人脸的名字 / Save the names of faces in the database |
| self.current_frame_face_cnt | 存储当前摄像头中捕获到的人脸数 / Counter for faces in current frame |
| self.current_frame_feature_list | 存储当前摄像头中捕获到的人脸特征 / Features of faces in current frame |
| self.current_frame_name_position_list | 存储当前摄像头中捕获到的所有人脸的名字坐标 / Positions of faces in current frame |
| self.current_frame_name_list | 存储当前摄像头中捕获到的所有人脸的名字 / Names of faces in current frame |
关于用到的 dlib 检测器，预测器，识别器:
1. dlib.get_frontal_face_detector
Link：http://dlib.net/python/index.html#dlib.get_frontal_face_detector
介绍：返回默认的人脸检测器，为下面的 fhog_object_detectorm / Returns the default face detector
2. class dlib.fhog_object_detector
Link：http://dlib.net/python/index.html#dlib.fhog_object_detector
介绍 :基于滑动窗的HOG进行目标检测, This object represents a sliding window histogram-of-oriented-gradients based object detector.
参数:
__call__(self: dlib.fhog_object_detector, image: array, upsample_num_times: int=0L) → dlib.rectangles
3. class dlib.shape_predictor
Link:
http://dlib.net/python/index.html#dlib.shape_predictor
简介:
人脸图像作为输入, 输出面部特征点;
This object is a tool that takes in an image region containing some object and outputs a set of point locations that define the pose of the object.
The classic example of this is human face pose prediction, where you take an image of a human face as input and are expected to identify
the locations of important facial landmarks such as the corners of the mouth and eyes, tip of the nose, and so forth.
参数 / parameters:
__call__(self: dlib.shape_predictor, image: array, box: dlib.rectangle) → dlib.full_object_detection
输入: dlib.rectangle 输出: dlib.full_object_detection
4. class dlib.face_recognition_model_v1
Link:
http://dlib.net/python/index.html#dlib.face_recognition_model_v1
简介 / intro:
将人脸转换为128D特征向量, 这样的话相似人脸会比较相近, 不相像的会比较远;
This object maps human faces into 128D vectors where pictures of the same person are mapped near to each other and pictures of different people are mapped far apart.
The constructor loads the face recognition model from a file. The model file is available here: http://dlib.net/files/dlib_face_recognition_resnet_model_v1.dat.bz2
参数 / parameters:
compute_face_descriptor(self: dlib.face_recognition_model_v1, img: numpy.ndarray[(rows,cols,3),uint8], face: dlib.full_object_detection, num_jitters: int=0L, padding: float=0.25) -> dlib.vector
通过 print(type()) 可以更清楚的看到 dlib 对象的传递:
# 人脸检测器/预测器/识别模型
detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor('data/data_dlib/shape_predictor_68_face_landmarks.dat')
facerec = dlib.face_recognition_model_v1("data/data_dlib/dlib_face_recognition_resnet_model_v1.dat")
faces = detector(img_gray, 0)
# 如果检测到人脸
if len(faces) != 0:
print(type(faces) # <class 'dlib.dlib.rectangles'>
# 遍历所有检测到的人脸
for i in range(len(faces)):
# 进行人脸比对
shape = predictor(img_rd, faces[i])
print(type(shape)) # <class 'dlib.dlib.full_object_detection'>
facerec.compute_face_descriptor(img_rd, shape)
print(type(facerec.compute_face_descriptor(img_rd, shape)) # <class 'dlib.dlib.vector'>
这样一个对象传递过程：
faces = detector(img_gray, 0) -> <class 'dlib.dlib.rectangles'> ->
shape = predictor(img_rd, faces[i]) -> <class 'dlib.dlib.full_object_detection'> ->
facerec.compute_face_descriptor(img_rd, shape) -> <class 'dlib.dlib.vector'>
欧氏距离对比的阈值设定，是在 return_euclidean_distance 函数的 dist 变量；
我这里程序里面指定的 欧氏距离判断阈值是 0.4，具体阈值可以根据实际情况或者测得结果进行修改；
这边做了一个，让人名跟随显示在头像下方，如果想要在人脸矩形框下方显示人名，首先需要知道 Dlib 生成的矩形框的尺寸怎么读取；
Dlib 返回的 dets 变量是一系列人脸的数据，此处对单张人脸处理，所以取 dets[0] 的参数；
可以通过 dets[0].top(), dets[0].bottom(), dets[0].left() 和 dets[0].right() 来确定要显示的人名的坐标；
图 8 dets[0].top() 等参数说明
得到矩形框的坐标，就可以获取人名的相对位置；
这是我这边取的坐标：
pos_text_1 = tuple([dets[0].left(), int(dets[0].bottom()+(dets[0].bottom()-dets[0].top())/4)])
图 9 face_reco_from_camera.py 生成的人脸识别窗口界面
如果想定制输出显示的名字而不是“Person_1”,"Person_2"...；
图 9 定制显示名字
1 # Copyright (C) 2020 coneypo
2 # SPDX-License-Identifier: MIT
3
4 # Author: coneypo
5 # Blog: http://www.cnblogs.com/AdaminXie
6 # GitHub: https://github.com/coneypo/Dlib_face_recognition_from_camera
7 # Mail: coneypo@foxmail.com
8
9 # 摄像头实时人脸识别 / Real-time face detection and recognition
10
11 import dlib
12 import numpy as np
13 import cv2
14 import pandas as pd
15 import os
16 import time
17 from PIL import Image, ImageDraw, ImageFont
18
19 # Dlib 正向人脸检测器 / Use frontal face detector of Dlib
20 detector = dlib.get_frontal_face_detector()
21
22 # Dlib 人脸 landmark 特征点检测器 / Get face landmarks
23 predictor = dlib.shape_predictor('data/data_dlib/shape_predictor_68_face_landmarks.dat')
24
25 # Dlib Resnet 人脸识别模型，提取 128D 的特征矢量 / Use Dlib resnet50 model to get 128D face descriptor
26 face_reco_model = dlib.face_recognition_model_v1("data/data_dlib/dlib_face_recognition_resnet_model_v1.dat")
27
28
29 class Face_Recognizer:
30 def __init__(self):
31 self.feature_known_list = [] # 用来存放所有录入人脸特征的数组 / Save the features of faces in the database
32 self.name_known_list = [] # 存储录入人脸名字 / Save the name of faces in the database
33
34 self.current_frame_face_cnt = 0 # 存储当前摄像头中捕获到的人脸数 / Counter for faces in current frame
35 self.current_frame_feature_list = [] # 存储当前摄像头中捕获到的人脸特征 / Features of faces in current frame
36 self.current_frame_name_position_list = [] # 存储当前摄像头中捕获到的所有人脸的名字坐标 / Positions of faces in current frame
37 self.current_frame_name_list = [] # 存储当前摄像头中捕获到的所有人脸的名字 / Names of faces in current frame
38
39 # Update FPS
40 self.fps = 0
41 self.frame_start_time = 0
42
43 # 从 "features_all.csv" 读取录入人脸特征 / Get known faces from "features_all.csv"
44 def get_face_database(self):
45 if os.path.exists("data/features_all.csv"):
46 path_features_known_csv = "data/features_all.csv"
47 csv_rd = pd.read_csv(path_features_known_csv, header=None)
48 for i in range(csv_rd.shape[0]):
49 features_someone_arr = []
50 for j in range(0, 128):
51 if csv_rd.iloc[i][j] == '':
52 features_someone_arr.append('0')
53 else:
54 features_someone_arr.append(csv_rd.iloc[i][j])
55 self.feature_known_list.append(features_someone_arr)
56 self.name_known_list.append("Person_"+str(i+1))
57 print("Faces in Database：", len(self.feature_known_list))
58 return 1
59 else:
60 print('##### Warning #####', '\n')
61 print("'features_all.csv' not found!")
62 print(
63 "Please run 'get_faces_from_camera.py' and 'features_extraction_to_csv.py' before 'face_reco_from_camera.py'",
64 '\n')
65 print('##### End Warning #####')
66 return 0
67
68 # 计算两个128D向量间的欧式距离 / Compute the e-distance between two 128D features
69 @staticmethod
70 def return_euclidean_distance(feature_1, feature_2):
71 feature_1 = np.array(feature_1)
72 feature_2 = np.array(feature_2)
73 dist = np.sqrt(np.sum(np.square(feature_1 - feature_2)))
74 return dist
75
76 # 更新 FPS / Update FPS of Video stream
77 def update_fps(self):
78 now = time.time()
79 self.frame_time = now - self.frame_start_time
80 self.fps = 1.0 / self.frame_time
81 self.frame_start_time = now
82
83 def draw_note(self, img_rd):
84 font = cv2.FONT_ITALIC
85
86 cv2.putText(img_rd, "Face Recognizer", (20, 40), font, 1, (255, 255, 255), 1, cv2.LINE_AA)
87 cv2.putText(img_rd, "FPS: " + str(self.fps.__round__(2)), (20, 100), font, 0.8, (0, 255, 0), 1, cv2.LINE_AA)
88 cv2.putText(img_rd, "Faces: " + str(self.current_frame_face_cnt), (20, 140), font, 0.8, (0, 255, 0), 1, cv2.LINE_AA)
89 cv2.putText(img_rd, "Q: Quit", (20, 450), font, 0.8, (255, 255, 255), 1, cv2.LINE_AA)
90
91 def draw_name(self, img_rd):
92 # 在人脸框下面写人脸名字 / Write names under rectangle
93 font = ImageFont.truetype("simsun.ttc", 30)
94 img = Image.fromarray(cv2.cvtColor(img_rd, cv2.COLOR_BGR2RGB))
95 draw = ImageDraw.Draw(img)
96 for i in range(self.current_frame_face_cnt):
97 # cv2.putText(img_rd, self.current_frame_name_list[i], self.current_frame_name_position_list[i], font, 0.8, (0, 255, 255), 1, cv2.LINE_AA)
98 draw.text(xy=self.current_frame_name_position_list[i], text=self.current_frame_name_list[i], font=font)
99 img_with_name = cv2.cvtColor(np.array(img), cv2.COLOR_RGB2BGR)
100 return img_with_name
101
102 # 修改显示人名 / Show names in chinese
103 def show_chinese_name(self):
104 # Default known name: person_1, person_2, person_3
105 if self.current_frame_face_cnt >= 1:
106 self.name_known_list[0] ='张三'.encode('utf-8').decode()
107 # self.name_known_list[1] ='李四'.encode('utf-8').decode()
108 # self.name_known_list[2] ='xx'.encode('utf-8').decode()
109 # self.name_known_list[3] ='xx'.encode('utf-8').decode()
110 # self.name_known_list[4] ='xx'.encode('utf-8').decode()
111
112 # 处理获取的视频流，进行人脸识别 / Face detection and recognition from input video stream
113 def process(self, stream):
114 # 1. 读取存放所有人脸特征的 csv / Get faces known from "features.all.csv"
115 if self.get_face_database():
116 while stream.isOpened():
117 print(">>> Frame start")
118 flag, img_rd = stream.read()
119 faces = detector(img_rd, 0)
120 kk = cv2.waitKey(1)
121 # 按下 q 键退出 / Press 'q' to quit
122 if kk == ord('q'):
123 break
124 else:
125 self.draw_note(img_rd)
126 self.current_frame_feature_list = []
127 self.current_frame_face_cnt = 0
128 self.current_frame_name_position_list = []
129 self.current_frame_name_list = []
130
131 # 2. 检测到人脸 / Face detected in current frame
132 if len(faces) != 0:
133 # 3. 获取当前捕获到的图像的所有人脸的特征 / Compute the face descriptors for faces in current frame
134 for i in range(len(faces)):
135 shape = predictor(img_rd, faces[i])
136 self.current_frame_feature_list.append(face_reco_model.compute_face_descriptor(img_rd, shape))
137 # 4. 遍历捕获到的图像中所有的人脸 / Traversal all the faces in the database
138 for k in range(len(faces)):
139 print(">>>>>> For face", k+1, " in camera")
140 # 先默认所有人不认识，是 unknown / Set the default names of faces with "unknown"
141 self.current_frame_name_list.append("unknown")
142
143 # 每个捕获人脸的名字坐标 / Positions of faces captured
144 self.current_frame_name_position_list.append(tuple(
145 [faces[k].left(), int(faces[k].bottom() + (faces[k].bottom() - faces[k].top()) / 4)]))
146
147 # 5. 对于某张人脸，遍历所有存储的人脸特征
148 # For every faces detected, compare the faces in the database
149 current_frame_e_distance_list = []
150 for i in range(len(self.feature_known_list)):
151 # 如果 person_X 数据不为空
152 if str(self.feature_known_list[i][0]) != '0.0':
153 print(" >>> With person", str(i + 1), ", the e distance: ", end='')
154 e_distance_tmp = self.return_euclidean_distance(self.current_frame_feature_list[k],
155 self.feature_known_list[i])
156 print(e_distance_tmp)
157 current_frame_e_distance_list.append(e_distance_tmp)
158 else:
159 # 空数据 person_X
160 current_frame_e_distance_list.append(999999999)
161 # 6. 寻找出最小的欧式距离匹配 / Find the one with minimum e distance
162 similar_person_num = current_frame_e_distance_list.index(min(current_frame_e_distance_list))
163 print(" >>> Minimum e distance with ", self.name_known_list[similar_person_num], ": ", min(current_frame_e_distance_list))
164
165 if min(current_frame_e_distance_list) < 0.4:
166 self.current_frame_name_list[k] = self.name_known_list[similar_person_num]
167 print(" >>> Face recognition result: " + str(self.name_known_list[similar_person_num]))
168 else:
169 print(" >>> Face recognition result: Unknown person")
170
171 # 矩形框 / Draw rectangle
172 for kk, d in enumerate(faces):
173 # 绘制矩形框
174 cv2.rectangle(img_rd, tuple([d.left(), d.top()]), tuple([d.right(), d.bottom()]),
175 (0, 255, 255), 2)
176
177 self.current_frame_face_cnt = len(faces)
178
179 # 7. 在这里更改显示的人名 / Modify name if needed
180 # self.show_chinese_name()
181
182 # 8. 写名字 / Draw name
183 img_with_name = self.draw_name(img_rd)
184
185 else:
186 img_with_name = img_rd
187
188 print(">>>>>> Faces in camera now:", self.current_frame_name_list)
189
190 cv2.imshow("camera", img_with_name)
191
192 # 9. 更新 FPS / Update stream FPS
193 self.update_fps()
194 print(">>> Frame ends\n\n")
195
196 # OpenCV 调用摄像头并进行 process
197 def run(self):
198 cap = cv2.VideoCapture(0)
199 # cap = cv2.VideoCapture("video.mp4")
200 cap.set(3, 480) # 640x480
201 self.process(cap)
202
203 cap.release()
204 cv2.destroyAllWindows()
205
206
207 def main():
208 Face_Recognizer_con = Face_Recognizer()
209 Face_Recognizer_con.run()
210
211
212 if __name__ == '__main__':
213 main()
face_reco_from_camera.py 输出 log：
Faces in Database： 3
>>> Frame start
>>>>>> For face 1 in camera
>>> With person 2 , the e distance: 0.24747225595381367
>>> With person 3 , the e distance: 0.22821104803792178
>>> Minimum e distance with Person_3 : 0.22821104803792178
>>> Face recognition result: Person_3
>>>>>> Faces in camera now: ['Person_3']
>>> Frame ends
>>> Frame start
>>>>>> For face 1 in camera
>>> With person 2 , the e distance: 0.2490812900317618
>>> With person 3 , the e distance: 0.22549497689337802
>>> Minimum e distance with Person_3 : 0.22549497689337802
>>> Face recognition result: Person_3
>>>>>> Faces in camera now: ['Person_3']
>>> Frame ends
>>> Frame start
>>>>>> For face 1 in camera
>>> With person 2 , the e distance: 0.24569769385882426
>>> With person 3 , the e distance: 0.2262102554355137
>>> Minimum e distance with Person_3 : 0.2262102554355137
>>> Face recognition result: Person_3
>>>>>> Faces in camera now: ['Person_3']
>>> Frame ends
>>> Frame start
>>>>>> For face 1 in camera
>>> With person 2 , the e distance: 0.24387949251367172
>>> With person 3 , the e distance: 0.22636200199905795
>>> Minimum e distance with Person_3 : 0.22636200199905795
>>> Face recognition result: Person_3
>>>>>> Faces in camera now: ['Person_3']
>>> Frame ends
>>> Frame start
>>>>>> For face 1 in camera
>>> With person 2 , the e distance: 0.2473446948271673
>>> With person 3 , the e distance: 0.22534075942468246
>>> Minimum e distance with Person_3 : 0.22534075942468246
>>> Face recognition result: Person_3
>>>>>> Faces in camera now: ['Person_3']
>>> Frame ends
>>> Frame start
>>>>>> For face 1 in camera
>>> With person 2 , the e distance: 0.24465000646050672
>>> With person 3 , the e distance: 0.2238005841538998
>>> Minimum e distance with Person_3 : 0.2238005841538998
>>> Face recognition result: Person_3
>>>>>> Faces in camera now: ['Person_3']
>>> Frame ends
如果对单个人脸，进行实时对比输出：
图 10 实时输出的欧氏距离结果
通过实时的输出结果，看的比较明显；
输出绿色部分：当是我自己时，计算出来的欧式距离基本都在 0.2 左右；
输出红色部分：而换一张图片上去比如特朗普，明显看到欧式距离计算结果 达到了 0.8，此时就可以判定，后来这张人脸不是一张人脸；
所以之前提到的欧式距离计算对比的阈值可以由此设定，本项目中取的是 dist=0.4；
dist 的确切取值自己权衡，http://dlib.net/face_recognition.py.html 的说明：
# When using a distance threshold of 0.6, the dlib model obtains an accuracy
# of 99.38% on the standard LFW face recognition benchmark, which is
# comparable to other state-of-the-art methods for face recognition as of
# February 2017. This accuracy means that, when presented with a pair of face
# images, the tool will correctly identify if the pair belongs to the same
# person or is from different people 99.38% of the time.
可以通过 logging.basicConfig(level=logging.DEBUG) 来设置代码的打印；
3. 总结
核心就是 提取人脸特征，然后计算欧式距离和预设的特征脸进行比对；
不过这个实时获取摄像头人脸进行比对，要实时的进行计算摄像头脸的特征值，然后还要计算欧氏距离，所以计算量比较大，可能摄像头视频流会出现卡顿；
此项目仅个人学习爱好研究，开源供大家一起学习；
# 请尊重他人劳动成果，转载或者使用源码请注明出处：http://www.cnblogs.com/AdaminXie
# 代码已上传到了我的 GitHub，如果对您有帮助欢迎 Star 支持我下：https://github.com/coneypo/Dlib_face_recognition_from_camera
# 如有问题请留言或者联系邮箱: coneypo@foxmail.com
