[AI] 深度视觉 - CV

前言

---

一、故事背景

本篇的初衷是为了初步整理计算机视觉相关的笔记，帮助个人理清思路。

当然了，计算机视觉是个及其庞大且随性的范畴，只能用到什么学什么，项目驱动为好。

本人目前致力于完善个人的AR知识体系、实践体系，聊及此，不经感叹 Real AR 门槛之变态高。

计算机视觉、机器学习、三维模型、手机软件开发（iphone, android or react native）、Serverless，

如果可以，还可以再算上一些附加的聊天系统、语音识别系统。

AR是一项综合多种技术于一身的产品，而不是技术。

目前来看，对该“理解”深度的忽视导致还未有一款真正获得市场好评的AR产品出现。

  

二、寻迹漫聊

Tracking

[Object Tracking] Overview of Object Tracking

[Object Tracking] Overview of algorithms for Object Tracking

Deep Learning

[Deep Learning] How deep is the Deep Learning - History

[Deep Learning] How deep is the Deep Learning - Revolution

[Deep Learning] How deep is the Deep Learning - Reinforcement Learning

[Object Tracking] Deep Boundary detection Tech

Human Recognition 

[CNN] Face Detection

[Face] Outline - Key features detection of human face

[CV Timeline] Applications of Face recognition and PoseNet

[CV] Detect face with mask

3D SLAM

[SLAM] Little about SLAM

Others: 

[CV] Collection of Computer Vision

 

三、新鲜韭菜

MeidaPipe

Real-Time 3D Object Detection on Mobile Devices with MediaPipe, Wednesday, March 11, 2020

在2019年6月举行的CVPR大会，MeidaPipe正式开源

大宝库

 样例代码大全

 

 

  

基础知识

---

一、基本数据结构

• 目的：了解 OpenCV 对 "几何图像&图片" 的描述方式

[OpenCV] Basic data types - Matrix【Mat有数学优化, IplImage, cvMat三种结构】

[OpenCV] IplImage and Operation【 IplImage为基础的图像操作】

[OpenCV] HighGUI【*** 与操作系统，文件系统，摄像机之类的硬件进行交互的一些函数纳入HighGui】

 

 

二、图像处理知识

• 预处理

[OpenCV] Image Processing - Image Elementary Knowledge【一些数学基本原理】

[OpenCV] Image Processing - Grayscale Transform & Histogram【利用“直方图"预处理图像】

 

• 频域、空间域滤波

[OpenCV] Image Processing - Frequency Domain Filtering【傅里叶】

[OpenCV] Image Processing - Spatial Filtering【算子】

[OpenCV] Image Processing - Fuzzy Set【模糊集，暂时忽略】

 

• 图像去噪

[OpenCV] Image Denoising

[Scikit-learn] Dynamic Bayesian Network - Conditional Random Field【条件随机场、模拟退火】

 

 

三、特征提取与匹配 

• 提取＆匹配

[OpenCV] Feature Extraction【提取特征】

[OpenCV] Feature Matching【特征匹配】

 

• 目标识别实践

[Object Tracking] Identify and Track Specific Object【特征提取与匹配的项目实践】

 

 

四、Sample 示范

• 数据结构

[OpenCV] Samples 02: Mat - 图像矩阵【Mat计算能力】

[OpenCV] Samples 01: Geometry - 几何图形【几何图案、文字等】

[OpenCV] Samples 07: create_mask【鼠标圈图】

 

• 工程技巧

[OpenCV] Samples 13: opencv_version【版本信息显示】

[OpenCV] Samples 10: imagelist_creator【图片地址list参数】

[OpenCV] Samples 18: Load image and check its attributes【图片属性】

 

• 色彩通道

[OpenCV] Samples 09: plImage <==> Mat【色域通道分离】

[OpenCV] Samples 16: Decompose and Analyse RGB channels【色域通道分离】

 

• 边缘操作

[OpenCV] Samples 08: edge【边缘检测】

[OpenCV] Samples 04: contours2【二值图案找轮廓】

[OpenCV] Samples 05: convexhull【散点的凸包轮廓】

 

• 模型算法

[OpenCV] Samples 03: kmeans【聚类算法】

[OpenCV] Samples 17: Floodfill【聚类算法】

[OpenCV] Samples 06: logistic regression【线性二分类】

[OpenCV] Samples 14: kalman filter【预测下一个状态】

[OpenCV] Samples 15: Background Subtraction and Gaussian mixture models【背景差分】

 

• 视频处理

[OpenCV] Samples 11: image sequence【视频流提取】

[OpenCV] Samples 12: laplace【视频流处理】

 

 

五、视觉对比

 [CV] Scene contrast

 

 

六、三维视觉

• 几何变换

[Link] 平面的投影变换(2)——有几种几何变换？【此人的其他相关文章也不错】

[Link] Photo Wake-Up: 3D Character Animation from a Single Photo【非常有潜力的技术】

 

七、目标跟踪

• 有姿态

[Object Tracking] Identify and Track Specific Object

[Object Tracking] LK & Deep-LK

 

• 无姿态

[Object Tracking] MeanShift

 

八、API剪裁

[OpenCV] Prune and insert APIs 

 

 

 

AR on Mobilephone

---

一、调研

• ViroMedia

// ViroReact: AR and VR using React Native

// Viro是个好东西，但不一定需要

[RN] 01, Init AR

[RN] 02, Start Viro

 

• ARCore

// 重点在定位系统

[ARCore] 00 - The First Demo

[ARCore] 01 - Plane Detection

[ARCore] 02 - Solar System

 

• Sumerian

// Host非常好

[Sumerian] 00 - It provides 3D Character Model

[Sumerian] 01 - Build a room

[Sumerian] 02 - Build a host

[Sumerian] 03 - Build a AR

 

 

二、实战

/* implement above. */ 

 

 

三、模型制作

[handbyhand] Web AR【概览】

 

 

 

Action Recognition

---

一、基础知识

[action] A Comprehensive Study of Deep Video Action Recognition

[action] Action Recognition by Skeleton

[action] MMLab: Slow-Fast Model

[action] PoseC3D

 

 

 

视觉设计

---

一、Sketch 

Sure, designed in Sketch and animated using the Element 3D plugin by Video Copilot inside After Effects.

By Sacha Jerrems

 

 

 

Kaggle 数据集

---

Rainforest Connection Species Audio Detection

Cassava Leaf Disease Classification

The Nature Conservancy Fisheries Monitoring

 

 

 

 

Talking Face Based on LLM

---

前言

一些过去的资料集合。我们未来的目标是 text --> talking face。

 

• 起初的"虚拟主播"的脸蛋儿照片生成

Artbreeder

StyleGAN3 参见：[AI] 深度网络 - DNN

 

• Voice --> Talking Face

[Submitted on 22 Sep 2021 (v1), last revised 24 Sep 2021 (this version, v2)]

Paper: Live Speech Portraits: Real-Time Photorealistic Talking-Head Animation

Code: https://github.com/YuanxunLu/LiveSpeechPortraits

We are really grateful to Andreas from Replicate for his amazing job in making the web demo! Now you can run the Demo on the browser.

 

• 更多最新的案例参见

[Avatar] Avatar by ImgGen