Halcon的Blob分析

一、Blob分析流程及常用算子

1.基本步骤

Blob分析的基本步骤，这是一种理想状态，也是最基本的套路，获取图像->分割图像（区分前景像素和背景像素）->特征提取（比如面积、重心、旋转角度等）。

halcon代码实现如下：
read_image(Image,‘particle’)
threshold(Image, BrightPixels,120,255)//阈值分割算子
connection(BrightPixels,Particles)//断开联通区域
area_center(Particles,Area,Row,Column)

实际上，提取Blob之前和分析Blob之后也存在重要的步骤。比如，提取Blob之前一般要对图像进行预处理，比如图像的变换和校正、平滑与去噪、以及增强处理；分析Blob之后需要将Blob进行选取，或者将Blob重心的像素值向物理坐标系坐标值的转化。因此，Blob实现方法需要具体情况具体分析。就拿阈值分割提取Blob而言，是用固定阈值还是动态阈值，这都是根据图片情况进行具体分析的。

2.完整的Blob分析步骤

在实际应用中，Blob 的分割会很复杂，需要处理更多步骤。 其原因有多种，比如杂乱或不均匀的照明、图像中有很多杂斑（很难提取目标位）。 此外，对Blob进行后处理以获取客户需要的直观显示数据，例如将特征转换为真实世界单位或结果可视化。

整个过程为：获取图像->应用ROI->定位ROI->矫正图像->图像预处理->动态获取分割参数->分割图像->处理区域->特征提取->将像素坐标转换到世界坐标->结果显示或者输出。

* ball.hdev: Inspection of Ball Bonding
* 
dev_update_window ('off')
dev_close_window ()
dev_open_window (0, 0, 728, 512, 'black', WindowID)
read_image (Bond, 'die/die_03')
dev_display (Bond)
set_display_font (WindowID, 14, 'mono', 'true', 'false')
disp_continue_message (WindowID, 'black', 'true')
stop ()

*灰度处理 二值化
threshold (Bond, Bright, 100, 255)
*形态学处理，一般用于定位，形状转换，rectangle2带方向的矩形
shape_trans (Bright, Die, 'rectangle2')
dev_set_color ('green')
dev_set_line_width (3)
dev_set_draw ('margin')
dev_display (Die)
disp_continue_message (WindowID, 'black', 'true')
stop ()
*区域锁定
reduce_domain (Bond, Die, DieGrey)

*再次二值化
threshold (DieGrey, Wires, 0, 50)
*用给定的形状特征填充区域：面积为1~100的区域
fill_up_shape (Wires, WiresFilled, 'area', 1, 100)
dev_display (Bond)
dev_set_draw ('fill')
dev_set_color ('red')
dev_display (WiresFilled)
disp_continue_message (WindowID, 'black', 'true')
stop ()

*形态学处理：此处为开运算/（也可以用腐蚀）,减少像素，circle–对圆形作用最大
*形态学 腐蚀
*erosion_circle (WiresFilled, RegionErosion, 15.5)
*形态学 膨胀
*dilation_circle (RegionErosion, RegionDilation, 15.5)
*形态学 开运算（效果同上 先腐蚀后膨胀）
opening_circle (WiresFilled, Balls, 15.5)
dev_set_color ('green')
dev_display (Balls)
disp_continue_message (WindowID, 'black', 'true')
stop ()
*将二值化分开的区域划分为不同的连通区域，断成不同的区域，方便后面作特征值提取处理
connection (Balls, SingleBalls)

*特征值提取
*根据圆度提取区域
* select_shape (SingleBalls, IntermediateBalls, 'circularity', 'and', 0.85, 1.0)
*根据面积提取区域
select_shape (SingleBalls, IntermediateBalls, 'area', 'and', 800, 1200)
*按照坐标进行排序
sort_region (IntermediateBalls, FinalBalls, 'first_point', 'true', 'column')
dev_display (Bond)
dev_set_colored (12)
dev_display (FinalBalls)
disp_continue_message (WindowID, 'black', 'true')
stop ()

*计算最小外接圆半径
smallest_circle (FinalBalls, Row, Column, Radius)
NumBalls := |Radius|
Diameter := 2 * Radius
meanDiameter := mean(Diameter)
minDiameter := min(Diameter)
dev_display (Bond)
disp_circle (WindowID, Row, Column, Radius)
dev_set_color ('white')
disp_message (WindowID, 'D: ' + Diameter$'.4', 'image', Row - 2 * Radius, Column, 'white', 'false')
dev_update_window ('on')

 

3.常见特征提取

1)、区域特征：
a：面积area；
b：力矩Moments；
c：平行于主轴的最小矩形smallest_rectangle1；
d：任意方向的最小矩形smallest_rectangle2；
e：最小圆形smallest_circle；
f：凸包面积convexity；
g：contlength区域边界长度；
h：圆形roundness；
j：圆度circularity；
k：紧密度compactness；
l：矩形度rectangularity；
2)、灰度特征
a：简单灰度值特征：区域的平均灰度值；
b：区域的最小和最大灰度值；

4.开运算和闭运算（形态学）

图像分割时，会用到开运算和闭运算，采用不同的分割策略，效果会不一样，这里仅先介绍下概念。
1)、开运算 先腐蚀后膨胀；
2)、闭运算 先膨胀后腐蚀；
3)、腐蚀；
4)、膨胀。
开运算和闭运算要用到的算子如下：
1）、开运算 opening(ConnectedRegions, ConnectedRegions, RegionOpening1)
2）、闭运算 closing(RegionOpening1, RegionOpening1, RegionClosing1)
3）、腐蚀 erosion1(RegionClosing1, RegionClosing1, RegionErosion1, 1)
4）、膨胀 dilation1(RegionErosion1, RegionErosion1, RegionDilation1, 1)
5）、圆形结构开运算 opening_circle(ConnectedRegions, RegionOpening, 3.5)
6）、圆形结构闭运算 closing_circle(RegionOpening, RegionClosing, 3.5)
7）、圆形结构腐蚀 erosion_circle(RegionClosing, RegionErosion, 3.5)
8）、圆形结构膨胀 dilation_circle(RegionErosion, RegionDilation, 3.5)

5.功能特征的取值：

求Region指定特征值：region_features(Regions : : Features : Value)

根据特征值选择区域：select_shape(Regions : SelectedRegions : Features, Operation, Min, Max : )

特征	英	译	备注
area	Area of the object	对象的面积
row	Row index of the center	中心点的行坐标
column	Column index of the center	中心点的列坐标
width	Width of the region	区域的宽度
height	Height of the region	区域的高度
row1	Row index of upper left corner	左上角行坐标
column1	Column index of upper left corner	左上角列坐标
row2	Row index of lower right corner	右下角行坐标
column2	Column index of lower right corner	右下角列坐标
circularity	Circularity	圆度	0~1
compactness	Compactness	紧密度	0~1
contlength	Total length of contour	轮廓线总长
convexity	Convexity	凸性
rectangularity	Rectangularity	矩形度	0~1
ra	Main radius of the equivalent ellipse	等效椭圆长轴半径长度
rb	Secondary radius of the equivalent ellipse	等效椭圆短轴半径长度
phi	Orientation of the equivalent ellipse	等效椭圆方向
anisometry	Anisometry	椭圆参数，Ra/Rb长轴与短轴的比值
bulkiness	Bulkiness	椭圆参数，蓬松度π*Ra*Rb/A
struct_factor	Structur Factor	椭圆参数，Anisometry*Bulkiness-1
outer_radius	Radius of smallest surrounding circle	最小外接圆半径
inner_radius	Radius of largest inner circle	最大内接圆半径
inner_width	Width of the largest axis-parallel rectangle that fits into the region	最大内接矩形宽度
inner_height	Height of the largest axis-parallel rectangle that fits into the region	最大内接矩形高度
dist_mean	Mean distance from the region border to the center	区域边界到中心的平均距离
dist_deviation	Deviation of the distance from the region border from the center	区域边界到中心距离的偏差
roundness	Roundness	圆度，与circularity计算方法不同
num_sides	Number of polygon sides	多边形边数
connect_num	Number of connection components	连通数
holes_num	Number of holes	区域内洞数
area_holes	Area of the holes of the object	所有洞的面积
max_diameter	Maximum diameter of the region	最大直径
orientation	Orientation of the region	区域方向
euler_number	Euler number	欧拉数，即连通数和洞数的差
rect2_phi	Orientation of the smallest surrounding rectangle	最小外接矩形的方向
rect2_len1	Half the length of the smallest surrounding rectangle	最小外接矩形长度的一半	smallest_rectangle2
rect2_len2	Half the width of the smallest surrounding rectangle	最小外接矩形宽度的一半
moments_m11	Geometric moments of the region	几何矩
moments_m20	Geometric moments of the region	几何矩
moments_m02	Geometric moments of the region	几何矩
moments_ia	Geometric moments of the region	几何矩
moments_ib	Geometric moments of the region	几何矩
moments_m11_invar	Geometric moments of the region	几何矩
moments_m20_invar	Geometric moments of the region	几何矩
moments_m02_invar	Geometric moments of the region	几何矩
moments_phi1	Geometric moments of the region	几何矩
moments_phi2	Geometric moments of the region	几何矩
moments_m21	Geometric moments of the region	几何矩
moments_m12	Geometric moments of the region	几何矩
moments_m03	Geometric moments of the region	几何矩
moments_m30	Geometric moments of the region	几何矩
moments_m21_invar	Geometric moments of the region	几何矩
moments_m12_invar	Geometric moments of the region	几何矩
moments_m03_invar	Geometric moments of the region	几何矩
moments_m30_invar	Geometric moments of the region	几何矩
moments_i1	Geometric moments of the region	几何矩
moments_i2	Geometric moments of the region	几何矩
moments_i3	Geometric moments of the region	几何矩
moments_i4	Geometric moments of the region	几何矩
moments_psi1	Geometric moments of the region	几何矩
moments_psi2	Geometric moments of the region	几何矩
moments_psi3	Geometric moments of the region	几何矩
moments_psi4	Geometric moments of the region	几何矩

二、Blob分析案例

1.豆子分割统计(形态学案例)

* 采集图像操作
dev_update_off ()
read_image (Image, 'pellets')
dev_close_window ()
get_image_size (Image, Width, Height)
dev_open_window (0, 0, Width, Height, 'black', WindowID)
dev_set_part (0, 0, Height - 1, Width - 1)
set_display_font (WindowID, 16, 'mono', 'true', 'false')
dev_set_colored (6)
dev_set_draw ('margin')
dev_set_line_width (3)
dev_display (Image)
disp_message (WindowID, 'Detect each single pellet', 'window', 12, 12, 'black', 'true')
disp_continue_message (WindowID, 'black', 'true')
stop ()

 

* 从背景分割颗粒的区域
* 二进制阈值分割 对dark(暗色) light(亮色)
binary_threshold (Image, LightRegion, 'max_separability', 'light', UsedThreshold)
opening_circle (LightRegion, Region, 2.5)
dev_display (Region)
disp_message (WindowID, 'First, segment the pellets', 'window', 12, 12, 'black', 'true')
disp_continue_message (WindowID, 'black', 'true')
stop ()

 

* 第一次分割出颗粒区域，只是分割处理而已不能进行统计
connection (Region, ConnectedRegionsWrong)
dev_display (Image)
dev_display (ConnectedRegionsWrong)
disp_message (WindowID, 'Simple connection fails', 'window', 12, 12, 'black', 'true')
disp_continue_message (WindowID, 'black', 'true')
stop ()

* 通过腐蚀的方式来分割出每个区域
erosion_circle (Region, RegionErosion, 7.5)
*自定义形状腐蚀
* shape_trans (RegionErosion, RegionTrans, 'convex')
* gen_rectangle1 (Rectangle, 1, 1, 20, 20)
* gen_circle (Circle, 10, 10, 10)
* erosion1 (Region, Circle, RegionErosion1, 1)
dev_display (Image)
dev_display (RegionErosion)
disp_message (WindowID, 'Erosion of the pellet regions', 'window', 12, 12, 'black', 'true')
disp_continue_message (WindowID, 'black', 'true')
stop ()

* 在次分割连通区域
connection (RegionErosion, ConnectedRegions)
dev_display (Image)
dev_display (ConnectedRegions)
disp_message (WindowID, 'Perform connection now', 'window', 12, 12, 'black', 'true')
disp_continue_message (WindowID, 'black', 'true')
stop ()

* 通过应用膨胀使其恢复到原始颗粒大小
dilation_circle (ConnectedRegions, RegionDilation, 7.5)
area_center (RegionDilation, Area, Row, Column)

* 统计颗粒数量
count_obj (RegionDilation, Number)
dev_display (Image)
dev_display (RegionDilation)
disp_message (WindowID, Number + ' pellets detected', 'window', 12, 12, 'black', 'true')
dev_set_color ('cyan')
disp_cross (WindowID, Row, Column, 16, rad(60))

 

2.车牌的字符分割(数字与字母)

***** 读取图片
dev_close_window ()
read_image (Image, '../chepai18.jpg')
dev_open_window_fit_image (Image, 0, 0, -1, -1, WindowHandle)
dev_display (Image)

 

***** 处理图片-定位车牌
rgb1_to_gray (Image, GrayImage)
threshold (GrayImage, Region, 80, 100)
connection (Region, ConnectedRegions)
select_shape (ConnectedRegions, SelectedRegions, 'area', 'and', 8870.97, 10000)

 

***** 处理图片- 转正图片
shape_trans (SelectedRegions, RegionTrans, 'rectangle2')
** 求出中心位置
area_center (RegionTrans, Area, Row, Column)
** 获取当前矩形的倾斜度
orientation_region (RegionTrans, Phi)
** 构建一个旋转的仿射变换矩阵
vector_angle_to_rigid (Row, Column, Phi, Row, Column, rad(0), HomMat2D)
hom_mat2d_slant (HomMat2D, rad(15), 'x', Column, Row, HomMat2DSlant)
affine_trans_region (RegionTrans, RegionAffineTrans, HomMat2DSlant, 'nearest_neighbor')
affine_trans_image (Image, ImageAffineTrans, HomMat2DSlant, 'constant', 'false')
reduce_domain (ImageAffineTrans, RegionAffineTrans, ImageReduced)

 

*************** 开始识别之车牌图片预处理
rgb1_to_gray (ImageReduced, GrayImage1)
threshold (GrayImage1, Region1, 128, 255)
erosion_rectangle1 (Region1, RegionErosion, 3, 3)
opening_rectangle1 (RegionErosion, RegionOpening, 1, 3)
connection (RegionOpening, ConnectedRegions1)
select_shape (ConnectedRegions1, SelectedRegions1, 'area', 'and', 200, 500)
sort_region (SelectedRegions1, SortedRegions, 'character', 'true', 'column')

 

3.车牌字符分割带中文(中文、数字、字母)

***** 读取图片
dev_close_window ()
read_image (Image, '素材/chepai18.jpg')
dev_open_window_fit_image (Image, 0, 0, -1, -1, WindowHandle)
dev_display (Image)
***** 处理图片-定位车牌
rgb1_to_gray (Image, GrayImage)
threshold (GrayImage, Region, 80, 100)
connection (Region, ConnectedRegions)
select_shape (ConnectedRegions, SelectedRegions, 'area', 'and', 8870.97, 10000)
***** 处理图片- 转正图片
shape_trans (SelectedRegions, RegionTrans, 'rectangle2')
area_center (RegionTrans, Area, Row, Column)
orientation_region (RegionTrans, Phi)
vector_angle_to_rigid (Row, Column, Phi, Row, Column, rad(0), HomMat2D)
hom_mat2d_slant (HomMat2D, rad(15), 'x', Column, Row, HomMat2DSlant)
affine_trans_region (RegionTrans, RegionAffineTrans, HomMat2DSlant, 'nearest_neighbor')
affine_trans_image (Image, ImageAffineTrans, HomMat2DSlant, 'constant', 'false')
reduce_domain (ImageAffineTrans, RegionAffineTrans, ImageReduced)
*************** 开始识别之车牌图片预处理,带中文字符分割
rgb1_to_gray (ImageReduced, GrayImage1)
threshold (GrayImage1, Regions, 176, 242)
closing_circle (Regions, RegionClosing, 3.5)
opening_circle (Regions, RegionOpening, 1.7)
connection (RegionOpening, ConnectedRegions1)
select_shape (ConnectedRegions1, SelectedRegions1, 'area', 'and', 20, 600)
sort_region (SelectedRegions1, SortedRegions, 'character', 'true', 'column')
*************** 组装苏字区域
select_obj (SortedRegions, ObjectSelected1, 1)
select_obj (SortedRegions, ObjectSelected2, 2)
select_obj (SortedRegions, ObjectSelected3, 3)
union2 (ObjectSelected1, ObjectSelected2, RegionUnion)
union2 (RegionUnion, ObjectSelected3, RegionUnion1)
select_obj (SortedRegions, ObjectSelected4, 4)
select_obj (SortedRegions, ObjectSelected5, 5)
select_obj (SortedRegions, ObjectSelected6, 6)
select_obj (SortedRegions, ObjectSelected7, 7)
select_obj (SortedRegions, ObjectSelected8, 8)
select_obj (SortedRegions, ObjectSelected9, 9)
***************** 把所有区域组成一个对象
gen_empty_obj (NumberObject)
concat_obj (NumberObject, RegionUnion1, NumberObject)
concat_obj (NumberObject, ObjectSelected4, NumberObject)
concat_obj (NumberObject, ObjectSelected5, NumberObject)
concat_obj (NumberObject, ObjectSelected6, NumberObject)
concat_obj (NumberObject, ObjectSelected7, NumberObject)
concat_obj (NumberObject, ObjectSelected8, NumberObject)
concat_obj (NumberObject, ObjectSelected9, NumberObject)
**************** 创建训练文件
TrainFile:='./Charactor.trf'
Words:=['苏','E','C','6','2','N','8']
** 完成图像与字符训练对应关系
write_ocr_trainf (NumberObject, GrayImage1, Words, TrainFile)
** 读取训练文件
read_ocr_trainf_names (TrainFile, CharacterNames, CharacterCount)
** 创建一个分类识别器
create_ocr_class_mlp (8, 10, 'constant', 'default', CharacterNames, 80, 'none', 10, 42, OCRHandle)
** 训练分类识别器
trainf_ocr_class_mlp (OCRHandle, TrainFile, 200, 1, 0.01, Error, ErrorLog)
** 保存分类识别文件
write_ocr_class_mlp (OCRHandle, './Charactor.omc')
*********************** 基于训练omc文件开始识别带中文车牌**************
read_ocr_class_mlp ('./Charactor.omc', OCRHandle1)
do_ocr_multi_class_mlp (NumberObject, GrayImage1, OCRHandle1, Class, Confidence)
dev_clear_window ()
dev_display (Image)
dev_set_color ('red')
set_display_font (WindowHandle, 30, 'mono', 'true', 'false')
for Index := 0 to |Class|-1 by 1
    set_tposition (WindowHandle,30, 120+36*Index)
    write_string (WindowHandle, Class[Index])
endfor

4.统计每个骰子的点数(距离变换+分水岭)

dev_close_window ()
dev_open_window (0, 0, 500, 500, 'black', WindowHandle)
set_display_font (WindowHandle, 30, 'mono', 'true', 'false')
list_files ('素材/dice/', ['files','follow_links'], ImageFiles)
tuple_regexp_select (ImageFiles, ['\\.(tif|tiff|gif|bmp|jpg|jpeg|jp2|png|pcx|pgm|ppm|pbm|xwd|ima|hobj)$','ignore_case'], ImageFiles)
ResultIndex:=1
for Index := 0 to |ImageFiles| - 1 by 1
    read_image (Image, ImageFiles[Index])
    get_image_size (Image, Width, Height)
    threshold (Image, Regions, 180, 255)

    *区域填充(把区域中孔洞进行填充)
    fill_up (Regions, RegionFillUp)

    *区域砍断
    connection (RegionFillUp, ConnectedRegions)
    *距离变换(水平垂直距离)越靠近边缘越暗
    distance_transform (ConnectedRegions, DistanceImage, 'octagonal', 'true', 1000, 1000)
    ** convert_image_type算子转化为BYTE类型时真实值小于零的值直接置0，而大于0的值则保持不变。
    convert_image_type (DistanceImage, ImageConverted, 'byte')

    *图像灰度反转
    invert_image (ImageConverted, ImageInvert)
    *灰度值缩放(扩大暗色区域)
    scale_image_max (ImageInvert, ImageScaleMax)
    *分水岭分割
    watersheds_threshold (ImageScaleMax, Basins, 10)
    intersection (Basins, ConnectedRegions, RegionIntersection)

    select_shape (RegionIntersection, SelectedRegions, 'area', 'and', 30000, 100000)
    count_obj (SelectedRegions, Number)
    dev_display (Image)
    for Index1 := 1 to Number by 1
        select_obj (SelectedRegions, ObjectSelected, Index1)
        reduce_domain (Image, ObjectSelected, ImageReduced)
        threshold (ImageReduced, Regions1, 0, 150)
        fill_up (Regions1, RegionFillUp1)
        connection (RegionFillUp1, ConnectedRegions1)
        select_shape (ConnectedRegions1, SelectedRegions1, 'area', 'and', 200, 20000)
        count_obj (SelectedRegions1, Number1)
        area_center (ObjectSelected, Area, Row, Column)
        disp_message (WindowHandle, Number1, 'image', Row, Column, 'green', 'false')
         
    endfor
    stop()
endfor

距离变换算法

计算区域每个点到区域边缘的距离变换，并将距离信息赋值到图像中。

distance_transform(Region : DistanceImage : Metric, Foreground, Width, Height : )

Region：输入参数，目标区域。

DistanceImage：输出参数，距离图像。

Metric：输入参数，用于测量距离的方法。默认 ‘city-block’

【

‘chamfer-3-4’,沿每个点到边缘的平行和垂直移动的权重为3，对角线移动的权重为4的方向计算最近距离。

‘chamfer-3-4-unnormalized’,同上，抑制归一化，可以减少运算量。

‘chessboard’,沿每个点到边缘的垂直、平行和对角的方向计算最近距离。

‘city-block’,沿每个点到边缘的垂直和平行的方向计算最近距离。

‘euclidean’,沿每个点到边缘的欧几里得距离。

‘octagonal’,沿每个点到边缘的垂直、平行和对角线（权值更高）的方向计算最近距离。

】

Foreground：输入参数，计算方式设置。默认 ‘true’ 【‘false’在区域外所有的点到区域的边缘, ‘true’计算区域内所有的点到区域边缘的距离】

Width：输入参数，输入的计算距离的范围宽度。1 ≤ Width

Height：输入参数，输入的计算距离的范围高度。1 ≤ Height

这里的Width，Height一般情况是等于图片的分辨率大小，但是范围值也可以设定为：要计算的区域包含在设定的分辨率内。

watersheds_threshold

分水岭，就是在峡谷最低哇处开口注水，然后地形地貌高耸的地方就会形成大坝，或者说是一条条脊。