关于视觉跟踪中评价标准的相关记录（The Evaluation of Visual Tracking Results on OTB-100 Dataset）
关于视觉跟踪中评价标准的相关记录（The Evaluation of Visual Tracking Results on OTB-100 Dataset）
2018-01-22 21:49:17
Benchmark website：http://cvlab.hanyang.ac.kr/tracker_benchmark/benchmark_v10.html
1. 修改 benchmark 的路径，改为你自己的数据集的路径：
2. 也可以修改 tracker 的设置，仅仅显示自己想要输出的那些跟踪算法：
3. 展示结果：
那么，问题来了，怎么将其拓展到 OTB100 dataset 上？怎么评价自己的跟踪结果？
1. 首先，将 tracking 的 txt 文档，生成 .mat 文件：
第一个是生成 .mat 文件的主函数。
1 %% #################################################### 2 3 % First, load the track_results.txt and groundtruth.txt 4 close all; clear all; clc; 5 warning off all; 6 addpath('./util'); 7 addpath(('C:\Users\WANG XIAO\Downloads\tracker_benchmark_v1.0\vlfeat-0.9.20-bin\vlfeat-0.9.20\toolbox')); 8 vl_setup 9 10 seqs=configSeqs; 11 trackers = configTrackers; 12 13 for ii=1:size(seqs, 2) 14 seqs{1, ii}.path = [seqs{1, ii}.path 'img\']; 15 end 16 17 numSeq=length(seqs); 18 numTrk=length(trackers); 19 evalType = 'TRE'; 20 finalPath = ['./results/results_' evalType '_CVPR13/']; 21 22 if ~exist(finalPath,'dir') 23 mkdir(finalPath); 24 end 25 26 tmpRes_path = ['./tmp/' evalType '/']; 27 bSaveImage=0; 28 29 if ~exist(tmpRes_path, 'dir') 30 mkdir(tmpRes_path); 31 end 32 33 pathAnno = './anno/'; 34 addpath(('./rstEval')); 35 addpath(['./trackers/VIVID_Tracker']); 36 results_base_path = 'C:\Users\WANG XIAO\Downloads\tracker_benchmark_v1.0\txt_files\SRDCF\'; 37 groundtruth_base_path ='C:\Users\WANG XIAO\Desktop\OTB100\Benchmark\'; 38 39 % all videos, call self with each video name. only keep valid directory names 40 dirs = dir(groundtruth_base_path); 41 videos = {dirs.name}; 42 videos(strcmp('.', videos) | strcmp('..', videos) | ... 43 strcmp('anno', videos) | ~[dirs.isdir]) = []; 44 45 [vn,~] = size(videos(:)); 46 numSeg = 20; 47 48 % ##################################################### 49 % The Main For Loop 50 % ##################################################### 51 for num = 1:numel(videos) 52 % get image ground truth for evaluation 53 [gt] = load_groundtruth_txt_info(groundtruth_base_path, videos{num}); 54 % get image track result for evaluation 55 [track_result] = load_results_txt_info(results_base_path, [videos{num} ]); 56 [num_of_frames, ~] = size(gt(:,1)); 57 toc = 100; 58 59 60 s = seqs{num}; 61 s.len = s.endFrame - s.startFrame + 1; 62 s.s_frames = cell(s.len,1); 63 nz = strcat('%0',num2str(s.nz),'d'); % number of zeros in the name of image 64 for i=1:s.len 65 image_no = s.startFrame + (i-1); 66 id = sprintf(nz,image_no); 67 s.s_frames{i} = strcat(s.path, id, '.', s.ext); 68 end 69 70 img = imread(s.s_frames{1}); 71 [imgH,imgW,ch]=size(img); 72 73 rect_anno = dlmread([pathAnno s.name '.txt']); 74 75 76 [subSeqs, subAnno, subTrackingResults] = splitSeqTREv2(s, numSeg, rect_anno, track_result); 77 78 79 80 81 % Second, translate the track_results.txt to .mat format file and save it. 82 for subIndex = 1:numSeg % 20 parts 83 84 current_part_track_results = subTrackingResults{1, subIndex}; 85 current_part_gt_results = subAnno{1, subIndex}; 86 87 results{subIndex}.res = current_part_track_results; 88 results{subIndex}.type = 'rect'; % 'ivtAff' 89 results{subIndex}.fps = num_of_frames / toc; 90 results{subIndex}.len = subSeqs{1, subIndex}.len ; 91 results{subIndex}.annoBegin = 1; 92 results{subIndex}.startFrame = subSeqs{1, subIndex}.startFrame; 93 results{subIndex}.anno = current_part_gt_results; 94 results{subIndex}.shiftType = 'left'; 95 videos{num} = [lower(videos{num}(1)) videos{num}(2:end)]; 96 if videos{num}(end-1)=='-' 97 videos{num} = [videos{num}(1:end-2) '.' videos{num}(end)]; 98 end 99 if strcmp(videos{num},'human4') 100 videos{num}='human4.2'; 101 end 102 103 104 end 105 106 matsavePath = './results/results_TRE_CVPR13/'; 107 mkdir(matsavePath); 108 109 save([[matsavePath videos{num}] '_SRDCF.mat'], 'results'); 110 111 112 end
1 function [ track_result ] = load_results_txt_info(base_path,video) 2 %LOAD_TXT_INFO 3 4 %see if there's a suffix, specifying one of multiple targets, for 5 %example the dot and number in 'Jogging.1' or 'Jogging.2'. 6 7 if numel(video) >= 2 && video(end-1) == '.' && ~isnan(str2double(video(end))), 8 suffix = video(end-1:end); %remember the suffix 9 video = video(1:end-2); %remove it from the video name 10 else 11 suffix = ''; 12 end 13 14 %full path to the video's files 15 if base_path(end) ~= '/' && base_path(end) ~= '\', 16 base_path(end+1) = '/'; 17 end 18 19 %try to load ground truth from text file (Benchmark's format) 20 % filename = [base_path video suffix '_ours.txt']; 21 try 22 filename = [base_path 'SRDCF_' suffix video '.txt']; 23 catch 24 filename = [base_path video suffix '_SRDCF.txt']; 25 end 26 27 f = fopen(filename); 28 assert(f ~= -1, ['No initial position or ground truth to load ("' filename '").']) 29 30 %the format is [x, y, width, height] 31 try 32 track_result = textscan(f, '%f,%f,%f,%f', 'ReturnOnError',false); 33 catch %#ok, try different format (no commas) 34 frewind(f); 35 track_result = textscan(f, '%f %f %f %f'); 36 % str = fgetl(f); 37 % track_result = textscan(str,'%f'); 38 % str = track_result{1}'; 39 end 40 track_result = cat(2, track_result{:}); 41 fclose(f); 42 43 end
1 function [subSeqs, subAnno, subTrackingResults]=splitSeqTREv2(seq, segNum,rect_anno, track_result) 2 % 20 segments for each sequences 3 % first, excluding all the occ/out-of-view frames 4 % then, sampling 5 6 minNum = 20; 7 8 fileName = ['initOmit/' seq.name '.txt']; 9 IdxExclude = []; 10 if exist(fileName) 11 IdxExclude=load(fileName)-seq.startFrame+1; 12 end 13 Idx = 1:seq.len; 14 for j = 1:size(IdxExclude,1) 15 Idx(IdxExclude(j,1):IdxExclude(j,2))=0; 16 end 17 Idx = Idx(find(Idx>0)); 18 19 for i=1:length(Idx) 20 r = rect_anno(Idx(i),:); 21 22 if r(1)<=0 | r(2)<=0 | r(3)<=0 | r(4)<=0 | isnan(sum(r)) 23 Idx(i) = 0; 24 end 25 end 26 27 Idx = Idx(find(Idx>0)); 28 29 for i = length(Idx):-1:1 30 if seq.len - Idx(i) + 1 >= minNum 31 endSeg = Idx(i); 32 endSegIdx = i; 33 break; 34 end 35 end 36 37 startFrIdxOne = [floor(1:endSegIdx/(segNum-1):endSegIdx) endSegIdx] ; 38 39 % endSeg = seq.len-minNum+1; 40 41 subAnno=[]; 42 subSeqs=[]; 43 subTrackingResults = []; 44 45 for i = 1:length(startFrIdxOne) 46 index = Idx(startFrIdxOne(i)); 47 subS.path = seq.path; 48 subS.nz = seq.nz; 49 subS.ext = seq.ext; 50 51 subS.startFrame = index+seq.startFrame-1; 52 subS.endFrame = seq.endFrame; 53 54 subS.len = subS.endFrame - subS.startFrame + 1; 55 56 subS.annoBegin = seq.startFrame; 57 subS.init_rect = rect_anno(index,:); 58 anno = rect_anno(index:end,:); 59 60 subS.s_frames = seq.s_frames(index:end); 61 62 subS.name = seq.name; 63 % subS.nameIdx = [seq.name '_' num2str(i)]; 64 65 subAnno{i} = anno; 66 subSeqs{i} = subS; 67 subTrackingResults{i} = track_result(subS.startFrame:subS.endFrame, :); 68 69 70 71 end
1 function [ ground_truth] = load_groundtruth_txt_info(base_path, video) 2 %LOAD_TXT_INFO 3 4 disp(['==>> deal with video: ', video]); 5 6 %see if there's a suffix, specifying one of multiple targets, for 7 %example the dot and number in 'Jogging.1' or 'Jogging.2'. 8 9 if numel(video) >= 2 && video(end-1) == '.' && ~isnan(str2double(video(end))), 10 suffix = video(end-1:end); %remember the suffix 11 video = video(1:end-2); %remove it from the video name 12 else 13 suffix = ''; 14 end 15 16 %full path to the video's files 17 if base_path(end) ~= '/' && base_path(end) ~= '\', 18 base_path(end+1) = '/'; 19 end 20 video_path = [base_path video '/']; 21 22 %try to load ground truth from text file (Benchmark's format) 23 filename = [video_path 'groundtruth_rect' suffix '.txt']; 24 f = fopen(filename); 25 assert(f ~= -1, ['No initial position or ground truth to load ("' filename '").']) 26 27 %the format is [x, y, width, height] 28 try 29 ground_truth = textscan(f, '%f,%f,%f,%f', 'ReturnOnError',false); 30 catch %#ok, try different format (no commas) 31 frewind(f); 32 ground_truth = textscan(f, '%f %f %f %f'); 33 end 34 35 ground_truth = cat(2, ground_truth{:}); 36 37 fclose(f); 38 39 end
有了这些 .mat 文件，就可以将其用于画 TRE 的曲线图了。
2. 用 perfPlot.m 函数来画出曲线图即可。
Stay Hungry，Stay Foolish ...
