模仿ShapeNet数据集格式构建仿真点云数据集

数据集结构

-- 0269156
     -- lidar_1.txt 
     -- lidar_2.txt
     -- lidar_3.txt
     ...
     -- lidar_n.txt
-- 0668753
-- 0668764
...
-- 0668768
-- train_test_split
    -- train.json
       ["shape_data/02691156/lidar_6","shape_data/02691156/lidar_12",...         "shape_data/02691156/lidar_20"] 
    -- test.json
    -- val.json
synsetoffset2category.txt
    "road"  	02691156

数据集制作

• 将CSV文件按ShapeNet数据集格式保存为txt文件

clc
clear
%% csv数据保存为mat
filename=dir('D:\leidashuji\lidar_label\code\csvdata\*.csv');
for i=1:length(filename)
csvname=[filename(i,1).folder '\' filename(i,1).name];
csvdata{i} = readmatrix(csvname);
data_orgin{i}(:,1:3) =csvdata{i}(:,1:3);
data_orgin{i}(:,4:6) =csvdata{i}(:,6:8);
data_orgin{i}(:,7) =csvdata{i}(:,4);
numPoint(i)=length(data_orgin{i});
pointcloud{i}=pointCloud(data_orgin{i}(:,1:3));
end
%%
for i=1:length(filename)
a=data_orgin{i};
path='D:\leidashuji\lidar_label\code\mat_data';
name=['lidar_',num2str(i)];
name1=['lidar_',num2str(i),'.txt'];
path_name{i}=fullfile(path,name);
path_name1{i}=fullfile(path,name1);
save (path_name{i}, 'a');
save (path_name1{i},'a','-ascii')
end

• 划分训练集，测试集，验证集

clc
clear
path='shape_data/02691156/';
filename_train=dir('D:\leidashuji\lidar_label\code\lidar_data\02691156\*.txt');%加载文件路径
train_num=0.7;
test_num=0.2;
val_num=0.1;%划分数据集比例
r=1:length(filename_train);
randIndex = randperm(size(r,2));%生成随机种子
r=r(:,randIndex);
train_list=[];
test_list=[];
val_list=[];
%将数据集按比例进行划分
for i=1:fix(train_num*length(filename_train))
train_list=[train_list,'"' path 'lidar_' num2str(r(i)) '", '];
end
for i=fix(train_num*length(filename_train))+1:fix(train_num*length(filename_train))+...
    fix(test_num*length(filename_train))
test_list=[test_list,'"' path 'lidar_' num2str(r(i)) '", '];
end
for i=fix(train_num*length(filename_train))+fix(test_num*length(filename_train))+1:length(filename_train)
val_list=[val_list,'"' path 'lidar_' num2str(r(i)) '", '];
end

• 将生成的test_list,train_list,val_list保存为.json文件

测试训练

• 将分类标签进行更改

seg_classes = {'road': [1,2,3]}#road代表大类名称，[1,2,3]代表属于该大类的标签数值

• 在train_partseg设置数据加载根目录

root = 'data/lidar_data/'#数据根目录
TRAIN_DATASET = PartNormalDataset(root=root, npoints=args.npoint, split='trainval', normal_channel=args.normal)
trainDataLoader = torch.utils.data.DataLoader(TRAIN_DATASET, batch_size=args.batch_size, shuffle=True, num_workers=10, drop_last=True)
TEST_DATASET = PartNormalDataset(root=root, npoints=args.npoint, split='test', normal_channel=args.normal)
testDataLoader = torch.utils.data.DataLoader(TEST_DATASET, batch_size=args.batch_size, shuffle=False, num_workers=10)

• 在ShapeNetDataLoader中设置数据路径

def __init__(self,root = 'Pointnet_Pointnet2_pytorch-master-0524\data\lidar_data', npoints=2500, split='train', class_choice=None, normal_channel=False):

self.meta = {}
        with open(os.path.join(self.root, 'train_test_split', 'train.json'), 'r') as f:
            train_ids = set([str(d.split('/')[2]) for d in json.load(f)])
        with open(os.path.join(self.root, 'train_test_split', 'val.json'), 'r') as f:
            val_ids = set([str(d.split('/')[2]) for d in json.load(f)])
        with open(os.path.join(self.root, 'train_test_split', 'test.json'), 'r') as f:
            test_ids = set([str(d.split('/')[2]) for d in json.load(f)])
        for item in self.cat:

• 训练结果

• 对于分类模型\(f\) 和大小为\(n\)的测试集\(D\)，\(Accuracy\)的定义为：

  

• mIOU: MIoU值越大（越接近1），模型效果越好

  P：Prediction预测值
  G：Ground Truth真实值

  

2022-05-28 10:12:21,461 - Model - INFO - Epoch 50 (50/50):
2022-05-28 10:12:21,461 - Model - INFO - Learning rate:0.000250
2022-05-28 10:12:35,241 - Model - INFO - Train accuracy is: 0.90082
2022-05-28 10:12:41,143 - Model - INFO - eval mIoU of road           0.519350
2022-05-28 10:12:41,143 - Model - INFO - Epoch 50 test Accuracy: 0.818262  Class avg mIOU: 0.519350   Inctance avg mIOU: 0.519350
2022-05-28 10:12:41,143 - Model - INFO - Best accuracy is: 0.83916
2022-05-28 10:12:41,144 - Model - INFO - Best class avg mIOU is: 0.58906
2022-05-28 10:12:41,144 - Model - INFO - Best inctance avg mIOU is: 0.58906

2022-05-29 11:44:36,180 - Model - INFO - Epoch 100 (100/100):
2022-05-29 11:44:36,180 - Model - INFO - Learning rate:0.000063
2022-05-29 11:44:49,908 - Model - INFO - Train accuracy is: 0.95139
2022-05-29 11:44:55,608 - Model - INFO - eval mIoU of road           0.625144
2022-05-29 11:44:55,608 - Model - INFO - Epoch 100 test Accuracy: 0.854199  Class avg mIOU: 0.625144   Inctance avg mIOU: 0.625144
2022-05-29 11:44:55,608 - Model - INFO - Best accuracy is: 0.85879
2022-05-29 11:44:55,608 - Model - INFO - Best class avg mIOU is: 0.65300
2022-05-29 11:44:55,608 - Model - INFO - Best inctance avg mIOU is: 0.65300

• 官方数据集的MIOU为0.8左右

Model	Inctance avg IoU	Class avg IoU
PointNet (Official)	83.7	80.4
PointNet2 (Official)	85.1	81.9
PointNet (Pytorch)	84.3	81.1
PointNet2_SSG (Pytorch)	84.9	81.8
PointNet2_MSG (Pytorch)	85.4	82.5

• 可视化

存在的问题

1. 测试集运行后，只了解输出的mIOU,暂时不了解如何将训练后的结果可视化