阅读mmdetection3d框架的源码探索其构建dataset的流程
在查看一些基于mmdetection3d
构建的代码的时候,一开始会摸不着头脑,它的dataset
到底是怎么构造的?
接下来就直接下载mmdetection3d
这个仓库,然后去分析里面的代码。
可以看到config(比如mmdetection3d/configs/centerpoint/centerpoint_02pillar_second_secfpn_4x8_cyclic_20e_nus.py
)有这样的字段:
train_pipeline = [
# 属于pipelines.loading中的操作
dict(
type='LoadPointsFromFile',
coord_type='LIDAR',
load_dim=5,
use_dim=5,
file_client_args=file_client_args),
# 输出的东西已经是一个名为results的字典了,里面最重要的就是results['points']了,是LiDARPoints类
dict(
type='LoadPointsFromMultiSweeps',
sweeps_num=9,
use_dim=[0, 1, 2, 3, 4],
file_client_args=file_client_args,
pad_empty_sweeps=True,
remove_close=True),
# 接收上面输出的results字典,然后对于里面的results['points']做增密操作。
# 输出的字典有15个key:dict_keys(['sample_idx', 'pts_filename', 'sweeps', 'timestamp', 'ann_info', 'img_fields', 'bbox3d_fields', 'pts_mask_fields', 'pts_seg_fields', 'bbox_fields', 'mask_fields', 'seg_fields', 'box_type_3d', 'box_mode_3d', 'points'])
dict(type='LoadAnnotations3D', with_bbox_3d=True, with_label_3d=True),
# 输出的字典有17个key:dict_keys(['sample_idx', 'pts_filename', 'sweeps', 'timestamp', 'ann_info', 'img_fields', 'bbox3d_fields', 'pts_mask_fields', 'pts_seg_fields', 'bbox_fields', 'mask_fields', 'seg_fields', 'box_type_3d', 'box_mode_3d', 'points', 'gt_bboxes_3d', 'gt_labels_3d'])
# 所以主要增加的键来自于这俩方法:self._load_bboxes_3d()和self._load_labels_3d()
# 属于piplines.transform_3d
dict(type='ObjectSample', db_sampler=db_sampler),
dict(
type='GlobalRotScaleTrans',
rot_range=[-0.3925, 0.3925],
scale_ratio_range=[0.95, 1.05],
translation_std=[0, 0, 0]),
dict(
type='RandomFlip3D',
sync_2d=False,
flip_ratio_bev_horizontal=0.5,
flip_ratio_bev_vertical=0.5),
dict(type='PointsRangeFilter', point_cloud_range=point_cloud_range),
dict(type='ObjectRangeFilter', point_cloud_range=point_cloud_range),
dict(type='ObjectNameFilter', classes=class_names),
dict(type='PointShuffle'),
# 属于piplines.formatting
dict(type='DefaultFormatBundle3D', class_names=class_names),
dict(type='Collect3D', keys=['points', 'gt_bboxes_3d', 'gt_labels_3d'])
# 最后只留下'image_meta'和上面的这三个keys。
]
只能大概猜测它是通过这些预先定义好了的类去规范整个dataset的加载过程吗?但是具体又是体现在哪个地方呢?或者说实际上load的时候怎么取到这些字典里面的值的呢?
下面来跟着代码走一下,
运行python tools/train.py configs/centerpoint/centerpoint_02pillar_second_secfpn_4x8_cyclic_20e_nus.py
然后打断点
首先看到tools/train.py
中的这里
经过build_dataset方法得到的dataset。
那么就要进去看build_dataset是做了啥
因为config里面写的data的type是type='CBGSDataset'
,所以会先走第39行,然后再次调用该函数,再走第43行。
然后就是进入build_from_cfg
函数了,
它最重要的代码就是截图所示,本质上是:
- 从
args
中移除键'type'
并获取其值obj_type
。 - 如果
obj_type
是字符串,从registry
中查找对应的类obj_cls
。 - 如果
obj_type
是类,直接使用obj_type
作为obj_cls
。 - 如果
obj_type
既不是字符串也不是类,抛出类型错误。 - 尝试使用
args
中的参数来实例化obj_cls
。 - 如果实例化过程中发生异常,捕获异常并抛出带有类名的异常信息,以便调试。
那么在这一层的build_from_cfg
函数中,obj_cls
指的是<class 'mmdet3d.datasets.nuscenes_dataset.NuScenesDataset'>
,也就是要构建这个类,然后返回去。
再往里走看这个NuScenesDataset
类的构造函数,它的基类是Custom3DDataset
而且这里面就有pipeline参数了,如果在图中它执行父类构造函数的时候停下来来看这个pipeline
,可以看到是
pipeline: [{'type': 'LoadPointsFromFile', 'coord_type': 'LIDAR', 'load_dim': 5, 'use_dim': 5, 'file_client_args': {...}}, {'type': 'LoadPointsFromMultiSweeps', 'sweeps_num': 9, 'use_dim': [...], 'file_client_args': {...}, 'pad_empty_sweeps': True, 'remove_close': True}, {'type': 'LoadAnnotations3D', 'with_bbox_3d': True, 'with_label_3d': True}, {'type': 'ObjectSample', 'db_sampler': {...}}, {'type': 'GlobalRotScaleTrans', 'rot_range': [...], 'scale_ratio_range': [...], 'translation_std': [...]}, {'type': 'RandomFlip3D', 'sync_2d': False, 'flip_ratio_bev_horizontal': 0.5, 'flip_ratio_bev_vertical': 0.5}, {'type': 'PointsRangeFilter', 'point_cloud_range': [...]}, {'type': 'ObjectRangeFilter', 'point_cloud_range': [...]}, {'type': 'ObjectNameFilter', 'classes': [...]}, {'type': 'PointShuffle'}, {'type': 'DefaultFormatBundle3D', 'class_names': [...]}, {'type': 'Collect3D', 'keys': [...]}]
它其实就是上面config里面的东西。
然后就执行父类构造函数啦,把pipeline
做了一下Compose
接着就可以看到Compose,大概是把这些流程都组合起来的意思,这个没有深究
所以,如果transforms
列表中的元素transform
是callable
的(即是类),那么就直接append到self.transform
里面,反之则要再重复调用build_from_cfg
函数,然后用transform(是个字典)
和PIPELINES
去构建这个类。再append到self.transform
里面。
至于这俩参数是什么,放一下详细解答:
transform是一个字典,例:
{'type': 'LoadPointsFromMultiSweeps', 'sweeps_num': 9, 'use_dim': [0, 1, 2, 3, 4], 'file_client_args': {'backend': 'disk'}, 'pad_empty_sweeps': True, 'remove_close': True}
PIPELINE是一个注册表,
Registry(name=pipeline, items={'Compose': <class 'mmdet.datasets.pipelines.compose.Compose'>, 'AutoAugment': <class 'mmdet.datasets.pipelines.auto_augment.AutoAugment'>, 'Shear': <class 'mmdet.datasets.pipelines.auto_augment.Shear'>, 'Rotate': <class 'mmdet.datasets.pipelines.auto_augment.Rotate'>, 'Translate': <class 'mmdet.datasets.pipelines.auto_augment.Translate'>, 'ColorTransform': <class 'mmdet.datasets.pipelines.auto_augment.ColorTransform'>, 'EqualizeTransform': <class 'mmdet.datasets.pipelines.auto_augment.EqualizeTransform'>, 'BrightnessTransform': <class 'mmdet.datasets.pipelines.auto_augment.BrightnessTransform'>, 'ContrastTransform': <class 'mmdet.datasets.pipelines.auto_augment.ContrastTransform'>, 'ToTensor': <class 'mmdet.datasets.pipelines.formatting.ToTensor'>, 'ImageToTensor': <class 'mmdet.datasets.pipelines.formatting.ImageToTensor'>, 'Transpose': <class 'mmdet.datasets.pipelines.formatting.Transpose'>, 'ToDataContainer': <class 'mmdet.datasets.pipelines.formatting.ToDataContainer'>, 'Collect': <class 'mmdet.datasets.pipelines.formatting.Collect'>, 'WrapFieldsToLists': <class 'mmdet.datasets.pipelines.formatting.WrapFieldsToLists'>, 'InstaBoost': <class 'mmdet.datasets.pipelines.instaboost.InstaBoost'>, 'LoadImageFromFile': <class 'mmdet.datasets.pipelines.loading.LoadImageFromFile'>, 'LoadImageFromWebcam': <class 'mmdet.datasets.pipelines.loading.LoadImageFromWebcam'>, 'LoadMultiChannelImageFromFiles': <class 'mmdet.datasets.pipelines.loading.LoadMultiChannelImageFromFiles'>, 'LoadAnnotations': <class 'mmdet.datasets.pipelines.loading.LoadAnnotations'>, 'LoadPanopticAnnotations': <class 'mmdet.datasets.pipelines.loading.LoadPanopticAnnotations'>, 'LoadProposals': <class 'mmdet.datasets.pipelines.loading.LoadProposals'>, 'FilterAnnotations': <class 'mmdet.datasets.pipelines.loading.FilterAnnotations'>, 'MultiScaleFlipAug': <class 'mmdet.datasets.pipelines.test_time_aug.MultiScaleFlipAug'>, 'Resize': <class 'mmdet.datasets.pipelines.transforms.Resize'>, 'RandomFlip': <class 'mmdet.datasets.pipelines.transforms.RandomFlip'>, 'RandomShift': <class 'mmdet.datasets.pipelines.transforms.RandomShift'>, 'Pad': <class 'mmdet.datasets.pipelines.transforms.Pad'>, 'Normalize': <class 'mmdet.datasets.pipelines.transforms.Normalize'>, 'RandomCrop': <class 'mmdet.datasets.pipelines.transforms.RandomCrop'>, 'SegRescale': <class 'mmdet.datasets.pipelines.transforms.SegRescale'>, 'PhotoMetricDistortion': <class 'mmdet.datasets.pipelines.transforms.PhotoMetricDistortion'>, 'Expand': <class 'mmdet.datasets.pipelines.transforms.Expand'>, 'MinIoURandomCrop': <class 'mmdet.datasets.pipelines.transforms.MinIoURandomCrop'>, 'Corrupt': <class 'mmdet.datasets.pipelines.transforms.Corrupt'>, 'Albu': <class 'mmdet.datasets.pipelines.transforms.Albu'>, 'RandomCenterCropPad': <class 'mmdet.datasets.pipelines.transforms.RandomCenterCropPad'>, 'CutOut': <class 'mmdet.datasets.pipelines.transforms.CutOut'>, 'Mosaic': <class 'mmdet.datasets.pipelines.transforms.Mosaic'>, 'MixUp': <class 'mmdet.datasets.pipelines.transforms.MixUp'>, 'RandomAffine': <class 'mmdet.datasets.pipelines.transforms.RandomAffine'>, 'YOLOXHSVRandomAug': <class 'mmdet.datasets.pipelines.transforms.YOLOXHSVRandomAug'>, 'CopyPaste': <class 'mmdet.datasets.pipelines.transforms.CopyPaste'>, 'DefaultFormatBundle': <class 'mmdet3d.datasets.pipelines.formating.DefaultFormatBundle'>, 'Collect3D': <class 'mmdet3d.datasets.pipelines.formating.Collect3D'>, 'DefaultFormatBundle3D': <class 'mmdet3d.datasets.pipelines.formating.DefaultFormatBundle3D'>, 'LoadMultiViewImageFromFiles': <class 'mmdet3d.datasets.pipelines.loading.LoadMultiViewImageFromFiles'>, 'LoadImageFromFileMono3D': <class 'mmdet3d.datasets.pipelines.loading.LoadImageFromFileMono3D'>, 'LoadPointsFromMultiSweeps': <class 'mmdet3d.datasets.pipelines.loading.LoadPointsFromMultiSweeps'>, 'PointSegClassMapping': <class 'mmdet3d.datasets.pipelines.loading.PointSegClassMapping'>, 'NormalizePointsColor': <class 'mmdet3d.datasets.pipelines.loading.NormalizePointsColor'>, 'LoadPointsFromFile': <class 'mmdet3d.datasets.pipelines.loading.LoadPointsFromFile'>, 'LoadPointsFromDict': <class 'mmdet3d.datasets.pipelines.loading.LoadPointsFromDict'>, 'LoadAnnotations3D': <class 'mmdet3d.datasets.pipelines.loading.LoadAnnotations3D'>, 'MultiScaleFlipAug3D': <class 'mmdet3d.datasets.pipelines.test_time_aug.MultiScaleFlipAug3D'>, 'RandomDropPointsColor': <class 'mmdet3d.datasets.pipelines.transforms_3d.RandomDropPointsColor'>, 'RandomFlip3D': <class 'mmdet3d.datasets.pipelines.transforms_3d.RandomFlip3D'>, 'RandomJitterPoints': <class 'mmdet3d.datasets.pipelines.transforms_3d.RandomJitterPoints'>, 'ObjectSample': <class 'mmdet3d.datasets.pipelines.transforms_3d.ObjectSample'>, 'ObjectNoise': <class 'mmdet3d.datasets.pipelines.transforms_3d.ObjectNoise'>, 'GlobalAlignment': <class 'mmdet3d.datasets.pipelines.transforms_3d.GlobalAlignment'>, 'GlobalRotScaleTrans': <class 'mmdet3d.datasets.pipelines.transforms_3d.GlobalRotScaleTrans'>, 'PointShuffle': <class 'mmdet3d.datasets.pipelines.transforms_3d.PointShuffle'>, 'ObjectRangeFilter': <class 'mmdet3d.datasets.pipelines.transforms_3d.ObjectRangeFilter'>, 'PointsRangeFilter': <class 'mmdet3d.datasets.pipelines.transforms_3d.PointsRangeFilter'>, 'ObjectNameFilter': <class 'mmdet3d.datasets.pipelines.transforms_3d.ObjectNameFilter'>, 'PointSample': <class 'mmdet3d.datasets.pipelines.transforms_3d.PointSample'>, 'IndoorPointSample': <class 'mmdet3d.datasets.pipelines.transforms_3d.IndoorPointSample'>, 'IndoorPatchPointSample': <class 'mmdet3d.datasets.pipelines.transforms_3d.IndoorPatchPointSample'>, 'BackgroundPointsFilter': <class 'mmdet3d.datasets.pipelines.transforms_3d.BackgroundPointsFilter'>, 'VoxelBasedPointSampler': <class 'mmdet3d.datasets.pipelines.transforms_3d.VoxelBasedPointSampler'>, 'AffineResize': <class 'mmdet3d.datasets.pipelines.transforms_3d.AffineResize'>, 'RandomShiftScale': <class 'mmdet3d.datasets.pipelines.transforms_3d.RandomShiftScale'>})
那么最后self.transforms
是什么,它是一个长度为12的列表,装的其实就是上面config里面的那一堆用预定义好的类来表示的操作:
再看到Compose
类的__call__
方法,
可以看到它其实是将data连续地走过了self.transforms
里面的所有类,也就是这样来完成pipeline
的具体实现的。
然后因为整个train.py
中的dataset是CBGSDataset
,所以去看它的__getitem__()
,
发现相当于调它的self.dataset
的__getitem__()
,
而前面有说过,它是包裹着NuscenesDataset
的,即self.dataset
的类就是NuscenesDataset
所以就得去看NuscenesDataset
在__getitem__()
的时候返回的是什么?发现得去看它的父类Custom3DDataset
的__getitem__()
,
点进去看prepare_train_data()
,
其实就是取出了当前sample的图或者LiDAR的数据input_dict
,
{input_dict: 'sample_idx': '0468b5e2192f40d88a4e3353c83d28a1', 'pts_filename': './data/nuscenes/samples/LIDAR_TOP/n008-2018-08-29-16-04-13-0400__LIDAR_TOP__1535573055397947.pcd.bin', 'sweeps': [{...}, {...}, {...}, {...}, {...}, {...}, {...}, {...}, {...}, {...}], 'timestamp': 1535573055.397947, 'ann_info': {'gt_bboxes_3d': LiDARInstance3DBoxes(
tensor([[-1.3053e+01, 2.3971e+01, -8.4190e-01, 1.9640e+00, 4.8260e+00,
1.7050e+00, -1.5572e+00, 5.6951e-05, 9.9719e-04],
[ 4.0986e+00, -5.1065e+01, -3.7889e+00, 2.2280e+00, 4.9170e+00,
2.7230e+00, -1.5088e+00, 0.0000e+00, 0.0000e+00],
[ 5.9061e+00, -2.5881e+01, -2.8135e+00, 1.8990e+00, 4.5950e+00,
1.6550e+00, -1.5894e+00, 1.4087e-02, 1.1929e-03],
[-6.5956e-01, -1.9181e+01, -2.5623e+00, 2.0340e+00, 5.0230e+00,
1.6550e+00, -3.1338e+00, 1.5844e-01, 8.3847e+00],
[-1.3778e+01, 4.6664e+00, -1.7571e+00, 2.1550e+00, 5.5660e+00,
1.9950e+00, -4.6786e+00, 0.0000e+00, 0.0000e+00],
[-1.4067e+01, 1.6442e+00, -1.7979e+00, 1.9260e+00, 4.8000e+00,
1.6530e+00, -1.5948e+00, 2.7373e-02, 2.3441e-02],
[-2.7166e+01, -3.5422e+00, -2.7281e+00, 1.7740e+00, 4.3200e+00,
1.6430e+00, -1.5752e+00, 0.0000e+00, 0.0000e+00],
[-1.5801e+01, -4.8262e+01, -3.9795e+00, 2.1070e+00, 4.9980e+00,
1.8140e+00, -1.5487e+00, -6.7043e-02, 3.5860e-02],
[-1.1155e+01, 7.9349e+01, 1.8333e+00, 2.2070e+00, 5.7040e+00,
2.0920e+00, -4.6931e+00, -9.7979e-02, 2.6185e-03],
[ 1.1782e+01, -2.5557e+00, -1.6983e+00, 1.8120e+00, 4.7570e+00,
1.5840e+00, -4.6788e+00, -4.9902e-03, 2.8623e-04],
[ 2.4727e+01, -1.3353e+01, -1.7999e+00, 1.9700e+00, 5.1180e+00,
1.8120e+00, -1.5493e+00, 0.0000e+00, 0.0000e+00],
[ 5.1789e+01, 1.9438e+01, -2.9660e-01, 1.9310e+00, 4.2030e+00,
1.9810e+00, -2.5001e+00, -3.7849e-02, -2.4194e-01],
[-9.9715e+00, 1.3736e+00, -1.6758e+00, 1.1540e+00, 7.9300e-01,
1.7680e+00, -1.2722e+00, -7.5932e-02, -4.1784e-01],
[-3.4043e+01, 1.4688e+01, -7.7760e-01, 8.2200e-01, 9.3900e-01,
2.0000e+00, -3.1572e+00, -1.7419e-02, 1.8329e+00],
[-9.1596e+00, 6.2475e+01, 1.2546e+00, 4.9400e-01, 6.4900e-01,
1.8350e+00, -3.2893e+00, 3.7277e-02, 2.1861e-02],
[-1.3011e+01, 3.4884e+01, -5.2392e-01, 2.1080e+00, 4.9040e+00,
1.6120e+00, -1.4880e+00, -2.8475e-04, -4.9859e-03],
[ 5.6555e+00, -3.4311e+01, -3.3156e+00, 1.8730e+00, 4.4170e+00,
1.8000e+00, -1.5213e+00, -1.0550e-03, -9.3994e-04],
[-1.4270e+01, -7.5958e-01, -1.8951e+00, 1.9500e+00, 4.7310e+00,
1.6730e+00, -1.5787e+00, 2.1642e-02, 1.0764e-02],
[-2.5297e+01, 3.7611e+01, -8.4330e-01, 2.9440e+00, 4.1020e+00,
1.5660e+00, 1.5101e+00, -8.7569e-03, 4.4248e-01],
[-1.1432e+01, -2.0810e+01, -2.7025e+00, 6.1800e-01, 6.9100e-01,
1.8900e+00, -1.7533e+00, 1.3554e+00, 3.2644e-01],
[-2.6757e+01, -8.8425e+00, -2.7591e+00, 2.1220e+00, 4.5250e+00,
1.8830e+00, -4.6994e+00, 2.8475e-04, 4.9859e-03],
[ 5.8472e+00, -3.1766e+01, -3.3697e+00, 1.8280e+00, 4.3640e+00,
2.0000e+00, -1.5213e+00, 2.8503e-03, 1.4843e-02],
[-1.3683e+01, 1.8342e+01, -1.3403e+00, 2.0270e+00, 4.3520e+00,
1.5400e+00, -1.6096e+00, 0.0000e+00, 0.0000e+00],
[ 8.5600e+00, -1.8746e+01, -2.4114e+00, 6.7100e-01, 1.4200e+00,
1.2430e+00, -3.7172e+00, -1.4976e-01, -9.7458e-02],
[-1.3618e+01, 7.3931e+00, -1.4792e+00, 2.2650e+00, 5.4260e+00,
1.8200e+00, -4.7102e+00, -1.8709e-03, -1.3796e-01],
[ 3.5973e+01, 4.1933e+01, 4.0528e-01, 1.5150e+00, 3.4580e+00,
2.0000e+00, -3.1788e+00, -8.3159e-03, 2.2258e-01],
[ 4.9209e+01, 2.1194e+01, -2.8792e-01, 1.8320e+00, 3.9100e+00,
2.0360e+00, -2.6253e+00, 5.8494e-02, -1.1541e-01],
[-2.6333e+01, 4.6259e+00, -2.2232e+00, 1.9040e+00, 3.9660e+00,
1.3920e+00, -4.6994e+00, 0.0000e+00, 0.0000e+00],
[-1.3432e+01, 1.2661e+01, -1.3381e+00, 2.0670e+00, 4.9390e+00,
1.3830e+00, -1.5171e+00, 1.7713e-02, -5.8042e-02],
[-2.4358e+01, 7.7235e+01, 6.3165e-01, 2.0340e+00, 4.0860e+00,
1.5930e+00, -4.7057e+00, 0.0000e+00, 0.0000e+00],
[-2.7353e+01, -5.3840e+01, -4.7062e+00, 2.3260e+00, 5.0220e+00,
2.6770e+00, 1.5664e+00, 0.0000e+00, 0.0000e+00],
[-1.8183e+01, -1.5049e+01, -2.6358e+00, 5.1600e-01, 6.8800e-01,
1.8210e+00, -4.5943e+00, -4.5360e-01, 7.0038e-02],
[ 5.7435e+00, -3.6962e+01, -3.4783e+00, 1.9290e+00, 4.5690e+00,
1.8810e+00, -1.5213e+00, -3.9922e-03, 2.2898e-04],
[ 1.2472e+01, 1.7255e+01, -8.0609e-01, 2.0270e+00, 4.5710e+00,
1.7280e+00, -1.5121e+00, -1.9961e-03, 1.1449e-04],
[-3.4475e+01, 3.4049e+01, 1.8847e-02, 5.1600e-01, 7.1900e-01,
1.8210e+00, -3.1083e+00, 2.2408e-02, 1.3386e+00]])), 'gt_labels_3d': array([0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 4, 8, 8, 8, 0, 0, 0, 0, 8, 0, 0,
0, 7, 1, 2, 4, 0, 0, 0, 0, 8, 0, 0, 8]), 'gt_names': array(['car', 'truck', 'car', 'car', 'truck', 'car', 'car', 'car', 'car',
'car', 'car', 'trailer', 'pedestrian', 'pedestrian', 'pedestrian',
'car', 'car', 'car', 'car', 'pedestrian', 'car', 'car', 'car',
'bicycle', 'truck', 'construction_vehicle', 'trailer', 'car',
'car', 'car', 'car', 'pedestrian', 'car', 'car', 'pedestrian'],
dtype='<U26')}}
然后再经过一个self.prepipeline
之后变成:
input_dict: {'sample_idx': '0468b5e2192f40d88a4e3353c83d28a1', 'pts_filename': './data/nuscenes/samples/LIDAR_TOP/n008-2018-08-29-16-04-13-0400__LIDAR_TOP__1535573055397947.pcd.bin', 'sweeps': [{...}, {...}, {...}, {...}, {...}, {...}, {...}, {...}, {...}, {...}], 'timestamp': 1535573055.397947, 'ann_info': {'gt_bboxes_3d': LiDARInstance3DBoxes(
tensor([[-1.3053e+01, 2.3971e+01, -8.4190e-01, 1.9640e+00, 4.8260e+00,
1.7050e+00, -1.5572e+00, 5.6951e-05, 9.9719e-04],
[ 4.0986e+00, -5.1065e+01, -3.7889e+00, 2.2280e+00, 4.9170e+00,
2.7230e+00, -1.5088e+00, 0.0000e+00, 0.0000e+00],
[ 5.9061e+00, -2.5881e+01, -2.8135e+00, 1.8990e+00, 4.5950e+00,
1.6550e+00, -1.5894e+00, 1.4087e-02, 1.1929e-03],
[-6.5956e-01, -1.9181e+01, -2.5623e+00, 2.0340e+00, 5.0230e+00,
1.6550e+00, -3.1338e+00, 1.5844e-01, 8.3847e+00],
[-1.3778e+01, 4.6664e+00, -1.7571e+00, 2.1550e+00, 5.5660e+00,
1.9950e+00, -4.6786e+00, 0.0000e+00, 0.0000e+00],
[-1.4067e+01, 1.6442e+00, -1.7979e+00, 1.9260e+00, 4.8000e+00,
1.6530e+00, -1.5948e+00, 2.7373e-02, 2.3441e-02],
[-2.7166e+01, -3.5422e+00, -2.7281e+00, 1.7740e+00, 4.3200e+00,
1.6430e+00, -1.5752e+00, 0.0000e+00, 0.0000e+00],
[-1.5801e+01, -4.8262e+01, -3.9795e+00, 2.1070e+00, 4.9980e+00,
1.8140e+00, -1.5487e+00, -6.7043e-02, 3.5860e-02],
[-1.1155e+01, 7.9349e+01, 1.8333e+00, 2.2070e+00, 5.7040e+00,
2.0920e+00, -4.6931e+00, -9.7979e-02, 2.6185e-03],
[ 1.1782e+01, -2.5557e+00, -1.6983e+00, 1.8120e+00, 4.7570e+00,
1.5840e+00, -4.6788e+00, -4.9902e-03, 2.8623e-04],
[ 2.4727e+01, -1.3353e+01, -1.7999e+00, 1.9700e+00, 5.1180e+00,
1.8120e+00, -1.5493e+00, 0.0000e+00, 0.0000e+00],
[ 5.1789e+01, 1.9438e+01, -2.9660e-01, 1.9310e+00, 4.2030e+00,
1.9810e+00, -2.5001e+00, -3.7849e-02, -2.4194e-01],
[-9.9715e+00, 1.3736e+00, -1.6758e+00, 1.1540e+00, 7.9300e-01,
1.7680e+00, -1.2722e+00, -7.5932e-02, -4.1784e-01],
[-3.4043e+01, 1.4688e+01, -7.7760e-01, 8.2200e-01, 9.3900e-01,
2.0000e+00, -3.1572e+00, -1.7419e-02, 1.8329e+00],
[-9.1596e+00, 6.2475e+01, 1.2546e+00, 4.9400e-01, 6.4900e-01,
1.8350e+00, -3.2893e+00, 3.7277e-02, 2.1861e-02],
[-1.3011e+01, 3.4884e+01, -5.2392e-01, 2.1080e+00, 4.9040e+00,
1.6120e+00, -1.4880e+00, -2.8475e-04, -4.9859e-03],
[ 5.6555e+00, -3.4311e+01, -3.3156e+00, 1.8730e+00, 4.4170e+00,
1.8000e+00, -1.5213e+00, -1.0550e-03, -9.3994e-04],
[-1.4270e+01, -7.5958e-01, -1.8951e+00, 1.9500e+00, 4.7310e+00,
1.6730e+00, -1.5787e+00, 2.1642e-02, 1.0764e-02],
[-2.5297e+01, 3.7611e+01, -8.4330e-01, 2.9440e+00, 4.1020e+00,
1.5660e+00, 1.5101e+00, -8.7569e-03, 4.4248e-01],
[-1.1432e+01, -2.0810e+01, -2.7025e+00, 6.1800e-01, 6.9100e-01,
1.8900e+00, -1.7533e+00, 1.3554e+00, 3.2644e-01],
[-2.6757e+01, -8.8425e+00, -2.7591e+00, 2.1220e+00, 4.5250e+00,
1.8830e+00, -4.6994e+00, 2.8475e-04, 4.9859e-03],
[ 5.8472e+00, -3.1766e+01, -3.3697e+00, 1.8280e+00, 4.3640e+00,
2.0000e+00, -1.5213e+00, 2.8503e-03, 1.4843e-02],
[-1.3683e+01, 1.8342e+01, -1.3403e+00, 2.0270e+00, 4.3520e+00,
1.5400e+00, -1.6096e+00, 0.0000e+00, 0.0000e+00],
[ 8.5600e+00, -1.8746e+01, -2.4114e+00, 6.7100e-01, 1.4200e+00,
1.2430e+00, -3.7172e+00, -1.4976e-01, -9.7458e-02],
[-1.3618e+01, 7.3931e+00, -1.4792e+00, 2.2650e+00, 5.4260e+00,
1.8200e+00, -4.7102e+00, -1.8709e-03, -1.3796e-01],
[ 3.5973e+01, 4.1933e+01, 4.0528e-01, 1.5150e+00, 3.4580e+00,
2.0000e+00, -3.1788e+00, -8.3159e-03, 2.2258e-01],
[ 4.9209e+01, 2.1194e+01, -2.8792e-01, 1.8320e+00, 3.9100e+00,
2.0360e+00, -2.6253e+00, 5.8494e-02, -1.1541e-01],
[-2.6333e+01, 4.6259e+00, -2.2232e+00, 1.9040e+00, 3.9660e+00,
1.3920e+00, -4.6994e+00, 0.0000e+00, 0.0000e+00],
[-1.3432e+01, 1.2661e+01, -1.3381e+00, 2.0670e+00, 4.9390e+00,
1.3830e+00, -1.5171e+00, 1.7713e-02, -5.8042e-02],
[-2.4358e+01, 7.7235e+01, 6.3165e-01, 2.0340e+00, 4.0860e+00,
1.5930e+00, -4.7057e+00, 0.0000e+00, 0.0000e+00],
[-2.7353e+01, -5.3840e+01, -4.7062e+00, 2.3260e+00, 5.0220e+00,
2.6770e+00, 1.5664e+00, 0.0000e+00, 0.0000e+00],
[-1.8183e+01, -1.5049e+01, -2.6358e+00, 5.1600e-01, 6.8800e-01,
1.8210e+00, -4.5943e+00, -4.5360e-01, 7.0038e-02],
[ 5.7435e+00, -3.6962e+01, -3.4783e+00, 1.9290e+00, 4.5690e+00,
1.8810e+00, -1.5213e+00, -3.9922e-03, 2.2898e-04],
[ 1.2472e+01, 1.7255e+01, -8.0609e-01, 2.0270e+00, 4.5710e+00,
1.7280e+00, -1.5121e+00, -1.9961e-03, 1.1449e-04],
[-3.4475e+01, 3.4049e+01, 1.8847e-02, 5.1600e-01, 7.1900e-01,
1.8210e+00, -3.1083e+00, 2.2408e-02, 1.3386e+00]])), 'gt_labels_3d': array([0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 4, 8, 8, 8, 0, 0, 0, 0, 8, 0, 0,
0, 7, 1, 2, 4, 0, 0, 0, 0, 8, 0, 0, 8]), 'gt_names': array(['car', 'truck', 'car', 'car', 'truck', 'car', 'car', 'car', 'car',
'car', 'car', 'trailer', 'pedestrian', 'pedestrian', 'pedestrian',
'car', 'car', 'car', 'car', 'pedestrian', 'car', 'car', 'car',
'bicycle', 'truck', 'construction_vehicle', 'trailer', 'car',
'car', 'car', 'car', 'pedestrian', 'car', 'car', 'pedestrian'],
dtype='<U26')}, 'img_fields': [], 'bbox3d_fields': ['gt_bboxes_3d'], 'pts_mask_fields': [], 'pts_seg_fields': [], 'bbox_fields': [], 'mask_fields': [], 'seg_fields': [], 'box_type_3d': <class 'mmdet3d.core.bbox.structures.lidar_box3d.LiDARInstance3DBoxes'>, 'box_mode_3d': <Box3DMode.LIDAR: 0>, 'points': DataContainer(tensor([[-1.2596e+01, -2.1491e+01, -1.2492e+00, 1.1000e+01, 4.9989e-01],
[ 8.3029e+00, -5.2680e+00, -1.1252e+00, 3.0000e+00, 1.0011e-01],
[-1.4655e+01, 8.3101e+00, -1.1988e+00, 4.0000e+00, 0.0000e+00],
...,
[-1.0501e+01, 5.2707e+00, -8.7517e-01, 2.0000e+00, 4.9784e-02],
[ 1.9245e+00, -2.5313e-01, -1.8701e+00, 9.0000e+00, 4.5012e-01],
[ 7.9341e-03, -8.9713e-02, -3.8146e-03, 6.0000e+00, 0.0000e+00]])), 'gt_bboxes_3d': DataContainer(LiDARInstance3DBoxes(
tensor([[ 1.1081e+01, 2.5450e+01, -8.5619e-01, 1.9973e+00, 4.9079e+00,
1.7339e+00, -1.4964e+00, -1.4681e-04, 1.0051e-03],
[-3.6702e+00, -2.6746e+01, -2.8613e+00, 1.9312e+00, 4.6730e+00,
1.6831e+00, -1.4643e+00, -1.4377e-02, -5.0396e-05],
[ 2.3823e+00, -1.9372e+01, -2.6058e+00, 2.0685e+00, 5.1083e+00,
1.6831e+00, 8.0225e-02, -9.0984e-01, 8.4799e+00],
[ 1.3541e+01, 5.9586e+00, -1.7869e+00, 2.1916e+00, 5.6605e+00,
2.0289e+00, 1.6250e+00, -0.0000e+00, 0.0000e+00],
[ 1.4103e+01, 2.9228e+00, -1.8284e+00, 1.9587e+00, 4.8815e+00,
1.6811e+00, -1.4588e+00, -2.9825e-02, 2.1301e-02],
[ 2.7837e+01, -1.1606e+00, -2.7744e+00, 1.8041e+00, 4.3933e+00,
1.6709e+00, -1.4784e+00, -0.0000e+00, 0.0000e+00],
[ 2.0320e+01, -4.7479e+01, -4.0470e+00, 2.1428e+00, 5.0828e+00,
1.8448e+00, -1.5049e+00, 6.4712e-02, 4.2319e-02],
[-1.1707e+01, -3.6420e+00, -1.7271e+00, 1.8428e+00, 4.8377e+00,
1.6109e+00, 1.6252e+00, 5.0297e-03, 7.3593e-04],
[-2.3856e+01, -1.5737e+01, -1.8305e+00, 2.0034e+00, 5.2049e+00,
1.8428e+00, -1.5043e+00, -0.0000e+00, 0.0000e+00],
[ 9.9788e+00, 2.2827e+00, -1.7043e+00, 1.1736e+00, 8.0646e-01,
1.7980e+00, -1.7814e+00, 1.1426e-01, -4.1650e-01],
[ 3.3174e+01, 1.7922e+01, -7.9080e-01, 8.3595e-01, 9.5494e-01,
2.0339e+00, 1.0356e-01, -1.4616e-01, 1.8584e+00],
[ 1.0063e+01, 3.6501e+01, -5.3281e-01, 2.1438e+00, 4.9872e+00,
1.6394e+00, -1.5656e+00, 7.3405e-04, -5.0255e-03],
[-2.6629e+00, -3.5264e+01, -3.3719e+00, 1.9048e+00, 4.4920e+00,
1.8306e+00, -1.5323e+00, 1.1528e-03, -8.5792e-04],
[ 1.4524e+01, 5.0582e-01, -1.9272e+00, 1.9831e+00, 4.8113e+00,
1.7014e+00, -1.4749e+00, -2.2887e-02, 8.9703e-03],
[ 2.2266e+01, 4.0362e+01, -8.5761e-01, 2.9940e+00, 4.1716e+00,
1.5926e+00, 1.7194e+00, -3.0672e-02, 4.4903e-01],
[ 1.3440e+01, -2.0060e+01, -2.7484e+00, 6.2849e-01, 7.0273e-01,
1.9221e+00, -1.3003e+00, -1.4022e+00, 2.0957e-01],
[ 2.7897e+01, -6.5666e+00, -2.8059e+00, 2.1580e+00, 4.6018e+00,
1.9150e+00, 1.6458e+00, -7.3405e-04, 5.0255e-03],
[-3.0846e+00, -3.2703e+01, -3.4269e+00, 1.8590e+00, 4.4381e+00,
2.0339e+00, -1.5323e+00, -4.2140e-03, 1.4782e-02],
[ 1.2222e+01, 1.9804e+01, -1.3630e+00, 2.0614e+00, 4.4259e+00,
1.5661e+00, -1.4440e+00, -0.0000e+00, 0.0000e+00],
[-6.9963e+00, -1.9755e+01, -2.4524e+00, 6.8239e-01, 1.4441e+00,
1.2641e+00, 6.6358e-01, 1.6042e-01, -8.5346e-02],
[ 1.3135e+01, 8.7066e+00, -1.5043e+00, 2.3034e+00, 5.5181e+00,
1.8509e+00, 1.6566e+00, 1.4224e-02, -1.3959e-01],
[-4.0190e+01, 3.9265e+01, 4.1216e-01, 1.5407e+00, 3.5167e+00,
2.0339e+00, 1.2515e-01, -1.1467e-02, 2.2622e-01],
[ 2.6263e+01, 7.0396e+00, -2.2609e+00, 1.9363e+00, 4.0333e+00,
1.4156e+00, 1.6458e+00, -0.0000e+00, 0.0000e+00],
[ 1.2475e+01, 1.4027e+01, -1.3608e+00, 2.1021e+00, 5.0228e+00,
1.4065e+00, -1.5365e+00, -1.2757e-02, -6.0382e-02],
[ 1.9765e+01, -1.3621e+01, -2.6806e+00, 5.2476e-01, 6.9968e-01,
1.8519e+00, 1.5407e+00, 4.5326e-01, 1.1149e-01],
[-2.5151e+00, -3.7958e+01, -3.5373e+00, 1.9617e+00, 4.6466e+00,
1.9129e+00, -1.5323e+00, 4.0238e-03, 5.8874e-04],
[-1.4177e+01, 1.6366e+01, -8.1977e-01, 2.0614e+00, 4.6486e+00,
1.7573e+00, -1.5415e+00, 2.0119e-03, 2.9437e-04],
[ 3.1882e+01, 3.7574e+01, 1.9167e-02, 5.2476e-01, 7.3120e-01,
1.8519e+00, 5.4691e-02, -1.4233e-01, 1.3541e+00],
[-1.5008e+01, 4.2524e+01, 5.4680e-01, 3.3987e+00, 6.4303e+00,
2.9808e+00, 3.0873e+00, -1.2606e-01, 5.3561e-01],
[ 1.7001e+00, 3.5514e+01, 6.0073e-01, 2.6858e+00, 7.2144e+00,
2.1082e+00, -3.1104e+00, -0.0000e+00, 0.0000e+00],
[-4.2222e+00, -6.2725e+00, -1.2759e+00, 2.1468e+00, 4.6262e+00,
2.3868e+00, 3.0942e+00, -5.8198e-02, -1.3791e-02],
[ 1.2403e+01, -8.7635e+00, -9.2305e-01, 2.5679e+00, 7.6843e+00,
2.2902e+00, 1.6714e+00, -5.7650e-03, 8.0131e-02],
[-2.5936e+01, 7.1599e+00, 3.5442e-01, 2.9574e+00, 1.1094e+01,
4.5296e+00, 1.9989e+00, -0.0000e+00, 0.0000e+00],
[-1.8544e+01, 3.2498e+01, 1.5000e+00, 3.1852e+00, 1.4053e+01,
3.6672e+00, 1.3203e+00, 3.1848e+00, 1.4379e+00],
[-2.5963e+01, 1.4695e+01, 1.3435e+00, 2.6004e+00, 7.5327e+00,
2.8587e+00, -1.4322e+00, -9.8903e+00, -4.8384e-01],
[-3.6054e+01, 1.4393e+01, 1.7633e+00, 2.6004e+00, 7.5327e+00,
2.8587e+00, -1.4321e+00, -1.0276e+01, -8.6155e-02],
[ 4.5645e+00, -1.6092e+01, -4.2814e-01, 2.3696e+00, 1.4248e+01,
3.9550e+00, 1.1303e-01, -0.0000e+00, 0.0000e+00],
[-8.5879e+00, -2.0463e+01, -2.0144e+00, 2.2119e+00, 6.5493e-01,
1.2661e+00, -1.4716e+00, -0.0000e+00, 0.0000e+00],
[-1.4564e+01, -7.3974e+00, -1.3176e+00, 3.0855e+00, 4.0170e-01,
9.5596e-01, 1.2726e+00, -0.0000e+00, 0.0000e+00],
[ 6.9650e+00, -1.0772e+01, -1.2148e+00, 5.2578e-01, 1.3058e+00,
1.6607e+00, -2.5509e+00, -1.5342e+00, -3.7594e+00],
[-3.3463e+01, -2.9462e+01, -2.1733e+00, 1.1888e+00, 2.0940e+00,
1.9638e+00, 7.8985e-01, 4.2958e-02, 6.0744e-02],
[ 9.4685e+00, -1.9686e+01, -9.4145e-01, 7.1392e-01, 2.1072e+00,
1.6465e+00, 8.3660e-02, 6.2571e-03, 2.1583e-01],
[ 1.9394e+01, 2.1123e+01, -1.1023e-02, 5.2578e-01, 1.3058e+00,
1.6607e+00, -2.5509e+00, -1.7439e+00, -3.7068e+00],
[-3.2073e+01, 1.0891e+01, 7.2284e-01, 5.3391e-01, 1.6495e+00,
1.7248e+00, -1.3910e+00, -3.7946e+00, 9.3973e-02],
[-9.1412e+00, 2.0717e+01, 7.1289e-01, 9.4884e-01, 2.1072e+00,
1.8702e+00, 1.4224e+00, 4.5628e+00, 1.5586e+00],
[-1.2042e+01, 1.3107e+01, 2.1672e-01, 5.3391e-01, 1.6495e+00,
1.7248e+00, -1.5480e+00, -3.3165e+00, -4.4372e-01],
[-8.6158e+00, 1.3458e+01, 7.0057e-02, 5.3391e-01, 1.6495e+00,
1.7248e+00, -1.4433e+00, -3.5085e+00, -2.4859e-01],
[ 5.0856e+00, 3.1870e+01, 1.0608e-01, 3.6916e-01, 3.6713e-01,
7.2409e-01, -2.7731e+00, 3.2728e-02, -6.6736e-03],
[-1.0092e+01, -6.3476e+00, -1.6635e+00, 5.1357e-01, 4.6272e-01,
1.0526e+00, -1.5827e+00, -0.0000e+00, 0.0000e+00]]))), 'gt_labels_3d': DataContainer(tensor([0, 0, 0, 1, 0, 0, 0, 0, 0, 8, 8, 0, 0, 0, 0, 8, 0, 0, 0, 7, 1, 2, 0, 0,
8, 0, 0, 8, 2, 2, 2, 2, 3, 3, 3, 3, 4, 5, 5, 6, 6, 6, 6, 7, 7, 7, 7, 9,
9])), 'transformation_3d_flow': ['R', 'S', 'T', 'VF'], 'pcd_rotation': tensor([[ 0.9961, -0.0879, 0.0000],
[ 0.0879, 0.9961, 0.0000],
[ 0.0000, 0.0000, 1.0000]]), 'pcd_rotation_angle': -0.08799006818782418, 'pcd_scale_factor': 1.016974603680348, 'pcd_trans': array([0., 0., 0.]), 'flip': False, 'flip_direction': None, 'pcd_horizontal_flip': False, 'pcd_vertical_flip': True}
然后再将它经过那个pipeline,最后的example就是:
example
是一个只有这四个键的字典,
example['points']
是DataContainer
类,而DataContainer.data
才是tensor
类。
那么example['points'].data.shape: torch.Size([242193, 5])
所以到这里就知道它是怎么加载dataset的以及在训练的时候,每次__getitem__
的时候拿到的是什么样子的数据了。