点云分割学习路径

1. 传统点云分割

点云分割是根据空间、几何和纹理等特征对点云进行划分，使得同一划分内的点云拥有相似的特征。点云的有效分割是许多应用的前提，例如在三维重建领域，需要对场景内的物体首先进行分类处理，然后才能进行后期的识别和重建。
传统的点云分割主要依赖聚类算法和基于随机采样一致性的分割算法，在很多技术上得到了广泛应用，但当点云规模不断增大时，传统的分割算法已经很难满足实际需要，这时就需要结合深度学习进行分割。

2.点云分割网络

PointNet/PointNet++

PCT网络（Point Cloud Transformer）

Cylinder网络

 Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation

JSNet网络

JSNet: Joint Instance and Semantic Segmentation of 3D Point Clouds

同时解决实例和语义分割问题

 

常用数据集：

　　1. Semantic3D

经典的大型室外场景点云分割数据集，由激光雷达扫描周围场景得到。Semantic3D提供了一个带有大标签的自然场景的3D点云数据集，总计超过40亿个点，8个类别标签。
数据集包含了各种城市和乡村场景，如农场，市政厅，运动场，城堡和广场。该数据集包含15个训练数据集和15个测试数据集，另外还包括4个缩减了的测试数据集。
数据集中的点都含有RGB和深度信息，并被标记为8个语义类别，分别是1：人造地形；2：自然地形；3：高植被；4：低植被；5：建筑物；6：硬景观；7：扫描人工制品，8：汽车，附加标签0：未标记点，标记没有地面真值的点。
数据集地址：http://www.semantic3d.net/

　　2. S3DIS

S3DIS数据集是斯坦福大学开发的带有像素级语义标注的语义数据集，是常用的室内场景分割数据集，使用Matterport相机收集数据，包含6个Area，13个语义元素，11种场景。
其中13个语义元素分别包括：天花板ceiling、地板floor、墙壁wall、梁beam、柱column、窗window、门door、桌子table、椅子chair、沙发sofa、书柜bookcase、板board、混杂元素(其他)clutter；
11种场景分别包括办公室office、会议室conference room、走廊hallway、礼堂auditorium、开放空间open space、大堂lobby、休息室lounge、储藏室pantry、复印室copy room、储藏室storage和卫生间WC。
数据集地址：http://buildingparser.stanford.edu/dataset.html

　　3.SemanticKITTI

SemanticKITTI数据集是一个基于KITTI Vision Benchmark里程计数据集的大型户外点云数据集，显示了市中心的交通、住宅区，以及德国卡尔斯鲁厄周围的高速公路场景和乡村道路。
原始里程计数据集由22个序列组成，作者将序列00到10拆分为训练集，将11到21拆分为测试集，并且为了与原始基准保持一致，作者对训练和测试集采用相同的划分，采用和KITTI数据集相同的标定方法，这使得该数据集和KITTI数据集等数据集可以通用。
SemanticKITTI数据集作者提供了精确的序列扫描注释，并且在点注释中显示了前所未有的细节，包含28个类，确保了类与Mapillary Visiotas数据集和Cityscapes数据集有很大的重叠，并在必要时进行了修改，以考虑稀疏性和垂直视野。
数据集地址：http://www.semantic-kitti.org/index.html

　　4. ShapeNet

ShapeNet数据集是一个由对象的三维CAD模型表示的形状存储库，注释丰富，规模较大。ShapeNet包含来自多种语义类别的3D模型，并按照WordNet分类法组织，能够完成部件分割任务，即不仅知道这个点云数据大的分割，还要将它的小部件进行分割。
它总共包括十六个大的类别，每个大的类别有可以分成若干个小类别，十六个类别具体包括：飞机Airplane、包Bag、帽子Cap、汽车Car、椅子Chair、耳机Earphone、吉他Guitar、刀Knife、灯Lamp、电脑Laptop、摩托车Motorbike、杯子Mug、手枪Pistol、火箭Rocket、滑板Skateboard、桌子Table。
数据集地址：https://www.shapenet.org/

　　5. PartNet

 PartNet数据集是用于细粒度和分层零件级3D对象理解的大规模基准。数据集包含573585个零件实例，涵盖26671个3D模型，涵盖24个对象类别。
PartNet数据集启用并充当许多任务的催化剂，例如形状分析，动态3D场景建模和仿真，可负担性分析等。数据集建立了用于评估3D零件识别的三个基准测试任务：细粒度语义分割，分层语义分割和实例分割。
数据集地址：https://shapenet.org/download/parts

 

 必读论文：

[1] PointNet: Deep learning on point sets for 3d classification and segmentation. CVPR 2017.

[2] Pointnet++: Deep hierarchical feature learning on point sets in a metric space. NeurIPS 2017.

[3] Deep learning for 3d point clouds: A survey. TPAMI 2020.

[4] Semantickitti: A dataset for semantic scene understanding of lidar sequences. ICCV2019.

[5] Scannet: Richly-annotated 3d reconstructions of indoor scenes. CVPR2017.

[6] Towards Semantic Segmentation of Urban-Scale 3D Point Clouds: A Dataset, Benchmarks and Challenges. CVPR 2021.

[7] Transfer Learning from Synthetic to Real LiDAR Point Cloud for Semantic Segmentation. AAAI 2022.

[8] Cylindrical and asymmetrical 3d convolution networks for lidar segmentation. CVPR2021.

[9] Rangenet++: Fast and accurate lidar semantic segmentation. IROS 2019.

[10] 4d spatio-temporal convnets: Minkowski convolutional neural networks. CVPR2019.

 

 

Traditinal methods

• PyramNet: Point cloud pyramid attention network and graph embedding module for classification and segmentation
• Fast semantic segmentation of 3D point clouds using a dense CRF with learned parameters
• Shape-based recognition of 3d point clouds in urban environments
• Fast semantic segmentation of 3d point clouds with strongly varying density
• Semantic point cloud interpretation based on optimal neighborhoods, relevant features and efficient classifiers
• Discriminative learning of markov random fields for segmentation of 3D scan data
• Robust 3D scan point classification using associative markov networks
• Contextual classification with functional max-margin markov networks

Point-based methods

Point-wise shared MLP

• PointNet: Deep learning on point sets for 3D classification and segmentation
• PointNet++: Deep hierarchical feature learning on point sets in a metric space
• PointSIFT: A SIFT-like network module for 3D point cloud semantic segmentation
• Know what your neighbors do: 3D semantic segmentation of point clouds
• RandLA-Net: Efficient semantic segmentation of large-scale point clouds
• Modeling point clouds with self-attention and gumbel subset sampling
• LSANet: Feature learning on point sets by local spatial attention
• PyramNet: Point cloud pyramid attention network and graph embedding module for classification and segmentation

Point Convolution

• PointCNN: Convolution on x-transformed points
• A-CNN: Annularly convolutional neural networks on point clouds
• KPConv: Flexible and deformable convolution for point clouds
• Dilated point convolutions: On the receptive field of point convolutions
• PointAtrousNet: Point atrous convolution for point cloud analysis
• PointAtrousGraph: Deep hierarchical encoder-decoder with atrous convolution for point clouds
• Tangent convolutions for dense prediction in 3D
• DAR-Net: Dynamic aggregation network for semantic scene segmentation
• ShellNet: Efficient point cloud convolutional neural networks using concentric shells statistics
• Point-voxel cnn for efficient 3D deep learning

Recurrent Neural Networ

• Exploring spatial context for 3D semantic segmentation of point clouds
• Recurrent slice networks for 3D segmentation of point clouds

Lattice Convolution

• SplatNet: Sparse lattice networks for point cloud processing
• LatticeNet: Fast point cloud segmentation using permutohedral lattices

Voxel-based methods

• Sparse single sweep lidar point cloud segmentation via learning contextual shape priors from scene completion paper, code
• Point cloud labeling using 3D convolutional neural network
• Segcloud: Semantic segmentation of 3D point cloud
• Fully-convolutional point networks for large-scale point clouds
• 3DCNN-DQN-RNN: A deep reinforcement learning framework for semantic parsing of large-scale 3D point clouds
• 3D semantic segmentation with submanifold sparse convolutional networks
• Efficient convolutions for real-time semantic segmentation of 3D point clouds
• VV-Net: Voxel vaenet with group convolutions for point cloud segmentation
• VolMap: A real-time model for semantic segmentation of a LiDAR surrounding view

Image-based methods

Range view-based methods

• SqueezeSeg: Convolutional neural nets with recurrent CRF for real-time road-object segmentation from 3D LiDAR point cloud
• SqueezeSegV2: Improved model structure and unsupervised domain adaptation for road-object segmentation from a LiDAR point cloud
• SqueezeSegV3: Spatially-adaptive convolution for efficient point-cloud segmentation
• Semantic segmentation of 3D LiDAR data in dynamic scene using semi-supervised learning
• RangeNet++: Fast and accurate LiDAR semantic segmentation
• LU-Net: An efficient network for 3D LiDAR point cloud semantic segmentation based on end-to-end-learned 3D features and U-Net
• 3D-MiniNet: Learning a 2D representation from point clouds for fast and efficient 3D LiDAR semantic segmentation
• DeepTemporalSeg: Temporally consistent semantic segmentation of 3D LiDAR scans
• LiSeg: Lightweight road-object semantic segmentation in 3D LiDAR scans for autonomous driving
• PointSeg: Real-time semantic segmentation based on 3D LiDAR point cloud
• RIU-Net: Embarrassingly simple semantic segmentation of 3D LiDAR point cloud
• SalsaNet: Fast road and vehicle segmentation in LiDAR point clouds for autonomous driving
• SalsaNext: Fast,uncertainty-aware semantic segmentation of LiDAR point clouds

Multi view-based methods

• Deep projective 3D semantic segmentation
• Unstructured point cloud semantic labeling using deep segmentation networks

Graph-based Methods

• Large-scale point cloud semantic segmentation with superpoint graphs
• Graph attention convolution for point cloud semantic segmentation
• Hierarchical point-edge interaction network for point cloud semantic segmentation
• Dynamic graph CNN for learning on point clouds

Survey

• Linking Points With Labels in 3D: A Review of Point Cloud Semantic Segmentation paper
• Are We Hungry for 3D LiDAR Data for Semantic Segmentation? A Survey and Experimental Study paper
• A Technical Survey and Evaluation of Traditional Point Cloud Clustering Methods for LiDAR Panoptic Segmentation paper
• A survey on deep learning-based precise boundary recovery of semantic segmentation for images and point clouds [paper] (https://www.sciencedirect.com/science/article/pii/S0303243421001185)