Task
对某一类别c1,选择已有标注的样本作为support,待测样本为 query,model 以 support 为参考去分割 query。
C-way K-shot:C为类别数;分割每个 query 使用 K 个 support images
Solution
提取support 和 query 的 feature,将两者的 feature 进行比对。训练时借助其他类别的标注数据去训练model。
数据集
pascal-5i
每个sub-dataset中,trainset来自pascal trainset,testset来自pascal valset;trainset 和 testset 的类别无交集
One-Shot Learning for Semantic Segmentation (2017.09)
dataset: pascal-5i
train时,support 和 query 均从Dtrain中抽样,class相同
test时,sample N = 1000 examples
Metric:mIoU
结构:conditioning branch 输出参数 θ 用于segmentation branch 输出feature 的分类
for k-shot:对每个support单独得到的mask做logical OR
SG-One: Similarity Guidance Network for One-Shot Semantic Segmentation (2018.11)
query image 送入 Guidance Branch 和 Segmentation Branch,通过concate 来使 Guidance Branch 输出指导Segmentation Branch。
CosineSimilarty map :as guidance to teach the segmentation branch to discover the desired object regions.
for k-shot:
1、 Y is the predicted semantic label
2、use the averaged vector to guide the segmentation process
CANet: Class-Agnostic Segmentation Networks with Iterative Refinement and Attentive Few-Shot Learning (2019.03)
Dense Comparison Module:
Feature Extractor:ResNet-50,使用中间 block 的 feature(middle-level features that may
constitute object parts shared by unseen classes)
use dilated convolutions in layers after block2
Dense Comparison:concatenated feature maps go through another convolutional block with 256 3 × 3 convolutional filters for comparison.
Iterative Optimization Module
循环迭代,alternatively use predicted masks in the last epoch and empty masks as the input to IOM(dropout)
for k-shot:The result from the attention branch serves as the weight λ, λ 通过 softmax 作为 sum 权重
使用弱标签
PANet: Few-Shot Image Semantic Segmentation with Prototype Alignment (2019.08 ICCV)
与检测的 PANet 无关,重名了
support 和 query image 公用 feature extractor,对 support feature 做 mask GAP 得到 prototypes。
query feature 与 prototypes 计算 cosine distance
Prototype alignment regularization (PAR):将support 和 query 互换,taking the query and the predicted mask as the new support to learn to segment the support images.
for k-shot:所有 support 一起做 Mask GAP