PyTorch固定参数
In situation of finetuning, parameters in backbone network need to be frozen. To achieve this target, there are two steps.
First, locate the layers and change their requires_grad attributes to be False.
for param in net.backbone.parameters(): param.requires_grad = False
for pname, param in net.named_parameters(): if(key_word in pname): param.requires_grad = False
Here we use parameters() or named_parameters() method, it will give both bias and weight.
Second, filter out those parameters who need to be updated and pass them to the optimizer.
optimizer = torch.optim.SGD(filter(lambda p: p.requires_grad == True, net.parameters()), lr=learning_rate, momentum=mom)