nn.MarginRankingLoss介绍

nn.MarginRankingLoss

复现论文代码中，它使用了MarginRankingLoss()函数，以下是我百度的内容：

排序损失函数

对于包含$\mathbf{N}$个样本的batch数据 $D(x_1,x_2,y)$, $x_1$,$x_2$是给定的待排序的两个输入，$y$代表真实的标签，属于{ 1 , − 1 } 。当Y = 1 是，$x_1$应该排在$x_2$前，Y = − 1 是，$x_1$应该排在$x_2$之后。

第n个样本对应的loss计算如下:

$$l_n = \max(0,-y*(x_1-x_2)+margin)$$

若$x_1$，$x_2$排序正确且$-y*(x_1-x_2)>margin$，则loss为0

class MarginRankingLoss(_Loss):
    __constants__ = ['margin', 'reduction']
    def __init__(self, margin=0., size_average=None, reduce=None, reduction='mean'):
        super(MarginRankingLoss, self).__init__(size_average, reduce, reduction)
        self.margin = margin
    def forward(self, input1, input2, target):
        return F.margin_ranking_loss(input1, input2, target, margin=self.margin, reduction=self.reduction)

pytorch中通过torch.nn.MarginRankingLoss类实现，也可以直接调用F.margin_ranking_loss 函数，代码中的size_average与reduce已经弃用。reduction有三种取值mean, sum, none，对应不同的返回ℓ ( x , y )。 默认为mean，对应于上述loss的计算

$$L=\{l_1,\dots, l_N\}$$

$$\ell(x, y)= \begin{cases}\mathrm{L}, & \text { if reduction = 'none' } \\ \frac{1}{N} \sum_{i=1}^{N} l_{i}, & \text { if reduction = 'mean' } \\ \sum_{i=1}^{N} l_{i} & \text { if reduction = 'sum' }\end{cases}$$

margin默认取0

例子：

import torch
import torch.nn.functional as F
import torch.nn as nn
import math
 
def validate_MarginRankingLoss(input1, input2, target, margin):
    val = 0
    for x1, x2, y in zip(input1, input2, target):
        loss_val = max(0, -y * (x1 - x2) + margin)
        val += loss_val
    return val / input1.nelement()
 
torch.manual_seed(10)
margin = 0
loss = nn.MarginRankingLoss()
input1 = torch.randn([3], requires_grad=True)
input2 = torch.randn([3], requires_grad=True)
target = torch.tensor([1, -1, -1])
print(target)
output = loss(input1, input2, target)
print(output.item())
 
output = validate_MarginRankingLoss(input1, input2, target, margin)
print(output.item())
 
loss = nn.MarginRankingLoss(reduction="none")
output = loss(input1, input2, target)
print(output)
 
'''
tensor([ 1, -1, -1])
0.015400052070617676
0.015400052070617676
tensor([0.0000, 0.0000, 0.0462], grad_fn=<ClampMinBackward>)
'''