TriggerBN +
motivation
用两个BN(一个用于干净样本, 一个用于对抗样本), 结果当使用\(\mathrm{BN}_{nat}\)的时候, 精度能够上升, 而使用\(\mathrm{BN}_{adv}\)的时候, 也有相当的鲁棒性, 但是二者不能兼得. 那么假设一个样本通过两种BN得到两个概率\(p_{nat}\)和\(p_{adv}\), 并利用
\[p = \alpha p_{nat} + (1-\alpha) p_{adv}, \quad \alpha \in [0, 1],
\]
来判断类别, 结果会如何呢 ?
注: 实验结果中, softmax为上述情形, 而non-softmax则是特征
\[f = \alpha f_{nat} + (1-\alpha) f_{adv}, \quad \alpha \in [0, 1], \\
p = \mathrm{Softmax}(f).
\]
settings
| Attribute | Value |
|---|---|
| attack | pgd-linf |
| batch_size | 128 |
| beta1 | 0.9 |
| beta2 | 0.999 |
| dataset | cifar10 |
| description | AT=0.5=default-sgd-0.1=pgd-linf-0.0314-0.25-10=128=default |
| epochs | 100 |
| epsilon | 0.03137254901960784 |
| learning_policy | [50, 75] x 0.1 |
| leverage | 0.5 |
| loss | cross_entropy |
| lr | 0.1 |
| model | resnet32 |
| momentum | 0.9 |
| optimizer | sgd |
| progress | False |
| resume | False |
| seed | 1 |
| stats_log | False |
| steps | 10 |
| stepsize | 0.25 |
| transform | default |
| weight_decay | 0.0005 |
results
x轴为\(\alpha\)从\(0\)变化到\(1\).
| Accuracy | Robustness | |
|---|---|---|
| softmax |  |
 |
| non-softmax |  |
 |
结论: 二者都具有trade-off的过程, 但是non-softmax下鲁棒性对\(\alpha\)更为敏感.
