Robust Graph Convolutional Networks Against Adversarial Attacks

目录

• 概
• 符号说明
• 算法

Zhu D., Zhang Z., Cui P. and Zhu W. Robust graph convolutional networks against adversarial attacks. In ACM International Conference on Knowledge Discovery and Data Mining (KDD), 2019.

概

RGCN 设计了一种 Gaussian-based 的模块, 利用 Variance 来设计 attention 用于提高网络的鲁棒性. 从我的理解来看, 它提高对抗鲁棒性的能力堪忧.

符号说明

• \(G = (V, E)\), 图;
• \(V = \{v_1, \ldots, v_N\}\), nodes;
• \(A\), 邻接矩阵;
• \(\tilde{A}\), add self-loop;
• \(\tilde{D}_{ii} = \sum_{j} \tilde{A}_{ij}\);
• \(\bm{h}_i^{(l)}\), node \(v_i\) 在第 \(l\) 层的特征表示;
• \(N(i) = \{v: (v, v_i) \in E\} \cup \{v_i\}\);
• \(\odot\), element-wise 乘法;

算法

• 初始化均值和方差特征:

  \[M^{(0)} = [\bm{\mu}_1^{(0)}, \bm{\mu}_2^{(0)}, \ldots, \bm{\mu}_N^{(0)}]^T, \\ \Sigma^{(0)} = [\bm{\sigma}_1^{(0)}, \bm{\sigma}_2^{(0)}, \ldots, \bm{\sigma}_N^{(0)}]^T, \\ \]

  假设

  \[\bm{h}_i^{(0)} \sim \mathcal{N}(\bm{\mu}_i^{(0)}, \bm{\sigma}_i^{(0)}); \]

• 第 \(l\) 层进行如下操作:

  1. 首先计算 attention:

     \[\bm{\alpha}_j^{(l)} = \exp(-\gamma \bm{\sigma}_j^{(l - 1)}), \]

     可以看出, 方差越大对应的权重越小 (因为作者认为方差越大的越容易是噪声);

  2. 计算如下该层的均值和方差特征:

     \[\bm{\mu}_i^{(l)} = \rho(\sum_{j \in N(i)} \frac{\bm{\mu}_j^{(l-1)} \odot \bm{\alpha}_j^{(l)}}{\sqrt{\tilde{D}_{ii} \tilde{D}_{jj}}} W_{\mu}^{(l)}), \\ \bm{\sigma}_i^{(l)} = \rho(\sum_{j \in N(i)} \frac{\bm{\sigma}_j^{(l-1)} \odot \bm{\alpha}_j^{(l)} \odot \bm{\alpha}_j^{(l)}}{\sqrt{\tilde{D}_{ii} \tilde{D}_{jj}}}W_{\sigma}^{(l)}); \]

     此时

     \[\bm{h}_i^{(l)} \sim \mathcal{N}(\bm{\mu}_i^{(l)}, \bm{\sigma}_i^{(l)}); \]

  3. 用矩阵表示为如下结果:

     \[M^{(l)} = \rho(\tilde{D}^{-1/2} \tilde{A} \tilde{D}^{-1/2} (M^{(l-1)} \odot \mathcal{A}^{(l)}) W_{\mu}^{(l)}), \\ \Sigma^{(l)} = \rho(\tilde{D}^{-1/2} \tilde{A} \tilde{D}^{-1/2} (\Sigma^{(l-1)} \odot \mathcal{A}^{(l)} \odot \mathcal{A}^{(l)}) W_{\sigma}^{(l)}); \]

• 最后一层, 我们得到最后的输出:

  \[\bm{z}_i = \bm{\mu}_i^{(L)} + \bm{\epsilon} \odot \sqrt{\bm{\sigma}_i^{(L)}}, \: \bm{\epsilon} \sim \mathcal{N}(\bm{0}, I); \]

• 最后通过如下损失进行训练:

  \[\mathcal{L} = \mathcal{L}_{cls} + \beta_1 \mathcal{L}_{reg1} + \beta_2 \mathcal{L}_{reg2}, \]

  其中 \(\mathcal{L}_{cls}\) 就是普通的分类损失 (基于 \(\bm{z}_i\)), 然后

  \[\mathcal{L}_{reg1} = \sum_{i=1}^N \text{KL}(\mathcal{N}(\bm{\mu}_i^{(1)}, \bm{\sigma_i}^{(1)}) \| \mathcal{N}(\bm{0}, I)) \]

  确保第一层的输出是正态分布?

  \[\mathcal{L}_{reg2} = \|W_{\mu}^{(0)}\|_2^2 + \|W_{\sigma}^{(0)}\|_2^2 \]

  为普通的对第一层的 L2 正则.