DeeperGCN All You Need to Train Deeper GCNs
概
本文介绍了一种连续可微的 aggregation function (极限状态下能够 cover 常用的 mean, max).
符号说明
- \(\mathcal{G} = (\mathcal{V}, \mathcal{E})\), 图;
- \(\mathbf{h}_v \in \mathbb{R}^D\), node features;
- \(\mathbf{h}_e \in \mathbb{R}^C\), edge features;
- \(\mathcal{N}(v)\), node \(v\) 的一阶邻居.
广义的 aggregation function
-
一般的 GCN 每一层需要历经:
\[\mathbf{m}_{vu}^{(l)} = \bm{\rho}^{(l)} (\mathbf{h}_v^{(l)}, \mathbf{h}_v^{(l)}, \mathbf{h}_{e_{vu}}^{(l)}), \: u \in \mathcal{N}(v), \\ \mathbf{m}_v^{(l)} = \bm{\zeta}^{(l)}(\{\mathbf{m}_{vu}^{(l)}| u \in \mathcal{N}(v)\}), \\ \mathbf{h}_v^{(l+1)} = \bm{\phi}^{(l)} (\mathbf{h}_v^{(l)}, \mathbf{m}_v^{(l)}). \]其中 \(\bm{\rho}, \bm{\zeta}, \bm{\phi}\) 为可训练的函数. 这里我们主要关注 \(\bm{\zeta}\).
-
SoftMax Aggregation:
\[\bm{\zeta}_{\beta}(\{\mathbf{m}_{vu}^{(l)}| u \in \mathcal{N}(v)\}) = \sum_{u \in \mathcal{N}_v} \frac{\exp(\beta \mathbf{m}_{vu})}{\sum_{i \in \mathcal{N}(v)} \exp(\beta \mathbf{m}_{vi})} \cdot \mathbf{m}_{vu}. \]显然当 \(\beta \rightarrow 0\) 的时候它退化为 \(\text{Mean}(\cdot)\), 当 \(\beta \rightarrow +\infty\) 的时候退化为 \(\text{Max}(\cdot)\).
-
PowerMean Aggregation:
\[\bm{\zeta}_{p}(\{\mathbf{m}_{vu}^{(l)}| u \in \mathcal{N}(v)\}) =(\frac{1}{|\mathcal{N}_v|} \sum_{u \in \mathcal{N}_v} \mathbf{m}_{vu}^p)^{1/p}, \]当 \(p = 1\) 的时候, 退化为 \(\text{Mean}(\cdot)\), 当 \(p \rightarrow +\infty\) 的时候退化为 \(\text{Max}(\cdot)\), 当 \(p=-1, p \rightarrow 0\) 的时候分别退化为 harmonic 和 geometric mean aggregation.
