Graph-Based Social Relation Reasoning

title: Graph-Based Social Relation Reasoning, 2020

task: we propose a simpler, faster, and more accurate method named graph relational reasoning network (GR2N) for social relation recognition.

abstract: Understanding social relations from an image has great potential for intelligent systems such as social chatbots and personal assistants.  Different from existing methods that process all social relations on an image independently, our method considers the paradigm of jointly inferring the relations by constructing a social relation graph.  Furthermore, the proposed GR2N constructs several virtual relation graphs to explicitly grasp the strong logical constraints among different types of social relations.

通过图像来理解社会关系对于智能系统，如社交聊天机器人和个人助理有着巨大的潜力。不同于现有的在一个图像上独立处理所有社会关系的方法，我们的方法考虑了通过构造一个社会关系图来共同推断关系的范式。此外，所提出的GR2N构造了若干虚拟关系图，以显式地把握不同类型社会关系之间的强逻辑约束。

由于潜在的隐私风险警告等广泛的应用，人们对在给定的静止图像中理解人与人之间的关系越来越感兴趣, 智能自主系统[52]，群活性分析[19]。

由于社会关系通常形成一个合理的社会场景，它们不是相互独立的，而是高度相关的。 独立地预测同一图象上的关系，需要从社会场景的高局部性出发，这可能会导致社会关系图的不合理和矛盾。（Independently predicting the relations on the same image suffers from the high locality in social scenes, which may result in an unreasonable and contradictory social relation graph.）

为此，我们认为，共同推断每个图像的所有关系有助于构建一个合理的、一致的社会关系图，同时对社会场景有一个透彻的理解。

To this end, we consider that jointly inferring all relations for each image helps construct a reasonable and consistent social relation graph with a thorough understanding of the social scene. 

此外，由于同一图像上的社会关系往往遵循较强的逻辑约束 logical constraints,，同时考虑所有关系可以有效地利用这些关系的一致性。

显然，同一图像上的关系在推理中是相互帮助的，这在现有的方法中并没有作为一个重要的线索加以利用。

we propose a graph relational reasoning network (GR2N)

现有的gnn方法大多只是通过消息传递来利用上下文信息，无法明确把握不同类型社会关系之间的逻辑约束。(Most existing GNNs' methods simply exploit contextual information via message passing, which fails to explicitly grasp the logical constraints among different types of social relations.) 

为了利用强逻辑约束，提出的GR2N用共享节点表示为不同的关系类型构造不同的虚拟关系图。(To exploit the strong logical constraints, the proposed GR2N constructs different virtual relation graphs for different relation types with shared node representations.)

我们的方法在每个虚拟关系图上学习特定于类型的消息，并通过汇总所有虚拟关系图上的所有邻居消息来更新节点表示。 最后，节点的最终表示可用来预测图上所有节点对的关系。Our method learns type-specificc messages on each virtual relation graph and updates the node representations by aggregating all neighbor messages across all virtual relation graphs. In the end, the final representations of nodes are utilized to predict the relations of all pairs of nodes on the graph.