论文拆解：SWCC

论文信息：

Jun Gao, Wei Wang, Changlong Yu, Huan Zhao, Wilfred Ng, Ruifeng Xu: Improving Event Representation via Simultaneous Weakly Supervised Contrastive Learning and Clustering. ACL 2022.

引言

第三段：存在问题

% 概述
In our work, we observe that there is a rich amount of information in co-occurring events, but previous works did not make good use of such information.

% 动机
Based on existing works on event relation extraction (Xue et al., 2016; Lee and Goldwasser, 2019; Zhang et al., 2020; Wang et al., 2020), we find that the co-occurrence relation, which refers to two events appearing in the same document, can be seen as a superset of currently defined explicit discourse relations.

% 具体
To be specific, these relations are often indicated by discourse markers (e.g., “because”, capturing the casual relation) (Lee and Goldwasser, 2019).

Therefore, two related events must exist in the same sentence or document.

More than that, the co-occurrence relation also includes other implicit event knowledge.

For example, events that occur in the same document may share the same topic and event type.

% 主流方法
To learn event representations, previous works (Granroth-Wilding and Clark, 2016; Weber et al., 2018) based on cooccurrence information usually exploit instancewise contrastive learning approaches related to the margin loss, which consists of an anchor, positive, and negative sample, where the anchor is more similar to the positive than the negative.

% 提出问题
However, they share two common limitations:

(1) such marginbased approaches struggle to capture the essential differences between events with different semantics, as they only consider one positive and one negative per anchor.

(2) Randomly sampled negative samples may contain samples semantically related to the anchor, but are undesirably pushed apart in embedding space.

This problem arises because these instance-wise contrastive learning approaches treat randomly selected events as negative samples, regardless of their semantic relevance.

第四段：本文方法

% 动机
We are motivated to address the above issues with the goal of making better use of cooccurrence information of events.

% 概述
To this end, we present SWCC: a Simultaneous Weakly supervised Contrastive learning and Clustering framework for event representation learning, where we exploit document-level co-occurrence information of events as weak supervision and learn event representations by simultaneously performing weakly supervised contrastive learning and prototype-based clustering.

% 具体：方法1
To address the first issue, we build our approach on the contrastive framework with the InfoNCE objective (van den Oord et al., 2019), which is a self-supervised contrastive learning method that uses one positive and multiple negatives.

Further, we extend the InfoNCE to a weakly supervised contrastive learning setting, allowing us to consider multiple positives and multiple negatives per anchor (as opposed to the previous works which use only one positive and one negative).

Co-occurring events are then incorporated as additional positives, weighted by a normalized co-occurrence frequency.

% 具体：方法2
To address the second issue, we introduce a prototype-based clustering method to avoid semantically related events being pulled apart.

Specifically, we impose a prototype for each cluster, which is a representative embedding for a group of semantically related events.

Then we cluster the data while enforce consistency between cluster assignments produced for different augmented representations of an event.

Unlike the instance-wise contrastive learning, our clustering method focuses on the cluster-level semantic concepts by contrasting between representations of events and clusters.

第五段：贡献总结

Overall, we make the following contributions:

% 概述：本文方法 + 实验效果

• We propose a simple and effective framework (SWCC) that learns event representations by making better use of co-occurrence information of events. Experimental results show that our approach outperforms previous approaches on several event related tasks.

% 具体：本文方法

• We introduce a weakly supervised contrastive learning method that allows us to consider multiple positives and multiple negatives, and a prototype-based clustering method that avoids semantically related events being pulled apart.

% 分析：科学发现

• We provide a thorough analysis of the prototypebased clustering method to demonstrate that the learned prototype vectors are able to implicitly capture various relations between events.