【论文理解】Learning Multiagent Communication with Backpropagation

Learning Multiagent Communication with Backpropagation
Sainbayar Sukhbaatar, Arthur Szlam, Rob Fergus 

---

Many tasks in AI require the collaboration of multiple agents. Typically, the communication protocol between agents is manually specified and not altered during training. In this paper we explore a simple neural model, called CommNet, that uses continuous communication for fully cooperative tasks. The model consists of multiple agents and the communication between them is learned alongside their policy. We apply this model to a diverse set of tasks, demonstrating the ability of the agents to learn to communicate amongst themselves, yielding improved performance over non-communicative agents and baselines. In some cases, it is possible to interpret the language devised by the agents, revealing simple but effective strategies for solving the task at hand.

 

人机对抗中多agent通信的模型，可以做baseline。核心就是上图，右边是整个框架，s1...代表是状态，a1...代表是输出动作，灰色的表示agent，每一层之间有通信；中间是层与层之间的通信，每个agent获得的reward和其它agent不相关，fi指的是通道channel；左边是单个通道对应单个agent的具体输入输出示意图，C指的是concatenation，连接即指的是通信，H指的是hidden state，即状态，输出的是下一层的状态。