Cases of Decomposition and ADMM in Federated Learning

This blog will introduce some cases in federated learning using the dual decomposition and ADMM methods in federated learning field. The innovations in the following papers are represented by:

1. A specific scene, for example, health care management, and it's usually a application or empirical study.
2. Changing the solving methods for the local or private variable.
3. Changing the communication network structure, i.e., $E$ in the decomposition problem.

It's preferred to read the previous blogs about Dual&Proximal, Operator&Optimization and Decomposition.

Federated Tensor Factorization for Computational Phenotyping

The authors applied the Federated Learning methods to extract phenotypes from EHR to support clinical decisions. There are two kinds of phenotypes: local variable patient mode and common variables feature modes. In their paper, both of them can be solved analytically.

Inexact-ADMM Based Federated Meta-Learning for Fast and Continual Edge Learning

The authors added a new regularity term to solve the catastrophic forgetting problem in the meta learning. Because of the augmented Lagriange, the optimal local variable at each iteration can be achieved by making the gradient be zero and the solution is also analytical. Meanwhile, they take the fourier expansion to speed the computation.

L-FGADMM: Layer-Wise Federated Group ADMM for Communication Efficient Decentralized Deep Learning

The authors changed the communication network and the adjacent consensus constraints supersede the global consensus constraint. In their work, client will only communicate with the two nearby neighbors. The concrete computation process was omitted in the paper.