Full Text:  <222>

CLC number: 

On-line Access: 2022-11-19

Received: 2022-06-21

Revision Accepted: 2022-10-27

Crosschecked: 0000-00-00

Cited: 0

Clicked: 116

Citations:  Bibtex RefMan EndNote GB/T7714

-   Go to

Article info.
Open peer comments

Frontiers of Information Technology & Electronic Engineering 

Accepted manuscript available online (unedited version)


Federated unsupervised representation learning


Author(s):  Fengda ZHANG, Kun KUANG, Long CHEN, Zhaoyang YOU, Tao SHEN, Jun XIAO, Yin ZHANG, Chao WU, Fei WU, Yueting ZHUANG, Xiaolin LI

Affiliation(s):  College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China; more

Corresponding email(s):  fdzhang@zju.edu.cn, kunkuang@zju.edu.cn

Key Words:  Federated learning; Unsupervised learning; Representation learning; Contrastive learning


Share this article to: More <<< Previous Paper|Next Paper >>>

Fengda ZHANG, Kun KUANG, Long CHEN, Zhaoyang YOU, Tao SHEN,Jun XIAO, Yin ZHANG, Chao WU, Fei WU, Yueting ZHUANG, Xiaolin LI. Federated unsupervised representation learning[J]. Frontiers of Information Technology & Electronic Engineering , 1998, -1(3): .

@article{title="Federated unsupervised representation learning",
author="Fengda ZHANG, Kun KUANG, Long CHEN, Zhaoyang YOU, Tao SHEN,Jun XIAO, Yin ZHANG, Chao WU, Fei WU, Yueting ZHUANG, Xiaolin LI",
journal="Frontiers of Information Technology & Electronic Engineering",
volume="-1",
number="-1",
pages="",
year="1998",
publisher="Zhejiang University Press & Springer",
doi="10.1631/FITEE.2200268"
}

%0 Journal Article
%T Federated unsupervised representation learning
%A Fengda ZHANG
%A Kun KUANG
%A Long CHEN
%A Zhaoyang YOU
%A Tao SHEN
%A Jun XIAO
%A Yin ZHANG
%A Chao WU
%A Fei WU
%A Yueting ZHUANG
%A Xiaolin LI
%J Frontiers of Information Technology & Electronic Engineering
%V -1
%N -1
%P
%@ 1869-1951
%D 1998
%I Zhejiang University Press & Springer

TY - JOUR
T1 - Federated unsupervised representation learning
A1 - Fengda ZHANG
A1 - Kun KUANG
A1 - Long CHEN
A1 - Zhaoyang YOU
A1 - Tao SHEN
A1 - Jun XIAO
A1 - Yin ZHANG
A1 - Chao WU
A1 - Fei WU
A1 - Yueting ZHUANG
A1 - Xiaolin LI
J0 - Frontiers of Information Technology & Electronic Engineering
VL - -1
IS - -1
SP -
EP -
%@ 1869-1951
Y1 - 1998
PB - Zhejiang University Press & Springer
ER -


Abstract: 
To leverage the enormous amount of unlabeled data on distributed edge devices, we formulate a new problem in federated learning called Federated Unsupervised Representation Learning (FURL) to learn a common representation model without supervision while preserving data privacy. FURL poses two new challenges: (1) data distribution shift (non-IID distribution) among clients would make local models focus on different categories, leading to the inconsistency of representation spaces. (2) without unified information among the clients in FURL, the representations across clients would be misaligned. To address these challenges, we propose the Federated Constrastive Averaging with Dictionary and Alignment (FedCA) algorithm. FedCA is composed of two key modules: (1) dictionary module to aggregate the representations of samples from each client and share with all clients for consistency of representation space; and (2) alignment module to align the representation of each client on a base model trained on a public data. We adopt the contrastive approach for local model training. Through extensive experiments with three evaluation protocols in IID and non-IID settings, we demonstrate that FedCA outperforms all baselines with significant margins.

Darkslateblue:Affiliate; Royal Blue:Author; Turquoise:Article

Reference

Open peer comments: Debate/Discuss/Question/Opinion

<1>

Please provide your name, email address and a comment





Journal of Zhejiang University-SCIENCE, 38 Zheda Road, Hangzhou 310027, China
Tel: +86-571-87952783; E-mail: cjzhang@zju.edu.cn
Copyright © 2000 - 2023 Journal of Zhejiang University-SCIENCE