A survey on federated learning in crowd intelligence

Qiang YANG; Yongxin TONG; Yansheng WANG; Lixin FAN; Wei WANG; Lei CHEN; Wei WANG; Yan KANG

doi:10.11959/j.issn.2096-6652.202218

您当前的位置：

首页 >

文章列表页 >

A survey on federated learning in crowd intelligence

Special Topic: Crowd Intelligence | 更新时间：2024-06-05

- A survey on federated learning in crowd intelligence
- Chinese Journal of Intelligent Science and Technology Vol. 4, Issue 1, Pages: 29-44(2022)
- 作者机构：
  
  1. 深圳前海微众银行股份有限公司，广东深圳 518063
  2. 香港科技大学，香港 999077
  3. 北京航空航天大学，北京 100191
  4. 南京大学，江苏南京 210033
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-6652.202218
  CLC： TP39
- Published Online：2022-03，
  
  Published：15 March 2022
- 稿件说明：
移动端阅览
Qiang YANG, Yongxin TONG, Yansheng WANG, et al. A survey on federated learning in crowd intelligence[J]. Chinese journal of intelligent science and technology, 2022, 4(1): 29-44.
DOI：

Qiang YANG, Yongxin TONG, Yansheng WANG, et al. A survey on federated learning in crowd intelligence[J]. Chinese journal of intelligent science and technology, 2022, 4(1): 29-44. DOI： 10.11959/j.issn.2096-6652.202218.

摘要

群体智能是在互联网高速普及下诞生的人工智能新范式。然而，数据孤岛与数据隐私保护问题导致群体间数据共享困难，群体智能应用难以构建。联邦学习是一类新兴的打破数据孤岛、联合构建群智模型的重要方法。首先，介绍了联邦学习的基础概念以及其与群体智能的关系；其次，基于群体智能视角对联邦学习算法框架进行了分类，从隐私、精度与效率3个角度讨论了联邦学习算法优化技术；而后，阐述了基于线性模型、树模型与神经网络模型的联邦学习算法模型；最后，介绍了联邦学习代表性开源平台与典型应用，并对联邦学习研究进行总结展望。

Abstract

Crowd intelligence is emerging as a new artificial intelligence paradigm owing to the rapid development of the Internet.However

the data isolation and data privacy preservation problems make it difficult to share data among the crowd and to build crowd intelligent applications.Federated learning is a novel solution that aims to collaboratively build models by breaking the data barriers in crowd.Firstly

the basic ideas of federated learning and a comparison with crowd intelligence were introduced.Secondly

federated learning algorithms were divided into three categories according to the crowd organization

and further optimization techniques on privacy

accuracy and efficiency were discussed.Thirdly

federated learning operators based on linear models

tree models and neural network models were presented respectively.Finally

mainstream federated learningopensource platforms and typical applications were introduced

followed by the conclusion.

关键词

Keywords

references

KONEČNÝ J , MCMAHAN H B , YU F X , et al . Federated learning:strategies for improving communication efficiency [J ] . arXiv preprint , 2016 ,arXiv:1610.05492.

MCMAHAN H B , MOORE E , RAMAGE D , et al . Communication-efficient learning of deep networks from decentralized data [J ] . arXiv preprint , 2016 ,arXiv:1602.05629.

YANG Q , LIU Y , CHEN T J , et al . Federated machine learning [J ] . ACM Transactions on Intelligent Systems and Technology , 2019 , 10 ( 2 ): 1 - 19 .

YANG Q , LIU Y , CHENG Y , et al . Federated learning-synthesis lectures on artificial intelligence and machine learning [M ] .[S.l:s.n. ] , 2019 .

YANG Q , FAN L X , YU H . Federated learning [M ] . Cham : Springer International Publishing , 2020 .

杨强 , 刘洋 , 程勇 , 等 . 联邦学习 [M ] . 北京 : 电子工业出版社 , 2020 .

YANG Q , LIU Y , CHENG Y , et al . Federated learning [M ] . Beijing : Publishing House of Electronics Industry , 2020 .

杨强 , 黄安埠 , 刘洋 , 等 . 联邦学习实战 [M ] . 北京 : 电子工业出版社 , 2021 .

YANG Q , HUANG A B , LIU Y , et al . Practicing federated learning [M ] . Beijing : Publishing House of Electronics Industry , 2021 .

杨强 , 刘洋 , 陈天健 , 等 . 联邦学习 [J ] . 中国计算机学会通讯 , 2018 , 14 ( 11 ): 49 - 55 .

YANG Q , LIU Y , CHEN T J , et al . Federated learning [J ] . Communications of the CCF , 2018 , 14 ( 11 ): 49 - 55 .

REYNOLDS C W , . Flocks,herds and schools:a distributed behavioral model [C ] // Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques . New York:ACM Press , 1987 : 25 - 34 .

PANDEY S R , TRAN N H , BENNIS M , et al . A crowdsourcing framework for on-device federated learning [J ] . IEEE Transactions on Wireless Communications , 2020 , 19 ( 5 ): 3241 - 3256 .

SINDINGER T S . Crowdsourcing:why the power of the crowd is driving the future of business [J ] . Human Resource Management International Digest , 2010 , 18 ( 3 ): 11 - 16 .

TONG Y X , WANG Y S , SHI D Y . Federated learning in the lens of crowdsourcing [J ] . Data Engineering , 2020 :26.

FINKEL J R , MANNING C D . Joint parsing and named entity recognition [C ] // Proceedings of 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics . Morristown:Association for Computational Linguistics , 2009 .

RICH C . Multitask learning [J ] . Machine Learning , 1997 , 28 ( 1 ): 41 - 75 .

ZHANG Y , YANG Q . An overview of multi-task learning [J ] . National Science Review , 2017 , 5 ( 1 ): 30 - 43 .

LI M , . Scaling distributed machine learning with the parameter server [C ] // Proceedings of 2014 International Conference on Big Data Science and Computing . New York:ACM Press , 2014 : 583 - 598 .

SATTLER F , MÜLLER K R , SAMEK W . Clustered federated learning:model-agnostic distributed multitask optimization under privacy constraints [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2020 , 32 ( 8 ): 3710 - 3722 .

ABAD M S H , OZFATURA E , GUNDUZ D , et al . Hierarchical federated learning ACROSS heterogeneous cellular networks [C ] // Proceedings of 2020 IEEE International Conference on Acoustics,Speech and Signal Processing . Piscataway:IEEE Press , 2020 : 8866 - 8870 .

GHOSH A , CHUNG J , YIN D , et al . An efficient framework for clustered federated learning [J ] . Advances in Neural Information Processing Systems , 2020 , 33 : 19586 - 19597 .

LIM W Y B , NG J S , XIONG Z H , et al . Decentralized edge intelligence:a dynamic resource allocation framework for hierarchical federated learning [J ] . IEEE Transactions on Parallel and Distributed Systems , 2022 , 33 ( 3 ): 536 - 550 .

WARNAT-HERRESTHAL S , SCHULTZE H , SHASTRY K L , et al . Swarm learning for decentralized and confidential clinical machine learning [J ] . Nature , 2021 , 594 ( 7862 ): 265 - 270 .

KIM H , PARK J , BENNIS M , et al . Blockchained on-device federated learning [J ] . IEEE Communications Letters , 2020 , 24 ( 6 ): 1279 - 1283 .

RIVEST R L , ADLEMAN L M , DERTOUZOS M L . On data banks and privacy homomorphisms [J ] . Foundations of Secure Computation , 1978 , 4 ( 11 ): 169 - 180 .

PHONG L T , AONO Y , HAYASHI T , et al . Privacy-preserving deep learning via additively homomorphic encryption [J ] . IEEE Transactions on Information Forensics and Security , 2018 , 13 ( 5 ): 1333 - 1345 .

HAO M , LI H W , LUO X Z , et al . Efficient and privacy-enhanced federated learning for industrial artificial intelligence [J ] . IEEE Transactions on Industrial Informatics , 2020 , 16 ( 10 ): 6532 - 6542 .

CHEN Y Q , QIN X , WANG J D , et al . FedHealth:a federated transfer learning framework for wearable healthcare [J ] . IEEE Intelligent Systems , 2020 , 35 ( 4 ): 83 - 93 .

ZHANG C L , LI S Y , XIA J Z , et al . Batchcrypt:efficient homomorphic encryption for cross-silo federated learning [C ] // Proceedings of 2020 USENIX Conference on Usenix Annual Technical Conference .[S.l.:s.n. ] , 2020 : 493 - 506 .

YAO A C , . Protocols for secure computations [C ] // Proceedings of the 23rd Annual Symposium on Foundations of Computer Science . Piscataway:IEEE Press , 1982 : 160 - 164 .

COCK M D , DOWSLEY R , NASCIMENTO A C A , et al . Privacypreserving classification of personal text messages with secure multi-party computation:an application to hate-speech detection [J ] . Advances in Neural Information Processing Systems 32 , 2019 :3752.

SHARMA S , XING C P , LIU Y , et al . Secure and efficient federated transfer learning [C ] // Proceedings of 2019 IEEE International Conference on Big Data . Piscataway:IEEE Press , 2019 : 2569 - 2576 .

ZHU H F , MONG GOH R S , NG W K . Privacy-preserving weighted federated learning within the secret sharing framework [J ] . IEEE Access , 2020 , 8 : 198275 - 198284 .

SHAMIR A . How to share a secret [J ] . Communications of the ACM , 1979 , 22 ( 11 ): 612 - 613 .

BONAWITZ K , IVANOV V , KREUTER B , et al . Practical secure aggregation for privacy-preserving machine learning [C ] // Proceedings of 2017 ACM SIGSAC Conference on Computer and Communications Security . New York:ACM Press , 2017 : 1175 - 1191 .

XU G W , LI H W , LIU S , et al . VerifyNet:secure and verifiable federated learning [J ] . IEEE Transactions on Information Forensics and Security , 2020 , 15 : 911 - 926 .

GAO D S , LIU Y , HUANG A B , et al . Privacy-preserving heterogeneous federated transfer learning [C ] // Proceedings of 2019 IEEE International Conference on Big Data . Piscataway:IEEE Press , 2019 : 2552 - 2559 .

LIU Y , MA Z , YAN Z , et al . Privacy-preserving federated K-means for proactive caching in next generation cellular networks [J ] . Information Sciences , 2020 , 521 : 14 - 31 .

DWORK C , . Differential privacy [C ] // Proceedings of the International Colloquium on Automata,Languages,and Programming . Heidelberg:Springer , 2006 : 1 - 12 .

GEYER R C , KLEIN T , NABI M . Differentially private federated learning:a client level perspective [J ] . arXiv preprint , 2017 ,arXiv:1712.07557.

MCMAHAN H B , RAMAGE D , TALWAR K , et al . Learning differentially private recurrent language models [J ] . arXiv preprint , 2017 ,arXiv:1710.06963.

WANG Y S , TONG Y X , SHI D Y , et al . An efficient approach for cross-silo federated learning to rank [C ] // Proceedings of 2021 IEEE 37th International Conference on Data Engineering . Piscataway:IEEE Press , 2021 : 1128 - 1139 .

SHI Y X , TONG Y X , SU Z Y , et al . Federated topic discovery:a semantic consistent approach [J ] . IEEE Intelligent Systems , 2020 .

JIANG D , TONG Y X , SONG Y F , et al . Industrial federated topic modeling [J ] . ACM Transactions on Intelligent Systems and Technology , 2021 , 12 ( 1 ): 1 - 22 .

JIANG D , SONG Y F , TONG Y X , et al . Federated topic modeling [C ] // Proceedings of the 28th ACM International Conference on Information and Knowledge Management . New York:ACM Press , 2019 : 1071 - 1080 .

HAO M , LI H W , XU G W , et al . Towards efficient and privacy-preserving federated deep learning [C ] // Proceedings of 2019 IEEE International Conference on Communications . Piscataway:IEEE Press , 2019 : 1 - 6 .

ABADI M , CHU A , GOODFELLOW I , et al . Deep learning with differential privacy [C ] // Proceedings of 2016 ACM SIGSAC Conference on Computer and Communications Security . New York:ACM Press , 2016 : 308 - 318 .

BHOWMICK A , DUCHI J , FREUDIGER J , et al . Protection against reconstruction and its applications in private federated learning [J ] . arXiv preprint , 2018 ,arXiv:1812.00984.

LU Y L , HUANG X H , DAI Y Y , et al . Differentially private asynchronous federated learning for mobile edge computing in urban informatics [J ] . IEEE Transactions on Industrial Informatics , 2020 , 16 ( 3 ): 2134 - 2143 .

WANG Y S , TONG Y X , SHI D Y . Federated latent dirichlet allocation:a local differential privacy based framework [J ] . Proceedings of the AAAI Conference on Artificial Intelligence.[S.l.:s.n.] , 2020 , 34 ( 4 ): 6283 - 6290 .

MOHRI M , SIVEK G , SURESH A T . Agnostic federated learning [C ] // Proceedings of the 36th International Conference on Machine Learning .[S.l.:s.n. ] , 2019 , 4615 - 4625 .

MANSOUR Y , MOHRI M , RO J , et al . Three approaches for personalization with applications to federated learning [J ] . arXiv preprint , 2020 ,arXiv:2002.10619.

DENG Y , KAMANI M M , MAHDAVI M . Adaptive personalized federated learning [J ] . arXiv preprint , 2020 ,arXiv:2003.13461.

LI T , SAHU A K , ZAHEER M , et al . Federated optimization in heterogeneous networks [J ] . arXiv preprint , 2018 ,arXiv:1812 .06127.

KARIMIREDDY S P , KALE S , MOHRI M , et al . SCAFFOLD:stochastic controlled averaging for federated learning [C ] // Proceeding of the International Conference on Machine Learning .[S.l.:s.n. ] , 2020 : 5132 - 5143 .

ROTHCHILD D , PANDA A , ULLAH E , et al . FetchSGD:communication-efficient federated learning with sketching [C ] // Proceedings of the 37th International Conference on Machine Learning .[S.l.:s.n. ] , 2020 : 8253 - 8265 .

HAMER J , MOHRI M , SURESH A T . FedBoost:a communication-efficient algorithm for federated learning [C ] // Proceedings of the 37th International Conference on Machine Learning .[S.l.:s.n. ] , 2020 : 3973 - 3983 .

SURESH A T , YU F X , KUMAR S , et al . Distributed mean estimation with limited communication [C ] // Proceedings of 2017 International Conference on Machine Learning .[S.l.:s.n. ] , 2017 : 3329 - 3337 .

CALDAS S , KONEČNY J ,, MCMAHAN H B , et al . Expanding the reach of federated learning by reducing client resource requirements [J ] . arXiv preprint , 2018 ,arXiv:1812.07210.

XU J J , DU W L , JIN Y C , et al . Ternary compression for communication-efficient federated learning [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2022 , 33 ( 3 ): 1162 - 1176 .

HADDADPOUR F , KAMANI M M , MOKHTARI A , et al . Federated learning with compression:unified analysis and sharp guarantees [C ] // Proceedings of the International Conference on Artificial Intelligence and Statistics .[S.l.:s.n. ] , 2021 : 2350 - 2358 .

CUI L Z , SU X X , MING Z X , et al . CREAT:blockchain-assisted compression algorithm of federated learning for content caching in edge computing [J ] . IEEE Internet of Things Journal , 2020 , 4370 ( 99 ): 1 .

WEI S Y , TONG Y X , ZHOU Z M , et al . Efficient and fair data valuation for horizontal federated learning [M ] // Federated learning . Cham : Springer , 2020 .

CHAI Z , ALI A , ZAWAD S , et al . TiFL:a tier-based federated learning system [C ] // Proceedings of the 29th International Symposium on High-Performance Parallel and Distributed Computing . New York:ACM Press , 2020 : 125 - 136 .2020.

HUANG T S , LIN W W , WU W T , et al . An efficiency-boosting client selection scheme for federated learning with fairness guarantee [J ] . IEEE Transactions on Parallel and Distributed Systems , 2021 , 32 ( 7 ): 1552 - 1564 .

LAI F , ZHU X F , MADHYASTHA H , et al . Oort:informed participant selection for scalable federated learning [J ] . arXiv preprint , 2020 ,arXiv:2010.06081.

WANG H , KAPLAN Z , NIU D , et al . Optimizing federated learning on non-IID data with reinforcement learning [C ] // Proceedings of 2020 IEEE Conference on Computer Communications . Piscataway:IEEE Press , 2020 : 1698 - 1707 .

SONG T S , TONG Y X , WEI S Y . Profit allocation for federated learning [C ] // Proceedings of 2019 IEEE International Conference on Big Data . Piscataway:IEEE Press , 2019 : 2577 - 2586 .

HARDY S , HENECKA W , IVEY-LAW H , , et al . Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption [J ] . arXiv preprint , 2017 ,arXiv:1711.10677.

KONEČNÝ J , MCMAHAN H B , RAMAGE D , et al . Federated optimization:distributed machine learning for on-device intelligence [J ] . arXiv preprint , 2016 ,arXiv:1610.02527.

TRUEX S , BARACALDO N , ANWAR A , et al . A hybrid approach to privacy-preserving federated learning [C ] // Proceedings of the 12th ACM Workshop on Artificial Intelligence and Security . New York:ACM Press , 2019 : 1 - 11 .

WU Y C , CAI S F , XIAO X K , et al . Privacy preserving vertical federated learning for tree-based models [J ] . Proceedings of the VLDB Endowment , 2020 , 13 ( 12 ): 2090 - 2103 .

CHENG K W , FAN T , JIN Y L , et al . SecureBoost:a lossless federated learning framework [J ] . IEEE Intelligent Systems , 2021 , 36 ( 6 ): 87 - 98 .

LIU Y , LIU Y T , LIU Z J , et al . Federated forest [J ] . IEEE Transactions on Big Data , 2020 , 2755 ( 99 ): 1 .

LIU Y , MA Z , LIU X M , et al . Revocable federated learning:a benchmark of federated forest [J ] . arXiv preprint , 2019 ,arXiv:1911.03242.

SAHU A K , LI T , SANJABI M , et al . On the convergence of federated optimization in heterogeneous networks [J ] . arXiv preprint , 2018 ,arXiv:1812.06127.

YUROCHKIN M , AGARWAL M , GHOSH S , et al . Bayesian nonparametric federated learning of neural networks [C ] // Proceedings of the International Conference on Machine Learning .[S.l.:s.n. ] , 2019 : 7252 - 7261 .

WANG H Y , YUROCHKIN M , SUN Y K , et al . Federated learning with matched averaging [J ] . arXiv preprint , 2020 ,arXiv:2002.06440.

HE C Y , ANNAVARAM M , AVESTIMEHR S . Group knowledge transfer:federated learning of large CNNs at the edge [J ] . arXiv preprint , 2020 ,arXiv:2007.14513.

BACCIU D , DI SARLI D , FARAJI P , et al . Federated reservoir computing neural networks [C ] // Proceedings of 2021 International Joint Conference on Neural Networks . Piscataway:IEEE Press , 2021 : 1 - 7 .

KRIZHEVSKY A , SUTSKEVER I , HINTON G E . ImageNet classification with deep convolutional neural networks [J ] . Communications of the ACM , 2017 , 60 ( 6 ): 84 - 90 .

SUNDERMEYER M , SCHLÜTER R ,, NEY H . LSTM neural networks for language modeling [C ] // Proceedings of the 13th Annual Conference of The International Speech Communication Association .[S.l.:s.n. ] , 2012 .

GRAVES A , MOHAMED A R , HINTON G . Speech recognition with deep recurrent neural networks [C ] // Proceedings of 2013 IEEE International Conference on Acoustics,Speech and Signal Processing . Piscataway:IEEE Press , 2013 : 6645 - 6649 .

SUTSKEVER I , VINYALS O , LE Q V . Sequence to sequence learning with neural networks [J ] . Advances in Neural Information Processing Systems , 2014 .

WANG R S , LI Z , CAO J , et al . Recurrent convolutional neural networks for text classification [C ] // Proceedings of 2019 International Joint Conference on Neural Networks .[S.l.:s.n. ] , 2015 .

CHO K , VAN MERRIENBOER B , BAHDANAU D , et al . On the properties of neural machine translation:encoder–decoder approaches [C ] // Proceedings of the 8th Workshop on Syntax,Semantics and Structure in Statistical Translation . Stroudsburg:Association for Computational Linguistics , 2014 .

SHUKLA S , SRIVASTAVA N . Federated matched averaging with information-gain based parameter sampling [C ] // Proceedings of the 1st International Conference on AI-ML-Systems .[S.l.:s.n. ] , 2021 : 1 - 7 .

LIU Y , FAN T , CHEN T J , et al . FATE:an industrial grade platform for collaborative learning with data protection [J ] . Journal of Machine Learning Research , 2021 , 22 ( 226 ): 1 - 6 .

BONAWITZ K , EICHNER H , GRIESKAMP W , et al . Towards federated learning at scale:system design [J ] . Proceedings of Machine Learning and Systems , 2019 , 1 : 374 - 388 .

ZILLER A , TRASK A , LOPARDO A , et al . PySyft:a library for easy federated learning [M ] // Federated learning systems . Cham : Springer , 2021 .

HE C Y , LI S Z , SO J , et al . FedML:a research library and benchmark for federated machine learning [J ] . arXiv preprint , 2020 ,arXiv:2007.13518.

BEUTEL D J , TOPAL T , MATHUR A , et al . Flower:a friendly federated learning research framework [J ] . arXiv preprint , 2020 ,arXiv:2007.14390.

LIU Y , YU J J Q , KANG J W , et al . Privacy-preserving traffic flow prediction:a federated learning approach [J ] . IEEE Internet of Things Journal , 2020 , 7 ( 8 ): 7751 - 7763 .

YU S , CHEN X , ZHOU Z , et al . When deep reinforcement learning meets federated learning:intelligent multitimescale resource management for multiaccess edge computing in 5G ultradense network [J ] . IEEE Internet of Things Journal , 2021 , 8 ( 4 ): 2238 - 2251 .

ZHAO N , WU H , YU F R , et al . Deep-reinforcement-learning-based latency minimization in edge intelligence over vehicular networks [J ] . IEEE Internet of Things Journal , 2022 , 9 ( 2 ): 1300 - 1312 .

SHI D Y , TONG Y X , ZHOU Z M , et al . Learning to assign:towards fair task assignment in large-scale ride hailing [C ] // Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery ＆ Data Mining .[S.l.:s.n. ] , 2021 : 3549 - 3557 .

ZHENG W B , YAN L , GOU C , et al . Federated meta-learning for fraudulent credit card detection [C ] // Proceedings of the 29th International Joint Conference on Artificial Intelligence . California:International Joint Conferences on Artificial Intelligence Organization , 2020 : 4654 - 4660 .

HAO M , LI H W , XU G W , et al . Privacy-aware and resource-saving collaborative learning for healthcare in cloud computing [C ] // Proceedings of 2020 IEEE International Conference on Communications . Piscataway:IEEE Press , 2020 : 1 - 6 .

BRISIMI T S , CHEN R D , MELA T , et al . Federated learning of predictive models from federated electronic health records [J ] . International Journal of Medical Informatics , 2018 , 112 : 59 - 67 .

ZHANG W S , ZHOU T , LU Q H , et al . Dynamic fusion-based federated learning for COVID-19 detection [J ] . IEEE Internet of Things Journal , 2021 , 8 ( 21 ): 15884 - 15891 .

Views

2898

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Collective knowledge graph: meta knowledge transfer and federated graph reasoning

Federated visualization:a new model for privacy-preserving visualization

Related Author

Mingyang CHEN

Wen ZHANG

Xiangnan CHEN

Hongting ZHOU

Huajun CHEN

Yating WEI

Zhiyong WANG

Shuyue ZHOU

Related Institution

College of Computer Science and Technology, Zhejiang University

School of Software Technology, Zhejiang University

Postal code：100079
Tel：（010）53879206 Email：cjist@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：35685 Visits Today：160

⁰