Privacy risks induced by generative large language models and governance paths

LI Yaling; CAI Jingjing; BAI Jieming

doi:10.11959/j.issn.2096-0271.202431

您当前的位置：

首页 >

文章列表页 >

Privacy risks induced by generative large language models and governance paths

Special Column: Artificial Intelligence Trustworthiness and Governance | 更新时间：2024-11-04

- Privacy risks induced by generative large language models and governance paths
- Chinese Journal of Intelligent Science and Technology Vol. 6, Issue 3, Pages: 394-401(2024)
- 作者机构：
  
  1.之江实验室发展战略与合作中心，浙江杭州，311121
  2.浙江省哲学社会科学试点实验室，浙江杭州，311121
  3.之江实验室智能社会治理实验室，浙江杭州，311121
- 作者简介：
- 基金信息：
  
  The National Key Research and Development Program of China(2022YFC3303103);Zhejiang Provincial Department of Science and Technology Soft Science Research Program(2024C35035)
- DOI：10.11959/j.issn.2096-0271.202431
  CLC： TP39
- Received：15 April 2024，
  
  Revised：2024-04-28，
  
  Published：15 September 2024
- 稿件说明：
移动端阅览
李亚玲,蔡京京,柏洁明.生成式大模型引发的隐私风险及治理路径[J].智能科学与技术学报,2024,06(03):394-401.

LI Yaling,CAI Jingjing,BAI Jieming.Privacy risks induced by generative large language models and governance paths[J].Chinese Journal of Intelligent Science and Technology,2024,06(03):394-401.
李亚玲,蔡京京,柏洁明.生成式大模型引发的隐私风险及治理路径[J].智能科学与技术学报,2024,06(03):394-401. DOI： 10.11959/j.issn.2096-0271.202431.

LI Yaling,CAI Jingjing,BAI Jieming.Privacy risks induced by generative large language models and governance paths[J].Chinese Journal of Intelligent Science and Technology,2024,06(03):394-401. DOI： 10.11959/j.issn.2096-0271.202431.

摘要

大语言模型（简称大模型）正引发人工智能技术的新一轮革命。由于大模型具有平台性、海量数据依赖性、用户交互频繁、易受到攻击等特征，其隐私泄露风险更加引人担忧。重点探讨了大模型引发的隐私泄露来源、内在机理。在梳理了国内外对大模型隐私风险的治理经验后，提出了包含政策、标准、数据、技术和生态在内的五维治理框架。最后，展望了大模型在多模态、智能体、具身智能、边缘智能等发展趋势下，可能面临的新型隐私泄露风险。

Abstract

Large language models(LLM) are leading to a new revolution in artificial intelligence technology. As LLM have the characteristics of platform

mass data dependence

frequent user interaction

and easy to be attacked

the risk of privacy disclosure is more worrying. The source and internal mechanism of privacy leakage caused by LLM were focused. After reviewing the domestic and foreign experience in the governance of LLM privacy risks

a five-dimensional governance framework that included policies

standards

data

technology

and ecology were proposed. Finally

this paper also looks forward to the new privacy disclosure risks that LLM may face under the development trend of multimodal

agent

embodied intelligence

edge intelligence

etc.

关键词

Keywords

references

HUANG X W, RUAN W J, HUANG W, et al. A survey of safety and trustworthiness of large language models through the lens of verification and validation[J]. Artificial Intelligence Review, 2024, 57(7): 175.

PLANT R, GIUFFRIDA V, GKATZIA D. You are what you write: preserving privacy in the era of large language models[EB ] .arXiv preprint, 2022, arXiv: 2204.09391 .

EMMERY C, KÁDÁR Á, CHRUPAŁA G. Adversarial stylometry in the wild: transferable lexical substitution attacks on author profiling[C]// Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. Stroudsburg: Association for Computational Linguistics, 2021.

CHEN S, KAHLA M, JIA R X, et al. Knowledge-enriched distributional model inversion attacks[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway: IEEE Press, 2021: 16158-16167.

CARLINI N, HAYES J, NASR M, et al. Extracting training data from diffusion models[EB ] .arXiv preprint, 2023, arXiv: 2301.13188 .

禇乐阳, 潘香霖, 陈向东. AI大模型在教育应用中的伦理风险与应对[J]. 苏州大学学报(教育科学版), 2024, 12(1): 87-96.

CHU L Y, PAN X L, CHEN X D. Ethic risks and responses in the application of AI large language models in education[J]. Journal of Soochow University(Educational Science Edition), 2024, 12(1): 87-96.

许志伟, 李海龙, 李博, 等. AIGC大模型测评综述:使能技术,安全隐患和应对[J].计算机科学与探索, 2024, 18(9): 2293-2325.

XU Z W, LI H L, LI B, et al. A survey of AIGC model evaluation: enabling technologies, vulnerabilities and mitigation[J]. Journal of Frontiers of Computer Science and Technology, 2024, 18(9): 2293-2325.

CARLINI N, TRAMER F, WALLACE E, et al. Extracting training data from large language models[EB ] . arXiv preprint, 2020, arXiv: 2012.07805 .

CARLINI N, IPPOLITO D, JAGIELSKI M, et al. Quantifying memorization across neural language models[EB ] . arXiv preprint, 2022, arXiv: 2202.07646 .

STAAB R, VERO M, BALUNOVIĆ M, et al. Beyond memorization: violating privacy via inference with large language modelss[EB ] . arXiv preprint, 2023, arXiv: 2310.07298 .

NASR M, CARLINI N, HAYASE J, et al. Scalable extraction of training data from (production) language models[EB ] . arXiv preprint, 2023, arXiv: 2311.17035 .

KRISHNA K, TOMAR G S, PARIKH A P, et al. Thieves on sesame street! model extraction of bert-based apis[EB ] . arXiv preprint, 2019, arXiv: 1910.12366 .

CARLINI N, PALEKA D, DVIJOTHAM K D, et al. Stealing part of a production language model[EB ] . arXiv preprint, 2024, arXiv: 2403.06634 .

KANDPAL N, WALLACE E, RAFFEL C. Deduplicating training data mitigates privacy risks in language models[C]//Proceedings of the 39th International Conference on Machine Learning. New York: PMLR, 2022: 10697-10707.

LEE K, IPPOLITO D, NYSTROM A, et al. Deduplicating training data makes language models better[EB ] . arXiv preprint, 2021, arXiv: 2107.06499 .

WANG Z, ZHONG W, WANG Y, et al. Data management for large language models: a survey[EB ] . arXiv preprint, 2023, arXiv: 2312.01700 .

SABLAYROLLES A, DOUZE M, SCHMID C, et al. Radioactive data: tracing through training[C]//Proceedings of the 37th International Conference on Machine Learning. New York: PMLR, 2020: 8326-8335.

SANDER T, FERNANDEZ P, DURMUS A, et al. Watermarking makes language models radioactive[EB ] . arXiv preprint, 2024, arXiv: 2402.14904 .

王飞跃, 王艳芬, 陈薏竹, 等. 联邦生态: 从联邦数据到联邦智能[J]. 智能科学与技术学报, 2020, 2(4): 305-311.

WANG F Y, WANG Y F, CHEN Y Z, et al. Federated ecology: from federated data to federated intelligence[J]. Chinese Journal of Intelligent Science and Technology, 2020, 2(4): 305-311.

YE R, WANG W H, CHAI J Y, et al. OpenFedLLM: training large language models on decentralized private data via federated learning[EB ] . arXiv preprint, 2024, arXiv: 2402.06954 .

CHEN T, BAO H, HUANG S, et al. THE-X: privacy-preserving transformer inference with homomorphic encryption[EB ] . arXiv preprint, 2022, arXiv: 2206.00216 .

LI D, SHAO R, WANG H, et al. MPCFormer: fast, performant and private Transformer inference with MPC[EB ] .arXiv preprint, 2022, arXiv: 2211.01452 .

ZENG W X, LI M, XIONG W J, et al. MPCViT: searching for accurate and efficient MPC-friendly vision transformer with heterogeneous attention[C]//Proceedings of the 2023 IEEE/CVF International Conference on Computer Vision(ICCV). Piscataway: IEEE Press, 2023: 5029-5040.

HAO M, LI H, CHEN H, et al. Iron: private inference on transformers[J]. Advances in Neural Information Processing Systems, 2022, 35: 15718-15731.

DONG Y, LU W J, ZHENG Y C, et al. PUMA: secure inference of LLaMA-7B in five minutes[EB ] . arXiv preprint, 2023, arXiv: 2307.12533 .

GUPTA K, JAWALKAR N, MUKHERJEE A, et al. SIGMA: secure GPT inference with function secret sharing[EB]. Cryptology ePrint Archive, 2023, cryptoeprint: 2023/1269.

HUANG W, WANG Y, CHENG A, et al. A fast, performant, secure distributed training framework for large language model[EB ] . arXiv preprint, 2024, arXiv: 2401.09796 .

ZIMERMAN I, BARUCH M, DRUCKER N, et al. Converting transformers to polynomial form for secure inference over homomorphic encryption[EB ] . arXiv preprint, 2023, arXiv: 2311.08610 .

SHI W, SHEA R, CHEN S, et al. Just fine-tune twice: Selective differential privacy for large language models[EB ] . arXiv preprint, 2022, arXiv: 2204.07667 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Rumor susceptibility and intervention strategies of large language model-driven social agents

Research on popularity stage prediction methods for open-source community projects: a case study of large language model projects

RadiFlow: AI agents reconstructing radiology workflow

Design and implementation of an intelligent question answering system for Macrobrachium rosenbergii aquaculture based on knowledge graph and large language model

The ACP and LLM-based framework for criminology: a review and prospect

Related Author

CHEN Yang

GONG Qingyuan

YUAN Jingwei

YIN Yongjie

LI Cong

FU Xiaoming

CHEN Yang

LU Xiaoyi

Related Institution

Research Institute of Intelligent Complex Systems, Fudan University

College of Computer Science and Artificial Intelligence, Fudan University

Institute of Computer Science, University of Göttingen, Göttingen

Research Center for Social Intelligence, Fudan University

Adaptive Networks and Control Laboratory, College of Future Information Technology, Fudan University

Postal code：100079
Tel：（010）53879206 Email：cjist@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：161982 Visits Today：145

⁰