RAG-PHI：检索增强生成驱动的平行人与平行智能

田永林; 王兴霞; 王雨桐; 王建功; 郭超; 范丽丽; 沈甜雨; 武万森; 张红梅; 朱正秋; 王飞跃

doi:10.11959/j.issn.2096-6652.2024015

您当前的位置：

首页 >

文章列表页 >

RAG-PHI：检索增强生成驱动的平行人与平行智能

专题：具身智能和人工智能体之后：基础设施模型与基础智能 | 更新时间：2024-06-05

- RAG-PHI：检索增强生成驱动的平行人与平行智能
- RAG-PHI: RAG-driven parallel human and parallel intelligence
- 智能科学与技术学报 2024年6卷第1期页码：41-51
- 作者机构：
  
  1.中国科学院自动化研究所多模态人工智能系统全国重点实验室，北京 100190
  2.中国科学院自动化研究所复杂系统管理与控制国家重点实验室，北京 100190
  3.中国科学院大学人工智能学院，北京 100049
  4.中国航空系统工程研究所，北京 100029
  5.北京理工大学信息与电子学院，北京 100081
  6.北京化工大学信息科学与技术学院，北京 100029
  7.国防科技大学系统工程学院，湖南长沙 410073
  8.北方自动控制技术研究所，山西太原 030006
  9.澳门科技大学创新工程学院，澳门 999078
- 作者简介：
  
  [ "田永林（1994- ），男，中国科学院自动化研究所博士后，主要研究方向为平行系统、自动驾驶、场景工程。" ]
  [ "王兴霞（1996- ），女，中国科学院自动化研究所多模态人工智能系统全国重点实验室博士生，主要研究方向为平行智能、平行油田、故障诊断。" ]
  [ "王雨桐（1994- ），女，博士，中国科学院自动化研究所多模态人工智能系统全国重点实验室助理研究员，主要研究方向为计算机视觉、图像异常检测。" ]
  [ "王建功（1996- ），男，博士，中国航空系统工程研究所工程师，主要研究方向为航空智能、大模型、计算机视觉。" ]
  [ "郭超（1992- ），男，博士，中国科学院自动化研究所助理研究员，主要研究方向为机器创作、人机协作、智能机器人系统、深度学习、强化学习。" ]
  [ "范丽丽（1991- ），女，博士，北京理工大学信息与电子学院博士后，主要研究方向为计算机视觉、跨模态感知与理解、类脑认知与决策。" ]
  [ "沈甜雨（1996- ），女，博士，北京化工大学信息科学与技术学院副教授，主要研究方向为机器视觉、智能感知与智能机器人系统。" ]
  [ "武万森（1996- ），男，国防科技大学系统工程学院博士生，主要研究方向为多模态深度学习、大语言模型、平行智能。" ]
  [ "张红梅（1987- ），女，就职于北方自动控制技术研究所，主要研究方向为智能控制、平行智能、大模型技术。" ]
  [ "朱正秋（1994- ），男，博士，国防科技大学系统工程学院讲师，主要研究方向为群智计算、群体智能、对话式AI智能体。" ]
  [ "王飞跃（1961- ），男，博士，中国科学院自动化研究所复杂系统管理与控制国家重点实验室主任、澳门科技大学特聘教授，主要研究方向为平行系统的方法与应用、社会计算、平行智能、知识自动化。" ]
- 基金信息：
  
  广东省重点领域研发计划(2020B090921003);广州市智能网联汽车重大科技专项(202007050002);中国科协青年人才托举工程(YESS20220372);澳门特别行政区科学技术发展基金资助项目(0093/2023/RIA2)
- DOI：10.11959/j.issn.2096-6652.2024015
  中图分类号： TP39
- 收稿：2024-02-03，
  
  修回：2024-03-03，
  
  纸质出版：2024-03-15
- 稿件说明：
移动端阅览
田永林, 王兴霞, 王雨桐, 等. RAG-PHI：检索增强生成驱动的平行人与平行智能[J]. 智能科学与技术学报, 2024,6(1):41-51.

TIAN Yonglin, WANG Xingxia, WANG Yutong, et al. RAG-PHI: RAG-driven parallel human and parallel intelligence[J]. Chinese Journal of Intelligent Science and Technology, 2024, 6(1): 41-51.
田永林, 王兴霞, 王雨桐, 等. RAG-PHI：检索增强生成驱动的平行人与平行智能[J]. 智能科学与技术学报, 2024,6(1):41-51. DOI： 10.11959/j.issn.2096-6652.2024015.

TIAN Yonglin, WANG Xingxia, WANG Yutong, et al. RAG-PHI: RAG-driven parallel human and parallel intelligence[J]. Chinese Journal of Intelligent Science and Technology, 2024, 6(1): 41-51. DOI： 10.11959/j.issn.2096-6652.2024015.

摘要

大模型技术的发展为融合生物人、数字人和机器人的平行人系统构建提供了崭新思路和智能基础。但是，在面临复杂系统的管控任务时，仍然存在着知识更新受限、专用能力不足以及信息隐私泄露等问题。为此，提出了检索增强生成驱动的平行人与平行智能框架——RAG-PHI，打造了开放式数据平台，为平行人系统接入实时知识、行业知识及私有知识；构建动态路由检索，进行上下文抓取和平行人能力重组；引入上下文提示学习增强思维与行为能力。进一步，面向平行人的组织管理、培训评估和运营生产模式，提出了基于RAG-PHI架构的平行人社区、平行人学校和平行人工厂架构，形成了检索增强生成和基础大模型协同驱动下的平行人生态，促进智业时代的生产力提升。

Abstract

The advancement of large models offers new perspectives and foundation intelligence for building parallel human ecosystems comprised of biological humans

digital humans

and robotic humans. However

challenges such as time-limited updates to knowledge

inadequate specialized capabilities

and risks of information privacy leakage persist in the management and control of complex systems. To tackle these issues

a retrieval-augmented generation-driven parallel human and parallel intelligence framework (RAG-PHI) is introduced. It proposes to establish an open data platform that facilitates the integration of real-time

industry-specific

and private knowledge into the parallel human system. It develops dynamic routing and retrieval for context capture and the reconfiguration of parallel human capabilities

along with introducing context-aware prompt learning to enhance cognitive and behavioral skills. Furthermore

towards the organization and management

training and evaluation

operation and production of parallel human

the structures of parallel human community

parallel human school

and parallel human factory are proposed by the RAG-PHI architecture. These are designed to foster a parallel human ecosystem powered by RAG and large foundation models

thereby enhancing productivity in the age of intelligent industries.

关键词

Keywords

references

田永林, 王雨桐, 王建功, 等. 视觉Transformer研究的关键问题: 现状及展望[J]. 自动化学报, 2022, 48(4): 957-979.

TIAN Y L, WANG Y T, WANG J G, et al. Key problems and progress of vision Transformers: the state of the art and prospects[J]. Acta Automatica Sinica, 2022, 48(4): 957-979.

WU T Y, HE S Z, LIU J P, et al. A brief overview of ChatGPT: the history, status quo and potential future development[J]. CAA Journal of Automatica Sinica, 2023, 10(5): 1122-1136.

WANG F Y, MIAO Q H, LI X, et al. What does ChatGPT say: the DAO from algorithmic intelligence to linguistic intelligence[J]. IEEE/CAA Journal of Automatica Sinica, 2023, 10(3): 575-579.

KIRILLOV A, MINTUN E, RAVI N, et al. Segment anything [EB]. arXiv preprint, 2023, arXiv: 2304.02643.

YU H, LIU X, TIAN Y, et al. Sora-based parallel vision for smart sensing of intelligent vehicles: from foundation models to foundation intelligence[J]. IEEE Transactions on Intelligent Vehicles, 2024, 9(2): 3030-3033.

WANG F Y, MIAO Q, LI L, et al. When does Sora show: the beginning of TAO to imaginative intelligence and scenarios engineering[J]. IEEE/CAA Journal of Automatica Sinica, 2024, 11(4): 809-815.

LI X, MIAO Q, LI L, et al. Sora for senarios engineering of intelligent vehicles: V&V, C&C, and beyonds[J]. IEEE Transactions on Intelligent Vehicles, 2024, 9(2): 3025-3029.

王飞跃. 平行管理: 复杂性管理智能的生态科技与智慧管理之DAO[J]. 自动化学报, 2022, 48(11): 2655-2669.

WANG F Y. Parallel management: the DAO to smart ecological technology for complexity management intelligence[J]. Acta Automatica Sinica, 2022, 48(11): 2655-2669.

SHEN Y L, SONG K T, TAN X, et al. HuggingGPT: solving AI tasks with ChatGPT and its friends in hugging face[EB]. arXiv preprint, 2023, arXiv: 2303.17580.

TIAN Y L, LI X, ZHANG H, et al. VistaGPT: generative parallel transformers for vehicles with intelligent systems for transport automation[J]. IEEE Transactions on Intelligent Vehicles, 2023, 8(9): 4198-4207.

WANG F Y. CAST Lab: a cyber-social-physical approach for traffic control and transportation management[J]. Intelligent Control and Systems Engineering Center in Chinese Academy of Sciences, 1999.

WANG F Y. The emergence of intelligent enterprises: from CPS to CPSS[J]. IEEE Intelligent Systems, 2010, 25(4): 85-88.

ZHANG J J, WANG F Y, WANG X, et al. Cyber-physical-social systems: the state of the art and perspectives[J]. IEEE Transactions on Computational Social Systems, 2018, 5(3): 829-840.

WANG X, YANG J, HAN J P, et al. Metaverses and DeMetaverses: from digital twins in CPS to parallel intelligence in CPSS[J]. IEEE Intelligent Systems, 2022, 37(4): 97-102.

YANG J, WANG X, TIAN Y L, et al. Parallel intelligence in CPSSs: being, becoming, and believing[J]. IEEE Intelligent Systems, 2023, 38(6): 75-80.

WANG F Y. Forward to the past: CASTLab's cyber-social-physical approach for ITS in 1999: history and perspectives[J]. IEEE Intelligent Transportation Systems Magazine, 2023, 15(4): 171-175.

杨静, 王晓, 王雨桐, 等. 平行智能与CPSS: 三十年发展的回顾与展望[J]. 自动化学报, 2023, 49(3): 614-634.

YANG J, WANG X, WANG Y T, et al. Parallel intelligence and CPSS in 30 years: an ACP approach[J]. Acta Automatica Sinica, 2023, 49(3): 614-634.

秦蕊, 梁小龙, 李娟娟, 等. 平行科研院所: 从数字化转型到智能化变革[J]. 智能科学与技术学报, 2023, 5(2): 212-221.

QIN R, LIANG X L, LI J J, et al. Parallel scientific research institutes: from digital transformation to intelligent revolution[J]. Chinese Journal of Intelligent Science and Technology, 2023, 5(2): 212-221.

田永林, 陈苑文, 杨静, 等. 元宇宙与平行系统: 发展现状、对比及展望[J]. 智能科学与技术学报, 2023(1): 121-132.

TIAN Y L, CHEN Y W, YANG J, et al. Metaverses and parallel systems: the state of the art, comparisons and prospects[J]. Chinese Journal of Intelligent Science and Technology, 2023(1): 121-132.

WANG X X, CHENG X, LU J W, et al. Metaverses-based parallel oil fields in CPSS: a framework and methodology[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2023, 53(4): 2138-2147.

WANG Y T, WANG X, WANG X X, et al. The ChatGPT after: building knowledge factories for knowledge workers with knowledge automation[J]. IEEE/CAA Journal of Automatica Sinica, 2023, 10(11): 2041-2044.

卢经纬, 郭超, 戴星原, 等. 问答ChatGPT之后: 超大预训练模型的机遇和挑战[J]. 自动化学报, 2023, 49(4): 705-717.

LU J W, GUO C, DAI X Y, et al. The ChatGPT after: opportunities and challenges of very large scale pre-trained models[J]. Acta Automatica Sinica, 2023, 49(4): 705-717.

WANG F Y, YANG J, WANG X X, et al. Chat with ChatGPT on industry 5.0: learning and decision-making for intelligent industries[J]. IEEE/CAA Journal of Automatica Sinica, 2023, 10(4): 831-834.

HUANG W, WANG C, ZHANG R, et al. VoxPoser: composable 3D value maps for robotic manipulation with language models [C]// Proceedings of PMLR. New York: PMLR, 2023: 540-562.

LEWIS P, PEREZ E, PIKTUS A, et al. Retrieval-augmented generation for knowledge-intensive NLP tasks[C]//Proceedings of the 34th International Conference on Neural Information Processing Systems. New York: ACM, 2020: 9459-9474.

王飞跃, 王雨桐. 数字科学家与平行科学: AI4S和S4AI的本源与目标[J]. 中国科学院院刊, 2024, 39(1): 27-33.

WANG F Y, WANG Y T. Digital scientists and parallel sciences: the origin and goal of AI for Science and Science for AI[J]. Bulletin of Chinese Academy of Sciences, 2024, 39(1): 27-33.

TEAM G, ANIL R, BORGEAUD S, et al. Gemini: a family of highly capable multimodal models[EB]. arXiv preprint, 2023, arXiv: 2312.11805.

HUANG W L, WANG C, ZHANG R H, et al. VoxPoser: composable 3D value maps for robotic manipulation with language models[EB]. arXiv preprint, 2023, arXiv:2307.05973.

FU Z P, ZHAO T Z, FINN C. Mobile ALOHA: learning bimanual mobile manipulation with low-cost whole-body teleoperation[EB]. arXiv preprint, 2024, arXiv: 2401.02117.

WANG F Y. Toward a paradigm shift in social computing: the ACP approach[J]. IEEE Intelligent Systems, 2007, 22(5): 65-67.

WANG F Y. The DAO to MetaControl for MetaSystems in metaverses: the system of parallel control systems for knowledge automation and control intelligence in CPSS[J]. IEEE/CAA Journal of Automatica Sinica, 2022, 9(11): 1899-1908.

WANG X X, CHENG X, LU J W, et al. Fault diagnosis for sucker rod pumping systems: a parallel system approach[J]. IEEE Journal of Radio Frequency Identification, 2022, 6: 743-747.

田永林, 陈苑文, 杨静, 等. 元宇宙与平行系统: 发展现状、对比及展望[J]. 智能科学与技术学报, 2023, 5(1): 121-132.

TIAN Y L, CHEN Y W, YANG J, et al. Metaverses and parallel systems: the state of the art, comparisons and prospects[J]. Chinese Journal of Intelligent Science and Technology, 2023, 5(1): 121-132.

YANG J, WANG Y T, WANG X X, et al. Generative AI empowering parallel manufacturing: building a "6S" collaborative production ecology for manufacturing 5.0[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2024(99): 1-15.

王飞跃, 王艳芬, 陈薏竹, 等. 联邦生态: 从联邦数据到联邦智能[J]. 智能科学与技术学报, 2020, 2(4): 305-311.

WANG F Y, WANG Y F, CHEN Y Z, et al. Federated ecology: from federated data to federated intelligence[J]. Chinese Journal of Intelligent Science and Technology, 2020, 2(4): 305-311.

朱静, 王飞跃, 王戈, 等. 联邦控制: 面向信息安全和权益保护的分布式控制方法[J]. 自动化学报, 2021, 47(8): 1912-1920.

ZHU J, WANG F Y, WANG G, et al. Federated control: a distributed control approach towards information security and rights protection[J]. Acta Automatica Sinica, 2021, 47(8): 1912-1920.

康孟珍, 王秀娟, 李冬, 等. 基于联邦学习的分布式农业组织[J]. 智能科学与技术学报, 2022, 4(2): 288-297.

KAGN M Z, WANG X J, LI D, et al. Distributed agricultural organization based on federated learning[J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(2): 288-297.

TIAN Y L, WANG J G, WANG Y T, et al. Federated vehicular transformers and their federations: privacy-preserving computing and cooperation for autonomous driving[J]. IEEE Transactions on Intelligent Vehicles, 2022, 7(3): 456-465.

LI J J, LIANG X L, QIN R, et al. From DAO to TAO: finding the essence of decentralization[C]//Proceedings of the 2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC). Piscataway: IEEE Press, 2023: 4283-4288.

LIANG X L, LI J J, QIN R, et al. TAO-based data governance in intelligent transportation systems[C]//Proceedings of the 2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC). Piscataway: IEEE Press, 2023: 5306-5311.

LI J J, WANG F Y. The TAO of blockchain intelligence for intelligent web 3.0[J]. IEEE/CAA Journal of Automatica Sinica, 2023, 10(12): 2183-2186.

LI X, YE P J, LI J J, et al. From features engineering to scenarios engineering for trustworthy AI: I&I, C&C, and V&V[J]. IEEE Intelligent Systems, 2022, 37(4): 18-26.

LI X, TIAN Y L, YE P J, et al. A novel scenarios engineering methodology for foundation models in metaverse[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2023, 53(4): 2148-2159.

WANG X, CHEN Y D, ZHU W W. A survey on curriculum learning[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(9): 4555-4576.

WANG X X, YANG J, WANG Y T, et al. Steps toward industry 5.0: building "6S" parallel industries with cyber-physical-social intelligence[J]. IEEE/CAA Journal of Automatica Sinica, 2023, 10(8): 1692-1703.

WANG F Y. New control paradigm for industry 5.0: from big models to foundation control and management[J]. IEEE/CAA Journal of Automatica Sinica, 2023, 10(8): 1643-1646.

王飞跃, 高彦臣, 商秀芹, 等. 平行制造与工业5.0: 从虚拟制造到智能制造[J]. 科技导报, 2018, 36(21): 10-22.

WANG F Y, GAO Y C, SHANG X Q, et al. Parallel manufacturing and industries 5.0: from virtual manufacturing to intelligent manufacturing[J]. Science & Technology Review, 2018, 36(21): 10-22.

LU J W, WANG X X, CHENG X, et al. Parallel factories for smart industrial operations: from big AI models to field foundational models and scenarios engineering[J]. IEEE/CAA Journal of Automatica Sinica, 2022, 9(12): 2079-2086.

YANG J, LI S M, WANG X X, et al. DeFACT in ManuVerse for parallel manufacturing: foundation models and parallel workers in smart factories[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2023, 53(4): 2188-2199.

彭绍明, 熊刚, 沈震, 等. 平行车间调度模型构建和系统设计[J]. 智能科学与技术学报, 2023(2): 254-266.

PENG S M, XIONG G, SHEN Z, et al. Parallel workshop scheduling model and system design[J]. Chinese Journal of Intelligent Science and Technology, 2023(2): 254-266.

浏览量

636

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

平行教学实验室：赋能K-12教育的新AI基础设施

平行澳门野外观测站：全域智能观测与应急管理自主系统

平行奥森：智慧生态与健康生活共融的元宇宙公园

AI交通科学家：大模型驱动的自主交通科研

智能的本质与意义