人机物CPSS智能融合的平行创作架构与关键技术研究

郭超; 鲁越; 王晓; 易达; 王虓; 王飞跃

doi:10.11959/j.issn.2096-6652.202246

您当前的位置：

首页 >

文章列表页 >

人机物CPSS智能融合的平行创作架构与关键技术研究

学术论文 | 更新时间：2024-06-05

- 人机物CPSS智能融合的平行创作架构与关键技术研究
- Architecture and key techniques of parallel creation through the fusion of human-cyber-physical intelligence in CPSS
- 智能科学与技术学报 2022年4卷第3期页码：344-354
- 作者机构：
  
  1. 中国科学院自动化研究所复杂系统管理与控制国家重点实验室，北京 100190
  2. 中国科学院大学人工智能学院，北京 100049
  3. 青岛智能产业技术研究院平行智能创新中心，山东青岛 266114
  4. 普渡大学, 美国西拉法叶 IN 47907-2050
  5. 中国国家博物馆科学与艺术研究所，北京 100006
- 作者简介：
  
  [ "郭超（1992- ），男，中国科学院自动化研究所复杂系统管理与控制国家重点实验室助理研究员，主要研究方向为机器学习、强化学习、机器艺术创作、图像生成、机器人绘画等" ]
  [ "鲁越（1994- ），男，中国科学院自动化研究所复杂系统管理与控制国家重点实验室博士生，主要研究方向为机器学习、小样本学习、风格迁移等" ]
  [ "王晓（1988- ），女，博士，中国科学院自动化研究所复杂系统管理与控制国家重点实验室副研究员，主要研究方向为群体行为的激发与汇聚激励、群体智能和社交网络挖掘与分析" ]
  [ "易达（1996- ），男，中国科学院自动化研究所复杂系统管理与控制国家重点实验室硕士生，主要研究方向为机器学习、深度学习、绘画图像生成、艺术风格迁移等" ]
  [ "王虓（1996- ），男，普渡大学计算机系博士生，主要研究方向为自监督学习、计算生物学和智能交通系统等" ]
  [ "王飞跃（1961- ），男，中国科学院自动化研究所复杂系统管理与控制国家重点实验室研究员，国防科技大学军事计算实验与平行系统技术研究中心教授，中国科学院大学中国经济与社会安全研究中心主任，主要研究方向为智能系统和复杂系统建模、分析与控制" ]
- 基金信息：
  
  国家自然科学基金项目(61533019)
- DOI：10.11959/j.issn.2096-6652.202246
  中图分类号： TP391
- 网络出版日期：2022-09，
  
  纸质出版日期：2022-09-15
- 稿件说明：
移动端阅览
郭超, 鲁越, 王晓, 等. 人机物CPSS智能融合的平行创作架构与关键技术研究[J]. 智能科学与技术学报, 2022,4(3):344-354.

Chao GUO, Yue LU, Xiao WANG, et al. Architecture and key techniques of parallel creation through the fusion of human-cyber-physical intelligence in CPSS[J]. Chinese journal of intelligent science and technology, 2022, 4(3): 344-354.
郭超, 鲁越, 王晓, 等. 人机物CPSS智能融合的平行创作架构与关键技术研究[J]. 智能科学与技术学报, 2022,4(3):344-354. DOI： 10.11959/j.issn.2096-6652.202246.

Chao GUO, Yue LU, Xiao WANG, et al. Architecture and key techniques of parallel creation through the fusion of human-cyber-physical intelligence in CPSS[J]. Chinese journal of intelligent science and technology, 2022, 4(3): 344-354. DOI： 10.11959/j.issn.2096-6652.202246.

摘要

随着人工智能探索领域的不断拓展，艺术创作成为人工智能发展和应用的重要研究热点。基于平行系统理论与 ACP 方法构建风格多样、内容逼真、笔触灵活和描述精准的平行艺术创作元宇宙，为提升人工智能的创造能力提供了一种可行的实现途径，并提供了应用案例。通过 AI 算法创作、人类筛选和评估、机器人执行，构建了人机物CPSS智能融合的平行创作架构，阐述了基于计算实验的绘画风格迁移、内容组合、笔触生成和图像描述等关键技术，并对所构建的平行创作系统进行了实验验证。平行创作系统融合了人、AI创作算法、机器人的优势，提升了人工智能艺术创作系统在虚拟和物理空间中的创作水平，促进了人机物协同艺术创作的发展。

Abstract

With the expansion of the fields covered by AI

artistic creation will become the next hot spot for AI research and applications.Building a metaverse with diverse styles

realistic contents

flexible strokes and accurate descriptions based on parallel system theory and ACP approach will provide a feasible way to improve AI creation capability.The intelligence of human

and robots were fused to develop a parallel creation architecture through the creation by AI

the evaluation by humans

and the execution by robots.The parallel creation with the key methods of style transfer

content combination

stroke generation and image captioning in computational experiments were explained.The parallel creation system was validated through the painting experiments.The parallel creation system will improve the creation capability of artificial intelligence in cyberspace and physical space

and promote the human-cyber-physical collaborative creation through the fusion of them.

关键词

Keywords

references

MCCARTHY J , MINSKY M , ROCHESTER N , et al . A proposal for the Dartmouth summer research project on artificial intelligence,August 31,1955 [J ] . AI Mag , 2006 , 27 : 12 - 14 .

BODEN M A . Creativity and artificial intelligence [J ] . Artificial Intelligence , 1998 , 103 ( 1/2 ): 347 - 356 .

SILVER D , HUANG A , MADDISON C J , et al . Mastering the game of Go with deep neural networks and tree search [J ] . Nature , 2016 , 529 ( 7587 ): 484 - 489 .

WANG F Y , ZHANG J J , ZHENG X H , et al . Where does AlphaGo go:from church-Turing thesis to AlphaGo thesis and beyond [J ] . IEEE/CAA Journal of Automatica Sinica , 2016 , 3 ( 2 ): 113 - 120 .

郭超 , 鲁越 , 林懿伦 , 等 . 平行艺术:人机协作的艺术创作 [J ] . 智能科学与技术学报 , 2019 , 1 ( 4 ): 335 - 341 .

GUO C , LU Y , LIN Y L , et al . Parallel art:artistic creation under human-machine collaboration [J ] . Chinese Journal of Intelligent Science and Technology , 2019 , 1 ( 4 ): 335 - 341 .

RAMESH A , DHARIWAL P , NICHOL A , et al . Hierarchical text-conditional image generation with CLIP latents [J ] . arXiv preprint,2022,arXiv:2204.06125 .

GONTHIER N , GOUSSEAU Y , LADJAL S , et al . Weakly supervised object detection in artworks [M ] // Lecture notes in computer science . Cham : Springer International Publishing , 2019 : 692 - 709 .

RODRIGUEZ C S , LECH M , PIROGOVA E . Classification of style in fine-art paintings using transfer learning and weighted image patches [C ] // Proceedings of 2018 12th International Conference on Signal Processing and Communication Systems . Piscataway:IEEE Press , 2018 : 1 - 7 .

CETINIC E , LIPIC T , GRGIC S . A deep learning perspective on beauty,sentiment,and remembrance of art [J ] . IEEE Access , 2019 , 7 : 73694 - 73710 .

DA YI , GUO C , BAI T X . Exploring painting synthesis with diffusion models [C ] // Proceedings of 2021 IEEE 1st International Conference on Digital Twins and Parallel Intelligence . Piscataway:IEEE Press , 2021 : 332 - 335 .

CETINIC E , SHE J . Understanding and creating art with AI:review and outlook [J ] . ACM Transactions on Multimedia Computing,Communications,and Applications , 2022 , 18 ( 2 ): 1 - 22 .

WANG F Y . Shadow systems:a new concept for nested and embedded co-simulation for intelligent systems [R ] . 1994 .

王飞跃 . 关于复杂系统研究的计算理论与方法 [J ] . 中国基础科学 , 2004 , 6 ( 5 ): 3 - 10 .

WANG F Y . Computational theory and method on complex system [J ] . China Basic Science , 2004 , 6 ( 5 ): 3 - 10 .

王飞跃 . 人工社会、计算实验、平行系统:关于复杂社会经济系统计算研究的讨论 [J ] . 复杂系统与复杂性科学 , 2004 , 1 ( 4 ): 25 - 35 .

WANG F Y . Artificial societies,computational experiments,and parallel systems:a discussion on computational theory of complex social-economic systems [J ] . Complex Systems and Complexity Science , 2004 , 1 ( 4 ): 25 - 35 .

王飞跃 . 平行系统方法与复杂系统的管理和控制 [J ] . 控制与决策 , 2004 , 19 ( 5 ): 485 - 489 , 514 .

WANG F Y . Parallel system methods for management and control of complex systems [J ] . Control and Decision , 2004 , 19 ( 5 ): 485 - 489 , 514 .

王飞跃 , 史帝夫·兰森 . 从人工生命到人工社会:复杂社会系统研究的现状和展望 [J ] . 复杂系统与复杂性科学 , 2004 , 1 ( 1 ): 33 - 41 .

WANG F Y , LANSING J S . From artificial life to artificial societies—new methods for studies of complex social systems [J ] . Complex Systems and Complexity Science , 2004 , 1 ( 1 ): 33 - 41 .

WANG F Y , WANG X , LI L X , et al . Steps toward parallel intelligence [J ] . IEEE/CAA Journal of Automatica Sinica , 2016 , 3 ( 4 ): 345 - 348 .

杨林瑶 , 陈思远 , 王晓 , 等 . 数字孪生与平行系统:发展现状、对比及展望 [J ] . 自动化学报 , 2019 , 45 ( 11 ): 2001 - 2031 .

YANG L Y , CHEN S Y , WANG X , et al . Digital twins and parallel systems:state of the art,comparisons and prospect [J ] . Acta Automatica Sinica , 2019 , 45 ( 11 ): 2001 - 2031 .

GATYS L A , ECKER A S , BETHGE M . Image style transfer using convolutional neural networks [C ] // Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2016 : 2414 - 2423 .

HERTZMANN A , . Painterly rendering with curved brush strokes of multiple sizes [C ] // Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques .[S.l.:s.n. ] , 1998 : 453 - 460 .

GOOCH B , COOMBE G , SHIRLEY P . Artistic vision:painterly rendering using computer vision techniques [C ] // Proceedings of the 2nd International Symposium on Non-photorealistic Animation and Rendering .[S.l.:s.n. ] , 2002 :83.

HERTZMANN A , JACOBS C E , OLIVER N , et al . Image analogies [C ] // Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques .[S.l.: s.n. ] , 2001 : 327 - 340 .

TOMASI C , MANDUCHI R . Bilateral filtering for gray and color images [C ] // Proceedings of the 6th International Conference on Computer Vision . Piscataway:IEEE Press , 1998 : 839 - 846 .

JOHNSON J , ALAHI A , LI F F . Perceptual losses for real-time style transfer and super-resolution [C ] // Computer Vision - ECCV 2016 .[S.l.:s.n. ] , 2016 : 694 - 711 .

CHEN D W , YUAN L , LIAO J , et al . Stylebank:an explicit representation for neural image style transfer [C ] // Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2017 : 1897 - 1906 .

HUANG X , BELONGIE S . Arbitrary style transfer in real-time with adaptive instance normalization [C ] // Proceedings of 2017 IEEE International Conference on Computer Vision . Piscataway:IEEE Press , 2017 : 1510 - 1519 .

COLTON S , WIGGINS G A . Computational creativity:the final frontier? [C ] // Proceedings of the 20th European Conference on Artificial Intelligence .[S.l.:s.n. ] , 2012 : 21 - 26 .

ESSER P , ROMBACH R , OMMER B . Taming transformers for high-resolution image synthesis [C ] // Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2021 : 12868 - 12878 .

RAMESH A , PAVLOV M , GOH G , et al . Zero-shot text-to-image generation [J ] . arXiv preprint,2021,arXiv:2102.12092 .

ZHENG Q Y , LI Z R , BARGTEIL A . Learning aesthetic layouts via visual guidance [J ] . arXiv preprint,2021,arXiv:2107.06262 .

LI J N , YANG J M , HERTZMANN A , et al . LayoutGAN:synthesizing graphic layouts with vector-wireframe adversarial networks [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2021 , 43 ( 7 ): 2388 - 2399 .

ZOU Z X , SHI T Y , QIU S , et al . Stylized neural painting [C ] // Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2021 : 15684 - 15693 .

HEGDE S , GATZIDIS C , TIAN F . Painterly rendering techniques:a state-of-the-art review of current approaches [J ] . Computer Animation and Virtual Worlds , 2013 , 24 ( 1 ): 43 - 64 .

HUANG Z W , ZHOU S C , HENG W . Learning to paint with model-based deep reinforcement learning [C ] // Proceedings of 2019 IEEE/CVF International Conference on Computer Vision . Piscataway:IEEE Press , 2019 : 8708 - 8717 .

VINYALS O , TOSHEV A , BENGIO S , et al . Show and tell:a neural image caption generator [C ] // Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2015 : 3156 - 3164 .

HOSSAIN M Z , SOHEL F , SHIRATUDDIN M F , et al . A comprehensive survey of deep learning for image captioning [J ] . ACM Computing Surveys , 2019 , 51 ( 6 ): 1 - 36 .

BAI S , AN S . A survey on automatic image caption generation [J ] . Neurocomputing , 2018 , 311 : 291 - 304 .

WANG W S , NA X X , CAO D P , et al . Decision-making in driver-automation shared control:a review and perspectives [J ] . IEEE/CAA Journal of Automatica Sinica , 2020 , 7 ( 5 ): 1289 - 1307 .

LIU T , TIAN B , AI Y F , et al . Parallel reinforcement learning-based energy efficiency improvement for a cyber-physical system [J ] . IEEE/CAA Journal of Automatica Sinica , 2020 , 7 ( 2 ): 617 - 626 .

WEI Q L , LI H Y , WANG F Y . Parallel control for continuous-time linear systems:a case study [J ] . IEEE/CAA Journal of Automatica Sinica , 2020 , 7 ( 4 ): 919 - 928 .

LI X S , LIU Y T , WANG K F , et al . A recurrent attention and interaction model for pedestrian trajectory prediction [J ] . IEEE/CAA Journal of Automatica Sinica , 2020 , 7 ( 5 ): 1361 - 1370 .

LIU K H , YE Z H , GUO H Y , et al . FISS GAN:a generative adversarial network for foggy image semantic segmentation [J ] . IEEE/CAA Journal of Automatica Sinica , 2021 , 8 ( 8 ): 1428 - 1439 .

LU J W , WEI Q L , WANG F Y . Parallel control for optimal tracking via adaptive dynamic programming [J ] . IEEE/CAA Journal of Automatica Sinica , 2020 , 7 ( 6 ): 1662 - 1674 .

SUN C , VIANNEY J M U , LI Y , et al . Proximity based automatic data annotation for autonomous driving [J ] . IEEE/CAA Journal of Automatica Sinica , 2020 , 7 ( 2 ): 395 - 404 .

WANG S Y , HOUSDEN J , BAI T X , et al . Robotic intra-operative ultrasound:virtual environments and parallel systems [J ] . IEEE/CAA Journal of Automatica Sinica , 2021 , 8 ( 5 ): 1095 - 1106 .

ZU C Y , YANG C , WANG J , et al . Simulation and field testing of multiple vehicles collision avoidance algorithms [J ] . IEEE/CAA Journal of Automatica Sinica , 2020 , 7 ( 4 ): 1045 - 1063 .

TAN J Y , XU C L , LI L , et al . Guidance control for parallel parking tasks [J ] . IEEE/CAA Journal of Automatica Sinica , 2020 , 7 ( 1 ): 301 - 306 .

张俊 , 许沛东 , 王飞跃 . 平行系统和数字孪生的一种数据驱动形式表示及计算框架 [J ] . 自动化学报 , 2020 , 46 ( 7 ): 1346 - 1356 .

ZHANG J , XU P D , WANG F Y . Parallel systems and digital twins:a data-driven mathematical representation and computational framework [J ] . Acta Automatica Sinica , 2020 , 46 ( 7 ): 1346 - 1356 .

JAYNES C , SEALES W B , CALVERT K , et al . The metaverse:a networked collection of inexpensive,self-configuring,immersive environments [C ] // Proceedings of the Workshop on Virtual Environments 2003 - EGVE’03 . New York:ACM Press , 2003 : 115 - 124 .

DUAN H H , LI J Y , FAN S Z , et al . Metaverse for social good:a university campus prototype [C ] // Proceedings of the 29th ACM International Conference on Multimedia . New York:ACM Press , 2021 : 153 - 161 .

WANG F Y . Parallel intelligence in metaverses:welcome to Hanoi! [J ] . IEEE Intelligent Systems , 2022 , 37 ( 1 ): 16 - 20 .

GUO C , BAI T X , LU Y , et al . Skywork-daVinci:a novel CPSS-based painting support system [C ] // Proceedings of 2020 IEEE 16th International Conference on Automation Science and Engineering . Piscataway:IEEE Press , 2020 : 673 - 678 .

GUO C , BAI T X , WANG X , et al . ShadowPainter:active learning enabled robotic painting through visual measurement and reproduction of the artistic creation process [J ] . Journal of Intelligent ＆ Robotic Systems , 2022 , 105 ( 3 ): 1 - 17 .

KOTOVENKO D , WRIGHT M , HEIMBRECHT A , et al . Rethinking style transfer:from pixels to parameterized brushstrokes [C ] // Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2021 : 12191 - 12200 .

CORNIA M , STEFANINI M , BARALDI L , et al . Meshed-memory transformer for image captioning [C ] // Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2020 : 10575 - 10584 .

LU Y , GUO C , DAI X Y , et al . Image captioning on fine art paintings via virtual paintings [C ] // Proceedings of 2021 IEEE 1st International Conference on Digital Twins and Parallel Intelligence . Piscataway:IEEE Press , 2021 : 156 - 159 .

LU Y , GUO C , DAI X Y , et al . Data-efficient image captioning of fine art paintings via virtual-real semantic alignment training [J ] . Neurocomputing , 2022 , 490 : 163 - 180 .

浏览量

823

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

平行夏尔希里：生态资源智能管护及其可持续发展新途径

元宇宙下的平行交通系统

平行安全：基于CPSS的生成式对抗安全智能系统

平行艺术：人机协作的艺术创作

平行机器：一种智能机器的管理与控制框架