智能艺术工厂：基于平行艺术智能体的创作自动化

郭超; 戴星原; 廖玮珑; 祁珏瑜; 鲁越; 倪清桦; WANG Lin; 王飞跃

doi:10.11959/j.issn.2096-6652.202421

您当前的位置：

首页 >

文章列表页 >

智能艺术工厂：基于平行艺术智能体的创作自动化

专题：新文科 | 更新时间：2024-08-06

- 智能艺术工厂：基于平行艺术智能体的创作自动化
- Intelligent art factory: achieving creative automation through parallel art agents
- 智能科学与技术学报 2024年6卷第2期页码：179-188
- 作者机构：
  
  1.中国科学院自动化研究所，北京 100190
  2.北京科技大学，北京 100083
  3.山东大学控制科学与工程学院，山东济南 250061
  4.澳门科技大学，澳门 999078
  5.加州大学伯克利分校劳伦斯国家实验室，美国伯克利 94720
- 作者简介：
  
  [ "郭超（1992- ），男，博士，中国科学院自动化研究所助理研究员，主要研究方向为机器学习、深度学习、机器艺术创作、智能机器人系统。" ]
  [ "戴星原（1993- ），男，博士，中国科学院自动化研究所多模态人工智能系统全国重点实验室助理研究员，主要研究方向为平行艺术、平行学习、平行智能。" ]
  [ "廖玮珑（2003- ），男，北京科技大学本科生，主要研究方向为AI智能体、智能机器人系统。" ]
  [ "祁珏瑜（2002- ），男，北京科技大学本科生，主要研究方向为图像生成模型、扩散模型。" ]
  [ "鲁越（1994- ），男，博士，山东大学控制科学与工程学院博士后，主要研究方向为机器学习、小样本学习、机器人系统。" ]
  [ "倪清桦（1999- ），女，澳门科技大学博士生，主要研究方向为平行智能、数字孪生、智能系统。" ]
  [ "Wang Lin（1976- ），女，博士，德国柏林德意志歌剧院、奥地利格拉茨歌剧院歌剧作曲家，加州大学伯克利劳伦斯国家实验室交叉学科研究员，南京艺术学院客座教授。主要创作方向为歌剧及交响乐作曲、剧作家、戏剧构作，主要研究方向为艺术与科学交叉、超学科研究和创作。" ]
  [ "王飞跃（1961- ），男，博士，中国科学院自动化研究所复杂系统管理与控制国家重点实验室主任，兼任中国国家博物馆科学与艺术研究所所长。主要研究方向为平行系统的方法与应用、社会计算、平行智能、知识自动化。" ]
- 基金信息：
  
  国家自然科学基金项目(62306319)
- DOI：10.11959/j.issn.2096-6652.202421
  中图分类号： TP39
- 收稿：2024-04-06，
  
  修回：2024-05-22，
  
  纸质出版：2024-06-15
- 稿件说明：
移动端阅览
郭超,戴星原,廖玮珑等.智能艺术工厂：基于平行艺术智能体的创作自动化[J].智能科学与技术学报,2024,06(02):179-188.

GUO Chao,DAI Xingyuan,LIAO Weilong,et al.Intelligent art factory: achieving creative automation through parallel art agents[J].Chinese Journal of Intelligent Science and Technology,2024,06(02):179-188.
郭超,戴星原,廖玮珑等.智能艺术工厂：基于平行艺术智能体的创作自动化[J].智能科学与技术学报,2024,06(02):179-188. DOI： 10.11959/j.issn.2096-6652.202421.

GUO Chao,DAI Xingyuan,LIAO Weilong,et al.Intelligent art factory: achieving creative automation through parallel art agents[J].Chinese Journal of Intelligent Science and Technology,2024,06(02):179-188. DOI： 10.11959/j.issn.2096-6652.202421.

摘要

近年来，多模态内容生成模型迅速发展，极大地提高了创意工作者的创作效率。然而，现有方法仍然需要领域专业知识，无法实现创作全流程的自动化，难以广泛服务于普通人群。因此，提出智能艺术工厂的理念和架构，基于大模型和人工智能智能体技术，构建智能艺术工厂的工作流程、基本功能和实现方法，建立面向创意创作的智能体员工和数字人团队，通过生物人、数字人和机器人的跨物理、信息、社会空间协作，实现创作流程自动化和创作知识自动化。智能艺术工厂将进一步解放大众的创造力。

Abstract

In recent years

multimodal content generation models have rapidly evolved

significantly improving the efficiency of creative workers. However

existing methods still require expertise and cannot realize the automation of the whole creation process

making it difficult to widely serve the public. This paper proposes the concept and architecture of intelligent art factory

based on advanced foundation models and AI agent systems. Creative production oriented agents and digital teams

in conjunction with the basic workflow

functions

and methods of the intelligent art factory are established. The creative process automation and creative knowledge automation are achieved through collaboration among biological humans

digital humans

and robots in cyber-physical-social spaces. The intelligent art factory seeks to further liberate the creativity of the public.

关键词

Keywords

references

ROMBACH R, BLATTMANN A, LORENZ D, et al. High-resolution image synthesis with latent diffusion models[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2022: 10674-10685.

LIU Y X, ZHANG K, LI Y, et al. Sora: a review on background, technology, limitations, and opportunities of large vision models[EB]. arXiv preprint, 2024, arXiv: 2402.17177.

WANG F Y, MIAO Q H, LI X, et al. What does ChatGPT say: the DAO from algorithmic intelligence to linguistic intelligence[J]. IEEE/CAA Journal of Automatica Sinica, 2023, 10(3): 575-579.

WANG F Y, MIAO Q H, LI L X, et al. When does sora show: the beginning of TAO to imaginative intelligence and scenarios engineering[J]. IEEE/CAA Journal of Automatica Sinica, 2024, 11(4): 809-815.

OUYANG L, WU J, XU J, et al. Training language models to follow instructions with human feedback[EB]. arXiv preprint, 2022: arXiv: 2203.02155.

WEI J, TAY Y, BOMMASANI R, et al. Emergent abilities of large language models[EB]. arXiv preprint, 2022, arXiv: 2206.07682.

WANG L, MA C, FENG X Y, et al. A survey on large language model based autonomous agents[EB]. arXiv preprint, 2023, arXiv: 2308.11432.

XI Z H, CHEN W X, GUO X, et al. The rise and potential of large language model based agents: a survey[EB]. arXiv preprint, 2023, arXiv: 2309.07864.

LI J K, WANG S Y, ZHANG M, et al. Agent hospital: a simulacrum of hospital with evolvable medical agents[EB]. arXiv preprint, 2024, arXiv: 2405.02957.

GUO C, LU Y, DOU Y, et al. Can ChatGPT boost artistic creation: the need of imaginative intelligence for parallel art[J]. IEEE/CAA Journal of Automatica Sinica, 2023, 10(4): 835-838.

WANG B, LI Y L, LV Z Y, et al. LAVE: LLM-powered agent assistance and language augmentation for video editing[C]//Proceedings of the 29th International Conference on Intelligent User Interfaces. New York: ACM, 2024: 699-714.

TAN W H, DING Z L, ZHANG W T, et al. Towards general computer control: a multimodal agent for red dead redemption II as a case study[EB]. arXiv preprint, 2024, arXiv: 2403.03186.

QIAN C, LIU W, LIU H Z, et al. ChatDev: communicative agents for software development[EB]. arXiv preprint, 2023, arXiv: 2307.07924.

HONG S R, ZHUGE M C, CHEN J, et al. MetaGPT: meta programming for A multi-agent collaborative framework[EB]. arXiv preprint, 2023, arXiv: 2308.00352.

YAO S Y, ZHAO J, YU D, et al. ReAct: synergizing reasoning and acting in language models[EB]. arXiv preprint, 2022, arXiv: 2210.03629.

WANG L, MA C, FENG X Y, et al. A survey on large language model based autonomous agents[J]. Frontiers of Computer Science, 2024, 18(6): 186345.

WEI J, WANG X Z, SCHUURMANS D, et al. Chain of thought prompting elicits reasoning in large language models[EB]. arXiv preprint, 2022, arXiv: 2201.11903.

LIU R B, YANG R X, JIA C Y, et al. Training socially aligned language models on simulated social interactions[EB]. arXiv preprint, 2023, arXiv: 2305.16960.

LI G, HAMMOUD H, ITANI H, et al. Camel: communicative agents for "mind" exploration of large language model society[J]. Advances in Neural Information Processing Systems, 2023, 36: 51991-52008.

LIANG T, HE Z W, JIAO W X, et al. Encouraging divergent thinking in large language models through multi-agent debate[EB]. arXiv preprint, 2023, arXiv: 2305.19118.

PAUL M, MAGLARAS L, FERRAG M A, et al. Digitization of healthcare sector: a study on privacy and security concerns[J]. ICT Express, 2023, 9(4): 571-588.

PARIKH D, ZITNICK C L. Exploring crowd co-creation scenarios for sketches[EB]. arXiv preprint, 2020, arXiv: 2005.07328.

DAVIS N, HSIAO C P, YASHRAJ SINGH K, et al. Empirically studying participatory sense-making in abstract drawing with a co-creative cognitive agent[C]//Proceedings of the 21st International Conference on Intelligent User Interfaces. New York: ACM, 2016: 196-207.

LEE Y J, ZITNICK C L, COHEN M F. ShadowDraw: real-time user guidance for freehand drawing[J]. ACM Transactions on Graphics, 2011, 30(4): 1-10.

HUANG Z Y, PENG Y C, HIBINO T, et al. dualFace: two-stage drawing guidance for freehand portrait sketching[EB]. arXiv preprint, 2021, arXiv: 2104.12297.

KARIMI P, REZWANA J, SIDDIQUI S, et al. Creative sketching partner: an analysis of human-AI co-creativity[C]//Proceedings of the 25th International Conference on Intelligent User Interfaces. New York: ACM, 2020: 221-230.

WANG C, PÉREZ-D'ARPINO C, XU D F, et al. Co-GAIL: learning diverse strategies for human-robot collaboration[EB]. arXiv preprint, 2021, arXiv: 2108.06038.

LIU H, HU M H, CHEN Y Z, et al. Angel's girl for blind painters: an efficient painting navigation system validated by multimodal evaluation approach[EB]. arXiv preprint, 2021, arXiv: 2107.12921.

SUH S, ZHAO J, LAW E. CodeToon: story ideation, auto comic generation, and structure mapping for code-driven storytelling[C]//Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. New York: ACM, 2022: 1-16.

CHAKRABARTY T, PADMAKUMAR V, BRAHMAN F, et al. Creativity support in the age of large language models: an empirical study involving emerging writers[EB]. arXiv preprint, 2023, arXiv: 2309.12570.

王飞跃. 平行系统方法与复杂系统的管理和控制[J]. 控制与决策, 2004, 19(5): 485-489, 514.

WANG F Y. Parallel system methods for management and control of complex systems[J]. Control and Decision, 2004, 19(5): 485-489, 514.

WANG F Y. Toward a paradigm shift in social computing: the ACP approach[J]. IEEE Intelligent Systems, 2007, 22(5): 65-67.

郭超, 鲁越, 林懿伦, 等. 平行艺术:人机协作的艺术创作[J]. 智能科学与技术学报, 2019, 1(4): 335-341.

GUO C, LU Y, LIN Y L, et al. Parallel art: artistic creation under human-machine collaboration[J]. Chinese Journal of Intelligent Science and Technology, 2019, 1(4): 335-341.

XUE X, YU X N, ZHOU D Y, et al. Computational experiments for complex social systems: integrated design of experiment system[J]. IEEE/CAA Journal of Automatica Sinica, 2024, 11(5): 1175-1189.

GUO C, BAI T X, LU Y, et al. Skywork-daVinci: a novel CPSS-based painting support system[C]//Proceedings of the 2020 IEEE 16th International Conference on Automation Science and Engineering(CASE). Piscataway: IEEE Press, 2020: 673-678.

倪清桦, 郭超, 王飞跃. 平行戏剧:新时代戏剧的人机协同创作与智能管理[J]. 智能科学与技术学报, 2023, 5(4): 436-445.

NI Q H, GUO C, WANG F Y. Parallel theaters: human-machine collaborative creation and intelligent management for theatrical art[J]. Chinese Journal of Intelligent Science and Technology, 2023, 5(4): 436-445.

王春法, 王飞跃, 鲁越, 等. 平行博物馆: 新时代博物馆运营的智能管理与控制[J]. 智能科学与技术学报, 2021, 3(2): 125-136.

WANG C F, WANG F Y, LU Y, et al. Parallel museums: intelligent management and control of museum operations in the new era[J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(2): 125-136.

鲁越, 郭超, 潘晴, 等. 平行博物馆系统: 框架、平台、方法及应用[J]. 模式识别与人工智能, 2023, 36(7): 575-589.

LU Y, GUO C, PAN Q, et al. Parallel museum systems: framework, platform, methods and applications[J]. Pattern Recognition and Artificial Intelligence, 2023, 36(7): 575-589.

GUO C, BAI T X, WANG X, et al. ShadowPainter: active learning enabled robotic painting through visual measurement and reproduction of the artistic creation process[J]. Journal of Intelligent & Robotic Systems, 2022, 105(3): 61.

GUO C, DOU Y, BAI T X, et al. ArtVerse: a paradigm for parallel human-machine collaborative painting creation in metaverses[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2023, 53(4): 2200-2208.

LU Y, GUO C, DAI X Y, et al. ArtCap: a dataset for image captioning of fine art paintings[J]. IEEE Transactions on Computational Social Systems, 2024, 11(1): 576-587.

LU Y, GUO C, DOU Y, et al. Could ChatGPT imagine: content control for artistic painting generation via large language models[J]. Journal of Intelligent & Robotic Systems, 2023, 109(2): 39.

LU Y, GUO C, LIN Y L, et al. Computational aesthetics of fine art paintings: the state of the art and outlook[J]. Acta Automatica Sinica, 2020, 46(11): 2239-2259.

LU Y, GUO C, DAI X Y, et al. Data-efficient image captioning of fine art paintings via virtual-real semantic alignment training[J]. Neurocomputing, 2022, 490: 163-180.

LU Y, GUO C, DAI X Y, et al. Generating emotion descriptions for fine art paintings via multiple painting representations[J]. IEEE Intelligent Systems, 2023, 38(3): 31-40.

DAI X Y, GUO C, TANG Y, et al. VistaRAG: toward safe and trustworthy autonomous driving through retrieval-augmented generation[J]. IEEE Transactions on Intelligent Vehicles, 2024, 9(4): 4579-4582.

WANG F Y. The DAO to MetaControl for MetaSystems in metaverses: the system of parallel control systems for knowledge automation and control intelligence in CPSS[J]. IEEE/CAA Journal of Automatica Sinica, 2022, 9(11): 1899-1908.

浏览量

299

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

平行艺术：人机协作的艺术创作

Cerberus：基于深度学习的跨站社交网络机器人检测系统