Intelligent art factory: achieving creative automation through parallel art agents

GUO Chao; DAI Xingyuan; LIAO Weilong; QI Jueyu; LU Yue; NI Qinghua; WANG Lin; WANG Fei-Yue

doi:10.11959/j.issn.2096-6652.202421

您当前的位置：

首页 >

文章列表页 >

Intelligent art factory: achieving creative automation through parallel art agents

Special Topic: New Liberal Arts | 更新时间：2024-08-06

- Intelligent art factory: achieving creative automation through parallel art agents
- Chinese Journal of Intelligent Science and Technology Vol. 6, Issue 2, Pages: 179-188(2024)
- 作者机构：
  
  1.中国科学院自动化研究所，北京 100190
  2.北京科技大学，北京 100083
  3.山东大学控制科学与工程学院，山东济南 250061
  4.澳门科技大学，澳门 999078
  5.加州大学伯克利分校劳伦斯国家实验室，美国伯克利 94720
- 作者简介：
- 基金信息：
  
  The National Natural Science Fundation of China(62306319)
- DOI：10.11959/j.issn.2096-6652.202421
  CLC： TP39
- Received：06 April 2024，
  
  Revised：2024-05-22，
  
  Published：15 June 2024
- 稿件说明：
移动端阅览
郭超,戴星原,廖玮珑等.智能艺术工厂：基于平行艺术智能体的创作自动化[J].智能科学与技术学报,2024,06(02):179-188.

GUO Chao,DAI Xingyuan,LIAO Weilong,et al.Intelligent art factory: achieving creative automation through parallel art agents[J].Chinese Journal of Intelligent Science and Technology,2024,06(02):179-188.
郭超,戴星原,廖玮珑等.智能艺术工厂：基于平行艺术智能体的创作自动化[J].智能科学与技术学报,2024,06(02):179-188. DOI： 10.11959/j.issn.2096-6652.202421.

GUO Chao,DAI Xingyuan,LIAO Weilong,et al.Intelligent art factory: achieving creative automation through parallel art agents[J].Chinese Journal of Intelligent Science and Technology,2024,06(02):179-188. DOI： 10.11959/j.issn.2096-6652.202421.

摘要

近年来，多模态内容生成模型迅速发展，极大地提高了创意工作者的创作效率。然而，现有方法仍然需要领域专业知识，无法实现创作全流程的自动化，难以广泛服务于普通人群。因此，提出智能艺术工厂的理念和架构，基于大模型和人工智能智能体技术，构建智能艺术工厂的工作流程、基本功能和实现方法，建立面向创意创作的智能体员工和数字人团队，通过生物人、数字人和机器人的跨物理、信息、社会空间协作，实现创作流程自动化和创作知识自动化。智能艺术工厂将进一步解放大众的创造力。

Abstract

In recent years

multimodal content generation models have rapidly evolved

significantly improving the efficiency of creative workers. However

existing methods still require expertise and cannot realize the automation of the whole creation process

making it difficult to widely serve the public. This paper proposes the concept and architecture of intelligent art factory

based on advanced foundation models and AI agent systems. Creative production oriented agents and digital teams

in conjunction with the basic workflow

functions

and methods of the intelligent art factory are established. The creative process automation and creative knowledge automation are achieved through collaboration among biological humans

digital humans

and robots in cyber-physical-social spaces. The intelligent art factory seeks to further liberate the creativity of the public.

关键词

Keywords

references

ROMBACH R, BLATTMANN A, LORENZ D, et al. High-resolution image synthesis with latent diffusion models[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE Press, 2022: 10674-10685.

LIU Y X, ZHANG K, LI Y, et al. Sora: a review on background, technology, limitations, and opportunities of large vision models[EB]. arXiv preprint, 2024, arXiv: 2402.17177.

WANG F Y, MIAO Q H, LI X, et al. What does ChatGPT say: the DAO from algorithmic intelligence to linguistic intelligence[J]. IEEE/CAA Journal of Automatica Sinica, 2023, 10(3): 575-579.

WANG F Y, MIAO Q H, LI L X, et al. When does sora show: the beginning of TAO to imaginative intelligence and scenarios engineering[J]. IEEE/CAA Journal of Automatica Sinica, 2024, 11(4): 809-815.

OUYANG L, WU J, XU J, et al. Training language models to follow instructions with human feedback[EB]. arXiv preprint, 2022: arXiv: 2203.02155.

WEI J, TAY Y, BOMMASANI R, et al. Emergent abilities of large language models[EB]. arXiv preprint, 2022, arXiv: 2206.07682.

WANG L, MA C, FENG X Y, et al. A survey on large language model based autonomous agents[EB]. arXiv preprint, 2023, arXiv: 2308.11432.

XI Z H, CHEN W X, GUO X, et al. The rise and potential of large language model based agents: a survey[EB]. arXiv preprint, 2023, arXiv: 2309.07864.

LI J K, WANG S Y, ZHANG M, et al. Agent hospital: a simulacrum of hospital with evolvable medical agents[EB]. arXiv preprint, 2024, arXiv: 2405.02957.

GUO C, LU Y, DOU Y, et al. Can ChatGPT boost artistic creation: the need of imaginative intelligence for parallel art[J]. IEEE/CAA Journal of Automatica Sinica, 2023, 10(4): 835-838.

WANG B, LI Y L, LV Z Y, et al. LAVE: LLM-powered agent assistance and language augmentation for video editing[C]//Proceedings of the 29th International Conference on Intelligent User Interfaces. New York: ACM, 2024: 699-714.

TAN W H, DING Z L, ZHANG W T, et al. Towards general computer control: a multimodal agent for red dead redemption II as a case study[EB]. arXiv preprint, 2024, arXiv: 2403.03186.

QIAN C, LIU W, LIU H Z, et al. ChatDev: communicative agents for software development[EB]. arXiv preprint, 2023, arXiv: 2307.07924.

HONG S R, ZHUGE M C, CHEN J, et al. MetaGPT: meta programming for A multi-agent collaborative framework[EB]. arXiv preprint, 2023, arXiv: 2308.00352.

YAO S Y, ZHAO J, YU D, et al. ReAct: synergizing reasoning and acting in language models[EB]. arXiv preprint, 2022, arXiv: 2210.03629.

WANG L, MA C, FENG X Y, et al. A survey on large language model based autonomous agents[J]. Frontiers of Computer Science, 2024, 18(6): 186345.

WEI J, WANG X Z, SCHUURMANS D, et al. Chain of thought prompting elicits reasoning in large language models[EB]. arXiv preprint, 2022, arXiv: 2201.11903.

LIU R B, YANG R X, JIA C Y, et al. Training socially aligned language models on simulated social interactions[EB]. arXiv preprint, 2023, arXiv: 2305.16960.

LI G, HAMMOUD H, ITANI H, et al. Camel: communicative agents for "mind" exploration of large language model society[J]. Advances in Neural Information Processing Systems, 2023, 36: 51991-52008.

LIANG T, HE Z W, JIAO W X, et al. Encouraging divergent thinking in large language models through multi-agent debate[EB]. arXiv preprint, 2023, arXiv: 2305.19118.

PAUL M, MAGLARAS L, FERRAG M A, et al. Digitization of healthcare sector: a study on privacy and security concerns[J]. ICT Express, 2023, 9(4): 571-588.

PARIKH D, ZITNICK C L. Exploring crowd co-creation scenarios for sketches[EB]. arXiv preprint, 2020, arXiv: 2005.07328.

DAVIS N, HSIAO C P, YASHRAJ SINGH K, et al. Empirically studying participatory sense-making in abstract drawing with a co-creative cognitive agent[C]//Proceedings of the 21st International Conference on Intelligent User Interfaces. New York: ACM, 2016: 196-207.

LEE Y J, ZITNICK C L, COHEN M F. ShadowDraw: real-time user guidance for freehand drawing[J]. ACM Transactions on Graphics, 2011, 30(4): 1-10.

HUANG Z Y, PENG Y C, HIBINO T, et al. dualFace: two-stage drawing guidance for freehand portrait sketching[EB]. arXiv preprint, 2021, arXiv: 2104.12297.

KARIMI P, REZWANA J, SIDDIQUI S, et al. Creative sketching partner: an analysis of human-AI co-creativity[C]//Proceedings of the 25th International Conference on Intelligent User Interfaces. New York: ACM, 2020: 221-230.

WANG C, PÉREZ-D'ARPINO C, XU D F, et al. Co-GAIL: learning diverse strategies for human-robot collaboration[EB]. arXiv preprint, 2021, arXiv: 2108.06038.

LIU H, HU M H, CHEN Y Z, et al. Angel's girl for blind painters: an efficient painting navigation system validated by multimodal evaluation approach[EB]. arXiv preprint, 2021, arXiv: 2107.12921.

SUH S, ZHAO J, LAW E. CodeToon: story ideation, auto comic generation, and structure mapping for code-driven storytelling[C]//Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. New York: ACM, 2022: 1-16.

CHAKRABARTY T, PADMAKUMAR V, BRAHMAN F, et al. Creativity support in the age of large language models: an empirical study involving emerging writers[EB]. arXiv preprint, 2023, arXiv: 2309.12570.

王飞跃. 平行系统方法与复杂系统的管理和控制[J]. 控制与决策, 2004, 19(5): 485-489, 514.

WANG F Y. Parallel system methods for management and control of complex systems[J]. Control and Decision, 2004, 19(5): 485-489, 514.

WANG F Y. Toward a paradigm shift in social computing: the ACP approach[J]. IEEE Intelligent Systems, 2007, 22(5): 65-67.

郭超, 鲁越, 林懿伦, 等. 平行艺术:人机协作的艺术创作[J]. 智能科学与技术学报, 2019, 1(4): 335-341.

GUO C, LU Y, LIN Y L, et al. Parallel art: artistic creation under human-machine collaboration[J]. Chinese Journal of Intelligent Science and Technology, 2019, 1(4): 335-341.

XUE X, YU X N, ZHOU D Y, et al. Computational experiments for complex social systems: integrated design of experiment system[J]. IEEE/CAA Journal of Automatica Sinica, 2024, 11(5): 1175-1189.

GUO C, BAI T X, LU Y, et al. Skywork-daVinci: a novel CPSS-based painting support system[C]//Proceedings of the 2020 IEEE 16th International Conference on Automation Science and Engineering(CASE). Piscataway: IEEE Press, 2020: 673-678.

倪清桦, 郭超, 王飞跃. 平行戏剧:新时代戏剧的人机协同创作与智能管理[J]. 智能科学与技术学报, 2023, 5(4): 436-445.

NI Q H, GUO C, WANG F Y. Parallel theaters: human-machine collaborative creation and intelligent management for theatrical art[J]. Chinese Journal of Intelligent Science and Technology, 2023, 5(4): 436-445.

王春法, 王飞跃, 鲁越, 等. 平行博物馆: 新时代博物馆运营的智能管理与控制[J]. 智能科学与技术学报, 2021, 3(2): 125-136.

WANG C F, WANG F Y, LU Y, et al. Parallel museums: intelligent management and control of museum operations in the new era[J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(2): 125-136.

鲁越, 郭超, 潘晴, 等. 平行博物馆系统: 框架、平台、方法及应用[J]. 模式识别与人工智能, 2023, 36(7): 575-589.

LU Y, GUO C, PAN Q, et al. Parallel museum systems: framework, platform, methods and applications[J]. Pattern Recognition and Artificial Intelligence, 2023, 36(7): 575-589.

GUO C, BAI T X, WANG X, et al. ShadowPainter: active learning enabled robotic painting through visual measurement and reproduction of the artistic creation process[J]. Journal of Intelligent & Robotic Systems, 2022, 105(3): 61.

GUO C, DOU Y, BAI T X, et al. ArtVerse: a paradigm for parallel human-machine collaborative painting creation in metaverses[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2023, 53(4): 2200-2208.

LU Y, GUO C, DAI X Y, et al. ArtCap: a dataset for image captioning of fine art paintings[J]. IEEE Transactions on Computational Social Systems, 2024, 11(1): 576-587.

LU Y, GUO C, DOU Y, et al. Could ChatGPT imagine: content control for artistic painting generation via large language models[J]. Journal of Intelligent & Robotic Systems, 2023, 109(2): 39.

LU Y, GUO C, LIN Y L, et al. Computational aesthetics of fine art paintings: the state of the art and outlook[J]. Acta Automatica Sinica, 2020, 46(11): 2239-2259.

LU Y, GUO C, DAI X Y, et al. Data-efficient image captioning of fine art paintings via virtual-real semantic alignment training[J]. Neurocomputing, 2022, 490: 163-180.

LU Y, GUO C, DAI X Y, et al. Generating emotion descriptions for fine art paintings via multiple painting representations[J]. IEEE Intelligent Systems, 2023, 38(3): 31-40.

DAI X Y, GUO C, TANG Y, et al. VistaRAG: toward safe and trustworthy autonomous driving through retrieval-augmented generation[J]. IEEE Transactions on Intelligent Vehicles, 2024, 9(4): 4579-4582.

WANG F Y. The DAO to MetaControl for MetaSystems in metaverses: the system of parallel control systems for knowledge automation and control intelligence in CPSS[J]. IEEE/CAA Journal of Automatica Sinica, 2022, 9(11): 1899-1908.

Views

299

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Parallel art:artistic creation under human-machine collaboration

Cerberus: a cross-site social bot detection system based on deep learning

Related Author

Chao GUO

Yue LU

Yilun LIN

Fan ZHUO

Fei-Yue WANG

Related Institution

The State Key Laboratory for Management and Control of Complex Systems, Institute of Automation, Chinese Academy of Sciences

University of Chinese Academy of Sciences

Central Academy of Fine Arts

Qingdao Academy of Intelligent Industries

Postal code：100079
Tel：（010）53879206 Email：cjist@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：162003 Visits Today：166

⁰