平行音乐：大模型时代的人机混合音乐创演

倪清桦; 鲁越; 林飞; 黄峻; 王艺瑾; 林伟华; 王飞跃

doi:10.11959/j.issn.2096-6652.202423

您当前的位置：

首页 >

文章列表页 >

平行音乐：大模型时代的人机混合音乐创演

专题：新文科 | 更新时间：2024-08-06

- 平行音乐：大模型时代的人机混合音乐创演
- Parallel music: human-machine hybrid music creation and performance in the era of large models
- 智能科学与技术学报 2024年6卷第2期页码：150-163
- 作者机构：
  
  1.澳门科技大学，澳门 999078
  2.山东大学控制科学与工程学院，山东济南 250061
  3.加州大学洛杉矶分校，美国洛杉矶 90095
  4.亚利桑那大学，美国图森 85721
  5.中国科学院自动化研究所复杂系统管理与控制国家重点实验室，北京 100190
- 作者简介：
  
  [ "倪清桦（1999- ），女，澳门科技大学博士生。主要研究方向为平行智能、数字孪生、智能系统等。" ]
  [ "鲁越（1994- ），男，博士，山东大学控制科学与工程学院博士后，主要研究方向为机器学习、小样本学习、机器人系统。" ]
  [ "林飞（1994- ），男，澳门科技大学博士生，主要研究方向为平行系统、大语言模型、多模态感知、生成式人工智能。" ]
  [ "黄峻（1998- ），男，澳门科技大学博士生，主要研究方向为平行智能、自动驾驶轨迹预测规划、提示工程。" ]
  [ "王艺瑾（1996- ），女，美国加州大学洛杉矶分校音乐学院音乐艺术博士生，主要研究方向为古典音乐、单簧管演奏等。" ]
  [ "林伟华（1961- ），男，博士，美国亚利桑那大学教授，资深音乐爱好者，主要研究方向为智能交通系统系统优化、物流和交通网络分析。" ]
  [ "王飞跃（1961- ），男，博士，中国科学院自动化研究所复杂系统管理与控制国家重点实验室主任，兼任中国国家博物馆科学与艺术研究所所长，主要研究方向为平行系统的方法与应用、社会计算、平行智能、知识自动化。" ]
- 基金信息：
  
  澳门特别行政区科学技术发展基金(0093/2023/RIA2)
- DOI：10.11959/j.issn.2096-6652.202423
  中图分类号： TP39
- 收稿：2024-04-18，
  
  修回：2024-05-27，
  
  纸质出版：2024-06-15
- 稿件说明：
移动端阅览
倪清桦,鲁越,林飞等.平行音乐：大模型时代的人机混合音乐创演[J].智能科学与技术学报,2024,06(02):150-163.

NI Qinghua,LU Yue,LIN Fei,et al.Parallel music: human-machine hybrid music creation and performance in the era of large models[J].Chinese Journal of Intelligent Science and Technology,2024,06(02):150-163.
倪清桦,鲁越,林飞等.平行音乐：大模型时代的人机混合音乐创演[J].智能科学与技术学报,2024,06(02):150-163. DOI： 10.11959/j.issn.2096-6652.202423.

NI Qinghua,LU Yue,LIN Fei,et al.Parallel music: human-machine hybrid music creation and performance in the era of large models[J].Chinese Journal of Intelligent Science and Technology,2024,06(02):150-163. DOI： 10.11959/j.issn.2096-6652.202423.

摘要

随着声音艺术等垂直领域基础模型的迅速发展，人工智能与音乐创作表演呈现愈发融合的趋势。面向音乐创演的流程与需求，提出新型音乐创演框架——平行音乐系统，该系统基于平行系统理论与人工社会、计算实验和平行执行方法，通过模拟真实世界的音乐创演环境，构建高度逼真的虚拟音乐创演场景，在虚拟和真实音乐系统间实现实时无缝交互，设计大模型时代音乐创演的人工系统、计算实验、平行执行各环节技术路线，形成由自然人、数字人、机器人组成的人机混合音乐创演团队。阐述了平行音乐的理论框架和关键技术，分析其在现代音乐领域中的创新意义和潜在价值，并探讨了平行音乐在乐曲生成和音乐治疗等场景中的应用案例。旨在利用大模型时代的人工智能技术赋能传统音乐创作路径，推动音乐创演的技术革新进程，释放音乐家的艺术创新潜力，以期为新时代音乐创演提供新视角与启发。

Abstract

As foundational models in specialized fields like sound art have rapidly advanced

the convergence of artificial intelligence with music creation and performance has become increasingly pronounced. A novel framework for music creation and performance called the parallel music system was proposed. Grounded in parallel systems theory and the artificial system

computational experiments

parallel execution (ACP) approach

this system stimulates the real-world music creation and performance environments to build highly authentic virtual scenarios. It facilitates real-time

seamless interactions between the virtual and real music systems and outlines the technological strategies for music creation and performance in the era of large models

incorporating a hybrid team of humans

digital beings

and robots. Additionally

the theoretical framework of parallel music and its critical technologies are discussed

the innovative contributions and potential in the modern music field are evaluated

the applications in scenarios like music generation and music therapy are examined. The objective is to harness the emerging AI technologies of the large model era to enhance traditional paths in music creation

drive technological advancements in music performance

unlock the creative potential of musicians

and offer fresh perspectives and inspirations for music creation and performance in the new era.

关键词

Keywords

references

CROSS I. Music, mind and evolution[J]. Psychology of Music, 2001, 29: 95-102.

JUSLIN P N, VÄSTFJÄLL D. Emotional responses to music: the need to consider underlying mechanisms[J]. The Behavioral and Brain Sciences, 2008, 31(5): 559-575.

MIRANDA E R, WANDERLEY M. New digital musical instruments: control and interaction beyond the keyboard (computer music and digital audio series)[M]. Wisconsin: A-R Editions, Inc., 2006.

王飞跃. 平行艺术：从艺术的智能化到智能的艺术化[R]. 2017.

WANG F Y. Parallel art: from intelligent art to artistic intelligence[R]. 2017.

王飞跃. 人工社会、计算实验、平行系统：关于复杂社会经济系统计算研究的讨论[J]. 复杂系统与复杂性科学, 2004, 1(4): 25-35.

WANG F Y. Artificial societies, computational experiments, and parallel systems: a discussion on computational theory of complex social- economic systems[J]. Complex Systems and Complexity Science, 2004, 1(4): 25-35.

KOEPKE P. The composing process[J]. American Music Teacher, 1958, 7(5): 6-20.

Erickson R. Composing music[J]. Perspectives of New Music, 1998, 26(2): 86-95.

HULSE B. Becoming-composer[J]. Perspectives of New Music, 2015, 53(1): 219-237.

ALPERSON P. The instrumentality of music[J]. The Journal of Aesthetics and Art Criticism, 2008, 66(1): 37-51.

MACRITCHIE J, ZICARI M, BLOM D. Identifying challenges and opportunities for student composer and performer peer learning through newly-composed classical piano scores[J]. British Journal of Music Education, 2018, 35(2): 153-175.

FRISCH R. Mastering the orchestra audition: a musician's guide[M]. Minneapolis: Kairos Publications, 2017.

TIMM E. Intonation facts[J]. Music Educators Journal, 1943, 29(3): 19-20.

HAFNER K. Piano tuners have built a bridge to 18th century[N]. New York Times, 2000-02-17.

LIM K A, RAPHAEL C. InTune: a system to support an instrumentalist's visualization of intonation[J]. Computer Music Journal, 2010, 34(3): 45-55.

BOULEZ P. Technology and the composer[J]. Leonardo, 1978, 11: 59-62.

SMITH N T, PETERS P, MOLINA K. Classical music futures: practices of innovation[M]. Cambridge: Open Book Publishers, 2024.

HILLER L AND ISAACSON L. Musical composition with a high-speed digital computer[J]. Journal of the Audio Engineering Society, 1958, 6(3): 154-160.

YANCHENKO A K, MUKHERJEE S. Classical music composition using state space models[EB]. arXiv preprint, 2017, arXiv:1708.03822.

EIGENFELDT A. Corpus-based recombinant composition using a genetic algorithm[J]. Soft Computing, 2012, 16(12): 2049-2056.

BICKERMAN G, BOSLEY S, SWIRE P, et al. Learning to create jazz melodies using deep belief nets[C]//Proceedings of the International Conference on Computational Creativity .[S.l.:s.n.]. 2010: 228-237.

GRACHTEN M, CHACÓN C E C. Strategies for conceptual change in convolutional neural networks[EB]. arXiv preprint, 2017, arXiv: 1711.01634.

BRETAN M, WEINBERG G, HECK L. A unit selection methodology for music generation using deep neural networks[EB]. arXiv preprint, 2016, arXiv: 1612.03789.

HADJERES G, PACHET F, NIELSEN F. DeepBach: a steerable model for bach chorales generation[EB]. arXiv preprint, 2016, arXiv: 1612.01010.

Widmer G. The musical expression project: A challenge for machine learning and knowledge discovery[C]// Proceedings of 2001 European Conference on Machine Learning (ECML). Berlin: Springer, 2001: 603-614.

THÖRN O, FÖGEL P, KNUDSEN P, et al. Anticipation in collaborative music performance using fuzzy systems: a case study[EB]. arXiv preprint, 1906, arXiv: 1906.02155.

DHARIWAL P, JUN H, PAYNE C, et al. Jukebox: a generative model for music[EB]. arXiv preprint, 2020, arXiv: 2005.00341.

CAILLON A, ESLING P. RAVE: a variational autoencoder for fast and high-quality neural audio synthesis[EB]. arXiv preprint, 2021, arXiv: 2111.05011.

WU S L, DONAHUE C, WATANABE S, et al. Music ControlNet: multiple time-varying controls for music generation[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2024, 32: 2692-2703.

YUAN R B, LIN H F, WANG Y, et al. ChatMusician: understanding and generating music intrinsically with LLM[EB]. arXiv preprint, 2024, arXiv: 2402.16153.

AGOSTINELLI A, DENK T I, BORSOS Z, et al. MusicLM: generating music from text[EB]. arXiv preprint, 2023, arXiv: 2301.11325.

XU T Z, LI J J, CHEN X S, et al. Mozart's touch: a lightweight multi-modal music generation framework based on pre-trained large models[EB]. arXiv preprint, 2024, arXiv: 2405.02801.

WANG F Y, ZHANG J J, ZHENG X H, et al. Where does AlphaGo go: from church-turing thesis to AlphaGo thesis and beyond[J]. IEEE/CAA Journal of Automatica Sinica, 2016, 3(2): 113-120.

WANG F Y, MIAO Q H, LI X, et al. What does ChatGPT say: the DAO from algorithmic intelligence to linguistic intelligence[J]. IEEE/CAA Journal of Automatica Sinica, 2023, 10(3): 575-579.

WANG F Y, MIAO Q H, LI L X, et al. When does sora show: the beginning of TAO to imaginative intelligence and scenarios engineering[J]. IEEE/CAA Journal of Automatica Sinica, 2024, 11(4): 809-815.

LU Y, GUO C, DAI X Y, et al. Data-efficient image captioning of fine art paintings via virtual-real semantic alignment training[J]. Neurocomputing, 2022, 490: 163-180.

鲁越, 郭超, 林懿伦, 等.绘画艺术图像的计算美学：研究前沿与展望[J]. 自动化学报, 2020, 46(11): 2239-2259.

LU Y, GUO C, LIN Y L, et al. Computational aesthetics of fine art paintings: the state of the art and outlook[J]. Acta Automatica Sinica, 2020, 46(11): 2239-2259.

林飞, 王飞跃, 田永林, 等. 平行药物系统：基于大语言模型和三类人的框架与方法[J]. 智能科学与技术学报, 2024, 6(1): 88-99.

LIN F, WANG F Y, TIAN Y L, et al. Parallel drug systems: framework and methods based on large language models and three types of humans[J]. Chinese Journal of Intelligent Science and Technology, 2024, 6(1): 88-99.

WANG F Y, LIN Y L, IOANNOU P A, et al. Transportation 5.0: the DAO to safe, secure, and sustainable intelligent transportation systems[J]. IEEE Transactions on Intelligent Transportation Systems, 2023, 24(10): 10262-10278.

黄峻, 田永林, 戴星原, 等. 基于深度学习的自动驾驶多模态轨迹预测方法:现状及展望[J]. 智能科学与技术学报, 2023, 5(2): 180-199.

HUANG J, TIAN Y L, DAI X Y, et al. Deep learning-based multimodal trajectory prediction methods for autonomous driving: state of the art and perspectives[J]. Chinese Journal of Intelligent Science and Technology, 2023, 5(2): 180-199.

LU Y, GUO C, DAI X Y, et al. ArtCap: a dataset for image captioning of fine art paintings[J]. IEEE Transactions on Computational Social Systems, 2024, 11(1): 576-587.

LU Y, GUO C, DOU Y, et al. Could ChatGPT imagine: content control for artistic painting generation via large language models[J]. Journal of Intelligent & Robotic Systems, 2023, 109(2): 39.

LU Y, GUO C, DAI X Y, et al. Generating emotion descriptions for fine art paintings via multiple painting representations[J]. IEEE Intelligent Systems, 2023, 38(3): 31-40.

GE L, LV B, LI N, et al. A Hypertension parallel healthcare system based on the ACP approach[J]. IEEE Journal of Radio Frequency Identification, 2022, 6: 724-728.

YU Y, YAO S Y, WANG K X, et al. Pursuing equilibrium of medical resources via data empowerment in parallel healthcare system[C]// Proceedings of 2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Honolulu: IEEE Press, 2023: 3186-3191.

王拥军, 王飞跃, 王戈, 等 . 平行医院：从医院信息管理系统到智慧医院操作系统[J]. 自动化学报, 2021, 47(11): 2585-2599.

WANG Y J, WANG F Y, WANG G, et al. Parallel hospitals: from hospital information system (HIS)to hospital smart operating system (HSOS)[J]. Acta Automatica Sinica, 2021, 47(11): 2585-2599.

倪清桦, 郭超, 王飞跃. 平行戏剧：新时代戏剧的人机协同创作与智能管理[J]. 智能科学与技术学报, 2023, 5(4): 436-445.

NI Q H, GUO C, WANG F Y. Parallel theaters: human-machine collaborative creation and intelligent management for theatrical art[J]. Chinese Journal of Intelligent Science and Technology, 2023, 5(4): 436-445.

皮佩定, 倪清桦, 杨静, 等. 平行夏尔希里：生态资源智能管护及其可持续发展新途径[J]. 智能科学与技术学报, 2023, 5(3): 283-292.

PI P D, NI Q H, YANG J, et al. Parallel Sharhili: a new approach to sustainable development and intelligent management of ecological resources[J]. Chinese Journal of Intelligent Science and Technology, 2023, 5(3): 283-292.

郭超, 鲁越, 林懿伦, 等. 平行艺术：人机协作的艺术创作[J]. 智能科学与技术学报, 2019, 1(4): 335-341.

GUO C, LU Y, LIN Y L, et al. Parallel art: artistic creation under human-machine collaboration[J]. Chinese Journal of Intelligent Science and Technology, 2019, 1(4): 335-341.

杨林瑶, 陈思远, 王晓, 等. 数字孪生与平行系统：发展现状、对比及展望[J]. 自动化学报, 2019, 45(11): 2001-2031.

YANG L Y, CHEN S Y, WANG X, et al. Digital twins and parallel systems: state of the art, comparisons and prospect[J]. Acta Automatica Sinica, 2019, 45(11): 2001-2031.

李士果, 卢建云. Hadoop 3.0大数据平台性能[J]. 电子技术与软件工程, 2019(5): 158-160.

LI S G, LU J Y. Hadoop 3.0 big data platform performance[J]. Electronic Technology & Software Engineering, 2019(5): 158-160.

浏览量

300

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

平行旅游：基础智能驱动的智慧出游服务

平行交通：虚实互动的智能交通管理与控制

基于ACP和LLM犯罪学研究进展：综述与展望

平行药物系统：基于大语言模型和三类人的框架与方法

RAG-PHI：检索增强生成驱动的平行人与平行智能