Parallel music: human-machine hybrid music creation and performance in the era of large models

NI Qinghua; LU Yue; LIN Fei; HUANG Jun; WANG Yijin; LIN Weihua; WANG Fei-Yue

doi:10.11959/j.issn.2096-6652.202423

您当前的位置：

首页 >

文章列表页 >

Parallel music: human-machine hybrid music creation and performance in the era of large models

Special Topic: New Liberal Arts | 更新时间：2024-08-06

- Parallel music: human-machine hybrid music creation and performance in the era of large models
- Chinese Journal of Intelligent Science and Technology Vol. 6, Issue 2, Pages: 150-163(2024)
- 作者机构：
  
  1.澳门科技大学，澳门 999078
  2.山东大学控制科学与工程学院，山东济南 250061
  3.加州大学洛杉矶分校，美国洛杉矶 90095
  4.亚利桑那大学，美国图森 85721
  5.中国科学院自动化研究所复杂系统管理与控制国家重点实验室，北京 100190
- 作者简介：
- 基金信息：
  
  The Science and Technology Development Fund, Macao SAR(0093/2023/RIA2)
- DOI：10.11959/j.issn.2096-6652.202423
  CLC： TP39
- Received：18 April 2024，
  
  Revised：2024-05-27，
  
  Published：15 June 2024
- 稿件说明：
移动端阅览
倪清桦,鲁越,林飞等.平行音乐：大模型时代的人机混合音乐创演[J].智能科学与技术学报,2024,06(02):150-163.

NI Qinghua,LU Yue,LIN Fei,et al.Parallel music: human-machine hybrid music creation and performance in the era of large models[J].Chinese Journal of Intelligent Science and Technology,2024,06(02):150-163.
倪清桦,鲁越,林飞等.平行音乐：大模型时代的人机混合音乐创演[J].智能科学与技术学报,2024,06(02):150-163. DOI： 10.11959/j.issn.2096-6652.202423.

NI Qinghua,LU Yue,LIN Fei,et al.Parallel music: human-machine hybrid music creation and performance in the era of large models[J].Chinese Journal of Intelligent Science and Technology,2024,06(02):150-163. DOI： 10.11959/j.issn.2096-6652.202423.

摘要

随着声音艺术等垂直领域基础模型的迅速发展，人工智能与音乐创作表演呈现愈发融合的趋势。面向音乐创演的流程与需求，提出新型音乐创演框架——平行音乐系统，该系统基于平行系统理论与人工社会、计算实验和平行执行方法，通过模拟真实世界的音乐创演环境，构建高度逼真的虚拟音乐创演场景，在虚拟和真实音乐系统间实现实时无缝交互，设计大模型时代音乐创演的人工系统、计算实验、平行执行各环节技术路线，形成由自然人、数字人、机器人组成的人机混合音乐创演团队。阐述了平行音乐的理论框架和关键技术，分析其在现代音乐领域中的创新意义和潜在价值，并探讨了平行音乐在乐曲生成和音乐治疗等场景中的应用案例。旨在利用大模型时代的人工智能技术赋能传统音乐创作路径，推动音乐创演的技术革新进程，释放音乐家的艺术创新潜力，以期为新时代音乐创演提供新视角与启发。

Abstract

As foundational models in specialized fields like sound art have rapidly advanced

the convergence of artificial intelligence with music creation and performance has become increasingly pronounced. A novel framework for music creation and performance called the parallel music system was proposed. Grounded in parallel systems theory and the artificial system

computational experiments

parallel execution (ACP) approach

this system stimulates the real-world music creation and performance environments to build highly authentic virtual scenarios. It facilitates real-time

seamless interactions between the virtual and real music systems and outlines the technological strategies for music creation and performance in the era of large models

incorporating a hybrid team of humans

digital beings

and robots. Additionally

the theoretical framework of parallel music and its critical technologies are discussed

the innovative contributions and potential in the modern music field are evaluated

the applications in scenarios like music generation and music therapy are examined. The objective is to harness the emerging AI technologies of the large model era to enhance traditional paths in music creation

drive technological advancements in music performance

unlock the creative potential of musicians

and offer fresh perspectives and inspirations for music creation and performance in the new era.

关键词

Keywords

references

CROSS I. Music, mind and evolution[J]. Psychology of Music, 2001, 29: 95-102.

JUSLIN P N, VÄSTFJÄLL D. Emotional responses to music: the need to consider underlying mechanisms[J]. The Behavioral and Brain Sciences, 2008, 31(5): 559-575.

MIRANDA E R, WANDERLEY M. New digital musical instruments: control and interaction beyond the keyboard (computer music and digital audio series)[M]. Wisconsin: A-R Editions, Inc., 2006.

王飞跃. 平行艺术：从艺术的智能化到智能的艺术化[R]. 2017.

WANG F Y. Parallel art: from intelligent art to artistic intelligence[R]. 2017.

王飞跃. 人工社会、计算实验、平行系统：关于复杂社会经济系统计算研究的讨论[J]. 复杂系统与复杂性科学, 2004, 1(4): 25-35.

WANG F Y. Artificial societies, computational experiments, and parallel systems: a discussion on computational theory of complex social- economic systems[J]. Complex Systems and Complexity Science, 2004, 1(4): 25-35.

KOEPKE P. The composing process[J]. American Music Teacher, 1958, 7(5): 6-20.

Erickson R. Composing music[J]. Perspectives of New Music, 1998, 26(2): 86-95.

HULSE B. Becoming-composer[J]. Perspectives of New Music, 2015, 53(1): 219-237.

ALPERSON P. The instrumentality of music[J]. The Journal of Aesthetics and Art Criticism, 2008, 66(1): 37-51.

MACRITCHIE J, ZICARI M, BLOM D. Identifying challenges and opportunities for student composer and performer peer learning through newly-composed classical piano scores[J]. British Journal of Music Education, 2018, 35(2): 153-175.

FRISCH R. Mastering the orchestra audition: a musician's guide[M]. Minneapolis: Kairos Publications, 2017.

TIMM E. Intonation facts[J]. Music Educators Journal, 1943, 29(3): 19-20.

HAFNER K. Piano tuners have built a bridge to 18th century[N]. New York Times, 2000-02-17.

LIM K A, RAPHAEL C. InTune: a system to support an instrumentalist's visualization of intonation[J]. Computer Music Journal, 2010, 34(3): 45-55.

BOULEZ P. Technology and the composer[J]. Leonardo, 1978, 11: 59-62.

SMITH N T, PETERS P, MOLINA K. Classical music futures: practices of innovation[M]. Cambridge: Open Book Publishers, 2024.

HILLER L AND ISAACSON L. Musical composition with a high-speed digital computer[J]. Journal of the Audio Engineering Society, 1958, 6(3): 154-160.

YANCHENKO A K, MUKHERJEE S. Classical music composition using state space models[EB]. arXiv preprint, 2017, arXiv:1708.03822.

EIGENFELDT A. Corpus-based recombinant composition using a genetic algorithm[J]. Soft Computing, 2012, 16(12): 2049-2056.

BICKERMAN G, BOSLEY S, SWIRE P, et al. Learning to create jazz melodies using deep belief nets[C]//Proceedings of the International Conference on Computational Creativity .[S.l.:s.n.]. 2010: 228-237.

GRACHTEN M, CHACÓN C E C. Strategies for conceptual change in convolutional neural networks[EB]. arXiv preprint, 2017, arXiv: 1711.01634.

BRETAN M, WEINBERG G, HECK L. A unit selection methodology for music generation using deep neural networks[EB]. arXiv preprint, 2016, arXiv: 1612.03789.

HADJERES G, PACHET F, NIELSEN F. DeepBach: a steerable model for bach chorales generation[EB]. arXiv preprint, 2016, arXiv: 1612.01010.

Widmer G. The musical expression project: A challenge for machine learning and knowledge discovery[C]// Proceedings of 2001 European Conference on Machine Learning (ECML). Berlin: Springer, 2001: 603-614.

THÖRN O, FÖGEL P, KNUDSEN P, et al. Anticipation in collaborative music performance using fuzzy systems: a case study[EB]. arXiv preprint, 1906, arXiv: 1906.02155.

DHARIWAL P, JUN H, PAYNE C, et al. Jukebox: a generative model for music[EB]. arXiv preprint, 2020, arXiv: 2005.00341.

CAILLON A, ESLING P. RAVE: a variational autoencoder for fast and high-quality neural audio synthesis[EB]. arXiv preprint, 2021, arXiv: 2111.05011.

WU S L, DONAHUE C, WATANABE S, et al. Music ControlNet: multiple time-varying controls for music generation[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2024, 32: 2692-2703.

YUAN R B, LIN H F, WANG Y, et al. ChatMusician: understanding and generating music intrinsically with LLM[EB]. arXiv preprint, 2024, arXiv: 2402.16153.

AGOSTINELLI A, DENK T I, BORSOS Z, et al. MusicLM: generating music from text[EB]. arXiv preprint, 2023, arXiv: 2301.11325.

XU T Z, LI J J, CHEN X S, et al. Mozart's touch: a lightweight multi-modal music generation framework based on pre-trained large models[EB]. arXiv preprint, 2024, arXiv: 2405.02801.

WANG F Y, ZHANG J J, ZHENG X H, et al. Where does AlphaGo go: from church-turing thesis to AlphaGo thesis and beyond[J]. IEEE/CAA Journal of Automatica Sinica, 2016, 3(2): 113-120.

WANG F Y, MIAO Q H, LI X, et al. What does ChatGPT say: the DAO from algorithmic intelligence to linguistic intelligence[J]. IEEE/CAA Journal of Automatica Sinica, 2023, 10(3): 575-579.

WANG F Y, MIAO Q H, LI L X, et al. When does sora show: the beginning of TAO to imaginative intelligence and scenarios engineering[J]. IEEE/CAA Journal of Automatica Sinica, 2024, 11(4): 809-815.

LU Y, GUO C, DAI X Y, et al. Data-efficient image captioning of fine art paintings via virtual-real semantic alignment training[J]. Neurocomputing, 2022, 490: 163-180.

鲁越, 郭超, 林懿伦, 等.绘画艺术图像的计算美学：研究前沿与展望[J]. 自动化学报, 2020, 46(11): 2239-2259.

LU Y, GUO C, LIN Y L, et al. Computational aesthetics of fine art paintings: the state of the art and outlook[J]. Acta Automatica Sinica, 2020, 46(11): 2239-2259.

林飞, 王飞跃, 田永林, 等. 平行药物系统：基于大语言模型和三类人的框架与方法[J]. 智能科学与技术学报, 2024, 6(1): 88-99.

LIN F, WANG F Y, TIAN Y L, et al. Parallel drug systems: framework and methods based on large language models and three types of humans[J]. Chinese Journal of Intelligent Science and Technology, 2024, 6(1): 88-99.

WANG F Y, LIN Y L, IOANNOU P A, et al. Transportation 5.0: the DAO to safe, secure, and sustainable intelligent transportation systems[J]. IEEE Transactions on Intelligent Transportation Systems, 2023, 24(10): 10262-10278.

黄峻, 田永林, 戴星原, 等. 基于深度学习的自动驾驶多模态轨迹预测方法:现状及展望[J]. 智能科学与技术学报, 2023, 5(2): 180-199.

HUANG J, TIAN Y L, DAI X Y, et al. Deep learning-based multimodal trajectory prediction methods for autonomous driving: state of the art and perspectives[J]. Chinese Journal of Intelligent Science and Technology, 2023, 5(2): 180-199.

LU Y, GUO C, DAI X Y, et al. ArtCap: a dataset for image captioning of fine art paintings[J]. IEEE Transactions on Computational Social Systems, 2024, 11(1): 576-587.

LU Y, GUO C, DOU Y, et al. Could ChatGPT imagine: content control for artistic painting generation via large language models[J]. Journal of Intelligent & Robotic Systems, 2023, 109(2): 39.

LU Y, GUO C, DAI X Y, et al. Generating emotion descriptions for fine art paintings via multiple painting representations[J]. IEEE Intelligent Systems, 2023, 38(3): 31-40.

GE L, LV B, LI N, et al. A Hypertension parallel healthcare system based on the ACP approach[J]. IEEE Journal of Radio Frequency Identification, 2022, 6: 724-728.

YU Y, YAO S Y, WANG K X, et al. Pursuing equilibrium of medical resources via data empowerment in parallel healthcare system[C]// Proceedings of 2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Honolulu: IEEE Press, 2023: 3186-3191.

王拥军, 王飞跃, 王戈, 等 . 平行医院：从医院信息管理系统到智慧医院操作系统[J]. 自动化学报, 2021, 47(11): 2585-2599.

WANG Y J, WANG F Y, WANG G, et al. Parallel hospitals: from hospital information system (HIS)to hospital smart operating system (HSOS)[J]. Acta Automatica Sinica, 2021, 47(11): 2585-2599.

倪清桦, 郭超, 王飞跃. 平行戏剧：新时代戏剧的人机协同创作与智能管理[J]. 智能科学与技术学报, 2023, 5(4): 436-445.

NI Q H, GUO C, WANG F Y. Parallel theaters: human-machine collaborative creation and intelligent management for theatrical art[J]. Chinese Journal of Intelligent Science and Technology, 2023, 5(4): 436-445.

皮佩定, 倪清桦, 杨静, 等. 平行夏尔希里：生态资源智能管护及其可持续发展新途径[J]. 智能科学与技术学报, 2023, 5(3): 283-292.

PI P D, NI Q H, YANG J, et al. Parallel Sharhili: a new approach to sustainable development and intelligent management of ecological resources[J]. Chinese Journal of Intelligent Science and Technology, 2023, 5(3): 283-292.

郭超, 鲁越, 林懿伦, 等. 平行艺术：人机协作的艺术创作[J]. 智能科学与技术学报, 2019, 1(4): 335-341.

GUO C, LU Y, LIN Y L, et al. Parallel art: artistic creation under human-machine collaboration[J]. Chinese Journal of Intelligent Science and Technology, 2019, 1(4): 335-341.

杨林瑶, 陈思远, 王晓, 等. 数字孪生与平行系统：发展现状、对比及展望[J]. 自动化学报, 2019, 45(11): 2001-2031.

YANG L Y, CHEN S Y, WANG X, et al. Digital twins and parallel systems: state of the art, comparisons and prospect[J]. Acta Automatica Sinica, 2019, 45(11): 2001-2031.

李士果, 卢建云. Hadoop 3.0大数据平台性能[J]. 电子技术与软件工程, 2019(5): 158-160.

LI S G, LU J Y. Hadoop 3.0 big data platform performance[J]. Electronic Technology & Software Engineering, 2019(5): 158-160.

Views

300

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Parallel tourism:foundation intelligence driven smart trip services

Parallel transportation:virtual-real interaction for intelligent traffic management and control

The ACP and LLM-based framework for criminology: a review and prospect

Parallel drug systems: framework and methods based on large language models and three types of humans

RAG-PHI: RAG-driven parallel human and parallel intelligence

Related Author

Wang Fei-Yue

Timothy J. Lee

WU Naiqi

LI Juanjuan

DAI Xingyuan

SONG Ping

Ni Qinghua

Lin Fei

Related Institution

Faculty of Innovation Engineering, Macau University of Science and Technology, Macao

State Key Laboratory for Management and Control of Complex Systems, Institute of Automation, Chinese Academy of Sciences

Dazhou Artificial Intelligence Institute

State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation,Chinese Academy of Sciences

School of Hospitality and Tourism Management, Macau University of Science and Technology, Macao

Postal code：100079
Tel：（010）53879206 Email：cjist@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：161982 Visits Today：145

⁰