Prospects and pathways for research on multi-level brain language decoding

Cheng Kaiwen; Du Yuanyuan; Zhao Yuhao

doi:10.11959/j.issn.2096-6652.ZNKX26030

您当前的位置：

首页 >

文章列表页 >

Prospects and pathways for research on multi-level brain language decoding

Surveys and Prospectives | 更新时间：2026-07-29

- Prospects and pathways for research on multi-level brain language decoding
- Chinese Journal of Intelligent Science and Technology Vol. 8, Issue 2, Pages: 183-195(2026)
- 作者机构：
  
  1.四川外国语大学语言智能学院，重庆 400031
  2.广东外语外贸大学英语语言文化学院，广东广州 510420
- 作者简介：
- 基金信息：
  
  Chongqing Social Science Planning Project(2023NDYB167);the Annual Project of the American Studies Center in Southwest Jiaotong University(ARC2023006);General Project of the Humanities and Social Sciences Research Fund of the Chinese Ministry of Education(25YJA710007)
- DOI：10.11959/j.issn.2096-6652.ZNKX26030
  CLC： TP391.9
- Received：10 February 2026，
  
  Revised：2026-04-27，
  
  Accepted：18 May 2026，
  
  Online First：27 July 2026，
  
  Published：15 June 2026
- 稿件说明：
移动端阅览
程凯文,杜圆圆,赵宇浩.多层级语言脑解码技术的研究前瞻与发展路径[J].智能科学与技术学报,2026,08(02):183-195.

Cheng Kaiwen,Du Yuanyuan,Zhao Yuhao.Prospects and pathways for research on multi-level brain language decoding[J].Chinese Journal of Intelligent Science and Technology,2026,08(02):183-195.
程凯文,杜圆圆,赵宇浩.多层级语言脑解码技术的研究前瞻与发展路径[J].智能科学与技术学报,2026,08(02):183-195. DOI： 10.11959/j.issn.2096-6652.ZNKX26030.

Cheng Kaiwen,Du Yuanyuan,Zhao Yuhao.Prospects and pathways for research on multi-level brain language decoding[J].Chinese Journal of Intelligent Science and Technology,2026,08(02):183-195. DOI： 10.11959/j.issn.2096-6652.ZNKX26030.

摘要

语言脑解码技术为失语患者重建交流能力提供了关键路径，也是脑机接口技术发展的重要驱动力之一。近年来，基于侵入性与非侵入性脑信号的多层级语言脑解码技术取得了显著进展，但不同语言层级在脑信号表征、任务复杂度与模型方法上存在明显差异，导致相关研究在技术路径和问题侧重上比较分散，尚缺乏系统的层级化梳理。鉴于此，系统梳理了2008—2025年语言脑解码研究在音素、声调、单词、短语、句子及语篇各层级的主要进展，分析了脑信号类型、任务设定与模型架构对解码精度的影响，并就多层级解码、开放词汇解码及预训练语言模型的融合应用等问题进行了重点讨论。在此基础上，提出了未来语言脑机接口研发与智能化语言康复的若干研究路径，以期为相关交叉研究提供理论支撑与技术参考。

Abstract

Brain language decoding provides a critical pathway for restoring communication in patients with aphasia and serves as an important driving force for the development of brain-computer interface (BCI) technology. In recent years

substantial progress has been made in multi-level brain language decoding based on both invasive and non-invasive brain signals. However

different linguistic levels vary markedly in brain signal representation

task complexity

and model selection

which has led to fragmented technical pathways and research emphases

while a systematic level-based synthesis is still lacking. Based on this

the major advances in brain language decoding research from 2008 to 2025 were reviewed across different linguistic levels

including phonemes

words

phrases

sentences

and discourse. The influence of brain signal modality

task setting

and model architecture on decoding accuracy was analyzed

and further key issues such as multi-level decoding

open-vocabulary decoding

and the integration of pre-trained language models were discussed. On this basis

several future research pathways were proposed for the development of language brain-computer interfaces and intelligent language rehabilitation

with the aim of providing theoretical support and technical reference for related interdisciplinary studies.

关键词

Keywords

references

Mitchell T M, Shinkareva S V, Carlson A, et al. Predicting human brain activity associated with the meanings of nouns[J]. Science, 2008, 320(5880): 1191-1195.

Tang J, LeBel A, Jain S, et al. Semantic reconstruction of continuous language from non-invasive brain recordings[J]. Nature Neuroscience, 2023, 26(5): 858-866.

Correia J, Formisano E, Valente G, et al. Brain-based translation: fMRI decoding of spoken words in bilinguals reveals language-independent semantic representations in anterior temporal lobe[J]. The Journal of Neuroscience, 2014, 34(1): 332-338.

Xu M, Li D, Li P. Brain decoding in multiple languages: Can cross-language brain decoding work?[J]. Brain and Language, 2021, 215: 104922.

Hu Y, Mohsenzadeh Y. Neural processing of naturalistic audiovisual events in space and time[J]. Communications Biology, 2025, 8: 110.

Goodfellow I, Bengio Y, Courville A. Deep learning[M]. Cambridge: MIT Press, 2016.

LeCun Y, Bengio Y, Hinton G. Deep learning[J]. Nature, 2015, 521(7553): 436-444.

Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]//Advances in Neural Information Processing Systems. Red Hook: Curran Associates, 2017: 5998-6008.

Anumanchipalli G K, Chartier J, Chang E F. Speech synthesis from neural decoding of spoken sentences[J]. Nature, 2019, 568(7753): 493-498.

Ramsey N F, Salari E, Aarnoutse E J, et al. Decoding spoken phonemes from sensorimotor cortex with high-density ECoG grids[J]. NeuroImage, 2018, 180: 301-311.

Pereira F, Lou B, Pritchett B, et al. Toward a universal decoder of linguistic meaning from brain activation[J]. Nature Communications, 2018, 9: 963.

Dash D, Ferrari P, Wang J. Decoding imagined and spoken phrases from non-invasive neural (MEG) signals[J]. Frontiers in Neuroscience, 2020, 14: 290.

Huang W, Yan H M, Cheng K W, et al. A dual-channel language decoding from brain activity with progressive transfer training[J]. Human Brain Mapping, 2021, 42(15): 5089-5100.

Holm E, Marraffini G F G, Slezak D F, et al. Hierarchical temporal correspondence between brain activity and different deep neural network architectures[C]//Advances in Neural Information Processing Systems. [S.l.:s.n.], 2025: 1-8.

曾小芹. 基于Python的中文结巴分词技术实现[J]. 信息与电脑(理论版), 2019, 31(18): 38-39, 42.

Zeng X Q. Technology implementation of Chinese jieba segmentation based on Python[J]. China Computer & Communication, 2019, 31(18): 38-39, 42.

Che W X, Feng Y L, Qin L B, et al. N-LTP: an open-source neural language technology platform for Chinese[C]//Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. Stroudsburg: ACL, 2021: 42-49.

Angrick M, Luo S Y, Rabbani Q, et al. Online speech synthesis using a chronically implanted brain-computer interface in an individual with ALS[J]. Scientific Reports, 2024, 14: 9617.

Vilela M, Hochberg L R. Applications of brain-computer interfaces to the control of robotic and prosthetic arms[M]//Brain-Computer Interfaces. Amsterdam: Elsevier, 2020: 87-99.

Ciabattoni L, Ferracuti F, Freddi A, et al. Human-in-the-loop approach to safe navigation of a smart wheelchair via brain computer interface[M]//Ambient Assisted Living. Cham: Springer, 2021: 197-209.

Willett F R, Avansino D T, Hochberg L R, et al. High-performance brain-to-text communication via handwriting[J]. Nature, 2021, 593(7858): 249-254.

Qian Y K, Liu C J, Yu P X, et al. Real-time decoding of full-spectrum Chinese using brain-computer interface[J]. Science Advances, 2025, 11(45): eadz9968.

Redwan S M, Uddin M P, Sharif M I, et al. A network theory investigation into the altered resting state functional connectivity in attentiondeficit hyperactivity disorder[PP]. V1. arXiv (2022-11-23) [2026-03-10]. arXiv:2212.02402.

Li D W, Luo X S, Guo J L, et al. Information-based multivariate decoding reveals imprecise neural encoding in children with attention deficit hyperactivity disorder during visual selective attention[J]. Human Brain Mapping, 2023, 44(3): 937-947.

Xu C, Zou J J, He F P, et al. Neural tracking of sound rhythms correlates with diagnosis, severity, and prognosis of disorders of consciousness[J]. Frontiers in Neuroscience, 2021, 15: 646543.

Rahman N, Khan D M, Masroor K, et al. Advances in brain-computer interface for decoding speech imagery from EEG signals: a systematic review[J]. Cognitive Neurodynamics, 2024, 18(6): 3565-3583.

陈霏, 潘昌杰. 基于发音想象的脑机接口的研究综述[J]. 信号处理, 2020, 36(6): 816-830.

Chen F, Pan C J. A review on brain-computer interface technologies for speech imagery[J]. Journal of Signal Processing, 2020, 36(6): 816-830.

杨淑淇, 谭颖. 深度学习在解码大脑语音信息中的应用[J]. 西南民族大学学报(自然科学版), 2025, 51(3): 315-322.

Yang S Q, Tan Y. Deep learning in decoding brain speech information[J]. Journal of Southwest University for Nationalities (Natrual Science Edition), 2025, 51(3): 315-322.

Liu Y, Zhao Z H, Xu M P, et al. Decoding and synthesizing tonal language speech from brain activity[J]. Science Advances, 2023, 9(23): eadh0478.

Lu J F, Li Y N, Zhao Z H, et al. Neural control of lexical tone production in human laryngeal motor cortex[J]. Nature Communications, 2023, 14: 6917.

Zhang X Y, Li H, Chen F. EEG-based classification of imaginary mandarin tones[C]//Proceedings of the 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). Piscataway: IEEE Press, 2020: 3889-3892.

Chen X P, Wang R, Khalilian-Gourtani A, et al. A neural speech decoding framework leveraging deep learning and speech synthesis[J]. Nature Machine Intelligence, 2024, 6(4): 467-480.

Littlejohn K T, Cho C J, Liu J R, et al. A streaming brain-to-voice neuroprosthesis to restore naturalistic communication[J]. Nature Neuroscience, 2025, 28(4): 902-912.

Fukuma R, Yanagisawa T, Nishimoto S, et al. Decoding visual stimulus in semantic space from electrocorticography signals[C]//Proceedings of the 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC). Piscataway: IEEE Press, 2018: 102-104.

Nishida S, Nishimoto S. Decoding naturalistic experiences from human brain activity via distributed representations of words[J]. NeuroImage, 2018, 180: 232-242.

Lin Y, Hsieh P J. Neural decoding of speech with semantic-based classification[J]. Cortex, 2022, 154: 231-240.

Zhang M F, Lévy J, D’Ascoli S, et al. From thought to action: how a hierarchy of neural dynamics supports language production[PP]. V2. arXiv (2025-02-18)[2026-03-10]. arXiv:2502.07429.

邹宇童, 李波, 徐晓翠. 面向分布式语义空间的fMRI解码研究[J]. 计算机技术与发展, 2026: 2026.0076.

Zou Y T, Li B, Xu X C. fMRI decoding research for distributed semantic spaces[J]. Computer Technology and Development, 2026: 2026.0076.

Buchweitz A, Shinkareva S V, Mason R A, et al. Identifying bilingual semantic neural representations across languages[J]. Brain and Language, 2012, 120(3): 282-289.

Correia J M, Jansma B, Hausfeld L, et al. EEG decoding of spoken words in bilingual listeners: from words to language invariant semantic-conceptual representations[J]. Frontiers in Psychology, 2015, 6: 71.

Du C D, Fu K C, Li J P, et al. Decoding visual neural representations by multimodal learning of brain-visual-linguistic features[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(9): 10760-10777.

任子琪. 基于深度生成模型的视觉神经信息解码与重构[D]. 西安: 西安电子科技大学, 2024.

Ren Z Q. Deep generative model-based neural decoding and visual reconstruction[D]. Xi'an: Xidian University, 2024.

Bruera A, Poesio M. Electroencephalography searchlight decoding reveals person- and place-specific responses for semantic category and familiarity[J]. Journal of Cognitive Neuroscience, 2025, 37(1): 135-154.

Wang Z, Ji H. Open vocabulary electroencephalography-to-text decoding and zero-shot sentiment classification[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2022, 36(5): 5350-5358.

刘晓龙, 杨帮华, 甘安安, 等. 基于CAM-Net模型的中文词语言语想象脑电解码研究[J]. 生物医学工程学杂志, 2025, 42(3): 473-479.

Liu X L, Yang B H, Gan A A, et al. Study on speech imagery electroencephalography decoding of Chinese words based on the CAM-Net model[J]. Journal of Biomedical Engineering, 2025, 42(3): 473-479.

Jia Z H, Wang H B, Shen Y Z, et al. Magnetoencephalography (MEG) based non-invasive Chinese speech decoding[J]. Journal of Neural Engineering, 2025, 22(6): 066014.

Zhao J Y, Gao R M, Brennan J R. Decoding the neural dynamics of headed syntactic structure building[J]. The Journal of Neuroscience, 2025, 45(17): e2126242025.

Matsuo E, Kobayashi I, Nishimoto S, et al. Generating natural language descriptions for semantic representations of human brain activity[C]//Proceedings of the ACL 2016 Student Research Workshop. Stroudsburg: ACL, 2016: 22-29.

Yang Y, Wang J, Bailer C, et al. Commonalities and differences in the neural representations of English, Portuguese, and Mandarin sentences: When knowledge of the brain-language mappings for two languages is better than one[J]. Brain and Language, 2017, 175: 77-85.

Li C F, Liu Y, Li J L, et al. Decoding bilingual EEG signals with complex semantics using adaptive graph attention convolutional network[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2024, 32: 249-258.

Huang W, Yang P F, Tang Y, et al. From sight to insight: a multi-task approach with the visual language decoding model[J]. Information Fusion, 2024, 112: 102573.

Zhang D H, Wang Z J, Qian Y K, et al. A brain-to-text framework for decoding natural tonal sentences[J]. Cell Reports, 2024, 43(11): 114924.

马磊, 崔文浩, 杨汶汶, 等. 非侵入性连续中文语言语义解码与重建[J]. 数据采集与处理, 2025, 40(3): 616-636.

Ma L, Cui W H, Yang W W, et al. Non-invasive continuous Chinese language semantic decoding and reconstruction[J]. Journal of Data Acquisition and Processing, 2025, 40(3): 616-636.

Wang B, Xu X R, Zhang L X, et al. Semantic reconstruction of continuouslanguage from MEG signals[PP]. V1. arXiv (2023-09-14)[2026-03-10]. arXiv:2309.07701.

Lewis M, Liu Y H, Goyal N, et al. BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2020: 7871-7880.

Wang J Q, Song Z X, Ma Z Y, et al. Enhancing EEG-to-text decoding through transferable representations from pre-trained contrastive EEG-text masked autoencoder[C]//Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg: ACL, 2024: 7278-7292.

Défossez A, Caucheteux C, Rapin J, et al. Decoding speech perception from non-invasive brain recordings[J]. Nature Machine Intelligence, 2023, 5(10): 1097-1107.

Lévy J, Zhang M F, Pinet S, et al. Brain-to-text decoding: a noninvasiveapproach via typing[PP]. V1. arXiv (2025-02-18) [2026-03-10]. arXiv:2502.17480.

Dehghani M, Boghrati R, Man K, et al. Decoding the neural representation of story meanings across languages[J]. Human Brain Mapping, 2017, 38(12): 6096-6106.

Hakala T, Lindh-Knuutila T, Hultén A, et al. Subword representations successfully decode brain responses to morphologically complex written words[J]. Neurobiology of Language, 2024, 5(4): 844-863.

Brysbaert M, Mandera P, Keuleers E. The word frequency effect in word processing: an updated review[J]. Current Directions in Psychological Science, 2018, 27(1): 45-50.

Wang S N, Zhang J J, Wang H Y, et al. Fine-grained neural decoding with distributed word representations[J]. Information Sciences, 2020, 507: 256-272.

Bi Y C. Dual coding of knowledge in the human brain[J]. Trends in Cognitive Sciences, 2021, 25(10): 883-895.

Huang W, Tang Y, Wang S Z, et al. Unraveling the differential efficiency of dorsal and ventral pathways in visual semantic decoding[J]. International Journal of Neural Systems, 2025, 35(3): 2550009.

Połczyńska M M, Bookheimer S Y. General principles governing the amount of neuroanatomical overlap between languages in bilinguals[J]. Neuroscience & Biobehavioral Reviews, 2021, 130: 1-14.

Liu X Y, Qu J, Li H L, et al. Similar activation patterns in the bilateral dorsal inferior frontal gyrus for monolingual and bilingual contexts in second language production[J]. Neuropsychologia, 2021, 156: 107857.

Zinszer B D, Anderson A J, Kang O, et al. Semantic structural alignment of neural representational spaces enables translation between English and Chinese words[J]. Journal of Cognitive Neuroscience, 2016, 28(11): 1749-1759.

Kim S Y, Qi T, Feng X X, et al. How does language distance between L1 and L2 affect the L2 brain network? An fMRI study of Korean-Chinese-English trilinguals[J]. NeuroImage, 2016, 129: 25-39.

Liberto G M D, Nie J P, Yeaton J, et al. Neural representation of linguistic feature hierarchy reflects second-language proficiency[J]. NeuroImage, 2021, 227: 117586.

Zhao Y H, Huang W, Cao S Q, et al. Revisiting the conceptual representation of Chinese-English Bilinguals: Evidence from AI-based brain language decoding[J]. Journal of Neurolinguistics, 2026, 79: 101341.

Sheikh U A, Carreiras M, Soto D. Decoding the meaning of unconsciously processed words using fMRI-based MVPA[J]. NeuroImage, 2019, 191: 430-440.

Sheikh U A, Carreiras M, Soto D. Neurocognitive mechanisms supporting the generalization of concepts across languages[J]. Neuropsychologia, 2021, 153: 107740.

Guo Z Z, Xu L S, Tan W J, et al. Impact of generation rate of speech imagery on neural activity and BCI decoding performance: a fNIRS study[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2025, 33: 1180-1190.

Huang W, Yan H M, Cheng K W, et al. A neural decoding algorithm that generates language from visual activity evoked by natural images[J]. Neural Networks, 2021, 144: 90-100.

Zhao Y H, Chen Y, Cheng K W, et al. Artificial intelligence based multimodal language decoding from brain activity: a review[J]. Brain Research Bulletin, 2023, 201: 110713.

Keller T A. The neural bases of sentence comprehension: a fMRI examination of syntactic and lexical processing[J]. Cerebral Cortex, 2001, 11(3): 223-237.

Wandelt S K, Bjånes D A, Pejsa K, et al. Representation of internal speech by single neurons in human supramarginal gyrus[J]. Nature Human Behaviour, 2024, 8(6): 1136-1149.

Devlin J, Chang M W, Lee K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1. Kerrville: Association for Computational Linguistics, 2019: 4171-4186.

Baltrušaitis T, Ahuja C, Morency L P. Multimodal machine learning: a survey and taxonomy[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(2): 423-443.

Wang P P, Zhou Y Y, Li Z N, et al. Neural decoding of Chinese sign language with machine learning for brain-computer interfaces[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2021, 29: 2721-2732.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

The knowledge origin of artificial intelligence

On intelligence: the origin and the goal

LinYun: a domain-specific large language model for intelligent question answering and decision support in forestry

“Breathing sovereignty” in new AI era: global technological competition with no genesis traps

Review of deep learning-based foreign object intrusion detection for transmission lines via UAV aerial photography

Related Author

CHENG Kaiwen

DU Yuanyuan

ZHAO Yuhao

Wang Fei-Yue

Zhao Weikang

Wang Haoyu

Hua Jing

Related Institution

School of Language Intelligence, Sichuan International Studies University, Chongqing 400031, China ²School of English Language and Culture, Guangdong University of Foreign Studies

广东外语外贸大学英语语言文化学院

The State Key Laboratory for Management and Control of Complex Systems, Institute of Automation, Chinese Academy of Sciences

Faculty of Innovation Engineering, Macau University of Science and Technology, Macao

Department of Engineering Science, Faculty of Innovation Engineering, Macau University of Science and Technology, Macao

Postal code：100079
Tel：（010）53879206 Email：cjist@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：185511 Visits Today：506

⁰