求解微分方程的人工智能与深度学习方法：现状及展望

卢经纬; 程相; 王飞跃

doi:10.11959/j.issn.2096-6652.202255

您当前的位置：

首页 >

文章列表页 >

求解微分方程的人工智能与深度学习方法：现状及展望

评论智能 | 更新时间：2024-06-05

- 求解微分方程的人工智能与深度学习方法：现状及展望
- Artificial intelligence and deep learning methods for solving differential equations: the state of the art and prospects
- 智能科学与技术学报 2022年4卷第4期页码：461-476
- 作者机构：
  
  1. 中国科学院自动化研究所复杂系统管理与控制国家重点实验室，北京 100190
  2. 青岛智能产业技术研究院，山东青岛 266114
  3. 中国科学院大学人工智能学院，北京 100049
- 作者简介：
  
  [ "卢经纬（1990- ），男，青岛智能产业技术研究院助理研究员，主要研究方向为最优控制、平行控制、自适应动态规划和深度强化学习" ]
  [ "程相（1994- ），男，中国科学院自动化研究所博士生，主要研究方向为智慧油田、深度学习和平行控制" ]
  [ "王飞跃（1961- ），男，中国科学院自动化研究所复杂系统管理与控制国家重点实验室主任，主要研究方向为平行系统的方法与应用、社会计算、平行智能以及知识自动化" ]
- 基金信息：
  
  国家自然科学基金项目;The National Natural Science Foundation of China(U1811463);行动元联合研究项目伺服驱动系统建模、决策和控制算法研究;Motion G, Inc.Collaborative Research Project for Modeling, Decision and Control Algorithms of Servo Drive Systems
- DOI：10.11959/j.issn.2096-6652.202255
  中图分类号： TP18
- 网络出版日期：2022-12，
  
  纸质出版日期：2022-12-15
- 稿件说明：
移动端阅览
卢经纬, 程相, 王飞跃. 求解微分方程的人工智能与深度学习方法：现状及展望[J]. 智能科学与技术学报, 2022,4(4):461-476.

Jingwei LU, Xiang CHENG, Fei-Yue WANG. Artificial intelligence and deep learning methods for solving differential equations: the state of the art and prospects[J]. Chinese journal of intelligent science and technology, 2022, 4(4): 461-476.
卢经纬, 程相, 王飞跃. 求解微分方程的人工智能与深度学习方法：现状及展望[J]. 智能科学与技术学报, 2022,4(4):461-476. DOI： 10.11959/j.issn.2096-6652.202255.

Jingwei LU, Xiang CHENG, Fei-Yue WANG. Artificial intelligence and deep learning methods for solving differential equations: the state of the art and prospects[J]. Chinese journal of intelligent science and technology, 2022, 4(4): 461-476. DOI： 10.11959/j.issn.2096-6652.202255.

摘要

随着基础理论和硬件计算能力的飞速发展，深度学习技术在众多领域取得了令人瞩目的成绩。作为描述客观物理世界的重要工具，长期以来微分方程是各领域研究人员关心的重点。近年来，深度学习和微分方程的结合逐渐成了研究的热点。由于深度学习能够从大量数据中高效地提取特征，微分方程能够反应客观的物理规律，因此二者的结合可以有效地提升深度学习的泛化性，同时增强深度学习的可解释性。首先，介绍了深度学习求解微分方程的基本问题。其次，介绍了两类深度学习求解微分方程的方法：数据驱动和物理知情方法。然后，讨论了微分方程深度学习求解方法在实际中的应用。与此同时，在充分调研的基础上提出了科学智能大模型——DeDAO（微分之道），以应对现有的挑战。最后，对微分方程深度学习求解方法进行了简要总结。

Abstract

With the rapid advancement of fundamental theories and computing capacity

deep learning techniques have made impressive achievements in many fields.Differential equations

as an important tool for describing the physical world

have long been a focus of interest for researchers in various fields.Combining the two methods has gained popularity as a study issue in recent years.Since deep learning can efficiently extract features from large amounts of data and differential equations can reflect objective physical laws

the combination of the two can effectively improve the generalization ability of deep learning and enhance the interpretability of deep learning.Firstly

the problem of solving differential equations by deep learning was briefly introduced.Then

two types of deep learning methods for solving differential equations were introduced: data-driven and physical-informed methods.Furthermore

the applications of relevant deep learning-based solving methods were discussed.Meanwhile

DeDAO (differential equations DAO)

a foundation model for artificial intelligence for science

was proposed to address existing challenges.Finally

conclusions of deep learning methods for solving differential equations were presented.

关键词

Keywords

references

ZILL D . A first course in differential equations with modeling applications [M ] . Stamford : Cengage Learning , 2012 .

HART J K , MARTINEZ K . Environmental sensor networks:a revolution in the earth system science? [J ] . Earth-Science Reviews , 2006 , 78 ( 3/4 ): 177 - 191 .

THOMAS J . Numerical partial differential equations:finite difference methods [M ] . New York : Springer Science ＆ Business Media , 2013 .

DONEA J , HUERTA A . Finite element methods for flow problems [M ] . Hoboken : John Wiley ＆ Sons , 2003 .

FEIT M , FLECK J , STEIGER A . Solution of the Schrödinger equation by a spectral method [J ] . Journal of Computational Physics , 1982 , 47 ( 3 ): 412 - 433 .

LIU G R , GUY T . An introduction to meshfree methods and their programming [M ] . New York : Springer Science ＆ Business Media , 2005 .

LECUN Y , BENGIO Y , HINTON G . Deep learning [J ] . Nature , 2015 , 521 ( 7553 ): 436 - 444 .

SUTTON R S , BARTO A G . Reinforcement learning:an introduction [M ] . Cambridge : MIT Press , 2018 .

李涛 , 魏庆来 . 基于深度强化学习的智能暖气温度控制系统 [J ] . 智能科学与技术学报 , 2020 , 2 ( 4 ): 348 - 353 .

LI T , WEI Q L . Intelligent heating temperature control system based on deep reinforcement learning [J ] . Chinese Journal of Intelligent Science and Technology , 2020 , 2 ( 4 ): 348 - 353 .

覃缘琪 , 季青原 , 葛俊 , 等 . 城市交通路网动态短时推理与精准预测研究 [J ] . 智能科学与技术学报 , 2022 , 4 ( 3 ): 380 - 395 .

QIN Y Q , JI Q Y , GE J , et al . Short-term traffic state reasoning and precise prediction in urban networks [J ] . Chinese Journal of Intelligent Science and Technology , 2022 , 4 ( 3 ): 380 - 395 .

曾淦雄 , 柯逍 . 基于 3D 卷积的图像序列特征提取与自注意力的车牌识别方法 [J ] . 智能科学与技术学报 , 2021 , 3 ( 3 ): 268 - 279 .

ZENG G X , KE X . 3D convolution-based image sequence feature extraction and self-attention for license plate recognition method [J ] . Chinese Journal of Intelligent Science and Technology , 2021 , 3 ( 3 ): 268 - 279 .

杜倩龙 , 宗成庆 , 苏克毅 . 利用上下文相似度增强词对齐效果的自然语言推理方法 [J ] . 智能科学与技术学报 , 2020 , 2 ( 1 ): 26 - 35 .

DU Q L , ZONG C Q , SU K Y . Enhancing alignment with context similarity for natural language inference [J ] . Chinese Journal of Intelligent Science and Technology , 2020 , 2 ( 1 ): 26 - 35 .

洪依 , 孙成立 , 冷严 . 基于超轻量通道注意力的端对端语音增强方法 [J ] . 智能科学与技术学报 , 2021 , 3 ( 3 ): 351 - 358 .

HONG Y , SUN C L , LENG Y . End-to-end speech enhancement based on ultra-lightweight channel attention [J ] . Chinese Journal of Intelligent Science and Technology , 2021 , 3 ( 3 ): 351 - 358 .

SILVER D , SCHRITTWIESER J , SIMONYAN K , et al . Mastering the game of Go without human knowledge [J ] . Nature , 2017 , 550 ( 7676 ): 354 - 359 .

JUMPER J , EVANS R , PRITZEL A , et al . Highly accurate protein structure prediction with AlphaFold [J ] . Nature , 2021 , 596 ( 7873 ): 583 - 589 .

CIODARO T , DEVA D , DE SEIXAS J M , et al . Online particle detection with neural networks based on topological calorimetry information [J ] . Journal of Physics:Conference Series , 2012 ,368.

HELMSTAEDTER M , BRIGGMAN K L , TURAGA S C , et al . Connectomic reconstruction of the inner plexiform layer in the mouse retina [J ] . Nature , 2013 , 500 ( 7461 ): 168 - 174 .

BOMMASANI R , HUDSON D A , ADELI E , et al . On the opportunities and risks of foundation models [J ] . arXiv preprint ， 2021 ,arXiv:2108.07258.

LI X , WANG Y T , YAN L , et al . ParallelEye-CS:a new dataset of synthetic images for testing the visual intelligence of intelligent vehicles [J ] . IEEE Transactions on Vehicular Technology , 2019 , 68 ( 10 ): 9619 - 9631 .

EVERINGHAM M , GOOL L , WILLIAMS C K I , et al . The pascal visual object classes (VOC) challenge [J ] . International Journal of Computer Vision , 2010 , 88 ( 2 ): 303 - 338 .

KARNIADAKIS G E , KEVREKIDIS I G , LU L , et al . Physics-informed machine learning [J ] . Nature Reviews Physics , 2021 , 3 ( 6 ): 422 - 440 .

RAISSI M , PERDIKARIS P , KARNIADAKIS G E . Physics-informed neural networks:a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations [J ] . Journal of Computational Physics , 2019 , 378 : 686 - 707 .

王飞跃 , 王景美 , 谢贻权 . 最佳格点集在最小二乘法中的应用 [J ] . 浙江大学学报(自然科学版) , 1988 , 22 ( 3 ): 33 - 43 .

WANG F Y , WANG J M , XIE Y Q . An application of good lattice points in least square method [J ] . Journal of Zhejiang University (Natural Science) , 1988 , 22 ( 3 ): 33 - 43 .

王飞跃 , 谢贻权 . 最小二乘加权残值方法的速进算法 [C ] // 第三届全国加权残值法会议论文集 . 成都:西南交通大学出版社 , 1989 : 312 - 316 .

WANG F Y , XIE Y Q . Sequential algorithms for least square weighted residual methods [C ] // Proceedings of the 3rd National Conference on Weighted Residual Methods . Chengdu:Southwest Jiaotong University Press , 1989 : 312 - 316 .

LAGARIS I E , LIKAS A , FOTIADIS D I . Artificial neural networks for solving ordinary and partial differential equations [J ] . IEEE Transactions on Neural Networks , 1998 , 9 ( 5 ): 987 - 1000 .

LU L , MENG X H , MAO Z P , et al . DeepXDE:a deep learning library for solving differential equations [J ] . SIAM Review , 2021 , 63 ( 1 ): 208 - 228 .

WINOVICH N , RAMANI K , LIN G . ConvPDE-UQ:convolutional neural networks with quantified uncertainty for heterogeneous elliptic partial differential equations on varied domains [J ] . Journal of Computational Physics , 2019 , 394 : 263 - 279 .

LI Z J , MEIDANI K , FARIMANI A . Transformer for partial differential equations' operator learning [J ] . arXiv preprint . 2022 ,arXiv:2205.13671.

DONG B , JIANG Q T , SHEN Z W . Image restoration:wavelet frame shrinkage,nonlinear evolution PDEs,and beyond [J ] . Multiscale Modeling ＆ Simulation , 2017 , 15 ( 1 ): 606 - 660 .

LONG Z C , LU Y P , DONG B . PDE-Net 2.0:learning PDEs from data with a numeric-symbolic hybrid deep network [J ] . Journal of Computational Physics , 2019 ,399.

LU L , JIN P Z , PANG G F , et al . Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators [J ] . Nature Machine Intelligence , 2021 , 3 ( 3 ): 218 - 229 .

WANG S F , WANG H W , PERDIKARIS P . Improved architectures and training algorithms for deep operator networks [J ] . Journal of Scientific Computing , 2022 , 92 ( 2 ): 1 - 42 .

HAN J , JENTZEN A , E W . Solving high-dimensional partial differential equations using deep learning [J ] . Proceedings of the National Academy of Sciences of the United States of America , 2018 , 115 ( 34 ): 8505 - 8510 .

SIRIGNANO J , SPILIOPOULOS K . DGM:a deep learning algorithm for solving partial differential equations [J ] . Journal of Computational Physics , 2018 , 375 : 1339 - 1364 .

CHEN Z , CHURCHILL V , WU K L , et al . Deep neural network modeling of unknown partial differential equations in nodal space [J ] . Journal of Computational Physics , 2022 ,449.

ZHA W S , ZHANG W , LI D L , et al . Convolution-based model-solving method for three-dimensional,unsteady,partial differential equations [J ] . Neural Computation , 2022 , 34 ( 2 ): 518 - 540 .

MEADE A J , FERNANDEZ A A . Solution of nonlinear ordinary differential equations by feedforward neural networks [J ] . Mathematical and Computer Modelling , 1994 , 20 ( 9 ): 19 - 44 .

VAN MILLIGEN B P , TRIBALDOS V , JIMÉNEZ J A , . Neural network differential equation and plasma equilibrium solver [J ] . Physical Review Letters , 1995 , 75 ( 20 ): 3594 - 3597 .

MAI-DUY N , TRAN-CONG T , . Mesh-free radial basis function network methods with domain decomposition for approximation of functions and numerical solution of Poisson’s equations [J ] . Engineering Analysis with Boundary Elements , 2002 , 26 ( 2 ): 133 - 156 .

REYNALDI A , LUKAS S , MARGARETHA H . Backpropagation and levenberg-marquardt algorithm for training finite element neural network [C ] // Proceedings of 2012 6th UKSim/AMSS European Symposium on Computer Modeling and Simulation . Piscataway:IEEE Press , 2012 : 89 - 94 .

JUMPER J , EVANS R , PRITZEL A , et al . Highly accurate protein structure prediction with AlphaFold [J ] . Nature , 2021 , 596 ( 7873 ): 583 - 589 .

RAISSI M , KARNIADAKIS G E . Hidden physics models:machine learning of nonlinear partial differential equations [J ] . Journal of Computational Physics , 2018 , 357 : 125 - 141 .

BAR-SINAI Y , HOYER S , HICKEY J , et al . Learning data-driven discretizations for partial differential equations [J ] . Proceedings of the National Academy of Sciences of the United States of America , 2019 , 116 ( 31 ): 15344 - 15349 .

PANG G F , LU L , KARNIADAKIS G E . fPINNs:fractional physics-informed neural networks [J ] . SIAM Journal on Scientific Computing , 2019 , 41 ( 4 ): A2603 - A2626 .

GAO H , SUN L N , WANG J X . PhyGeoNet:physics-informed geometry-adaptive convolutional neural networks for solving parameterized steady-state PDEs on irregular domain [J ] . Journal of Computational Physics , 2021 ,428.

ZHU Y H , ZABARAS N , KOUTSOURELAKIS P-S , et al . Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data [J ] . Journal of Computational Physics , 2019 , 394 : 56 - 81 .

SUN L N , GAO H , PAN S W , et al . Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data [J ] . Computer Methods in Applied Mechanics and Engineering , 2020 ,361.

MENG X H , LI Z , ZHANG D K , et al . PPINN:parareal physicsinformed neural network for time-dependent PDEs [J ] . Computer Methods in Applied Mechanics and Engineering , 2020 ,370.

YANG L , MENG X H , KARNIADAKIS G E . B-PINNs:Bayesian physics-informed neural networks for forward and inverse PDE problems with noisy data [J ] . Journal of Computational Physics , 2021 ,425.

WANG Y Z , SUN H , SUN G Z . DSP-PIGAN:a precision-consistency machine learning algorithm for solving partial differential equations [C ] // Proceedings of 2021 13th International Conference on Machine Learning and Computing .[S.l.:s.n. ] , 2021 : 21 - 26 .

WANG S , WANG H , PERDIKARIS P . Learning the solution operator of parametric partial differential equations with physics-informed DeepONets [J ] . Science Advances , 2021 , 7 ( 40 ).

PASZKE A , GROSS S , MASSA F , et al . Pytorch:an imperative style,high-performance deep learning library [C ] // Proceedings of Advances in Neural Information Processing Systems .[S.l.:s.n. ] , 2019 .

ABADI M , AGARWAL A , BARHAM P , et al . Tensorflow:large-scale machine learning on heterogeneous distributed systems [J ] . arXiv preprint , 2015 ,arXiv:1603.04467.

王昀卓 . 求解偏微分方程的神经网络方法 [D ] . 合肥:中国科学技术大学 , 2021 .

WANG Y Z . Neural network for solving partial differential equations [D ] . Hefei:University of Science and Technology of China , 2021 .

HAN J H , NICA M , STINCHCOMBE A R . A derivative-free method for solving elliptic partial differential equations with deep neural networks [J ] . Journal of Computational Physics , 2020 ,419.

OSORIO J D , WANG Z , KARNIADAKIS G , et al . Forecasting solarthermal systems performance under transient operation using a data-driven machine learning approach based on the deep operator network architecture [J ] . Energy Conversion and Management , 2022 ,252.

FRANKLIN T S , SOUZA L S , FONTES R M , et al . A physicsinformed neural networks (PINN) oriented approach to flow metering in oil wells:an ESP lifted oil well system as a case study [J ] . Digital Chemical Engineering , 2022 ,5.

SHI H J , MENG M Q H . Deep koopman operator with control for nonlinear systems [J ] . IEEE Robotics and Automation Letters , 2022 , 7 ( 3 ): 7700 - 7707 .

张子龙 , 潘秋景 , 仉文岗 , 等 . 基于物理信息神经网络的盾构隧道诱发地表沉降预测 [J ] . 工程力学 , 2022 , 39 : 1 - 14 .

ZHANG Z L , PAN Q J , ZHANG W G , et al . Prediction of surface settlements induced by shield tunnelling using physics-informed neural networks [J ] . Engineering Mechanics , 2022 , 39 : 1 - 14 .

万鹏 , 李迎光 , 华家玘 , 等 . 基于元学习和 PINN 的变工况刀具磨损精确预测方法 [J ] . 南京航空航天大学学报 , 2022 , 54 ( 3 ): 387 - 396 .

WAN P , LI Y G , HUA J Q , et al . Accurate prediction method of tool wear under varying cutting conditions based on meta learning and PINN [J ] . Journal of Nanjing University of Aeronautics ＆ Astronautics , 2022 , 54 ( 3 ): 387 - 396 .

郑素佩 , 林云云 , 封建湖 , 等 . 浅水波方程的粘性正则化 PINN 算法 [J ] . 计算物理 , 2022 : 1 - 15 .

ZHENG S P , LIN Y Y , FENG J H , et al . Viscous regularization PINN algorithm for shallow water equations [J ] . Chinese Journal of Computational Physics , 2022 : 1 - 15 .

GOSWAMI S , ANITESCU C , CHAKRABORTY S , et al . Transfer learning enhanced physics informed neural network for phase-field modeling of fracture [J ] . Theoretical and Applied Fracture Mechanics , 2020 ,106.

ZHANG E R , YIN M L , KARNIADAKIS G E . Physics-informed neural networks for nonhomogeneous material identification in elasticity imaging [J ] . arXiv preprint , 2020 ,arXiv:2009.04525.

BECK C , BECKER S , GROHS P , et al . Solving the Kolmogorov PDE by means of deep learning [J ] . Journal of Scientific Computing , 2021 , 88 ( 3 ): 1 - 28 .

GLAU K , WUNDERLICH L . The deep parametric PDE method and applications to option pricing [J ] . Applied Mathematics and Computation , 2022 ,432.

BARMPARIS G D , TSIRONIS G P . Physics-informed machine learning for the COVID-19 pandemic:adherence to social distancing and short-term predictions for eight countries [J ] . Quantitative Biology , 2022 , 10 ( 2 ): 139 .

GRIMM V , HEINLEIN A , KLAWONN A , et al . Estimating the time-dependent contact rate of SIR and SEIR models in mathematical epidemiology using physics-informed neural networks [J ] . Electronic Transactions on Numerical Analysis , 2021 , 56 : 1 - 27 .

CAVANAGH H , MOSBACH A , SCALLIET G , et al . Physicsinformed deep learning characterizes morphodynamics of Asian soybean rust disease [J ] . Nature Communications , 2021 , 12 : 1 - 9 .

ZHANG S T , WU J R , WANG Y G , et al . A physics-informed statistical learning framework for forecasting local suspended sediment concentrations in marine environment [J ] . Water Research , 2022 ,218.

GROSS M R , HYMAN J D , SRINIVASAN S , et al . A physicsinformed machine learning workflow to forecast production in a fractured Marcellus shale reservoir [C ] // Proceedings of the 9th Unconventional Resources Technology Conference . Tulsa:American Association of Petroleum Geologists , 2021 .

KASHINATH K , MUSTAFA M , ALBERT A , et al . Physics-informed machine learning:case studies for weather and climate modelling [J ] . Philosophical Transactions Series A,Mathematical,Physical,and Engineering Sciences , 2021 , 379 ( 2194 ).

POMBO D V , RINCON M J , BACHER P , et al . Assessing stacked physics-informed machine learning models for co-located wind-solar power forecasting [J ] . Sustainable Energy,Grids and Networks , 2022 ,32.

JHONG B C , LIN C Y , JHONG Y D , et al . Assessing the effective spatial characteristics of input features through physics-informed machine learning models in inundation forecasting during typhoons [J ] . Hydrological Sciences Journal , 2022 , 67 ( 10 ): 1527 - 1545 .

BUKHARI A H , RAJA M A Z , SHOAIB M , et al . Fractional order Lorenz based physics informed SARFIMA-NARX model to monitor and mitigate megacities air pollution [J ] . Chaos,Solitons ＆ Fractals , 2022 ,161.

RICE J L , XU W W , AUGUST A . Analyzing Koopman approaches to physics-informed machine learning for long-term sea-surface temper ature forecasting [J ] . arXiv preprint , 2020 ,arXiv:2010.00399.

HORNIK K , STINCHCOMBE M , WHITE H . Multilayer feedforward networks are universal approximators [J ] . Neural Networks , 1989 , 2 ( 5 ): 359 - 366 .

WANG F Y . The emergence of intelligent enterprises:from CPS to CPSS [J ] . IEEE Intelligent Systems , 2010 , 25 ( 4 ): 85 - 88 .

王飞跃 . 平行系统方法与复杂系统的管理和控制 [J ] . 控制与决策 , 2004 , 19 ( 5 ): 485 - 489 , 514 .

WANG F Y . Parallel system methods for management and control of complex systems [J ] . Control and Decision , 2004 , 19 ( 5 ): 485 - 489 , 514 .

王飞跃 . 平行控制与数字孪生：经典控制理论的回顾与重铸 [J ] . 智能科学与技术学报 , 2020 , 2 ( 3 ): 293 - 300 .

WANG F Y . Parallel control and digital twins:control theory revisited and reshaped [J ] . Chinese Journal of Intelligent Science and Technology , 2020 , 2 ( 3 ): 293 - 300 .

WANG F Y . The DAO to MetaControl for MetaSystems in metaverses:the system of parallel control systems for knowledge automation and control intelligence in CPSS [J ] . IEEE/CAA Journal of Automatica Sinica , 2022 , 9 ( 11 ): 1899 - 1908 .

WANG F Y . Shadow systems:a new concept for nested and embedded co-simulation for intelligent systems [R ] . 1994 .

WEI Q L , LI H Y , WANG F Y . Parallel control for continuous-time linear systems:a case study [J ] . IEEE/CAA Journal of Automatica Sinica , 2020 , 7 ( 4 ): 919 - 928 .

LU J W , WEI Q L , WANG F Y . Parallel control for optimal tracking via adaptive dynamic programming [J ] . IEEE/CAA Journal of Automatica Sinica , 2020 , 7 ( 6 ): 1662 - 1674 .

LU J W , WEI Q L , WANG Z Y , et al . Event-triggered optimal control for discrete-time multi-player non-zero-sum games using parallel control [J ] . Information Sciences , 2022 , 584 : 519 - 535 .

LU J W , WEI Q L , LIU Y J , et al . Event-triggered optimal parallel tracking control for discrete-time nonlinear systems [J ] . IEEE Transactions on Systems,Man,and Cybernetics:Systems , 2022 , 52 ( 6 ): 3772 - 3784 .

LU J W , WEI Q L , ZHOU T M , et al . Event-triggered near-optimal control for unknown discrete-time nonlinear systems using parallel control [J ] . IEEE Transactions on Cybernetics , 2022 .

DEVLIN J , CHANG M W , LEE K , et al . BERT:pre-training of deep bidirectional transformers for language understanding [C ] // Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies .[S.l.:s.n. ] , 2019 : 4171 - 4186 .

BROWN T , MANN B , RYDER N , et al . Language models are few-shot learners [C ] // Proceedings of the Conference on Neural Information Processing Systems .[S.l.:s.n. ] , 2020 : 1877 - 1901 .

RADFORD A , KIM J W , HALLACY C , et al . Learning transferable visual models from natural language supervision [C ] // Proceedings of the International Conference on Machine Learning .[S.l.:s.n. ] , 2021 : 8748 - 8763 .

ALAYRAC J B , DONAHUE J , LUC P , et al . Flamingo:a visual language model for few-shot learning [J ] . arXiv preprint , 2022 ,arXiv:2204.14198.

SU B , DU D , YANG Z , et al . A molecular multimodal foundation model associating molecule graphs with natural language [J ] . arXiv preprint , 2022 ,arXiv:1810.05481.

XIE Z D , ZHANG Z , CAO Y , et al . SimMIM:a simple framework for masked image modeling [C ] // Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2022 : 9643 - 9653 .

王飞跃 . 平行管理:复杂性管理智能的生态科技与智慧管理之DAO [J ] . 自动化学报 , 2022 , 48 ( 11 ): 2655 - 2669 .

WANG F Y . Parallel management:the DAO to smart ecological technology for complexity management intelligence [J ] . Acta Automatica Sinica , 2022 , 48 ( 11 ): 2655 - 2669 .

SHAZEER N , MIRHOSEINI A , MAZIARZ K , et al . Outrageously large neural networks:the sparsely-gated mixture-of-experts layer [C ] // Proceedings of the International Conference on Learning Representations .[S.l.:s.n. ] , 2017 .

CHENG X , LU J , WANG F Y . Deep learning meets optimal latticesfor continuous traffic flow prediction [J ] . International Journal of Intelligent Control and Systems , 2022 , 2 ( 1 ): 7 - 12 .

浏览量

1516

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

融合音画同步的唇形合成研究

基于深度学习的MRI脑卒中病灶分割方法综述

移动目标轨迹预测方法研究综述

强化学习：迈向知行合一的智能机制与算法

人工智能进入后深度学习时代