Deep learning-based multimodal trajectory prediction methods for autonomous driving: state of the art and perspectives

Jun HUANG; Yonglin TIAN; Xingyuan DAI; Xiao WANG; Zhixing PING

doi:10.11959/j.issn.2096-6652.202317

您当前的位置：

首页 >

文章列表页 >

Deep learning-based multimodal trajectory prediction methods for autonomous driving: state of the art and perspectives

Surveys and Prospectives | 更新时间：2024-04-04

- Deep learning-based multimodal trajectory prediction methods for autonomous driving: state of the art and perspectives
- Chinese Journal of Intelligent Science and Technology Vol. 5, Issue 2, Pages: 180-199(2023)
- 作者机构：
  
  1. 中国科学院自动化研究所复杂系统管理与控制国家重点实验室，北京 100190
  2. 澳门科技大学，澳门 999078
  3. 青岛智能产业技术研究院，山东青岛 266000
  4. 北京怀柔平行传感智能研究院，北京 101400
  5. 中国科学院自动化研究所北京市智能化技术与系统工程技术研究中心，北京 100190
  6. 安徽大学人工智能学院，安徽合肥 230601
  7. 自主无人系统技术教育部工程研究中心，安徽合肥 230601
  8. 安徽省无人系统与智能技术工程研究中心，安徽合肥 230601
  9. 北方自动控制技术研究所，山西太原 030006
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-6652.202317
  CLC： TP29
- Revised：2023-05-26，
  
  Online First：2023-06，
  
  Published：15 June 2023
- 稿件说明：
移动端阅览
Jun HUANG, Yonglin TIAN, Xingyuan DAI, et al. Deep learning-based multimodal trajectory prediction methods for autonomous driving: state of the art and perspectives[J]. Chinese Journal of Intelligent Science and Technology, 2023, 5(2): 180-199.
DOI：

Jun HUANG, Yonglin TIAN, Xingyuan DAI, et al. Deep learning-based multimodal trajectory prediction methods for autonomous driving: state of the art and perspectives[J]. Chinese Journal of Intelligent Science and Technology, 2023, 5(2): 180-199. DOI： 10.11959/j.issn.2096-6652.202317.

摘要

对周围车辆轨迹的精确预测可以辅助自动驾驶车辆做出合理的即时决策。虽然相比传统轨迹预测算法，深度学习方法已取得较好效果，但是自动驾驶车辆在异构高动态复杂变化环境下实现多模态高精度预测仍存在信息丢失、交互和不确定性难以建模、预测缺乏可解释性等问题。Transformer具备的长距离建模能力和并行计算能力使其不仅在自然语言处理领域取得巨大成功，而且在扩展至自动驾驶多模态轨迹预测任务时也解决了以上问题。基于此，对过去基于深度神经网络的方法，特别是对基于Transformer的方法进行全面总结与回顾；同时分析了Transformer相较于传统序列网络、图神经网络、生成模型的优势，并结合现有难题进行针对性分析与分类。Transformer模型可以更好地应用于多模态轨迹预测任务，此类模型具有更好的泛化性和可解释性。最后，对多模态轨迹预测未来发展方向进行了展望。

Abstract

Although deep learning methods have achieved better results than traditional trajectory prediction algorithms

there are still problems such as information loss

interaction and uncertainty difficulties in modelling

and lack of interpretability of predictions when implementing multimodal high-precision prediction for autonomous vehicles in heterogeneous

highly dynamic and complex changing environments.The newly developed Transformer's long-range modelling capability and parallel computing ability make it a great success not only in the field of natural language processing

but also in solving the above problems when extended to the task of multimodal trajectory prediction for autonomous driving.Based on this

the aim of this paper is to provide a comprehensive summary and review of past deep neural network-based approaches

in particular the Transformer-based approach.The advantages of Transformer over traditional sequential network

graphical neural network and generative model were also analyzed and classified in relation to existing challenges

simultaneously.Transformer models can be better applied to multimodal trajectory prediction tasks

and that such models have better generalisation and interpretability.Finally

the future directions of multimodal trajectory prediction were presented.

关键词

Keywords

references

LEVINSON J , ASKELAND J , BECKER J , et al . Towards fully autonomous driving:systems and algorithms [C ] // Proceedings of 2011 IEEE Intelligent Vehicles Symposium (IV) . Piscataway:IEEE Press , 2011 : 163 - 168 .

MAURER M , GERDES J C , LENZ B , et al . Autonomous driving:technical,legal and social aspects [M ] . Springer Berlin : Nature , 2016 .

YURTSEVER E , LAMBERT J , CARBALLO A , et al . A survey of autonomous driving:common practices and emerging technologies [J ] . IEEE Access , 2020 , 8 : 58443 - 58469 .

张慧 , 王坤峰 , 王飞跃 . 深度学习在目标视觉检测中的应用进展与展望 [J ] . 自动化学报 , 2017 , 43 ( 8 ): 1289 - 1305 .

ZHANG H , WANG K F , WANG F Y . Advances and perspectives on applications of deep learning in visual object detection [J ] . Acta Automatica Sinica , 2017 , 43 ( 8 ): 1289 - 1305 .

田永林 , 王雨桐 , 王建功 , 等 . 视觉 Transformer 研究的关键问题:现状及展望 [J ] . 自动化学报 , 2022 , 48 ( 4 ): 957 - 979 .

TIAN Y L , WANG Y T , WANG J G , et al . Key problems and progress of vision transformers:the state of the art and prospects [J ] . Acta Automatica Sinica , 2022 , 48 ( 4 ): 957 - 979 .

卢经纬 , 郭超 , 戴星原 , 等 . 问答ChatGPT之后:超大预训练模型的机遇和挑战 [J ] . 自动化学报 , 2023 , 49 ( 4 ): 705 - 717 .

LU J W , GUO C , DAI X Y , et al . The ChatGPT after:opportunities and challenges of very large scale pre-trained models [J ] . Acta Automatica Sinica , 2023 , 49 ( 4 ): 705 - 717 .

王飞跃 , 缪青海 , 张军平 , 等 . 探讨AI for Science的影响与意义:现状与展望 [J ] . 智能科学与技术学报 , 2023 , 5 ( 1 ): 1 - 6 .

WANG F Y , MIAO Q H , ZHANG J P , et al . The DAOs to AI for Science by DeSci:the state of the art and perspective [J ] . Chinese Journal of Intelligent Science and Technology , 2023 , 5 ( 1 ): 1 - 6 .

李娟娟 , 秦蕊 , 丁文文 , 等 . 基于Web3的去中心化自治组织与运营新框架 [J ] . 自动化学报 , 2023 , 49 ( 5 ): 985 - 998 .

LI J J , QIN R , DING W W , et al . A new framework for Web3-powered decentralized autonomous organizations and operations [J ] . Acta Automatica Sinica , 2023 , 49 ( 5 ): 985 - 998 .

LECUN Y , BOSER B , DENKER J S , et al . Backpropagation applied to handwritten zip code recognition [J ] . Neural Computation , 1989 , 1 ( 4 ): 541 - 551 .

YE P J , WANG X , ZHENG W B , et al . Parallel cognition:hybrid intelligence for human-machine interaction and management [J ] . Frontiers of Information Technology ＆ Electronic Engineering , 2022 , 23 ( 12 ): 1765 - 1779 .

WANG X , YANG J , HAN J P , et al . Metaverses and DeMetaverses:from digital twins in CPS to parallel intelligence in CPSS [J ] . IEEE Intelligent Systems , 2022 , 37 ( 4 ): 97 - 102 .

XIE G , SHANGGUAN A Q , FEI R , et al . Motion trajectory prediction based on a CNN-LSTM sequential model [J ] . Science China Information Sciences , 2020 , 63 ( 11 ): 1 - 21 .

GIRSHICK R . Fast R-CNN [C ] // Proceedings of 2015 IEEE International Conference on Computer Vision (ICCV) . Piscataway:IEEE Press , 2015 : 1440 - 1448 .

HE K M , GKIOXARI G , DOLLÁR P , et al . Mask R-CNN [C ] // Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV) . Piscataway:IEEE Press , 2017 : 2980 - 2988 .

GRAVES A . Generating sequences with recurrent neural networks [J ] . arXiv preprint . 2013 ,2013,arXiv:1308.0850.

BAHDANAU D , CHO K , BENGIO Y . Neural machine translation by jointly learning to align and translate [J ] . arXiv preprint , 2014 ,arXiv:1409.0473.

HOCHREITER S , SCHMIDHUBER J . Long short-term memory [J ] . Neural Computation , 1997 , 9 ( 8 ): 1735 - 1780 .

FU R , ZHANG Z , LI L . Using LSTM and GRU neural network methods for traffic flow prediction [C ] // Proceedings of 2016 31st Youth Academic Annual Conference of Chinese Association of Automation (YAC) . Piscataway:IEEE Press , 2016 : 324 - 328 .

DEY R , SALEM F M . Gate-variants of Gated Recurrent Unit (GRU) neural networks [C ] // Proceedings of 2017 IEEE 60th International Midwest Symposium on Circuits and Systems (MWSCAS) . Piscataway:IEEE Press , 2017 : 1597 - 1600 .

CHUNG J , GULCEHRE C , CHO K H , et al . Empirical evaluation of gated recurrent neural networks on sequence modeling [J ] . arXiv preprint , 2014 ,arXiv:1412.3555.

MESSAOUD K , YAHIAOUI I , VERROUST-BLONDET A , et al . Nonlocal social pooling for vehicle trajectory prediction [C ] // Proceedings of 2019 IEEE Intelligent Vehicles Symposium (IV) . Piscataway:IEEE Press , 2019 : 975 - 980 .

MESSAOUD K , YAHIAOUI I , VERROUST-BLONDET A , , et al . Attention based vehicle trajectory prediction [J ] . IEEE Transactions on Intelligent Vehicles , 2021 , 6 ( 1 ): 175 - 185 .

MESSAOUD K , DEO N , TRIVEDI M M , et al . Trajectory prediction for autonomous driving based on multi-head attention with joint agentmap representation [C ] // Proceedings of 2021 IEEE Intelligent Vehicles Symposium (IV) . Piscataway:IEEE Press , 2021 : 165 - 170 .

VASWANI A , SHAZEER N , PARMAR N , et al . Attention is all you need [C ] // Proceedings of the 31st International Conference on Neural Information Processing Systems . New York:ACM , 2017 : 6000 - 6010 .

LIU Y C , ZHANG J H , FANG L J , et al . Multimodal motion prediction with stacked transformers [C ] // Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2021 : 7573 - 7582 .

ZHANG K , LI L . Explainable multimodal trajectory prediction using attention models [J ] . Transportation Research Part C:Emerging Technologies , 2022 ,143:103829.

WU Z H , PAN S R , CHEN F W , et al . A comprehensive survey on graph neural networks [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2021 , 32 ( 1 ): 4 - 24 .

SCARSELLI F , GORI M , TSOI A C , et al . The graph neural network model [J ] . IEEE Transactions on Neural Networks , 2009 , 20 ( 1 ): 61 - 80 .

DIEHL F , BRUNNER T , LE M T , et al . Graph neural networks for modelling traffic participant interaction [C ] // Proceedings of 2019 IEEE Intelligent Vehicles Symposium . Piscataway:IEEE Press , 2019 : 695 - 701 .

KIPF T N , WELLING M . Semi-supervised classification with graph convolutional networks [J ] . arXiv preprint , 2016 ,arXiv:1609.02907.

ZHANG S , TONG H H , XU J J , et al . Graph convolutional networks:a comprehensive review [J ] . Computational Social Networks , 2019 , 6 ( 1 ): 1 - 23 .

VELIČKOVIĆ P , CUCURULL G , CASANOVA A , et al . Graph attention networks [J ] . arXiv preprint , 2017 ,arXiv:1710.10903.

JI H Y , WANG X , SHI C , et al . Heterogeneous graph propagation network [J ] . IEEE Transactions on Knowledge and Data Engineering , 2023 , 35 ( 1 ): 521 - 532 .

WANG X , HE X N , CAO Y X , et al . KGAT:knowledge graph attention network for recommendation [C ] // Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery ＆ Data Mining . New York:ACM , 2019 : 950 - 958 .

KIPF T N , WELLING M . Variational graph auto-encoders [J ] . arXiv preprint , 2016 ,arXiv:1611.07308.

PAN S , HU R , LONG G , et al . Adversarially regularized graph autoencoder for graph embedding [J ] . arXiv preprint , 2018 ,arXiv:1802.04407.

ZHOU D W , ZHENG L C , HAN J W , et al . A data-driven graph generative model for temporal interaction networks [C ] // Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery ＆ Data Mining . New York:ACM , 2020 : 401 - 411 .

BONGINI P , BIANCHINI M , SCARSELLI F . Molecular generative graph neural networks for drug discovery [J ] . Neurocomputing , 2021 , 450 : 242 - 252 .

YAN S , XIONG Y , LIN D . Spatial temporal graph convolutional networks for skeleton-based action recognition [J ] . arXiv preprint , 2018 ,arXiv:1801.07455.

GUO S N , LIN Y F , FENG N , et al . Attention based spatial-temporal graph convolutional networks for traffic flow forecasting [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2019 , 33 ( 1 ): 922 - 929 .

GOODFELLOW I , POUGET-ABADIE J , MIRZA M , et al . Generative adversarial networks [J ] . Communications of the ACM , 2020 , 63 ( 11 ): 139 - 144 .

CRESWELL A , WHITE T , DUMOULIN V , et al . Generative adversarial networks:an overview [J ] . IEEE Signal Processing Magazine , 2018 , 35 ( 1 ): 53 - 65 .

HO J , ERMON S . Generative adversarial imitation learning [C ] // Proceedings of the 30th International Conference on Neural Information Processing Systems . New York:ACM , 2016 : 4572 - 4580 .

SONG J M , REN H Y , SADIGH D , et al . Multi-agent generative adversarial imitation learning [C ] // Proceedings of the 32nd International Conference on Neural Information Processing Systems . New York:ACM , 2018 : 7472 - 7483 .

KOSARAJU V , SADEGHIAN A , MARTÍN-MARTÍN R , , et al . SocialBiGAT:multimodal trajectory forecasting using bicycle-GAN and graph attention networks [J ] . arXiv preprint , 2019 ,arXiv:1907.03395.

ZHAO T , ZHAO R , ESKENAZI M . Learning discourse-level diversity for neural dialog models using conditional variational autoencoders [J ] . arXiv preprint , 2017 ,arXiv:1703.10960.

BAO J M , CHEN D , WEN F , et al . CVAE-GAN:fine-grained image generation through asymmetric training [C ] // Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV) . Piscataway:IEEE Press , 2017 : 2764 - 2773 .

IVANOVIC B , LEUNG K , SCHMERLING E , et al . Multimodal deep generative models for trajectory prediction:a conditional variational autoencoder approach [J ] . IEEE Robotics and Automation Letters , 2021 , 6 ( 2 ): 295 - 302 .

HONG J , SAPP B , PHILBIN J . Rules of the road:predicting driving behavior with a convolutional model of semantic interactions [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway:IEEE Press , 2019 : 8446 - 8454 .

LEFÈVRE S , VASQUEZ D , LAUGIER C . A survey on motion prediction and risk assessment for intelligent vehicles [J ] . ROBOMECH Journal , 2014 , 1 ( 1 ): 1 - 14 .

GONZÁLEZ D , PÉREZ J , MILANÉS V , et al . A review of motion planning techniques for automated vehicles [J ] . IEEE Transactions on Intelligent Transportation Systems , 2016 , 17 ( 4 ): 1135 - 1145 .

LIU J B , MAO X Y , FANG Y Q , et al . A survey on deep-learning approaches for vehicle trajectory prediction in autonomous driving [C ] // Proceedings of 2021 IEEE International Conference on Robotics and Biomimetics (ROBIO) . Piscataway:IEEE Press , 2021 : 978 - 985 .

MOZAFFARI S , AL-JARRAH O Y , DIANATI M , et al . Deep learningbased vehicle behavior prediction for autonomous driving applications:a review [J ] . IEEE Transactions on Intelligent Transportation Systems , 2022 , 23 ( 1 ): 33 - 47 .

LEON F , GAVRILESCU M . A review of tracking and trajectory prediction methods for autonomous driving [J ] . Mathematics , 2021 , 9 ( 6 ): 660 .

GOMES I , WOLF D . A review on intention-aware and interactionaware trajectory prediction for autonomous vehicles [J ] . 2022

SHIRAZI M S , MORRIS B T . Looking at intersections:a survey of intersection monitoring,behavior and safety analysis of recent studies [J ] . IEEE Transactions on Intelligent Transportation Systems , 2017 , 18 ( 1 ): 4 - 24 .

HUANG Y J , DU J T , YANG Z R , et al . A survey on trajectory-prediction methods for autonomous driving [J ] . IEEE Transactions on Intelligent Vehicles , 2022 , 7 ( 3 ): 652 - 674 .

KAELBLING L P , LITTMAN M L , MOORE A W . Reinforcement learning:a survey [J ] . Journal of Artificial Intelligence Research , 1996 , 4 : 237 - 285 .

NAYAK A , ESKANDARIAN A , DOERZAPH Z . Uncertainty estimation of pedestrian future trajectory using Bayesian approximation [J ] . IEEE Open Journal of Intelligent Transportation Systems , 2022 , 3 : 617 - 630 .

WANG X , TANG K , DAI X Y , et al . Safety-balanced driving-style aware trajectory planning in intersection scenarios with uncertain environment [J ] . IEEE Transactions on Intelligent Vehicles , 2023 , 8 ( 4 ): 2888 - 2898 .

DEO N , TRIVEDI M M . Multi-modal trajectory prediction of surrounding vehicles with maneuver based LSTMs [C ] // Proceedings of 2018 IEEE Intelligent Vehicles Symposium (IV) . Piscataway:IEEE Press , 2018 : 1179 - 1184 .

SHOUNO O . Deep unsupervised learning of a topological map of vehicle maneuvers for characterizing driving styles [C ] // Proceedings of 2018 21st International Conference on Intelligent Transportation Systems (ITSC) . Piscataway:IEEE Press , 2018 : 2917 - 2922 .

LIPTON Z C . The mythos of model interpretability:in machine learning,the concept of interpretability is both important and slippery [J ] . Queue , 2018 , 16 ( 3 ): 31 - 57 .

CHAKRABORTY S , TOMSETT R , RAGHAVENDRA R , et al . Interpretability of deep learning models:A survey of results [C ] // 2017 IEEE SmartWorld,Ubiquitous Intelligence ＆ Computing,Advanced ＆ Trusted Computed,Scalable Computing ＆ Communications,Cloud ＆ Big Data Computing,Internet of People and Smart City Innovation . Piscataway:IEEE Press , 2017 : 1 - 6 .

GUNNING D , STEFIK M , CHOI J , et al . XAI-explainable artificial intelligence [J ] . Science Robotics , 2019 , 4 ( 37 ): eaay7120 .

DU M N , LIU N H , HU X . Techniques for interpretable machine learning [J ] . Communications of the ACM , 2019 , 63 ( 1 ): 68 - 77 .

HUME D . An inquiry concerning human understanding [M ] . New York : Oxford University Press , 2003 .

YUAN Y , KITANI K . Diverse trajectory forecasting with determinantal point processes [J ] . arXiv preprint , 2019 ,arXiv:1907.04967.

KIM J , CANNY J . Interpretable learning for self-driving cars by visualizing causal attention [C ] // Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV) . Piscataway:IEEE Press , 2017 : 2961 - 2969 .

CAESAR H , BANKITI V , LANG A H , et al . nuScenes:a multimodal dataset for autonomous driving [C ] // Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway:IEEE Press , 2020 : 11618 - 11628 .

HOUSTON J , ZUIDHOF G , BERGAMINI L , et al . One thousand and one hours:Self-driving motion prediction dataset [C ] // Proceedings of Conference on Robot Learning . New York:PMLR , 2021 : 409 - 418 .

SUN P , KRETZSCHMAR H , DOTIWALLA X , et al . Scalability in perception for autonomous driving:waymo open dataset [C ] // Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway:IEEE Press , 2020 : 2443 - 2451 .

CHANG M F , LAMBERT J , SANGKLOY P , et al . Argoverse:3D tracking and forecasting with rich maps [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway:IEEE Press , 2019 : 8740 - 8749 .

WILSON B , QI W , AGARWAL T , et al . Argoverse 2:next generation datasets for self-driving perception and forecasting [J ] . arXiv preprint , 2023 ,arXiv:2301.00493.

ZHAN W , SUN L , WANG D , et al . Interaction dataset:an international,adversarial and cooperative motion dataset in interactive driving scenarios with semantic maps [J ] . arXiv preprint , 2019 ,arXiv:1910.03088.

KRAJEWSKI R , BOCK J , KLOEKER L , et al . The highD dataset:a drone dataset of naturalistic vehicle trajectories on German highways for validation of highly automated driving systems [C ] // Proceedings of 2018 21st International Conference on Intelligent Transportation Systems (ITSC) . Piscataway:IEEE Press , 2018 : 2118 - 2125 .

HUANG X Y , CHENG X J , GENG Q C , et al . The ApolloScape dataset for autonomous driving [C ] // Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) . Piscataway:IEEE Press , 2018 : 954 - 960 .

GEIGER A , LENZ P , STILLER C , et al . Vision meets robotics:the KITTI dataset [J ] . International Journal of Robotics Research , 2013 , 32 ( 11 ): 1231 - 1237 .

FANG L J , JIANG Q H , SHI J P , et al . TPNet:trajectory proposal net work for motion prediction [C ] // Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway:IEEE Press , 2020 : 6796 - 6805 .

NIKHIL N , MORRIS B T . Convolutional Neural Network for Trajectory Prediction [C ] // European Conference on Computer Vision . Cham:Springer , 2019 : 186 - 196 .

CHOU F C , LIN T H , CUI H G , et al . Predicting motion of vulnerable road users using high-definition maps and efficient ConvNets [C ] // Proceedings of 2020 IEEE Intelligent Vehicles Symposium (IV) . Piscataway:IEEE Press , 2020 : 1655 - 1662 .

CUI H G , RADOSAVLJEVIC V , CHOU F C , et al . Multimodal trajectory predictions for autonomous driving using deep convolutional networks [C ] // Proceedings of 2019 International Conference on Robotics and Automation (ICRA) . Piscataway:IEEE Press , 2019 : 2090 - 2096 .

PHAN-MINHT , GRIGOREE C , BOULTONF A , et al . CoverNet:multimodal behavior prediction using trajectory sets [C ] // Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway:IEEE Press , 2020 : 14062 - 14071 .

CUI H G , NGUYEN T , CHOU F C , et al . Deep kinematic models for kinematically feasible vehicle trajectory predictions [C ] // Proceedings of 2020 IEEE International Conference on Robotics and Automation (ICRA) . Piscataway:IEEE Press , 2020 : 10563 - 10569 .

DEO N , TRIVEDI M M . Convolutional social pooling for vehicle trajectory prediction [C ] // Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) . Piscataway:IEEE Press , 2018 : 1468 - 1476 .

MARZINOTTO A , COLLEDANCHISE M , SMITH C , et al . Towards a unified behavior trees framework for robot control [C ] // Proceedings of 2014 IEEE International Conference on Robotics and Automation (ICRA) . Piscataway:IEEE Press , 2014 : 5420 - 5427 .

ZHANG Y , ZOU Y , TANG J , et al . A lane-changing prediction method based on temporal convolution network [J ] . arXiv preprint , 2020 ,arXiv:2011.01224.

GILLES T , SABATINI S , TSISHKOU D , et al . HOME:heatmap output for future motion estimation [C ] // Proceedings of 2021 IEEE International Intelligent Transportation Systems Conference (ITSC) . Piscataway:IEEE Press , 2021 : 500 - 507 .

GILLES T , SABATINI S , TSISHKOU D , et al . GOHOME:graphoriented heatmap output for future motion estimation [C ] // Proceedings of 2022 International Conference on Robotics and Automation (ICRA) . Piscataway:IEEE Press , 2022 : 9107 - 9114 .

GILLES T , SABATINI S , TSISHKOU D , et al . Thomas:trajectory heatmap output with learned multi-agent sampling [J ] . arXiv preprint . 2021 ,arXiv:2110.06607.

YE M S , CAO T Y , CHEN Q F . TPCN:temporal point cloud networks for motion forecasting [C ] // Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway:IEEE Press , 2021 : 11313 - 11322 .

GIULIARI F , HASAN I , CRISTANI M , et al . Transformer networks for trajectory forecasting [C ] // Proceedings of 2020 25th International Conference on Pattern Recognition (ICPR) . Piscataway:IEEE Press , 2020 : 10335 - 10342 .

ALTCHÉ F , DE LA FORTELLE A . An LSTM network for highway trajectory prediction [C ] // Proceedings of 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC) . Piscataway:IEEE Press , 2017 : 353 - 359 .

ZYNER A , WORRALL S , NEBOT E . Naturalistic driver intention and path prediction using recurrent neural networks [J ] . IEEE Transactions on Intelligent Transportation Systems , 2019 , 21 ( 4 ): 1584 - 1594 .

PARK S H , KIM B , KANG C M , et al . Sequence-to-sequence prediction of vehicle trajectory via LSTM encoder-decoder architecture [C ] // Proceedings of 2018 IEEE Intelligent Vehicles Symposium (IV) . Piscataway:IEEE Press , 2018 : 1672 - 1678 .

SONG H , DING W , CHEN Y , et al . Pip:planning-informed trajectory prediction for autonomous driving [C ] // Proceedings of Computer VisionECCV 2020:16th European Conference .[S.l.:s.n. ] , 2020 : 598 - 614 .

XIN L , WANG P , CHAN C Y , et al . Intention-aware long horizon trajectory prediction of surrounding vehicles using dual LSTM networks [C ] // Proceedings of 2018 21st International Conference on Intelligent Transportation Systems (ITSC) . Piscataway:IEEE Press , 2018 : 1441 - 1446 .

SHIRAZI M S , MORRIS B T . Trajectory prediction of vehicles turning at intersections using deep neural networks [J ] . Machine Vision and Applications , 2019 , 30 ( 6 ): 1097 - 1109 .

XING Y , LV C , CAO D P . Personalized vehicle trajectory prediction based on joint time-series modeling for connected vehicles [J ] . IEEE Transactions on Vehicular Technology , 2020 , 69 ( 2 ): 1341 - 1352 .

ZHANG T , SONG W J , FU M Y , et al . Vehicle motion prediction at intersections based on the turning intention and prior trajectories model [J ] . IEEE/CAA Journal of Automatica Sinica , 2021 , 8 ( 10 ): 1657 - 1666 .

BAE S , SAXENA D , NAKHAEI A , et al . Cooperation-aware lane change maneuver in dense traffic based on model predictive control with recurrent neural network [C ] // Proceedings of 2020 American Control Conference (ACC) . Piscataway:IEEE Press , 2020 : 1209 - 1216 .

MO X Y , XING Y , LV C . Interaction-aware trajectory prediction of connected vehicles using CNN-LSTM networks [C ] // Proceedings of IECON 2020 the 46th Annual Conference of the IEEE Industrial Electronics Society . Piscataway:IEEE Press , 2020 : 5057 - 5062 .

HUANG Z , WANG J , PI L , et al . LSTM based trajectory prediction model for cyclist utilizing multiple interactions with environment [J ] . Pattern Recognition , 2021 ,112:107800.

WANG S B , ZHAO P , YU B , et al . Vehicle trajectory prediction by knowledge-driven LSTM network in urban environments [J ] . Journal of Advanced Transportation , 2020 , 2020 : 1 - 20 .

JU C , WANG Z , LONG C , et al . Interaction-aware Kalman neural networks for trajectory prediction [C ] // Proceedings of 2020 IEEE Intelligent Vehicles Symposium (IV) . Piscataway:IEEE Press , 2020 : 1793 - 1800 .

LIN L , LI W Z , BI H K , et al . Vehicle trajectory prediction using LSTMs with spatial-temporal attention mechanisms [J ] . IEEE Intelligent Transportation Systems Magazine , 2022 , 14 ( 2 ): 197 - 208 .

SHI X J , GAO Z H , LAUSEN L , et al . Deep learning for precipitation nowcasting:a benchmark and a new model [C ] // Proceedings of the 31st International Conference on Neural Information Processing Systems . New York:ACM , 2017 : 5622 - 5632 .

DING W C , CHEN J , SHEN S J . Predicting vehicle behaviors over an extended horizon using behavior interaction network [C ] // Proceedings of 2019 International Conference on Robotics and Automation (ICRA) . Piscataway:IEEE Press , 2019 : 8634 - 8640 .

MERCAT J , GILLES T , EL ZOGHBY N , et al . Multi-head attention for multi-modal joint vehicle motion forecasting [C ] // Proceedings of 2020 IEEE International Conference on Robotics and Automation (ICRA) . Piscataway:IEEE Press , 2020 : 9638 - 9644 .

ZHANG K P , ZHAO L , DONG C X , et al . AI-TP:attention-based interaction-aware trajectory prediction for autonomous driving [J ] . IEEE Transactions on Intelligent Vehicles , 2023 , 8 ( 1 ): 73 - 83 .

TANG C , ZHAN W , TOMIZUKA M . Exploring social posterior collapse in variational autoencoder for interaction modeling [J ] . Advances in Neural Information Processing Systems , 2021 , 34 : 8481 - 8494 .

DEVLIN J , CHANG M W , LEE K , et al . Bert:pre-training of deep bidirectional transformers for language understanding [J ] . arXiv preprint , 2018 ,arXiv:1810.04805.

LEE N , CHOI W , VERNAZA P , et al . DESIRE:distant future prediction in dynamic scenes with interacting agents [C ] // Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway:IEEE Press , 2017 : 2165 - 2174 .

ZHAO H , GAO J , LAN T , et al . Tnt:target-driven trajectory prediction [C ] // Proceedings of Conference on Robot Learning . New York:PMLR , 2021 : 895 - 904 .

LIU M , CHENG H , CHEN L , et al . LAformer:trajectory prediction for autonomous driving with lane-aware scene constraints [J ] . arXiv preprint , 2023 ,arXiv:2302.13933.

YUAN Y , WENG X S , OU Y L , et al . AgentFormer:agent-aware transformers for socio-temporal multi-agent forecasting [C ] // Proceedings of 2021 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway:IEEE Press , 2021 : 9793 - 9803 .

PINI S , PERONE C S , AHUJA A , et al . Safe Real-World Autonomous Driving by Learning to Predict and Plan with a Mixture of Experts [J ] . arXiv preprint , 2022 ,arXiv:2211.02131.

CHEN W , WANG F , SUN H . S2tnet:spatio-temporal transformer networks for trajectory prediction in autonomous driving [C ] // Proceedings of Asian Conference on Machine Learning . New York:PMLR , 2021 : 454 - 469 .

BHAT M , FRANCIS J , OH J . Trajformer:trajectory prediction with local self-attentive contexts for autonomous driving [J ] . arXiv preprint , 2020 ,arXiv.2011.14910.

ZHAO J , LI X , XUE Q , et al . Spatial-channel transformer network for trajectory prediction on the traffic scenes [J ] . arXiv preprint , 2021 ,arXiv:2101.11472.

HU J , SHEN L , SUN G . Squeeze-and-excitation networks [C ] // Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2018 : 7132 - 7141 .

HUANG Z Y , MO X Y , LV C . Multi-modal motion prediction with transformer-based neural network for autonomous driving [C ] // Proceedings of 2022 International Conference on Robotics and Automation (ICRA) . Piscataway:IEEE Press , 2022 : 2605 - 2611 .

ZHANG K P , FENG X L , WU L , et al . Trajectory prediction for autonomous driving using spatial-temporal graph attention transformer [J ] . IEEE Transactions on Intelligent Transportation Systems , 2022 , 23 ( 11 ): 22343 - 22353 .

LIU Z , LIN Y T , CAO Y , et al . Swin transformer:hierarchical vision transformer using shifted windows [C ] // Proceedings of 2021 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway:IEEE Press , 2021 : 9992 - 10002 .

NAYAKANTI N , AL-RFOU R , ZHOU A , et al . Wayformer:motion forecasting via simple ＆ efficient attention networks [J ] . arXiv preprint , 2022 ,arXiv:2207.05844.

CHAMI I , ABU-EL-HAIJA S , PEROZZI B , et al . Machine learning on graphs:A model and comprehensive taxonomy [J ] . Journal of Machine Learning Research , 2022 , 23 ( 89 ): 1 - 64 .

LIANG M , YANG B , HU R , et al . Learning Lane Graph Representations for Motion Forecasting [C ] // Proceedings of European Conference on Computer Vision . Cham:Springer , 2020 : 541 - 556 .

BEHBAHANI F , SHIARLIS K , CHEN X , et al . Learning from demonstration in the wild [C ] // Proceedings of 2019 International Conference on Robotics and Automation (ICRA) . Piscataway:IEEE Press , 2019 : 775 - 781 .

CASAS S , GULINO C , LIAO R J , et al . SpAGNN:spatially-aware graph neural networks for relational behavior forecasting from sensor data [C ] // Proceedings of 2020 IEEE International Conference on Robotics and Automation (ICRA) . Piscataway:IEEE Press , 2020 : 9491 - 9497 .

RHINEHART N , MCALLISTER R , KITANI K , et al . PRECOG:PREdiction conditioned on goals in visual multi-agent settings [C ] // Proceedings of 2019 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway:IEEE Press , 2020 : 2821 - 2830 .

CASAS S , GULINO C , SUO S , et al . Implicit Latent Variable Model for Scene-Consistent Motion Forecasting [C ] // Proceeding of European Conference on Computer Vision . Cham:Springer , 2020 : 624 - 641 .

LI X , YING X W , CHUAH M C . GRIP:graph-based interactionaware trajectory prediction [C ] // Proceedings of 2019 IEEE Intelligent Transportation Systems Conference (ITSC) . Piscataway:IEEE Press , 2019 : 3960 - 3966 .

LI X , YING X , CHUAH M C . Grip++:enhanced graph-based interaction-aware trajectory prediction for autonomous driving [J ] . arXiv preprint , 2019 ,arXiv:1907.07792.

CHANDRA R , GUAN T R , PANUGANTI S , et al . Forecasting trajectory and behavior of road-agents using spectral clustering in graphLSTMs [J ] . IEEE Robotics and Automation Letters , 2020 , 5 ( 3 ): 4882 - 4890 .

GAO J Y , SUN C , ZHAO H , et al . VectorNet:encoding HD maps and agent dynamics from vectorized representation [C ] // Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway:IEEE Press , 2020 : 11522 - 11530 .

GU J R , SUN C , ZHAO H . DenseTNT:end-to-end trajectory prediction from dense goal sets [C ] // Proceedings of 2021 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway:IEEE Press , 2021 : 15283 - 15292 .

CHAI Y , SAPP B , BANSAL M , et al . Multipath:multiple probabilistic anchor trajectory hypotheses for behavior prediction [J ] . arXiv preprint , 2019 ,arXiv:1910.05449.

HUANG Y F , BI H K , LI Z X , et al . STGAT:modeling spatialtemporal interactions for human trajectory prediction [C ] // Proceedings of 2019 IEEE/CVF International Conference on Computer Vision (ICCV) . Piscataway:IEEE Press , 2019 : 6271 - 6280 .

GUPTA A , JOHNSON J , LI F F , et al . Social GAN:socially acceptable trajectories with generative adversarial networks [C ] // Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2018 : 2255 - 2264 .

LI J C , MA H B , TOMIZUKA M . Conditional generative neural system for probabilistic trajectory prediction [C ] // Proceedings of 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . Piscataway:IEEE Press , 2019 : 6150 - 6156 .

YANG B , HE C , WANG P , et al . TPPO:a novel trajectory predictor with pseudo oracle [J ] . arXiv preprint , 2020 ,arXiv:2002.01852.

SADEGHIAN A , KOSARAJU V , SADEGHIAN A , et al . SoPhie:an attentive GAN for predicting paths compliant to social and physical constraints [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway:IEEE Press , 2019 : 1349 - 1358 .

LIU S H , LIU H B , BI H K , et al . CoL-GAN:plausible and collisionless trajectory prediction by attention-based GAN [J ] . IEEE Access , 2020 , 8 : 101662 - 101671 .

CHOI S , KWEON N , YANG C , et al . DSA-GAN:driving style attention generative adversarial network for vehicle trajectory prediction [C ] // Proceedings of 2021 IEEE International Intelligent Transportation Systems Conference (ITSC) . Piscataway:IEEE Press , 2021 : 1515 - 1520 .

KINGMA D P , WELLING M . Auto-encoding variational bayes [J ] . arXiv preprint , 2013 ,arXiv:1312.6114.

BALDI P . Autoencoders,unsupervised learning and deep architectures [C ] // Proceedings of the 2011 International Conference on Unsupervised and Transfer Learning workshop . New York:ACM , 2011 : 37 - 50 .

GÓMEZ-HUÉLAMO C , CONDE M V , ORTIZ M , et al . Exploring attention GAN for vehicle motion prediction [C ] // Proceedings of 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC) . Piscataway:IEEE Press , 2022 : 4011 - 4016 .

WANG W , LIU C , ZHAO D . How much data are enough? A statistical approach with case study on longitudinal driving behavior [J ] . IEEE Transactions on Intelligent Vehicles , 2017 , 2 ( 2 ): 85 - 98 .

王晓 , 杨林瑶 , 胡斌 , 等 . 平行推理:一种基于ACP方法的虚实互动的知识协同框架 [J ] . 智能科学与技术学报 , 2023 , 5 ( 1 ): 69 - 82 .

WANG X , YANG L Y , HU B , et al . Parallel reasoning:a virtual-real interactive knowledge collaboration framework based on ACP approach [J ] . Chinese Journal of Intelligent Science and Technology , 2023 , 5 ( 1 ): 69 - 82 .

田永林 , 陈苑文 , 杨静 , 等 . 元宇宙与平行系统:发展现状、对比及展望 [J ] . 智能科学与技术学报 , 2023 , 5 ( 1 ): 121 - 132 .

TIAN Y L , CHEN Y W , YANG J , et al . Metaverses and parallel systems:the state of the art,comparisons and prospects [J ] . Chinese Journal of Intelligent Science and Technology , 2023 , 5 ( 1 ): 121 - 132 .

MIAO Q H , LV Y S , HUANG M , et al . Parallel learning:overview and perspective for computational learning across Syn2Real and Sim2Real [J ] . IEEE/CAA Journal of Automatica Sinica , 2023 , 10 ( 3 ): 603 - 631 .

Views

1285

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

A GAN generation method of sound effect using only a single short audio for training

A lip reading method based on adaptive pooling attention Transformer

Dual-stage gated segmented multimodal emotion recognition method

Survey on multi-agent reinforcement learning methods from the perspective of population

Review of pedestrian trajectory prediction methods

Related Author

HONG Zijing

MIAO Xiangyang

JIANG Lin

WANG Xiao

DENG Tao

HU Zhenxiao

YAO Yun

XU Guangxian

Related Institution

Xiangjiang Laboratory

School of Artificial Intelligence and Advanced Computing, Hunan University of Technology and Business

School of Intelligent Engineering and Intelligent Manufacturing, Hunan University of Technology and Business

School of Electronic and Information Engineering, Liaoning Technical University

School of Electrical and Control Engineering, Liaoning Technical University

Postal code：100079
Tel：（010）53879206 Email：cjist@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：162003 Visits Today：166

⁰