机器人持续学习进展与展望

赵超; 许杰; 陈星宇; 梅魁志; 兰旭光

doi:10.11959/j.issn.2096-6652.202235

您当前的位置：

首页 >

文章列表页 >

机器人持续学习进展与展望

综述与展望 | 更新时间：2024-06-05

- 机器人持续学习进展与展望
- A review of continual learning for robotics
- 智能科学与技术学报 2022年4卷第3期页码：308-323
- 作者机构：
  
  1. 西安交通大学人工智能与机器人研究所，陕西西安 710049
  2. 西安交通大学人工智能学院，陕西西安 710049
- 作者简介：
  
  [ "赵超（1996- ），男，西安交通大学人工智能与机器人研究所博士生，主要研究方向为机器人持续学习" ]
  [ "许杰（1996- ），男，西安交通大学人工智能与机器人研究所博士生，主要研究方向为人机交互、人体运动预测" ]
  [ "陈星宇（1990- ），男，博士，西安交通大学人工智能与机器人研究所助理教授，主要研究方向为计算机视觉、机器人环境感知、人-机器人共融协作" ]
  [ "梅魁志（1978- ），男，博士，西安交通大学人工智能与机器人研究所教授、博士生导师，主要研究方向为机器人场景感知与定位、深度学习/智能系统的软硬件协同设计" ]
  [ "兰旭光（1976- ），男，博士，西安交通大学人工智能与机器人研究所教授、博士生导师，主要研究方向计算机视觉、统计学习与模式识别、深度强化学习与机器人学习、人-机器人共融协作" ]
- 基金信息：
  
  国家自然科学基金资助项目;The National Natural Science Foundation of China(62125305);国家自然科学基金资助项目;The National Natural Science Foundation of China(62076193);国家自然科学基金资助项目;The National Natural Science Foundation of China(62088102);科技创新2030—“新一代人工智能”重大项目;The National Key Research and Development Program of China(2021ZD0112700);中央高校基础研究基金资助项目;The Fundamental Research Funds for the Central Universities(xtr072022001)
- DOI：10.11959/j.issn.2096-6652.202235
  中图分类号： TP242.6
- 网络出版日期：2022-09，
  
  纸质出版日期：2022-09-15
- 稿件说明：
移动端阅览
赵超, 许杰, 陈星宇, 等. 机器人持续学习进展与展望[J]. 智能科学与技术学报, 2022,4(3):308-323.

Chao ZHAO, Jie XU, Xingyu CHEN, et al. A review of continual learning for robotics[J]. Chinese journal of intelligent science and technology, 2022, 4(3): 308-323.
赵超, 许杰, 陈星宇, 等. 机器人持续学习进展与展望[J]. 智能科学与技术学报, 2022,4(3):308-323. DOI： 10.11959/j.issn.2096-6652.202235.

Chao ZHAO, Jie XU, Xingyu CHEN, et al. A review of continual learning for robotics[J]. Chinese journal of intelligent science and technology, 2022, 4(3): 308-323. DOI： 10.11959/j.issn.2096-6652.202235.

摘要

当前机器人技术面临的较大限制之一是难以适应不断变化的任务，当机器人面对新环境或者学习新任务时，会不可避免地遗忘旧环境或旧任务的经验。为了总结机器人持续学习的研究和发展现状，首先介绍了持续学习的框架和评价基准，然后阐述了持续学习在机器人任务中的必要性和面临的挑战，并对持续学习的发展现状进行了回顾，最后展望了机器人持续学习的发展前景，提出了一些有价值的研究问题。

Abstract

One of the limitations of robotics is that it is difficult for robots to adapt to fickle tasks.A robot will inevitably forget the knowledge from old environments or tasks when facing new environments or tasks.In order to summarize research in continual learning for robotics

firstly

the framework and evaluation protocols in continual learning were introduced.And then necessity and challenge of continual learning in the robotics were expounded.The research for continual learning was also summarized.Finally

the prospect of continual learning was predicted and some valuable research directions were put forward.

关键词

Keywords

references

RING M B . Continual learning in reinforcement environments [M ] . Austin : University of Texas , 1994 .

TJOMSLAND J , KALKAN S , GUNES H . Mind your manners! A dataset and a continual learning approach for assessing social appropriateness of robot actions [J ] . Frontiers in Robotics and AI , 2022 ,9:669420.

GROSSMANN A . Continual learning for mobile robots [D ] . Birmingham:University of Birmingham , 2001 .

NEHMZOW U , RIANO L . A proposal for continual learning in robotics [C ] // Proceedings of Towards Autonomous Robotic Systems .[S.l.:s.n. ] , 2009 .

LIU B , XIAO X S , STONE P . A lifelong learning approach to mobile robot navigation [J ] . IEEE Robotics and Automation Letters , 2021 , 6 ( 2 ): 1090 - 1096 .

HADSELL R , RAO D , RUSU A A , et al . Embracing change:continual learning in deep neural networks [J ] . Trends in Cognitive Sciences , 2020 , 24 ( 12 ): 1028 - 1040 .

PARISI G I , KEMKER R , PART J L , et al . Continual lifelong learning with neural networks:a review [J ] . Neural Networks , 2019 , 113 : 54 - 71 .

DE LANGE M , ALJUNDI R , MASANA M , et al . A continual learning survey:defying forgetting in classification tasks [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2022 , 44 ( 7 ): 3366 - 3385 .

VERWIMP E , DE LANGE M , TUYTELAARS T . Rehearsal revealed:the limits and merits of revisiting samples in continual learning [C ] // Proceedings of 2021 IEEE/CVF International Conference on Computer Vision . Piscataway:IEEE Press , 2021 : 9365 - 9374 .

LESORT T , LOMONACO V , STOIAN A , et al . Continual learning for robotics:definition,framework,learning strategies,opportunities and challenges [J ] . Information Fusion , 2020 , 58 : 52 - 68 .

LIU Y Y , SCHIELE B , SUN Q R . Adaptive aggregation networks for class-incremental learning [C ] // Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2021 : 2544 - 2553 .

ZHAI M Y , CHEN L , HE J W , et al . Piggyback GAN:efficient lifelong learning for image conditioned generation [M ] // Computer vision -ECCV 2020 . Cham : Springer International Publishing , 2020 : 397 - 413 .

MITCHELL T , COHEN W , HRUSCHKA E , et al . Never-ending learning [J ] . Communications of the ACM , 2018 , 61 ( 5 ): 103 - 115 .

MCCLOSKEY M , COHEN N J . Catastrophic interference in connectionist networks:the sequential learning problem [J ] . Psychology of Learning and Motivation , 1989 , 24 : 109 - 165 .

TRAORÉ R , CASELLES-DUPRÉ H , LESORT T , et al . Continual reinforcement learning deployed in real-life using policy distillation and Sim2Real transfer [C ] // Proceedings of ICML Workshop on Multi-Task and Lifelong Reinforcement Learning .[S.l.:s.n. ] , 2019 .

ZHANG Y , YANG Q . A survey on multi-task learning [J ] . IEEE Transactions on Knowledge and Data Engineering , 2021 .

PAN S J , YANG Q . A survey on transfer learning [J ] . IEEE Transactions on Knowledge and Data Engineering , 2010 , 22 ( 10 ): 1345 - 1359 .

HOI S C H , SAHOO D , LU J , et al . Online learning:a comprehensive survey [J ] . Neurocomputing , 2021 , 459 : 249 - 289 .

GOODFELLOW I J , MIRZA M , XIAO D , et al . An empirical investigation of catastrophic forgetting in gradient-based neural networks [C ] // Proceedings of the 2nd International Conference on Learning Representations .[S.l.:s.n. ] , 2014 .

LOPEZ-PAZ D , RANZATO M . Gradient episodic memory for continual learning [J ] . arXiv preprint,2017,arXiv:1706.08840 .

REBUFFI S A , KOLESNIKOV A , SPERL G , et al . iCaRL:incremental classifier and representation learning [C ] // Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2017 : 5533 - 5542 .

SAENKO K , KULIS B , FRITZ M , et al . Adapting visual category models to new domains [J ] . Lecture Notes in Computer Science , 2010 : 213 - 226 .

LECUN Y , BOTTOU L , BENGIO Y , et al . Gradient-based learning applied to document recognition [J ] . Proceedings of the IEEE , 1998 , 86 ( 11 ): 2278 - 2324 .

GANIN Y , LEMPITSKY V . Unsupervised domain adaptation by backpropagation [C ] // Proceedings of the 32nd International Conference on Machine Learning .[S.l.:s.n. ] , 2015 : 1180 - 1189 .

HULL J J . A database for handwritten text recognition research [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 1994 , 16 ( 5 ): 550 - 554 .

NETZER Y , WANG T . Reading digits in natural images with unsupervised feature learning [J ] . Advances in Neural Information Processing Systems , 2011 .

BELLEMARE M G , NADDAF Y , VENESS J , et al . The arcade learning environment:an evaluation platform for general agents [J ] . Journal of Artificial Intelligence Research , 2013 , 47 : 253 - 279 .

JUNG S , AHN H , CHA S , et al . Continual learning with node-importance based adaptive group sparse regularization [J ] . Advances in Neural Information Processing Systems , 2020 , 33 : 3647 - 3658 .

RUSU A A , RABINOWITZ N C , DESJARDINS G , et al . Progressive neural networks [J ] . arXiv preprint,2016,arXiv:1606.04671 .

WANG L Y , ZHANG M T , JIA Z F , et al . AFEC:active forgetting of negative transfer in continual learning [J ] . arXiv preprint,2021,arXiv:2110.12187 .

MENDEZ J A , WANG B Y , EATON E . Lifelong policy gradient learning of factored policies for faster training without forgetting [J ] . Advances in Neural Information Processing Systems , 2020 .

XU K , VERMA S , FINN C , et al . Continual learning of control primitives:skill discovery via reset-games [J ] . Advances in Neural Information Processing Systems , 2020 .

YANG F , YANG C , LIU H , et al . Evaluations of the gap between supervised and reinforcement lifelong learning on robotic manipulation tasks [C ] // Proceedings of the Conference on Robot Learning .[S.l.:s.n. ] , 2022 : 547 - 556 .

TODOROV E , EREZ T , TASSA Y . MuJoCo:a physics engine for model-based control [C ] // Proceedings of 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems . Piscataway:IEEE Press , 2012 : 5026 - 5033 .

BROCKMAN G , CHEUNG V , PETTERSSON L , et al . OpenAI Gym [J ] . arXiv preprint,2016,arXiv:1606.01540 .

JULIAN R , SWANSON B , SUKHATME G S , et al . Never stop learning:the effectiveness of fine-tuning in robotic reinforcement learning [J ] . arXiv preprint,2020,arXiv:2004.10190 .

DÍAZ-RODRÍGUEZ N , LOMONACO V , FILLIAT D , et al . Don’t forget,there is more than forgetting:new metrics for continual learning [J ] . arXiv preprint,2018,arXiv:1810.13166 .

CHAUDHRY A , DOKANIA P K , AJANTHAN T , et al . Riemannian walk for incremental learning:understanding forgetting and intransigence [C ] // Proceedings of the European Conference on Computer Vision .[S.l.:s.n. ] , 2018 : 532 - 547 .

YIN H , YANG P , LI P . Mitigating forgetting in online continual learning with neuron calibration [J ] . Advances in Neural Information Processing Systems , 2021 ,34.

WOŁCZYK M , ZAJĄC M , PASCANU R , et al . Continual world:a robotic benchmark for continual reinforcement learning [J ] . Advances in Neural Information Processing Systems , 2021 .

JIANG Y , BHARADWAJ S , WU B , et al . Temporal-logic-based reward shaping for continuing learning tasks [J ] . arXiv preprint,2020,arXiv:2007.01498 .

KEMKER R , MCCLURE M , ABITINO A , et al . Measuring catastrophic forgetting in neural networks [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2018 , 32 ( 1 ): 3390 - 3398 .

RIEMER M , CASES I , AJEMIAN R , et al . Learning to learn without forgetting by maximizing transfer and minimizing interference [C ] // Proceedings of the 7th International Conference on Learning Representations .[S.l.:s.n. ] , 2019 .

HENNING C , CERVERA M , D’ANGELO F , , et al . Posterior meta-replay for continual learning [J ] . Advances in Neural Information Processing Systems , 2021 ,34.

DONG J , CONG Y , SUN G , et al . I3DOL:Incremental 3D object learning without catastrophic forgetting [C ] // Proceedings of the AAAI Conference on Artificial Intelligence .[S.l.:s.n. ] , 2021 : 6066 - 6074 .

ZHOU M , XIAO J , CHANG Y F , et al . Image de-raining via continual learning [C ] // Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2021 : 4905 - 4914 .

CHERAGHIAN A , RAHMAN S , RAMASINGHE S , et al . Synthesized feature based few-shot class-incremental learning on a mixture of subspaces [C ] // Proceedings of 2021 IEEE/CVF International Conference on Computer Vision . Piscataway:IEEE Press , 2021 : 8641 - 8650 .

KUKLEVA A , KUEHNE H , SCHIELE B . Generalized and incremental few-shot learning by explicit learning and calibration without forgetting [C ] // Proceedings of 2021 IEEE/CVF International Conference on Computer Vision . Piscataway:IEEE Press , 2021 : 9000 - 9009 .

LOO N , SWAROOP S , TURNER R E . generalized variational continual learning [C ] // Proceedings of the International Conference on Learning Representations .[S.l.:s.n. ] , 2020 .

GORUR D . Linear mode connectivity in multitask and continual learning [J ] . arXiv preprint,2020,arXiv:2010.04495 .

KIRKPATRICK J , PASCANU R , RABINOWITZ N , et al . Overcoming catastrophic forgetting in neural networks [J ] . Proceedings of the National Academy of Sciences of the United States of America , 2017 , 114 ( 13 ): 3521 - 3526 .

VERMA V K , LIANG K J , MEHTA N , et al . Efficient feature transformations for discriminative and generative continual learning [J ] . arXiv preprint,2021,arXiv:2103.13558 .

CHA S , HSU H , HWANG T , et al . CPR:classifier-projection regularization for continual learning [C ] // Proceedings of the International Conference on Learning Representations .[S.l.:s.n. ] , 2020 .

HINTON G , VINYALS O , DEAN J . Distilling the knowledge in a neural network [J ] . arXiv preprint,2015,arXiv:1503.02531 .

ZHAI M Y , CHEN L , TUNG F , et al . Lifelong GAN:continual learning for conditional image generation [C ] // Proceedings of 2019 IEEE/CVF International Conference on Computer Vision . Piscataway:IEEE Press , 2019 : 2759 - 2768 .

TAO X Y , HONG X P , CHANG X Y , et al . Few-shot class-incremental learning [C ] // Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2020 : 12180 - 12189 .

MENG Q E , SHIN'ICHI S ,, . ADINet:attribute driven incremental network for retinal image classification [C ] // Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2020 : 4032 - 4041 .

FINI E , LATHUILIÈRE S , SANGINETO E , et al . Online continual learning under extreme memory constraints [J ] . Lecture Notes in Computer Science , 2020 : 720 - 735 .

SHMELKOV K , SCHMID C , ALAHARI K . Incremental learning of object detectors without catastrophic forgetting [C ] // Proceedings of 2017 IEEE International Conference on Computer Vision.Piscataway:IEEE Press .[S.l.:s.n. ] , 2017 : 3420 - 3429 .

SMITH J , HSU Y C , BALLOCH J , et al . Always be dreaming:a new approach for data-free class-incremental learning [C ] // Proceedings of 2021 IEEE/CVF International Conference on Computer Vision . Piscataway:IEEE Press , 2021 : 9354 - 9364 .

CASTRO F M , MARÍN-JIMÉNEZ M J , GUIL N , et al . End-to-end incremental learning [J ] . Lecture Notes in Computer Science , 2018 : 241 - 257 .

KIM J Y , CHOI D W . Split-and-bridge:adaptable class incremental learning within a single neural network [J ] . arXiv preprint,2021,arXiv:2107.01349 .

DENG D R , CHEN G Y , HAO J Y , et al . Flattening sharpness for dynamic gradient projection memory benefits continual learning [J ] . arXiv preprint,2021,arXiv:2110.04593 .

TANG S X , CHEN D P , ZHU J G , et al . Layerwise optimization by gradient decomposition for continual learning [C ] // Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2021 : 9629 - 9638 .

SAHA G , GARG I , ROY K . Gradient projection memory for continual learning [J ] . arXiv preprint,2021,arXiv:2103.09762 .

JIN X S , DU J Y , REN X . Gradient based memory editing for task-free continual learning [J ] . arXiv preprint,2020,arXiv:2006.15294 .

CAI Z P , SENER O , KOLTUN V . Online continual learning with natural distribution shifts:an empirical study with visual data [C ] // Proceedings of 2021 IEEE/CVF International Conference on Computer Vision . Piscataway:IEEE Press , 2021 : 8261 - 8270 .

CHAUDHRY A , KHAN N , DOKANIA P K , et al . Continual learning in low-rank orthogonal subspaces [J ] . Advances in Neural Information Processing Systems , 2020 : 1 - 12 .

LIN S , YANG L , FAN D L , et al . TRGP:trust region gradient projection for continual learning [J ] . arXiv preprint,2022,arXiv:2202.02931 .

ALJUNDI R , KELCHTERMANS K , TUYTELAARS T . Task-free continual learning [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2019 : 11246 - 11255 .

KIM C D , JEONG J , MOON S , et al . Continual learning on noisy data streams via self-purified replay [C ] // Proceedings of 2021 IEEE/CVF International Conference on Computer Vision . Piscataway:IEEE Press , 2021 : 517 - 527 .

CHA H , LEE J , SHIN J . Co2L:contrastive continual learning [C ] // Proceedings of 2021 IEEE/CVF International Conference on Computer Vision . Piscataway:IEEE Press , 2021 : 9496 - 9505 .

KUZINA A , EGOROV E , BURNAEV E . BooVAE:boosting approach for continual learning of VAE [J ] . Advances in Neural Information Processing Systems , 2021 .

MARACANI A , MICHIELI U , TOLDO M , et al . RECALL:replay-based continual learning in semantic segmentation [C ] // Proceedings of 2021 IEEE/CVF International Conference on Computer Vision . Piscataway:IEEE Press , 2021 : 7006 - 7015 .

AYUB A , WAGNER A R . EEC:learning to encode and regenerate images for continual learning [J ] . arXiv preprint,2021,arXiv:2101.04904 .

ROSTAMI M , KOLOURI S , PILLY P , et al . Generative continual concept learning [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2020 , 34 ( 4 ): 5545 - 5552 .

SHIN H , LEE J K , KIM J , et al . Continual learning with deep generative replay [J ] . arXiv preprint,2017,arXiv:1705.08690 .

OSTAPENKO O , PUSCAS M , KLEIN T , et al . Learning to remember:a synaptic plasticity driven framework for continual learning [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2019 : 11313 - 11321 .

HU X T , TANG K H , MIAO C Y , et al . Distilling causal effect of data in class-incremental learning [C ] // Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2021 : 3956 - 3965 .

LI T J , KE Q H , RAHMANI H , et al . Else-Net:elastic semantic network for continual action recognition from skeleton data [C ] // Proceedings of 2021 IEEE/CVF International Conference on Computer Vision . Piscataway:IEEE Press , 2021 : 13414 - 13423 .

SINGH P , VERMA V K , MAZUMDER P , et al . Calibrating CNNs for lifelong learning [J ] . Advances in Neural Information Processing Systems , 2020 .

PELLEGRINI L , GRAFFIETI G , LOMONACO V , et al . Latent replay for real-time continual learning [C ] // Proceedings of 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems . Piscataway:IEEE Press , 2020 : 10203 - 10209 .

KUNDU J N , VENKATESH R M , VENKAT N , et al . Class-incremental domain adaptation [M ] // Computer vision - ECCV 2020 . Cham : Springer International Publishing , 2020 : 53 - 69 .

WEI K , DENG C , YANG X , et al . Incremental embedding learning via zero-shot translation [C ] // Proceedings of the AAAI Conference on Artificial Intelligence .[S.l.:s.n. ] , 2021 : 10254 - 10262 .

STAN S , ROSTAMI M . Unsupervised model adaptation for continual semantic segmentation [C ] // Proceedings of the AAAI Conference on Artificial Intelligence .[S.l.:s.n. ] , 2021 : 2593 - 2601 .

ZHU F , ZHANG X Y , WANG C , et al . Prototype augmentation and self-supervision for incremental learning [C ] // Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2021 : 5867 - 5876 .

DE LANGE M , TUYTELAARS T . Continual prototype evolution:learning online from non-stationary data streams [C ] // Proceedings of 2021 IEEE/CVF International Conference on Computer Vision . Piscataway:IEEE Press , 2021 : 8230 - 8239 .

OLIVERI G , VAN LAAKE L C , CARISSIMO C , et al . Continuous learning of emergent behavior in robotic matter [J ] . Proceedings of the National Academy of Sciences of the United States of America , 2021 , 118 ( 21 ): e2017015118 .

YAN S P , XIE J W , HE X M . DER:dynamically expandable representation for class incremental learning [C ] // Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2021 : 3013 - 3022 .

VENIAT T , DENOYER L , RANZATO M . Efficient continual learning with modular networks and task-driven priors [C ] // Proceedings of the 9th International Conference on Learning Representations .[S.l.:s.n. ] , 2021 .

XU J , MA J , GAO X , et al . Adaptive progressive continual learning [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2021 .

CHEN T , ZHENYU Z , SIJIA L , et al . Long live the lottery:the existence of winning tickets in lifelong learning [J ] . International Conference on Learning Representations , 2021 , 51 ( 4 ): 273 - 287 .

SHI Y J , YUAN L , CHEN Y P , et al . Continual learning via bit-level information preserving [C ] // Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2021 : 16669 - 16678 .

MAZUMDER P , SINGH P , RAI P . Few-shot lifelong learning [J ] . arXiv preprint,2021,arXiv:2103.00991 .

CACCIA M , RODRIGUEZ P , OSTAPENKO O , et al . Online fast adaptation and knowledge accumulation:a new approach to continual learning [J ] . Advances in Neural Information Processing Systems , 2020 .

HURTADO J , RAYMOND-SAEZ A , SOTO A . Optimizing reusable knowledge for continual learning via metalearning [J ] . Advances in Neural Information Processing Systems , 2021 .

VON OSWALD J , ZHAO D , KOBAYASHI S , et al . Learning where to learn:gradient sparsity in meta and continual learning [J ] . arXiv preprint,2021,arXiv:2110.14402 .

GUPTA G , YADAV K , PAULL L . La-MAML:look-ahead meta learning for continual learning [C ] // Proceedings of the Advances in Neural Information Processing Systems .[S.l.:s.n. ] , 2020 .

VOLPI R , LARLUS D , ROGEZ G . Continual adaptation of visual representations via domain randomization and meta-learning [C ] // Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognitio . Piscataway:IEEE Press , 2021 : 4441 - 4451 .

LUO Y D , HUANG Z , ZHANG Z , et al . Learning from the past:continual meta-learning with Bayesian graph neural networks [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2020 , 34 ( 4 ): 5021 - 5028 .

ZHANG J , ZHANG J T , GHOSH S , et al . Regularize,expand and compress:nonexpansive continual learning [C ] // Proceedings of 2020 IEEE Winter Conference on Applications of Computer Vision . Piscataway:IEEE Press , 2020 : 843 - 851 .

EBRAHIMI S , ELHOSEINY M , DARRELL T , et al . Uncertain ty-guided continual learning with Bayesian neural networks [J ] . IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops , 2019 : 75 - 78 .

RAVAGLIA L , RUSCI M , NADALINI D , et al . A TinyML platform for on-device continual learning with quantized latent replays [J ] . IEEE Journal on Emerging and Selected Topics in Circuits and Systems , 2021 , 11 ( 4 ): 789 - 802 .

PUTRA R V W , SHAFIQUE M . lpSpikeCon:enabling low-precision spiking neural network processing for efficient unsupervised continual learning on autonomous agents [J ] . arXiv preprint,2022,arXiv:2205.12295 .

BURHAN HAFEZ M , WERMTER S . Behavior self-organization supports task inference for continual robot learning [C ] // Proceedings of 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems . Piscataway:IEEE Press , 2021 : 6739 - 6746 .

XU J , ZHU Z . Reinforced continual learning [J ] . Advances in Neural Information Processing Systems , 2018 : 899 - 908 .

LECARPENTIER E , ABEL D , ASADI K , et al . Lipschitz lifelong reinforcement learning [C ] // Proceedings of the AAAI Conference on Artificial Intelligenc .[S.l.:s.n. ] , 2021 : 8270 - 8278 .

BRAFMAN R I , TENNENHOLTZ M . R-MAX - a general polynomial time algorithm for near-optimal reinforcement learning [J ] . IJCAI International Joint Conference on Artificial Intelligence , 2001 , 3 : 953 - 958 .

ABEL D , JINNAI Y , YUE G , et al . Policy and value transfer in lifelong reinforcement learning [C ] // Proceedings of the 35th International Conference on Machine Learning .[S.l.:s.n. ] , 2018 : 30 - 41 .

LOMONACO V , DESAI K R , CULURCIELLO E , et al . Continual reinforcement learning in 3D non-stationary environments [C ] // Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops . Piscataway:IEEE Press , 2020 : 999 - 1008 .

CHANDAK Y , THEOCHAROUS G , NOTA C , et al . Lifelong learning with a changing action set [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2020 , 34 ( 4 ): 3373 - 3380 .

MNIH V , KAVUKCUOGLU K , SILVER D , et al . Human-level control through deep reinforcement learning [J ] . Nature , 2015 , 518 ( 7540 ): 529 - 533 .

ROLNICK D , AHUJA A , SCHWARZ J , et al . Experience replay for continual learning [J ] . arXiv preprint,2019,arXiv:1811.11682 .

LU K , GROVER A , ABBEEL P , et al . Reset-free lifelong Learning with skill-space planning [C ] // Proceedings of the International Conference on Learning Representations .[S.l.:s.n. ] , 2020 .

ALJUNDI R . Continual learning in neural networks [J ] . arXiv preprint,2019,arXiv:1910.02718 .

MORENO-MUÑOZ P , RAMÍREZ D , ARTÉS-RODRÍGUEZ A , . Continual learning for infinite hierarchical change-point detection [C ] // Proceedings of 2020 IEEE International Conference on Acoustics,Speech and Signal Processing . Piscataway:IEEE Press , 2020 : 3582 - 3586 .

BING L . Learning on the job:online lifelong and continual learning [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2020 , 34 ( 9 ): 13544 - 13549 .

FLESCH T , BALAGUER J , DEKKER R , et al . Comparing continual task learning in minds and machines [J ] . Proceedings of the National Academy of Sciences of the United States of America , 2018 , 115 ( 44 ): 10313 - 10322 .

BARRON H C . Neural inhibition for continual learning and memory [J ] . Current Opinion in Neurobiology , 2021 , 67 : 85 - 94 .

浏览量

831

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

暂无数据