动态环境下基于语义信息与几何约束的视觉SLAM系统

李嘉铭; 解明扬; 张民; 王从庆

doi:10.11959/j.issn.2096-6652.202342

您当前的位置：

首页 >

文章列表页 >

动态环境下基于语义信息与几何约束的视觉SLAM系统

学术论文 | 更新时间：2024-04-07

- 动态环境下基于语义信息与几何约束的视觉SLAM系统
- Visual SLAM based on semantic information and geometric constraints in dynamic environment
- 智能科学与技术学报 2023年5卷第4期页码：477-485
- 作者机构：
  
  南京航空航天大学自动化学院，江苏南京 211106
- 作者简介：
  
  [ "李嘉铭（1999- ），男，南京航空航天大学自动化学院硕士研究生，主要研究方向为移动机器人自主导航。" ]
  [ "解明扬（1988- ），男，博士，南京航空航天大学自动化学院副研究员，主要研究方向为智能机器人技术。" ]
  [ "张民（1973- ），男，博士，南京航空航天大学自动化学院副研究员，主要研究方向为多无人机/编队协同控制。" ]
  [ "王从庆（1960- ），男，博士，南京航空航天大学自动化学院教授，主要研究方向为智能机器人与智能控制。" ]
- 基金信息：
  
  启元实验室创新基金项目(S20210201102);南京航空航天大学科研与实践创新计划(xcxjh20220342)
- DOI：10.11959/j.issn.2096-6652.202342
  中图分类号： TP242.6
- 收稿：2023-08-14，
  
  修回：2023-10-11，
  
  纸质出版：2023-12-15
- 稿件说明：
移动端阅览
李嘉铭, 解明扬, 张民, 等. 动态环境下基于语义信息与几何约束的视觉SLAM系统[J]. 智能科学与技术学报, 2023,5(4):477-485.

LI Jiaming, XIE Mingyang, ZHANG Min, et al. Visual SLAM based on semantic information and geometric constraints in dynamic environment[J]. Chinese Journal of Intelligent Science and Technology, 2023, 5(4): 477-485.
李嘉铭, 解明扬, 张民, 等. 动态环境下基于语义信息与几何约束的视觉SLAM系统[J]. 智能科学与技术学报, 2023,5(4):477-485. DOI： 10.11959/j.issn.2096-6652.202342.

LI Jiaming, XIE Mingyang, ZHANG Min, et al. Visual SLAM based on semantic information and geometric constraints in dynamic environment[J]. Chinese Journal of Intelligent Science and Technology, 2023, 5(4): 477-485. DOI： 10.11959/j.issn.2096-6652.202342.

摘要

目前同时定位与地图构建技术多假设外部环境是静态的，忽略了动态目标对SLAM系统的影响，这一假设很大程度上影响了无人系统自主导航的定位精度与鲁棒性。针对这一问题，提出了一种结合目标检测语义信息与多视图极线几何约束的动态SLAM系统，根据运动概率来判别与剔除系统中的动态特征点，并在TUM数据集及实机平台上进行了算法性能测试。实验结果表明在高动态环境下，该系统相较于ORB-SLAM2在绝对轨迹误差上至少可减小94%以上，相对位移轨迹和旋转角度误差分别至少降低了41%和40%，表明该算法能够有效剔除动态特征点，提高了动态环境下视觉SLAM系统的定位精度和鲁棒性。

Abstract

Most existing visual SLAM systems assume that the external environment is static

ignoring the influence of dynamic objects on the SLAM system. This assumption largely affects the accuracy and robustness of autonomous navigation. To address this issue

a dynamic SLAM system was proposed

which combined semantic information based on object detection and geometric information from multi-view geometry constraints by defining and discriminating the dynamic feature points in the system based on the moving probability. Experiment results on the public TUM dataset and our robot in real environment showed that

when comparing with ORB-SLAM2

the absolute trajectory error could be reduced larger than 94%

and the average relative position and attitude errors were reduced at least 41% and 40%

respectively

in high dynamic environments. It means that the proposed SLAM system effectively removes dynamic feature points

thus improving the localization accuracy and robustness of the visual SLAM system within high dynamic environments.

关键词

Keywords

references

SMITH R C , CHEESEMAN P . On the representation and estimation of spatial uncertainty [J ] . The International Journal of Robotics Research , 1986 , 5 ( 4 ): 56 - 68 .

王霞 , 左一凡 . 视觉SLAM研究进展 [J ] . 智能系统学报 , 2020 , 15 ( 5 ): 825 - 834 .

WANG X , ZUO Y F . Advances in visual SLAM research [J ] . CAAI Transactions on Intelligent Systems , 2020 , 15 ( 5 ): 825 - 834 .

MUR-ARTAL R , MONTIEL J M M , TARDÓS J D . ORB-SLAM: a versatile and accurate monocular SLAM system [J ] . IEEE Transactions on Robotics , 2015 , 31 ( 5 ): 1147 - 1163 .

MUR-ARTAL R , TARDÓS J D . ORB-SLAM2: an open-source SLAM system for monocular, stereo, and RGB-D cameras [J ] . IEEE Transactions on Robotics , 2017 , 33 ( 5 ): 1255 - 1262 .

CAMPOS C , ELVIRA R , RODRÍGUEZ J J G , et al . ORB-SLAM3: an accurate open-source library for visual, visual-inertial, and multimap SLAM [J ] . IEEE Transactions on Robotics , 2021 , 37 ( 6 ): 1874 - 1890 .

LABBÉ M , MICHAUD F . Online global loop closure detection for large-scale multi-session graph-based SLAM [C ] // Proceedings of 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems . Piscataway : IEEE Press , 2014 : 2661 - 2666 .

CARUSO D , ENGEL J , CREMERS D . Large-scale direct SLAM for omnidirectional cameras [C ] // Proceedings of 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . Piscataway : IEEE Press , 2015 : 141 - 148 .

郑南宁 . 人工智能新时代 [J ] . 智能科学与技术学报 , 2019 , 1 ( 1 ): 1 - 3 .

ZHENG N N . The new era of artificial intelligence [J ] . Chinese Journal of Intelligent Science and Technology , 2019 , 1 ( 1 ): 1 - 3 .

YU C , LIU Z X , LIU X J , et al . DS-SLAM: a semantic visual SLAM towards dynamic environments [C ] // Proceedings of 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . Piscataway : IEEE Press , 2019 : 1168 - 1174 .

BADRINARAYANAN V , KENDALL A , CIPOLLA R . SegNet: a deep convolutional encoder-decoder architecture for image segmentation [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2017 , 39 ( 12 ): 2481 - 2495 .

BESCOS B , FÁCIL J M , CIVERA J , et al . DynaSLAM: tracking, mapping, and inpainting in dynamic scenes [J ] . IEEE Robotics and Automation Letters , 2018 , 3 ( 4 ): 4076 - 4083 .

HE K M , GKIOXARI G , DOLLÁR P , et al . Mask R-CNN [C ] // Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV) . Piscataway : IEEE Press , 2017 : 2980 - 2988 .

BESCOS B , CAMPOS C , TARDÓS J D , et al . DynaSLAM II: tightly-coupled multi-object tracking and SLAM [J ] . IEEE Robotics and Automation Letters , 2021 , 6 ( 3 ): 5191 - 5198 .

JI T T , WANG C , XIE L H . Towards real-time semantic RGB-D SLAM in dynamic environments [C ] // Proceedings of 2021 IEEE International Conference on Robotics and Automation (ICRA) . Piscataway : IEEE Press , 2021 : 11175 - 11181 .

YUAN X , CHEN S . SaD-SLAM: a visual SLAM based on semantic and depth information [C ] // Proceedings of 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . Piscataway : IEEE Press , 2020 : 4930 - 4935 .

LIU Y B , MIURA J . RDS-SLAM: real-time dynamic SLAM using semantic segmentation methods [J ] . IEEE Access , 2021 , 9 : 23772 - 23785 .

ZHONG F W , WANG S , ZHANG Z Q , et al . Detect-SLAM: making object detection and SLAM mutually beneficial [C ] // Proceedings of 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) . Piscataway : IEEE Press , 2018 : 1001 - 1010 .

LIU W , ANGUELOV D , ERHAN D , et al . SSD: single shot MultiBox detector [M ] // Computer Vision-ECCV 2016 . Cham : Springer International Publishing , 2016 : 21 - 37 .

SOARES J C V , GATTASS M , MEGGIOLARO M A . Crowd-SLAM: visual SLAM towards crowded environments using object detection [J ] . Journal of Intelligent & Robotic Systems , 2021 , 102 ( 2 ): 1 - 16 .

REDMON J, FARHADI A. YOLOv3: an incremental improvement[EB].2018, arXiv preprint, arXiv:1804.02767.

ZHANG C Y , HUANG T , ZHANG R C , et al . PLD-SLAM: a new RGB-D SLAM method with point and line features for indoor dynamic scene [J ] . ISPRS International Journal of Geo-Information , 2021 , 10 ( 3 ): 163 .

WANG Y N , XU K , TIAN Y B , et al . DRG-SLAM: a semantic RGB-D SLAM using geometric features for indoor dynamic scene [C ] // Proceedings of 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . Piscataway : IEEE Press , 2022 : 1352 - 1359 .

LECUN Y , BENGIO Y , HINTON G . Deep learning [J ] . Nature , 2015 , 521 ( 7553 ): 436 - 444 .

JOCHER G , CHAURASIA A , STOKEN A , et al . Ultralytics/YOLOv5: v7.0-YOLOv5 sota realtime instance segmentation [Z ] . Zenodo , 2022 .

STURM J , ENGELHARD N , ENDRES F , et al . A benchmark for the evaluation of RGB-D SLAM systems [C ] // Proceedings of 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems . Piscataway : IEEE Press , 2012 : 573 - 580 .

浏览量

424

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

低纹理动态场景下基于特征增强的视觉惯导SLAM方法

基于多尺度卷积神经网络特征融合的植株叶片检测技术

基于SE-YOLOv5s的绝缘子检测

一种基于深度学习的机械臂分拣方法

Cerberus：基于深度学习的跨站社交网络机器人检测系统