Visual SLAM based on semantic information and geometric constraints in dynamic environment

LI Jiaming; XIE Mingyang; ZHANG Min; WANG Congqing

doi:10.11959/j.issn.2096-6652.202342

您当前的位置：

首页 >

文章列表页 >

Visual SLAM based on semantic information and geometric constraints in dynamic environment

Papers and Reports | 更新时间：2024-04-07

- Visual SLAM based on semantic information and geometric constraints in dynamic environment
- Chinese Journal of Intelligent Science and Technology Vol. 5, Issue 4, Pages: 477-485(2023)
- 作者机构：
  
  南京航空航天大学自动化学院，江苏南京 211106
- 作者简介：
- 基金信息：
  
  The Qiyuan Lab Innovation Fund Project(S20210201102);Research and Practice Innovation Program of NUAA(xcxjh20220342)
- DOI：10.11959/j.issn.2096-6652.202342
  CLC： TP242.6
- Received：14 August 2023，
  
  Revised：2023-10-11，
  
  Published：15 December 2023
- 稿件说明：
移动端阅览
LI Jiaming, XIE Mingyang, ZHANG Min, et al. Visual SLAM based on semantic information and geometric constraints in dynamic environment[J]. Chinese Journal of Intelligent Science and Technology, 2023, 5(4): 477-485.
DOI：

LI Jiaming, XIE Mingyang, ZHANG Min, et al. Visual SLAM based on semantic information and geometric constraints in dynamic environment[J]. Chinese Journal of Intelligent Science and Technology, 2023, 5(4): 477-485. DOI： 10.11959/j.issn.2096-6652.202342.

摘要

目前同时定位与地图构建技术多假设外部环境是静态的，忽略了动态目标对SLAM系统的影响，这一假设很大程度上影响了无人系统自主导航的定位精度与鲁棒性。针对这一问题，提出了一种结合目标检测语义信息与多视图极线几何约束的动态SLAM系统，根据运动概率来判别与剔除系统中的动态特征点，并在TUM数据集及实机平台上进行了算法性能测试。实验结果表明在高动态环境下，该系统相较于ORB-SLAM2在绝对轨迹误差上至少可减小94%以上，相对位移轨迹和旋转角度误差分别至少降低了41%和40%，表明该算法能够有效剔除动态特征点，提高了动态环境下视觉SLAM系统的定位精度和鲁棒性。

Abstract

Most existing visual SLAM systems assume that the external environment is static

ignoring the influence of dynamic objects on the SLAM system. This assumption largely affects the accuracy and robustness of autonomous navigation. To address this issue

a dynamic SLAM system was proposed

which combined semantic information based on object detection and geometric information from multi-view geometry constraints by defining and discriminating the dynamic feature points in the system based on the moving probability. Experiment results on the public TUM dataset and our robot in real environment showed that

when comparing with ORB-SLAM2

the absolute trajectory error could be reduced larger than 94%

and the average relative position and attitude errors were reduced at least 41% and 40%

respectively

in high dynamic environments. It means that the proposed SLAM system effectively removes dynamic feature points

thus improving the localization accuracy and robustness of the visual SLAM system within high dynamic environments.

关键词

Keywords

references

SMITH R C , CHEESEMAN P . On the representation and estimation of spatial uncertainty [J ] . The International Journal of Robotics Research , 1986 , 5 ( 4 ): 56 - 68 .

王霞 , 左一凡 . 视觉SLAM研究进展 [J ] . 智能系统学报 , 2020 , 15 ( 5 ): 825 - 834 .

WANG X , ZUO Y F . Advances in visual SLAM research [J ] . CAAI Transactions on Intelligent Systems , 2020 , 15 ( 5 ): 825 - 834 .

MUR-ARTAL R , MONTIEL J M M , TARDÓS J D . ORB-SLAM: a versatile and accurate monocular SLAM system [J ] . IEEE Transactions on Robotics , 2015 , 31 ( 5 ): 1147 - 1163 .

MUR-ARTAL R , TARDÓS J D . ORB-SLAM2: an open-source SLAM system for monocular, stereo, and RGB-D cameras [J ] . IEEE Transactions on Robotics , 2017 , 33 ( 5 ): 1255 - 1262 .

CAMPOS C , ELVIRA R , RODRÍGUEZ J J G , et al . ORB-SLAM3: an accurate open-source library for visual, visual-inertial, and multimap SLAM [J ] . IEEE Transactions on Robotics , 2021 , 37 ( 6 ): 1874 - 1890 .

LABBÉ M , MICHAUD F . Online global loop closure detection for large-scale multi-session graph-based SLAM [C ] // Proceedings of 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems . Piscataway : IEEE Press , 2014 : 2661 - 2666 .

CARUSO D , ENGEL J , CREMERS D . Large-scale direct SLAM for omnidirectional cameras [C ] // Proceedings of 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . Piscataway : IEEE Press , 2015 : 141 - 148 .

郑南宁 . 人工智能新时代 [J ] . 智能科学与技术学报 , 2019 , 1 ( 1 ): 1 - 3 .

ZHENG N N . The new era of artificial intelligence [J ] . Chinese Journal of Intelligent Science and Technology , 2019 , 1 ( 1 ): 1 - 3 .

YU C , LIU Z X , LIU X J , et al . DS-SLAM: a semantic visual SLAM towards dynamic environments [C ] // Proceedings of 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . Piscataway : IEEE Press , 2019 : 1168 - 1174 .

BADRINARAYANAN V , KENDALL A , CIPOLLA R . SegNet: a deep convolutional encoder-decoder architecture for image segmentation [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2017 , 39 ( 12 ): 2481 - 2495 .

BESCOS B , FÁCIL J M , CIVERA J , et al . DynaSLAM: tracking, mapping, and inpainting in dynamic scenes [J ] . IEEE Robotics and Automation Letters , 2018 , 3 ( 4 ): 4076 - 4083 .

HE K M , GKIOXARI G , DOLLÁR P , et al . Mask R-CNN [C ] // Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV) . Piscataway : IEEE Press , 2017 : 2980 - 2988 .

BESCOS B , CAMPOS C , TARDÓS J D , et al . DynaSLAM II: tightly-coupled multi-object tracking and SLAM [J ] . IEEE Robotics and Automation Letters , 2021 , 6 ( 3 ): 5191 - 5198 .

JI T T , WANG C , XIE L H . Towards real-time semantic RGB-D SLAM in dynamic environments [C ] // Proceedings of 2021 IEEE International Conference on Robotics and Automation (ICRA) . Piscataway : IEEE Press , 2021 : 11175 - 11181 .

YUAN X , CHEN S . SaD-SLAM: a visual SLAM based on semantic and depth information [C ] // Proceedings of 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . Piscataway : IEEE Press , 2020 : 4930 - 4935 .

LIU Y B , MIURA J . RDS-SLAM: real-time dynamic SLAM using semantic segmentation methods [J ] . IEEE Access , 2021 , 9 : 23772 - 23785 .

ZHONG F W , WANG S , ZHANG Z Q , et al . Detect-SLAM: making object detection and SLAM mutually beneficial [C ] // Proceedings of 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) . Piscataway : IEEE Press , 2018 : 1001 - 1010 .

LIU W , ANGUELOV D , ERHAN D , et al . SSD: single shot MultiBox detector [M ] // Computer Vision-ECCV 2016 . Cham : Springer International Publishing , 2016 : 21 - 37 .

SOARES J C V , GATTASS M , MEGGIOLARO M A . Crowd-SLAM: visual SLAM towards crowded environments using object detection [J ] . Journal of Intelligent & Robotic Systems , 2021 , 102 ( 2 ): 1 - 16 .

REDMON J, FARHADI A. YOLOv3: an incremental improvement[EB].2018, arXiv preprint, arXiv:1804.02767.

ZHANG C Y , HUANG T , ZHANG R C , et al . PLD-SLAM: a new RGB-D SLAM method with point and line features for indoor dynamic scene [J ] . ISPRS International Journal of Geo-Information , 2021 , 10 ( 3 ): 163 .

WANG Y N , XU K , TIAN Y B , et al . DRG-SLAM: a semantic RGB-D SLAM using geometric features for indoor dynamic scene [C ] // Proceedings of 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . Piscataway : IEEE Press , 2022 : 1352 - 1359 .

LECUN Y , BENGIO Y , HINTON G . Deep learning [J ] . Nature , 2015 , 521 ( 7553 ): 436 - 444 .

JOCHER G , CHAURASIA A , STOKEN A , et al . Ultralytics/YOLOv5: v7.0-YOLOv5 sota realtime instance segmentation [Z ] . Zenodo , 2022 .

STURM J , ENGELHARD N , ENDRES F , et al . A benchmark for the evaluation of RGB-D SLAM systems [C ] // Proceedings of 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems . Piscataway : IEEE Press , 2012 : 573 - 580 .

Views

424

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Feature-enhanced visual-inertial SLAM method for low-texture dynamic environment

Plant leaf detection technology based on multi-scale CNN feature fusion

Insulator detection based on SE-YOLOv5s

A robot sorting method based on deep learning

Cerberus: a cross-site social bot detection system based on deep learning

Related Author

XU Xin

YANG Chen

YU Zirui

WANG Yao

Ying LI

Long CHEN

Zhaohong HUANG

Yang SUN

Related Institution

Chengyi University College, Jimei University

College of Computer Engineering, Jimei University

Fujian Provincial Key Laboratory of Information Processing and Intelligent Control

College of Computer and Control Engineering, Minjiang University

College of Computer Science and Mathematics, Fujian University of Technology

Postal code：100079
Tel：（010）53879206 Email：cjist@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：162003 Visits Today：166

⁰