基于视觉显著性的车载单目相机自运动估计及前车尺度估计方法

艾明欣; 刘铁; 王净; 丁佳丽; 袁泽剑; 尚媛园

doi:10.11959/j.issn.2096-6652.202129

您当前的位置：

首页 >

文章列表页 >

基于视觉显著性的车载单目相机自运动估计及前车尺度估计方法

专刊：目标智能检测与识别 | 更新时间：2024-06-05

- 基于视觉显著性的车载单目相机自运动估计及前车尺度估计方法
- A method based on visual saliency for vehicle-mounted monocular camera ego-motion estimation and vehicle scale estimation
- 智能科学与技术学报 2021年3卷第3期页码：280-293
- 作者机构：
  
  1. 首都师范大学信息工程学院，北京 100048
  2. 西安交通大学人工智能学院，陕西西安 710049
- 作者简介：
  
  [ "艾明欣（1999− ），女，首都师范大学信息工程学院硕士生，主要研究方向为计算机视觉、模式识别等" ]
  [ "刘铁（1981− ）男，博士，首都师范大学信息工程学院副教授，主要研究方向为计算机视觉、模式识别、多媒体计算、图像处理、大规模数据计算等" ]
  [ "王净（1997− ），女，首都师范大学信息工程学院硕士生，主要研究方向为计算机视觉、模式识别等" ]
  [ "丁佳丽（1996− ），女，首都师范大学信息工程学院硕士生，主要研究方向为计算机视觉、显著性物体检测等" ]
  [ "袁泽剑（1971− ），男，博士，西安交通大学人工智能学院教授，主要研究方向为图像处理、模式识别、计算机视觉、机器学习、概率机器人等" ]
  [ "尚媛园（1977− ），女，博士，首都师范大学信息工程学院教授，主要研究方向为图像处理、计算机视觉、嵌入式系统等" ]
- 基金信息：
  
  国家自然科学基金资助项目;The National Natural Science Foundation of China(61876112);国家自然科学基金资助项目;The National Natural Science Foundation of China(61976170);北京市自然科学基金资助项目;The Natural Science Foundation of Beijing(L201022)
- DOI：10.11959/j.issn.2096-6652.202129
  中图分类号： TP181
- 网络出版日期：2021-09，
  
  纸质出版日期：2021-09-15
- 稿件说明：
移动端阅览
艾明欣, 刘铁, 王净, 等. 基于视觉显著性的车载单目相机自运动估计及前车尺度估计方法[J]. 智能科学与技术学报, 2021,3(3):280-293.

Mingxin AI, Tie LIU, Jing WANG, et al. A method based on visual saliency for vehicle-mounted monocular camera ego-motion estimation and vehicle scale estimation[J]. Chinese journal of intelligent science and technology, 2021, 3(3): 280-293.
艾明欣, 刘铁, 王净, 等. 基于视觉显著性的车载单目相机自运动估计及前车尺度估计方法[J]. 智能科学与技术学报, 2021,3(3):280-293. DOI： 10.11959/j.issn.2096-6652.202129.

Mingxin AI, Tie LIU, Jing WANG, et al. A method based on visual saliency for vehicle-mounted monocular camera ego-motion estimation and vehicle scale estimation[J]. Chinese journal of intelligent science and technology, 2021, 3(3): 280-293. DOI： 10.11959/j.issn.2096-6652.202129.

摘要

提出一种基于视觉显著性的车载单目相机自运动估计及前车尺度估计方法。首先，针对车载相机自运动估计，通过视觉显著性计算方法检测并去除含有噪声的单目图像序列中的运动目标，同时考虑图像的纹理区域和平滑区域，利用加权显著图保留有用特征点，进而对车载相机进行鲁棒的自运动估计。其次，将前车距离转化为前车尺度估计问题，通过描述子匹配与李代数中正则化的强度匹配相结合的方法最小化损失函数，通过设计视觉注意力机制选择有纹理无遮挡的图像块，并对选定的图像块中的像素赋权以减轻被噪声破坏像素的影响，从而实现鲁棒、准确的尺度估计。最后，利用多个具有挑战性的数据集对所提方法进行分析验证。结果表明，单目相机自运动估计方法达到了基于立体相机方法的水平，前车尺度估计方法在充分发挥强鲁棒性优势的同时保证了预测精度。

Abstract

A method based on visual saliency for ego-motion estimation and scale estimation of the vehicle in front was proposed.Firstly

for the ego-motion estimation of vehicle-mounted camera

the visual saliency calculation method was used to detect and remove moving objects in the monocular image sequence containing noise.While considering the image texture and smooth region

the weighted saliency map was used to retain useful feature points

to improve the robustness of ego-motion estimation.Secondly

the distance of the vehicle in front was converted into a vehicle scale estimation

by integrating descriptor match and the strength of regularization match of the lie algebra to minimize loss function.The visual attention mechanism was used to get texture image block without shade

and the pixel in the image block weight to mitigate the effects of destroyed by noise pixel

so as to realize the robust and accurate scale estimation.Finally

several challenging datasets were used to analyze and verify the proposed method.The results show that the monocular camera ego-motion estimation method reaches the level of the stereo camera-based method

and the vehicle scale estimation method ensures the prediction accuracy while giving full play to the advantages of strong robustness.

关键词

Keywords

references

JAEGLE A , PHILLIPS S , DANIILIDIS K . Fast,robust,continuous monocular egomotion computation [C ] // Proceedings of 2016 IEEE International Conference on Robotics and Automation (ICRA) . Piscataway:IEEE Press , 2016 : 773 - 780 .

LOURAKIS M I A , ARGYROS A A . SBA:a software package for generic sparse bundle adjustment [J ] . ACM Transactions on Mathematical Software , 2009 , 36 ( 1 ): 1 - 30 .

袁野 , 和晓歌 , 朱定坤 , 等 . 视觉图像显著性检测综述 [J ] . 计算机科学 , 2020 , 47 ( 7 ): 84 - 91 .

YUAN Y , HE X G , ZHU D K , et al . Survey of visual image saliency detection [J ] . Computer Science , 2020 , 47 ( 7 ): 84 - 91 .

GEIGER A , LENZ P , URTASUN R . Are we ready for autonomous driving? The KITTI vision benchmark suite [C ] // Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2012 : 3354 - 3361 .

刘凯 , 李浥东 , 林伟鹏 . 车辆再识别技术综述 [J ] . 智能科学与技术学报 , 2020 , 2 ( 1 ): 10 - 25 .

LIU K , LI Y D , LIN W P . A survey on vehicle re-identification [J ] . Chinese Journal of Intelligent Science and Technology , 2020 , 2 ( 1 ): 10 - 25 .

熊刚 , 董西松 , 朱凤华 , 等 . 城市交通大数据技术及智能应用系统 [J ] . 大数据 , 2015 , 1 ( 4 ): 81 - 96 .

XIONG G , DONG X S , ZHU F H , et al . Big data technologies and intelligent application system for urban transportation [J ] . Big Data Research , 2015 , 1 ( 4 ): 81 - 96 .

KIM G , CHO J S . Vision-based vehicle detection and inter-vehicle distance estimation [C ] // Proceedings of 2012 12th International Conference on Control,Automation and Systems . Piscataway:IEEE Press , 2012 : 625 - 629 .

MUKHTAR A , XIA L K , TANG T B . Vehicle detection techniques for collision avoidance systems:a review [J ] . IEEE Transactions on Intelligent Transportation Systems , 2015 , 16 ( 5 ): 2318 - 2338 .

DAGAN E , MANO O , STEIN G P , et al . Forward collision warning with a single camera [C ] // Proceedings of 2004 IEEE Intelligent Vehicles Symposium . Piscataway:IEEE Press , 2004 : 37 - 42 .

LIU T , YUAN Z J , SUN J , et al . Learning to detect a salient object [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2011 , 33 ( 2 ): 353 - 367 .

LI G B , XIE Y , LIN L , et al . Instance-level salient object segmentation [C ] // Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway:IEEE Press , 2017 : 247 - 256 .

LIU T , DUAN H B , SHANG Y Y , et al . Automatic salient object sequence rebuilding for video segment analysis [J ] . Science China Information Sciences , 2018 , 61 ( 1 ): 012205 .

HOLZER S , POLLEFEYS M , ILIC S , et al . Online learning of linear predictors for real-time tracking [C ] // Proceedings of 2012 European Conference on Computer Vision . Heidelberg:Springer , 2012 .

DANELLJAN M , HÄGER G , KHAN F S , et al . Accurate scale estimation for robust visual tracking [C ] // Proceedings of the 2014 British Machine Vision Conference .[S.l. ] : BMVA Press , 2014 : 470 - 483 .

CHEN D P , YUAN Z J , HUA G , et al . Multi-timescale collaborative tracking [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2017 , 39 ( 1 ): 141 - 155 .

CHANG H C , LAI S H , LU K R . A robust and efficient video stabilization algorithm [C ] // Proceedings of 2004 IEEE International Conference on Multimedia and Expo . Piscataway:IEEE Press , 2004 : 29 - 32 .

BAY H , ESS A , TUYTELAARS T , et al . Speeded-up robust features (SURF) [J ] . Computer Vision and Image Understanding , 2008 , 110 ( 3 ): 346 - 359 .

查冰 , 张力 , 艾海滨 . 不同特征提取算法对相机运动估计的适用性研究 [J ] . 测绘科学 , 2018 , 43 ( 3 ): 92 - 98 .

ZHA B , ZHANG L , AI H B . The applicability of different feature extraction algorithms to camera motion estimation [J ] . Science of Surveying and Mapping , 2018 , 43 ( 3 ): 92 - 98 .

SHI J B , TOMASI . Good features to track [C ] // Proceedings of 1994 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 1994 : 593 - 600 .

LUCAS B D , KANADE T . An iterative image registration technique with an application to stereo vision [C ] // Proceedings of the 7th International Joint Conference on Artificial Intelligence . San Francisco:Morgan Kaufmann Publishers Inc , 1981 : 674 - 679 .

CAI J H , WALKER R . Robust motion estimation for camcorders mounted in mobile platforms [C ] // Proceedings of 2008 Digital Image Computing:Techniques and Applications . Piscataway:IEEE Press , 2008 : 491 - 497 .

CHEN Q X , LIU T , SHANG Y Y , et al . Salient object detection:integrate salient features in the deep learning framework [J ] . IEEE Access , 2019 , 7 : 152483 - 152492 .

BOLME D S , BEVERIDGE J R , DRAPER B A , et al . Visual object tracking using adaptive correlation filters [C ] // Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2010 : 2544 - 2550 .

HARE S , GOLODETZ S , SAFFARI A , et al . Struck:structured output tracking with kernels [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2016 , 38 ( 10 ): 2096 - 2109 .

BAKER S , MATTHEWS I . Lucas-Kanade 20 years on:a unifying framework [J ] . International Journal of Computer Vision , 2004 , 56 ( 3 ): 221 - 255 .

BENHIMANE S , MALIS E . Real-time image-based tracking of planes using efficient second-order minimization [C ] // Proceedings of 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems . Piscataway:IEEE Press , 2004 : 943 - 948 .

BENHIMANE S , MALIS E . Homography-based 2D visual tracking and servoing [J ] . The International Journal of Robotics Research , 2007 , 26 ( 7 ): 661 - 676 .

LIU T , ZHENG N N , DING W , et al . Video attention:learning to detect a salient object sequence [C ] // Proceedings of 2008 19th International Conference on Pattern Recognition . Piscataway:IEEE Press , 2008 : 1 - 4 .

TIAN Y N , YANG G D , WANG Z , et al . Apple detection during different growth stages in orchards using the improved YOLO-V3 model [J ] . Computers and Electronics in Agriculture , 2019 , 157 : 417 - 426 .

TOMASI C , KANADE T . Detection and tracking of point features [J ] . International Journal of Computer Vision , 1991 , 9 ( 3 ): 137 - 154 .

RAPHAEL E , KIEFER R , REISMAN P , et al . Development of a camera-based forward collision alert system [J ] . SAE International Journal of Passenger Cars-Mechanical Systems , 2011 , 4 ( 1 ): 467 - 478 .

刘世蔷 , 黄影平 , 胡兴 , 等 . 基于自适应外点剔除与解耦算法的视觉里程计 [J ] . 信息与控制 , 2019 , 48 ( 2 ): 172 - 178 .

LIU S Q , HUANG Y P , HU X , et al . Visual odometry based on adaptive outlier removal and decoupling algorithm [J ] . Information and Control , 2019 , 48 ( 2 ): 172 - 178 .

CVIŠIĆ I , ĆESIĆ J , MARKOVIĆ I , et al . SOFT-SLAM:computationally efficient stereo visual simultaneous localization and mapping for autonomous unmanned aerial vehicles [J ] . Journal of Field Robotics , 2018 , 35 ( 4 ): 578 - 595 .

MUR-ARTAL R , TARDÓS J D , . ORB-SLAM2:an open-source SLAM system for monocular,stereo,and RGB-D cameras [J ] . IEEE Transactions on Robotics , 2017 , 33 ( 5 ): 1255 - 1262 .

IN T , PAN J , CAO S Z , et al . A general optimization-based framework for local odometry estimation with multiple sensors [J ] . arXiv preprint,2019,arXiv:1901.03638 .

浏览量

701

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

暂无数据