3D convolution-based image sequence feature extraction and self-attention for license plate recognition method

Ganxiong ZENG; Xiao KE

doi:10.11959/j.issn.2096-6652.202128

您当前的位置：

首页 >

文章列表页 >

3D convolution-based image sequence feature extraction and self-attention for license plate recognition method

Special Issue: Intelligent Object Detection and Recognition | 更新时间：2024-06-05

- 3D convolution-based image sequence feature extraction and self-attention for license plate recognition method
- Chinese Journal of Intelligent Science and Technology Vol. 3, Issue 3, Pages: 268-279(2021)
- 作者机构：
  
  1. 福州大学数学与计算机科学学院，福建福州 350116
  2. 福建省网络计算与智能信息处理重点实验室，福建福州 350116
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-6652.202128
  CLC： TP183，TP39
- Published Online：2021-09，
  
  Published：15 September 2021
- 稿件说明：
移动端阅览
Ganxiong ZENG, Xiao KE. 3D convolution-based image sequence feature extraction and self-attention for license plate recognition method[J]. Chinese journal of intelligent science and technology, 2021, 3(3): 268-279.
DOI：

Ganxiong ZENG, Xiao KE. 3D convolution-based image sequence feature extraction and self-attention for license plate recognition method[J]. Chinese journal of intelligent science and technology, 2021, 3(3): 268-279. DOI： 10.11959/j.issn.2096-6652.202128.

摘要

近年来，基于自注意力机制的神经网络在计算机视觉任务中得到广泛的应用。随着智能交通系统的广泛应用，面对复杂多变的交通场景，车牌识别任务的难度不断提高，准确识别的需求更加迫切。因此提出一个基于自注意力的免矫正的车牌识别方法T-LPR。首先对图像进行切片和序列化，并使用3D卷积对切片序列进行特征提取，从而得到图像的嵌入向量序列。然后将嵌入向量序列输入基于Transformer Encoder的编码器中，学习各个嵌入向量之间的关系并输出最终的编码结果。最后使用分类器进行分类。在多个公共数据集上的实验结果表明，所提方法对各类困难场景下的车牌识别都非常有效。

Abstract

In recent years

neural networks based on self-attentive mechanism have been widely used in computer vision tasks.As the intelligent transportation system is widely used

the task difficulty of license plate recognition is increasing and the need for correct recognition is getting more pressing in the face of complex and changing traffic scenes.Therefore

a rectification-free license plate recognition method T-LPR based on self-attention was proposed.Firstly

the images were sliced and sequenced

and 3D convolution was used for feature extraction of the sliced sequences to obtain a sequence of image embedding vectors.Secondly

the sequence of embedding vectors was fed into an encoder based on Transformer Encoder

which learned the relationship between the individual embedding vectors and outputs the final encoding result.Finally

the final encoding result was classified by a classifier.Experimental results on several public datasets show that T-LPR proposed is very effective for recognizing license plates in all kinds of difficult scenarios.

关键词

Keywords

references

刘晓波 , 蒋阳升 , 唐优华 , 等 . 综合交通大数据应用技术的发展展望 [J ] . 大数据 , 2019 , 5 ( 3 ): 55 - 68 .

LIU X B , JIANG Y S , TANG Y H , et al . Development prospect of in-tegrated transportation big data application technology [J ] . Big Data Research , 2019 , 5 ( 3 ): 55 - 68 .

KHAN M A , SHARIF M , JAVED M Y , et al . License number plate recognition system using entropy-based features selection approach with SVM [J ] . IET Image Processing , 2018 , 12 ( 2 ): 200 - 209 .

薛磊 , 杨晓敏 , 吴炜 , 等 . 一种基于KNN与改进SVM的车牌字符识别算法 [J ] . 四川大学学报(自然科学版) , 2006 , 43 ( 5 ): 1031 - 1036 .

XUE L , YANG X M , WU W , et al . An algorithm based on KNN and improved SVM for licence plate recognition [J ] . Journal of Sichuan University (Natural Science Edition) , 2006 , 43 ( 5 ): 1031 - 1036 .

张钹 . 人工智能进入后深度学习时代 [J ] . 智能科学与技术学报 , 2019 , 1 ( 1 ): 4 - 6 .

ZHANG B . Artificial intelligence is entering the post deep-learning era [J ] . Chinese Journal of Intelligent Science and Technology , 2019 , 1 ( 1 ): 4 - 6 .

LI H , WANG P , YOU M Y , et al . Reading car license plates using deep neural networks [J ] . Image and Vision Computing , 2018 , 72 : 14 - 23 .

JAIN V , SASINDRAN Z , RAJAGOPAL A , et al . Deep automatic license plate recognition system [C ] // Proceedings of the 10th Indian Conference on Computer Vision,Graphics and Image Processing . New York:ACM Press , 2016 : 1 - 8 .

XU Z B , YANG W , MENG A J , et al . Towards end-to-end license plate detection and recognition:a large dataset and baseline [C ] // Proceedings of the European Conference on Computer Vision . Heidelberg:Springer , 2018 : 261 - 277 .

LAROCA R , SEVERO E , ZANLORENSI L A , et al . A robust real-time automatic license plate recognition based on the YOLO detector [C ] // Proceedings of 2018 International Joint Conference on Neural Networks (IJCNN) . Piscataway:IEEE Press , 2018 : 1 - 10 .

ZHUANG J F , HOU S H , WANG Z L , et al . Towards human-level license plate recognition [C ] // Proceedings of the European Conference on Computer Vision . Heidelberg:Springer , 2018 : 314 - 329 .

SILVA S M , JUNG C R . License plate detection and recognition in unconstrained scenarios [C ] // Proceedings of the European Conference on Computer Vision . Heidelberg:Springer , 2018 : 593 - 609 .

ZHANG L J , WANG P , LI H , et al . A robust attentional framework for license plate recognition in the wild [J ] . IEEE Transactions on Intelligent Transportation Systems (Early Access) , 2020 : 1 - 10 .

ZHERZDEV S , GRUZDEV A . LPRNet:license plate recognition via deep neural networks [J ] . arXiv preprint,2018,arXiv:1806,10447 .

CHEN L C , PAPANDREOU G , KOKKINOS I , et al . DeepLab:semantic image segmentation with deep convolutional nets,atrous convolution,and fully connected CRFs [J ] . arXiv preprint,2016,arXiv:1606.00915 .

HE K M , GKIOXARI G , DOLLÁR P , et al . Mask R-CNN [C ] // Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV) . Piscataway:IEEE Press , 2017 : 2980 - 2988 .

REDMON J , FARHADI A . YOLOv3:an incremental improvement [J ] . arXiv preprint,2018,arXiv:1804.02767 .

REDMON J , FARHADI A . YOLO9000:better,faster,stronger [C ] // Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway:IEEE Press , 2017 : 6517 - 6525 .

ZHANG Z G , WANG C . The research of vehicle plate recognition technical based on BP neural network [J ] . AASRI Procedia , 2012 , 1 : 74 - 81 .

TAO X Z , LI L , LU L . A lightweight convolutional neural network for license plate character recognition [C ] // Proceedings of 2019 International Symposium on Intelligence Computation and Applications . Heidelberg:Springer , 2019 : 379 - 387 .

LI H , WANG P , SHEN C H . Toward end-to-end car license plate detection and recognition with deep neural networks [J ] . IEEE Transactions on Intelligent Transportation Systems , 2019 , 20 ( 3 ): 1126 - 1136 .

LUO C J , JIN L W , SUN Z H . MORAN:a multi-object rectified attention network for scene text recognition [J ] . Pattern Recognition , 2019 , 90 : 109 - 118 .

WANG T W , ZHU Y Z , JIN L W , et al . Decoupled attention network for text recognition [C ] // Proceedings of the AAAI Conference on Artificial Intelligence , 2020 , 34 ( 7 ): 12216 - 12224 .

VASWANI A , SHAZEER N , PARMAR N , et al . Attention is all you need [J ] . arXiv preprint,2017,arXiv:1706.03762 .

DEVLIN J , CHANG M W , LEE K , et al . BERT:pre-training of deep bidirectional transformers for language understanding [J ] . arXiv preprint,2019,arXiv:1810.04805 .

DOSOVITSKIY A , BEYER L , KOLESNIKOV A , et al . An image is worth 16×16 words:transformers for image recognition at scale [J ] . arXiv preprint,2020,arXiv:2010.11929 .

CARION N , MASSA F , SYNNAEVE G , et al . End-to-end object detection with transformers [C ] // Proceedings of the European conference on Computer Vision . Heidelberg:Springer , 2020 : 213 - 229 .

ZHENG S X , LU J C , ZHAO H S , et al . Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers [J ] . arXiv preprint,2020,arXiv:2012.15840 .

LI X Y , ZHANG Y Y , LIU C H , et al . VidTr:video transformer without convolutions [J ] . arXiv preprint,2021,arXiv:2104.11746 .

YUAN Y L , ZOU W B , ZHAO Y , et al . A robust and efficient approach to license plate detection [J ] . IEEE Transactions on Image Processing , 2017 , 26 ( 3 ): 1102 - 1114 .

LI P Y , NGUYEN M , YAN W Q . Rotation correction for license plate recognition [C ] // Proceedings of 2018 4th International Conference on Control,Automation and Robotics . Piscataway:IEEE Press , 2018 : 400 - 404 .

GOU C , WANG K F , YAO Y J , et al . Vehicle license plate recognition based on extremal regions and restricted boltzmann machines [J ] . IEEE Transactions on Intelligent Transportation Systems , 2016 , 17 ( 4 ): 1096 - 1107 .

BJÖRKLUND T , FIANDROTTI A , ANNARUMMA M , et al . Robust license plate recognition using neural networks trained on synthetic images [J ] . Pattern Recognition , 2019 , 93 : 134 - 146 .

GAO F , CAI Y C , GE Y S , et al . EDF-LPR:a new encoder-decoder framework for license plate recognition [J ] . IET Intelligent Transport Systems , 2020 , 14 ( 8 ): 959 - 969 .

SCHUSTER M , PALIWAL K K . Bidirectional recurrent neural networks [J ] . IEEE Transactions on Signal Processing , 1997 , 45 ( 11 ): 2673 - 2681 .

GRAVES A , FERNÁNDEZ S , GOMEZ F , et al . Connectionist temporal classification:labelling unsegmented sequence data with recurrent neural networks [C ] // Proceedings of the 23rd International Conference on Machine Learning . New York:ACM Press , 2006 : 369 - 376 .

CHOLLET F , . Xception:deep learning with depthwise separable convolutions [C ] // Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2017 : 1800 - 1807 .

HOCHREITER S , SCHMIDHUBER J . Long short-term memory [J ] . Neural Computation , 1997 , 9 ( 8 ): 1735 - 1780 .

SHI B G , BAI X , YAO C . An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2017 , 39 ( 11 ): 2298 - 2304 .

ZHANG Y P , NIE S , LIU W J , et al . Sequence-to-sequence domain adaptation network for robust text image recognition [C ] // Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2019 : 2735 - 2744 .

JADERBERG M , SIMONYAN K , ZISSERMAN A , et al . Spatial transformer networks [J ] . arXiv preprint,2016,arXiv:1506.02025 .

YANG M K , GUAN Y S , LIAO M H , et al . Symmetry-constrained rectification network for scene text recognition [C ] // Proceedings of 2019 IEEE/CVF International Conference on Computer Vision . Piscataway:IEEE Press , 2019 : 9146 - 9155 .

LU N , YU W W , QI X B , et al . MASTER:multi-aspect non-local network for scene text recognition [J ] . Pattern Recognition , 2021 , 117 : 107980 .

BA J L , KIROS J R , HINTON G E . Layer normalization [J ] . arXiv preprint,2016,arXiv:1607.06450 .

REN S Q , HE K M , GIRSHICK R , et al . Faster R-CNN:towards real-time object detection with region proposal networks [J ] . arXiv preprint,2015,arXiv:1506.01497 .

ŠPAŇHEL J , SOCHOR J , JURÁNEK R , et al . Holistic recognition of low quality license plates by CNN using track annotated data [C ] // Proceedings of 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) . Piscataway:IEEE Press , 2017 : 1 - 6 .

LIU W , ANGUELOV D , ERHAN D , et al . SSD:single shot multibox detector [C ] // Proceedings of the European conference on Computer Vision . Heidelberg:Springer , 2016 : 21 - 37 .

MASOOD S Z , SHU G , DEHGHAN A , et al . License plate detection and recognition using deeply learned convolutional neural networks [J ] . arXiv preprint,2017,arXiv:1703.07330 .

Views

772

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

Postal code：100079
Tel：（010）53879206 Email：cjist@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：35685 Visits Today：160

⁰