基于无人机图像的城市道路停车巡检方法

王琛; 张凌云; 刘波; 张航

doi:10.3963/j.jssn.1674-4861.2024.04.010

基于无人机图像的城市道路停车巡检方法

doi: 10.3963/j.jssn.1674-4861.2024.04.010

长安大学工程机械学院西安 710064

基金项目:

国家自然科学基金青年基金项目 52202435

西安市科协青年人才托举计划项目 959202313091

详细信息

作者简介:
王琛(1987—)，博士，副教授. 研究方向：无人系统设计等. E-mail：wangchenjustin@chd.edu.cn

通讯作者:
张航(1991—)，博士，讲师. 研究方向：智能无人系统设计等. E-mail：zhhang@chd.edu.cn

中图分类号: U491
计量
- 文章访问数: 45
- HTML全文浏览量: 20
- PDF下载量: 4
- 被引次数: 0
出版历程
- 收稿日期: 2024-01-30
- 网络出版日期: 2024-11-25

An Inspection Method of Urban Road Parking Based on UAV Image

Mechanical Engineering College, Chang'an University, Xi'an 710064, China

摘要

摘要: 快速准确的对城市道路停放车辆进行巡检对于城市智慧管理具有重要意义。针对当前巡检方法的低效率、高成本和不准确，研究了1种基于无人机图像的巡检方法。首先，为满足全天候巡检，采用光照度增强算法对暗光条件的图像进行增强，同时采用去模糊算法对模糊图像质量进行改善。针对YOLOv5现有算法检测精度不高，实时性不强等问题，改进网络为使模型收敛速度更快，采用Focal-EIOU Loss优化损失函数；为了可以更好的适应不同大小和形状的目标，采用C2F模块替代C3模块，使用不同大小的卷积核提取特征；为了提高网络的鲁棒性及抗干扰能力，通过添加SimAM注意力机制，不增加网络模块参数且能预测特征图的3D注意力权值；采用CARAFE算子进行上采样增加感受野，全面利用特征图的语义信息。实验结果表明：改进后YOLOv5模型的准确率提高了5.1%，召回率提高了5.9%，平均精度mAP值提高了3.6%。其次，采用字符识别网络SVTR对车牌号进行识别，仅通过单个视觉模型就能完成特征提取和文本转录2个任务。通过无人机场工程应用平台进行试验，试验结果表明：该巡检方法可以准确、快速且智能完成巡检，准确度达到90%，检测速度达到170帧/s，基本满足巡检精度和实时性要求。
- 交通工程 /
- 道路停车 /
- 巡检方法 /
- 图像处理 /
- 无人机
Abstract: Efficient and accurate inspection of parked vehicles on urban roads is of significant importance for smart city management. Addressing the issues of low efficiency, high cost, and inaccuracy in current inspection methods, a drone-based image inspection approach is investigated. To enable all-weather inspection, an illumination enhance-ment algorithm is employed to boost images captured under low-light conditions, while a deblurring algorithm is uti-lized to improve the quality of blurred images. To tackle the limitations of the existing YOLOv5 algorithm, includ-ing insufficient detection accuracy and real-time performance, several modifications are introduced. The Fo-cal-EIOU Loss function is optimized to accelerate model convergence. The C3 module is replaced with the C2F module, utilizing varying sizes of convolutional kernels to extract features, enhancing adaptability to targets of dif-ferent sizes and shapes. Furthermore, the SimAM attention mechanism is incorporated to improve the network's ro-bustness and anti-interference capability, predicting 3D attention weights for feature maps without increasing model parameters. The CARAFE operator is adopted for upsampling to expand the receptive field, comprehensively lever-aging semantic information from feature maps. Experimental results demonstrate that the modified YOLOv5 model achieves 5.1% increase in accuracy, 5.9% improvement in recall, and 3.6% enhancement in mean Average Precision (mAP). Secondly, the SVTR character recognition network is utilized to identify license plate numbers, accomplish-ing both feature extraction and text transcription tasks within a single vision model. Finally, field tests conducted on a drone-based engineering application platform reveal that this inspection method can accurately, rapidly, and intelli-gently complete inspections, achieving an accuracy of 90% and a detection speed of 170 frames per second, essentially meeting inspection precision and real-time requirements.
- traffic engineering /
- road parking /
- inspection method /
- image processing /
- drone

HTML全文

图 1 算法流程图

Figure 1. Overall flow chart of algorithm

下载: 全尺寸图片幻灯片

图 2 DeblurgAN网络结构

Figure 2. Network Structure of deblurGAN

下载: 全尺寸图片幻灯片

图 3 IOU计算图

Figure 3. Calculation diagram of IOU

下载: 全尺寸图片幻灯片

图 4 C3模块和C2F模块的结构图

Figure 4. Structure diagram of C3 module and C2F module

下载: 全尺寸图片幻灯片

图 5 CARAFE算子流程图

Figure 5. Flowchart of CARAFE operator

下载: 全尺寸图片幻灯片

图 6 改进后YOLOv5结构图

Figure 6. Structure diagram of improved YOLOv5

下载: 全尺寸图片幻灯片

图 7 SVTR结构图

Figure 7. Diagram of SVTR structure

下载: 全尺寸图片幻灯片

图 8 车牌颜色提取流程图

Figure 8. Flow Chart of license plate color extraction

下载: 全尺寸图片幻灯片

图 9 ROI区域原理图

Figure 9. Area Schematic of ROI

下载: 全尺寸图片幻灯片

图 10 车辆检测图

Figure 10. Chart of vehicle inspection

下载: 全尺寸图片幻灯片

图 11 巡检系统总体图

Figure 11. Diagram of general inspection system

下载: 全尺寸图片幻灯片

图 12 数据集样本图

Figure 12. Graph of data set sample

下载: 全尺寸图片幻灯片

图 13 车牌识别结果

Figure 13. Result of license plate recognition

下载: 全尺寸图片幻灯片

图 14 改进前后算法训练结果

Figure 14. Results of algorithm training before and after improvement

下载: 全尺寸图片幻灯片

图 15 不同速度测试结果图

Figure 15. Graphs of test results at different speeds

下载: 全尺寸图片幻灯片

图 16 去模糊前后识别结果图

Figure 16. Identify the result map before and after deblurring

下载: 全尺寸图片幻灯片

图 17 光照增强前后检测结果图

Figure 17. Detection result diagram before and after light enhancement

下载: 全尺寸图片幻灯片

图 18 违停检测结果图

Figure 18. Stop violation detection result diagram

下载: 全尺寸图片幻灯片

表 1 消融实验结果

Table 1. Results of Ablation Experiment

改进的网络结构	P/%	R/%	mAP@0.5/%
YOLOv5	84.3	75.4	83.5
YOLOv5+Focal-EIOU	83.1	78.8	84.1
YOLOv5+Focal-EIOU+C2F	84.2	78.6	84.9
YOLOv5+Focal-EIOU+C2F+SimAM	85.1	79.5	85.1
YOLOv5+Focal-EIOU+C2F+SimAM +CARAFE	89.4	81.3	87.1

下载: 导出CSV

表 2 不同模型对比结果

Table 2. Compare experiment results with different models

模型	精确率/%	召回率/%	mAP/%	FPS/(帧/s)
SSD	87.6	76.4	85.1	85.7
Faster R-CNN	88.4	79.5	83.5	62.4
YOLOv5s	84.3	75.4	83.4	187.5
YOLOv6s	85.4	77.1	82.2	120.4
YOLOv7	80.2	68.2	70.7	108.5
YOLOv8s	81.8	73.9	75.6	195.4
YOLOv6s_U	87.5	78.8	84.6	202.1
本文模型	89.4	81.3	87.1	195.6

下载: 导出CSV

表 3 无人机场参数表

Table 3. Table of Unmanned Airfield Parameters

指标	参数
机场作业半径/km	7
防水等级	IP55
工作环境温度/℃	－35~50
RTK基站定位	1 + 10^-4(水平)
精准度/cm	2 + 10^-4(垂直)

下载: 导出CSV

表 4 无人机参数表

Table 4. Table of UAV Parameters

指标	参数
云台相机	禅思H20N
最大飞行时间/min	41
最大水平飞行速度/(m/s)	23
最大变焦倍数/x	200
工作环境温度/℃	－20~50
云台可控转动范围/(°)	平移：±90
云台可控转动范围/(°)	俯仰：－120~+45
RTK基站定位精准度/cm	1 + 10^-4(水平)
RTK基站定位精准度/cm	1.5 + 10^-4(垂直)

下载: 导出CSV

表 5 实验环境

Table 5. Experimental Environment

场景	描述
道路1	主干道2 km, 监控多，违停少
道路2	辅道1 km, 监控少，违停多

下载: 导出CSV

表 6 环境光照强度对照表

Table 6. Table of ambient light intensity comparison

场所/环境	光照度/lux
晴天室外	30 000~300 000
阴天室外	50~500
夜间路灯	0.1~0.3
黑夜	0.001~0.02

下载: 导出CSV

表 7 系统测试实验结果

Table 7. Results of System Test

场景	算法				FPS/(帧/s)	准确率/%
场景	车牌识别	违停判断	光照增强	去模糊	FPS/(帧/s)	准确率/%
	√				198.2	89.2
光照较好		√			195.4	88.4
光照较好	√			√	187.2	93.5
		√		√	182.9	91.6
	√				198.4	72.5
	√				193.6	69.7
光照较差	√		√		185.2	88.4
光照较差		√	√		180.7	87.3
	√		√	√	174.5	92.1
		√	√	√	170.1	90.9

下载: 导出CSV

参考文献(24)

[1]	梁定康, 钱瑞, 陈义豪, 等. 基于视觉的无人机巡检违章违停系统设计与实现[J]. 电子科技, 2018, 31(5): 76-77, 92. LIANG D K, QIAN R, CHEN Y H, et al. Design and implementation of vision-based unmanned aerial vehicle inspectionand parking violation system[J]. Electronic Science and Technology, 2018, 31(5): 76-77, 92. (in Chinese)
[2]	冯东明, 余星宇, 黎剑安, 等. 基于无人机的悬索桥主缆自动巡检及小样本表观病害识别[J]. 中国公路学报, 2024, 37 (2): 29-39. FENG D M, YU X Y, LI J A, et al. Automatic inspection of main cable of suspension bridge based on UAV and smallsample apparent disease recognition[J]. China Journal of Highway and Transportation, 2024, 37(2): 29-39. (in Chinese)
[3]	刘东海, 马子茹, 黄斌, 等. 基于无人机巡检与深度学习的河道整治施工进度图像识别[J]. 水资源与水工程学报, 2024, 35(4): 92-100, 110. LIU D H, MA Z R, HUANG B, et al. Image recognition of river regulation construction progress based on UAV inspection and deep learning[J]. Journal of WaterResources and Water Engineering, 2024, 35(4): 92-100, 110. (in Chinese)
[4]	赵慧敏, 罗贺, 阴酉龙, 等. 面向集合任务的多无人机电力巡检任务分配方法研究[J/OL]. 系统工程理论与实践, (2024-05-08)[2024-06-10]. http://kns.cnki.net/kcms/detail/11.2267.N.20240507.1609.003.html. ZHAO H M, LUO H, YIN Y L, et al. Research on task assignment method of multi-UAV power inspection for ensembletask[J/OL]. System engineering theory and practice, (2024-05-08)[2024-06-10]. http://kns.cnki.net/kcms/detail/11.2267.N.20240507.1609.003.html. (in Chinese)
[5]	黄小龙. 无人机巡检图像和蒲公英算法的输电线缆绝缘子缺陷自动检测[J]. 集成电路与嵌入式系统, 2024, 24(8): 64-71. HUANG X L. Automatic insulator defect detection of transmission line based on UAV inspection image and dandelion algorithm[J]. Integrated Circuits and Embedded Systems, 2024, 24(8): 64-71. (in Chinese)
[6]	马丽. 无人机智能巡检技术在光伏发电并网巡检中的应用[J]. 电气技术与经济, 2024, (8): 144-146. MA L. Application of UAV intelligent inspection technology in grid-connected photovoltaic power generation inspection[J]. Electrical Technology & Economy, 2024, (8): 144-146. (in Chinese)
[7]	田爱军, 蔡旭阳, 陈玮, 等. 无人机端路面车辆违停检测及取证系统[J]. 测控技术, 2021, 40(5): 67-74. TIAN A J, CAI X Y, CHEN W, et al. UAV end road vehicle parking detection and evidence collection System[J]. Measurement and Control Technology, 2021, 40(5): 67-74. (in Chinese)
[8]	吴志高, 陈明. 基于改进YOLOV7的微藻轻量级检测方法[J]. 大连海洋大学学报, 2023, 38(1): 129-139. WU Z G, CHEN M. Lightweight detection method of microalgae based on improved YOLOV7[J]. Journal of Dalian Ocean University, 2023, 38(1): 129-139. (in Chinese)
[9]	原晓佩, 陈小锋, 廉明. 基于Haar-like和LBP的多特征融合目标检测算法[J]. 计算机科学, 2021, 48(11): 219-225. YUAN X P, CHEN X F, LIAN M. Multi-feature fusion target detection algorithm based on Haar-like and LBP[J]. Journal of Computer Science, 2021, 48(11): 219-225. (in Chinese)
[10]	耿庆田, 于繁华, 王宇婷, 等. 基于特征融合的车型检测新算法[J]. 吉林大学学报(工学版)2018, 48(3): 929-935. GENG Q T, YU F H, WANG Y T, et al. A new vehicle detection algorithm based on feature fusion[J]. Journal of Jilin University(Engineering and Technology Edition), 2018, 48 (3): 929-935. (in Chinese)
[11]	王燕, 张继凯, 尹乾. 基于Faster R-CNN的车牌识别算法[J]. 北京师范大学学报(自然科学版), 2020, 56(5): 647-653. WANG Y, ZHANG J K, YIN Q. License plate recognition algorithm based on faster R-CNN[J]. Journal of Beijing Normal University(Natural Science Edition), 2020, 56(5): 647-653. (in Chinese)
[12]	ZOU Y J, ZHANG Y J, YAN J, et al. A robust license plate recognition model based on bi-lstm[J]. IEEE Access, 2020, 8: 211630-211641.
[13]	饶文军, 谷玉海, 朱腾腾, 等. 基于深度学习的车牌智能识别方法[J]. 重庆理工大学学报(自然科学), 2021, 35(3): 119-127. RAO W J, GU Y H, ZHU T T, et al. Intelligent license plate recognition method based on deep learning[J]. Journal of Chongqing University of Technology (Natural Science), 2021, 35(3): 119-127. (in Chinese)
[14]	XU X Y, PAN J, ZHANG Y J, et al. Motion blur kernel estimation via deep learning[J]. IEEE Transactions on Image Processing, 2017, 27(1): 194-205.
[15]	SUN J, CAO W F, XU Z B, et al. Learning a convolutional neural network for non-uniform motion blur removal[C]. The IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA: IEEE, 2015.
[16]	王司雨, 李良荣, 顾平, 等. 1种混合的单图像去运动模糊方法[J]. 电子技术应用, 2019, 45(9): 81-84. WANG S Y, LI L R, GU P, et al. A hybrid single image demotion-blurring method[J]. Application of Electronic Technology, 2019, 45(9): 81-84. (in Chinese)
[17]	KUPYN O, BUDZAN V, MYKHAILYCHM, et al. Deblurgan: blind motion deblurring using conditional adversarial networks[C]. The IEEE Conferenceon Computer Vision and Pattern Recognition, Salt Lake City, UT, USA: IEEE, 2018.
[18]	钱炜, 潘琦. 基于生成对抗网络的喷雾液滴运动模糊恢复[J]. 国外电子测量技术, 2022, 41(11): 15-21. QIAN W, PAN Q. Spray droplet motion fuzzy recovery based on generative adduction network[J]. Foreign Electronic Measurement Technology, 2022, 41(11): 15-21. (in Chinese)
[19]	董良振, 田建艳, 杨胜强, 等. 基于光照校正和图像融合的零件表面图像增强[J]. 计算机工程, 2024, 50(6): 245-254. DONG L Z, TIAN J Y, YANG S Q, et al. Part surface image enhancement based on illumination correction and image fusion[J]. Computer Engineering, 2024, 50(6): 245-254. (in Chinese)
[20]	ALI U, LEE I H, MAHMOOD M T. Guided image filtering in shape-from-focus: a comparative analysis[J]. Pattern Recognition, 2021, 111: 107670.
[21]	张红英, 朱恩弘, 吴亚东. 1种基于细节层分离的单曝光HDR图像生成算法[J]. 自动化学报, 2019, 45(11): 2159-2170. ZHANG H Y, ZHU E H, WU Y D. A single exposure HDR image generation algorithm based on detail layer separation[J]. Acta Automatica Sinica, 2019, 45(11): 2159-2170. (in Chinese)
[22]	ZHU P, CHEN B, LIU B, et al. Object detection for hazardous material vehicles based on improved YOLOv5 algorithm[J]. Electronics, 2023, 12(5): 1257-1273.
[23]	戴愿, 刘伟铭, 王珩, 等. 基于改进YOLOv5s模型的地铁屏蔽门与列车门间异物快速检测方法[J]. 交通信息与安全, 2023, 41(2): 18-27. doi: 10.3963/j.jssn.1674-4861.2023.02.002 DAI Y, LIU W M, WANG H, et al. Rapid detection method of foreign bodies between subway screen doors and train doors based on improved YOLOv5s model[J]. Journal of Transport Information and Safety, 2023, 41(2): 18-27. (in Chinese) doi: 10.3963/j.jssn.1674-4861.2023.02.002
[24]	WANG J, CHEN K, XU R, et al. Carafe: content-aware reassembly of features[C]. The IEEE/CVF International Conference on Computer Vision, Seoul, Korea(South): IEEE, 2019.