航拍视频车辆检测目标关联与时空轨迹匹配

冯汝怡; 李志斌; 吴启范; 范昌彦

doi:10.3963/j.jssn.1674-4861.2021.02.008

航拍视频车辆检测目标关联与时空轨迹匹配

doi: 10.3963/j.jssn.1674-4861.2021.02.008

东南大学交通学院南京 210096

基金项目:

国家自然科学基金项目 71871057

详细信息

作者简介:
冯汝怡(1999—)，博士研究生.研究方向：智能交通.E-mail: fengruyi@seu.edu.cn

通讯作者:
李志斌(1983—)，博士，教授.研究方向：智能交通.E-mail: lizhibin@seu.edu.cn

中图分类号: U491.4
计量
- 文章访问数: 756
- HTML全文浏览量: 561
- PDF下载量: 71
- 被引次数: 0
出版历程
- 收稿日期: 2020-07-06

Association of Vehicle Object Detection and the Time-space Trajectory Matching from Aerial Videos

School of Transportation, Southeast University, Nanjing 210096, China

摘要

摘要: 高解析度轨迹数据蕴含丰富车辆行驶与交通流时空信息。为从航拍视频中提取车辆轨迹，构建了车辆检测目标跨帧关联与轨迹匹配融合方法。采用卷积神经网络YOLOv5构建视频全域车辆目标检测，提出车辆动力学与轨迹置信度约束下跨帧目标关联算法，建立了基于最大相关性的断续轨迹匹配与融合构建算法，实现轨迹车辆唯一编号。将轨迹从图像坐标转换为车道基准下Frenet坐标，构建集合经验模态分解(EEMD)模型进行轨迹数据噪声消除。采用南京市快速路无人机拍摄的2组开源航拍视频，涵盖拥堵与自由流交通状态，对轨迹提取算法进行效果测试。结果表明，在自由流和拥挤条件下轨迹准确率分别为98.86%和98.83%，轨迹召回率为93.00%和86.69%，构建算法的轨迹提取速度为0.07 s/辆/m。该方法处理得到的详细车辆时空轨迹信息能为交通流、交通安全、交通管控研究提供广泛的数据支撑，数据公开于http://seutraffic.com/。
- 车辆轨迹 /
- 车辆检测 /
- 航拍视频 /
- 轨迹构建 /
- 轨迹数据
Abstract: High resolution track data contains rich information about vehicle travel and traffic flow. The fusion method of cross-frame vehicle detection association and trajectory matching is developed to extract the vehicle trajectories from the aerial video. The convolutional neural network, YOLOv5, is used to obtain video-wide vehicle object detection. Base on the result of detection, a correlation algorithm of a cross-frame target under the constraints of vehicle dynamics and trajectory confidence is proposed. Then, broken track matching and constructing algorithms based on the maximum correlation are established for identifying unique vehicles. The trajectory is converted from image coordinates to Freenet coordinates under lane reference, and the ensemble empirical mode decomposition(EEMD)model has been constructed to eliminate data noise. Two sets of open-source aerial videos, coving congestion and free-flow traffic status, are taken by a drone on the Nanjing expressway to test the effect of the trajectory extraction algorithm. The results show that the trajectory accuracies are 98.86 and 98.83% under the free flow and congested conditions, respectively. Besides, the track recall rates are 93.00 and 86.69%. The trajectory extraction speed of the algorithm is 0.07 s/vehicle/m. The vehicle trajectory dataset processed by this method can provide extensive data support for traffic flow, traffic safety, and traffic control research. The dataset is published at http://seutraffic.com/.
- vehicle trajectory /
- vehicle detection /
- UAV aerial video /
- trajectory construction /
- trajectory data

HTML全文

图 1 典型车辆时空轨迹图

Figure 1. Typical vehicle time-space trajectory map

下载: 全尺寸图片幻灯片

图 2 轨迹构建算法框架图

Figure 2. Framework of the proposed algorithm

下载: 全尺寸图片幻灯片

图 3 轨迹关联示意图

Figure 3. Illustration for trajectory correlation

下载: 全尺寸图片幻灯片

图 4 坐标转换示意图

Figure 4. Illustration of coordinate transform

下载: 全尺寸图片幻灯片

图 5 假阳性检测结果的可能原因

Figure 5. Possible cause for false positive detection

下载: 全尺寸图片幻灯片

图 6 车辆轨迹降噪结果

Figure 6. Results for trajectory denoising

下载: 全尺寸图片幻灯片

图 7 关键交通参数与NGSIM数据对比

Figure 7. Comparison of key traffic parameters with NGSIM

下载: 全尺寸图片幻灯片

图 8 视频1车辆轨迹时空图

Figure 8. Time-space vehicle trajectory for video 1

下载: 全尺寸图片幻灯片

图 9 视频2车辆轨迹时空图

Figure 9. Time-space vehicle trajectory for video 2

下载: 全尺寸图片幻灯片

表 1 轨迹构建结果

Table 1. Results of trajectory construction

变量	测试视频1	测试视频2
轨迹数量真值(GT)	500	541
真阳数(TP)	465	469
假阴数(TN）	35	72
假阳数(FP)	4	43
召回率(Recall)/%	93.00	86.69
准确率(Precision)/%	99.15	91.6

下载: 导出CSV

参考文献(20)

[1]	赵秀江. 基于视频图像处理技术的行车轨迹线采集及其分析研究[D]. 长沙: 湖南大学, 2012. ZHAO Xiujiang. Collection and analysis research of travel trajectory line based on image processing technology[D]. Changsha: Hunan University, 2012. (in Chinese)
[2]	刘晨强. 车辆轨迹数据与换道行为特性研究[D]. 北京: 北京工业大学, 2018. LIU Chenqiang. Research of vehicle trajectory data and lane change characteristics[D]. Beijing: Beijing University of Technology, 2018. in Chinese
[3]	Federal Highwavy Administration. Next generation simulation (NGSIM) vehicle trajectories and supporting data[EB/OL]. (2006-12-1)[2021-03-11]. https://data.transportation.gov/Automobiles/Next-Generation-Simulation-NGSIM-Vehicle-Trajector/ 8ect-6jqj.
[4]	MONTANINO M, PUNZO V. Making NGSIM data usable for studies on traffic flow theory: multistep method for vehicle trajectory reconstruction[J]. Transportation Research Record: Journal of the Transportation Research Board, 2013(2390): 99-111. http://www.researchgate.net/profile/Marcello_Montanino/publication/269853999_Making_NGSIM_Data_Usable_for_Studies_on_Traffic_Flow_Theory/links/54abda5a0cf25c4c472fb1bb.pdf
[5]	石建军, 刘晨强. NGSIM车辆轨迹重构[J]. 北京工业大学学报, 2019, 45(6): 601-609. https://www.cnki.com.cn/Article/CJFDTOTAL-BJGD201906010.htm SHI Jianjun, LIU Chenqiang. Trajectory reconstruction of vehicles in NGSIM[J]. Journal of Beijing University of Technology, 2019, 45(6): 601-609. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-BJGD201906010.htm
[6]	PUNZO V, BORZACCHIELLO M T, CIUO B. On the assessment of vehicle trajectory data accuracy and application to the Next Generation Simulation(NGSIM)program data[J]. Transportation Research Part C: Emerging Technologies, 2011(19), 1243-1262.
[7]	姬红利. 基于航拍视频的多目标检测和跟踪[D]. 天津: 天津大学, 2014. JI Hongli. Multiple target detection and tracking based on aerial videos[D]. Tianjin: Tianjin University, 2014. (in Chinese)
[8]	EMMANOUIL N B, ELENI I V, JOHN C G. Unmanned aerial aircraft systems for transportation engineering: Current practice and future challenges[J]. International Journal of Transportation Science and Technology, 2016, 5(3): 111-122. doi: 10.1016/j.ijtst.2017.02.001
[9]	成名, 金立左. 基于视觉显著性的航拍车辆检测[J]. 工业控制计算机, 2016(4): 75-77. doi: 10.3969/j.issn.1001-182X.2016.04.035 CHENG Ming, JIN Lizuo. Aerial vehicle detection based on visual significance[J]. Industrial Control Computer, 2016(4): 75-77. (in Chinese) doi: 10.3969/j.issn.1001-182X.2016.04.035
[10]	毛征, 刘松松, 张辉等. 不同光照和姿态下的航拍车辆检测方法[J]. 北京工业大学学报, 2016, 42(7): 982-988. https://www.cnki.com.cn/Article/CJFDTOTAL-BJGD201607004.htm MAO Zheng, LIU Songsong, ZHANG Hui. Vehicle detection from aerial photographing under different illumination and pose[J]. Journal of Beijing University of Technology, 2016, 42 (7): 982-988. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-BJGD201607004.htm
[11]	XIAO Jiangjian, YANG Changjiang, HAN Feng, et al. Vehicle and person tracking in aerial videos[C]. Multimodal Technologies for Perception of Humans, Baltimore, USA: SpringerVerlag, 2007.
[12]	SALEEMI I, SHAH M. Multiframe many–many point correspondence for vehicle tracking in high density wide area aerial videos[J]. International Journal of Computer Vision, 2013 (104): 198-219. doi: 10.1007/s11263-013-0624-1
[13]	AZEVEDO C L, CARDOSO J L, BEN-AKIVA M, et al. Automatic vehicle trajectory extraction by aerial remote sensing[J]. ProcediaSocial and Behavioral Sciences, 2014(111): 849-858. http://www.sciencedirect.com/science/article/pii/S1877042814001207
[14]	CAO Xianbin, WU Changxia, LAN Jinhe, et al. Vehicle detection and motion analysis in low-altitude airborne video under urban environment[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2011, 21(10): 1522-1533. doi: 10.1109/TCSVT.2011.2162274
[15]	REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: Unified, real-time object detection[C]. Computer Vision & Pattern Recognition, Las Vegas, USA: IEEE, 2016.
[16]	FAN Q, BROWN L, SMITH J. A closer look at faster R-CNN for vehicle detection[C]. IEEE Intelligent Vehicles Symposium(IV), Gotenburg, Sweden: ITSS, 2016.
[17]	KUHN H W. The Hungarian method for the assignment problem[J]. Naval Research Logistics Quarterly, 1955(1/2): 83-97. http://bib.oxfordjournals.org/external-ref?access_num=10.1002/nav.3800020109&link_type=DOI
[18]	MAO Tianlu, WANG Hua, DENG Zhigang, et al. An efficientlane model for complex traffic simulation[J]. Computer Animation & Virtual Worlds, 2015, 26(3/4): 397-403.
[19]	WU Zhaohua, HUANG N E. Ensemble emprical mode decomposition: a noise-assisted data analysis algorithm[J]. Advances in Adaptive Data Analysis, 2009(1): 1-41.
[20]	CHEN Xinqiang, LI Zhibin, WANG Yinhai, et al. Anomaly detection and cleaning of highway elevation data from Google Earth using ensemble empirical mode decomposition[J]. Journal of Transportation Engineering, 2018, 144(5): 04018015.1-04018015.14. http://www.researchgate.net/publication/324014183_Anomaly_Detection_and_Cleaning_of_Highway_Elevation_Data_from_Google_Earth_Using_Ensemble_Empirical_Mode_Decomposition