面向自动驾驶的车路协同感知点云融合模式时延影响分析

叶青; 赵聪; 朱逸凡; 俞山川

doi:10.3963/j.jssn.1674-4861.2023.04.008

面向自动驾驶的车路协同感知点云融合模式时延影响分析

doi: 10.3963/j.jssn.1674-4861.2023.04.008

叶青^{1, 2,},
赵聪^3, ,,
朱逸凡³,
俞山川²

1.
重庆大学大数据与软件学院重庆 401331
2.
招商局重庆交通科研设计院有限公司自动驾驶技术交通运输行业研发中心重庆 400060
3.
同济大学道路与交通工程教育部重点实验室上海 201804

基金项目:

国家重点研发计划项目 2021YFB1600403

重庆市技术创新与应用发展专项重点项目 cstc2021jscx-gksbX0057

详细信息

作者简介:
叶青（1994—），博士研究生. 研究方向：智能交通系统、车路协同与自动驾驶. E-mail：yeqing2@cmhk.com

通讯作者:
赵聪（1992—），博士，副研究员. 研究方向：车路协同数字感知与智能决策. E-mail: zhc@tongji.edu.cn

中图分类号: U491.5
计量
- 文章访问数: 625
- HTML全文浏览量: 301
- PDF下载量: 35
- 被引次数: 0
出版历程
- 收稿日期: 2021-06-07
- 网络出版日期: 2023-11-23

An Analysis of the Impact of Time Delay of Fusion Modes for Point Clouds from Cooperative Road Vehicle Systems on Autonomous Driving

YE Qing^{1, 2
,},
ZHAO Cong^{3
, ,},
ZHU Yifan³,
YU Shanchuan²

1.
School of Big Data & Software Engineering, Chongqing University, Chongqing 401331, China
2.
Research and Development Center of Transport Industry of Self-driving Technology, China Merchants Chongqing Communications Technology Research & Design Institute Co., Ltd., Chongqing 400060, China
3.
Key Laboratory of Road and Traffic Engineering of the Ministry of Education, Tongji University, Shanghai 201804, China

摘要

摘要: 新一代通信技术的快速发展为车路协同感知提供了基础，可进一步提升自动驾驶车辆在复杂场景中的感知能力，现有研究对不同的协同感知信息融合模式进行了探索，但忽略了对感知精度与通信时延平衡性的分析。针对自动驾驶协同感知中点云融合模式的时延特征，本文以前融合、特征融合、后融合3种模式为研究对象，提出了基于模拟仿真的时延影响分析框架。考虑通信时延引起的协同感知结果时滞性，利用扩展卡尔曼滤波算法对存在时延的协同感知结果进行预测性补偿，创新提出了滞后补偿误差和等效时延评价指标，用以综合评价不同融合模式对协同感知结果的影响；针对不同点云融合模式的感知结果，构建了平均感知精度与平移误差分布关系模型，依据目标检测平移误差的分布特征生成带有感知误差的仿真轨迹，进而对协同感知效果进行评估。结合TrajNet++行人轨迹数据集，以不同时延参数及点云融合模式，对1 200条轨迹进行了180 000次数值仿真。结果显示，感知目标的已知轨迹长度越短、速度越高，时延对协同感知精度的影响越大，以100 ms时延下后融合为基准，当特征融合时延在500 ms以内、前融合时延在700 ms以内时，可以达到相同或更高的协同感知精度。针对目标易突然出现且速度快的复杂场景，宜采用低时延、低精度的后融合模式，反之，宜采用具有高时延、高精度的特征融合或前融合模式。本研究可为自动驾驶协同感知的点云融合模式选择提供依据。
- 智能交通 /
- 时延影响 /
- 轨迹预测 /
- 自动驾驶 /
- 协同感知 /
- 点云融合
Abstract: The rapid development of the new generation of communication technologies provides a foundation for cooperative perception between autonomous vehicles (AVs) and road. This advancement holds the potential to significantly enhance the perception capabilities of AVs in complex scenarios. Previous studies have explored different information fusion modes for cooperative perception, but neglected to analyze the balance between perception accuracy and communication delay. Aiming at the delay characteristics of point cloud fusion in cooperative perception of AVs, a delay impact analysis framework is proposed based on simulation, concentrating on three fusion modes: pre-fusion, feature fusion, and post-fusion. Considering the time lag of cooperative perception results caused by communication delay, the Extended Kalman Filter algorithm is used to make predictive compensation for cooperative perception results with delay. The novel metrics, namely Lag Compensation Error and equivalent time delay, are proposed for comprehensive evaluation of the impact of different fusion modes on cooperative perception results. Based on perception results from various point cloud fusion modes, a model is established to fit the relationship between average perception accuracy and translation error distribution. Utilizing the distribution characteristics of translation errors, this model serves as the basis for generating simulated trajectories with perception errors and subsequently the evaluating of cooperative perception performance. Finally, leveraging the TrajNet++ pedestrian trajectory dataset, 180 000 numerical simulations are conducted across 1 200 trajectories with various point cloud fusion modes and different delay parameters. The results demonstrate that the shorter trajectory lengths and the higher target speeds amplify the impact of delay on cooperative perception accuracy. In comparison to post-fusion with a 100 ms delay as the benchmark, the equivalent or superior cooperative perception accuracy is feasible when the feature-fusion delay is below 500 ms and the front-fusion delay is below 700 ms. In complex scenarios involving sudden target appearances or high-speed targets, it is recommended to choose low-delay, low-accuracy post-fusion modes. Conversely, it is advisable to consider feature fusion or pre-fusion modes with high delay and high accuracy. This study can provide a basis for the selection of point cloud fusion modes for cooperative perception of autonomous driving.
- intelligent transportation /
- delay impact /
- trajectory prediction /
- autonomous driving /
- collaborative perception /
- point cloud fusion

HTML全文

图 1 车路点云示意图

Figure 1. Schematic diagram of point cloud of vehicle and road

下载: 全尺寸图片幻灯片

图 2 点云前融合流程

Figure 2. Process of pre-fusion modefor point cloud

下载: 全尺寸图片幻灯片

图 3 点云特征融合流程

Figure 3. Process of feature-fusion mode for point cloud

下载: 全尺寸图片幻灯片

图 4 点云后融合流程

Figure 4. Process of post-fusion mode for point cloud

下载: 全尺寸图片幻灯片

图 5 自动驾驶协同感知流程

Figure 5. Cooperative perception process for autonomous driving

下载: 全尺寸图片幻灯片

图 6 时延影响分析框架

Figure 6. The framework of latency impact analysis

下载: 全尺寸图片幻灯片

图 7 基于点云的目标检测平均精度与平均平移误差关系

Figure 7. Relationship between mean average precision and mean average translation error of object detection based on point cloud

下载: 全尺寸图片幻灯片

图 8 不同融合模式和已知轨迹时长下的滞后补偿误差

Figure 8. LCE under different fusion modes and known trajectory duration

下载: 全尺寸图片幻灯片

图 9 不同融合模式和目标速度时长下的滞后补偿误差

Figure 9. LCE under different fusion modes and object velocity

下载: 全尺寸图片幻灯片

图 10 不同条件下的等效时延

Figure 10. Equivalent latency under different conditions

下载: 全尺寸图片幻灯片

表 1 点云目标检测模型的平均精度和平均平移误差

Table 1. mean average precision and mean average translation error of object detection models based on point cloud

序号	平均精度L_mAP	平均平移误差L_mATE
1	0.723 9	0.236 6
2	0.697 2	0.237 0
3	0.674 0	0.254 6
4	0.681 5	0.255 8
5	0.671 5	0.248 8
6	0.668 0	0.253 1
7	0.668 0	0.253 5
8	0.662 8	0.240 3
9	0.666 9	0.254 2
10	0.665 9	0.272 6

下载: 导出CSV

表 2 点云融合模式评价指标

Table 2. Evaluationindexes of point cloud fusion modes

点云融合模式	时延/ms	平均精度L_mAP
前融合	高（100~1 000）	0.762 9
特征融合	中（50~200）	0.635 1
后融合	低（0~100）	0.489 9

下载: 导出CSV

表 3 仿真实验参数

Table 3. Simulation experiment parameters 单位: ms

参数	取值	取值步长
时延	100~1 000	100
点云融合模式	前融合、特征融合、后融合	无

下载: 导出CSV

参考文献(20)

[1]	许庆, 王嘉伟, 王建强, 等. 网联通信时延下的混合队列控制特性分析[J]. 交通信息与安全, 2021, 39(1): 128-136. XU Q, WANG J W, WANG J Q, et al. A performance analysis of mixed platoon control under communication delay[J]. Journal of Transport Information and Safety, 2021, 39(1): 128-136. (in Chinese)
[2]	陈志军, 张晶明, 熊盛光, 等. 智能网联车辆生态驾驶研究现状及展望[J]. 交通信息与安全. 2022, 40(4): 13-25. CHEN Z J, ZHANG J M, XIONG S G, et al. A review on research status and trends of eco-driving on intelligent connected vehicles[J]. Journal of Transport Information and Safety, 2022, 40(4): 13-25. (in Chinese)
[3]	ARNOLD E, DIANATI M, DE TEMPLE R, et al. Cooperative perception for 3D object detection in driving scenarios using infrastructure sensors[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(3): 1852-1864.
[4]	ZHOU Y, TUZEL O. VoxelNet: End-to-end learning for point cloudbased 3D object detection[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, United States: IEEE, 2018.
[5]	LANG A H, VORA S, CAESAR H, et al. Pointpillars: Fast encoders for object detection from point clouds[C]. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, United States: IEEE, 2019.
[6]	QI C R, SU H, MO K, et al. PointNet: Deep learning on point sets for 3D classification and segmentation[C]. IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Honolulu, Hawaii, United States: IEEE, 2017.
[7]	LI Y, BU R, SUN M, et al. Pointcnn: Convolution on x-transformed points[J]. Advances in Neural Information Processing Systems, 2018(31): 828-838.
[8]	THOMA RS, ANDRICH C, GALDO G, et al. Cooperative passive coherent location: a promising 5G service to support road safety[J]. IEEE Communications Magazine, 2019, 57 (9): 86-92.
[9]	CHEN Q, TANG S, YANG Q, et al. Cooper: Cooperative perception for connected autonomous vehicles based on 3D point clouds[C]. 39^th International Conference on Distributed Computing Systems, Dallas, United States: IEEE, 2019.
[10]	CHEN Q, MA X, TANG S, et al. F-Cooper: Feature based cooperative perception for autonomous vehicle edge computing system using 3D point clouds[C]. 4^th ACM/IEEE Symposium on Edge Computing, Washington, D. C., United States: ACM/IEEE, 2019.
[11]	XU R, XIANG H, XIA X, et al. Opv2v: An open benchmark dataset and fusion pipeline for perception with vehicle-to-vehicle communication[C]. International Conference on Robotics and Automation(ICRA), Philadelphia, United States: IEEE, 2022.
[12]	XU R, TU Z, XIANG H, et al. CoBEVT: Cooperative bird's eye view semantic segmentation with sparse transformers[J]. ProceedingsofMachine Learning Research, 2023, 205: 989-1000.
[13]	XU R, XIANG H, TU Z, et al. V2x-vit: Vehicle-to-everything cooperative perception with vision transformer[C]. European Conference on Computer Vision, Tel Aviv, Israel: Springer, 2022.
[14]	TSUKADA M, OI T, ITO A, et al. AutoC2X: Open-source software to realize V2X cooperative perception among autonomous vehicles[C]. IEEE 92^nd Vehicular Technology Conference, Victoria, BC, Canada: IEEE, 2020.
[15]	GHOSH A, MAEDER A, BAKER M, et al. 5G evolution: A view on 5G cellular technology beyond 3GPP release 15[J]. IEEE Access, 2019, 7: 127639-127651.
[16]	HARTLEY R, GHAFFARI M, EUSTICE R M, et al. Contact-aided invariant extended Kalman filtering for robot state estimation[J]. The International Journal of Robotics Research, 2020, 39(4): 402-430.
[17]	KOTHARI P, KREISS S, ALAHI A. Human trajectory forecasting in crowds: a deep learning perspective[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 23 (7): 7386-7400.
[18]	ZHAO C, SONG A, DU Y, et al. TrajGAT: A map-embedded graph attention network for real-time vehicle trajectory imputation of roadside perception[J]. Transportation Research Part C: Emerging Technologies, 2022, 142: 103787.
[19]	GUO J, CARRILLO D, TANG S, et al. CoFF: Cooperative spatial feature fusion for 3D object detection on autonomous vehicles[J]. IEEE Internet of Things Journal, 2021, 8(14): 11078-11087.
[20]	CAESAR H, BANKITI V, LANG AH, et al. Nuscenes: A multimodal dataset for autonomous driving[C]. IEEE Conference on Computer Vision and Pattern Recognition, Washington, D. C., United States: IEEE, 2020.