A Cooperative Control Model of Continuous Signal Intersections for Connected Vehicles
-
摘要: 智能交通信号控制技术是缓解交通拥堵的重要手段。为解决传统强化学习算法应用到连续多交叉口的局限性问题，提出了1种基于上下层神经网络的连续交叉口交通信号控制模型。控制模型由下层神经网络选择当前状态下可能的最优控制策略，再由上层神经网络根据各路口车均延误进行二次调整，将最终控制策略应用到多交叉口的相位配时中。以典型连续3个交叉口为例，通过SUMO仿真平台对模型进行仿真验证，在低与高饱和度下，该控制模型分别对车均延误降低了23.6%和26%，排队长度降低了8.4%和9.4%。实验数据表明，该模型可有效提高连续交叉口道路通行能力，为缓解城市交通拥堵提供了1种有效技术手段。Abstract: Intelligent traffic signal control is an essential means to alleviate traffic congestion. A continuous traffic signal control model based on the upper and lower neural networks is proposed to solve the limitation of the traditional reinforcement learning algorithm at continuous multiple intersections. In this model, the local optimal control strategy in the current state is selected by the lower neural network. Then, the secondary adjustment can be made by the upper neural network according to the delay of vehicles at intersections. A global control strategy is applied to the phase timing of multiple intersections. The model is verified by the SUMO simulation platform, taking three typical continuous intersections as case studies. The average vehicle delay reduces by 23.6% and 26% under low and high saturation, and the queue length reduces by 8.4% and 9.4%. The results show that the road capacity of continuous intersections can be improved based on the proposed model, which provides an effective technical method to alleviate urban traffic congestion.
-
图 1 连续交叉口上下层信号控制模型框架
Figure 1. Frame of upper-and-lower signal control model for continuous intersections
图 4 处理车辆信息的图卷积神经网络
Figure 4. Graph convolutional neural network for processing vehicle information
图 13 流量为2 400~3 600 veh/h的车均延误
Figure 13. Vehicle average delay at 2 400~3 600 veh / h
表 1 神经网络参数表
Table 1. Parameters of the neural network
参数 值 重放内存大小M 20 000 训练批次B 64 初始贪心率ϵ 1 最终贪心率ϵ 0.01 目标网络更新率α衰减系数γ 0.001 0.99 Relu函数泄露值 0.01 学习率 0.000 1 
下载: 导出CSV
表 2 车辆参数表
Table 2. Parameters of vehicles
车辆参数 数值 最大速度/(km/h) 50 最大加速度/(m/s2) 4.0 减速加速度/(m/s2) 4.5 车身长度/m 4.8 最小车间距/m 1
下载: 导出CSV
表 3 车流到达率
Table 3. Traffic arrival rates
交叉口 车流量/(Veh/h) 直行比例/% 右转比例/% 左转比例/% 1 2 400 57.14 28.57 14.29 2 2 400 54.55 27.27 18.18 3 2 400 57.14 28.57 14.29 1 3 600 44.44 33.33 22.22 2 3 600 57.14 28.57 14.29 3 3 600 50.00 25.00 25.00 1 4 800 57.14 28.57 14.29 2 4 800 60.00 20.00 20.00 3 4 800 57.14 28.57 14.29
下载: 导出CSV
表 4 各模型在不同流量下的车均延误统计
Table 4. Vehicle delay under different flow rates
车流量/(veh/h) 上下层Agent/m 单层DQN/m 数解法绿波带/m 2 400 34.9 36.1 53.4 3 600 46.3 50.6 57.4 4 800 48.9 56.4 64.2
下载: 导出CSV
表 5 各模型在不同流量下的排队长度统计
Table 5. Average queue length under different flow rates
车流量/(veh/h) 上下层Agent/m 单层DQN/m 数解法绿波带/m 2 400 12.4 13.5 17.6 3 600 15.3 16.4 18.4 4 800 20.7 22.8 23.2
下载: 导出CSV
-
[1] 王庞伟, 于洪斌, 张为, 等. 城市车路协同系统下实时交通状态评价方法[J]. 中国公路学报, 2019, 32(6): 176-187. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGL201906019.htmWANG Pangwei, YU Hongbin, ZHANG Wei, et al. Real-time traffic state evaluation method under urban vehicle-road collaboration system[J]. China Journal of Highway and Transport, 2019, 32(6): 176-187. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-ZGGL201906019.htm [2] 赵盼明, 刘钊, 刘玉, 等. 基于模糊控制的小区域交叉口群过饱和状态信号协调优化[J]. 交通信息与安全, 2018, 36(4): 51-59. doi: 10.3963/j.issn.1674-4861.2018.04.008ZHAO Panming, LIU Zhao, LIU Yu, et al. Signal coordination and optimization based on fuzzy control in the supersaturated state of small area intersections[J]. Journal of Transport Information and Safety, 2018, 36(4): 51-59. (in Chinese) doi: 10.3963/j.issn.1674-4861.2018.04.008 [3] SILVER D. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540): 529-533. doi: 10.1038/nature14236 [4] RAVINDRAN B. Reinforcement learning: ln introduction[J]. IEEETransactionson Neural Networks, 1992, 9(5): 1054-1054. [5] 曹建峰. 分段式优化数解法智能交通绿波带算法[J]. 物联网技术, 2013(8): 82-84. doi: 10.3969/j.issn.2095-1302.2013.08.027CAO Jianfeng. Green wave band algorithm for intelligent transportation based on piecewise optimized number solution[J]. Internet of Things Technologies, 2013(8): 82-84. (in Chinese) doi: 10.3969/j.issn.2095-1302.2013.08.027 [6] 常玉林, 张其强, 张鹏. 城市干线双向绿波控制优化设计[J]. 重庆理工大学学报(自然科学版), 2014, 28(12): 108-112. doi: 10.3969/j.issn.1674-8425(z).2014.12.021CHANG Yulin, ZHANG Qiqiang, ZHANG Peng. Two-way green wave control optimization design of urban trunk lines[J]. Journal of Chongqing University of Technology(Natural Science Edition), 2014, 28(12): 108-112. (in Chinese) doi: 10.3969/j.issn.1674-8425(z).2014.12.021 [7] 宋现敏, 张亚南, 马林. 交叉口动态车道与交通信号协同优化方法[J]. 交通运输系统工程与信息, 2020, 20(6): 121-128.SONG Xianmin, ZHANG Yanan, MA Lin. Cooperative optimization method of dynamic lane and traffic signal at intersection[J]. Journal of Transportation Systems Engineering and Information Technology, 2020, 20(6): 121-128. (in Chinese) [8] MOUSAVI S, SCHUKAT M, HOWLEY E. Traffic light control using deep policy-gradient and value-function-based reinforcement learning[J]. Intelligent Transport Systems, 2017, 11(7): 417-423. doi: 10.1049/iet-its.2017.0153 [9] LI L, LYU Yisheng, WANG Feiyue. Traffic signal timing via deep reinforcement learning[J]. CAA Journal of Automatica Sinica, 2016, 3(3): 247-254. doi: 10.1109/JAS.2016.7508798 [10] 文峰, 张可新. 基于深度强化学习的交通信号配时优化研究[J]. 沈阳理工大学学报, 2019, 38(1): 48-52+63. https://www.cnki.com.cn/Article/CJFDTOTAL-SGXY201901011.htmWEN Feng, ZHANG Kexin. Research on traffic signal timing optimization based on deep reinforcement learning[J]. Journal of Shenyang Ligong University, 2019, 38(1): 48-52 + 63. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-SGXY201901011.htm [11] 刘义, 何均宏. 强化学习在城市交通信号灯控制方法中的应用[J]. 科技导报, 2019, 37(6): 84-90. https://www.cnki.com.cn/Article/CJFDTOTAL-KJDB201906013.htmLIU Yi, HE Junhong. The application of reinforcement learning in the control method of urban traffic lights[J]. Science & Technology Review, 2019, 37(6): 84-90. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-KJDB201906013.htm [12] XU Ming, WU Jianping, HUANG Ling, et al. Network-wide traffic signal control based on the discovery of critical nodes and deep reinforcement learning[J]. Journal of Intelligent Transportation Systems, 2020, 24(1): 1-10. doi: 10.1080/15472450.2018.1527694 [13] TOUHBI S, BABRAM M A, NGUYEN-HUU T, et al. Adaptive traffic signal control: exploring reward definition for reinforcement learning[J]. Procedia Computer Science, 2017(109): 513-520. http://www.sciencedirect.com/science/article/pii/S1877050917309912 [14] WU Yuankai, TAN Huachun, PENG Jiankun, et al. Deep reinforcement learning of energy management with continuous control strategy and traffic information for aseries-parallel plugin hybrid electric bus[J]. Applied Energy, 2019(247): 454-466. http://www.sciencedirect.com/science/article/pii/S030626191930652X [15] AREL I, LIU C, URBANIK T, et al. Reinforcement learning-based multi-agent system for network traffic signal control[J]. IET Intelligent Transport Systems, 2010, 4(2): 128-135. doi: 10.1049/iet-its.2009.0070 [16] JIN J, MA X, KOSONEN I. An Intelligent control system for traffic lights with simulation-based evaluation[J]. Control Engineering Practice, 2017(58): 24-33. http://www.sciencedirect.com/science/article/pii/S096706611630212X [17] ZHAO Dongbin, DAI Yujie, ZHANG Zhen. Computational intelligence in urban traffic signal control: Asurvey[J]. IEEE Transactions on Systems, Man, and Cybernetics, 2011, 42(4): 485-494. http://ieeexplore.ieee.org/document/5978226 [18] WAN Chiahao, HWANG Mingchorng. Value-based deep reinforcement learning for adaptive isolated intersectionsignal control[J]. IET Intelligent Transport Systems, 2018, 12(9): 1005-1010. doi: 10.1049/iet-its.2018.5170 [19] TAN Tian, BAO Feng, DENG Yue, et al. Cooperative deep reinforcement learning for large-scale traffic grid signal control[J]. IEEE Transactions on Cybernetics, 2019, 50(6): 2687-2700. http://ieeexplore.ieee.org/document/8676356 -
点击查看大图
图(13) / 表(5)
计量
- 文章访问数: 968
- HTML全文浏览量: 465
- PDF下载量: 85
- 被引次数: 0