Modeling Car Following Behavior of Autonomous Driving Vehicles Based on Deep Reinforcement Learning

In order to enhance the performance of car following behavior of autonomous vehicles and mitigate the negative effects of traffic oscillations, a deep reinforcement learning-based car following model for automated driving is investigated. The existing reward function is improved by incorporating energy consumption, and the related terms for representing energy consumption are established based on the VT-Micro model. In addition, the method of using the time gap between vehicles to establish the reward function related to driving efficiency is improved by adding virtual speed to the time gap, in order to avoid computation overflow and unrealistic short following distance in the traffic oscillation scenario. To overcome the limitations of training on closed-loop simulated roads and simulated vehicle trajectories, human driver behavior extracted from the NGSIM trajectory data during traffic oscillation are used to develop the training environment. By applying the twin delayed deep deterministic policy gradient algorithm (TD3), a multi-objective car following model is then developed. A system for evaluating model performance is established to compare the performance of the TD3 model with traditional models in car following and traffic oscillations scenarios. Study results of car following scenarios show that the TD3 model and the traditional adaptive cruise control (ACC) model perform similarly in terms of comfort and driving efficiency, but both outperform the human drivers. In terms of safety, the TD3 model reduces safety hazards by 53.65% compared to the traditional ACC model, and 36.24% compared to the human drivers. Regarding energy consumption, the TD3 model reduces the energy consumption of the conventional ACC model and human drivers by 6.73% and 15.65%, respectively. Study results show that the TD3 model can reduce the negative impacts of traffic oscillations. In the scenario with a 100% TD3 model penetration rate, driving discomfort decreases by 55.95%, driving efficiency increases by 8.82%, crash risks reduce by 73.21%, and fuel consumption drops by 5.97%, compared to a 100% human-driven environment.