Low-carbon smart parks achieve selfbalanced carbon emission and absorption through the cooperative scheduling of direct current(DC)-based distributed photovoltaic,energy storage units,and loads.Direct current power li...Low-carbon smart parks achieve selfbalanced carbon emission and absorption through the cooperative scheduling of direct current(DC)-based distributed photovoltaic,energy storage units,and loads.Direct current power line communication(DC-PLC)enables real-time data transmission on DC power lines.With traffic adaptation,DC-PLC can be integrated with other complementary media such as 5G to reduce transmission delay and improve reliability.However,traffic adaptation for DC-PLC and 5G integration still faces the challenges such as coupling between traffic admission control and traffic partition,dimensionality curse,and the ignorance of extreme event occurrence.To address these challenges,we propose a deep reinforcement learning(DRL)-based delay sensitive and reliable traffic adaptation algorithm(DSRTA)to minimize the total queuing delay under the constraints of traffic admission control,queuing delay,and extreme events occurrence probability.DSRTA jointly optimizes traffic admission control and traffic partition,and enables learning-based intelligent traffic adaptation.The long-term constraints are incorporated into both state and bound of drift-pluspenalty to achieve delay awareness and enforce reliability guarantee.Simulation results show that DSRTA has lower queuing delay and more reliable quality of service(QoS)guarantee than other state-of-the-art algorithms.展开更多
With the advancement of connected vehicle(CV)technology,an increasing number of CVs will appear on urban roads.Data collected by CVs can be used to optimize signal parameters at intersections,thus improving traffic ef...With the advancement of connected vehicle(CV)technology,an increasing number of CVs will appear on urban roads.Data collected by CVs can be used to optimize signal parameters at intersections,thus improving traffic efficiency.In this study,we design a real-time adaptive signal control method for an arterial road with multiple intersections with low penetration rates.By utilizing vehicle arrival information collected by CVs,our method rapidly determines optimal signal phasing and timing(SPaT).The proposed adaptive signal control method was tested with the Simulation of Urban Mobility(SUMO)software,and was found to reduce total travel delay in the network better than a fixed coordination control method.The performance of the proposed method in reducing travel delay is expected to improve as CV detection range increases.展开更多
Multi-agent Reinforcement Learning(MARL)has become one of the best methods in Adaptive Traffic Signal Control(ATSC).Traffic flow is a very regular traffic volume,which is highly critical to signal control policy.Howev...Multi-agent Reinforcement Learning(MARL)has become one of the best methods in Adaptive Traffic Signal Control(ATSC).Traffic flow is a very regular traffic volume,which is highly critical to signal control policy.However,dynamic control policies will directly affect traffic flow formation,and it is impossible to provide observation through the original traffic flow prediction.This paper proposes a method for estimating traffic flow according to the time window in Reinforcement Learning(RL)training.Therefore,it is verified on both the regular road network and the real road network.Our method further reduces the intersection delay and queue length compared with the original method.展开更多
The importance of using adaptive traffic signal control for figuring out the unpredictable traffic congestion in today's metropolitan life cannot be overemphasized. The vehicular ad hoc network(VANET), as an integ...The importance of using adaptive traffic signal control for figuring out the unpredictable traffic congestion in today's metropolitan life cannot be overemphasized. The vehicular ad hoc network(VANET), as an integral component of intelligent transportation systems(ITSs), is a new potent technology that has recently gained the attention of academics to replace traditional instruments for providing information for adaptive traffic signal controlling systems(TSCSs). Meanwhile, the suggestions of VANET-based TSCS approaches have some weaknesses:(1) imperfect compatibility of signal timing algorithms with the obtained VANET-based data types, and(2) inefficient process of gathering and transmitting vehicle density information from the perspective of network quality of service(Qo S). This paper proposes an approach that reduces the aforementioned problems and improves the performance of TSCS by decreasing the vehicle waiting time, and subsequently their pollutant emissions at intersections. To achieve these goals, a combination of vehicle-to-vehicle(V2V) and vehicle-to-infrastructure(V2I) communications is used. The V2 V communication scheme incorporates the procedure of density calculation of vehicles in clusters, and V2 I communication is employed to transfer the computed density information and prioritized movements information to the road side traffic controller. The main traffic input for applying traffic assessment in this approach is the queue length of vehicle clusters at the intersections. The proposed approach is compared with one of the popular VANET-based related approaches called MC-DRIVE in addition to the traditional simple adaptive TSCS that uses the Webster method. The evaluation results show the superiority of the proposed approach based on both traffic and network Qo S criteria.展开更多
Realising adaptive traffic signal control(ATSC)through reinforcement learning(RL)is an important means to easetraffic congestion.This paper finds the computing power of the central processing unit(CPU)cannot fully use...Realising adaptive traffic signal control(ATSC)through reinforcement learning(RL)is an important means to easetraffic congestion.This paper finds the computing power of the central processing unit(CPU)cannot fully usedwhen Simulation of Urban MObility(SUMO)is used as an environment simulator for RL.We propose a multi-process framework under value-basedRL.First,we propose a shared memory mechanism to improve exploration efficiency.Second,we use the weight sharing mechanism to solve the problem of asynchronous multi-process agents.We also explained the reason shared memory in ATSC does not lead to early local optima of the agent.Wehave verified in experiments the sampling efficiency of the 10-process method is 8.259 times that of the single process.The sampling efficiency of the 20-process method is 13.409 times that of the single process.Moreover,the agent can also converge to the optimal solution.展开更多
Reinforcement learning-based traffic signal control systems (RLTSC) can enhance dynamic adaptability, save vehicle travelling timeand promote intersection capacity. However, the existing RLTSC methods do not consider ...Reinforcement learning-based traffic signal control systems (RLTSC) can enhance dynamic adaptability, save vehicle travelling timeand promote intersection capacity. However, the existing RLTSC methods do not consider the driver’s response time requirement, sothe systems often face efficiency limitations and implementation difficulties.We propose the advance decision-making reinforcementlearning traffic signal control (AD-RLTSC) algorithm to improve traffic efficiency while ensuring safety in mixed traffic environment.First, the relationship between the intersection perception range and the signal control period is established and the trust region state(TRS) is proposed. Then, the scalable state matrix is dynamically adjusted to decide the future signal light status. The decision will bedisplayed to the human-driven vehicles (HDVs) through the bi-countdown timer mechanism and sent to the nearby connected automatedvehicles (CAVs) using the wireless network rather than be executed immediately. HDVs and CAVs optimize the driving speedbased on the remaining green (or red) time. Besides, the Double Dueling Deep Q-learning Network algorithm is used for reinforcementlearning training;a standardized reward is proposed to enhance the performance of intersection control and prioritized experiencereplay is adopted to improve sample utilization. The experimental results on vehicle micro-behaviour and traffic macro-efficiencyshowed that the proposed AD-RLTSC algorithm can simultaneously improve both traffic efficiency and traffic flow stability.展开更多
Adaptive Traffic Signal Control(ATSC)adjusts signal timings to real-time traffic measure ments,increasing operational efficiency within a network.However,ATSC is both expen sive to install and operate making it infeas...Adaptive Traffic Signal Control(ATSC)adjusts signal timings to real-time traffic measure ments,increasing operational efficiency within a network.However,ATSC is both expen sive to install and operate making it infeasible to deploy at all signalized intersections within a network.This study presents a bi-level optimization framework that applies heuristic methods to identify a limited set of locations for ATSC deployment within an urban network.At the upper-level,the Population Based Incremental Learning(PBIL)algo rithm is employed to generate,evaluate,learn,and update different ATSC configurations.The lower-level uses the delay-based Max-Pressure algorithm to simulate the ATSC config uration within a microsimulation platform.The study proposes improvements to the PBIL algorithm by considering constraints on the maximum number of intersections for ATSC deployment and incorporates prior information about the intersection performance(i.e.,informed search).Simulation results on the traffic network of State College,PA reveal that the proposed PBIL algorithm consistently outperforms baseline methods that select loca tions only based on queue-lengths or delays in terms of reducing overall network travel times.The study also reveals that intersections experiencing the highest delays or longest queues are not always the best candidates for ATSC.Moreover,applying ATSC at all inter sections does not always provide the best performance;in fact,ATSC applied to some loca tions could increase travel times by contributing additional congestion downstream.Additionally,the modified PBIL algorithm with the informed search strategy is more effi cient at identifying promising solutions suggesting it can be readily applied to more gen eralized optimization problems.展开更多
Macroscopic Fundamental Diagrams(MFDs)are valuable for designing and evaluating network-wide traffic management schemes.Since obtaining empirical MFDs can be expensive,analytical methodologies are crucial to estimate ...Macroscopic Fundamental Diagrams(MFDs)are valuable for designing and evaluating network-wide traffic management schemes.Since obtaining empirical MFDs can be expensive,analytical methodologies are crucial to estimate variations in MFD shapes under different control strategies and predict their efficacy in mitigating congestion.Analyses of urban grid networks'abstractions can provide an inexpensive methodology to obtain a qualitative understanding of impacts of control policies.However,existing abstractions are valid only for simple intersection layouts with unidirectional and single-lane links and two conflicting movement groups.Naturally,the real intersections are more complex,with multiple incoming and outgoing lanes,heterogeneous incoming links'capacities and several conflicting movement groups.To this end,we consider a grid network with differences in capacities of horizontal and vertical directions,allowing us to investigate the characteristics of control policies that can avoid pernicious gridlock in heterogeneous networks.We develop a new,more comprehensive network abstraction of such grid networks to analyze and compare the impacts of two families of decentralized Traffic Signal Controllers(TSCs)on the network's stability.The obtained theoretical insights are verified using microsimulation results of grid networks with multiple signalized intersections.The analyses suggest that considering both upstream and downstream congestion information in deciding signal plans can encourage more evenly distributed traffic in the network,making them more robust and effective at all congestion levels.The study provides a framework to understand general expectations from decentralized control policies when network inhomogeneity arises due to variations in incoming link capacities and turning directions.展开更多
Optimization of adaptive traffic signal timing is one of the most complex problems in traffic control systems. This paper presents an adaptive transit signal priority （TSP） strategy that applies the parallel genetic...Optimization of adaptive traffic signal timing is one of the most complex problems in traffic control systems. This paper presents an adaptive transit signal priority （TSP） strategy that applies the parallel genetic algorithm （PGA） to optimize adaptive traffic signal control in the presence of TSP. The method can optimize the phase plan, cycle length, and green splits at isolated intersections with consideration for the performance of both the transit and the general vehicles. A VISSIM （VISual SIMulation） simulation testbed was developed to evaluate the performance of the proposed PGA-based adaptive traffic signal control with TSP. The simulation results show that the PGA-based optimizer for adaptive TSP outperformed the fully actuated NEMA control in all test cases. The results also show that the PGA-based optimizer can produce TSP timing plans that benefit the transit vehicles while minimizing the impact of TSP on the general vehicles.展开更多
基金supported by the Science and Technology Project of State Grid Corporation of China under grant 52094021N010(5400-202199534A-0-5-ZN)。
文摘Low-carbon smart parks achieve selfbalanced carbon emission and absorption through the cooperative scheduling of direct current(DC)-based distributed photovoltaic,energy storage units,and loads.Direct current power line communication(DC-PLC)enables real-time data transmission on DC power lines.With traffic adaptation,DC-PLC can be integrated with other complementary media such as 5G to reduce transmission delay and improve reliability.However,traffic adaptation for DC-PLC and 5G integration still faces the challenges such as coupling between traffic admission control and traffic partition,dimensionality curse,and the ignorance of extreme event occurrence.To address these challenges,we propose a deep reinforcement learning(DRL)-based delay sensitive and reliable traffic adaptation algorithm(DSRTA)to minimize the total queuing delay under the constraints of traffic admission control,queuing delay,and extreme events occurrence probability.DSRTA jointly optimizes traffic admission control and traffic partition,and enables learning-based intelligent traffic adaptation.The long-term constraints are incorporated into both state and bound of drift-pluspenalty to achieve delay awareness and enforce reliability guarantee.Simulation results show that DSRTA has lower queuing delay and more reliable quality of service(QoS)guarantee than other state-of-the-art algorithms.
基金supported by the Program of Humanities and Social Science of the Ministry of Education of China(No.24YJA630013)the Natural Science Foundation of Ningbo of China(No.2024J125)the“Innovation Yongjiang 2035”Key R&D Programme(No.2024H032),China。
文摘With the advancement of connected vehicle(CV)technology,an increasing number of CVs will appear on urban roads.Data collected by CVs can be used to optimize signal parameters at intersections,thus improving traffic efficiency.In this study,we design a real-time adaptive signal control method for an arterial road with multiple intersections with low penetration rates.By utilizing vehicle arrival information collected by CVs,our method rapidly determines optimal signal phasing and timing(SPaT).The proposed adaptive signal control method was tested with the Simulation of Urban Mobility(SUMO)software,and was found to reduce total travel delay in the network better than a fixed coordination control method.The performance of the proposed method in reducing travel delay is expected to improve as CV detection range increases.
基金supported by National Natural Science Foundation of China:[Grant Number 61763028]“Innovation Star”Project for Outstanding Graduate Students in Gansu Province:[Grant Number 2021CXZX-515].
文摘Multi-agent Reinforcement Learning(MARL)has become one of the best methods in Adaptive Traffic Signal Control(ATSC).Traffic flow is a very regular traffic volume,which is highly critical to signal control policy.However,dynamic control policies will directly affect traffic flow formation,and it is impossible to provide observation through the original traffic flow prediction.This paper proposes a method for estimating traffic flow according to the time window in Reinforcement Learning(RL)training.Therefore,it is verified on both the regular road network and the real road network.Our method further reduces the intersection delay and queue length compared with the original method.
基金Project supported by the UM High Impact Research MoE Grant from the Ministry of Education,Malaysia(No.UM.C/625/1/HIR/MOHE/FCSIT/09)
文摘The importance of using adaptive traffic signal control for figuring out the unpredictable traffic congestion in today's metropolitan life cannot be overemphasized. The vehicular ad hoc network(VANET), as an integral component of intelligent transportation systems(ITSs), is a new potent technology that has recently gained the attention of academics to replace traditional instruments for providing information for adaptive traffic signal controlling systems(TSCSs). Meanwhile, the suggestions of VANET-based TSCS approaches have some weaknesses:(1) imperfect compatibility of signal timing algorithms with the obtained VANET-based data types, and(2) inefficient process of gathering and transmitting vehicle density information from the perspective of network quality of service(Qo S). This paper proposes an approach that reduces the aforementioned problems and improves the performance of TSCS by decreasing the vehicle waiting time, and subsequently their pollutant emissions at intersections. To achieve these goals, a combination of vehicle-to-vehicle(V2V) and vehicle-to-infrastructure(V2I) communications is used. The V2 V communication scheme incorporates the procedure of density calculation of vehicles in clusters, and V2 I communication is employed to transfer the computed density information and prioritized movements information to the road side traffic controller. The main traffic input for applying traffic assessment in this approach is the queue length of vehicle clusters at the intersections. The proposed approach is compared with one of the popular VANET-based related approaches called MC-DRIVE in addition to the traditional simple adaptive TSCS that uses the Webster method. The evaluation results show the superiority of the proposed approach based on both traffic and network Qo S criteria.
基金Gansu Education Department:[Grant Number 2021CXZX-515]National Natural Science Foundation of China:[Grant Number 61763028].
文摘Realising adaptive traffic signal control(ATSC)through reinforcement learning(RL)is an important means to easetraffic congestion.This paper finds the computing power of the central processing unit(CPU)cannot fully usedwhen Simulation of Urban MObility(SUMO)is used as an environment simulator for RL.We propose a multi-process framework under value-basedRL.First,we propose a shared memory mechanism to improve exploration efficiency.Second,we use the weight sharing mechanism to solve the problem of asynchronous multi-process agents.We also explained the reason shared memory in ATSC does not lead to early local optima of the agent.Wehave verified in experiments the sampling efficiency of the 10-process method is 8.259 times that of the single process.The sampling efficiency of the 20-process method is 13.409 times that of the single process.Moreover,the agent can also converge to the optimal solution.
基金Science&Technology Research and Development Program of China Railway(Grant No.N2021G045)the Beijing Municipal Natural Science Foundation(Grant No.L191013)the Joint Funds of the Natural Science Foundation of China(Grant No.U1934222).
文摘Reinforcement learning-based traffic signal control systems (RLTSC) can enhance dynamic adaptability, save vehicle travelling timeand promote intersection capacity. However, the existing RLTSC methods do not consider the driver’s response time requirement, sothe systems often face efficiency limitations and implementation difficulties.We propose the advance decision-making reinforcementlearning traffic signal control (AD-RLTSC) algorithm to improve traffic efficiency while ensuring safety in mixed traffic environment.First, the relationship between the intersection perception range and the signal control period is established and the trust region state(TRS) is proposed. Then, the scalable state matrix is dynamically adjusted to decide the future signal light status. The decision will bedisplayed to the human-driven vehicles (HDVs) through the bi-countdown timer mechanism and sent to the nearby connected automatedvehicles (CAVs) using the wireless network rather than be executed immediately. HDVs and CAVs optimize the driving speedbased on the remaining green (or red) time. Besides, the Double Dueling Deep Q-learning Network algorithm is used for reinforcementlearning training;a standardized reward is proposed to enhance the performance of intersection control and prioritized experiencereplay is adopted to improve sample utilization. The experimental results on vehicle micro-behaviour and traffic macro-efficiencyshowed that the proposed AD-RLTSC algorithm can simultaneously improve both traffic efficiency and traffic flow stability.
基金This research was supported by NSF Grant CMMI-1749200.
文摘Adaptive Traffic Signal Control(ATSC)adjusts signal timings to real-time traffic measure ments,increasing operational efficiency within a network.However,ATSC is both expen sive to install and operate making it infeasible to deploy at all signalized intersections within a network.This study presents a bi-level optimization framework that applies heuristic methods to identify a limited set of locations for ATSC deployment within an urban network.At the upper-level,the Population Based Incremental Learning(PBIL)algo rithm is employed to generate,evaluate,learn,and update different ATSC configurations.The lower-level uses the delay-based Max-Pressure algorithm to simulate the ATSC config uration within a microsimulation platform.The study proposes improvements to the PBIL algorithm by considering constraints on the maximum number of intersections for ATSC deployment and incorporates prior information about the intersection performance(i.e.,informed search).Simulation results on the traffic network of State College,PA reveal that the proposed PBIL algorithm consistently outperforms baseline methods that select loca tions only based on queue-lengths or delays in terms of reducing overall network travel times.The study also reveals that intersections experiencing the highest delays or longest queues are not always the best candidates for ATSC.Moreover,applying ATSC at all inter sections does not always provide the best performance;in fact,ATSC applied to some loca tions could increase travel times by contributing additional congestion downstream.Additionally,the modified PBIL algorithm with the informed search strategy is more effi cient at identifying promising solutions suggesting it can be readily applied to more gen eralized optimization problems.
文摘Macroscopic Fundamental Diagrams(MFDs)are valuable for designing and evaluating network-wide traffic management schemes.Since obtaining empirical MFDs can be expensive,analytical methodologies are crucial to estimate variations in MFD shapes under different control strategies and predict their efficacy in mitigating congestion.Analyses of urban grid networks'abstractions can provide an inexpensive methodology to obtain a qualitative understanding of impacts of control policies.However,existing abstractions are valid only for simple intersection layouts with unidirectional and single-lane links and two conflicting movement groups.Naturally,the real intersections are more complex,with multiple incoming and outgoing lanes,heterogeneous incoming links'capacities and several conflicting movement groups.To this end,we consider a grid network with differences in capacities of horizontal and vertical directions,allowing us to investigate the characteristics of control policies that can avoid pernicious gridlock in heterogeneous networks.We develop a new,more comprehensive network abstraction of such grid networks to analyze and compare the impacts of two families of decentralized Traffic Signal Controllers(TSCs)on the network's stability.The obtained theoretical insights are verified using microsimulation results of grid networks with multiple signalized intersections.The analyses suggest that considering both upstream and downstream congestion information in deciding signal plans can encourage more evenly distributed traffic in the network,making them more robust and effective at all congestion levels.The study provides a framework to understand general expectations from decentralized control policies when network inhomogeneity arises due to variations in incoming link capacities and turning directions.
文摘Optimization of adaptive traffic signal timing is one of the most complex problems in traffic control systems. This paper presents an adaptive transit signal priority （TSP） strategy that applies the parallel genetic algorithm （PGA） to optimize adaptive traffic signal control in the presence of TSP. The method can optimize the phase plan, cycle length, and green splits at isolated intersections with consideration for the performance of both the transit and the general vehicles. A VISSIM （VISual SIMulation） simulation testbed was developed to evaluate the performance of the proposed PGA-based adaptive traffic signal control with TSP. The simulation results show that the PGA-based optimizer for adaptive TSP outperformed the fully actuated NEMA control in all test cases. The results also show that the PGA-based optimizer can produce TSP timing plans that benefit the transit vehicles while minimizing the impact of TSP on the general vehicles.
