Controlling Complex Dynamic Transportation Systems: Development and Adaptation of a Novel Distributed Cooperative Multi-Agent Learning Technique

Intelligent transportation systems continue to increase complexity, scale, and scope as more devices contain embedded compute. Cooperation among vehicles, intersections, and other members of the greater traffic ecosystem at a system-of-systems level is critical to improving the efficiency of the multi-billion-dollar asset that is the U.S. roadway infrastructure. This work introduces a negotiations strategy among multi-agent reinforcement learning agents and applies this to both traffic signal control and supervisory control of vehicle platooning. The traffic signal control implementation builds off of many prior research thrusts, and was shown to improve vehicle throughput by an average of 671veh/hr over actuated traffic under static arrival conditions. Additionally, the negotiations strategy saved 9.8sec and a maximum of 14.3sec in total travel time for a probe vehicle traversing the intersection system under static conditions. The realized efficiency increases resulted in an emissions reduction of 809kg/hr. In the case of platooning, the presented framework departs from the bulk of research, which tends to focus on the safety-critical and fine control of platooned vehicle motion; instead, this work chooses to approach platooning from a supervisory angle. Under the guidance of negotiated supervisory control, the maximum observed fuel savings was 20.8% in a low-traffic-density scenario over the nominal case where no vehicles are directed to platoon and closer to 7.2% when traffic is calibrated to match US highway 101. In each of these cases, the differences in mean speed between legacy driven vehicles and the vehicles supervised by the negotiations was < 5m/s. While the underlying representations for each of these systems is necessarily different. The results suggest that approaching the increasingly well-instrumented and controlled network of roadways with supervisory level controls yields an improvement in performance. The extension of this approach and future works are also addressed.