Eugene Vinitsky

I’m a graduate student in Controls at UC Berkeley working with Prof. Alexandre Bayen. I'm interested in how we can use learning algorithms and control theory to find good policies for cooperative agents. I'm particularly motivated by the sudden proliferation of partially automated vehicles on our roadways: this presents an good venue to study robot-human interaction in both theory and practice. This is an opportunity to find ways for humans and robots to collaborate, akin to the brief era of Centaur Chess, when human-computer chess teams sharply outplayed either human or computer. Humans are "bad" drivers, both from a safety and efficiency perspective, and autonomous vehicles can be used to not only replace human driving, but also to encourage more effective human driving.

In particular, I’ve focused on

  • Using multi-agent reinforcement learning to help autonomous vehicles mitigate poor human driving patterns
  • Transferring these controllers from simulator to reality
  • Developing open-source tools to allow researchers to study the interaction of autonomous vehicles and humans
I'm also one of the primary developers of Flow, a library for benchmarking and training autonomous vehicle controllers.

Email  /  CV  /  Google Scholar  /  LinkedIn

profile photo
News
  1. I'm one of the workshop hosts for the Workshop on Lagrangian Control for Traffic Flow Smoothing in Mixed Autonomy Settings at CDC 2019 in Nice. Come by!
  2. Our work on developing new benchmarks for traffic control was covered in Science.
Research

I'm interested in multi-agent reinforcement learning, transportation, controller design, and finding ways to port controllers from sim to real. Most of my research can be seen as figuring out the specifics of how to apply techniques from domains like control and ML to bettering transportation systems. Papers are roughly in order of relation to current research.

Lagrangian Control through Deep-RL: Applications to bottleneck decongestion
Eugene Vinitsky, Kanaad Parvate , Aboudy Kreidieh, Cathy Wu , Alexandre Bayen
Intelligent Transportation Systems Conference, 2018  
Code

We introduce an autonomous vehicle (AV) based alternative to ramp metering to improve transportation networks. Ramp metering is the standard technique for maximizing the outflow of traffic bottlenecks but is expensive to maintain. Instead, we can take advantage of readily available cruise controllers to optimize the system. Using reinforcement learning, we design controllers that even at at a low penetration rate of 10%, are able to improve the outflow of a small model of the San-Francisco Oakland Bay Bridge.

Benchmarks for reinforcement learning in mixed-autonomy traffic
Eugene Vinitsky, Aboudy Kreidieh, Luc Le Flem, Nishant Kheterpal, Kathy Jang, Cathy Wu, Fangyu Wu, Richard Liaw, Eric Liang, Alexandre Bayen
Conference on Robot Learning, 2018  
Code / BAIR Blog Post / Coverage in Science

Benchmarks are an essential part of algorithmic computer science/control, making it easy to rank algorithms and control schemes. To remedy the lack of benchmarks in intelligent traffic/autonomous vehicle control we release a set of four new benchmarks. These cover intersections, on-ramp merges, traffic light control, and bottleneck control. We benchmark four standard deep RL algorithms on these tasks and open-source our benchmarks to enable to the community to test their controls against our results.

Zero-Shot Autonomous Vehicle Policy Transfer: From Simulation to Real-World via Adversarial Learning
Behdad Chalaki, Logan Beaver, Benjamin Remer, Kathy Jang, Eugene Vinitsky, Alexandre Bayen, Andreas Malikopoulos
Submission to ICRA, 2019  
project page / code

We train an adversary to perturb the states and action spaces of our controller, yielding a controller that is robust to the sim to real gap. we transfer a controller from a simulator to a minicity that is able to efficiently control traffic through a roundabout under variable inflows.

Simulation to scaled city: zero-shot policy transfer for traffic control via autonomous vehicles
Kathy Jang, Eugene Vinitsky, Behdad Chalaki, Benjamin Remer, Logan Beaver, Andreas Malikopoulos, Alexandre Bayen
ICCPS, 2019  
project page / code

We train a vehicle to bring a platoon of vehicles efficiently through a roundabout in a simulator. By adding appropriate Gaussian noise to the state and action space, the controller transfers directly from the simulator with no loss.

On the Approximability of Time Disjoint Walks
Alexandre Bayen, Jesse Goodman, Eugene Vinitsky
COCOA, 2018  
arxiv

We introduce a new combinatorial optimization problem: Time Disjoint Walks. This is the natural combinatorial problem that arises when you try to route autonomous vehicles through a network without colissions. We show that for standard DAGs the resulting problem is APX hard and provide tight bounds on the performance of a greedy algorithm.

Emergent Behaviors in Mixed-Autonomy Traffic
Cathy Wu, Aboudy Kreidieh, Eugene Vinitsky, Alexandre Bayen
Conference on Robot Learning, 2017  
Code \ Project site

We demonstrate that in a variety of settings with mixed human and autonomous vehicles, interesting and unexpected behaviors can emerge. We also introduce the notion of a state equivalence class, a permutation invariant ordering of the inputs, that drastically improves the sample complexity of the RL algorithms.

Flow: Architecture and Benchmarking for Reinforcement Learning in Traffic Control
Cathy Wu , Aboudy Kreidieh, Kanaad Parvate , Eugene Vinitsky, Alexandre Bayen
Code

This paper marked the release of Flow. Flow is a traffic control benchmarking framework. It provides a suite of traffic control scenarios (benchmarks), tools for designing custom traffic scenarios, and integration with deep reinforcement learning and traffic microsimulation libraries. Flow makes pythonic development of traffic control easy and aims to ease the process of studying mixed-autonomy traffic systems.

Blog Posts
  1. Autonomous Vehicles for Social Good: Berkeley AI Blog
  2. A quick writeup of a lifetime of reading textbooks: Thoughts on Better Textbooks
Service
flowgo Graduate Student Instructor and Course Co-creator, EE290O, Fall 2018

Prof. Bayen, Prof. Wu, Yashar Zeynali, Aboudy Kriedieh, and I put together a course on the use of deep multi-agent reinforcement learning for the study of transportation systems. The lecture notes and homeworks are available above.



All source code stolen from this lovely guy source code,