Benchmarking Interactive Behavior Planning for Self-Driving Vehicles using BARK
Tutorial at IROS 2020, Oct. 25, Las Vegas, USA
About the Tutorial
BARK is an open-source benchmarking platform to foster a comparison of approaches developed in the field of interactive behavior planning for self-driving vehicles.
BARK offers a unique approach with behavior models being used exchangeably for prediction, planning, and simulation.
To grow the community around a common benchmarking platform, we want to enable participants to employ BARK within research and industry projects.
Attendees of the tutorial will:
- Understand requirements on benchmarking platforms to systematically develop interactive behavior planners
- Gain hands-on-experience with BARK as benchmarking platform for interactive planners
- Implement and benchmark their own interactive-behavior planner
Topics of interest:
- Behavior Benchmarking
- Autonomous Driving
- Multi-Agent Systems
- Interactive Planning
Target audience:
- PhD students (in robotics, in software engineering) who want to practically learn about cutting edge benchmarking approaches for interactive decision making algorithms. This tutorial provides a good entry point to understand the state-of-the-art in this field. It gives them an understanding of the current challenges and upcoming future research directions. By gaining hands-on-experience with problems in interactive decision making, new researches are encouraged to directly tackle the open problems in this field.
- Experienced robotics researchers from academia and industry who want to get to know a new benchmarking platform for autonomous driving and multi-agent systems tailored to the analysis of interactive decision making algorithms. BARK offers a unique approach with behavior models being used exchangeably for prediction, planning, and simulation. This could attract domain experts providing them a comprehensive framework for research in behavior modeling.
- Practitioners from industry who want to extend their workflow and easily benchmark developed planning approaches against other state-of-the-art models in a variety of standard scenarios.
To take away as much as possible of the tutorial, the participants should have some background in decision making, i.e understand the fundamental principles of decision problems, e.g. MDPs and POMDPs, and have experience with Python and a basic understanding of C++.
Resources
Relevant links giving an impression on the background on which this tutorial is based:
Contact
Please email us at bernhard(at)fortiss.org if you have any comments and feedback.
News:
(Last update June 5th)
- June 5th 2020: Tutorial website available
Program
The tutorial mixes slide presentations and interactive sessions.
A short spot light talk will provide hands on knowledge about industry research.
| Duration | Content | Type | Speaker |
|---|---|---|---|
| 5 mins | Tutorial Overview | Slides | Julian Bernhard |
| 15 mins | Requirements on benchmarking environments for interactive behavior generation, BARK in a nutshell, Example Use Cases of BARK | Slides | Julian Bernhard |
| 40 mins | Industry-Research in interactive behavior generation | Sides | Konstantin Hubmann from Toyota Research Institute (TRI), USA |
| 5 mins | BARK Installation and Setup | Hands-on | Patrick Hart |
| 15 mins | Development of a simple behavior planner in BARK | iPython Notebook | Patrick Hart |
| 20 mins | Scenario generation with BARK | iPython Notebook | Julian Bernhard |
| 20 mins | Benchmarking process with BARK | iPython Notebook | Klemens Esterle |
| 5 mins | INTERACTION dataset with BARK | iPython Notebook | Tobias Kessler |
Guests Speakers:
Constantin Hubmann
Constantin Hubmann received the bachelor's and master's degree in electrical engineering and information technology from Technische Universität München (TUM), München, Germany, in 2012 and 2014, respectively. He pursued his doctoral degree with the Chair of Measurement and Control Engineering, Karlsruher Institut für Technologie, Karlsruhe, Germany and in cooperation with the BMW Group. Subsequently, he joined the Toyota Research Institute (TRI) in Los Altos, USA, as a research scientist. His research interests include belief state planning and decision making under uncertainty for autonomous driving. He recently became very interested in learning from demonstrations.
About BARK
Predicting and planning interactive behaviors in complex traffic situations presents a challenging task.
Especially in scenarios involving multiple traffic participants that interact densely, autonomous vehicles still struggle to interpret situations and to eventually achieve their own driving goal.
BARK is a semantic simulation framework for autonomous agents with a special focus on autonomous driving.
Its behavior model-centric design allows for the rapid development, training, and benchmarking of various decision-making algorithms.
In BARK, behavior models are (re-)used for planning, prediction, and simulation.
A range of models is currently available, ranging from simplistic car-following to rule-based intersection crossing models and models derived from the planning community such as Monte-Carlo Tree Search and Reinforcement Learning-based behavior models.
In addition to that, BARK provides an easy to use benchmarking process to compare various behavior models based on using BARK's data-based or sampling-based scenario generation.
BARK supports the replay of the Interaction Dataset with the possibility to (partially) exchange recorded behavior with interactive models.
Coverage-based sampling criteria are supported not only over physical scenario properties such as velocity and vehicle distance ranges but also over hidden behavior model parameters.
Organizers
Prof. Alois Knoll
Since 2001, Alois C. Knoll has been a professor of Computer Science at the Department of Informatics of the Technische Universität München. His research interests include cognitive, medical and sensor-based robotics, multi-agent systems, data fusion, adaptive systems, multimedia information retrieval, model-driven development of embedded systems with applications to automotive software and electric transportation, as well as simulation systems for robotics and traffic.
In these fields, he has published over 800 technical papers and guest-edited international journals.
He was the coordinator of the EU-Project ECHORD++, a large initiative for bringing together robotics industry, research institutes and universities with the aim of advancing new robot technologies to market-readiness. He is currently one of the directors of the EU Human Brain Project, (and head of the section on Neurorobotics), which runs from 2013 – 2023, with a total expected project budget of 1 billion EUR, of which ~120 million EUR are expected to go to the Neurorobotics Platform. He has also coordinated many other nationally and internationally funded projects. He is a senior member of the IEEE, a member of the German Society of Informatics and a Fellow of the School of Engineering of the University of Tokyo (2010).
Julian Bernhard
Julian Bernhard is a PhD student at TUM and fortiss. His research focuses on risk-constrained interactive motion planning under behavioral uncertainty. He studied Electrical Engineering at the Technische Universität München with focus on machine learning, control theory and signal processing and graduated with the Master of Science in 2014. Afterwards, he worked as a consulting engineer in the field of signal processing. In 2017, he joined fortiss as a staff researcher in the competence field autonomous systems and sensor systems and started his PhD. He gained experience in autonomous driving during industry cooperations for autonomous valet parking, during working on autonomous vehicle prototypes and is one of the initiators and main developers of BARK.
Klemens Esterle
Klemens Esterle is a PhD student at TUM and fortiss. His research focuses on combining formal methods to model multi-agent constraints from traffic rules with interactive behavior planning. He studied Mechatronics Engineering at the Technische Universität Dresden, graduating in 2014. Afterwards, he worked as a simulation engineer in the field of systems engineering. In 2017, he joined fortiss as a staff researcher in the competence field autonomous systems and sensor systems and started his PhD. He gained experience in autonomous driving during industry cooperations for autonomous valet parking, prediction and simulation. He is one of the initiators and main developers of BARK.
Patrick Hart
Patrick Hart is a PhD student at the Technical University of Munich and fortiss. His main research interest is how agents can learn complex behaviors in uncertain environments. He obtained his Master's degree from the Karlsruhe Institute of Technology and wrote his master thesis at the FZI Karlsruhe about search-based motion planning for autonomous vehicles in 2016. In 2017, he then joined the autonomous systems group at fortiss and began to pursue his PhD. At fortiss, he gained valuable insights in the field of autonomous driving working on autonomous valet parking, autonomous vehicles, and on simulation. He is the initiator of BARK machine learning (BARK-ML) and one of the initiators and main developers of BARK.
Tobias Kessler
Tobias works as a scientist at fortiss and is a PhD student at the Technical University of Munich. His main research focus is planning cooperative and interactive behaviors for autonomous vehicles alongside with a smooth and safe motion using optimization-based methods such as mixed-integer programming. He also maintains the autonomous driving stack, based on Apollo, of the institute’s vehicle prototype. He is one of the initiators and developers of the simulation tool BARK and has an industrial background in vehicle dynamics simulation. Tobias holds a Master’s degree in Mathematics from the Technical University of Munich.
