Event

LM Robotics Seminar: Reinforcement Learning with Temporal Logic Specifications

Friday, November 12, 2021
2:00 p.m.
2121 JMP

Reinforcement Learning with Temporal Logic Specifications

Derya Aksaray

Assistant Professor
Aerospace Engineering and Mechanics
University of Minnesota, Twin Cities

Streaming Link: 

Abstract

Dynamical systems such as drones, mobile robots, or autonomous cars are envisioned to achieve complex specifications which may include spatial (e.g., regions of interest), temporal (e.g., time bounds), and logical (e.g., preconditions, concurrency among tasks) requirements. As the specifications get more complex, representing them via algebraic equations becomes harder. Alternatively, such specifications can be compactly expressed and used in control synthesis by utilizing the framework of temporal logics. In this talk, I will address the problem of learning optimal control policies for satisfying temporal logic (TL) specifications in the face of uncertainty. Standard reinforcement learning (RL) algorithms, which aim to maximize the expected sum of discounted rewards, are not directly applicable when the goal is to generate trajectories that optimally satisfy a TL specification. To overcome this limitation, I will formulate an approximate problem that can be solved via standard RL algorithms such as Q-learning and present the suboptimality bound of the proposed solution. Then, I will consider the case where a TL specification is given as the constraint rather than the objective and present a novel approach for satisfying the TL constraint with a desired probability throughout the learning process. I will motivate this part by multi-use of autonomous systems, e.g., a drone executing a pick-up and delivery mission as its primary task (constraint) while learning to fly over regions of interest (aerial monitoring) as its secondary task (objective).

Biography

Derya Aksaray is currently an Assistant Professor in the Department of Aerospace Engineering and Mechanics at the University of Minnesota (UMN). Before joining UMN, she held post-doctoral researcher positions at the Massachusetts Institute of Technology from 2016- 2017 and at Boston University from 2014-2016. She received her Ph.D. degree in Aerospace Engineering from the Georgia Institute of Technology in 2014. Her research interests lie primarily in the areas of control theory, formal methods, and machine learning with applications to autonomous systems and aerial robotics.

Host

Yancy Diaz-Mercado

Contact: appicard@umd.edu