Toward Robust Manipulation in Complex Environments
Recent advances in deeplearning and GPU-based computing have enabled significant progress in several areas of robotics, including navigation, visual recognition, and object manipulation. This progress has turned applications such as autonomous driving and delivery tasks in warehouses, hospitals, or hotelsinto realistic application scenarios. However, robust manipulation in complex settings is still an open research problem. Various efforts at NVIDIA robotics research investigate how deep learning along with physics-based and photo-realistic simulation can be used to train manipulators in virtual environments and then deploy them in the real world. Our work shows promising results on different pieces of the manipulation puzzle, including manipulator control, touch sensing, object pose detection, and object pick and place. In this talk, I will present some of these advances. I will describe a robot manipulator that can open and close cabinet doors and drawers, detect and pickup objects, and move these objects to desired locations. Our baseline system is designed to be applicable in a wide variety of environments, only relying on 3D articulated models of the kitchen and the relevant objects. I will discuss lessons learned so far, and various research directions toward enabling more robust and general manipulation systems that do not rely on existing models.
Dieter Fox is Senior Director of Robotics Research at NVIDIA and Professor in the Paul G. Allen School of Computer Science & Engineering at the University of Washington, where he heads the UW Robotics and State Estimation Lab. Dieter obtained his Ph.D. from the University of Bonn, Germany. His research is in robotics and artificial intelligence, with a focus on state estimation and perception applied to problems such as mapping, object detection and tracking, manipulation, and activity recognition. He has published more than 200 technical papers and is the co-author of the textbook “Probabilistic Robotics”. He is a Fellow of the IEEE, AAAI, and ACM, and recipient of the 2020 Pioneer in Robotics and Automation Award. Dieter also received several best paper awards at major robotics, AI, and computer vision conferences. He was an editor of the IEEE Transactions on Robotics, program co-chair of the 2008 AAAI Conference on Artificial Intelligence, and program chair of the 2013 Robotics: Science and Systems conference.