For internal staff only
Abstract: The ability to build an actionable representation of the environment of a robot is crucial for autonomy and prerequisite to a large variety of applications, ranging from home, service, care and consumer robots to industrial inspection and maintenance to disaster response and defense missions. Notably, a large part of the promise of autonomous robots depends on long-term operation in domains shared with humans and other agents. These environments are typically highly complex, semantically rich, and immensely dynamic with agents regularly moving through and interacting with the scene.
This talk presents an autonomy pipeline to address these challenges. We first present methods for detection and representation complex semantics, short-term motion, and long-term changes in dense robot maps, and show how these challenges can be addressed jointly and in real-time in a unified framework we call Khronos. We show how Dynamic Scene Graphs (DSGs) can represent semantic symbols in a task-driven fashion and facilitate the prediction of likely future evolutions of the scene based on the data the robot has already collected. Lastly, we show how robots as embodied agents can leverage our actionable scene representations and predictions to complete tasks such as actively gathering data that helps them improve their world models and perception capabilities over time. The presented methods are demonstrated on-board fully autonomous aerial and ground robots, run in real-time on the limited hardware available, and are available as open-source software.
Bio: Lukas Schmid is currently a postdoctoral research fellow at the MIT SPARK Lab led by Prof. Luca Carlone at the Massachusetts Institute of Technology. Before that, he briefly was a postdoctoral researcher at the Autonomous Systems Lab (ASL) led by Prof. Roland Siegwart at ETH Zürich. He earned his PhD in 2022 from ETH Zurich advised by Prof. Roland Siegwart, Cesar Cadena, and Juan Nieto, where he also obtained his M.Sc. in Robotics, Systems, and Control (RSC) in 2019. During his PhD, he was a visiting researcher at the Microsoft Mixed Reality and AI Lab led by Prof. Marc Pollefeys.
His work has been recognized with the ETH Medal for outstanding PhD Theses, the ETH Medal for outstanding Master Theses, the Willi Studer Prize for the best M.Sc. graduate of the year, the first place in the 2024 "Nothing Stands Still" Hilti SLAM challenge, and a Swiss National Science Foundation postdoc-mobility career award.
His research focuses on active and passive perception of complex, dynamic, and changing environments for robotic interaction and augmented reality. This includes research on dense geometric and semantic scene representations, scene abstraction, and understanding. Second, he extends these map representations to model challenging and dynamic environments by recognizing short-term moving and long-term changing entities, predicting spatial and temporal changes, and building systems that continuously learn and adapt over time. Lastly, he works on utilizing these world representations and predictions for active perception and interaction planning. This includes autonomously building such representations in unknown environments and actively adapting to new scenes. He demonstrates these novel concepts through implementation on-board compute-constrained autonomous mobile robots in interactive real-time missions.
Website: schmluk.github.io