Sim2Real Transfer Learning for Long Horizon Robotic Manipulation Tasks

Advisors: Daniel Swoboda, Till Hofmann

Supervisor: Prof. Hector Geffner

Level: Master’s Thesis

In imitiation learning, a robot learns to execute a task by replicating and generalizing previously recorded demonstrations. This can be done from human demonstration or by using a trajectory planner. Typically, imitation learning focuses on learning in simulation and it has been used successfully for learning tasks from natural language input [Shridhar et al., 2023] and input from task planners [Swoboda, 2024]. However, transferring the learned policies from simulation to reality is a challenging task (e.g., [Höfer et al., 2021]).

The goal of this thesis is to extend our approach for learning skill execution in simulation [Swoboda, 2024] to a real-world application. Your tasks will involve adapting existing data collection to generate annotated demonstrations and collecting symbolic descriptions of the environment for learning domain grounding from real world examples. You’ll adapt and evaluate a transformer-based neural network architecture on a variety of tasks, assessing its performance and reliability, and its generalization capabilities in both simulation and real-world scenarios.

In addition to the academic aspect, this thesis project offers practical and networking experiences in connection with the RWTH Speed Funds initiative. You’ll have the opportunity to present your results to peers in an academic setting. If you’re a motivated, research-oriented student with an interest in robotics and machine learning, we are keen to hear from you. This thesis is scheduled to start in October 2024.

Goals

1. Analyze existing transfer learning approaches and evaluate their suitability in our setting
2. Implement transfer learning, extending our imitation learning approach
3. Evaluate the approach in real-world household robotics scenario

What we expect:

• Good Python skills
• Experience with Linux and ROS/ROS2
• Good knowledge of Deep Neural Networks, Imitation Learning, etc.
• Experience with Robotics and Control

What we offer:

• A fully equipped robotics laboratory with a Hello Robot Stretch 3 domestic robot and possibly a 7-DOF robotics manipulator
• Good coffee and Club Mate
• A team of students and researchers dedicated to robotics

Interested?

Contact Daniel Swoboda.

References

[1] S. Höfer et al., “Sim2Real in Robotics and Automation: Applications and Challenges,” IEEE Transactions on Automation Science and Engineering, vol. 18, no. 2, pp. 398–400, Apr. 2021, doi: 10.1109/TASE.2021.3064065.

[2] D. Swoboda, “Learning skill execution and domain grounding for integrated robot task and motion planning,” Master’s Thesis, RWTH Aachen University, 2024.

[3] M. Shridhar, L. Manuelli, and D. Fox, “Perceiver-actor: a multi-task transformer for robotic manipulation,” in Proceedings of The 6th Conference on Robot Learning, PMLR, Mar. 2023, pp. 785–799. Accessed: Apr. 17, 2023. [Online]. Available: https://proceedings.mlr.press/v205/shridhar23a.html