Advanced Topics in Reinforcement Learning and Planning

Course type

Seminar

Study programs

• Master Informatik
• Master Data Science
• Master Software Systems Engineering
• Master Simulation Sciences
• Master Computational Engineering Science
• Master Human-Centered Intelligent Systems

Content

Students will present papers from a list of important recent works in machine learning and reasoning compiled by the professor. Topics include deep and reinforcement learning, transformer and GNN architectures, and more. The focus will be in the use of these techniques in simple applications that are crisp and easy to understand. The list of papers will also include those underlying well-known learning systems like as Dalle-E, ChatGPT, Alphazero, Gato, etc.

Recommended prior knowledge

Bachelor degree in CS or equivalent and basic AI and ML courses, with some previous exposure to deep learning and reinforcement learning.

Topics

Planning, Hierarchical Search and Skills

• MaestroMotif: Skill Design from Artificial Intelligence Feedback
  M Klissarov, M Henaff, R Raileanu, S Sodhani, P Vincent, A Zhang, PL Bacon, D Precup, MC Machado, P D’Oro
  2025, ICLR

• Prompting Decision Transformers for Zero-Shot Reach-Avoid Policies
  K Li, M Zitnik
  2025, arXiv

Algorithmic Reasoning and Out-of-Distribution Generalization

• Neural Algorithmic Reasoning Without Intermediate Supervision
  G Rodionov, L Prokhorenkova
  2023, NeurIPS

• Unlocking Out-of-Distribution Generalization in Transformers via Recursive Latent Space Reasoning
  A Altabaa, S Chen, J Lafferty, Z Yang
  2025, arXiv

• Simulation of Graph Algorithms with Looped Transformers
  AB de Luca, K Fountoulakis
  2024, arXiv

• Which Algorithms Can Graph Neural Networks Learn?
  S Wittig, A Vasileiou, RR Nerem, T Stoll, F Geerts, Y Wang, C Morris
  2026, arXiv

Expressivity and Length Generalization: Logic and Transformer Programs

• Expressive Power of Graph Transformers via Logic
  V Ahvonen, M Funk, D Heiman, A Kuusisto, C Lutz
  2026, AAAI

• Barriers to Universal Reasoning With Transformers (And How to Overcome Them)
  O Kraus, Y Sarrof, Y Yao, A Koller, M Hahn
  2026, arXiv

World Models: Object-Centric, Abstract and Action Models

• Identifiable Token Correspondence for World Models
  Y Kim, R Sun, I Kim, B Park, HO Song
  2026, ICML
  Code

• Neurosymbolic Action Model Learning under Partial Observability
  A Kikaj, L De Smet, G Marra, L De Raedt
  2026, arXiv

Questions?

If you have any questions, please contact Ulzhalgas Rakhman.