Ameet Talwalkar (CMU and Datadog)December 9, 2025 Title and AbstractGeometric perspective on reward identifiability and transferability in inverse reinforcement learning Inverse reinforcement learning (IRL) seeks to infer a reward function from expert demonstrations. A core challenge in this setting is reward ambiguity: many rewards can explain the same demonstrations, yet only some transfer to new environments. We introduce a convex-analytic view that ties identifiability and transferability to the geometry of the set of occupancy measures. The key idea is that both hinge on how potential-shaping subspaces—the normals to sets of occupancy measures—intersect. In the infinite-data limit, this recovers established rank conditions. With finite data, we show that identifiability and transferability are governed by the principal angles between these subspaces, providing a more refined measure of similarity between transition laws. This geometric framework unifies and significantly generalizes previous results while offering an intuitive picture of when rewards are identifiable and transferable. BioAndreas Schlaginhaufen is a 4th-year PhD student at EPFL, advised by Prof. Maryam Kamgarpour. His research focuses on the theory and algorithms of learning from demonstrations and preferences, with broader interests in multi-agent systems and applications to robotics |