Hierarchical adversarial imitation learning from motion capture data

Abstract

A lot of problems in Imitation Learning can be split into multiple low-level tasks that need to be executed in sequence or parallel. Likewise in the area of human motion prediction, the actions of the human are frequently defined by a combination of linked subgoals. However, recent work often focuses on solving the problem statement as a single task or lies its attention on the general accuracy of the predicted motion. In this thesis, a two-step framework is introduced which is able to execute a high-level policy using low-level tasks. We implement different network structures for goal-conditioned human motion prediction by including goal loss regularization and adversarial methods, and demonstrate that they can be combined for long-term planning using Maximum Entropy Inverse Reinforcement Learning on a simplified model of the environment. The framework is tested on a simple pick and place task in simulation, using recorded demonstrations of multiple humans executing the higher-level task. Results show, that the trained networks are capable of reaching subgoals with great accuracy and can be used in combination with the presented Inverse Reinforcement Learning algorithm to learn the long-term objective.