Gaze based intent prediction for human robot collaboration using neural networks

Abstract

Predicting the intentions of humans is very useful for human robot collaboration, since it can enable robots to proactively adapt to future activities of the human. This thesis compares intent prediction performance of different machine learning models based on neural networks that use different ways to encode input data. A dataset with a human picking up objects and placing objects was recorded for this work. It includes three dimensional positional data from motion capture and eye gaze data. Four different representations of the eye gaze, together with object and human skeleton positional data were evaluated and compared by their classification performance for intent prediction in a pick and place scenario. The resulting system can predict which object the human intends to pick up next and where to place it afterwards. This thesis shows the importance of including eye gaze in pick and place intent prediction.