Predicting user intent during teleoperation using neural networks

Abstract

The goal of the thesis is to develop a method to predict user intent during robot teleoperation using machine learning methods. Teleoperation refers to controlling a robot through an interface when the robot is located distant from the user. Traditionally the robot can be teleoperated by giving commands that are directly mapped to robot actions. However, this can lead to intensive workload for the human operator and can be prone to operational mistakes. Teleoperation can be mitigated by combining user control and robot autonomy, where the robot can be semi-autonomously operated towards mid-level goals. Here, predicting the user intent can increase the performance of the robot by allowing the robot to anticipate and disambiguate user commands. In this thesis, we develop methods to predict the user intent while the user is controlling the robot using hand motion. In addition to predicting the intent, we model the uncertainty of how confident the robot is towards the prediction. We consider robot reaching tasks where human intent is represented as the object to be grasped or the final grasping position on the surface of the object. We use neural networks to predict the intent and estimate the uncertainty using Beta distribution and mixture multivariate Gaussian distribution and further learning the grasp densities. The positive results show that we can predict the user intent before the robot grasps and can be utilized in shared autonomy scenarios to provide better assistance.