Detecting ambiguity in conversational systems

Abstract

The question of detection of user search queries has been explored by many authors. With the advent of speech based search interfaces, narrowing down the scope of search based on user intent becomes even more important. A prominent part of determining the user's goals is first detecting whether the query is ambiguous, based on which, clarifying questions can be posed. Previous works have mostly attempted to classify user intent into pre-defined categories that may not be suitable for open-domain settings. This thesis explores multiple methods to detect the level of ambiguity of the first query input by the user. Two principal approaches are presented in this work, both of which depend on information provided by documents retrieved from the search operation. The first approach creates a graph based on the similarities between the documents and the second approach generates a graph from the concepts covered in those documents. The graphs are then processed by a graph convolutional network and classified into four levels of ambiguity. The models are tested on data provided by the ClariQ challenge and are found to depend on the documents taken into scope as well as the distribution of the documents in the search results. The best results obtained by the models have been shown to improve over traditional sentence classification approaches and have been compared to the top ranked entries in the challenge. Additionally, ways to improve the datasets and the models have been proposed.