05 Fakultät Informatik, Elektrotechnik und Informationstechnik
Permanent URI for this collectionhttps://elib.uni-stuttgart.de/handle/11682/6
Browse
Search Results
Item Open Access A visual analytics approach for explainability of deep neural networks(2018) Kuznecov, PaulDeep Learning has advanced the state-of-the-art in many fields, including machine translation, where Neural Machine Translation (NMT) has become the dominant approach in recent years. However, NMT still faces many challenges such as domain adaption, over- and under-translation, and handling long sentences, making the need for human translators apparent. Additionally, NMT systems pose the problems of explainability, interpretability, and interaction with the user, creating a need for better analytics systems. This thesis introduces NMTVis, an integrated Visual Analytics system for NMT aimed at translators. The system supports users in multiple tasks during translation: finding, filtering and selecting machine-generated translations that possibly contain translation errors, interactive post-editing of machine translations, and domain adaption from user corrections to improve the NMT model. Multiple metrics are proposed as a proxy for translation quality to allow users to quickly find sentences for correction using a parallel coordinates plot. Interactive, dynamic graph visualizations are used to enable exploration and post-editing of translation hypotheses by visualizing beam search and attention weights generated by the NMT model. A web-based user study showed that a majority of participants rated the system positively regarding functional effectiveness, ease of interaction and intuitiveness of visualizations. The user study also revealed a preference for NMTVis over traditional text-based translation systems, especially for large documents. Additionally, automated experiments were conducted which showed that using the system can reduce post-editing effort and improve translation quality for domain-specific documents.Item Open Access Neural-based methods for user simulation in dialog systems(2018) Schmidt, MaximilianSpoken Dialog Systems allow users to interact with a Dialog Manager (DM) using natural language, thereby following a goal to fulfill their task. State-of-the-art solutions cast the problem as Markov Decision Process, leveraging Reinforcement Learning (RL) algorithms to find an optimal dialog strategy for the DM. For this purpose, several thousand dialogs need to be seen by the RL agent. A user simulator comes in handy to generate responses on demand, however the current state-of-the-art agenda-based user simulators lack the ability to model real human subjects. In this thesis, this problem is addressed by implementing a user simulator using a Recurrent Neural Network which approximates the agenda-based model in a first step. Going onwards, it is shown to learn noise and variance treated as varying user behavior. This is used to train the simulator on real data thus modeling real users.Item Open Access Exploring simplified subtitles to support spoken language understanding(2018) Angerbauer, KatrinUnderstanding spoken language is a crucial skill we need throughout our lives. Yet, it can be difficult for various reasons, especially for those who are hard-of-hearing or just learning to speak a language. Captions or subtitles are a common means to make spoken information accessible. Verbatim transcriptions of talks or lectures are often cumbersome to read, as we generally speak faster than we read. Thus, subtitles are often edited to improve their readability, either manually or automatically. This thesis explores the automatic summarization of sentences and employs the method of sentence compression by deletion with recurrent neural networks. We tackle the task of sentence compression from different directions. On one hand, we look at a technical solution for the problem. On the other hand, we look at the human-centered perspective by investigating the effect of compressed subtitles on comprehension and cognitive load in a user study. Thus, the contribution is twofold: We present a neural network model for sentence compression and the results of a user study evaluating the concept of simplified subtitles. Regarding the technical aspect 60 different configurations of the model were tested. The best-scoring models achieved results comparable to state of the art approaches. We use a Sequence to Sequence architecture together with a compression ratio parameter to control the resulting compression ratio. Thereby, a compression ratio accuracy of 42.1 % was received for the best-scoring model configuration, which can be used as baseline for future experiments in that direction. Results from the 30 participants of the user study show that shortened subtitles could be enough to foster comprehension, but result in higher cognitive load. Based on that feedback we gathered design suggestions to improve future implementations in respect to their usability. Overall, this thesis provides insights on the technological side as well as from the end-user perspective to contribute to an easier access to spoken language.Item Open Access Deep reinforcement learning in dialog systems(2018) Väth, DirkThis thesis explores advanced deep reinforcement learning methods for learning dialog policies. While many recent contributions in the area of reinforcement learning focus on learning how to play Atari games, this thesis applies them in a real-world scenario. When talking to a dialog system, the dialog policy is the component which chooses the response based on the history of the interaction between user and system. Nowadays, dialog policies may be learned automatically by training a reinforcement learning agent with a user simulator. In this thesis, a baseline method for dialog policy learning is implemented and extended by various state-of-the art deep reinforcement learning methods. An ablation study discusses the significance of each extension, highlighting beneficial and harmful additions. Each extended agent is shown to perform better than the baseline method with all agents outperforming policies from an existing benchmark. Two agents even prove to be on par with handcrafted dialog policies. Along with the quantitative evaluation, qualitative results are provided in the form of chats between a user and a trained agent.Item Open Access How word-embedding methods improve information extraction and can be used for multilingual approaches(2018) Zendler, UlrichExpanding entity sets and extracting relations are key tasks in natural language processing (NLP), which is accomplished in various approaches. Recent successful attempts are all using word-embeddings like the ones presented by Mikolov et al. While most work concentrates on how to improve these tasks in general without considering a specific domain, it is of interest how to achieve even higher precisions when focusing on a specific domain and optimizing the methods towards a single purpose. Therefore this thesis suggests methods and adjustments to optimize the proposals for entity set expansion for the domain of drugs. While this is the main purpose of this thesis, it will also present a novel idea, how to improve the precision in relation extraction by using word-embeddings, which could be combined with existing successful relation extraction methods. And finally another key aspect of many international companies is tagged, by presenting a solution for multilingual information extraction system (IES), which is capable of preprocessing text of multiple languages, expanding entity sets independent of the language used and extracting relations on the texts.Item Open Access Machine question answering with attention-based convolutional neural networks(2018) Blohm, MatthiasThe task of answering textual questions with the help of deep learning techniques is currently an interesting challenge. Although promising results have been achieved in previous works, these approaches leave much room for further considerations and improvements. This thesis deals with the question, how a system can be realized, which is able to capture and process textual contents, and to draw the right conclusions for answering multiple-choice questions with the help of modern methods. For this, current techniques such as convolutional neural networks and attention mechanisms are used and tested on the benchmark datasets MovieQA, WikiQA and InsuranceQA, three corpora with question-answer entries from the domains movies, Wikipedia resp. insurances, each with a slightly different task. The implementation is done using the framework TensorFlow; For the representation of the textual content, pre-trained word vectors of the tool GloVe are used. In addition to improving the system, this work also aims to analyze and evaluate its learning behavior. This is done with the aid of so-called adversarial examples, where by modifying textual context information it is checked whether the neural network concentrates on the correct content when answering a question, and at which degree of manipulation a successful performance of the neural network gets impossible. At the same time, the limitations of such text comprehension systems are shown, which are often able to compare text sequences, but do not develop a deeper understanding of the meaning and content of the inputs. The text comprehension system created in this work achieves a new state-of-the-art for MovieQA with an accuracy of 82.73% correctly answered questions.