05 Fakultät Informatik, Elektrotechnik und Informationstechnik
Permanent URI for this collectionhttps://elib.uni-stuttgart.de/handle/11682/6
Browse
Search Results
Item Open Access Models for data-efficient reinforcement learning on real-world applications(2021) Dörr, Andreas; Toussaint, Marc (Prof. Dr.)Large-scale deep Reinforcement Learning is strongly contributing to many recently published success stories of Artificial Intelligence. These techniques enabled computer systems to autonomously learn and master challenging problems, such as playing the game of Go or complex strategy games such as Star-Craft on human levels or above. Naturally, the question arises which problems could be addressed with these Reinforcement Learning technologies in industrial applications. So far, machine learning technologies based on (semi-)supervised learning create the most visible impact in industrial applications. For example, image, video or text understanding are primarily dominated by models trained and derived autonomously from large-scale data sets with modern (deep) machine learning methods. Reinforcement Learning, on the opposite side, however, deals with temporal decision-making problems and is much less commonly found in the industrial context. In these problems, current decisions and actions inevitably influence the outcome and success of a process much further down the road. This work strives to address some of the core problems, which prevent the effective use of Reinforcement Learning in industrial settings. Autonomous learning of new skills is always guided by existing priors that allow for generalization from previous experience. In some scenarios, non-existing or uninformative prior knowledge can be mitigated by vast amounts of experience for a particular task at hand. Typical industrial processes are, however, operated in very restricted, tightly calibrated operating points. Exploring the space of possible actions or changes to the process naively on the search for improved performance tends to be costly or even prohibitively dangerous. Therefore, one reoccurring subject throughout this work is the emergence of priors and model structures that allow for efficient use of all available experience data. A promising direction is Model-Based Reinforcement Learning, which is explored in the first part of this work. This part derives an automatic tuning method for one of themostcommonindustrial control architectures, the PID controller. By leveraging all available data about the system’s behavior in learning a system dynamics model, the derived method can efficiently tune these controllers from scratch. Although we can easily incorporate all data into dynamics models, real systems expose additional problems to the dynamics modeling and learning task. Characteristics such as non-Gaussian noise, latent states, feedback control or non-i.i.d. data regularly prevent using off-the-shelf modeling tools. Therefore, the second part of this work is concerned with the derivation of modeling solutions that are particularly suited for the reinforcement learning problem. Despite the predominant focus on model-based reinforcement learning as a promising, data-efficient learning tool, this work’s final part revisits model assumptions in a separate branch of reinforcement learning algorithms. Again, generalization and, therefore, efficient learning in model-based methods is primarily driven by the incorporated model assumptions (e.g., smooth dynamics), which real, discontinuous processes might heavily violate. To this end, a model-free reinforcement learning is presented that carefully reintroduces prior model structure to facilitate efficient learning without the need for strong dynamic model priors. The methods and solutions proposed in this work are grounded in the challenges experienced when operating with real-world hardware systems. With applications on a humanoid upper-body robot or an autonomous model race car, the proposed methods are demonstrated to successfully model and master their complex behavior.Item Open Access B-splines on sparse grids for uncertainty quantification(2021) Rehme, Michael F.; Pflüger, Dirk (Prof. Dr.)Item Open Access Architekturkonzepte zur Datenverwaltung in Data Lakes(2021) Giebler, Corinna; Mitschang, Bernhard (Prof. Dr.-Ing.)Die zunehmende Digitalisierung in zahlreichen Bereichen und die damit verbundene Vielzahl an heterogenen Daten, die gespeichert, verwaltet und analysiert werden müssen, stellen eine Herausforderung für traditionelle Datenmanagementkonzepte dar. Insbesondere gilt es, den potentiellen Wert der Daten auszunutzen und durch neue Erkenntnisse Kosten zu senken und Effizienz zu erhöhen. Um die Verwaltung und flexible Analyse der generierten Daten zu ermöglichen, wurde das Konzept des Data Lake entwickelt. Daten heterogener Struktur werden hier in ihrer Rohform gespeichert, sodass auch lange nach ihrer Erfassung beliebige Anwendungsfälle darauf realisiert werden können. Soll allerdings ein solcher Data Lake für die praktische Nutzung in z.B. einem Unternehmen umgesetzt werden, zeigen sich zahlreiche Probleme und Lücken auf. Methodische Grundlagen sind unvollständig, vage oder fehlen ganz. So gibt es keine vollständige Data-Lake-Architektur oder einen Leitfaden, um eine solche zu erstellen. Auch fehlt es an einer passenden Datenorganisation, um die Vielzahl an Anwendungsfällen und Nutzergruppen eines unternehmensweiten Data Lake zu unterstützen. In dieser Arbeit werden diese Lücken adressiert. Hierzu werden drei Forschungsziele formuliert: Z1-Identifikation der Eigenschaften eines Data Lake, Z2-Erstellung eines Leitfadens zur Definition einer vollständigen Data-Lake-Architektur und Z3-Erarbeitung einer internen Data-Lake-Organisation. Diese Forschungsziele werden durch insgesamt sieben Forschungsbeiträge abgedeckt. Hierfür wird zunächst in einer vollständigen Literaturrecherche das Konzept des Data Lake identifiziert und definiert. Im zweiten Schritt stellt diese Arbeit das Data Lake Architecture Framework (DLAF) vor, welches die Definition einer vollständigen Data-Lake-Architektur ermöglicht. Abschließend bietet das Zonenreferenzmodell einen systematischen Ansatz zur Datenorganisation in Data Lakes. Die Umsetzbarkeit der erarbeiteten Lösungen wird mithilfe einer prototypischen Implementierung für ein reales Anwendungsszenario gezeigt. Eine abschließende Evaluation bestätigt, dass die entwickelten Lösungen vollständig sind, zahlreiche Vorteile bieten und so die Industrialisierung von Data Lakes unterstützen.Item Open Access Scalable biophysical simulations of the neuromuscular system(2021) Maier, Benjamin; Schulte, Miriam (Prof. Dr.)The human neuromuscular system consisting of skeletal muscles and neural circuits is a complex system that is not yet fully understood. Surface electromyography (EMG) can be used to study muscle behavior from the outside. Computer simulations with detailed biophysical models provide a non-invasive tool to interpret EMG signals and gain new insights into the system. The numerical solution of such multi-scale models imposes high computational work loads, which restricts their application to short simulation time spans or coarse resolutions. We tackled this challenge by providing scalable software employing instruction-level and task-level parallelism, suitable numerical methods and efficient data handling. We implemented a comprehensive, state-of-the-art, multi-scale multi-physics model framework that can simulate surface EMG signals and muscle contraction as a result of neuromuscular stimulation. This work describes the model framework and its numerical discretization, develops new algorithms for mesh generation and parallelization, covers the use and implementation of our software OpenDiHu, and evaluates its computational performance in numerous use cases. We obtain a speedup of several hundred compared to a baseline solver from the literature and demonstrate, that our distributed-memory parallelization and the use of High Performance Computing resources enables us to simulate muscular surface EMG of the biceps brachii muscle with realistic muscle fiber counts of several hundred thousands. We find that certain model effects are only visible with such high resolution. In conclusion, our software contributes to more realistic simulations of the neuromuscular system and provides a tool for applied researchers to complement in vivo experiments with in-silico studies. It can serve as a building block to set up comprehensive models for more organs in the musculoskeletal system.Item Open Access Improving sample-efficiency for model-free off-policy deep reinforcement learning in contact-rich manipulation(2021) Hoppe, Sabrina; Toussaint, Marc (Prof. Dr.)For centuries, humans have been dreaming of intelligent machines which can move and behave like humans. With the industrial revolution, autonomously moving machinery came into existence. The intelligence to make such a machine behave or even learn autonomously is still a huge challenge however. Since first steps in this direction have been taken we have seen a rising demand for intelligent robots in many fields including production automation where robots may help to cope with increasingly flexible and rapidly changing manufacturing processes. Researchers in the field of reinforcement learning (RL) investigate how to design algorithms such that agents can learn autonomously. For instance robots can be enabled to figure out the optimal way to perform production tasks by themselves. In this thesis, I will focus on insertion tasks which frequently occur in manufacturing. Today's algorithms for such tasks typically trade off sample efficiency against generalization capabilities. This means that one extreme type of algorithms makes an agent learn quickly by pre-defining a lot of structure or making specific assumptions about the task at hand, but this typically implies that the algorithm becomes very specific to this task. The other extreme type of algorithms, model-free learning methods, are very broadly applicable but then typically require vast amounts of data to learn reasonable behavior. In this thesis, I will start from flexible, general-purpose model-free RL algorithms and examine ways how to add small amounts of common sense or human prior knowledge to considerably speed up learning. Such an improvement can be characterized by the type of information that is used as well as by the approach that is chosen to make use of the information. The types of information I will use in this thesis include the robot itself, i.e., for example dynamics information; prior knowledge about the task, for instance a coarse solution strategy that intuitively seems sensible for humans; and mathematical insights into the type and structure of data that the agent has collected. I will also suggest a number of methods to integrate such information in a learning process: as a way to make informed choices about new actions for the agent to try (the so-called exploration), as criteria for an engineer on how to formally describe the task (i.e., how to design a suitable Markov Decision Process), and by choosing and adapting the type of function approximation that is used inside the learning process. All methods that I will present in this thesis have been evaluated on real-world robotic manipulation tasks that have been derived or taken from industrial production plants. The results show that the proposed ways to make use of additional information significantly increase the efficiency of learning processes and can improve their stability even in adverse settings.Item Open Access Load-balancing for scalable simulations with large particle numbers(2021) Hirschmann, Steffen; Pflüger, Dirk (Prof. Dr.)Item Open Access Numerical simulation of fracture in high-velocity impact(2021) Becker, Marvin; Mehl, Miriam (Prof. Dr.)This thesis contains (i) experimental and (ii) numerical techniques to improve the description of fracture. In terms of (i), we provide validation data for high-velocity impact which were extracted by a software which we designed for this purpose. The algorithms to process the motion of parallax and noise containing images are described. In terms of (ii) we compare the FEM and SPH method and propose improved choices for modelling the impact.Item Open Access Provenance-based visual data exploration(2021) Ben Lahmar, Houssem; Herschel, Melanie (Prof. Dr.)Item Open Access Super-resolution enhancement for computed tomography imaging and image processing(2021) Sun, Kaicong; Simon, Sven (Prof. Dr.-Ing.)Item Open Access Explaining existing and missing results over nested data in big data analytics systems(2021) Diestelkämper, Ralf; Herschel, Melanie (Prof. Dr.)Debugging analytical queries in big data analytics systems, such as Apache Spark, Flink, or Hive is a tedious process, especially when large datasets with nested data are involved. To ease this debugging process, we present novel approaches to obtain explanations for existing and missing data in the query result based on a formal data and execution model that faithfully captures the execution semantics of big data analytics systems to provide practically meaningful explanations. These explanations describe why data are present or absent from the result. Our first contribution is a novel, distributed, and scalable algorithm that matches tree-patterns on nested data in big data analytics systems. It enables us to precisely address and query nested data values and arbitrary combinations of them. We leverage this tree-pattern matching algorithm to request explanations for queries over large, nested data. The algorithm matches the pattern onto the data in two steps. It computes matches on the schema in the first step and applies these matches on the data values in a second step. Hence, it avoids complex global state that prevents other state-of-the-art algorithms to scale horizontally on large compute clusters and dataset sizes. In addition to the tree-pattern matching algorithm, we leverage provenance to find the explanations. Provenance describes the origins and derivation of the result data. To provide explanations for existing data, we introduce the novel structural provenance. It traces structural manipulations in addition to data dependencies through the query pipelines. It provides more comprehensive explanations than other existing approaches since it distinguishes between accessed and manipulated data at the granaluratity of individual nested attributes. We define formal capture rules for the structural provenance that extend our execution model. Capturing the strucural provenance according to these rules imposes a high runtime overhead. Thus, we contribute the Pebble algorithm that implements an optimized, lightweight structural provenance to scale to large, nested datasets. Pebble’s explanations enable novel use-cases beyond debugging, such as finding data-usage patterns or fine-grained auditing. Furthermore, we contribute a novel approach to query-based explanations for missing data in a query result. Query-based explanations pinpoint operators in the query that prevent expected data from appearing in the result. This data is called missing data or missing answer. Our approach is the first to support nested data and to consider operators that modify the schema and structure of the data such as the nesting or projection operator as potential causes of missing answers. Additionally, it accounts for mistakenly referenced attributes in the query. Hence, our explanations apply to a wider range of datasets and to novel error scenarios compared to existing, provenance-based solutions. Our approach extends these solutions with reparameterizations. Reparameterizations describe parameter modifications in query operators. We formally introduce them based on our execution model and derive a formal definition of our query-based explanations. To efficiently compute the explanations over large, nested datasets, we propose a novel heuristic algorithm, called Breadcrumb. It applies two unique techniques: (i) It reasons about multiple schema alternatives to account for the mistakenly referenced attributes and (ii) it re-validates each intermediate result to check whether its data can contribute to the missing answer. That is necessary to provide correct explanations for nested data. We implement the tree-pattern matching algorithm, Pebble, and Breadcrumb in Apache Spark to show that each algorithm scales with increasing dataset sizes. Therefore, we run the algorithms on at least two nested real-world workloads of up to 500GB. We illustrate that tree-patterns simplify the query pipelines and show that Pebble and Breadcrumb provide more comprehensive explanations than other state-of-the-art solutions which enable novel use-cases.