05 Fakultät Informatik, Elektrotechnik und Informationstechnik
Permanent URI for this collectionhttps://elib.uni-stuttgart.de/handle/11682/6
Browse
Search Results
Item Open Access A framework for distributed training of physics-informed neural networks using JAX(2021) Braun, JohannesThe intention of this thesis is to evaluate the high-performance machine learning framework JAX. In the course of this work, a physics-informed neural network that solves the Burgers’ equation is implemented. This problem is chosen, as it is a well known and researched numerical problem and thus allows for great comparability. Here, a basic version of the physics-informed neural network with Flax is first created, which is an ecosystem for JAX that allows to implement neural networks. This version was then first improved with the tools offered via JAX. Afterwards, a SPMD version of this physics-informed neural network is also implemented, where multiple graphics processor units are utilized in the training. Additionally, the physics-informed neural network is extended to predict the parameters of the partial differential equation that describes the Burgers’ equation. This was done by the physics-informed neural network, while still learning to estimate the Burgers’ equation. For the optimized basic physics-informed neural network and the physics-informed neural network that also estimates the parameters of the partial differential equation promising results with JAX were achieved. The outcome of the SPMD physics-informed neural network was dissatisfactory, as it did not yield any improvements compared to the basic version. Although, this might stem from the small amount of data points used for each iteration and further points discussed in this paper. Additionally, a caveat must be voiced, as it often becomes apparent that the documentations of JAX and Flax are a work in progress. Because of this, a lot of crucial features have to be found out by trial and error, while working with these frameworks. Yet still, JAXand hence also Flax are considered a compelling framework to implement high-performance neural networks. Especially because of its potent Autograd and straightforward XLA just in time compilation. Through these components a performant physics-informed neural network can be quickly setup as shown in this thesis. Here, Autograd aids in creating the necessary gradients for the physical loss. Whereas the XLA just in time compilation yields drastic improvements to the run time of the training performed on the physics-informed neural network. These features then lead to previously mentioned promising results for the basic physics-informed neural network and the physics-informed neural network that also estimates the parameters of the partial differential equation.Item Open Access Models for data-efficient reinforcement learning on real-world applications(2021) Dörr, Andreas; Toussaint, Marc (Prof. Dr.)Large-scale deep Reinforcement Learning is strongly contributing to many recently published success stories of Artificial Intelligence. These techniques enabled computer systems to autonomously learn and master challenging problems, such as playing the game of Go or complex strategy games such as Star-Craft on human levels or above. Naturally, the question arises which problems could be addressed with these Reinforcement Learning technologies in industrial applications. So far, machine learning technologies based on (semi-)supervised learning create the most visible impact in industrial applications. For example, image, video or text understanding are primarily dominated by models trained and derived autonomously from large-scale data sets with modern (deep) machine learning methods. Reinforcement Learning, on the opposite side, however, deals with temporal decision-making problems and is much less commonly found in the industrial context. In these problems, current decisions and actions inevitably influence the outcome and success of a process much further down the road. This work strives to address some of the core problems, which prevent the effective use of Reinforcement Learning in industrial settings. Autonomous learning of new skills is always guided by existing priors that allow for generalization from previous experience. In some scenarios, non-existing or uninformative prior knowledge can be mitigated by vast amounts of experience for a particular task at hand. Typical industrial processes are, however, operated in very restricted, tightly calibrated operating points. Exploring the space of possible actions or changes to the process naively on the search for improved performance tends to be costly or even prohibitively dangerous. Therefore, one reoccurring subject throughout this work is the emergence of priors and model structures that allow for efficient use of all available experience data. A promising direction is Model-Based Reinforcement Learning, which is explored in the first part of this work. This part derives an automatic tuning method for one of themostcommonindustrial control architectures, the PID controller. By leveraging all available data about the system’s behavior in learning a system dynamics model, the derived method can efficiently tune these controllers from scratch. Although we can easily incorporate all data into dynamics models, real systems expose additional problems to the dynamics modeling and learning task. Characteristics such as non-Gaussian noise, latent states, feedback control or non-i.i.d. data regularly prevent using off-the-shelf modeling tools. Therefore, the second part of this work is concerned with the derivation of modeling solutions that are particularly suited for the reinforcement learning problem. Despite the predominant focus on model-based reinforcement learning as a promising, data-efficient learning tool, this work’s final part revisits model assumptions in a separate branch of reinforcement learning algorithms. Again, generalization and, therefore, efficient learning in model-based methods is primarily driven by the incorporated model assumptions (e.g., smooth dynamics), which real, discontinuous processes might heavily violate. To this end, a model-free reinforcement learning is presented that carefully reintroduces prior model structure to facilitate efficient learning without the need for strong dynamic model priors. The methods and solutions proposed in this work are grounded in the challenges experienced when operating with real-world hardware systems. With applications on a humanoid upper-body robot or an autonomous model race car, the proposed methods are demonstrated to successfully model and master their complex behavior.Item Open Access Evaluating dynamic load balancing of ECM workload pattern employed in cloud environments managed by a Kubernetes/Docker eco-system(2021) Hagemann, PascalThe transition to cloud-based ECM solutions requires adaptation and enhancements for dynamic cloud environment. Cloud computing and containerization offer key concepts to create solutions for this task. They also open new opportunities to benefit from cloud usage by leveraging the pay-as-you-go model. Cloud users and providers both benefit from the increased efficiency of dynamic applications. But legacy applications are not yet able to leverage the benefits provided by cloud orchestration. This thesis, therefore researches the feasibility of a dynamic load balancing approach applied to an ECM application deployed into a cloud environment. To evaluate the approach, a prototype using open-source software is created on a Kubernetes orchestrated cluster. Previous work included the port of the containerized ECM application into the Kubernetes environment. The present prototype builds up on this approach by enhancing the ECM application components with metrics export capabilities. A monitoring system based on Prometheus is introduced to gather these metrics from the ECM application and other system components. Information provided by these metrics are used to add elasticity to application components. The prototype proves that dynamic load balancing of the ECM application in the cloud is feasible. Two major challenges for an efficient deployment of the application were identified, (1) the generation of useful metrics and (2) removing dependencies from individual components. Further research into optimizations of stateful service components is required. This further ensures an efficient usage in cloud based elastic topologies, especially considering stateful database applications.Item Open Access B-splines on sparse grids for uncertainty quantification(2021) Rehme, Michael F.; Pflüger, Dirk (Prof. Dr.)Item Open Access Analyse von Clustering-Algorithmen zur Partitionierung von Trainingsdaten für komplexe Mehrklassenprobleme(2021) Braun, KaiIn den letzten Jahrzenten nahm die Anzahl der Datenmengen im Bereich der Informationstechnologie immer weiter zu. Mittlerweile können sie mithilfe algorithmischer Verfahren automatisch ausgewertet werden, um neue Informationen und Erkenntnisse zu erlangen. Diese algorithmischen Verfahren werden im Gebiet des Machine-Learnings fortlaufend weiterentwickelt. Daten aus der Industrie weisen meist komplexe Charakteristika auf, welche die Vorhersagegenauigkeit herkömmlicher Machine-Learning Klassifikatoren verringert. Bekannte Beispiele dafür sind etwa die Multi-Class Imbalance oder ein heterogenes Produktportfolio. Aus einer Vorarbeit \cite {Hirsch2020} geht hervor, dass eine geeignete Partitionierung der Daten die Vorhersagegenauigkeit der Klassifikatoren signifikant verbessert. Jedoch wird bei diesem Verfahren spezifisches Domänenwissen verlangt, mit dessen Hilfe die Daten partitioniert werden. Eine Methode, welche Daten unabhängig von Domänenwissen partitioniert, ist das Clustering. Im Rahmen dieser Arbeit soll geprüft werden, inwiefern eine rein datengetriebene Partitionierung mittels Clustering ähnliche Verbesserungen der Vorhersagegenauigkeit hervorbringen kann wie das domänengetriebene Verfahren. Hierfür wird ein Konzept zur datengetriebenen Partitionierung und anschließenden Klassifikation entwickelt. Ziel der datengetriebenen Partitionierung ist es herauszufinden, durch welche Clustering-Algorithmen eine verbesserte Vorhersagegenauigkeit möglich ist. Im Rahmen der Arbeit wird für die datengetriebene Partitionierung und Klassifikation folgendes Konzept entwickelt: Der erste Schritt der datengetriebenen Partitionierung sieht vor, den gesamten Datensatz mittels Clustering in einzelne Partitionen zu unterteilen. Im zweiten Schritt werden Klassifikatoren auf den einzelnen Partitionen angewendet. Dabei soll geprüft werden, durch welche Partitionierung der Daten die besten Klassifikationsergebnisse erreicht werden können. Mithilfe verschiedener Clustering-Metriken werden die Clustering-Ergebnisse der einzelnen Algorithmen evaluiert. Außerdem wird untersucht, welche Einflussfaktoren sich am stärksten auf die Klassifikationsergebnisse auswirken. Des Weiteren wird auf spezifische Herausforderungen, wie die Klassenungleichverteilung oder die fehlenden Features, eingegangen. Die Messergebnisse ergeben, dass eine Partitionierung mittels K-Means oder X-Means die besten Werte liefert.Item Open Access Verteiltes Subgraph-Counting mit Colorcoding(2021) Güzel, EmreFür das Subgraph-Counting-Problem und das Subgraph-Enumeration-Problem sind die Anzahlen und Häufigkeiten der Isomorphismen für bestimmte Subgraphen in größeren Graphen von Interesse. Zur Lösung dieser Probleme gibt es verschiedene Ansätze. Einer dieser Ansätze basiert auf dem Colorcoding [AYZ95] Verfahren von Alon et al. Die beiden Algorithmen PARSE [ZKKM10] und CC [BCK+17] [BCK+18] versuchen diese Probleme mit diesem Ansatz zu lösen. Dabei hat PARSE eine verteilte Version, wohingegen CC lediglich auf einer einzigen Maschine ausführbar ist. Zusätzlich zu diesen beiden Algorithmen wird in dieser Arbeit DistCC, eine verteilte Version CC’s, vorgestellt. DistCC orientiert sich bei der Art und Weise seiner Verteilung am Beispiel von PARSE. Dank der Verteilung sind die Laufzeiten DistCC’s teilweise deutlich kürzer, als die Laufzeiten CC’s. Außerdem wurden neben der natürlichen Sortierung der Graphen auch zwei neue Sortierungen eingeführt, die die Laufzeiten CC’s und somit auch DistCC’s teilweise verbessern.Item Open Access Digital twin for fleet management of connected vehicles(2021) Gerhards, Jan PatrickConnected cars are capable of communicating with other devices (e.g., other vehicles, smartphones, road side units) in their environment. The domain in which they operate is a highly dynamic one. Reasons for this include changing demands of users and heterogeneity of technologies and interfaces. Due to this, developing a concept for a digital twin of a connected car that is capable of operating under these conditions is a challenging task. Especially handling of data and behavior of connected cars is an area of interest. In this thesis, we propose a digital twin architecture which can be used to implement connected car scenarios. We analyze challenges of the domain and assess in which ways and under what circumstances connected cars may interact. Based on these expectations, we introduce our concept for a digital twin. Our main goal is to provide interoperability between different car manufacturers, while still allowing developers to implement functionality in a way they deem appropriate. In order to verify our concept, a prototype of a digital twin is implemented. We explain this implementation and share experiences gained during this development. Finally, we draw conclusions on the concept as a whole and how it may be used in practice.Item Open Access Neural Networks on Microsoft HoloLens 2(2021) Lazar, LéonThe goal of the present Bachelor thesis is to enable comparing different approaches of integrating Neural Networks in HoloLens 2 applications in a quantitative and qualitative manner by defining highly diagnostic criteria. Moreover, multiple different approaches to accomplish the integration are proposed, implemented and evaluated using the aforementioned criteria. Finally, the work gives an expressive overview of all working approaches. The basic requirements are that Neural Networks trained by TensorFlow/Keras can be used and executed directly on the HoloLens 2 without requiring an internet connection. Furthermore, the Neural Networks have to be integrable in Mixed/Augmented Reality applications. In total four approaches are proposed: TensorFlow.js, Unity Barracuda, TensorFlow.NET, and Windows Machine Learning which is an already existing approach. For each working approach a benchmarking application is developed which runs a common reference model on a test datatset to measure inference time and accuracy. Moreover, a small proof of concept application is developed in order to show that the approach also works with real Augmented Reality applications. The application uses a MobileNetV2 model to classify image frames coming from the webcam and displays the results to the user. All the feasible approaches are evaluated using the aforementioned evaluation criteria which include ease of implementation, performance, accuracy, compatibility with Machine Learning frameworks and pre-trained models, and integrability with 3D frameworks. The Barracuda, TensorFlow.js and WinML approaches turned out to be feasible. Barracuda, which only can be integrated in Unity applications, is the most performant framework since it can make use of GPU inference. After that follows TensorFlow.js which can be integrated in JavaScript Augmented Reality frameworks such as A-Frame. Windows ML can currently only use CPU inference on the HoloLens 2 and is therefore the slowest one. It can be integrated in Unity projects with some difficulties as well as plain Win32 and UWP apps. Barracuda and Windows Machine Learning are also integrated in a biomechanical visualization application based on Unity for performing simulations. The results of this thesis make the different approaches for integrating Neural Networks on the HoloLens 2 comparable. Now an informed decision which approach is the best for a specific application can be made. Furthermore, the work shows that the use of Barracuda or TensorFlow.js on the HoloLens 2 is feasible and superior compared to the existing WinML approach.Item Open Access OpenCL-HPX integration(2021) Schupikov, MichaelDistributed applications combine the computational capabilities of heterogeneous nodes. As such, they offer challenges regarding data transfer and synchronization. HPX is a library for concurrent, parallel applications. It strives not only to address challenges regarding distributed systems, but also to conform to current and upcoming C++ standards. One of the solutions found in heterogeneous systems is provided in form of the OpenCL standard. It enables the cooperation between hardware resources through a unified interface. In this work, we combine HPX and OpenCL in form of an executor. The OpenCL executor enables HPX users to benefit from more resources on heterogeneous nodes. We describe the executor’s design and its implementation. Furthermore, we present the testing methods to ensure the correctness of the executor. Finally, we provide benchmarks on NVIDIA and AMD GPUs.Item Open Access Extracting and segmenting high-variance references from PDF documents with BERT(2021) Evci, HasanThe extraction and segmentation of references from scientific articles is a core task of modern digital libraries. Once references are extracted and segmented, the bibliographic information can be made publicly available and linked, enabling efficient literature study. However, references often vary in their structure and content. This makes the extraction and segmentation of references a challenging but valuable task. The purpose of this thesis is to investigate whether Bidirectional Encoder Representations from Transformers (BERT) is suitable for the extraction and segmentation of bibliographic references. Therefore, we follow a deep learning approach for the extraction and segmentation of references from PDF documents. We use a neural network architecture based on BERT, a deep language representation model that has significantly increased performance on many natural language processing tasks. Over the BERT output, we put a linear-chain Conditional Random Field. We experiment with different BERT models and input formats and also examine two approaches for reference extraction and segmentation. The experiments are evaluated on a challenging dataset that contains both English and German social science publications with highly varying references. Our results show that the best performing BERT models were pre-trained on similar data to the data that we used for the fine-tuning of the BERT models on the task of reference extraction and reference segmentation. Moreover, our findings show that long, context-based input sequences yield the best results. The extraction model identifies and extracts references with an average F1-score of 81.9%. References are segmented with an average F1-score of 93.6%. We show that our models compare well to one other previously published work. Our conclusion is that BERT is a suitable choice for reference extraction and reference segmentation.