Browsing by Author "Zimmermann, Heiko"

Now showing 1 - 3 of 3

Open Access
Bayesian functional optimization of neural network activation functions
(2017) Zimmermann, Heiko
In the past we have seen many great successes of Bayesian optimization as a black-box and hyperparameter optimization method in many applications of machine learning. Most existing approaches aim to optimize an unknown objective function by treating it as a random function and place a parametric prior over it. Recently an alternative approach was introduced which allows Bayesian optimization to work in nonparametric settings to optimize functionals (Bayesian functional optimization). Another well recognized framework that powers some of today’s most competitive machine learning algorithms are artificial neural networks which are state of the art tools to parameterize and train complex nonlinear models. However, while normally a lot of attention is paid to the network’s layout and structure the neuron’s nonlinear activation function is often still chosen from the set of commonly used function. While recent work addressing this problem mainly considers steepest-descent-based methods to jointly train individual neuron activation functions and the network parameters, we use Bayesian functional optimization to search for globally optimal shared activation functions. Therefore, we formulate the problem as a functional optimization problem and model the activation functions as elements in a reproducing kernel Hilbert space. Our experiments have shown that Bayesian functional optimization outperforms a similar parametric approach using standard Bayesian optimization and works well for higher dimensional problems. Compared to the baseline models with fixed sigmoid and jointly trained shared activation function we achieved an improvement of the relative classification error over 39% and over 20%, respectively.
Open Access
Optimality principles in human point-to-manifold reaching accounting for muscle dynamics
(2020) Wochner, Isabell; Driess, Danny; Zimmermann, Heiko; Häufle, Daniel F. B.; Toussaint, Marc; Schmitt, Syn
Human arm movements are highly stereotypical under a large variety of experimental conditions. This is striking due to the high redundancy of the human musculoskeletal system, which in principle allows many possible trajectories toward a goal. Many researchers hypothesize that through evolution, learning, and adaption, the human system has developed optimal control strategies to select between these possibilities. Various optimality principles were proposed in the literature that reproduce human-like trajectories in certain conditions. However, these studies often focus on a single cost function and use simple torque-driven models of motion generation, which are not consistent with human muscle-actuated motion. The underlying structure of our human system, with the use of muscle dynamics in interaction with the control principles, might have a significant influence on what optimality principles best model human motion. To investigate this hypothesis, we consider a point-to-manifold reaching task that leaves the target underdetermined. Given hypothesized motion objectives, the control input is generated using Bayesian optimization, which is a machine learning based method that trades-off exploitation and exploration. Using numerical simulations with Hill-type muscles, we show that a combination of optimality principles best predicts human point-to-manifold reaching when accounting for the muscle dynamics.
Open Access
Visuelle Analyse von Social-Media-Informationsdiffusion für den Zivilschutz
(2014) Zimmermann, Heiko
In den letzten Jahren haben vor allem soziale Medien zunehmend Einzug in das Leben unserer Internetgesellschaft gehalten. Dabei generieren Benutzer sogenannter sozialer Netzwerke eine Vielzahl an Daten, die untereinander ausgetauscht und der Öffentlichkeit zugänglich gemacht werden. Richtig gefiltert und ausgewertet stellen diese Daten wichtige Informationsquellen dar, welche in der Marktforschung, der Bekämpfung von Kriminalität und nicht zuletzt beim Katastrophenschutz Verwendung finden. Gerade bei Letzterem ist es oftmals notwendig, die gewonnenen Informationen auf einer Karte zu verorten. Dies geschieht meist über GPS-Koordinaten, die den Nachrichten angehängt sind. Von besonderem Interesse sind dabei neben den Inhalten der Nachrichten auch die Verbreitungswege der enthaltenen Informationen. In dieser Bachelorarbeit werden, ausgehend von einem Workshop mit Domäneexperten, Konzepte zur Gewinnung und Visualisierung von Kommunikationsnetzwerken, am Beispiel von Twitter, entwickelt. Dabei dienen als Datengrundlage sowohl online verfügbare Twitterdaten, als auch ein zuvor gesammelter, lokal gespeicherter Datensatz, der zuvor auf enthaltene Kommunikationsnetzwerke untersucht wird. Zur Visualisierung der Daten auf einer Karte wird ein Knoten-Kanten-Diagramm und eine, aus den Kommunikationsnetzwerken erzeugte, Heatmap verwendet. Bei einem abschließenden Anwendungsfall wird ein Datensatz, mit Hilfe der Heatmap, auf dessen Verbreitungswege untersucht.