08 Fakultät Mathematik und Physik

Permanent URI for this collectionhttps://elib.uni-stuttgart.de/handle/11682/9

Browse

Filters

Settings

Institute: search.filters.UBSInstitut.Institut für Stochastik und Anwendungen

Search Results

Now showing 1 - 10 of 29
  • Thumbnail Image
    ItemOpen Access
    Learning with high-dimensional data
    (2022) Fischer, Simon; Steinwart, Ingo (Prof. Dr.)
    This thesis is divided into three parts. In the first part we introduce a framework that allows us to investigate learning scenarios with restricted access to the data. We use this framework to model high-dimensional learning scenarios as an infinite-dimensional one in which the learning algorithm has only access to some finite-dimensional projections of the data. Finally, we provide a prototypical example of such an infinite-dimensional classification problem in which histograms can achieve polynomial learning rates. In the second part we present some individual results that might by useful for the investigation of kernel-based learning methods using Gaussian kernels in high- or infinite-dimensional learning problems. To be more precise, we present log-covering number bounds for Gaussian reproducing kernel Hilbert spaces on general bounded subsets of the Euclidean space. Unlike previous results in this direction we focus on small explicit constants and their dependence on crucial parameters such as the kernel width as well as the size and dimension of the underlying space. Afterwards, we generalize these bounds to Gaussian kernels defined on special infinite-dimensional compact subsets of the sequence space ℓ_2. More precisely, the considered domains are given by the image of the unit ℓ_∞-ball under some diagonal operator. In the third part we contribute some new insights to the compactness properties of diagonal operators from ℓ_p to ℓ_q for p ≠ q.
  • Thumbnail Image
    ItemOpen Access
    An approximation of solutions to heat equations defined by generalized measure theoretic Laplacians
    (2020) Ehnes, Tim; Hambly, Ben
    We consider the heat equation defined by a generalized measure theoretic Laplacian on [0, 1]. This equation describes heat diffusion in a bar such that the mass distribution of the bar is given by a non-atomic Borel probabiliy measure μ, where we do not assume the existence of a strictly positive mass density. We show that weak measure convergence implies convergence of the corresponding generalized Laplacians in the strong resolvent sense. We prove that strong semigroup convergence with respect to the uniform norm follows, which implies uniform convergence of solutions to the corresponding heat equations. This provides, for example, an interpretation for the mathematical model of heat diffusion on a bar with gaps in that the solution to the corresponding heat equation behaves approximately like the heat flow on a bar with sufficiently small mass on these gaps.
  • Thumbnail Image
    ItemOpen Access
    Smoothing spline regression estimates for randomly right censored data
    (2013) Winter, Stefan; Dippon, Jürgen (PD Dr.)
    This thesis investigates a class of nonparametric regression estimates (smoothing spline estimates) in the presence randomly right censored data. In particular, it is shown that suitably defined estimates of the conditional mean lifetime, the conditional survival function, and the conditional variance of the lifetime achieve the optimal rates of convergence up to a logarithmic factor. These results are valid without any regularity assumption on the distribution of the covariates (beside boundedness) and for adaptive estimators. Furthermore, the consistency of the considered smoothing spline estimates is discussed and the estimators are applied to simulated and real data sets.
  • Thumbnail Image
    ItemOpen Access
    Regression from linear models to neural networks : double descent, active learning, and sampling
    (2023) Holzmüller, David; Steinwart, Ingo (Prof. Dr.)
    Regression, that is, the approximation of functions from (noisy) data, is a ubiquitous task in machine learning and beyond. In this thesis, we study regression in three different settings. First, we study the double descent phenomenon in non-degenerate unregularized linear regression models, proving that these models are always very noise-sensitive when the number of parameters is close to the number of samples. Second, we study batch active learning algorithms for neural network regression from a more applied perspective: We introduce a framework for building existing and new algorithms and provide a large-scale benchmark showing that a new algorithm can achieve state-of-the-art performance. Third, we study convergence rates for non-log-concave sampling and log-partition estimation algorithms, including approximation-based methods, and prove many results on optimal rates, efficiently achievable rates, multi-regime behaviors, reductions, and the relation to optimization.
  • Thumbnail Image
    ItemOpen Access
    A projection and a variational regularization method for sparse inverse problems
    (2012) Offtermatt, Jonas; Kaltenbacher, Barbara (Prof. Dr.)
    The solution of sparse inverse problems has become a highly active topic over the past decade. This thesis aims at providing new methods for the regularization of sparse and possibly ill-posed inverse problems. In this work a projection and a variational regularization method for the solution of sparse inverse problems are presented. The description and analysis of each of these two methods is complemented by an additional related topic. The projection method, developed in Chapter 4, is based on an adaptive regularization method for a distributed parameter in a parabolic PDE, originally introduced by Chavent and coauthors. Here we adapt this approach for general sparse inverse problems. Furthermore a well-definedness result is presented and it is proven that the minimizer achieved by the algorithm solves the original problem in a least squares sense. Additionally, we illustrated the efficiency of the algorithm by two numerical examples from applications in systems biology and data analysis. The sequence of subspaces adaptively chosen by the introduced algorithm leads us to the analysis of regularization by discretization in preimage space. This regularization method is known to convergence only under additional assumptions on the solution. In Chapter 5 regularization by discretization in case of noisy data under a suitable source condition is considered. We present some results of well-definedness, stability and convergence for linear and nonlinear inverse problems in case the regularization subspace is chosen by the discrepancy principle. In Chapter 6 the second main part of this thesis starts. There we present a variational method for sparse inverse problems. Before introducing a new regularization functional, we take a closer look at Bayesian regularization theory. We give a brief introduction and present the connection between deterministic Tikhonov regularization and stochastic Bayesian inversion in case of Gaussian densities, developed by Kaipio and Somersalo. Then we discuss the convergence results from Hofinger and Pikkarainen for the stochastic theory, which are based on this close connection. Also we outline a concept for a general convergence result and prove a generalization result for the existence of a R-minimizing solution. Again we illustrate the gained results with some numerical examples. We use the close connection between stochastic and deterministic regularization to develop a new regularization functional for sparse inverse problems in Chapter 8. There we establish well-definedness, stability and convergence proofs for this functional, based on the results from Hofmann et al. Additionally, we prove convergence rates for the new functional. However, only in a generalized Bregman distance introduced by Grasmair, as the generated regularization term is not convex. The proposed functional is differentiable and thus can be used in gradient based optimization methods, e. g. a Quasi Newton method. We illustrate the efficiency and accuracy of this approach again with some numerical examples. The thesis starts with a general and detailed introduction into inverse problems. First a motivation and introduction to inverse problems is given in Chapter 1. Then a brief overview over recent results in regularization theory is presented in Chapter 2. Finally Chapter 3 closes the introductory part with a motivation and some first notations on sparsity in inverse problems.
  • Thumbnail Image
    ItemOpen Access
    Estimation of minimum mean squared error with variable metric from censored observations
    (2008) Strobel, Matthias; Walk, Harro (Prof. Dr.)
    Eine optimale Metrik auf einem gegebenen Stichprobenraum wird approximiert. Dabei wird die Zensierung der Daten berücksichtigt. Anhand von medizinischen Daten wird die Bedeutung der Prädiktoren bei der Behandlung von Brustkrebs untersucht.
  • Thumbnail Image
    ItemOpen Access
    Ordinal patterns in clusters of subsequent extremes of regularly varying time series
    (2020) Oesting, Marco; Schnurr, Alexander
    In this paper, we investigate temporal clusters of extremes defined as subsequent exceedances of high thresholds in a stationary time series. Two meaningful features of these clusters are the probability distribution of the cluster size and the ordinal patterns giving the relative positions of the data points within a cluster. Since these patterns take only the ordinal structure of consecutive data points into account, the method is robust under monotone transformations and measurement errors. We verify the existence of the corresponding limit distributions in the framework of regularly varying time series, develop non-parametric estimators and show their asymptotic normality under appropriate mixing conditions. The performance of the estimators is demonstrated in a simulated example and a real data application to discharge data of the river Rhine.
  • Thumbnail Image
    ItemOpen Access
    Neue Methoden für die Optimierung verfahrenstechnischer Prozesse
    (2013) Hokenmaier, Stephania; Kaltenbacher, Barbara (Univ.-Prof. Dipl.-Ing. Dr.)
    Die Verfahrenstechnik beschäftigt sich mit der Trennung, Umwandlung oder Verflüssigung chemischer Stoffe, wie zum Beispiel die Zerlegung von Luft in ihre Hauptbestandteile Sauerstoff, Stickstoff und Argon oder die Reinigung und Verflüssigung von Erdgas. Mit Hilfe von Prozesssimulatoren können verfahrenstechnische Anlagen, oder Teile davon, modelliert, simuliert und anschließend optimiert werden. Die Engineering Division der Linde AG, eines der weltweit führenden Unternehmen für die Planung und den Bau von verfahrenstechnischen Anlagen, betreibt und entwickelt den hauseigenen Prozesssimulator OPTISIM (R). Steigende Anforderungen an die Optimierung im Bezug auf Problemgröße, Effizienz und Robustheit, insbesondere bei aufretenden Unstetigkeiten und der Verwendung von Approximationen während der Simulation und Optimierung, erfordern den Einsatz neuer Optimierungsverfahren. In dieser Arbeit wurde ein simultaner Optimierungsansatz implementiert, bei dem die Modellgleichungen als Gleichungsnebenbedingungen zu dem Optimierungsproblem hinzugefügt werden. Zur Lösung dieser komplexen Optimierungsaufgaben wurde der Optimierer IPOPT in OPTISIM (R) eingebunden. Hierbei wurde die globale Konvergenz der Filter-Liniensuche von Biegler und Wächter, die in IPOPT verwendet wird, unter der Annahme von Störungen in den Gleichungsnebenbedingungen und deren Ableitungen untersucht. Die Störungen modellieren hierbei die verwendeten Approximationen sowie bis zu einem gewissen Maße auch auftretende Unstetigkeiten. Des Weiteren wurde ein globaler Optimierungsalgorithmus mit einer Methode des mehrfachen Starts umgesetzt. Hierzu wurde eine spezielle Behandlung der Startwerte entwickelt, sodass die Optimierung von unterschiedlichen Punkten aus gestartet werden kann.
  • Thumbnail Image
    ItemOpen Access
    Stochastic partial differential equations on Cantor-like sets
    (2020) Ehnes, Tim; Freiberg, Uta Renata (Prof. Dr.)
  • Thumbnail Image
    ItemOpen Access
    Lossless transformations and excess risk bounds in statistical inference
    (2023) Györfi, László; Linder, Tamás; Walk, Harro
    We study the excess minimum risk in statistical inference, defined as the difference between the minimum expected loss when estimating a random variable from an observed feature vector and the minimum expected loss when estimating the same random variable from a transformation (statistic) of the feature vector. After characterizing lossless transformations, i.e., transformations for which the excess risk is zero for all loss functions, we construct a partitioning test statistic for the hypothesis that a given transformation is lossless, and we show that for i.i.d. data the test is strongly consistent. More generally, we develop information-theoretic upper bounds on the excess risk that uniformly hold over fairly general classes of loss functions. Based on these bounds, we introduce the notion of a δ -lossless transformation and give sufficient conditions for a given transformation to be universally δ -lossless. Applications to classification, nonparametric regression, portfolio strategies, information bottlenecks, and deep learning are also surveyed.