Browsing by Author "Schäfer Rodrigues Silva, Aline"

Now showing 1 - 3 of 3

Open Access
Diagnosing similarities in probabilistic multi-model ensembles : an application to soil-plant-growth-modeling
(2022) Schäfer Rodrigues Silva, Aline; Weber, Tobias K. D.; Gayler, Sebastian; Guthke, Anneli; Höge, Marvin; Nowak, Wolfgang; Streck, Thilo
There has been an increasing interest in using multi-model ensembles over the past decade. While it has been shown that ensembles often outperform individual models, there is still a lack of methods that guide the choice of the ensemble members. Previous studies found that model similarity is crucial for this choice. Therefore, we introduce a method that quantifies similarities between models based on so-called energy statistics. This method can also be used to assess the goodness-of-fit to noisy or deterministic measurements. To guide the interpretation of the results, we combine different visualization techniques, which reveal different insights and thereby support the model development. We demonstrate the proposed workflow on a case study of soil–plant-growth modeling, comparing three models from the Expert-N library. Results show that model similarity and goodness-of-fit vary depending on the quantity of interest. This confirms previous studies that found that “there is no single best model” and hence, combining several models into an ensemble can yield more robust results.
Open Access
Quantifying and visualizing model similarities for multi-model methods
(Stuttgart : Eigenverlag des Instituts für Wasser- und Umweltsystemmodellierung der Universität Stuttgart, 2022) Schäfer Rodrigues Silva, Aline; Nowak, Wolfgang (Prof. Dr.-Ing.)
Modeling environmental systems is typically limited by an incomplete system understanding due to scarce and imprecise measurements. This leads to different types of uncertainties, among which conceptual uncertainty plays a key role, but is difficult to address. Conceptual uncertainty refers to the problem of finding the most appropriate model representation of the physical system. This includes the problem of choosing from several plausible model hypotheses, but also the problem that the true system description might not even be among this set of hypotheses. In this thesis, I address the first of these issues, the uncertainty of choosing a model from a finite set. To account for this uncertainty of model choice, modelers typically use multi-model methods. This means that they consider not only one but several models and apply statistical methods to either combine them or select the most appropriate one. For any of these methods, it is crucial to know how similar the individual models are. But even though multi-model methods have become increasingly popular, no methods were available that quantify the similarities between models and visualize them intuitively. This dissertation aims at closing these gaps. In particular, it tackles the challenges of judging whether simplified models are a suitable replacement for a more detailed model, and of visualizing model similarities in a way that helps modelers to gain an intuitive understanding of the model set. I defined three research questions that address these challenges and form the basis of this thesis. 1. How can we systematically assess how similar conceptually simplified model versions are compared to an original, more detailed model? 2. How can we extend the similarity analysis so it is suitable for computationally expensive models? 3. How can we visualize the similarities between probabilistic model predictions? With the first contribution, I show that the so-called model confusion matrix can be used to quantify model similarities and thus identify the best conceptual simplification of a detailed reference model. This matrix was introduced by Schöniger et al. [2015] to estimate the data need of competing models. Here, I demonstrate that the matrix can be used, beyond this original purpose, to analyze model similarities. With the second contribution, I address the problem of assessing this matrix for computationally expensive models. Since calculating this matrix requires many model runs, the existing method was not yet suitable for models that have long run times. This problem is solved by extending the surrogate-based Bayesian model selection [Mohammadi et al., 2018] so that two models can be compared based on their surrogates while accounting for approximation errors. With the third contribution, I demonstrate how the similarity of probabilistic model predictions can be quantified based on so-called energy statistics. By comparing different visualization techniques, I show how multi-model ensembles can be visualized intuitively so that modelers can get a better understanding of the model set. The presented methods are widely applicable and can thus help to bring the importance of model similarities further into the focus of multi-model developers and users. Thus, depending on the research problem, the individual models or an appropriate multi-model method can be selected in a more targeted manner.
Open Access
Surrogate-based Bayesian comparison of computationally expensive models : application to microbially induced calcite precipitation
(2021) Scheurer, Stefania; Schäfer Rodrigues Silva, Aline; Mohammadi, Farid; Hommel, Johannes; Oladyshkin, Sergey; Flemisch, Bernd; Nowak, Wolfgang
Geochemical processes in subsurface reservoirs affected by microbial activity change the material properties of porous media. This is a complex biogeochemical process in subsurface reservoirs that currently contains strong conceptual uncertainty. This means, several modeling approaches describing the biogeochemical process are plausible and modelers face the uncertainty of choosing the most appropriate one. The considered models differ in the underlying hypotheses about the process structure. Once observation data become available, a rigorous Bayesian model selection accompanied by a Bayesian model justifiability analysis could be employed to choose the most appropriate model, i.e. the one that describes the underlying physical processes best in the light of the available data. However, biogeochemical modeling is computationally very demanding because it conceptualizes different phases, biomass dynamics, geochemistry, precipitation and dissolution in porous media. Therefore, the Bayesian framework cannot be based directly on the full computational models as this would require too many expensive model evaluations. To circumvent this problem, we suggest to perform both Bayesian model selection and justifiability analysis after constructing surrogates for the competing biogeochemical models. Here, we will use the arbitrary polynomial chaos expansion. Considering that surrogate representations are only approximations of the analyzed original models, we account for the approximation error in the Bayesian analysis by introducing novel correction factors for the resulting model weights. Thereby, we extend the Bayesian model justifiability analysis and assess model similarities for computationally expensive models. We demonstrate the method on a representative scenario for microbially induced calcite precipitation in a porous medium. Our extension of the justifiability analysis provides a suitable approach for the comparison of computationally demanding models and gives an insight on the necessary amount of data for a reliable model performance.