02 Fakultät Bau- und Umweltingenieurwissenschaften
Permanent URI for this collectionhttps://elib.uni-stuttgart.de/handle/11682/3
Browse
5 results
Search Results
Item Open Access Diagnosing similarities in probabilistic multi-model ensembles : an application to soil-plant-growth-modeling(2022) Schäfer Rodrigues Silva, Aline; Weber, Tobias K. D.; Gayler, Sebastian; Guthke, Anneli; Höge, Marvin; Nowak, Wolfgang; Streck, ThiloThere has been an increasing interest in using multi-model ensembles over the past decade. While it has been shown that ensembles often outperform individual models, there is still a lack of methods that guide the choice of the ensemble members. Previous studies found that model similarity is crucial for this choice. Therefore, we introduce a method that quantifies similarities between models based on so-called energy statistics. This method can also be used to assess the goodness-of-fit to noisy or deterministic measurements. To guide the interpretation of the results, we combine different visualization techniques, which reveal different insights and thereby support the model development. We demonstrate the proposed workflow on a case study of soil–plant-growth modeling, comparing three models from the Expert-N library. Results show that model similarity and goodness-of-fit vary depending on the quantity of interest. This confirms previous studies that found that “there is no single best model” and hence, combining several models into an ensemble can yield more robust results.Item Open Access Strategies for simplifying reactive transport models : a Bayesian model comparison(2020) Schäfer Rodrigues Silva, Aline; Guthke, Anneli; Höge, Marvin; Cirpka, Olaf A.; Nowak, WolfgangFor simulating reactive transport on aquifer scale, various modeling approaches have been proposed. They vary considerably in their computational demands and in the amount of data needed for their calibration. Typically, the more complex a model is, the more data are required to sufficiently constrain its parameters. In this study, we assess a set of five models that simulate aerobic respiration and denitrification in a heterogeneous aquifer at quasi steady state. In a probabilistic framework, we test whether simplified approaches can be used as alternatives to the most detailed model. The simplifications are achieved by neglecting processes such as dispersion or biomass dynamics, or by replacing spatial discretization with travel‐time‐based coordinates. We use the model justifiability analysis proposed by Schöniger, Illman, et al. (2015, https://doi.org/10.1016/j.jhydrol.2015.07.047) to determine how similar the simplified models are to the reference model. This analysis rests on the principles of Bayesian model selection and performs a tradeoff between goodness‐of‐fit to reference data and model complexity, which is important for the reliability of predictions. Results show that, in principle, the simplified models are able to reproduce the predictions of the reference model in the considered scenario. Yet, it became evident that it can be challenging to define appropriate ranges for effective parameters of simplified models. This issue can lead to overly wide predictive distributions, which counteract the apparent simplicity of the models. We found that performing the justifiability analysis on the case of model simplification is an objective and comprehensive approach to assess the suitability of candidate models with different levels of detail.Item Open Access Diagnosis of model errors with a sliding time‐window Bayesian analysis(2022) Hsueh, Han‐Fang; Guthke, Anneli; Wöhling, Thomas; Nowak, WolfgangDeterministic hydrological models with uncertain, but inferred‐to‐be‐time‐invariant parameters typically show time‐dependent model errors. Such errors can occur if a hydrological process is active in certain time periods in nature, but is not resolved by the model or by its input. Such missing processes could become visible during calibration as time‐dependent best‐fit values of model parameters. We propose a formal time‐windowed Bayesian analysis to diagnose this type of model error, formalizing the question “In which period of the calibration time‐series does the model statistically disqualify itself as quasi‐true?” Using Bayesian model evidence (BME) as model performance metric, we determine how much the data in time windows of the calibration time‐series support or refute the model. Then, we track BME over sliding time windows to obtain a dynamic, time‐windowed BME (tBME) and search for sudden decreases that indicate an onset of model error. tBME also allows us to perform a formal, sliding likelihood‐ratio test of the model against the data. Our proposed approach is designed to detect error occurrence on various temporal scales, which is especially useful in hydrological modeling. We illustrate this by applying our proposed method to soil moisture modeling. We test tBME as model error indicator on several synthetic and real‐world test cases that we designed to vary in error sources (structure and input) and error time scales. Results prove the successful detection errors in dynamic models. Moreover, the time sequence of posterior parameter distributions helps to investigate the reasons for model error and provide guidance for model improvement.Item Open Access Bayesian model weighting : the many faces of model averaging(2020) Höge, Marvin; Guthke, Anneli; Nowak, WolfgangModel averaging makes it possible to use multiple models for one modelling task, like predicting a certain quantity of interest. Several Bayesian approaches exist that all yield a weighted average of predictive distributions. However, often, they are not properly applied which can lead to false conclusions. In this study, we focus on Bayesian Model Selection (BMS) and Averaging (BMA), Pseudo-BMS/BMA and Bayesian Stacking. We want to foster their proper use by, first, clarifying their theoretical background and, second, contrasting their behaviours in an applied groundwater modelling task. We show that only Bayesian Stacking has the goal of model averaging for improved predictions by model combination. The other approaches pursue the quest of finding a single best model as the ultimate goal, and use model averaging only as a preliminary stage to prevent rash model choice. Improved predictions are thereby not guaranteed. In accordance with so-called ℳ-settings that clarify the alleged relations between models and truth, we elicit which method is most promising.Item Open Access The method of forced probabilities : a computation trick for Bayesian model evidence(2022) Banerjee, Ishani; Walter, Peter; Guthke, Anneli; Mumford, Kevin G.; Nowak, WolfgangBayesian model selection objectively ranks competing models by computing Bayesian Model Evidence (BME) against test data. BME is the likelihood of data to occur under each model, averaged over uncertain parameters. Computing BME can be problematic: exact analytical solutions require strong assumptions; mathematical approximations (information criteria) are often strongly biased; assumption-free numerical methods (like Monte Carlo) are computationally impossible if the data set is large, for example like high-resolution snapshots from experimental movies. To use BME as ranking criterion in such cases, we develop the “Method of Forced Probabilities (MFP)”. MFP swaps the direction of evaluation: instead of comparing thousands of model runs on random model realizations with the observed movie snapshots, we force models to reproduce the data in each time step and record the individual probabilities of the model following these exact transitions. MFP is fast and accurate for models that fulfil the Markov property in time, paired with high-quality data sets that resolve all individual events. We demonstrate our approach on stochastic macro-invasion percolation models that simulate gas migration in porous media, and list additional examples of probable applications. The corresponding experimental movie was obtained from slow gas injection into water-saturated, homogeneous sand in a 25 x 25 x 1 cm acrylic glass tank. Despite the movie not always satisfying the high demands (resolving all individual events), we can apply MFP by suggesting a few workarounds. Results confirm that the proposed method can compute BME in previously unfeasible scenarios, facilitating a ranking among competing model versions for future model improvement.