Causality in an engineered system pertains to how a system output changes due to a controlled change or intervention on the system or system environment. Engineered systems designs reflect a causal theory regarding how a system will work, and predicting the reliability of such systems typically requires knowledge of this underlying causal structure. The aim of this work is to introduce causal modeling tools that inform reliability predictions based on biased data sources. We present a novel application of the popular structural causal modeling (SCM) framework to reliability estimation in an engineering application, illustrating how this framework can inform whether reliability is estimable and how to estimate reliability given a set of data and assumptions about the subject matter and data generating mechanism. When data are insufficient for estimation, sensitivity studies based on problem-specific knowledge can inform how much reliability estimates can change due to biases in the data and what information should be collected next to provide the most additional information. We apply the approach to a pedagogical example related to a real, but proprietary, engineering application, considering how two types of biases in data can influence a reliability calculation.
Current quantification of margin and uncertainty (QMU) guidance lacks a consistent framework for communicating the credibility of analysis results. Recent efforts at providing QMU guidance have pushed for broadening the analyses supporting QMU results beyond extrapolative statistical models to include a more holistic picture of risk, including information garnered from both experimental campaigns and computational simulations. Credibility guidance would assist in the consideration of belief-based aspects of an analysis. Such guidance exists for presenting computational simulation-based analyses and is under development for the integration of experimental data into computational simulations (calibration or validation), but is absent for the ultimate QMU product resulting from experimental or computational analyses. A QMU credibility assessment framework comprised of five elements is proposed: requirement definitions and quantity of interest selection, data quality, model uncertainty, calibration/parameter estimation, and validation. Through considering and reporting on these elements during a QMU analysis, the decision-maker will receive a more complete description of the analysis and be better positioned to understand the risks involved with using the analysis to support a decision. A molten salt battery application is used to demonstrate the proposed QMU credibility framework.
Causality in an engineered system pertains to how a system output changes due to a controlled change or intervention on the system or system environment. Engineered systems designs reflect a causal theory regarding how a system will work, and predicting the reliability of such systems typically requires knowledge of this underlying causal structure. The aim of this work is to introduce causal modeling tools that inform reliability predictions based on biased data sources. We present a novel application of the popular structural causal modeling (SCM) framework to reliability estimation in an engineering application, illustrating how this framework can inform whether reliability is estimable and how to estimate reliability given a set of data and assumptions about the subject matter and data generating mechanism. When data are insufficient for estimation, sensitivity studies based on problem-specific knowledge can inform how much reliability estimates can change due to biases in the data and what information should be collected next to provide the most additional information. We apply the approach to a pedagogical example related to a real, but proprietary, engineering application, considering how two types of biases in data can influence a reliability calculation.
In experiments conducted on the Z-machine at Sandia National Laboratories, dynamic material properties cannot be analyzed using traditional analytic methods, necessitating solving an inverse problem. Bayesian model calibration is a statistical framework for solving an inverse problem to estimate parameters input into a computational model in the presence of multiple uncertainties. Disentangling input parameter uncertainty and model misspecification is often poorly identified problem. When using computational models for physical parameter estimation, the issue of parameter identifiability must be carefully considered to obtain accurate and precise estimates of physical parameters. Additionally, in dynamic material properties applications, the experimental output is a function, velocity over time. While we can sample an arbitrarily large number of points from the measured velocity, these curves only contain a finite amount of information about the calibration parameters. In this report, we propose modifications to the Bayesian model calibration framework to simplify and improve the estimation of physical parameters with functional outputs. Specifically, we propose scaling the likelihood function by an effective sample size rather than modeling the discrepancy function; and modularizing input nuisance parameters with weakly identified parameters. We evaluate the performance of these proposed methods using a statistical simulation study and then apply these methods to estimate parameters of the tantalum equation of state. We conclude that these proposed methods can provide simple, fast, and statistically valid alternatives to the full Bayesian model calibration procedure; and that these methods can be used to estimate parameters of the equation of state for tantalum.
The current standard Bayesian approach to model calibration, which assigns a Gaussian process prior to the discrepancy term, often suffers from issues of unidentifiability and computational complexity and instability. When the goal is to quantify uncertainty in physical parameters for extrapolative prediction, then there is no need to perform inference on the discrepancy term. With this in mind, we introduce Gibbs posteriors as an alternative Bayesian method for model calibration, which updates the prior with a loss function connecting the data to the parameter. The target of inference is the physical parameter value which minimizes the expected loss. We propose to tune the loss scale of the Gibbs posterior to maintain nominal frequentist coverage under assumptions of the form of model discrepancy, and present a bootstrap implementation for approximating coverage rates. Our approach is highly modular, allowing an analyst to easily encode a wide variety of such assumptions. Furthermore, we provide a principled method of combining posteriors calculated from data subsets. We apply our methods to data from an experiment measuring the material properties of tantalum.
Dynamic material properties experiments provide access to the most extreme temperatures and pressures attainable in a laboratory setting; the data from these experiments are often used to improve our understanding of material models at these extreme conditions. We apply Bayesian model calibration to dynamic material property applications where the experimental output is a function: velocity over time. This framework can accommodate more uncertainties and facilitate analysis of new types of experiments relative to techniques traditionally used to analyse dynamic material experiments. However, implementation of Bayesian model calibration requires more sophisticated statistical techniques, because of the functional nature of the output as well as parameter and model discrepancy identifiability. We propose a novel Bayesian model calibration process to simplify and improve the estimation of the material property calibration parameters. Specifically, we propose scaling the likelihood function by an effective sample size rather than modelling the auto–correlation function to accommodate the functional output. Additionally, we propose sensitivity analyses by using the notion of 'modularization' to assess the effect of experiment–specific nuisance input parameters on estimates of the physical parameters. Furthermore, the Bayesian model calibration framework proposed is applied to dynamic compression of tantalum to extreme pressures, and we conclude that the procedure results in simple, fast and valid inferences on the material properties for tantalum.