Our primary aim in this work is to understand how to efficiently obtain reliable uncertainty quantification in automatic learning algorithms with limited training datasets. Standard approaches rely on cross-validation to tune hyper parameters. Unfortunately, when our datasets are too small, holdout datasets become unreliable—albeit unbiased—measures of prediction quality due to the lack of adequate sample size. We should not place confidence in holdout estimators under conditions wherein the sample variance is both large and unknown. More poigniantly, our training experiments on limited data (Duersch and Catanach, 2021) show that even if we could improve estimator quality under these conditions, the typical training trajectory may never even encounter generalizable models.
The Bayesian optimal experimental design (OED) problem seeks to identify data, sensor configurations, or experiments which can optimally reduce uncertainty. The goal of OED is to find an experiment that maximizes the expected information gain (EIG) about quantities of interest given prior knowledge about expected data. Therefore, within the context of seismic monitoring, we can use Bayesian OED to configure sensor networks by choosing sensor locations, types, and fidelity in order to improve our ability to identify and locate seismic sources. In this work, we develop the framework necessary to use Bayesian OED to optimize the ability to locate seismic events from arrival time data of detected seismic phases. In order to do utilize Bayesian OED we must develop four elements:1. A likelihood function that describes the uncertainty of detection and travel times; 2. A Bayesian solver that takes a prior and likelihood to identify the posterior; 3. An algorithm to compute EIG; and, 4. An optimizer that finds a sensor network which maximizes EIG. Once we have developed this framework, we can explore many relevant questions to monitoring such as: how and what multiphenomenology data can be used to optimally reduce uncertainty, how to trade off sensor fidelity and earth model uncertainty, and how sensor types, number, and locations influence uncertainty
Understanding the effect of a high-altitude electromagnetic pulse (HEMP) on the equipment in the United States electrical power grid is important to national security. A present challenge to this understanding is evaluating the vulnerability of transformers to a HEMP. Evaluating vulnerability by direct testing is cost-prohibitive, due to the wide variation in transformers, their high cost, and the large number of tests required to establish vulnerability with confidence. Alternatively, material and component testing can be performed to quantify a model for transformer failure, and the model can be used to assess vulnerability of a wide variety of transformers. This project develops a model of the probability of equipment failure due to effects of a HEMP. Potential failure modes are cataloged, and a model structure is presented which can be quantified by the results of small-scale coupon tests.
Catanach, Thomas A.; McCardell, Reed M.; Baetica, Ania-Ariadna B.; Murray, RIchard M.
It has been an ongoing scientific debate whether biological parameters are conserved across experimental setups with different media, pH values, and other experimental conditions. Our work explores this question using Bayesian probability as a rigorous framework to assess the biological context of parameters in a model of the cell growth controller in You et al. When this growth controller is uninduced, the E. coli cell population grows to carrying capacity; however, when the circuit is induced, the cell population growth is regulated to remain well below carrying capacity. This growth control controller regulates the E. coli cell population by cell to cell communication using the signaling molecule AHL and by cell death using the bacterial toxin CcdB. To evaluate the context dependence of parameters such as the cell growth rate, the carrying capacity, the AHL degradation rate, the leakiness of AHL, the leakiness of toxin CcdB, and the IPTG induction factor, we collect experimental data from the growth control circuit in two different media, at two different pH values, and with several induction levels. We define a set of possible context dependencies that describe how these parameters may differ with the experimental conditions and we develop mathematical models of the growth controller across the different experimental contexts. We then determine whether these parameters are shared across experimental contexts or whether they are context dependent. For each of these possible context dependencies, we use Bayesian inference to assess its plausibility and to estimate the parameters of the growth controller. Ultimately, we find that there is significant experimental context dependence in this circuit. Furthermore, we also find that the estimated parameter values are sensitive to our assumption of a context relationship.