Publications

Results 26–50 of 287

Search results

Jump to search filters

The role of stiffness in training and generalization of ResNets

Journal of Machine Learning for Modeling and Computing

Najm, Habib N.; Sargsyan, Khachik; D'Elia, Marta

Neural ordinary differential equations (NODEs) have recently regained popularity as large-depth limits of a large class of neural networks. In particular, residual neural networks (ResNets) are equivalent to an explicit Euler discretization of an underlying NODE, where the transition from one layer to the next is one time step of the discretization. The relationship between continuous and discrete neural networks has been of particular interest. Notably, analysis from the ordinary differential equation viewpoint can potentially lead to new insights for understanding the behavior of neural networks in general. In this work, we take inspiration from differential equations to define the concept of stiffness for a ResNet via the interpretation of a ResNet as the discretization of a NODE. Here, we then examine the effects of stiffness on the ability of a ResNet to generalize, via computational studies on example problems coming from climate and chemistry models. We find that penalizing stiffness does have a unique regularizing effect, but we see no benefit to penalizing stiffness over L2 regularization (penalization of network parameter norms) in terms of predictive performance.

More Details

Quantifying modeling uncertainty in simplified beam models for building response prediction

Structural Control and Health Monitoring

Ghahari, S.F.; Sargsyan, Khachik; Celebi, Mehmet; Taciroglu, Ertugrul

The use of simple models for response prediction of building structures is preferred in earthquake engineering for risk evaluations at regional scales, as they make computational studies more feasible. The primary impediment in their gainful use presently is the lack of viable methods for quantifying (and reducing upon) the modeling errors/uncertainties they bear. This study presents a Bayesian calibration method wherein the modeling error is embedded into the parameters of the model. The method is specifically described for coupled shear-flexural beam models here, but it can be applied to any parametric surrogate model. The major benefit the method offers is the ability to consider the modeling uncertainty in the forward prediction of any degree-of-freedom or composite response regardless of the data used in calibration. The method is extensively verified using two synthetic examples. In the first example, the beam model is calibrated to represent a similar beam model but with enforced modeling errors. In the second example, the beam model is used to represent the detailed finite element model of a 52-story building. Both examples show the capability of the proposed solution to provide realistic uncertainty estimation around the mean prediction.

More Details

Using a surrogate-assisted Bayesian framework to calibrate the runoff-generation scheme in the Energy Exascale Earth System Model (E3SM) v1

Geoscientific Model Development

Xu, Donghui; Bisht, Gautam; Sargsyan, Khachik; Liao, Chang; Ruby Leung, L.

Runoff is a critical component of the terrestrial water cycle, and Earth system models (ESMs) are essential tools to study its spatiotemporal variability. Runoff schemes in ESMs typically include many parameters so that model calibration is necessary to improve the accuracy of simulated runoff. However, runoff calibration at a global scale is challenging because of the high computational cost and the lack of reliable observational datasets. In this study, we calibrated 11 runoff relevant parameters in the Energy Exascale Earth System Model (E3SM) Land Model (ELM) using a surrogate-assisted Bayesian framework. First, the polynomial chaos expansion machinery with Bayesian compressed sensing is used to construct computationally inexpensive surrogate models for ELM-simulated runoff at 0.5 × 0.5 for 1991-2010. The error metric between the ELM simulations and the benchmark data is selected to construct the surrogates, which facilitates efficient calibration and avoids the more conventional, but challenging, construction of high-dimensional surrogates for the ELM simulated runoff. Second, the Sobol' index sensitivity analysis is performed using the surrogate models to identify the most sensitive parameters, and our results show that, in most regions, ELM-simulated runoff is strongly sensitive to 3 of the 11 uncertain parameters. Third, a Bayesian method is used to infer the optimal values of the most sensitive parameters using an observation-based global runoff dataset as the benchmark. Our results show that model performance is significantly improved with the inferred parameter values. Although the parametric uncertainty of simulated runoff is reduced after the parameter inference, it remains comparable to the multimodel ensemble uncertainty represented by the global hydrological models in ISMIP2a. Additionally, the annual global runoff trend during the simulation period is not well constrained by the inferred parameter values, suggesting the importance of including parametric uncertainty in future runoff projections.

More Details

Quantification of the effect of uncertainty on impurity migration in PISCES-A simulated with GITR

Nuclear Fusion

Younkin, T.R.; Sargsyan, Khachik; Casey, Tiernan A.; Najm, Habib N.; Canik, J.M.; Green, D.L.; Doerner, R.P.; Nishijima, D.; Baldwin, M.; Drobny, J.; Curreli, D.; Wirth, B.D.

A Bayesian inference strategy has been used to estimate uncertain inputs to global impurity transport code (GITR) modeling predictions of tungsten erosion and migration in the linear plasma device, PISCES-A. This allows quantification of GITR output uncertainty based on the uncertainties in measured PISCES-A plasma electron density and temperature profiles (n e, T e) used as inputs to GITR. The technique has been applied for comparison to dedicated experiments performed for high (4 × 1022 m-2 s-1) and low (5 × 1021 m-2 s-1) flux 250 eV He-plasma exposed tungsten (W) targets designed to assess the net and gross erosion of tungsten, and corresponding W impurity transport. The W target design and orientation, impurity collector, and diagnostics, have been designed to eliminate complexities associated with tokamak divertor plasma exposures (inclined target, mixed plasma species, re-erosion, etc) to benchmark results against the trace impurity transport model simulated by GITR. The simulated results of the erosion, migration, and re-deposition of W during the experiment from the GITR code coupled to materials response models are presented. Specifically, the modeled and experimental W I emission spectroscopy data for a 429.4 nm line and net erosion through the target and collector mass difference measurements are compared. The methodology provides predictions of observable quantities of interest with quantified uncertainty, allowing estimation of moments, together with the sensitivities to plasma temperature and density.

More Details

UQTk Version 3.1.2 User Manual

Sargsyan, Khachik; Safta, Cosmin; Boll, Luke; Johnston, Katherine; Khalil, Mohammad; Chowdhary, Kenny; Rai, Prashant; Casey, Tiernan A.; Zeng, Xiaoshu; Debusschere, Bert J.

The UQ Toolkit (UQTk) is a collection of libraries and tools for the quantification of uncertainty in numerical model predictions. Version 3.1.2 offers intrusive and non-intrusive methods for propagating input uncertainties through computational models, tools for sensitivity analysis, methods for sparse surrogate construction, and Bayesian inference tools for inferring parameters from experimental data. This manual discusses the download and installation process for UQTk, provides pointers to the UQ methods used in the toolkit, and describes some of the examples provided with the toolkit.

More Details

Breaking Down the Computational Barriers to Real-Time Urban Flood Forecasting

Geophysical Research Letters

Ivanov, Valeriy Y.; Xu, Donghui; Dwelle, M.C.; Sargsyan, Khachik

Flooding impacts are on the rise globally, and concentrated in urban areas. Currently, there are no operational systems to forecast flooding at spatial resolutions that can facilitate emergency preparedness and response actions mitigating flood impacts. We present a framework for real-time flood modeling and uncertainty quantification that combines the physics of fluid motion with advances in probabilistic methods. The framework overcomes the prohibitive computational demands of high-fidelity modeling in real-time by using a probabilistic learning method relying on surrogate models that are trained prior to a flood event. This shifts the overwhelming burden of computation to the trivial problem of data storage, and enables forecasting of both flood hazard and its uncertainty at scales that are vital for time-critical decision-making before and during extreme events. The framework has the potential to improve flood prediction and analysis and can be extended to other hazard assessments requiring intense high-fidelity computations in real-time.

More Details

Adsorbate Partition Functions via Phase Space Integration: Quantifying the Effect of Translational Anharmonicity on Thermodynamic Properties

Journal of Physical Chemistry C

Blondal, Katrin; Sargsyan, Khachik; Bross, David H.; Ruscic, Branko; Goldsmith, C.F.

A new method for computing anharmonic thermophysical properties for adsorbates on metal surfaces is presented. Classical Monte Carlo phase space integration is performed to calculate the partition function for the motion of a hydrogen atom on Cu(111). A minima-preserving neural network potential energy surface is used within the integration routine. Two different sampling schema for generating the training data are presented, and two different density functionals are used. The results are benchmarked against direct state counting results by using discrete variable representation. The phase space integration results are in excellent quantitative agreement with the benchmark results. Additionally, both the discrete variable representation and the phase space integration results confirm that the motion of H on Cu(111) is highly anharmonic. The results were applied to calculate the free energy of dissociative adsorption of H2 and the resulting Langmuir isotherms at 400, 800, and 1200 K in a partial pressure range of 0-1 bar. It shows that the anharmonic effects lead to significantly higher predicted surface site fractions of hydrogen.

More Details
Results 26–50 of 287
Results 26–50 of 287
Top