Publications

Debusschere, Bert D.; Jakeman, John D.; Chowdhary, Kamaljit S.; Safta, Cosmin S.; Sargsyan, Khachik S.; Rai, P.R.; Ghanem, R.G.; Knio, O.K.; La Maitre, O.L.; Winokur, J.W.; Li, G.L.; Ghattas, O.G.; Moser, R.M.; Simmons, C.S.; Alexanderian, A.A.; Gattiker, J.G.; Higdon, D.H.; Lawrence, E.L.; Bhat, S.B.; Marzouk, Y.M.; Bigoni, D.B.; Cui, T.C.; Parno, M.P.

Abstract not provided.

Mathematics of Computation

Narayan, Akil; Jakeman, John D.; Zhou, Tao

We propose, theoretically investigate, and numerically validate an algorithm for the Monte Carlo solution of least-squares polynomial approximation problems in a collocation framework. Our investigation is motivated by applications in the collocation approximation of parametric functions, which frequently entails construction of surrogates via orthogonal polynomials. A standard Monte Carlo approach would draw samples according to the density defining the orthogonal polynomial family. Our proposed algorithm instead samples with respect to the (weighted) pluripotential equilibrium measure of the domain, and subsequently solves a weighted least-squares problem, with weights given by evaluations of the Christoffel function. We present theoretical analysis to motivate the algorithm, and numerical results that show our method is superior to standard Monte Carlo methods in many situations of interest.

Wildey, Timothy M.; Butler, Troy B.; Jakeman, John D.

Abstract not provided.

Wildey, Timothy M.; Jakeman, John D.; Butler, Troy B.

Abstract not provided.

Jakeman, John D.

Abstract not provided.

Journal of Computational Physics

Gorodetsky, Alex A.; Geraci, Gianluca G.; Eldred, Michael S.; Jakeman, John D.

We describe and analyze a variance reduction approach for Monte Carlo (MC) sampling that accelerates the estimation of statistics of computationally expensive simulation models using an ensemble of models with lower cost. These lower cost models — which are typically lower fidelity with unknown statistics — are used to reduce the variance in statistical estimators relative to a MC estimator with equivalent cost. We derive the conditions under which our proposed approximate control variate framework recovers existing multifidelity variance reduction schemes as special cases. We demonstrate that existing recursive/nested strategies are suboptimal because they use the additional low-fidelity models only to efficiently estimate the unknown mean of the first low-fidelity model. As a result, they cannot achieve variance reduction beyond that of a control variate estimator that uses a single low-fidelity model with known mean. However, there often exists about an order-of-magnitude gap between the maximum achievable variance reduction using all low-fidelity models and that achieved by a single low-fidelity model with known mean. We show that our proposed approach can exploit this gap to achieve greater variance reduction by using non-recursive sampling schemes. The proposed strategy reduces the total cost of accurately estimating statistics, especially in cases where only low-fidelity simulation models are accessible for additional evaluations. Several analytic examples and an example with a hyperbolic PDE describing elastic wave propagation in heterogeneous media are used to illustrate the main features of the methodology.

SIAM Journal on Scientific Computing

Jakeman, John D.; Narayan, Akil; Zhou, Tao

We propose an algorithm for recovering sparse orthogonal polynomial expansions via collocation. A standard sampling approach for recovering sparse polynomials uses Monte Carlo sampling, from the density of orthogonality, which results in poor function recovery when the polynomial degree is high. Our proposed approach aims to mitigate this limitation by sampling with respect to the weighted equilibrium measure of the parametric domain and subsequently solves a preconditioned'1-minimization problem, where the weights of the diagonal preconditioning matrix are given by evaluations of the Christoffel function. Our algorithm can be applied to a wide class of orthogonal polynomial families on bounded and unbounded domains, including all classical families. We present theoretical analysis to motivate the algorithm and numerical results that show our method is superior to standard Monte Carlo methods in many situations of interest. Numerical examples are also provided to demonstrate that our proposed algorithm leads to comparable or improved accuracy even when compared with Legendre- and Hermite-specific algorithms.

Jakeman, John D.

Abstract not provided.

Jakeman, John D.

Abstract not provided.

Jakeman, John D.

Abstract not provided.

Gulian, Mamikon G.; Swiler, Laura P.; Frankel, Ari L.; Safta, Cosmin S.; Jakeman, John D.

Abstract not provided.

Gulian, Mamikon G.; Swiler, Laura P.; Frankel, Ari L.; Jakeman, John D.; Safta, Cosmin S.

Abstract not provided.

Zeng, Xiaoshu Z.; Geraci, Gianluca G.; Eldred, Michael S.; Jakeman, John D.; Gorodetsky, Alex A.; Ghanem, Roger G.

Abstract not provided.

Wildey, Timothy M.; Jakeman, John D.

Abstract not provided.

International Journal for Numerical Methods in Engineering

Jakeman, John D.; Friedman, Sam; Eldred, Michael S.; Tamellini, Lorenzo; Gorodetsky, Alex A.; Allaire, Doug

We present an adaptive algorithm for constructing surrogate models of multi-disciplinary systems composed of a set of coupled components. With this goal we introduce “coupling” variables with a priori unknown distributions that allow surrogates of each component to be built independently. Once built, the surrogates of the components are combined to form an integrated-surrogate that can be used to predict system-level quantities of interest at a fraction of the cost of the original model. The error in the integrated-surrogate is greedily minimized using an experimental design procedure that allocates the amount of training data, used to construct each component-surrogate, based on the contribution of those surrogates to the error of the integrated-surrogate. The multi-fidelity procedure presented is a generalization of multi-index stochastic collocation that can leverage ensembles of models of varying cost and accuracy, for one or more components, to reduce the computational cost of constructing the integrated-surrogate. Extensive numerical results demonstrate that, for a fixed computational budget, our algorithm is able to produce surrogates that are orders of magnitude more accurate than methods that treat the integrated system as a black-box.

Jakeman, John D.; Butler, Troy B.; Eldred, Michael S.; Geraci, Gianluca G.; Gorodetsky, Alex A.; Wildey, Timothy M.

Abstract not provided.

International Journal for Numerical Methods in Engineering

Jakeman, John D.; Eldred, Michael S.; Geraci, Gianluca; Gorodetsky, Alex

In this paper, we present an adaptive algorithm to construct response surface approximations of high-fidelity models using a hierarchy of lower fidelity models. Our algorithm is based on multi-index stochastic collocation and automatically balances physical discretization error and response surface error to construct an approximation of model outputs. This surrogate can be used for uncertainty quantification (UQ) and sensitivity analysis (SA) at a fraction of the cost of a purely high-fidelity approach. We demonstrate the effectiveness of our algorithm on a canonical test problem from the UQ literature and a complex multiphysics model that simulates the performance of an integrated nozzle for an unmanned aerospace vehicle. We find that, when the input-output response is sufficiently smooth, our algorithm produces approximations that can be over two orders of magnitude more accurate than single fidelity approximations for a fixed computational budget.

Eldred, Michael S.; Geraci, Gianluca G.; Gorodetsky, Alex A.; Jakeman, John D.

Abstract not provided.

Friedman, Sam F.; Jakeman, John D.; Eldred, Michael S.; Tamellini, Lorenzo T.; Gorodestky, Alex G.; Allaire, Doug A.

We present an adaptive algorithm for constructing surrogate models for integrated systems composed of a set of coupled components. With this goal we introduce ‘coupling’ variables with a priori unknown distributions that allow approximations of each component to be built independently. Once built, the surrogates of the components are combined and used to predict system-level quantities of interest (QoI) at a fraction of the cost of interrogating the full system model. We use a greedy experimental design procedure, based upon a modification of Multi-Index Stochastic Collocation (MISC), to minimize the error of the combined surrogate. This is achieved by refining each component surrogate in accordance with its relative contribution to error in the approximation of the system-level QoI. Our adaptation of MISC is a multi-fidelity procedure that can leverage ensembles of models of varying cost and accuracy, for one or more components, to produce estimates of system-level QoI. Several numerical examples demonstrate the efficacy of the proposed approach on systems involving feed-forward and feedback coupling. For a fixed computational budget, the proposed algorithm is able to produce approximations that are orders of magnitude more accurate than approximations that treat the integrated system as a black-box.

Jakeman, John D.; Friedman, Samuel F.; Eldred, Michael S.; Tamellini, Lorenzo T.; Gorodetsky, Alex A.; Allaire, Doug A.

Abstract not provided.

Jakeman, John D.; Wildey, Timothy M.; Eldred, Michael S.

Abstract not provided.

Wildey, Timothy M.; Jakeman, John D.; Butler, Troy B.; Cyr, Eric C.; Shadid, John N.

Abstract not provided.

Wildey, Timothy M.; Shadid, John N.; Cyr, Eric C.; Jakeman, John D.; Butler, Troy B.

Abstract not provided.

Perego, Mauro P.; Price, Stephen F.; Stadler, Georg S.; Eldred, Michael S.; Jackson, Charles J.; Jakeman, John D.; Salinger, Andrew G.; Kalashnikova, Irina

Abstract not provided.

Eldred, Michael S.; Domino, Stefan P.; Barone, Matthew F.; Jakeman, John D.

Abstract not provided.

Wildey, Timothy M.; Jakeman, John D.; Butler, Troy B.

Abstract not provided.

Kalashnikova, Irina; Salinger, Andrew G.; Perego, Mauro P.; Jakeman, John D.; Eldred, Michael S.; Demeshko, Irina D.; Tuminaro, Raymond S.; Price, Stephen F.

Abstract not provided.

Mathematics for Industry

Jakeman, Anthony J.; Jakeman, John D.

Uncertainty pervades the representation of systems in the water–environment–agriculture cross-sector. Successful methods to address uncertainties have largely focused on standard mathematical formulations of biophysical processes in a single sector, such as partial or ordinary differential equations. More attention to integrated models of such systems is warranted. Model components representing the different sectors of an integrated model can have less standard, and different, formulations to one another, as well as different levels of epistemic knowledge and data informativeness. Thus, uncertainty is not only pervasive but also crosses boundaries and propagates between system components. Uncertainty assessment (UA) cries out for more eclectic treatment in these circumstances, some of it being more qualitative and empirical. Here in this paper, we discuss the various sources of uncertainty in such a cross-sectoral setting and ways to assess and manage them. We have outlined a fast-growing set of methodologies, particularly in the computational mathematics literature on uncertainty quantification (UQ), that seem highly pertinent for uncertainty assessment. There appears to be considerable scope for advancing UA by integrating relevant UQ techniques into cross-sectoral problem applications. Of course this will entail considerable collaboration between domain specialists who often take first ownership of the problem and computational methods experts.

Najm, H.N.; Eldred, Michael S.; Debusschere, Bert D.; Chowdhary, Kamaljit S.; Jakeman, John D.; Safta, Cosmin S.; Sargsyan, Khachik S.

Abstract not provided.

Safta, Cosmin S.; Reid, Timothy R.; Jakeman, John D.; Sargsyan, Khachik S.

Abstract not provided.

Peterson, Kara J.; Parks, Michael L.; Ackerman, Eric A.; Bambha, Ray B.; Bull, Diana L.; Frederick, Jennifer M.; Hardesty, Jasper O.; Ilgen, Anastasia G.; Jakeman, John D.; Powell, Amy J.; Peterson, Matthew G.; Roesler, Erika L.; Safta, Cosmin S.; Stracuzzi, David J.; Kalashnikova, Irina

Abstract not provided.

Environmental Modelling and Software

Wang, Qian; Guillaume, Joseph H.A.; Jakeman, John D.; Yang, Tao; Iwanaga, Takuya; Croke, Barry; Jakeman, Anthony J.

Despite widespread use of factor fixing in environmental modeling, its effect on model predictions has received little attention and is instead commonly presumed to be negligible. We propose a proof-of-concept adaptive method for systematically investigating the impact of factor fixing. The method uses Global Sensitivity Analysis methods to identify groups of sensitive parameters, then quantifies which groups can be safely fixed at nominal values without exceeding a maximum acceptable error, demonstrated using the 21-dimensional Sobol’ G-function. Three error measures are considered for quantities of interest, namely Relative Mean Absolute Error, Pearson Product-Moment Correlation and Relative Variance. Results demonstrate that factor fixing may cause large errors in the model results unexpectedly, when preliminary analysis suggests otherwise, and that the default value selected affects the number of factors to fix. To improve the applicability and methodological development of factor fixing, a new research agenda encompassing five opportunities is discussed for further attention.

Reese, William M.; Hart, Joseph L.; van Bloemen Waanders, Bart G.; Perergo, Mauro P.; Jakeman, John D.; Saibaba, Arvind S.

Abstract not provided.

Morrow, Zachary B.; van Bloemen Waanders, Bart G.; Jakeman, John D.

Abstract not provided.

Communications in Computational Physics

Harbrecht, Helumt; Jakeman, John D.; Zaspel, Peter

Gaussian processes and other kernel-based methods are used extensively to construct approximations of multivariate data sets. The accuracy of these approximations is dependent on the data used. This paper presents a computationally efficient algorithm to greedily select training samples that minimize the weighted Lp error of kernel-based approximations for a given number of data. The method successively generates nested samples, with the goal of minimizing the error in high probability regions of densities specified by users. The algorithm presented is extremely simple and can be implemented using existing pivoted Cholesky factorization methods. Training samples are generated in batches which allows training data to be evaluated (labeled) in parallel. For smooth kernels, the algorithm performs comparably with the greedy integrated variance design but has significantly lower complexity. Numerical experiments demonstrate the efficacy of the approach for bounded, unbounded, multi-modal and non-tensor product densities. We also show how to use the proposed algorithm to efficiently generate surrogates for inferring unknown model parameters from data using Bayesian inference.

Wildey, Timothy M.; Butler, Troy B.; Jakeman, John D.

Abstract not provided.

Jakeman, John D.

Abstract not provided.

SIAM-ASA Journal on Uncertainty Quantification

Adcock, Ben; Bao, Anyi; Jakeman, John D.; Narayan, Akil

The recovery of approximately sparse or compressible coefficients in a polynomial chaos expansion is a common goal in many modern parametric uncertainty quantification (UQ) problems. However, relatively little effort in UQ has been directed toward theoretical and computational strategies for addressing the sparse corruptions problem, where a small number of measurements are highly corrupted. Such a situation has become pertinent today since modern computational frameworks are sufficiently complex with many interdependent components that may introduce hardware and software failures, some of which can be difficult to detect and result in a highly polluted simulation result. In this paper we present a novel compressive sampling-based theoretical analysis for a regularized \ell1 minimization algorithm that aims to recover sparse expansion coefficients in the presence of measurement corruptions. Our recovery results are uniform (the theoretical guarantees hold for all compressible signals and compressible corruptions vectors) and prescribe algorithmic regularization parameters in terms of a user-defined a priori estimate on the ratio of measurements that are believed to be corrupted. We also propose an iteratively reweighted optimization algorithm that automatically refines the value of the regularization parameter and empirically produces superior results. Our numerical results test our framework on several medium to high dimensional examples of solutions to parameterized differential equations and demonstrate the effectiveness of our approach.

Adcock, Ben A.; Bao, Anyi B.; Jakeman, John D.; Naryan, Akil N.

The recovery of approximately sparse or compressible coefficients in a polynomial chaos expansion is a common goal in many modern parametric uncertainty quantification (UQ) problems. However, relatively little effort in UQ has been directed toward theoretical and computational strategies for addressing the sparse corruptions problem, where a small number of measurements are highly corrupted. Such a situation has become pertinent today since modern computational frameworks are sufficiently complex with many interdependent components that may introduce hardware and software failures, some of which can be difficult to detect and result in a highly polluted simulation result. In this paper we present a novel compressive sampling-based theoretical analysis for a regularized t1 minimization algorithm that aims to recover sparse expansion coefficients in the presence of measurement corruptions. Our recovery results are uniform (the theoretical guarantees hold for all compressible signals and compressible corruptions vectors), and prescribe algorithmic regularization parameters in terms of a user-defined a priori estimate on the ratio of measurements that are believed to be corrupted. We also propose an iteratively reweighted optimization algorithm that automatically refines the value of the regularization parameter, and empirically produces superior results. Our numerical results test our framework on several medium-to-high dimensional examples of solutions to parameterized differential equations, and demonstrate the effectiveness of our approach.

Bao, Anyi B.; Adcock, Ben A.; Jakeman, John D.; Narayan, Akil N.

Abstract not provided.

Perego, Mauro P.; Jakeman, John D.; Price, S.P.; Salinger, Andrew G.; Stadler, G.S.; Kalashnikova, Irina

Abstract not provided.

Wildey, Timothy M.; Butler, Troy B.; Jakeman, John D.; Marvin, Brad M.

Abstract not provided.

Swiler, Laura P.; Gulian, Mamikon G.; Frankel, Ari L.; Safta, Cosmin S.; Jakeman, John D.

Abstract not provided.

Jakeman, John D.

Abstract not provided.

Jakeman, John D.; Eldred, Michael S.

Abstract not provided.

Wildey, Timothy M.; Butler, Troy B.; Jakeman, John D.

Abstract not provided.

Adams, Brian M.; Jakeman, John D.; Swiler, Laura P.; Stephens, John A.; Vigil, Dena V.; Wildey, Timothy M.; Bauman, Lara E.; Bohnhoff, William J.; Dalbey, Keith D.; Eddy, John P.; Ebeida, Mohamed S.; Eldred, Michael S.; Hough, Patricia D.; Hu, Kenneth H.

The Dakota (Design Analysis Kit for Optimization and Terascale Applications) toolkit provides a exible and extensible interface between simulation codes and iterative analysis methods. Dakota contains algorithms for optimization with gradient and nongradient-based methods; uncertainty quanti cation with sampling, reliability, and stochastic expansion methods; parameter estimation with nonlinear least squares methods; and sensitivity/variance analysis with design of experiments and parameter study methods. These capabilities may be used on their own or as components within advanced strategies such as surrogate-based optimization, mixed integer nonlinear programming, or optimization under uncertainty. By employing object-oriented design to implement abstractions of the key components required for iterative systems analyses, the Dakota toolkit provides a exible and extensible problem-solving environment for design and performance analysis of computational models on high performance computers. This report serves as a user's manual for the Dakota software and provides capability overviews and procedures for software execution, as well as a variety of example studies.

Adams, Brian M.; Jakeman, John D.; Swiler, Laura P.; Stephens, John A.; Vigil, Dena V.; Wildey, Timothy M.; Bauman, Lara E.; Bohnhoff, William J.; Dalbey, Keith D.; Eddy, John P.; Ebeida, Mohamed S.; Eldred, Michael S.; Hough, Patricia D.; Hu, Kenneth H.

The Dakota (Design Analysis Kit for Optimization and Terascale Applications) toolkit provides a exible and extensible interface between simulation codes and iterative analysis methods. Dakota contains algorithms for optimization with gradient and nongradient-based methods; uncertainty quanti cation with sampling, reliability, and stochastic expansion methods; parameter estimation with nonlinear least squares methods; and sensitivity/variance analysis with design of experiments and parameter study methods. These capabilities may be used on their own or as components within advanced strategies such as surrogate-based optimization, mixed integer nonlinear programming, or optimization under uncertainty. By employing object-oriented design to implement abstractions of the key components required for iterative systems analyses, the Dakota toolkit provides a exible and extensible problem-solving environment for design and performance analysis of computational models on high performance computers. This report serves as a theoretical manual for selected algorithms implemented within the Dakota software. It is not intended as a comprehensive theoretical treatment, since a number of existing texts cover general optimization theory, statistical analysis, and other introductory topics. Rather, this manual is intended to summarize a set of Dakota-related research publications in the areas of surrogate-based optimization, uncertainty quanti cation, and optimization under uncertainty that provide the foundation for many of Dakota's iterative analysis capabilities.

SIAM Journal on Scientific Computing

Qin, Tong; Chen, Zhen; Jakeman, John D.; Xiu, Dongbin

We present a numerical framework for recovering unknown nonautonomous dynamical systems with time-dependent inputs. To circumvent the difficulty presented by the nonautonomous nature of the system, our method transforms the solution state into piecewise integration of the system over a discrete set of time instances. The time-dependent inputs are then locally parameterized by using a proper model, for example, polynomial regression, in the pieces determined by the time instances. This transforms the original system into a piecewise parametric system that is locally time invariant. We then design a deep neural network structure to learn the local models. Once the network model is constructed, it can be iteratively used over time to conduct global system prediction. We provide theoretical analysis of our algorithm and present a number of numerical examples to demonstrate the effectiveness of the method.

Wildey, Timothy M.; Butler, Troy B.; Jakeman, John D.; Seidl, Daniel T.; van Bloemen Waanders, Bart G.

Abstract not provided.