Publications

Abstract not provided.

Abstract not provided.

Abstract not provided.

This report summarizes methods to incorporate information (or lack of information) about inter-variable dependence into risk assessments that use Dempster-Shafer theory or probability bounds analysis to address epistemic and aleatory uncertainty. The report reviews techniques for simulating correlated variates for a given correlation measure and dependence model, computation of bounds on distribution functions under a specified dependence model, formulation of parametric and empirical dependence models, and bounding approaches that can be used when information about the intervariable dependence is incomplete. The report also reviews several of the most pervasive and dangerous myths among risk analysts about dependence in probabilistic models.

Abstract not provided.

The views of state of art in verification and validation (V & V) in computational physics are discussed. These views are described in the framework in which predictive capability relies on V & V, as well as other factors that affect predictive capability. Some of the research topics addressed are development of improved procedures for the use of the phenomena identification and ranking table (PIRT) for prioritizing V & V activities, and the method of manufactured solutions for code verification. It also addressed development and use of hierarchical validation diagrams, and the construction and use of validation metrics incorporating statistical measures.

This report presents a perspective on the role of code comparison activities in verification and validation. We formally define the act of code comparison as the Code Comparison Principle (CCP) and investigate its application in both verification and validation. One of our primary conclusions is that the use of code comparisons for validation is improper and dangerous. We also conclude that while code comparisons may be argued to provide a beneficial component in code verification activities, there are higher quality code verification tasks that should take precedence. Finally, we provide a process for application of the CCP that we believe is minimal for achieving benefit in verification processes.

Several simple test problems are used to explore the following approaches to the representation of the uncertainty in model predictions that derives from uncertainty in model inputs: probability theory, evidence theory, possibility theory, and interval analysis. Each of the test problems has rather diffuse characterizations of the uncertainty in model inputs obtained from one or more equally credible sources. These given uncertainty characterizations are translated into the mathematical structure associated with each of the indicated approaches to the representation of uncertainty and then propagated through the model with Monte Carlo techniques to obtain the corresponding representation of the uncertainty in one or more model predictions. The different approaches to the representation of uncertainty can lead to very different appearing representations of the uncertainty in model predictions even though the starting information is exactly the same for each approach. To avoid misunderstandings and, potentially, bad decisions, these representations must be interpreted in the context of the theory/procedure from which they derive.

Abstract not provided.

Abstract not provided.

This report summarizes a variety of the most useful and commonly applied methods for obtaining Dempster-Shafer structures, and their mathematical kin probability boxes, from empirical information or theoretical knowledge. The report includes a review of the aggregation methods for handling agreement and conflict when multiple such objects are obtained from different sources.

Abstract not provided.

Dempster-Shafer theory offers an alternative to traditional probabilistic theory for the mathematical representation of uncertainty. The significant innovation of this framework is that it allows for the allocation of a probability mass to sets or intervals. Dempster-Shafer theory does not require an assumption regarding the probability of the individual constituents of the set or interval. This is a potentially valuable tool for the evaluation of risk and reliability in engineering applications when it is not possible to obtain a precise measurement from experiments, or when knowledge is obtained from expert elicitation. An important aspect of this theory is the combination of evidence obtained from multiple sources and the modeling of conflict between them. This report surveys a number of possible combination rules for Dempster-Shafer structures and provides examples of the implementation of these rules for discrete and interval-valued data.

This report presents general concepts in a broadly applicable methodology for validation of Accelerated Strategic Computing Initiative (ASCI) codes for Defense Programs applications at Sandia National Laboratories. The concepts are defined and analyzed within the context of their relative roles in an experimental validation process. Examples of applying the proposed methodology to three existing experimental validation activities are provided in appendices, using an appraisal technique recommended in this report.

The verification and validation (V & V) in computational fluid dynamics was presented. The methods and procedures for assessing V & V were presented. The issues such as code verification versus solution verification, model validation versus solution validation, the distinction between error and uncertainity, conceptual sources of error and uncertainity, and the relationship between validation and prediction was discussed. Methods for determining the accuracy of numerical solutions were presented and the importance of software testing during verification activities were emphasized.

Abstract not provided.

This research effort focuses on methodology for quantifying the effects of model uncertainty and discretization error on computational modeling and simulation. The work is directed towards developing methodologies which treat model form assumptions within an overall framework for uncertainty quantification, for the purpose of developing estimates of total prediction uncertainty. The present effort consists of work in three areas: framework development for sources of uncertainty and error in the modeling and simulation process which impact model structure; model uncertainty assessment and propagation through Bayesian inference methods; and discretization error estimation within the context of non-deterministic analysis.

Verification and validation are the primary means to assess accuracy and reliability in computational simulations. This paper presents an extensive review of the literature in computational validation and develops a number of extensions to existing ideas. We discuss the early work in validation by the operations research, statistics, and CFD communities. The emphasis in our review is to bring together the diverse contributors to validation methodology and procedures. The disadvantages of standard practice of qualitative graphical validation are pointed out and the arguments for and the literature on validation quantification are presented. We discuss the attributes of a beneficial validation experiment hierarchy and then we give an example for a complex system; a hypersonic cruise missile. We present six recommended characteristics of how a validation experiment is designed, executed, and analyzed. Since one of the key features of a validation experiment is a careful experimental uncertainty estimation analysis, we discuss a statistical procedure that has been developed for improving the estimation of experimental uncertainty. One facet of code verification, the estimation of computational error and uncertainty, is discussed in some detail, but we do not address many other important issues in code verification. We argue for the separation of the concepts of error and uncertainty in computational simulations. Error estimation, primarily that due to numerical solution error, is discussed with regard to its importance in validation. In the same vein, we explain the need to move toward nondeterministic simulations in CFD validation, that is, the propagation of input quantity uncertainty in CFD simulations which yield probabilistic output quantities. We discuss the relatively new concept of validation quantification, also referred to as validation metrics. The inadequacy, in our view, of hypothesis testing in computational validation is discussed. We close the paper by presenting our ideas on validation metrics and we apply them to two conceptual examples. © 2000 The American Institute of Aeronautics and Astronautics Inc.

Abstract not provided.

A bibliography has been compiled dealing with the verification and validation of computational simulations. The references listed in this bibliography are concentrated in the field of computational fluid dynamics (CFD). However, references from the following fields are also included: operations research, heat transfer, solid dynamics, software quality assurance, software accreditation, military systems, and nuclear reactor safety. This bibliography, containing 221 references, is not meant to be comprehensive. It was compiled during the last ten years in response to the author's interest and research in the methodology for verification and validation. The emphasis in the bibliography is in the following areas: philosophy of science underpinnings, development of terminology and methodology, high accuracy solutions for CFD verification, experimental datasets for CFD validation, and the statistical quantification of model validation. This bibliography should provide a starting point for individual researchers in many fields of computational simulation in science and engineering.

The present paper addresses the question: ``What are the general classes of uncertainty and error sources in complex, computational simulations?`` This is the first step of a two step process to develop a general methodology for quantitatively estimating the global modeling and simulation uncertainty in computational modeling and simulation. The second step is to develop a general mathematical procedure for representing, combining and propagating all of the individual sources through the simulation. The authors develop a comprehensive view of the general phases of modeling and simulation. The phases proposed are: conceptual modeling of the physical system, mathematical modeling of the system, discretization of the mathematical model, computer programming of the discrete model, numerical solution of the model, and interpretation of the results. This new view is built upon combining phases recognized in the disciplines of operations research and numerical solution methods for partial differential equations. The characteristics and activities of each of these phases is discussed in general, but examples are given for the fields of computational fluid dynamics and heat transfer. They argue that a clear distinction should be made between uncertainty and error that can arise in each of these phases. The present definitions for uncertainty and error are inadequate and. therefore, they propose comprehensive definitions for these terms. Specific classes of uncertainty and error sources are then defined that can occur in each phase of modeling and simulation. The numerical sources of error considered apply regardless of whether the discretization procedure is based on finite elements, finite volumes, or finite differences. To better explain the broad types of sources of uncertainty and error, and the utility of their categorization, they discuss a coupled-physics example simulation.

The issues of verification, calibration, and validation of computational fluid dynamics (CFD) codes has been receiving increasing levels of attention in the research literature and in engineering technology. Both CFD researchers and users of CFD codes are asking more critical and detailed questions concerning the accuracy, range of applicability, reliability and robustness of CFD codes and their predictions. This is a welcomed trend because it demonstrates that CFD is maturing from a research tool to the world of impacting engineering hardware and system design. In this environment, the broad issue of code quality assurance becomes paramount, However, the philosophy and methodology of building confidence in CFD code predictions has proven to be more difficult than many expected. A wide variety of physical modeling errors and discretization errors are discussed. Here, discretization errors refer to all errors caused by conversion of the original partial differential equations to algebraic equations, and their solution. Boundary conditions for both the partial differential equations and the discretized equations will be discussed. Contrasts are drawn between the assumptions and actual use numerical method consistency and stability. Commen are also made concerning the existence and uniqueness solutions for both the partial differential equations and the discrete equations. Various techniques are suggested for the detection and estimation of errors caused by physical modeling and discretization of the partial differential equations.

Extensive surface pressure measurements were obtained on a hypersonic vehicle configuration at Mach 8. All of the experimental results were obtained in the Sandia National Laboratories Mach 8 hypersonic wind tunnel for lamipar boundary layer conditions. The basic vehicle configuration IS a spherically blunted 100 half-angle cone with a slice parallel with the axis of the vehicle. The bluntness ratio of the geometly IS 10% and the slice begins at 70% of the length of the vehicle. Surface pressure measurements were obtained for angles of attack from -10 to +180. for various roll angles, at 96 locations on the body surface. A new and innovative uncertainty analysis was devised to estimate the contributors to surface pressure meaSment uncenainty. Quantitative estimates were computed for the uncertainty contributions due to the complete insmmentation system, nonunifoxmity of flow in the test section of the wind Nnnel. and variations in the wind tunnel model. This extensive set of high-quality surface pressure measurements is recommended for use in the calibration and validation of computational fluid dynamics codes for hyuersonic flow conditions.

A comprehensive review of experimental base pressure and base heating data related to supersonic and hypersonic flight vehicles has been completed. Particular attention was paid to free-flight data as well as wind tunnel data for models without rear sting support. Using theoretically based correlation parameters, a series of internally consistent, empirical prediction equations has been developed for planar and axisymmetric geometries (wedges, cones, and cylinders). These equations encompass the speed range from low supersonic to hypersonic flow and laminar and turbulent forebody boundary layers. A wide range of cone and wedge angles and cone bluntness ratios was included in the data base used to develop the correlations. The present investigation also included preliminary studies of the effect of angle of attack and specific-heat ratio of the gas.

Aerodynamic force and moment measurements and flow visualization results are presented for a hypersonic vehicle configuration at Mach 8. The basic vehicle configuration is a spherically blunted 10{degree} half-angle cone with a slice parallel with the axis of the vehicle. On the slice portion of the vehicle, a flap could be attached so that deflection angles of 10{degree}, 20{degree} and 30{degree} could be obtained. All of the experimental results were obtained in the Sandia Mach 8 hypersonic wind tunnel for laminar boundary layer conditions. Flow visualization results include shear stress sensitive liquid crystal photographs, surface streak flow photographs (using liquid crystals), and spark schlieren photographs and video. The liquid crystals were used as an aid in verifying that a laminar boundary layer existed over the entire body. The surface flow photo-graphs show attached and separated flow on both the leeside of the vehicle and near the flap. A detailed uncertainty analysis was conducted to estimate the contributors to body force and moment measurement uncertainty. Comparisons are made with computational results to evaluate both the experimental and numerical results. This extensive set of high-quality experimental force and moment measurements is recommended for use in the calibration and validation of relevant computational aerodynamics codes.