The Multi-Fidelity Toolkit (MFTK) is a simulation tool being developed at Sandia National Laboratories for aerodynamic predictions of compressible flows over a range of physics fidelities and computational speeds. These models include the Reynolds-Averaged-Navier-Stokes (RANS) equations, the Euler equations, and modified Newtonian aerodynamics (MNA) equations, and they can be invoked independently or coupled with hierarchical Kriging to interpolate between high-fidelity simulations using lower-fidelity data. However, as with any new simulation capability, verification and validation are necessary to gather credibility evidence. This work describes formal code- and solution-verification activities as well as model validation with uncertainty considerations. Code verification is performed on the MNA model by comparing with an analytical solution for flat-plate and inclined-plate geometries. Solution-verification activities include grid-refinement studies of HIFiRE-1 wind tunnel measurements, which are used for validation, for all model fidelities. A thorough treatment of the validation comparison with prediction error and validation uncertainty is also presented.
The Multi-Fidelity Toolkit (MFTK) is a simulation tool being developed at Sandia National Laboratories for aerodynamic predictions of compressible flows over a range of physics fidelities and computational speeds. These models include the Reynolds-Averaged Navier–Stokes (RANS) equations, the Euler equations, and modified Newtonian aerodynamics (MNA) equations, and they can be invoked independently or coupled with hierarchical Kriging to interpolate between high-fidelity simulations using lower-fidelity data. However, as with any new simulation capability, verification and validation are necessary to gather credibility evidence. This work describes formal model validation with uncertainty considerations that leverages experimental data from the HIFiRE-1 wind tunnel tests. The geometry is a multi-conic shape that produces complex flow phenomena under hypersonic conditions. A thorough treatment of the validation comparison with prediction error and validation uncertainty is also presented.
Though the method-of-moments implementation of the electric-field integral equation plays an important role in computational electromagnetics, it provides many code-verification challenges due to the different sources of numerical error and their possible interactions. Matters are further complicated by singular integrals, which arise from the presence of a Green's function. In this report, we document our research to address these issues, as well as its implementation and testing in Gemma.
The method of manufactured solutions (MMS) has become increasingly popular in conducting code verification studies on predictive codes, such as nuclear power system codes and computational fluid dynamic codes. The reason for the popularity of this approach is that it can be used when an analytical solution is not available. Using MMS, code developers are able to verify that their code is free of coding errors that impact the observed order of accuracy. While MMS is still an excellent tool for code verification, it does not identify coding errors that are of the same order as the numerical method. This paper presents a method that combines MMS with modified equation analysis (MEA), which calculates the local truncation error (LTE) to identify coding error up to and including the order of the numerical method. This method is referred to as modified equation analysis methd of manufactured solutions (MEAMMS). MEAMMS is then applied to a custom-built code, which solves the shallow water equations, to test the performance of the code verification method. MEAMMS is able to detect all coding errors that impact the implementation of the numerical scheme. To show how MEAMMS is different than MMS, they are both applied to the same first-order numerical method test problem with a first-order coding error. When there are first-order coding errors, only MEAMMS is able to identify them. Finally, this shows that MEAMMS is able to identify a larger set of coding errors while still being able to identify the coding errors MMS is able to identify.
Gemma verification activities for FY20 can be divided into three categories: the development of specialized quadrature rules, initial progress towards the development of manufactured solutions for code verification, and automated code-verification testing. In the method-of-moments implementation of the electric-field integral equation, the presence of a Green’s function in the four-dimensional integrals yields singularities in the integrand when two elements are nearby. To address these challenges, we have developed quadrature rules to integrate the functions through which the singularities can be characterized. Code verification is necessary to develop confidence in the implementation of the numerical methods in Gemma. Therefore, we have begun investigating the use of manufactured solutions to more thoroughly verify Gemma. Manufactured solutions provide greater flexibility for testing aspects of the code; however, the aforementioned singularities provide challenges, and existing work is limited in rigor and quantity. Finally, we have implemented automated code-verification testing using the VVTest framework to automate the mesh refinement and execution of a Gemma simulation to generate mesh convergence data. This infrastructure computes the observed order of accuracy from these data and compares it with the theoretical order of accuracy to either develop confidence in the implementation of the numerical methods or detect coding errors.
The Method of Manufactured Solutions (MMS) has proven to be useful for completing code verification studies. MMS allows the code developer to verify that the observed order-of-accuracy matches the theoretical order-of accuracy. Even though the solution to the partial differential equation is not intuitive, it provides an exact solution to a problem that most likely could not be solved analytically. The code developer can then use the exact solution as a debugging tool. While the order-of-accuracy test has been historically treated as the most rigorous of all code verification methods, it fails to indicate code”bugs” that are of the same order as the theoretical order-of-accuracy. The only way to test for these types of code bugs is to verify that the theoretical local truncation error for a particular grid matches the difference between the manufactured solution (MS) and the solution on that grid. The theoretical local truncation error can be computed by using the modified equation analysis (MEA) with the MS and its analytic derivatives, which we call modified equation analysis method of manufactured solutions (MEAMMS). In addition to describing the MEAMMS process, this study shows the results of completing a code verification study on a conservation of mass code. The code was able to compute the leading truncation error term as well as additional higher-order terms. When the code verification process was complete, not only did the observed order-of-accuracy match the theoretical order-of-accuracy for all numerical schemes implemented in the code, but it was also able to cancel the discretization error to within round-off error for a 64-bit system.