Publications

McGregor, Duncan A.; Love, Edward L.; Sirajuddin, David S.; Swan, Matthew S.; Collins, David I.; Pawlowski, Roger P.; Cartwright, Keith C.; Stafford, David S.

This is the user manual for EMPIRE, a simulation code for electromagnetics and plasma physics.

Crockatt, Michael M.; Shadid, John N.; Pawlowski, Roger P.; Conde, Sidafa C.; Mabuza, Sibu M.; Bonilla, Jesus B.

Abstract not provided.

Pawlowski, Roger P.

Abstract not provided.

Pawlowski, Roger P.

Abstract not provided.

Aaziz, Omar R.; Allan, Benjamin A.; Brandt, James M.; Cook, Jeanine C.; Devine, Karen D.; Elliott, James E.; Gentile, Ann C.; Hammond, Simon D.; Kelley, Brian M.; Lopatina, Lena L.; Moore, Stan G.; Olivier, Stephen L.; Pedretti, Kevin P.; Poliakoff, David Z.; Pawlowski, Roger P.; Regier, Phillip A.; Schmitz, Mark E.; Schwaller, Benjamin S.; Surjadidjaja, Vanessa S.; Swan, Matthew S.; Tucker, Nick T.; Tucker, Tom T.; Vaughan, Courtenay T.; Walton, Sara P.

Scientific applications run on high-performance computing (HPC) systems are critical for many national security missions within Sandia and the NNSA complex. However, these applications often face performance degradation and even failures that are challenging to diagnose. To provide unprecedented insight into these issues, the HPC Development, HPC Systems, Computational Science, and Plasma Theory & Simulation departments at Sandia crafted and completed their FY21 ASC Level 2 milestone entitled "Integrated System and Application Continuous Performance Monitoring and Analysis Capability." The milestone created a novel integrated HPC system and application monitoring and analysis capability by extending Sandia's Kokkos application portability framework, Lightweight Distributed Metric Service (LDMS) monitoring tool, and scalable storage, analysis, and visualization pipeline. The extensions to Kokkos and LDMS enable collection and storage of application data during run time, as it is generated, with negligible overhead. This data is combined with HPC system data within the extended analysis pipeline to present relevant visualizations of derived system and application metrics that can be viewed at run time or post run. This new capability was evaluated using several week-long, 290-node runs of Sandia's ElectroMagnetic Plasma In Realistic Environments ( EMPIRE ) modeling and design tool and resulted in 1TB of application data and 50TB of system data. EMPIRE developers remarked this capability was incredibly helpful for quickly assessing application health and performance alongside system state. In short, this milestone work built the foundation for expansive HPC system and application data collection, storage, analysis, visualization, and feedback framework that will increase total scientific output of Sandia's HPC users.

Tomas, Ignacio T.; Shadid, John N.; Crockatt, Michael M.; Pawlowski, Roger P.; Maier, Matthias M.; Guermond, Jean-Luc G.

This report summarizes the findings and outcomes of the LDRD-express project with title “Fluid models of charged species transport: numerical methods with mathematically guaranteed properties”. The primary motivation of this project was the computational/mathematical exploration of the ideas advanced aiming to improve the state-of-the-art on numerical methods for the one-fluid Euler-Poisson models and gain some understanding on the Euler-Maxwell model. Euler-Poisson and Euler-Maxwell, by themselves are not the most technically relevant PDE plasma-models. However, both of them are elementary building blocks of PDE-models used in actual technical applications and include most (if not all) of their mathematical difficulties. Outside the classical ideal MHD models, rigorous mathematical and numerical understanding of one-fluid models is still a quite undeveloped research area, and the treatment/understanding of boundary conditions is minimal (borderline non-existent) at this point in time. This report focuses primarily on bulk-behaviour of Euler-Poisson’s model, touching boundary conditions only tangentially.

Brandt, James M.; Cook, Jeanine C.; Aaziz, Omar R.; Allan, Benjamin A.; Devine, Karen D.; Elliott, James J.; Gentile, Ann C.; Hammond, Simon D.; Kelley, Brian M.; Lopatina, Lena L.; Moore, Stan G.; Olivier, Stephen L.; Pedretti, Kevin P.; Poliakoff, David Z.; Pawlowski, Roger P.; Regier, Phillip A.; Schmitz, Mark E.; Schwaller, Benjamin S.; Surjadidjaja, Vanessa S.; Swan, Matthew S.; Tucker, Tom T.; Tucker, Nick T.; Vaughan, Courtenay T.; Walton, Sara P.

Abstract not provided.

Crockatt, Michael M.; Shadid, John N.; Conde, Sidafa C.; Pawlowski, Roger P.; Mabuza, Sibu M.

Abstract not provided.

Communications in Computational Physics

Bettencourt, Matthew T.; Brown, Dominic A.S.; Cartwright, Keith L.; Cyr, Eric C.; Glusa, Christian A.; Lin, Paul T.; Moore, Stan G.; McGregor, Duncan A.O.; Pawlowski, Roger P.; Phillips, Edward G.; Roberts, Nathan V.; Wright, Steven A.; Maheswaran, Satheesh; Jones, John P.; Jarvis, Stephen A.

In this paper we introduce EMPIRE-PIC, a finite element method particle-in-cell (FEM-PIC) application developed at Sandia National Laboratories. The code has been developed in C++ using the Trilinos library and the Kokkos Performance Portability Framework to enable running on multiple modern compute architectures while only requiring maintenance of a single codebase. EMPIRE-PIC is capable of solving both electrostatic and electromagnetic problems in two- and three-dimensions to second-order accuracy in space and time. In this paper we validate the code against three benchmark problems - a simple electron orbit, an electrostatic Langmuir wave, and a transverse electromagnetic wave propagating through a plasma. We demonstrate the performance of EMPIRE-PIC on four different architectures: Intel Haswell CPUs, Intel's Xeon Phi Knights Landing, ARM Thunder-X2 CPUs, and NVIDIA Tesla V100 GPUs attached to IBM POWER9 processors. This analysis demonstrates scalability of the code up to more than two thousand GPUs, and greater than one hundred thousand CPUs.

Pawlowski, Roger P.; Phipps, Eric T.; Trott, Christian R.; Cyr, Eric C.; Shadid, John N.

Abstract not provided.

Crockatt, Michael M.; Shadid, John N.; Conde, Sidafa C.; Pawlowski, Roger P.; Mabuza, Sibu M.

Abstract not provided.

Phipps, Eric T.; Pawlowski, Roger P.; Trott, Christian R.

Abstract not provided.

Journal of Computational Physics

Mabuza, Sibusiso M.; Shadid, John N.; Cyr, Eric C.; Pawlowski, Roger P.; Kuzmin, Dmitri

In this work, a stabilized continuous Galerkin (CG) method for magnetohydrodynamics (MHD) is presented. Ideal, compressible inviscid MHD equations are discretized in space on unstructured meshes using piecewise linear or bilinear finite element bases to get a semi-discrete scheme. Stabilization is then introduced to the semi-discrete method in a strategy that follows the algebraic flux correction paradigm. This involves adding some artificial diffusion to the high order, semi-discrete method and mass lumping in the time derivative term. The result is a low order method that provides local extremum diminishing properties for hyperbolic systems. The difference between the low order method and the high order method is scaled element-wise using a limiter and added to the low order scheme. The limiter is solution dependent and computed via an iterative linearity preserving nodal variation limiting strategy. The stabilization also involves an optional consistent background high order dissipation that reduces phase errors. The resulting stabilized scheme is a semi-discrete method that can be applied to inviscid shock MHD problems and may be even extended to resistive and viscous MHD problems. To satisfy the divergence free constraint of the MHD equations, we add parabolic divergence cleaning to the system. Various time integration methods can be used to discretize the scheme in time. We demonstrate the robustness of the scheme by solving several shock MHD problems.

Miller, Sean M.; Cyr, Eric C.; Phillips, Edward G.; Pawlowski, Roger P.; Beckwith, Kristian B.; Bettencourt, Matthew T.; Cartwright, Keith C.; Shields, Sidney S.

Abstract not provided.

Pawlowski, Roger P.; Phipps, Eric T.

Abstract not provided.

Cyr, Eric C.; Shadid, John N.; Tuminaro, Raymond S.; Chacon, Luis C.; Pawlowski, Roger P.; Phillips, Edward G.; Miller, Sean M.

Abstract not provided.

Journal of Computational Physics

Miller, Sean M.; Cyr, E.C.; Shadid, John N.; Kramer, R.M.J.; Phillips, Edward G.; Conde, Sidafa C.; Pawlowski, Roger P.

Multi-fluid plasma models, where an electron fluid is modeled in addition to multiple ion and neutral species as well as the full set of Maxwell's equations, are useful for representing physics beyond the scope of classic MHD. This advantage presents challenges in appropriately dealing with electron dynamics and electromagnetic behavior characterized by the plasma and cyclotron frequencies and the speed of light. For physical systems, such as those near the MHD asymptotic regime, this requirement drastically increases runtimes for explicit time integration even though resolving fast dynamics may not be critical for accuracy. Implicit time integration methods, with efficient solvers, can help to step over fast time-scales that constrain stability, but do not strongly influence accuracy. As an extension, Implicit-explicit (IMEX) schemes provide an additional mechanism to choose which dynamics are evolved using an expensive implicit solve or resolved using a fast explicit solve. In this study, in addition to IMEX methods we also consider a physics compatible exact sequence spatial discretization. This combines nodal bases (H-Grad) for fluid dynamics with a set of vector bases (H-Curl and H-Div) for Maxwell's equations. This discretization allows for multi-fluid plasma modeling without violating Gauss' laws for the electric and magnetic fields. This initial study presents a discussion of the major elements of this formulation and focuses on demonstrating accuracy in the linear wave regime and in the MHD limit for both a visco-resistive and a dispersive ideal MHD problem.

Pawlowski, Roger P.

Abstract not provided.

Cyr, Eric C.; Miller, Sean M.; Pawlowski, Roger P.; Phillips, Edward G.; Beckwith, Kristian B.; Bettencourt, Matthew T.; Cartwright, Keith C.; Kramer, Richard M.; Roberts, Nathan V.; Shadid, John N.; Shields, Sidney S.

Abstract not provided.

Conde, Sidafa C.; Mabuza, Sibu M.; Shadid, John N.; Cyr, Eric C.; Smith, Thomas M.; Pawlowski, Roger P.

Abstract not provided.

Cyr, Eric C.; Miller, Sean M.; Phillips, Edward G.; Beckwith, Kristian B.; Bettencourt, Matthew T.; Cartwright, Keith C.; Kramer, Richard M.; Pawlowski, Roger P.; Roberts, Nathan V.; Shadid, John N.; Shields, Sidney S.

Abstract not provided.

Tuminaro, Raymond S.; Shadid, John N.; Cyr, Eric C.; Lin, Paul L.; Pawlowski, Roger P.; Phillips, Edward G.; Wiesner, Tobias A.; Adler, James A.; Benson, Tom B.; Chacon, Luis C.; Farrell, Patrick F.; Rappaport, Ari x.; MacLachlan, Scott M.; Hu, Jonathan J.; Berger-Vergiat, Luc B.; Glusa, Christian A.; Siefert, Christopher S.

Abstract not provided.

Lin, Paul L.; Shadid, John N.; Phillips, Edward G.; Pawlowski, Roger P.

Abstract not provided.

Pawlowski, Roger P.

Abstract not provided.

Lin, Paul L.; Shadid, John N.; Phillips, Edward G.; Pawlowski, Roger P.

Abstract not provided.

Mabuza, Sibusiso M.; Shadid, John N.; Conde, Sidafa C.; Cyr, Eric C.; Smith, Thomas M.; Pawlowski, Roger P.; Kuzmin, Dmitri K.

Abstract not provided.

Bettencourt, Matthew T.; Shields, Sidney S.; Beckwith, Kristian B.; Cartwright, Keith C.; Cyr, Eric C.; Kramer, Richard M.; Lin, Paul L.; McDoniel, William M.; Miller, Sean M.; Pawlowski, Roger P.; Phillips, Edward G.; Roberts, Nathan V.

Abstract not provided.

Pawlowski, Roger P.

Abstract not provided.

Pawlowski, Roger P.

Abstract not provided.

Phipps, Eric T.; Pawlowski, Roger P.; Bettencourt, Matthew T.; Cyr, Eric C.; Roberts, Nathan V.

Abstract not provided.

Journal of Computational and Applied Mathematics

Lin, Paul L.; Shadid, John N.; Hu, J.J.; Pawlowski, Roger P.; Cyr, E.C.

This work explores the current performance and scaling of a fully-implicit stabilized unstructured finite element (FE) variational multiscale (VMS) capability for large-scale simulations of 3D incompressible resistive magnetohydrodynamics (MHD). The large-scale linear systems that are generated by a Newton nonlinear solver approach are iteratively solved by preconditioned Krylov subspace methods. The efficiency of this approach is critically dependent on the scalability and performance of the algebraic multigrid preconditioner. This study considers the performance of the numerical methods as recently implemented in the second-generation Trilinos implementation that is 64-bit compliant and is not limited by the 32-bit global identifiers of the original Epetra-based Trilinos. The study presents representative results for a Poisson problem on 1.6 million cores of an IBM Blue Gene/Q platform to demonstrate very large-scale parallel execution. Additionally, results for a more challenging steady-state MHD generator and a transient solution of a benchmark MHD turbulence calculation for the full resistive MHD system are also presented. These results are obtained on up to 131,000 cores of a Cray XC40 and one million cores of a BG/Q system.

Huang, Andy H.; Gao, Xujiao G.; Pawlowski, Roger P.; Gates, Jason M.; Musson, Lawrence M.; Hennigan, Gary L.; Negoita, Mihai N.

Abstract not provided.

Ober, Curtis C.; Pawlowski, Roger P.; Conde, Sidafa C.; Kalashnikova, Irina; Phipps, Eric T.; Phillips, Edward G.; Mota, Alejandro M.; Phlipot, Greg [.; Ridzal, Denis R.; Hansen, Michael A.; Fisher, Travis C.

Abstract not provided.

Cyr, Eric C.; Beckwith, Kristian B.; Bettencourt, Matthew T.; Cartwright, Keith C.; Kramer, Richard M.; Miller, Sean T.; Pawlowski, Roger P.; Phillips, Edward G.; Roberts, Nathan V.; Shadid, John N.

Abstract not provided.

Huang, Andy H.; Gao, Xujiao G.; Pawlowski, Roger P.; Gates, Jason M.; Musson, Lawrence M.; Hennigan, Gary L.; Negoita, Mihai N.

Abstract not provided.

Bettencourt, Matthew T.; Kramer, Richard M.; Cartwright, Keith C.; Phillips, Edward G.; Ober, Curtis C.; Pawlowski, Roger P.; Swan, Matthew S.; Kalashnikova, Irina; Phipps, Eric T.; Conde, Sidafa C.; Cyr, Eric C.; Ulmer, Craig D.; Kordenbrock, Todd H.; Levy, Scott L.; Templet, Gary J.; Hu, Jonathan J.; Lin, Paul L.; Glusa, Christian A.; Siefert, Christopher S.; Glass, Micheal W.

This report documents the outcome from the ASC ATDM Level 2 Milestone 6358: Assess Status of Next Generation Components and Physics Models in EMPIRE. This Milestone is an assessment of the EMPIRE (ElectroMagnetic Plasma In Realistic Environments) application and three software components. The assessment focuses on the electromagnetic and electrostatic particle-in-cell solu- tions for EMPIRE and its associated solver, time integration, and checkpoint-restart components. This information provides a clear understanding of the current status of the EMPIRE application and will help to guide future work in FY19 in order to ready the application for the ASC ATDM L 1 Milestone in FY20. It is clear from this assessment that performance of the linear solver will have to be a focus in FY19.

Pawlowski, Roger P.; Phillips, Edward G.; Shadid, John N.; Lin, Paul L.; Cyr, Eric C.; Conde, Sidafa C.; Bettencourt, Matthew T.; Phipps, Eric T.; Trott, Christian R.

Abstract not provided.

Pawlowski, Roger P.; Phillips, Edward G.; Shadid, John N.; Cyr, Eric C.; Conde, Sidafa C.; Bettencourt, Matthew T.; Phipps, Eric T.; Bond, Stephen D.; Trott, Christian R.

Abstract not provided.

Huang, Andy H.; Gao, Xujiao G.; Pawlowski, Roger P.; Gates, Jason M.; Musson, Lawrence M.; Hennigan, Gary L.; Negoita, Mihai N.

Abstract not provided.

Lin, Paul L.; Shadid, John N.; Phillips, Edward G.; Pawlowski, Roger P.; Cyr, Eric C.

Abstract not provided.

Lin, Paul L.; Shadid, John N.; Phillips, Edward G.; Pawlowski, Roger P.

Abstract not provided.

Toth, Alexander R.; Pawlowski, Roger P.; Bartlett, Roscoe B.

Abstract not provided.

Pawlowski, Roger P.; Turner, Daniel Z.

Abstract not provided.

Lin, Paul L.; Shadid, John N.; Phillips, Edward G.; Pawlowski, Roger P.; Cyr, Eric C.

Abstract not provided.

Pawlowski, Roger P.; Bettencourt, Matthew T.; Cyr, Eric C.; Miller, Sean M.; Phillips, Edward G.; Phipps, Eric T.; Shadid, John N.; Trott, Christian R.

Abstract not provided.

Ober, Curtis C.; Pawlowski, Roger P.; Kalashnikova, Irina; Conde, Sidafa C.; Cyr, Eric C.

Abstract not provided.

Pawlowski, Roger P.

Abstract not provided.

Bettencourt, Matthew T.; Cyr, Eric C.; Kramer, Richard M.; Miller, Sean M.; Pawlowski, Roger P.; Phillips, Edward G.; Robinson, Allen C.; Shadid, John N.

Abstract not provided.

Bettencourt, Matthew T.; Cyr, Eric C.; Kramer, Richard M.; Miller, Sean M.; Pawlowski, Roger P.; Phillips, Edward G.; Robinson, Allen C.; Shadid, John N.

Abstract not provided.

Lin, Paul L.; Shadid, John N.; Phillips, Edward G.; Hu, Jonathan J.; Cyr, Eric C.; Pawlowski, Roger P.

Abstract not provided.

Mayr, Matthias M.; Cyr, Eric C.; Shadid, John N.; Pawlowski, Roger P.; Wildey, Timothy M.; Scovazzi, Guglielmo S.; Zeng, Xianyi Z.; Phillips, Edward G.; Conde, Sidafa C.

Abstract not provided.

Mayr, Matthias M.; Cyr, Eric C.; Shadid, John N.; Pawlowski, Roger P.; Wildey, Timothy M.; Scovazzi, Guglielmo S.; Zeng, Xianyi Z.; Phillips, Edward G.; Conde, Sidafa C.

Abstract not provided.

Lin, Paul L.; Shadid, John N.; Phillips, Edward G.; Hu, Jonathan J.; Cyr, Eric C.; Pawlowski, Roger P.; Tsuji, Paul T.; Sondak, David S.

Abstract not provided.

Lin, Paul L.; Shadid, John N.; Pawlowski, Roger P.; Bettencourt, Matthew T.; Cyr, Eric C.

Abstract not provided.

Pawlowski, Roger P.

Abstract not provided.

Ober, Curtis C.; Bartlett, Roscoe B.; Coffey, Todd S.; Pawlowski, Roger P.

Time integration is a central component for most transient simulations. It coordinates many of the major parts of a simulation together, e.g., a residual calculation with a transient solver, solution with the output, various operator-split physics, and forward and adjoint solutions for inversion. Even though there is this variety in these transient simulations, there is still a common set of algorithms and proce- dures to progress transient solutions for ordinary-differential equations (ODEs) and differential-alegbraic equations (DAEs). Rythmos is a collection of these algorithms that can be used for the solution of transient simulations. It provides common time-integration methods, such as Backward and Forward Euler, Explicit and Im- plicit Runge-Kutta, and Backward-Difference Formulas. It can also provide sensitivities, and adjoint components for transient simulations. Rythmos is a package within Trilinos, and requires some other packages (e.g., Teuchos and Thrya) to provide basic time-integration capabilities. It also can be cou- pled with several other Trilinos packages to provide additional capabilities (e.g., AztecOO and Belos for linear solutions, and NOX for non-linear solutions). The documentation is broken down into three parts: Theory Manual, User's Manual, and Developer's Guide. The Theory Manual contains the basic theory of the time integrators, the nomenclature and mathematical structure utilized within Rythmos, and verification results demonstrating that the designed order of accuracy is achieved. The User's Manual provides information on how to use the Rythmos, description of input parameters through Teuchos Parameter Lists, and description of convergence test examples. The Developer's Guide is a high-level discussion of the design and structure of Rythmos to provide information to developers for the continued development of capabilities. Details of individual components can be found in the Doxygen webpages. Notice: This document is a collection of notes gathered over many years, however its development has been dormant for the past several years due to the lack of funding. Pre-release copies of this document have circulated to internal Sandia developers, who have found it useful in their application development. Also external Sandia developers have made inquiries for additional Rythmos documentation. To make this documentation publicly available, we are releasing this documentation in an " as-is " condition. We apologize for its incomplete state and obvious lack of readability, however we hope that the information contained in this document will still be helpful to users in their application development.

Lin, Paul L.; Shadid, John N.; Phillips, Edward G.; Hu, Jonathan J.; Cyr, Eric C.; Pawlowski, Roger P.; Prokopenko, Andrey P.; Tsuji, Paul T.

Abstract not provided.

Gates, Jason M.; Pawlowski, Roger P.; Cyr, Eric C.

Abstract not provided.

Pawlowski, Roger P.; Bartlett, Roscoe B.; Bettencourt, Matthew T.; Carleton, James B.; Conde, Sidafa C.; Cyr, Eric C.; Kim, Kyungjoo K.; Mota, Alejandro M.; Perego, Mauro P.; Shadid, John N.; Sjaardema, Gregory D.; Toth, Alexander R.; Bradley, Andrew M.; Spotz, William S.; Ober, Curtis C.; Kalashnikova, Irina

Abstract not provided.

Ober, Curtis C.; Pawlowski, Roger P.; Cyr, Eric C.

Abstract not provided.

Pawlowski, Roger P.

Abstract not provided.

Cyr, Eric C.; Shadid, John N.; Wildey, Timothy M.; Phillips, Edward G.; Robinson, Allen C.; Miller, Sean M.; Pawlowski, Roger P.

Abstract not provided.

Shadid, John N.; Phillips, Edward G.; Cyr, Eric C.; Pawlowski, Roger P.; Bettencourt, Matthew T.; Cartwright, Keith C.; Lin, Paul L.; Kramer, Richard M.; Niederhaus, John H.; Radtke, Gregg A.; Robinson, Allen C.

Abstract not provided.

Journal of Computational Physics

Shadid, John N.; Smith, Thomas M.; Cyr, E.C.; Wildey, T.M.; Pawlowski, Roger P.

A critical aspect of applying modern computational solution methods to complex multiphysics systems of relevance to nuclear reactor modeling, is the assessment of the predictive capability of specific proposed mathematical models. In this respect the understanding of numerical error, the sensitivity of the solution to parameters associated with input data, boundary condition uncertainty, and mathematical models is critical. Additionally, the ability to evaluate and or approximate the model efficiently, to allow development of a reasonable level of statistical diagnostics of the mathematical model and the physical system, is of central importance. In this study we report on initial efforts to apply integrated adjoint-based computational analysis and automatic differentiation tools to begin to address these issues. The study is carried out in the context of a Reynolds averaged Navier-Stokes approximation to turbulent fluid flow and heat transfer using a particular spatial discretization based on implicit fully-coupled stabilized FE methods. Initial results are presented that show the promise of these computational techniques in the context of nuclear reactor relevant prototype thermal-hydraulics problems.

Lin, Paul L.; Shadid, John N.; Hu, Jonathan J.; Pawlowski, Roger P.; Cyr, Eric C.; Prokopenko, Andrey V.

Abstract not provided.

Bettencourt, Matthew T.; Cyr, Eric C.; Pawlowski, Roger P.; Kramer, Richard M.; Phillips, Edward G.; Shadid, John N.; Robinson, Allen C.

Abstract not provided.

Computer Methods in Applied Mechanics and Engineering

Shadid, John N.; Pawlowski, Roger P.; Cyr, E.C.; Tuminaro, Raymond S.; Chacón, L.; Weber, Paula D.

The computational solution of the governing balance equations for mass, momentum, heat transfer and magnetic induction for resistive magnetohydrodynamics (MHD) systems can be extremely challenging. These difficulties arise from both the strong nonlinear, nonsymmetric coupling of fluid and electromagnetic phenomena, as well as the significant range of time- and length-scales that the interactions of these physical mechanisms produce. This paper explores the development of a scalable, fully-implicit stabilized unstructured finite element (FE) capability for 3D incompressible resistive MHD. The discussion considers the development of a stabilized FE formulation in context of the variational multiscale (VMS) method, and describes the scalable implicit time integration and direct-to-steady-state solution capability. The nonlinear solver strategy employs Newton-Krylov methods, which are preconditioned using fully-coupled algebraic multilevel preconditioners. These preconditioners are shown to enable a robust, scalable and efficient solution approach for the large-scale sparse linear systems generated by the Newton linearization. Verification results demonstrate the expected order-of-accuracy for the stabilized FE discretization. The approach is tested on a variety of prototype problems, that include MHD duct flows, an unstable hydromagnetic Kelvin-Helmholtz shear layer, and a 3D island coalescence problem used to model magnetic reconnection. Initial results that explore the scaling of the solution methods are also presented on up to 128K processors for problems with up to 1.8B unknowns on a CrayXK7.

Bettencourt, Matthew T.; Cyr, Eric C.; Pawlowski, Roger P.; Kramer, Richard M.; Phillips, Edward G.; Shadid, John N.; Robinson, Allen C.

Abstract not provided.

Journal of Computational Physics

Hamilton, Steven; Berrill, Mark; Clarno, Kevin; Pawlowski, Roger P.; Toth, Alex; Kelley, C.T.; Evans, Thomas; Philip, Bobby

This paper evaluates the performance of multiphysics coupling algorithms applied to a light water nuclear reactor core simulation. The simulation couples the k-eigenvalue form of the neutron transport equation with heat conduction and subchannel flow equations. We compare Picard iteration (block Gauss-Seidel) to Anderson acceleration and multiple variants of preconditioned Jacobian-free Newton-Krylov (JFNK). The performance of the methods are evaluated over a range of energy group structures and core power levels. A novel physics-based approximation to a Jacobian-vector product has been developed to mitigate the impact of expensive on-line cross section processing steps. Numerical simulations demonstrating the efficiency of JFNK and Anderson acceleration relative to standard Picard iteration are performed on a 3D model of a nuclear fuel assembly. Both criticality (k-eigenvalue) and critical boron search problems are considered.

Cyr, Eric C.; Shadid, John N.; Phillips, Edward G.; Pawlowski, Roger P.; Seefeldt, Ben S.

Abstract not provided.

Journal of Computational Physics

Sondak, D.; Shadid, John N.; Oberai, A.A.; Pawlowski, Roger P.; Cyr, E.C.; Smith, Thomas M.

New large eddy simulation (LES) turbulence models for incompressible magnetohydrodynamics (MHD) derived from the variational multiscale (VMS) formulation for finite element simulations are introduced. The new models include the variational multiscale formulation, a residual-based eddy viscosity model, and a mixed model that combines both of these component models. Each model contains terms that are proportional to the residual of the incompressible MHD equations and is therefore numerically consistent. Moreover, each model is also dynamic, in that its effect vanishes when this residual is small. The new models are tested on the decaying MHD Taylor Green vortex at low and high Reynolds numbers. The evaluation of the models is based on comparisons with available data from direct numerical simulations (DNS) of the time evolution of energies as well as energy spectra at various discrete times. A numerical study, on a sequence of meshes, is presented that demonstrates that the large eddy simulation approaches the DNS solution for these quantities with spatial mesh refinement.

Phillips, Edward G.; Shadid, John N.; Cyr, Eric C.; Pawlowski, Roger P.

Abstract not provided.

Lin, Paul L.; Shadid, John N.; Hu, Jonathan J.; Cyr, Eric C.; Pawlowski, Roger P.; Prokopenko, Andrey V.

Abstract not provided.

Lin, Paul L.; Shadid, John N.; Hu, Jonathan J.; Cyr, Eric C.; Pawlowski, Roger P.; Prokopenko, Andrey V.

Abstract not provided.

Pawlowski, Roger P.; Salinger, Andrew G.

Abstract not provided.

Lin, Paul L.; Shadid, John N.; Hu, Jonathan J.; Cyr, Eric C.; Pawlowski, Roger P.; Prokopenko, Andrey V.

Abstract not provided.

Demeshko, Irina D.; Edwards, Harold C.; Heroux, Michael A.; Salinger, Andrew G.; Pawlowski, Roger P.; Phipps, Eric T.

Abstract not provided.

Cyr, Eric C.; Bettencourt, Matthew T.; Demeshko, Irina D.; Pawlowski, Roger P.

Abstract not provided.

Mathematics and Computations, Supercomputing in Nuclear Applications and Monte Carlo International Conference, M and C+SNA+MC 2015

Pawlowski, Roger P.; Clarno, K.; Montgomery, R.; Salko, R.; Evans, T.; Turner, J.; Gaston, D.

The Tiamat code is being developed by CASL (Consortium for Advanced Simulation of LWRs) as an integrated tool for predicting pellet-clad interaction and improving the high-fidelity core simulator. Tiamat is a large-scale parallel code that couples the the multi-dimensional Bison-CASL fuel performance code on every fuel rod with the COBRA-TF (CTF) sub-channel thermal-hydraulics code and either the Insilico or MPACT neutronics codes. Tiamat solves a transient problem where each time step is subcycled using Picard iteration to converge the fully-coupled nonlinear system. This report discusses the solution algorithms and software design of the simulator. Results are shown for a five assembly cross and compared against a separate core simulator developed by CASL. Tiamat has demonstrated that it can compute quantities of interest for analyzing pellet-clad interaction.

Cyr, Eric C.; Shadid, John N.; Phillips, Edward G.; Tuminaro, Raymond S.; Pawlowski, Roger P.; Lin, Paul L.; Chacon, Luis C.

Abstract not provided.

Pawlowski, Roger P.

Abstract not provided.

Demeshko, Irina D.; Edwards, Harold C.; Heroux, Michael A.; Phipps, Eric T.; Salinger, Andrew G.; Pawlowski, Roger P.

Abstract not provided.

Demeshko, Irina D.; Edwards, Harold C.; Heroux, Michael A.; Pawlowski, Roger P.; Phipps, Eric T.; Salinger, Andrew G.; Trott, Christian R.

Abstract not provided.

Pawlowski, Roger P.

Abstract not provided.

Pawlowski, Roger P.

Abstract not provided.

Keiter, Eric R.; Warrender, Christina E.; Mei, Ting M.; Russo, Thomas V.; Schiek, Richard S.; Thornquist, Heidi K.; Verley, Jason V.; Coffey, Todd S.; Pawlowski, Roger P.

This manual describes the use of the Xyce Parallel Electronic Simulator. Xyce has been designed as a SPICE-compatible, high-performance analog circuit simulator, and has been written to support the simulation needs of the Sandia National Laboratories electrical designers. This development has focused on improving capability over the current state-of-the-art in the following areas: Capability to solve extremely large circuit problems by supporting large-scale parallel computing platforms (up to thousands of processors). This includes support for most popular parallel and serial computers. A differential-algebraic-equation (DAE) formulation, which better isolates the device model package from solver algorithms. This allows one to develop new types of analysis without requiring the implementation of analysis-specific device models. Device models that are specifically tailored to meet Sandias needs, including some radiationaware devices (for Sandia users only). Object-oriented code design and implementation using modern coding practices. Xyce is a parallel code in the most general sense of the phrase a message passing parallel implementation which allows it to run efficiently a wide range of computing platforms. These include serial, shared-memory and distributed-memory parallel platforms. Attention has been paid to the specific nature of circuit-simulation problems to ensure that optimal parallel efficiency is achieved as the number of processors grows.

Keiter, Eric R.; Mei, Ting M.; Russo, Thomas V.; Pawlowski, Roger P.; Schiek, Richard S.; Coffey, Todd S.; Thornquist, Heidi K.; Verley, Jason V.; Warrender, Christina E.

This document is a reference guide to the Xyce Parallel Electronic Simulator, and is a companion document to the Xyce Users Guide [1] . The focus of this document is (to the extent possible) exhaustively list device parameters, solver options, parser options, and other usage details of Xyce. This document is not intended to be a tutorial. Users who are new to circuit simulation are better served by the Xyce Users Guide [1] .

SIAM Journal on Scientific Computing

Phillips, Edward G.; Elman, Howard C.; Cyr, Eric C.; Shadid, John N.; Pawlowski, Roger P.

The magnetohydrodynamics (MHD) equations are used to model the flow of electrically conducting fluids in such applications as liquid metals and plasmas. This system of nonself-adjoint, nonlinear PDEs couples the Navier-Stokes equations for fluids and Maxwell's equations for electromagnetics. There has been recent interest in fully coupled solvers for the MHD system because they allow for fast steady-state solutions that do not require pseudo-time-stepping. When the fully coupled system is discretized, the strong coupling can make the resulting algebraic systems difficult to solve, requiring effective preconditioning of iterative methods for efficiency. In this work, we consider a finite element discretization of an exact penalty formulation for the stationary MHD equations posed in two-dimensional domains. This formulation has the benefit of implicitly enforcing the divergence-free condition on the magnetic field without requiring a Lagrange multiplier. We consider extending block preconditioning techniques developed for the Navier-Stokes equations to the full MHD system. We analyze operators arising in block decompositions from a continuous perspective and apply arguments based on the existence of approximate commutators to develop new preconditioners that account for the physical coupling. This results in a family of parameterized block preconditioners for both Picard and Newton linearizations. We develop an automated method for choosing the relevant parameters and demonstrate the robustness of these preconditioners for a range of the physical nondimensional parameters and with respect to mesh refinement.

Shadid, John N.; Pawlowski, Roger P.; Cyr, Eric C.; Wildey, Timothy M.

This report describes work directed towards completion of the Thermal Hydraulics Methods (THM) CFD Level 3 Milestone THM.CFD.P7.05 for the Consortium for Advanced Simulation of Light Water Reactors (CASL) Nuclear Hub effort. The focus of this milestone was to demonstrate the thermal hydraulics and adjoint based error estimation and parameter sensitivity capabilities in the CFD code called Drekar::CFD. This milestone builds upon the capabilities demonstrated in three earlier milestones; THM.CFD.P4.02 [12], completed March, 31, 2012, THM.CFD.P5.01 [15] completed June 30, 2012 and THM.CFD.P5.01 [11] completed on October 31, 2012.

ACM Transaction on Mathematical Software

Salinger, Andrew G.; Ostien, Jakob O.; Pawlowski, Roger P.; Phipps, Eric T.; Sun, WaiChing S.; Gao, Xujiao G.; Hansen, Glen H.; Kalashnikova, Irina; Mota, Alejandro M.; Muller, Richard P.; Nielsen, Erik N.

Abstract not provided.

Shadid, John N.; Pawlowski, Roger P.; Cyr, Eric C.; Wildey, Timothy M.; Weber, Paula D.

Abstract not provided.

Russo, Thomas V.; Mei, Ting M.; Keiter, Eric R.; Schiek, Richard S.; Thornquist, Heidi K.; Verley, Jason V.; Coffey, Todd S.; Pawlowski, Roger P.; Warrender, Christina E.

This manual describes the use of the Xyce Parallel Electronic Simulator. Xyce has been designed as a SPICE-compatible, high-performance analog circuit simulator, and has been written to support the simulation needs of the Sandia National Laboratories electrical designers. This development has focused on improving capability over the current state-of-the-art in the following areas: Capability to solve extremely large circuit problems by supporting large-scale parallel computing platforms (up to thousands of processors). This includes support for most popular parallel and serial computers. A differential-algebraic-equation (DAE) formulation, which better isolates the device model package from solver algorithms. This allows one to develop new types of analysis without requiring the implementation of analysis-specific device models. Device models that are specifically tailored to meet Sandias needs, including some radiationaware devices (for Sandia users only). Object-oriented code design and implementation using modern coding practices. Xyce is a parallel code in the most general sense of the phrase a message passing parallel implementation which allows it to run efficiently a wide range of computing platforms. These include serial, shared-memory and distributed-memory parallel platforms. Attention has been paid to the specific nature of circuit-simulation problems to ensure that optimal parallel efficiency is achieved as the number of processors grows.

Keiter, Eric R.; Mei, Ting M.; Russo, Thomas V.; Pawlowski, Roger P.; Schiek, Richard S.; Coffey, Todd S.; Thornquist, Heidi K.; Verley, Jason V.; Warrender, Christina E.

This document is a reference guide to the Xyce Parallel Electronic Simulator, and is a companion document to the Xyce Users Guide [1] . The focus of this document is (to the extent possible) exhaustively list device parameters, solver options, parser options, and other usage details of Xyce. This document is not intended to be a tutorial. Users who are new to circuit simulation are better served by the Xyce Users Guide [1] .

Pawlowski, Roger P.

Abstract not provided.

Pawlowski, Roger P.

Abstract not provided.

Shadid, John N.; Tuminaro, Raymond S.; Pawlowski, Roger P.

Abstract not provided.

Shadid, John N.; Wildey, Timothy M.; Pawlowski, Roger P.

Abstract not provided.

Pawlowski, Roger P.; Cyr, Eric C.; Shadid, John N.

Abstract not provided.

Pawlowski, Roger P.; Cyr, Eric C.; Shadid, John N.; Tuminaro, Raymond S.; Lin, Paul L.

Abstract not provided.

Cyr, Eric C.; Shadid, John N.; Pawlowski, Roger P.

This document summarizes the results from a level 3 milestone study within the CASL VUQ effort. We compare the adjoint-based a posteriori error estimation approach with a recent variant of a data-centric verification technique. We provide a brief overview of each technique and then we discuss their relative advantages and disadvantages. We use Drekar::CFD to produce numerical results for steady-state Navier Stokes and SARANS approximations. 3

Shadid, John N.; Pawlowski, Roger P.

Abstract not provided.

Pawlowski, Roger P.

Abstract not provided.

Rao, Rekha R.; O'Hern, Timothy J.; Domino, Stefan P.; Wyatt, Nicholas B.; Tikare, Veena T.; Erickson, Lindsay C.; Shelden, Bion S.; Brotherton, Christopher M.; Cipiti, Benjamin B.; Pawlowski, Roger P.; Noble, David R.; Jove Colon, Carlos F.; Moffat, Harry K.; Roberts, Christine C.; Nemer, Martin N.

Abstract not provided.

Ridzal, Denis R.; Shadid, John N.; Pawlowski, Roger P.

Abstract not provided.

Shadid, John N.; Tuminaro, Raymond S.; Pawlowski, Roger P.; Lin, Paul L.

Abstract not provided.

Gao, Xujiao G.; Bochev, Pavel B.; Peterson, Kara J.; Pawlowski, Roger P.; Hennigan, Gary L.; Musson, Lawrence M.

Abstract not provided.

Lecture Notes in Computational Science and Engineering

Phipps, Eric T.; Pawlowski, Roger P.

Expression templates are a well-known set of techniques for improving the efficiency of operator overloading-based forward mode automatic differentiation schemes in the C++ programming language by translating the differentiation from individual operators to whole expressions. However standard expression template approaches result in a large amount of duplicate computation, particularly for large expression trees, degrading their performance. In this paper we describe several techniques for improving the efficiency of expression templates and their implementation in the automatic differentiation package Sacado (Phipps et al., Advances in automatic differentiation, Lecture notes in computational science and engineering, Springer, Berlin, 2008; Phipps and Gay, Sacado automatic differentiation package. http://trilinos.sandia.gov/packages/sacado/, 2011). We demonstrate their improved efficiency through test functions as well as their application to differentiation of a large-scale fluid dynamics simulation code. © 2012 Springer-Verlag.

Phipps, Eric T.; Pawlowski, Roger P.; Salinger, Andrew G.

Abstract not provided.

Proposed for publication in SIAM Journal on Scientific Computing.

Shadid, John N.; Tuminaro, Raymond S.; Pawlowski, Roger P.

Abstract not provided.

Phipps, Eric T.; Pawlowski, Roger P.

Abstract not provided.

Keiter, Eric R.; Warrender, Christina E.; Verley, Jason V.; Mei, Ting M.; Russo, Thomas V.; Pawlowski, Roger P.; Schiek, Richard S.; Santarelli, Keith R.; Coffey, Todd S.; Thornquist, Heidi K.

Abstract not provided.

Keiter, Eric R.; Warrender, Christina E.; Verley, Jason V.; Mei, Ting M.; Russo, Thomas V.; Pawlowski, Roger P.; Schiek, Richard S.; Santarelli, Keith R.; Coffey, Todd S.; Thornquist, Heidi K.

Abstract not provided.

Pawlowski, Roger P.

Abstract not provided.

Pawlowski, Roger P.

Abstract not provided.

Scientific Programming

Pawlowski, Roger P.; Phipps, Eric T.; Salinger, Andrew G.

An approach for incorporating embedded simulation and analysis capabilities in complex simulation codes through template-based generic programming is presented. This approach relies on templating and operator overloading within the C++ language to transform a given calculation into one that can compute a variety of additional quantities that are necessary for many state-of-the-art simulation and analysis algorithms. An approach for incorporating these ideas into complex simulation codes through general graph-based assembly is also presented. These ideas have been implemented within a set of packages in the Trilinos framework and are demonstrated on a simple problem from chemical engineering. © 2012 - IOS Press and the authors. All rights reserved.

Pawlowski, Roger P.; Shadid, John N.; Wildey, Timothy M.

Abstract not provided.

Pawlowski, Roger P.; Summers, Randall M.

Abstract not provided.

Pawlowski, Roger P.; Belcourt, Kenneth N.; Hooper, Russell H.; Schmidt, Rodney C.

Abstract not provided.

Pawlowski, Roger P.; Cyr, Eric C.; Shadid, John N.

Abstract not provided.

Phipps, Eric T.; Pawlowski, Roger P.; Salinger, Andrew G.

Abstract not provided.

Schmidt, Rodney C.; Belcourt, Kenneth N.; Hooper, Russell H.; Pawlowski, Roger P.

LIME is a small software package for creating multiphysics simulation codes. The name was formed as an acronym denoting 'Lightweight Integrating Multiphysics Environment for coupling codes.' LIME is intended to be especially useful when separate computer codes (which may be written in any standard computer language) already exist to solve different parts of a multiphysics problem. LIME provides the key high-level software (written in C++), a well defined approach (with example templates), and interface requirements to enable the assembly of multiple physics codes into a single coupled-multiphysics simulation code. In this report we introduce important software design characteristics of LIME, describe key components of a typical multiphysics application that might be created using LIME, and provide basic examples of its use - including the customized software that must be written by a user. We also describe the types of modifications that may be needed to individual physics codes in order for them to be incorporated into a LIME-based multiphysics application.

Pawlowski, Roger P.; Phipps, Eric T.; Salinger, Andrew G.

Abstract not provided.

Pawlowski, Roger P.; Cyr, Eric C.; Shadid, John N.

Abstract not provided.

Scientific Programming

Pawlowski, Roger P.; Phipps, Eric T.; Owen, Steven J.; Siefert, Christopher S.; Staten, Matthew L.

Abstract not provided.

Shadid, John N.; Pawlowski, Roger P.; Cyr, Eric C.; Weber, Paula D.

Abstract not provided.

Shadid, John N.; Pawlowski, Roger P.; Cyr, Eric C.

Abstract not provided.

Keiter, Eric R.; Warrender, Christina E.; Mei, Ting M.; Russo, Thomas V.; Pawlowski, Roger P.; Schiek, Richard S.; Santarelli, Keith R.; Coffey, Todd S.; Thornquist, Heidi K.

This document is a reference guide to the Xyce Parallel Electronic Simulator, and is a companion document to the Xyce Users Guide. The focus of this document is (to the extent possible) exhaustively list device parameters, solver options, parser options, and other usage details of Xyce. This document is not intended to be a tutorial. Users who are new to circuit simulation are better served by the Xyce Users Guide. The Xyce Parallel Electronic Simulator has been written to support, in a rigorous manner, the simulation needs of the Sandia National Laboratories electrical designers. It is targeted specifically to run on large-scale parallel computing platforms but also runs well on a variety of architectures including single processor workstations. It also aims to support a variety of devices and models specific to Sandia needs. This document is intended to complement the Xyce Users Guide. It contains comprehensive, detailed information about a number of topics pertinent to the usage of Xyce. Included in this document is a netlist reference for the input-file commands and elements supported within Xyce; a command line reference, which describes the available command line arguments for Xyce; and quick-references for users of other circuit codes, such as Orcad's PSpice and Sandia's ChileSPICE.

Keiter, Eric R.; Warrender, Christina E.; Mei, Ting M.; Russo, Thomas V.; Pawlowski, Roger P.; Schiek, Richard S.; Santarelli, Keith R.; Coffey, Todd S.; Thornquist, Heidi K.

This manual describes the use of the Xyce Parallel Electronic Simulator. Xyce has been designed as a SPICE-compatible, high-performance analog circuit simulator, and has been written to support the simulation needs of the Sandia National Laboratories electrical designers. This development has focused on improving capability over the current state-of-the-art in the following areas: (1) Capability to solve extremely large circuit problems by supporting large-scale parallel computing platforms (up to thousands of processors). Note that this includes support for most popular parallel and serial computers; (2) Improved performance for all numerical kernels (e.g., time integrator, nonlinear and linear solvers) through state-of-the-art algorithms and novel techniques. (3) Device models which are specifically tailored to meet Sandia's needs, including some radiation-aware devices (for Sandia users only); and (4) Object-oriented code design and implementation using modern coding practices that ensure that the Xyce Parallel Electronic Simulator will be maintainable and extensible far into the future. Xyce is a parallel code in the most general sense of the phrase - a message passing parallel implementation - which allows it to run efficiently on the widest possible number of computing platforms. These include serial, shared-memory and distributed-memory parallel as well as heterogeneous platforms. Careful attention has been paid to the specific nature of circuit-simulation problems to ensure that optimal parallel efficiency is achieved as the number of processors grows. The development of Xyce provides a platform for computational research and development aimed specifically at the needs of the Laboratory. With Xyce, Sandia has an 'in-house' capability with which both new electrical (e.g., device model development) and algorithmic (e.g., faster time-integration methods, parallel solver algorithms) research and development can be performed. As a result, Xyce is a unique electrical simulation capability, designed to meet the unique needs of the laboratory.

Shadid, John N.; Pawlowski, Roger P.; Cyr, Eric C.

Abstract not provided.

Pawlowski, Roger P.; Belcourt, Kenneth N.; Schmidt, Rodney C.

Abstract not provided.

Pawlowski, Roger P.; Red-Horse, John R.

Abstract not provided.

Shadid, John N.; Pawlowski, Roger P.; Tuminaro, Raymond S.

Abstract not provided.

Creighton, J.R.; Pawlowski, Roger P.; Coltrin, Michael E.; Figiel, J.J.

Abstract not provided.

Transactions on Mathematical Software

Pawlowski, Roger P.

Abstract not provided.

Salinger, Andrew G.; Owen, Steven J.; Pawlowski, Roger P.; Phipps, Eric T.; Siefert, Christopher S.; Staten, Matthew L.

Abstract not provided.

Shadid, John N.; Pawlowski, Roger P.; Cyr, Eric C.

Abstract not provided.

Creighton, J.R.; Baucom, Kevin C.; Coltrin, Michael E.; Figiel, J.J.; Cross, Karen C.; Koleske, Daniel K.; Pawlowski, Roger P.; Heller, Edwin J.; Bogart, Katherine B.; Coker, Eric N.

Our ability to field useful, nano-enabled microsystems that capitalize on recent advances in sensor technology is severely limited by the energy density of available power sources. The catalytic nanodiode (reported by Somorjai's group at Berkeley in 2005) was potentially an alternative revolutionary source of micropower. Their first reports claimed that a sizable fraction of the chemical energy may be harvested via hot electrons (a 'chemicurrent') that are created by the catalytic chemical reaction. We fabricated and tested Pt/GaN nanodiodes, which eventually produced currents up to several microamps. Our best reaction yields (electrons/CO{sub 2}) were on the order of 10{sup -3}; well below the 75% values first reported by Somorjai (we note they have also been unable to reproduce their early results). Over the course of this Project we have determined that the whole concept of 'chemicurrent', in fact, may be an illusion. Our results conclusively demonstrate that the current measured from our nanodiodes is derived from a thermoelectric voltage; we have found no credible evidence for true chemicurrent. Unfortunately this means that the catalytic nanodiode has no future as a micropower source.

Creighton, J.R.; Coltrin, Michael E.; Baucom, Kevin C.; Pawlowski, Roger P.; Figiel, J.J.

Abstract not provided.

Rao, Rekha R.; Pawlowski, Roger P.; Brotherton, Christopher M.; Cipiti, Benjamin B.; Domino, Stefan P.; Jove Colon, Carlos F.; Moffat, Harry K.; Nemer, Martin N.; Noble, David R.; O'Hern, Timothy J.

Imported oil exacerabates our trade deficit and funds anti-American regimes. Nuclear Energy (NE) is a demonstrated technology with high efficiency. NE's two biggest political detriments are possible accidents and nuclear waste disposal. For NE policy, proliferation is the biggest obstacle. Nuclear waste can be reduced through reprocessing, where fuel rods are separated into various streams, some of which can be reused in reactors. Current process developed in the 1950s is dirty and expensive, U/Pu separation is the most critical. Fuel rods are sheared and dissolved in acid to extract fissile material in a centrifugal contactor. Plants have many contacts in series with other separations. We have taken a science and simulation-based approach to develop a modern reprocessing plant. Models of reprocessing plants are needed to support nuclear materials accountancy, nonproliferation, plant design, and plant scale-up.

Pawlowski, Roger P.

Abstract not provided.

Salinger, Andrew G.; Pawlowski, Roger P.

Dynamical systems theory provides a powerful framework for understanding the behavior of complex evolving systems. However applying these ideas to large-scale dynamical systems such as discretizations of multi-dimensional PDEs is challenging. Such systems can easily give rise to problems with billions of dynamical variables, requiring specialized numerical algorithms implemented on high performance computing architectures with thousands of processors. This talk will describe LOCA, the Library of Continuation Algorithms, a suite of scalable continuation and bifurcation tools optimized for these types of systems that is part of the Trilinos software collection. In particular, we will describe continuation and bifurcation analysis techniques designed for large-scale dynamical systems that are based on specialized parallel linear algebra methods for solving augmented linear systems. We will also discuss several other Trilinos tools providing nonlinear solvers (NOX), eigensolvers (Anasazi), iterative linear solvers (AztecOO and Belos), preconditioners (Ifpack, ML, Amesos) and parallel linear algebra data structures (Epetra and Tpetra) that LOCA can leverage for efficient and scalable analysis of large-scale dynamical systems.

Pawlowski, Roger P.

Abstract not provided.

Phipps, Eric T.; Pawlowski, Roger P.; Red-Horse, John R.; Schmidt, Rodney C.; Eldred, Michael S.

Abstract not provided.

Keiter, Eric R.; Russo, Thomas V.; Schiek, Richard S.; Mei, Ting M.; Thornquist, Heidi K.; Coffey, Todd S.; Santarelli, Keith R.; Pawlowski, Roger P.

This document is a reference guide to the Xyce Parallel Electronic Simulator, and is a companion document to the Xyce Users Guide. The focus of this document is (to the extent possible) exhaustively list device parameters, solver options, parser options, and other usage details of Xyce. This document is not intended to be a tutorial. Users who are new to circuit simulation are better served by the Xyce Users Guide.

Keiter, Eric R.; Santarelli, Keith R.; Hoekstra, Robert J.; Russo, Thomas V.; Schiek, Richard S.; Mei, Ting M.; Thornquist, Heidi K.; Pawlowski, Roger P.; Coffey, Todd S.

The Xyce Parallel Electronic Simulator has been written to support, in a rigorous manner, the simulation needs of the Sandia National Laboratories electrical designers. Specific requirements include, among others, the ability to solve extremely large circuit problems by supporting large-scale parallel computing platforms, improved numerical performance and object-oriented code design and implementation. The Xyce release notes describe: Hardware and software requirements New features and enhancements Any defects fixed since the last release Current known defects and defect workarounds For up-to-date information not available at the time these notes were produced, please visit the Xyce web page at http://www.cs.sandia.gov/xyce.

Shadid, John N.; Tuminaro, Raymond S.; Pawlowski, Roger P.

Abstract not provided.

Cyr, Eric C.; Shadid, John N.; Pawlowski, Roger P.; Tuminaro, Raymond S.

Abstract not provided.

Keiter, Eric R.; Mei, Ting M.; Russo, Thomas V.; Pawlowski, Roger P.; Schiek, Richard S.; Santarelli, Keith R.; Coffey, Todd S.; Thornquist, Heidi K.

This manual describes the use of the Xyce Parallel Electronic Simulator. Xyce has been designed as a SPICE-compatible, high-performance analog circuit simulator, and has been written to support the simulation needs of the Sandia National Laboratories electrical designers. This development has focused on improving capability over the current state-of-the-art in the following areas: (1) Capability to solve extremely large circuit problems by supporting large-scale parallel computing platforms (up to thousands of processors). Note that this includes support for most popular parallel and serial computers. (2) Improved performance for all numerical kernels (e.g., time integrator, nonlinear and linear solvers) through state-of-the-art algorithms and novel techniques. (3) Device models which are specifically tailored to meet Sandia's needs, including some radiation-aware devices (for Sandia users only). (4) Object-oriented code design and implementation using modern coding practices that ensure that the Xyce Parallel Electronic Simulator will be maintainable and extensible far into the future. Xyce is a parallel code in the most general sense of the phrase - a message passing parallel implementation - which allows it to run efficiently on the widest possible number of computing platforms. These include serial, shared-memory and distributed-memory parallel as well as heterogeneous platforms. Careful attention has been paid to the specific nature of circuit-simulation problems to ensure that optimal parallel efficiency is achieved as the number of processors grows. The development of Xyce provides a platform for computational research and development aimed specifically at the needs of the Laboratory. With Xyce, Sandia has an 'in-house' capability with which both new electrical (e.g., device model development) and algorithmic (e.g., faster time-integration methods, parallel solver algorithms) research and development can be performed. As a result, Xyce is a unique electrical simulation capability, designed to meet the unique needs of the laboratory.

Keiter, Eric R.; Mei, Ting M.; Russo, Thomas V.; Pawlowski, Roger P.; Schiek, Richard S.; Santarelli, Keith R.; Coffey, Todd S.; Thornquist, Heidi K.

This document is a reference guide to the Xyce Parallel Electronic Simulator, and is a companion document to the Xyce Users’ Guide. The focus of this document is (to the extent possible) exhaustively list device parameters, solver options, parser options, and other usage details of Xyce. This document is not intended to be a tutorial. Users who are new to circuit simulation are better served by the Xyce Users’ Guide.

Pawlowski, Roger P.

Abstract not provided.

Phipps, Eric T.; Salinger, Andrew G.; Pawlowski, Roger P.

Abstract not provided.

Pawlowski, Roger P.

Abstract not provided.

Keiter, Eric R.; Mei, Ting M.; Russo, Thomas V.; Pawlowski, Roger P.; Schiek, Richard S.; Santarelli, Keith R.; Coffey, Todd S.; Thornquist, Heidi K.

This document is a reference guide to the Xyce Parallel Electronic Simulator, and is a companion document to the Xyce Users Guide. The focus of this document is (to the extent possible) exhaustively list device parameters, solver options, parser options, and other usage details of Xyce. This document is not intended to be a tutorial. Users who are new to circuit simulation are better served by the Xyce Users Guide.

Keiter, Eric R.; Mei, Ting M.; Russo, Thomas V.; Pawlowski, Roger P.; Schiek, Richard S.; Santarelli, Keith R.; Coffey, Todd S.; Thornquist, Heidi K.

This manual describes the use of the Xyce Parallel Electronic Simulator. Xyce has been designed as a SPICE-compatible, high-performance analog circuit simulator, and has been written to support the simulation needs of the Sandia National Laboratories electrical designers. This development has focused on improving capability over the current state-of-the-art in the following areas: (1) Capability to solve extremely large circuit problems by supporting large-scale parallel computing platforms (up to thousands of processors). Note that this includes support for most popular parallel and serial computers. (2) Improved performance for all numerical kernels (e.g., time integrator, nonlinear and linear solvers) through state-of-the-art algorithms and novel techniques. (3) Device models which are specifically tailored to meet Sandia's needs, including some radiation-aware devices (for Sandia users only). (4) Object-oriented code design and implementation using modern coding practices that ensure that the Xyce Parallel Electronic Simulator will be maintainable and extensible far into the future. Xyce is a parallel code in the most general sense of the phrase - a message passing parallel implementation - which allows it to run efficiently on the widest possible number of computing platforms. These include serial, shared-memory and distributed-memory parallel as well as heterogeneous platforms. Careful attention has been paid to the specific nature of circuit-simulation problems to ensure that optimal parallel efficiency is achieved as the number of processors grows. The development of Xyce provides a platform for computational research and development aimed specifically at the needs of the Laboratory. With Xyce, Sandia has an 'in-house' capability with which both new electrical (e.g., device model development) and algorithmic (e.g., faster time-integration methods, parallel solver algorithms) research and development can be performed. As a result, Xyce is a unique electrical simulation capability, designed to meet the unique needs of the laboratory.

Hoekstra, Robert J.; Hennigan, Gary L.; Pawlowski, Roger P.; Phipps, Eric T.; Shadid, John N.; Musson, Lawrence M.; Keiter, Eric R.

Abstract not provided.

Pawlowski, Roger P.; Salinger, Andrew G.

Abstract not provided.

Pawlowski, Roger P.; Salinger, Andrew G.

Singlet oxygen generators are multiphase flow chemical reactors used to generate energetic oxygen to be used as a fuel for chemical oxygen iodine lasers. In this paper, a theoretical model of the generator is presented along with its solutions over ranges of parameter space and oxygen maximizing optimizations. The singlet oxygen generator (SOG) is a low-pressure, multiphase flow chemical reactor that is used to produce molecular oxygen in an electronically excited state, i.e. singlet delta oxygen. The primary product of the reactor, the energetic oxygen, is used in a stage immediately succeeding the SOG to dissociate and energize iodine. The gas mixture including the iodine is accelerated to a supersonic speed and lased. Thus the SOG is the fuel generator for the chemical oxygen iodine laser (COIL). The COIL has important application for both military purposes--it was developed by the US Air Force in the 1970s--and, as the infrared beam is readily absorbed by metals, industrial cutting and drilling. The SOG appears in various configurations, but the one in focus here is a crossflow droplet generator SOG. A gas consisting of molecular chlorine and a diluent, usually helium, is pumped through a roughly rectangular channel. An aqueous solution of hydrogen peroxide and potassium hydroxide is pumped through small holes into the channel and perpendicular to the direction of the gas flow. So doing causes the solution to become aerosolized. Dissociation of the potassium hydroxide draws a proton from the hydrogen peroxide generating an HO{sub 2} radical in the liquid. Chlorine diffuses into the liquid and reacts with the HO{sub 2} ion producing the singlet delta oxygen; some of the oxygen diffuses back into the gas phase. The focus of this work is to generate a predictive multiphase flow model of the SOG in order to optimize its design. The equations solved are the so-called Eulerian-Eulerian form of the multiphase flow Navier-Stokes equations wherein one set of the equations represents the gas phase and another equation set of size m represents the liquid phase. In this case, m is representative of the division of the liquid phase into distinct representations of the various droplet sizes distributed in the reactor. A stabilized Galerkin formulation is used to solve the equation set on a computer. The set of equations is large. There are five equations representing the gas phase: continuity, vector momentum, heat. There are 5m representing the liquid phase: number density, vector momentum, heat. Four mass transfer equations represent the gas phase constituents and there are m advection diffusion equations representing the HO{sub 2} ion concentration in the liquid phase. Thus we are taking advantage of and developing algorithms to harness the power of large parallel computing architectures to solve the steady-state form of these equations numerous times so as to explore the large parameter space of the equations via continuation methods and to maximize the generation of singlet delta oxygen via optimization methods. Presented here will be the set of equations that are solved and the methods we are using to solve them. Solutions of the equations will be presented along with solution paths representing varying aerosol loading-the ratio of liquid to gas mass flow rates-and simple optimizations centered around maximizing the oxygen production and minimizing the amount of entrained liquid in the gas exit stream. Gas-entrained liquid is important to minimize as it can destroy the lenses and mirrors present in the lasing cavity.

Pawlowski, Roger P.; Shadid, John N.; Phipps, Eric T.

Abstract not provided.

Pawlowski, Roger P.; Shadid, John N.; Phipps, Eric T.; Bartlett, Roscoe B.

Abstract not provided.

Hopkins, Matthew M.; Pawlowski, Roger P.; Moffat, Harry K.; Carnes, Brian C.; Hooper, Russell H.

Many current and future modeling applications at Sandia including ASC milestones will critically depend on the simultaneous solution of vastly different physical phenomena. Issues due to code coupling are often not addressed, understood, or even recognized. The objectives of the LDRD has been both in theory and in code development. We will show that we have provided a fundamental analysis of coupling, i.e., when strong coupling vs. a successive substitution strategy is needed. We have enabled the implementation of tighter coupling strategies through additions to the NOX and Sierra code suites to make coupling strategies available now. We have leveraged existing functionality to do this. Specifically, we have built into NOX the capability to handle fully coupled simulations from multiple codes, and we have also built into NOX the capability to handle Jacobi Free Newton Krylov simulations that link multiple applications. We show how this capability may be accessed from within the Sierra Framework as well as from outside of Sierra. The critical impact from this LDRD is that we have shown how and have delivered strategies for enabling strong Newton-based coupling while respecting the modularity of existing codes. This will facilitate the use of these codes in a coupled manner to solve multi-physic applications.

Pawlowski, Roger P.; Hill, Judith C.; Salinger, Andrew G.

Abstract not provided.

Pawlowski, Roger P.; Banks, Jeffrey W.; Shadid, John N.

Abstract not provided.

Pawlowski, Roger P.

Abstract not provided.

Pawlowski, Roger P.; Hopkins, Matthew M.

Abstract not provided.

Keiter, Eric R.; Hoekstra, Robert J.; Coffey, Todd S.; Heroux, Michael A.; Pawlowski, Roger P.; Hutchinson, Scott A.

Abstract not provided.

Phipps, Eric T.; Salinger, Andrew G.; Pawlowski, Roger P.

Abstract not provided.

Bader, Brett W.; Pawlowski, Roger P.; Kolda, Tamara G.

This report documents research to develop robust and efficient solution techniques for solving large-scale systems of nonlinear equations. The most widely used method for solving systems of nonlinear equations is Newton's method. While much research has been devoted to augmenting Newton-based solvers (usually with globalization techniques), little has been devoted to exploring the application of different models. Our research has been directed at evaluating techniques using different models than Newton's method: a lower order model, Broyden's method, and a higher order model, the tensor method. We have developed large-scale versions of each of these models and have demonstrated their use in important applications at Sandia. Broyden's method replaces the Jacobian with an approximation, allowing codes that cannot evaluate a Jacobian or have an inaccurate Jacobian to converge to a solution. Limited-memory methods, which have been successful in optimization, allow us to extend this approach to large-scale problems. We compare the robustness and efficiency of Newton's method, modified Newton's method, Jacobian-free Newton-Krylov method, and our limited-memory Broyden method. Comparisons are carried out for large-scale applications of fluid flow simulations and electronic circuit simulations. Results show that, in cases where the Jacobian was inaccurate or could not be computed, Broyden's method converged in some cases where Newton's method failed to converge. We identify conditions where Broyden's method can be more efficient than Newton's method. We also present modifications to a large-scale tensor method, originally proposed by Bouaricha, for greater efficiency, better robustness, and wider applicability. Tensor methods are an alternative to Newton-based methods and are based on computing a step based on a local quadratic model rather than a linear model. The advantage of Bouaricha's method is that it can use any existing linear solver, which makes it simple to write and easily portable. However, the method usually takes twice as long to solve as Newton-GMRES on general problems because it solves two linear systems at each iteration. In this paper, we discuss modifications to Bouaricha's method for a practical implementation, including a special globalization technique and other modifications for greater efficiency. We present numerical results showing computational advantages over Newton-GMRES on some realistic problems. We further discuss a new approach for dealing with singular (or ill-conditioned) matrices. In particular, we modify an algorithm for identifying a turning point so that an increasingly ill-conditioned Jacobian does not prevent convergence.

Hooper, Russell H.; Hopkins, Matthew M.; Pawlowski, Roger P.

Abstract not provided.

Hutchinson, Scott A.; Keiter, Eric R.; Hoekstra, Robert J.; Russo, Thomas V.; Rankin, Eric R.; Pawlowski, Roger P.; Fixel, Deborah A.; Schiek, Richard S.; Bogdan, Carolyn W.

This manual describes the use of theXyceParallel Electronic Simulator.Xycehasbeen designed as a SPICE-compatible, high-performance analog circuit simulator, andhas been written to support the simulation needs of the Sandia National Laboratorieselectrical designers. This development has focused on improving capability over thecurrent state-of-the-art in the following areas:%04Capability to solve extremely large circuit problems by supporting large-scale par-allel computing platforms (up to thousands of processors). Note that this includessupport for most popular parallel and serial computers.%04Improved performance for all numerical kernels (e.g., time integrator, nonlinearand linear solvers) through state-of-the-art algorithms and novel techniques.%04Device models which are specifically tailored to meet Sandia's needs, includingmany radiation-aware devices.3 XyceTMUsers' Guide%04Object-oriented code design and implementation using modern coding practicesthat ensure that theXyceParallel Electronic Simulator will be maintainable andextensible far into the future.Xyceis a parallel code in the most general sense of the phrase - a message passingparallel implementation - which allows it to run efficiently on the widest possible numberof computing platforms. These include serial, shared-memory and distributed-memoryparallel as well as heterogeneous platforms. Careful attention has been paid to thespecific nature of circuit-simulation problems to ensure that optimal parallel efficiencyis achieved as the number of processors grows.The development ofXyceprovides a platform for computational research and de-velopment aimed specifically at the needs of the Laboratory. WithXyce, Sandia hasan %22in-house%22 capability with which both new electrical (e.g., device model develop-ment) and algorithmic (e.g., faster time-integration methods, parallel solver algorithms)research and development can be performed. As a result,Xyceis a unique electricalsimulation capability, designed to meet the unique needs of the laboratory.4 XyceTMUsers' GuideAcknowledgementsThe authors would like to acknowledge the entire Sandia National Laboratories HPEMS(High Performance Electrical Modeling and Simulation) team, including Steve Wix, CarolynBogdan, Regina Schells, Ken Marx, Steve Brandon and Bill Ballard, for their support onthis project. We also appreciate very much the work of Jim Emery, Becky Arnold and MikeWilliamson for the help in reviewing this document.Lastly, a very special thanks to Hue Lai for typesetting this document with LATEX.TrademarksThe information herein is subject to change without notice.Copyrightc 2002-2003 Sandia Corporation. All rights reserved.XyceTMElectronic Simulator andXyceTMtrademarks of Sandia Corporation.Orcad, Orcad Capture, PSpice and Probe are registered trademarks of Cadence DesignSystems, Inc.Silicon Graphics, the Silicon Graphics logo and IRIX are registered trademarks of SiliconGraphics, Inc.Microsoft, Windows and Windows 2000 are registered trademark of Microsoft Corporation.Solaris and UltraSPARC are registered trademarks of Sun Microsystems Corporation.Medici, DaVinci and Taurus are registered trademarks of Synopsys Corporation.HP and Alpha are registered trademarks of Hewlett-Packard company.Amtec and TecPlot are trademarks of Amtec Engineering, Inc.Xyce's expression library is based on that inside Spice 3F5 developed by the EECS De-partment at the University of California.All other trademarks are property of their respective owners.ContactsBug Reportshttp://tvrusso.sandia.gov/bugzillaEmailxyce-support%40sandia.govWorld Wide Webhttp://www.cs.sandia.gov/xyce5 XyceTMUsers' GuideThis page is left intentionally blank6

Hopkins, Matthew M.; Pawlowski, Roger P.; Hooper, Russell H.

Abstract not provided.

Hooper, Russell H.; Hopkins, Matthew M.; Pawlowski, Roger P.; Notz, Patrick N.; Gianoulakis, Steven E.

Abstract not provided.

Proposed for publication in Computer Methods in Applied Mechanics and Engineering.

Shadid, John N.; Salinger, Andrew G.; Pawlowski, Roger P.; Lin, Paul L.; Hennigan, Gary L.; Tuminaro, Raymond S.; Lehoucq, Richard B.

The solution of the governing steady transport equations for momentum, heat and mass transfer in fluids undergoing non-equilibrium chemical reactions can be extremely challenging. The difficulties arise from both the complexity of the nonlinear solution behavior as well as the nonlinear, coupled, non-symmetric nature of the system of algebraic equations that results from spatial discretization of the PDEs. In this paper, we briefly review progress on developing a stabilized finite element (FE) capability for numerical solution of these challenging problems. The discussion considers the stabilized FE formulation for the low Mach number Navier-Stokes equations with heat and mass transport with non-equilibrium chemical reactions, and the solution methods necessary for detailed analysis of these complex systems. The solution algorithms include robust nonlinear and linear solution schemes, parameter continuation methods, and linear stability analysis techniques. Our discussion considers computational efficiency, scalability, and some implementation issues of the solution methods. Computational results are presented for a CFD benchmark problem as well as for a number of large-scale, 2D and 3D, engineering transport/reaction applications.

Pawlowski, Roger P.; Keiter, Eric R.; Hoekstra, Robert J.; Hutchinson, Scott A.; Russo, Thomas V.

Abstract not provided.

Pawlowski, Roger P.; Hooper, Russell H.; Kolda, Tamara G.; Phipps, Eric T.; Bader, Brett W.; Salinger, Andrew G.

Abstract not provided.

Hopkins, Matthew M.; Hooper, Russell H.; Pawlowski, Roger P.

Abstract not provided.

Hooper, Russell H.; Hopkins, Matthew M.; Pawlowski, Roger P.

Abstract not provided.

Proposed for publication in Computation Methods in Applied Mechanics and Engineering.

Shadid, John N.; Salinger, Andrew G.; Pawlowski, Roger P.; Lin, Paul L.; Hennigan, Gary L.; Tuminaro, Raymond S.; Lehoucq, Richard B.

The solution of the governing steady transport equations for momentum, heat and mass transfer in fluids undergoing non-equilibrium chemical reactions can be extremely challenging. The difficulties arise from both the complexity of the nonlinear solution behavior as well as the nonlinear, coupled, non-symmetric nature of the system of algebraic equations that results from spatial discretization of the PDEs. In this paper, we briefly review progress on developing a stabilized finite element ( FE) capability for numerical solution of these challenging problems. The discussion considers the stabilized FE formulation for the low Mach number Navier-Stokes equations with heat and mass transport with non-equilibrium chemical reactions, and the solution methods necessary for detailed analysis of these complex systems. The solution algorithms include robust nonlinear and linear solution schemes, parameter continuation methods, and linear stability analysis techniques. Our discussion considers computational efficiency, scalability, and some implementation issues of the solution methods. Computational results are presented for a CFD benchmark problem as well as for a number of large-scale, 2D and 3D, engineering transport/reaction applications.

Keiter, Eric R.; Hutchinson, Scott A.; Hoekstra, Robert J.; Russo, Thomas V.; Rankin, Eric R.; Pawlowski, Roger P.; Wix, Steven D.; Fixel, Deborah A.

This manual describes the use of the Xyce Parallel Electronic Simulator. Xyce has been designed as a SPICE-compatible, high-performance analog circuit simulator capable of simulating electrical circuits at a variety of abstraction levels. Primarily, Xyce has been written to support the simulation needs of the Sandia National Laboratories electrical designers. This development has focused on improving capability the current state-of-the-art in the following areas: {sm_bullet} Capability to solve extremely large circuit problems by supporting large-scale parallel computing platforms (up to thousands of processors). Note that this includes support for most popular parallel and serial computers. {sm_bullet} Improved performance for all numerical kernels (e.g., time integrator, nonlinear and linear solvers) through state-of-the-art algorithms and novel techniques. {sm_bullet} Device models which are specifically tailored to meet Sandia's needs, including many radiation-aware devices. {sm_bullet} A client-server or multi-tiered operating model wherein the numerical kernel can operate independently of the graphical user interface (GUI). {sm_bullet} Object-oriented code design and implementation using modern coding practices that ensure that the Xyce Parallel Electronic Simulator will be maintainable and extensible far into the future. Xyce is a parallel code in the most general sense of the phrase - a message passing of computing platforms. These include serial, shared-memory and distributed-memory parallel implementation - which allows it to run efficiently on the widest possible number parallel as well as heterogeneous platforms. Careful attention has been paid to the specific nature of circuit-simulation problems to ensure that optimal parallel efficiency is achieved as the number of processors grows. One feature required by designers is the ability to add device models, many specific to the needs of Sandia, to the code. To this end, the device package in the Xyce These input formats include standard analytical models, behavioral models look-up Parallel Electronic Simulator is designed to support a variety of device model inputs. tables, and mesh-level PDE device models. Combined with this flexible interface is an architectural design that greatly simplifies the addition of circuit models. One of the most important feature of Xyce is in providing a platform for computational research and development aimed specifically at the needs of the Laboratory. With Xyce, Sandia now has an 'in-house' capability with which both new electrical (e.g., device model development) and algorithmic (e.g., faster time-integration methods) research and development can be performed. Ultimately, these capabilities are migrated to end users.

Keiter, Eric R.; Hutchinson, Scott A.; Hoekstra, Robert J.; Russo, Thomas V.; Rankin, Eric R.; Pawlowski, Roger P.; Fixel, Deborah A.; Wix, Steven D.

This document is a reference guide to the Xyce Parallel Electronic Simulator, and is a companion document to the Xyce Users' Guide. The focus of this document is (to the extent possible) exhaustively list device parameters, solver options, parser options, and other usage details of Xyce. This document is not intended to be a tutorial. Users who are new to circuit simulation are better served by the Xyce Users' Guide.

Keiter, Eric R.; Pawlowski, Roger P.; Hoekstra, Robert J.; Hutchinson, Scott A.; Russo, Thomas V.; Coffey, Todd S.; Kolda, Tamara G.

Abstract not provided.

Keiter, Eric R.; Pawlowski, Roger P.; Hutchinson, Scott A.; Hoekstra, Robert J.; Russo, Thomas V.; Rankin, Eric R.

Abstract not provided.

Simmons, J.A.; Fischer, Arthur J.; Crawford, Mary H.; Abrams, B.L.; Biefeld, Robert M.; Koleske, Daniel K.; Allerman, A.A.; Figiel, J.J.; Creighton, J.R.; Coltrin, Michael E.; Tsao, Jeffrey Y.; Mitchell, Christine C.; Kerley, Thomas M.; Wang, George T.; Bogart, Katherine B.; Seager, Carleton H.; Campbell, Jonathan C.; Follstaedt, D.M.; Norman, Adam K.; Kurtz, S.R.; Wright, Alan F.; Myers, S.M.; Missert, Nancy A.; Copeland, Robert G.; Provencio, P.N.; Wilcoxon, Jess P.; Hadley, G.R.; Wendt, J.R.; Kaplar, Robert K.; Shul, Randy J.; Rohwer, Lauren E.; Tallant, David T.; Simpson, Regina L.; Moffat, Harry K.; Salinger, Andrew G.; Pawlowski, Roger P.; Emerson, John A.; Thoma, Steven T.; Cole, Phillip J.; Boyack, Kevin W.; Garcia, Marie L.; Allen, Mark S.; Burdick, Brent B.; Rahal, Nabeel R.; Monson, Mary A.; Chow, Weng W.; Waldrip, Karen E.

This SAND report is the final report on Sandia's Grand Challenge LDRD Project 27328, 'A Revolution in Lighting -- Building the Science and Technology Base for Ultra-Efficient Solid-state Lighting.' This project, which for brevity we refer to as the SSL GCLDRD, is considered one of Sandia's most successful GCLDRDs. As a result, this report reviews not only technical highlights, but also the genesis of the idea for Solid-state Lighting (SSL), the initiation of the SSL GCLDRD, and the goals, scope, success metrics, and evolution of the SSL GCLDRD over the course of its life. One way in which the SSL GCLDRD was different from other GCLDRDs was that it coincided with a larger effort by the SSL community - primarily industrial companies investing in SSL, but also universities, trade organizations, and other Department of Energy (DOE) national laboratories - to support a national initiative in SSL R&D. Sandia was a major player in publicizing the tremendous energy savings potential of SSL, and in helping to develop, unify and support community consensus for such an initiative. Hence, our activities in this area, discussed in Chapter 6, were substantial: white papers; SSL technology workshops and roadmaps; support for the Optoelectronics Industry Development Association (OIDA), DOE and Senator Bingaman's office; extensive public relations and media activities; and a worldwide SSL community website. Many science and technology advances and breakthroughs were also enabled under this GCLDRD, resulting in: 55 publications; 124 presentations; 10 book chapters and reports; 5 U.S. patent applications including 1 already issued; and 14 patent disclosures not yet applied for. Twenty-six invited talks were given, at prestigious venues such as the American Physical Society Meeting, the Materials Research Society Meeting, the AVS International Symposium, and the Electrochemical Society Meeting. This report contains a summary of these science and technology advances and breakthroughs, with Chapters 1-5 devoted to the five technical task areas: 1 Fundamental Materials Physics; 2 111-Nitride Growth Chemistry and Substrate Physics; 3 111-Nitride MOCVD Reactor Design and In-Situ Monitoring; 4 Advanced Light-Emitting Devices; and 5 Phosphors and Encapsulants. Chapter 7 (Appendix A) contains a listing of publications, presentations, and patents. Finally, the SSL GCLDRD resulted in numerous actual and pending follow-on programs for Sandia, including multiple grants from DOE and the Defense Advanced Research Projects Agency (DARPA), and Cooperative Research and Development Agreements (CRADAs) with SSL companies. Many of these follow-on programs arose out of contacts developed through our External Advisory Committee (EAC). In h s and other ways, the EAC played a very important role. Chapter 8 (Appendix B) contains the full (unedited) text of the EAC reviews that were held periodically during the course of the project.

Heroux, Michael A.; Kolda, Tamara G.; Long, Kevin R.; Hoekstra, Robert J.; Pawlowski, Roger P.; Phipps, Eric T.; Salinger, Andrew G.; Williams, Alan B.; Heroux, Michael A.; Hu, Jonathan J.; Lehoucq, Richard B.; Thornquist, Heidi K.; Tuminaro, Raymond S.; Willenbring, James M.; Bartlett, Roscoe B.; Howle, Victoria E.

The Trilinos Project is an effort to facilitate the design, development, integration and ongoing support of mathematical software libraries. In particular, our goal is to develop parallel solver algorithms and libraries within an object-oriented software framework for the solution of large-scale, complex multi-physics engineering and scientific applications. Our emphasis is on developing robust, scalable algorithms in a software framework, using abstract interfaces for flexible interoperability of components while providing a full-featured set of concrete classes that implement all abstract interfaces. Trilinos uses a two-level software structure designed around collections of packages. A Trilinos package is an integral unit usually developed by a small team of experts in a particular algorithms area such as algebraic preconditioners, nonlinear solvers, etc. Packages exist underneath the Trilinos top level, which provides a common look-and-feel, including configuration, documentation, licensing, and bug-tracking. Trilinos packages are primarily written in C++, but provide some C and Fortran user interface support. We provide an open architecture that allows easy integration with other solver packages and we deliver our software to the outside community via the Gnu Lesser General Public License (LGPL). This report provides an overview of Trilinos, discussing the objectives, history, current development and future plans of the project.

Salinger, Andrew G.; Pawlowski, Roger P.; Lehoucq, Richard B.; Romero, L.A.; Wilkes, Edward D.

LOCA, the Library of Continuation Algorithms, is a software library for performing stability analysis of large-scale applications. LOCA enables the tracking of solution branches as a function of a system parameter, the direct tracking of bifurcation points, and, when linked with the ARPACK library, a linear stability analysis capability. It is designed to be easy to implement around codes that already use Newton's method to converge to steady-state solutions. The algorithms are chosen to work for large problems, such as those that arise from discretizations of partial differential equations, and to run on distributed memory parallel machines. This manual presents LOCA's continuation and bifurcation analysis algorithms, and instructions on how to implement LOCA with an application code. The LOCA code is being made publicly available at www.cs.sandia.gov/loca.

Pawlowski, Roger P.; Salinger, Andrew G.; Salinger, Andrew G.

Abstract not provided.

Salinger, Andrew G.; Pawlowski, Roger P.; Shadid, John N.; van Bloemen Waanders, Bart G.

Abstract not provided.

Pawlowski, Roger P.; Salinger, Andrew G.; Habermehl, Scott D.; Ho, Pauline H.

Abstract not provided.

Salinger, Andrew G.; Pawlowski, Roger P.; Romero, L.A.

Abstract not provided.