Publications

72 Results
Skip to search filters

Towards Predictive Plasma Science and Engineering through Revolutionary Multi-Scale Algorithms and Models (Final Report)

Laity, George R.; Robinson, Allen C.; Cuneo, M.E.; Alam, Mary K.; Beckwith, Kristian B.; Bennett, Nichelle L.; Bettencourt, Matthew T.; Bond, Stephen D.; Cochrane, Kyle C.; Criscenti, Louise C.; Cyr, Eric C.; De Zetter, Karen J.; Drake, Richard R.; Evstatiev, Evstati G.; Fierro, Andrew S.; Gardiner, Thomas A.; Glines, Forrest W.; Goeke, Ronald S.; Hamlin, Nathaniel D.; Hooper, Russell H.; Koski, Jason K.; Lane, James M.; Larson, Steven R.; Leung, Kevin L.; McGregor, Duncan A.; Miller, Philip R.; Miller, Sean M.; Ossareh, Susan J.; Phillips, Edward G.; Simpson, Sean S.; Sirajuddin, David S.; Smith, Thomas M.; Swan, Matthew S.; Thompson, Aidan P.; Tranchida, Julien G.; Bortz-Johnson, Asa J.; Welch, Dale R.; Russell, Alex M.; Watson, Eric D.; Rose, David V.; McBride, Ryan D.

This report describes the high-level accomplishments from the Plasma Science and Engineering Grand Challenge LDRD at Sandia National Laboratories. The Laboratory has a need to demonstrate predictive capabilities to model plasma phenomena in order to rapidly accelerate engineering development in several mission areas. The purpose of this Grand Challenge LDRD was to advance the fundamental models, methods, and algorithms along with supporting electrode science foundation to enable a revolutionary shift towards predictive plasma engineering design principles. This project integrated the SNL knowledge base in computer science, plasma physics, materials science, applied mathematics, and relevant application engineering to establish new cross-laboratory collaborations on these topics. As an initial exemplar, this project focused efforts on improving multi-scale modeling capabilities that are utilized to predict the electrical power delivery on large-scale pulsed power accelerators. Specifically, this LDRD was structured into three primary research thrusts that, when integrated, enable complex simulations of these devices: (1) the exploration of multi-scale models describing the desorption of contaminants from pulsed power electrodes, (2) the development of improved algorithms and code technologies to treat the multi-physics phenomena required to predict device performance, and (3) the creation of a rigorous verification and validation infrastructure to evaluate the codes and models across a range of challenge problems. These components were integrated into initial demonstrations of the largest simulations of multi-level vacuum power flow completed to-date, executed on the leading HPC computing machines available in the NNSA complex today. These preliminary studies indicate relevant pulsed power engineering design simulations can now be completed in (of order) several days, a significant improvement over pre-LDRD levels of performance.

More Details
TYPE SAND Report YEAR 2021
OSTIDOI

EMPIRE-PIC: A performance portable unstructured particle-in-cell code

Communications in Computational Physics

Bettencourt, Matthew T.; Brown, Dominic A.S.; Cartwright, Keith L.; Cyr, Eric C.; Glusa, Christian A.; Lin, Paul T.; Moore, Stan G.; McGregor, Duncan A.O.; Pawlowski, Roger P.; Phillips, Edward G.; Roberts, Nathan V.; Wright, Steven A.; Maheswaran, Satheesh; Jones, John P.; Jarvis, Stephen A.

In this paper we introduce EMPIRE-PIC, a finite element method particle-in-cell (FEM-PIC) application developed at Sandia National Laboratories. The code has been developed in C++ using the Trilinos library and the Kokkos Performance Portability Framework to enable running on multiple modern compute architectures while only requiring maintenance of a single codebase. EMPIRE-PIC is capable of solving both electrostatic and electromagnetic problems in two- and three-dimensions to second-order accuracy in space and time. In this paper we validate the code against three benchmark problems - a simple electron orbit, an electrostatic Langmuir wave, and a transverse electromagnetic wave propagating through a plasma. We demonstrate the performance of EMPIRE-PIC on four different architectures: Intel Haswell CPUs, Intel's Xeon Phi Knights Landing, ARM Thunder-X2 CPUs, and NVIDIA Tesla V100 GPUs attached to IBM POWER9 processors. This analysis demonstrates scalability of the code up to more than two thousand GPUs, and greater than one hundred thousand CPUs.

More Details
TYPE Journal Article YEAR 2021
ScopusOSTIDOI

Enabling Scalable Multifluid Plasma Simulations Through Block Preconditioning

Lecture Notes in Computational Science and Engineering

Phillips, Edward G.; Shadid, John N.; Cyr, Eric C.; Miller, Sean M.

Recent work has demonstrated that block preconditioning can scalably accelerate the performance of iterative solvers applied to linear systems arising in implicit multiphysics PDE simulations. The idea of block preconditioning is to decompose the system matrix into physical sub-blocks and apply individual specialized scalable solvers to each sub-block. It can be advantageous to block into simpler segregated physics systems or to block by discretization type. This strategy is particularly amenable to multiphysics systems in which existing solvers, such as multilevel methods, can be leveraged for component physics and to problems with disparate discretizations in which scalable monolithic solvers are rare. This work extends our recent work on scalable block preconditioning methods for structure-preserving discretizatons of the Maxwell equations and our previous work in MHD system solvers to the context of multifluid electromagnetic plasma systems. We argue how a block preconditioner can address both the disparate discretization, as well as strongly-coupled off-diagonal physics that produces fast time-scales (e.g. plasma and cyclotron frequencies). We propose a block preconditioner for plasma systems that allows reuse of existing multigrid solvers for different degrees of freedom while capturing important couplings, and demonstrate the algorithmic scalability of this approach at time-scales of interest.

More Details
TYPE Conference Poster YEAR 2020
ScopusOSTIDOI

IMEX and exact sequence discretization of the multi-fluid plasma model

Journal of Computational Physics

Miller, Sean M.; Cyr, E.C.; Shadid, John N.; Kramer, R.M.J.; Phillips, Edward G.; Conde, Sidafa C.; Pawlowski, Roger P.

Multi-fluid plasma models, where an electron fluid is modeled in addition to multiple ion and neutral species as well as the full set of Maxwell's equations, are useful for representing physics beyond the scope of classic MHD. This advantage presents challenges in appropriately dealing with electron dynamics and electromagnetic behavior characterized by the plasma and cyclotron frequencies and the speed of light. For physical systems, such as those near the MHD asymptotic regime, this requirement drastically increases runtimes for explicit time integration even though resolving fast dynamics may not be critical for accuracy. Implicit time integration methods, with efficient solvers, can help to step over fast time-scales that constrain stability, but do not strongly influence accuracy. As an extension, Implicit-explicit (IMEX) schemes provide an additional mechanism to choose which dynamics are evolved using an expensive implicit solve or resolved using a fast explicit solve. In this study, in addition to IMEX methods we also consider a physics compatible exact sequence spatial discretization. This combines nodal bases (H-Grad) for fluid dynamics with a set of vector bases (H-Curl and H-Div) for Maxwell's equations. This discretization allows for multi-fluid plasma modeling without violating Gauss' laws for the electric and magnetic fields. This initial study presents a discussion of the major elements of this formulation and focuses on demonstrating accuracy in the linear wave regime and in the MHD limit for both a visco-resistive and a dispersive ideal MHD problem.

More Details
TYPE Journal Article YEAR 2019
ScopusOSTIDOI

ASC ATDM Level 2 Milestone #6358: Assess Status of Next Generation Components and Physics Models in EMPIRE

Bettencourt, Matthew T.; Kramer, Richard M.; Cartwright, Keith C.; Phillips, Edward G.; Ober, Curtis C.; Pawlowski, Roger P.; Swan, Matthew S.; Kalashnikova, Irina; Phipps, Eric T.; Conde, Sidafa C.; Cyr, Eric C.; Ulmer, Craig D.; Kordenbrock, Todd H.; Levy, Scott L.; Templet, Gary J.; Hu, Jonathan J.; Lin, Paul L.; Glusa, Christian A.; Siefert, Christopher S.; Glass, Micheal W.

This report documents the outcome from the ASC ATDM Level 2 Milestone 6358: Assess Status of Next Generation Components and Physics Models in EMPIRE. This Milestone is an assessment of the EMPIRE (ElectroMagnetic Plasma In Realistic Environments) application and three software components. The assessment focuses on the electromagnetic and electrostatic particle-in-cell solu- tions for EMPIRE and its associated solver, time integration, and checkpoint-restart components. This information provides a clear understanding of the current status of the EMPIRE application and will help to guide future work in FY19 in order to ready the application for the ASC ATDM L 1 Milestone in FY20. It is clear from this assessment that performance of the linear solver will have to be a focus in FY19.

More Details
TYPE SAND Report YEAR 2018
OSTIDOI

Scalable preconditioners for structure preserving discretizations of maxwell equations in first order form

SIAM Journal on Scientific Computing

Phillips, Edward G.; Shadid, John N.; Cyr, Eric C.

Multiple physical time-scales can arise in electromagnetic simulations when dissipative effects are introduced through boundary conditions, when currents follow external time-scales, and when material parameters vary spatially. In such scenarios, the time-scales of interest may be much slower than the fastest time-scales supported by the Maxwell equations, therefore making implicit time integration an efficient approach. The use of implicit temporal discretizations results in linear systems in which fast time-scales, which severely constrain the stability of an explicit method, can manifest as so-called stiff modes. This study proposes a new block preconditioner for structure preserving (also termed physics compatible) discretizations of the Maxwell equations in first order form. The intent of the preconditioner is to enable the efficient solution of multiple-time-scale Maxwell type systems. An additional benefit of the developed preconditioner is that it requires only a traditional multigrid method for its subsolves and compares well against alternative approaches that rely on specialized edge-based multigrid routines that may not be readily available. Results demonstrate parallel scalability at large electromagnetic wave CFL numbers on a variety of test problems.

More Details
TYPE Journal Article YEAR 2018
ScopusOSTIDOI

Towards a scalable multifidelity simulation approach for electrokinetic problems at the mesoscale

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Hong, Brian D.; Perego, Mauro P.; Bochev, Pavel B.; Frischknecht, Amalie F.; Phillips, Edward G.

In this work we present a computational capability featuring a hierarchy of models with different fidelities for the solution of electrokinetics problems at the micro-/nano-scale. A multifidelity approach allows the selection of the most appropriate model, in terms of accuracy and computational cost, for the particular application at hand. We demonstrate the proposed multifidelity approach by studying the mobility of a colloid in a micro-channel as a function of the colloid charge and of the size of the ions dissolved in the fluid.

More Details
TYPE Conference Poster YEAR 2018
ScopusOSTIDOI

A block preconditioner for an exact penalty formulation for stationary MHD

SIAM Journal on Scientific Computing

Phillips, Edward G.; Elman, Howard C.; Cyr, Eric C.; Shadid, John N.; Pawlowski, Roger P.

The magnetohydrodynamics (MHD) equations are used to model the flow of electrically conducting fluids in such applications as liquid metals and plasmas. This system of nonself-adjoint, nonlinear PDEs couples the Navier-Stokes equations for fluids and Maxwell's equations for electromagnetics. There has been recent interest in fully coupled solvers for the MHD system because they allow for fast steady-state solutions that do not require pseudo-time-stepping. When the fully coupled system is discretized, the strong coupling can make the resulting algebraic systems difficult to solve, requiring effective preconditioning of iterative methods for efficiency. In this work, we consider a finite element discretization of an exact penalty formulation for the stationary MHD equations posed in two-dimensional domains. This formulation has the benefit of implicitly enforcing the divergence-free condition on the magnetic field without requiring a Lagrange multiplier. We consider extending block preconditioning techniques developed for the Navier-Stokes equations to the full MHD system. We analyze operators arising in block decompositions from a continuous perspective and apply arguments based on the existence of approximate commutators to develop new preconditioners that account for the physical coupling. This results in a family of parameterized block preconditioners for both Picard and Newton linearizations. We develop an automated method for choosing the relevant parameters and demonstrate the robustness of these preconditioners for a range of the physical nondimensional parameters and with respect to mesh refinement.

More Details
TYPE Journal Article YEAR 2014
ScopusOSTI
72 Results
72 Results