Publications

Moore, Stan G.

This report investigates free expansion of Aluminum and provides a take home message of "The physically realistic SNAP machine-learning potential captures liquid-vapor coexistence behavior for free expansion of aluminum at a level not generally accessible to hydrocodes".

Moore, Stan G.

The experiment investigates free expansion of a supercritical fluid into a two-phase liquid-vapor coexistence region. A huge molecular dynamics simulation (6 billion Lennard-Jones atoms) was run on 5760 GPUs (33% of LLNL Sierra) using LAMMPS/Kokkos software. This improved visualization workflow and started preliminary simulations of aluminum using SNAP machine learning potential.

Computer Physics Communications

Thompson, Aidan P.; Aktulga, H.M.; Berger, Richard; Bolintineanu, Dan S.; Brown, W.M.; Crozier, Paul C.; in 't Veld, Pieter J.; Kohlmeyer, Axel; Moore, Stan G.; Nguyen, Trung D.; Shan, Ray; Stevens, Mark J.; Tranchida, Julien; Trott, Christian R.; Plimpton, Steven J.

Since the classical molecular dynamics simulator LAMMPS was released as an open source code in 2004, it has become a widely-used tool for particle-based modeling of materials at length scales ranging from atomic to mesoscale to continuum. Reasons for its popularity are that it provides a wide variety of particle interaction models for different materials, that it runs on any platform from a single CPU core to the largest supercomputers with accelerators, and that it gives users control over simulation details, either via the input script or by adding code for new interatomic potentials, constraints, diagnostics, or other features needed for their models. As a result, hundreds of people have contributed new capabilities to LAMMPS and it has grown from fifty thousand lines of code in 2004 to a million lines today. In this paper several of the fundamental algorithms used in LAMMPS are described along with the design strategies which have made it flexible for both users and developers. We also highlight some capabilities recently added to the code which were enabled by this flexibility, including dynamic load balancing, on-the-fly visualization, magnetic spin dynamics models, and quantum-accuracy machine learning interatomic potentials. Program Summary: Program Title: Large-scale Atomic/Molecular Massively Parallel Simulator (LAMMPS) CPC Library link to program files: https://doi.org/10.17632/cxbxs9btsv.1 Developer's repository link: https://github.com/lammps/lammps Licensing provisions: GPLv2 Programming language: C++, Python, C, Fortran Supplementary material: https://www.lammps.org Nature of problem: Many science applications in physics, chemistry, materials science, and related fields require parallel, scalable, and efficient generation of long, stable classical particle dynamics trajectories. Within this common problem definition, there lies a great diversity of use cases, distinguished by different particle interaction models, external constraints, as well as timescales and lengthscales ranging from atomic to mesoscale to macroscopic. Solution method: The LAMMPS code uses parallel spatial decomposition, distributed neighbor lists, and parallel FFTs for long-range Coulombic interactions [1]. The time integration algorithm is based on the Størmer-Verlet symplectic integrator [2], which provides better stability than higher-order non-symplectic methods. In addition, LAMMPS supports a wide range of interatomic potentials, constraints, diagnostics, software interfaces, and pre- and post-processing features. Additional comments including restrictions and unusual features: This paper serves as the definitive reference for the LAMMPS code. References: [1] S. Plimpton, Fast parallel algorithms for short-range molecular dynamics. J. Comp. Phys. 117 (1995) 1–19. [2] L. Verlet, Computer experiments on classical fluids: I. Thermodynamical properties of Lennard–Jones molecules, Phys. Rev. 159 (1967) 98–103.

Moore, Stan G.; Thompson, Aidan P.

Abstract not provided.

Aaziz, Omar R.; Allan, Benjamin A.; Brandt, James M.; Cook, Jeanine C.; Devine, Karen D.; Elliott, James E.; Gentile, Ann C.; Hammond, Simon D.; Kelley, Brian M.; Lopatina, Lena L.; Moore, Stan G.; Olivier, Stephen L.; Pedretti, Kevin P.; Poliakoff, David Z.; Pawlowski, Roger P.; Regier, Phillip A.; Schmitz, Mark E.; Schwaller, Benjamin S.; Surjadidjaja, Vanessa S.; Swan, Matthew S.; Tucker, Nick T.; Tucker, Tom T.; Vaughan, Courtenay T.; Walton, Sara P.

Scientific applications run on high-performance computing (HPC) systems are critical for many national security missions within Sandia and the NNSA complex. However, these applications often face performance degradation and even failures that are challenging to diagnose. To provide unprecedented insight into these issues, the HPC Development, HPC Systems, Computational Science, and Plasma Theory & Simulation departments at Sandia crafted and completed their FY21 ASC Level 2 milestone entitled "Integrated System and Application Continuous Performance Monitoring and Analysis Capability." The milestone created a novel integrated HPC system and application monitoring and analysis capability by extending Sandia's Kokkos application portability framework, Lightweight Distributed Metric Service (LDMS) monitoring tool, and scalable storage, analysis, and visualization pipeline. The extensions to Kokkos and LDMS enable collection and storage of application data during run time, as it is generated, with negligible overhead. This data is combined with HPC system data within the extended analysis pipeline to present relevant visualizations of derived system and application metrics that can be viewed at run time or post run. This new capability was evaluated using several week-long, 290-node runs of Sandia's ElectroMagnetic Plasma In Realistic Environments ( EMPIRE ) modeling and design tool and resulted in 1TB of application data and 50TB of system data. EMPIRE developers remarked this capability was incredibly helpful for quickly assessing application health and performance alongside system state. In short, this milestone work built the foundation for expansive HPC system and application data collection, storage, analysis, visualization, and feedback framework that will increase total scientific output of Sandia's HPC users.

Brandt, James M.; Cook, Jeanine C.; Aaziz, Omar R.; Allan, Benjamin A.; Devine, Karen D.; Elliott, James J.; Gentile, Ann C.; Hammond, Simon D.; Kelley, Brian M.; Lopatina, Lena L.; Moore, Stan G.; Olivier, Stephen L.; Pedretti, Kevin P.; Poliakoff, David Z.; Pawlowski, Roger P.; Regier, Phillip A.; Schmitz, Mark E.; Schwaller, Benjamin S.; Surjadidjaja, Vanessa S.; Swan, Matthew S.; Tucker, Tom T.; Tucker, Nick T.; Vaughan, Courtenay T.; Walton, Sara P.

Abstract not provided.

Moore, Stan G.

Abstract not provided.

Moore, Stan G.

Abstract not provided.

Moore, Stan G.; Plimpton, Steven J.; Gallis, Michail A.

Abstract not provided.

Moore, Stan G.

Abstract not provided.

Moore, Stan G.

Abstract not provided.

Moore, Stan G.; Thompson, Aidan P.

Atomistic simulations can capture physics of free expansion into two-phase region.

Communications in Computational Physics

Bettencourt, Matthew T.; Brown, Dominic A.S.; Cartwright, Keith L.; Cyr, Eric C.; Glusa, Christian A.; Lin, Paul T.; Moore, Stan G.; McGregor, Duncan A.O.; Pawlowski, Roger P.; Phillips, Edward G.; Roberts, Nathan V.; Wright, Steven A.; Maheswaran, Satheesh; Jones, John P.; Jarvis, Stephen A.

In this paper we introduce EMPIRE-PIC, a finite element method particle-in-cell (FEM-PIC) application developed at Sandia National Laboratories. The code has been developed in C++ using the Trilinos library and the Kokkos Performance Portability Framework to enable running on multiple modern compute architectures while only requiring maintenance of a single codebase. EMPIRE-PIC is capable of solving both electrostatic and electromagnetic problems in two- and three-dimensions to second-order accuracy in space and time. In this paper we validate the code against three benchmark problems - a simple electron orbit, an electrostatic Langmuir wave, and a transverse electromagnetic wave propagating through a plasma. We demonstrate the performance of EMPIRE-PIC on four different architectures: Intel Haswell CPUs, Intel's Xeon Phi Knights Landing, ARM Thunder-X2 CPUs, and NVIDIA Tesla V100 GPUs attached to IBM POWER9 processors. This analysis demonstrates scalability of the code up to more than two thousand GPUs, and greater than one hundred thousand CPUs.

Moore, Stan G.

Abstract not provided.

Moore, Stan G.; Thompson, Aidan P.

Abstract not provided.

Weinberg, Evan W.; Moore, Stan G.

Abstract not provided.

Hu, Jonathan J.; Glusa, Christian A.; Moore, Stan G.; Lin, Paul T.; Phillips, Edward G.; Bettencourt, Matthew T.; Elliott, James J.; Siefert, Christopher S.; Rajamanickam, Sivasankaran R.

Abstract not provided.

Thompson, Aidan P.; Moore, Stan G.; Gayatri, Rahul G.

Abstract not provided.

Moore, Stan G.; Thompson, Aidan P.; Gayatri, Rahul G.

Abstract not provided.

Moore, Stan G.; Stagg, Alan K.

Abstract not provided.

Moore, Stan G.

Abstract not provided.

AIP Conference Proceedings

Gallis, Michail A.; Torczynski, J.R.; Bitter, Neal B.; Koehler, Timothy P.; Moore, Stan G.; Plimpton, Steven J.; Papadakis, G.

The Direct Simulation Monte Carlo (DSMC) method has been used for more than 50 years to simulate rarefied gases. The advent of modern supercomputers has brought higher-density near-continuum flows within range. This in turn has revived the debate as to whether the Boltzmann equation, which assumes molecular chaos, can be used to simulate continuum flows when they become turbulent. In an effort to settle this debate, two canonical turbulent flows are examined, and the results are compared to available continuum theoretical and numerical results for the Navier-Stokes equations.

Moore, Stan G.

Abstract not provided.

Physics of Fluids

Plimpton, Steven J.; Moore, Stan G.; Borner, A.; Stagg, Alan K.; Koehler, T.P.; Torczynski, J.R.; Gallis, Michail A.

The gold-standard definition of the Direct Simulation Monte Carlo (DSMC) method is given in the 1994 book by Bird [Molecular Gas Dynamics and the Direct Simulation of Gas Flows (Clarendon Press, Oxford, UK, 1994)], which refined his pioneering earlier papers in which he first formulated the method. In the intervening 25 years, DSMC has become the method of choice for modeling rarefied gas dynamics in a variety of scenarios. The chief barrier to applying DSMC to more dense or even continuum flows is its computational expense compared to continuum computational fluid dynamics methods. The dramatic (nearly billion-fold) increase in speed of the largest supercomputers over the last 30 years has thus been a key enabling factor in using DSMC to model a richer variety of flows, due to the method's inherent parallelism. We have developed the open-source SPARTA DSMC code with the goal of running DSMC efficiently on the largest machines, both current and future. It is largely an implementation of Bird's 1994 formulation. Here, we describe algorithms used in SPARTA to enable DSMC to operate in parallel at the scale of many billions of particles or grid cells, or with billions of surface elements. We give a few examples of the kinds of fundamental physics questions and engineering applications that DSMC can address at these scales.

Moore, Stan G.

Abstract not provided.

Moore, Stan G.; Stagg, Alan K.

Abstract not provided.

Moore, Stan G.

Abstract not provided.

Moore, Stan G.; Gallis, Michail A.

Abstract not provided.

Tranchida, Julien G.; Wood, Mitchell A.; Moore, Stan G.

This report summarizes the result of the LDRD Exploratory Express project 211666-01, titled "Coupled Magnetic Spin Dynamics and Molecular Dynamics in a Massively Parallel Framework" .

Molecular Physics

Mattox, Timothy I.; Larentzos, James P.; Moore, Stan G.; Stone, Christopher P.; Ibanez, Daniel A.; Thompson, Aidan P.; Lísal, Martin; Brennan, John K.; Plimpton, Steven J.

Simulating energetic materials with complex microstructure is a grand challenge, where until recently, an inherent gap in computational capabilities had existed in modelling grain-scale effects at the microscale. We have enabled a critical capability in modelling the multiscale nature of the energy release and propagation mechanisms in advanced energetic materials by implementing, in the widely used LAMMPS molecular dynamics (MD) package, several novel coarse-graining techniques that also treat chemical reactivity. Our innovative algorithmic developments rooted within the dissipative particle dynamics framework, along with performance optimisations and application of acceleration technologies, have enabled extensions in both the length and time scales far beyond those ever realised by atomistic reactive MD simulations. In this paper, we demonstrate these advances by modelling a shockwave propagating through a microstructured material and comparing performance with the state-of-the-art in atomistic reactive MD techniques. As a result of this work, unparalleled explorations in energetic materials research are now possible.

Tranchida, Julien G.; Wood, Mitchell A.; Moore, Stan G.; Plimpton, Steven J.; Thompson, Aidan P.; Thibaudeau, Pascal T.

Abstract not provided.

Moore, Stan G.

Abstract not provided.

Moore, Stan G.

Abstract not provided.

Moore, Stan G.; Fuller, Timothy J.

Abstract not provided.

Moore, Stan G.; Gallis, Michail A.; Plimpton, Steven J.

Abstract not provided.

Hemmert, Karl S.; Moore, Stan G.; Gallis, Michail A.; Davis, Mike D.; Levesque, John L.; Hjelm, Nathan T.; Lujan, James L.; Morton, David P.; Nam, Hai A.; Parga, Alex P.; Jr., Paul P.; Shipman, Galen S.; Torrez, Alfred T.

Abstract not provided.

Hemmert, Karl S.; Moore, Stan G.; Gallis, Michail A.; Davis, Mike D.; Levesque, John L.; Hjelm, Nathan T.; Lujan, James L.; Morton, David P.; Nam, Hai A.; Parga, Alex P.; Jr., Paul P.; Shipman, Galen S.; Torrez, Alfred T.

Abstract not provided.

Moore, Stan G.

Abstract not provided.

Gallis, Michail A.; Moore, Stan G.; Plimpton, Steven J.

Abstract not provided.

Physical Review E

Koski, Jason K.; Moore, Stan G.; Grest, Gary S.; Stevens, Mark J.

The role of an external field on capillary waves at the liquid-vapor interface of a dipolar fluid is investigated using molecular dynamics simulations. For fields parallel to the interface, the interfacial width squared increases linearly with respect to the logarithm of the size of the interface across all field strengths tested. The value of the slope decreases with increasing field strength, indicating that the field dampens the capillary waves. With the inclusion of the parallel field, the surface stiffness increases with increasing field strength faster than the surface tension. For fields perpendicular to the interface, the interfacial width squared is linear with respect to the logarithm of the size of the interface for small field strengths, and the surface stiffness is less than the surface tension. Above a critical field strength that decreases as the size of the interface increases, the interface becomes unstable due to the increased amplitude of the capillary waves.

Moore, Stan G.; Ibanez-Granados, Daniel A.

Abstract not provided.

Moore, Stan G.; Thompson, Aidan P.; Wood, Mitchell W.

LAMMPS is a classical molecular dynamics code (lammps.sandia.gov) used to model materials science problems at Sandia National Laboratories and around the world. LAMMPS was one of three Sandia codes selected to participate in the Trinity KNL (TR2) Open Science period. During this period, three different problems of interest were investigated using LAMMPS. The first was benchmarking KNL performance using different force field models. The second was simulating void collapse in shocked HNS energetic material using an all-atom model. The third was simulating shock propagation through poly-crystalline RDX energetic material using a coarse-grain model, the results of which were used in an ACM Gordon Bell Prize submission. This report describes the results of these simulations, lessons learned, and some hardware issues found on Trinity KNL as part of this work.

Moore, Stan G.

Abstract not provided.

Plimpton, Steven J.; Moore, Stan G.; Trott, Christian R.

Abstract not provided.

Abbott, Lauren J.; Moore, Stan G.

Abstract not provided.

Fierro, Andrew S.; Moore, Christopher H.; Moore, Stan G.; Biedermann, Laura B.; Hopkins, Matthew M.

Abstract not provided.

Moore, Stan G.

Abstract not provided.

Moore, Stan G.

Abstract not provided.

Moore, Stan G.

Abstract not provided.

Biedermann, Laura B.; Chow, Weng W.; Moore, Christopher H.; Moore, Stan G.; Fierro, Andrew S.; Hopkins, Matthew M.; Elizondo-Decanini, Juan M.

Abstract not provided.

Moore, Stan G.

Abstract not provided.

Trott, Christian R.; Hammond, Simon D.; Moore, Stan G.; Shan, Tzu-Ray S.

Abstract not provided.

Costa, Timothy C.; Bond, Stephen D.; Littlewood, David J.; Moore, Stan G.

The problem of computing quantum-accurate design-scale solutions to mechanics problems is rich with applications and serves as the background to modern multiscale science research. The prob- lem can be broken into component problems comprised of communicating across adjacent scales, which when strung together create a pipeline for information to travel from quantum scales to design scales. Traditionally, this involves connections between a) quantum electronic structure calculations and molecular dynamics and between b) molecular dynamics and local partial differ- ential equation models at the design scale. The second step, b), is particularly challenging since the appropriate scales of molecular dynamic and local partial differential equation models do not overlap. The peridynamic model for continuum mechanics provides an advantage in this endeavor, as the basic equations of peridynamics are valid at a wide range of scales limiting from the classical partial differential equation models valid at the design scale to the scale of molecular dynamics. In this work we focus on the development of multiscale finite element methods for the peridynamic model, in an effort to create a mathematically consistent channel for microscale information to travel from the upper limits of the molecular dynamics scale to the design scale. In particular, we first develop a Nonlocal Multiscale Finite Element Method which solves the peridynamic model at multiple scales to include microscale information at the coarse-scale. We then consider a method that solves a fine-scale peridynamic model to build element-support basis functions for a coarse- scale local partial differential equation model, called the Mixed Locality Multiscale Finite Element Method. Given decades of research and development into finite element codes for the local partial differential equation models of continuum mechanics there is a strong desire to couple local and nonlocal models to leverage the speed and state of the art of local models with the flexibility and accuracy of the nonlocal peridynamic model. In the mixed locality method this coupling occurs across scales, so that the nonlocal model can be used to communicate material heterogeneity at scales inappropriate to local partial differential equation models. Additionally, the computational burden of the weak form of the peridynamic model is reduced dramatically by only requiring that the model be solved on local patches of the simulation domain which may be computed in parallel, taking advantage of the heterogeneous nature of next generation computing platforms. Addition- ally, we present a novel Galerkin framework, the 'Ambulant Galerkin Method', which represents a first step towards a unified mathematical analysis of local and nonlocal multiscale finite element methods, and whose future extension will allow the analysis of multiscale finite element methods that mix models across scales under certain assumptions of the consistency of those models.

Trott, Christian R.; Shan, Tzu-Ray S.; Moore, Stan G.; Thompson, Aidan P.; Plimpton, Steven J.

Abstract not provided.

Trott, Christian R.; Thompson, Aidan P.; Plimpton, Steven J.; Moore, Stan G.; Shan, Tzu-Ray S.

Abstract not provided.

Moore, Stan G.; Moore, Christopher H.; Boerner, Jeremiah J.

Abstract not provided.

Costa, Timothy C.; Bond, Stephen D.; Littlewood, David J.; Moore, Stan G.

Abstract not provided.

Moore, Christopher H.; Moore, Stan G.; Hopkins, Matthew M.; Boerner, Jeremiah J.; Cartwright, Keith C.; Biedermann, Laura B.

Abstract not provided.

Moore, Christopher H.; Hopkins, Matthew M.; Moore, Stan G.; Boerner, Jeremiah J.; Cartwright, Keith C.

Abstract not provided.

Moore, Stan G.; Shan, Tzu-Ray S.

Abstract not provided.

Moore, Christopher H.; Hopkins, Matthew M.; Moore, Stan G.; Boerner, Jeremiah J.; Cartwright, Keith C.

Abstract not provided.

Hooper, Russell H.; Moore, Stan G.

Aleph models continuum electrostatic and steady and transient thermal fields using a finite-element method. Much work has gone into expanding the core solver capability to support enriched modeling consisting of multiple interacting fields, special boundary conditions and two-way interfacial coupling with particles modeled using Aleph's complementary particle-in-cell capability. This report provides quantitative evidence for correct implementation of Aleph's field solver via order- of-convergence assessments on a collection of problems of increasing complexity. It is intended to provide Aleph with a pedigree and to establish a basis for confidence in results for more challenging problems important to Sandia's mission that Aleph was specifically designed to address.

Moore, Stan G.; Moore, Christopher H.; Boerner, Jeremiah J.

Abstract not provided.

Thompson, Aidan P.; Schultz, Peter A.; Crozier, Paul C.; Moore, Stan G.; Swiler, Laura P.; Stephens, John A.; Trott, Christian R.; Foiles, Stephen M.; Tucker, Garritt J.

This report summarizes the result of LDRD project 12-0395, titled "Automated Algorithms for Quantum-level Accuracy in Atomistic Simulations." During the course of this LDRD, we have developed an interatomic potential for solids and liquids called Spectral Neighbor Analysis Poten- tial (SNAP). The SNAP potential has a very general form and uses machine-learning techniques to reproduce the energies, forces, and stress tensors of a large set of small configurations of atoms, which are obtained using high-accuracy quantum electronic structure (QM) calculations. The local environment of each atom is characterized by a set of bispectrum components of the local neighbor density projected on to a basis of hyperspherical harmonics in four dimensions. The SNAP coef- ficients are determined using weighted least-squares linear regression against the full QM training set. This allows the SNAP potential to be fit in a robust, automated manner to large QM data sets using many bispectrum components. The calculation of the bispectrum components and the SNAP potential are implemented in the LAMMPS parallel molecular dynamics code. Global optimization methods in the DAKOTA software package are used to seek out good choices of hyperparameters that define the overall structure of the SNAP potential. FitSnap.py, a Python-based software pack- age interfacing to both LAMMPS and DAKOTA is used to formulate the linear regression problem, solve it, and analyze the accuracy of the resultant SNAP potential. We describe a SNAP potential for tantalum that accurately reproduces a variety of solid and liquid properties. Most significantly, in contrast to existing tantalum potentials, SNAP correctly predicts the Peierls barrier for screw dislocation motion. We also present results from SNAP potentials generated for indium phosphide (InP) and silica (SiO 2 ). We describe efficient algorithms for calculating SNAP forces and energies in molecular dynamics simulations using massively parallel computers and advanced processor ar- chitectures. Finally, we briefly describe the MSM method for efficient calculation of electrostatic interactions on massively parallel computers.

Moore, Stan G.

Abstract not provided.

Hopkins, Matthew M.; Boerner, Jeremiah J.; Moore, Stan G.; Crozier, Paul C.; Musson, Lawrence M.

Abstract not provided.

Moore, Stan G.

Abstract not provided.

The Journal of Chemical Physics

Moore, Stan G.; Crozier, Paul C.

Abstract not provided.

Moore, Christopher H.; Hopkins, Matthew M.; Boerner, Jeremiah J.; Crozier, Paul C.; Moore, Stan G.

Abstract not provided.

Hopkins, Matthew M.; Boerner, Jeremiah J.; Moore, Christopher H.; Crozier, Paul C.; Moore, Stan G.; Bettencourt, Matthew T.; Hooper, Russell H.

Abstract not provided.

Moore, Stan G.; Crozier, Paul C.; Moore, Christopher H.; Bettencourt, Matthew T.; Hopkins, Matthew M.

Abstract not provided.

Moore, Stan G.; Crozier, Paul C.; Moore, Christopher H.; Bettencourt, Matthew T.; Hopkins, Matthew M.

Abstract not provided.

Hopkins, Matthew M.; Hooper, Russell H.; Boerner, Jeremiah J.; Moore, Christopher H.; Barnat, Edward V.; Crozier, Paul C.; Moore, Stan G.; Bettencourt, Matthew T.; Musson, Lawrence M.

Abstract not provided.