Publications

Bertagna, Luca B.; Guba, Oksana G.; Bradley, Andrew M.; Salinger, Andrew G.; Taylor, Mark A.; Foucar, James G.

Abstract not provided.

International Conference for High Performance Computing, Networking, Storage and Analysis, SC

Bertagna, Luca B.; Guba, Oksana G.; Taylor, Mark A.; Foucar, James G.; Larkin, Jeff; Bradley, Andrew M.; Rajamanickam, Sivasankaran R.; Salinger, Andrew G.

We present an effort to port the nonhydrostatic atmosphere dynamical core of the Energy Exascale Earth System Model (E3SM) to efficiently run on a variety of architectures, including conventional CPU, many-core CPU, and GPU. We specifically target cloud-resolving resolutions of 3 km and 1 km. To express on-node parallelism we use the C++ library Kokkos, which allows us to achieve a performance portable code in a largely architecture-independent way. Our C++ implementation is at least as fast as the original Fortran implementation on IBM Power9 and Intel Knights Landing processors, proving that the code refactor did not compromise the efficiency on CPU architectures. On the other hand, when using the GPUs, our implementation is able to achieve 0.97 Simulated Years Per Day, running on the full Summit supercomputer. To the best of our knowledge, this is the most achieved to date by any global atmosphere dynamical core running at such resolutions.

Bertagna, Luca B.; Guba, Oksana G.; Foucar, James G.; Taylor, Mark A.; Bradley, Andrew M.; Salinger, Andrew G.

Abstract not provided.

Bertagna, Luca B.; Guba, Oksana G.; Taylor, Mark A.; Foucar, James G.; Larkin, Jeff L.; Bradley, Andrew M.; Rajamanickam, Sivasankaran R.; Salinger, Andrew G.

Abstract not provided.

Hillman, Benjamin H.; Caldwell, Peter C.; Salinger, Andrew G.; Bertagna, Luca B.; Beydoun, Hassan B.; Peter, Bogenschutz.P.; Bradley, Andrew M.; Donahue, Aaron D.; Eldred, Christopher; Foucar, James G.; Golaz, Chris G.; Guba, Oksana G.; Jacob, Robert J.; Johnson, Jeff J.; Keen, Noel K.; Krishna, Jayesh K.; Lin, Wuyin L.; Liu, Weiran L.; Pressel, Kyle P.; Singh, Balwinder S.; Steyer, Andrew S.; Taylor, Mark A.; Terai, Chris T.; Ullrich, Paul A.; Wu, Danqing W.; Yuan, Xingqui Y.

Abstract not provided.

Foucar, James G.

The version of STK (Sierra ToolKit) that has long been provided with Trilinos is no longer supported by the core develop- ment team. With the introduction of a the new STK library into Trilinos, the old STK has been renamed to stk classic. This document contains a rough guide of how to port a stk classic code to STK.

Noble, David R.; Sjaardema, Gregory D.; Clausen, Jonathan C.; Thomas, Jesse D.; Foucar, James G.; Franko, Kenneth J.; Heinstein, Martin W.; Williams, Alan B.

Abstract not provided.

Hensinger, David M.; Drake, Richard R.; Foucar, James G.

Generating finite-element meshes is a serious bottleneck for large parallel simulations. When mesh generation is limited to serial machines and element counts approach a billion, this bottleneck becomes a roadblock. Pamgen is a parallel mesh generation library that allows on-the-fly scalable generation of hexahedral and quadrilateral finite element meshes for several simple geometries. It has been used to generate more that 1.1 billion elements on 17,576 processors. Pamgen generates an unstructured finite element mesh on each processor at the start of a simulation. The mesh is specified by commands passed to the library as a 'C'-programming language string. The resulting mesh geometry, topology, and communication information can then be queried through an API. pamgen allows specification of boundary condition application regions using sidesets (element faces) and nodesets (collections of nodes). It supports several simple geometry types. It has multiple alternatives for mesh grading. It has several alternatives for the initial domain decomposition. Pamgen makes it easy to change details of the finite element mesh and is very useful for performance studies and scoping calculations.