Publications

Laros, James H.; Pedretti, Kevin P.; Kelly, Suzanne M.; Vaughan, Courtenay T.

Abstract not provided.

Barrett, Brian B.; Kelly, Suzanne M.; Klundt, Ruth A.; Laros, James H.; Leung, Vitus J.; Levenhagen, Michael J.; Lofstead, Gerald F.; Moreland, Kenneth D.; Oldfield, Ron A.; Pedretti, Kevin P.; Rodrigues, Arun; Barrett, Richard F.; Ward, Harry L.; Vandyke, John P.; Vaughan, Courtenay T.; Wheeler, Kyle B.; Brandt, James M.; Brightwell, Ronald B.; Curry, Matthew L.; Fabian, Nathan D.; Ferreira, Kurt; Gentile, Ann C.; Hemmert, Karl S.

Abstract not provided.

Barrett, Brian B.; Kelly, Suzanne M.; Klundt, Ruth A.; Laros, James H.; Leung, Vitus J.; Levenhagen, Michael J.; Lofstead, Gerald F.; Moreland, Kenneth D.; Oldfield, Ron A.; Pedretti, Kevin P.; Rodrigues, Arun; Barrett, Richard F.; Ward, Harry L.; Vandyke, John P.; Vaughan, Courtenay T.; Wheeler, Kyle B.; Brandt, James M.; Brightwell, Ronald B.; Curry, Matthew L.; Fabian, Nathan D.; Ferreira, Kurt; Gentile, Ann C.; Hemmert, Karl S.

This report documents thirteen of Sandia's contributions to the Computational Systems and Software Environment (CSSE) within the Advanced Simulation and Computing (ASC) program between fiscal years 2009 and 2012. It describes their impact on ASC applications. Most contributions are implemented in lower software levels allowing for application improvement without source code changes. Improvements are identified in such areas as reduced run time, characterizing power usage, and Input/Output (I/O). Other experiments are more forward looking, demonstrating potential bottlenecks using mini-application versions of the legacy codes and simulating their network activity on Exascale-class hardware. The purpose of this report is to prove that the team has completed milestone 4467-Demonstration of a Legacy Application's Path to Exascale. Cielo is expected to be the last capability system on which existing ASC codes can run without significant modifications. This assertion will be tested to determine where the breaking point is for an existing highly scalable application. The goal is to stretch the performance boundaries of the application by applying recent CSSE RD in areas such as resilience, power, I/O, visualization services, SMARTMAP, lightweight LWKs, virtualization, simulation, and feedback loops. Dedicated system time reservations and/or CCC allocations will be used to quantify the impact of system-level changes to extend the life and performance of the ASC code base. Finally, a simulation of anticipated exascale-class hardware will be performed using SST to supplement the calculations. Determine where the breaking point is for an existing highly scalable application: Chapter 15 presented the CSSE work that sought to identify the breaking point in two ASC legacy applications-Charon and CTH. Their mini-app versions were also employed to complete the task. There is no single breaking point as more than one issue was found with the two codes. The results were that applications can expect to encounter performance issues related to the computing environment, system software, and algorithms. Careful profiling of runtime performance will be needed to identify the source of an issue, in strong combination with knowledge of system software and application source code.

Laros, James H.; Pedretti, Kevin P.; Kelly, Suzanne M.; Vaughan, Courtenay T.

Abstract not provided.

IEEE International Symposium on Parallel and Distributed Processing Workshops and Phd Forum

Vaughan, Courtenay T.; Rajan, Mahesh R.; Barrett, Richard F.; Doerfler, Douglas W.; Pedretti, Kevin P.

Cielo, a Cray XE6, is the Department of Energy NNSA Advanced Simulation and Computing (ASC) campaign's newest capability machine. Rated at 1.37 PFLOPS, it consists of 8,944 dual-socket oct-core AMD Magny-Cours compute nodes, linked using Cray's Gemini interconnect. Its primary mission objective is to enable a suite of the ASC applications implemented using MPI to scale to tens of thousands of cores. Cielo is an evolutionary improvement to a successful architecture previously available to many of our codes, thus enabling a basis for understanding the capabilities of this new architecture. Using three codes strategically important to the ASC campaign, and supplemented with some micro-benchmarks that expose the fundamental capabilities of the XE6, we report on the performance characteristics and capabilities of Cielo. © 2011 IEEE.

Barrett, Richard F.; Heroux, Michael A.; Lin, Paul L.; Vaughan, Courtenay T.; Williams, Alan B.

Abstract not provided.

Laros, James H.; Pedretti, Kevin P.; Kelly, Suzanne M.; Vaughan, Courtenay T.

Abstract not provided.

Concurreny and Computation: Practice and Experience

Rajan, Mahesh R.; Vaughan, Courtenay T.; Doerfler, Douglas W.; Barrett, Richard F.; Lin, Paul L.; Pedretti, Kevin P.; Hemmert, Karl S.

Abstract not provided.

Vaughan, Courtenay T.

Abstract not provided.

Barrett, Richard F.; Heroux, Michael A.; Lin, Paul L.; Vaughan, Courtenay T.; Williams, Alan B.

Abstract not provided.

Rajan, Mahesh R.; Vaughan, Courtenay T.; Barrett, Richard F.; Doerfler, Douglas W.; Lin, Paul L.; Pedretti, Kevin P.; Hemmert, Karl S.

Abstract not provided.

Parallel Processing Letters

Barrett, Richard F.; Vaughan, Courtenay T.; Rajan, Mahesh R.; Doerfler, Douglas W.; Pedretti, Kevin P.

Abstract not provided.

Vaughan, Courtenay T.

Abstract not provided.

Vaughan, Courtenay T.

Abstract not provided.

Rajan, Mahesh R.; Vaughan, Courtenay T.; Doerfler, Douglas W.; Lin, Paul L.; Pedretti, Kevin P.; Hemmert, Karl S.

Abstract not provided.

Vaughan, Courtenay T.; Rajan, Mahesh R.; Barrett, Richard F.; Doerfler, Douglas W.; Pedretti, Kevin P.

Abstract not provided.

Doerfler, Douglas W.; Rajan, Mahesh R.; Epperson, Marcus E.; Vaughan, Courtenay T.; Pedretti, Kevin P.; Barrett, Richard F.; Barrett, Brian B.

Abstract not provided.

Pedretti, Kevin P.; Kelly, Suzanne M.; Vaughan, Courtenay T.

Abstract not provided.

Rajan, Mahesh R.; Vaughan, Courtenay T.; Doerfler, Douglas W.; Lin, Paul L.; Pedretti, Kevin P.

Abstract not provided.

Rajan, Mahesh R.; Vaughan, Courtenay T.; Doerfler, Douglas W.; Lin, Paul L.; Pedretti, Kevin P.; Barrett, Richard F.; Hemmert, Karl S.

Abstract not provided.

Vaughan, Courtenay T.; Rajan, Mahesh R.; Barrett, Richard F.; Doerfler, Douglas W.; Pedretti, Kevin P.

Cielo, a Cray XE6, is the Department of Energy NNSA Advanced Simulation and Computing (ASC) campaign's newest capability machine. Rated at 1.37 PFLOPS, it consists of 8,944 dual-socket oct-core AMD Magny-Cours compute nodes, linked using Cray's Gemini interconnect. Its primary mission objective is to enable a suite of the ASC applications implemented using MPI to scale to tens of thousands of cores. Cielo is an evolutionary improvement to a successful architecture previously available to many of our codes, thus enabling a basis for understanding the capabilities of this new architecture. Using three codes strategically important to the ASC campaign, and supplemented with some micro-benchmarks that expose the fundamental capabilities of the XE6, we report on the performance characteristics and capabilities of Cielo.

Vaughan, Courtenay T.; Doerfler, Douglas W.

Abstract not provided.

Vaughan, Courtenay T.; Doerfler, Douglas W.

Abstract not provided.

International Journal of Distributed Systems and Technologies

Rajan, Mahesh; Doerfler, Douglas W.; Vaughan, Courtenay T.; Epperson, Marcus E.; Ogden, Jeff

In a recent acquisition by DOE/NNSA several large capacity computing clusters called TLCC have been installed at the DOE labs: SNL, LANL and LLNL. TLCC architecture with ccNUMA, multi-socket, multi-core nodes, and InfiniBand interconnect, is representative of the trend in HPC architectures. This paper examines application performance on TLCC contrasting them with Red Storm/Cray XT4. TLCC and Red Storm share similar AMD processors and memory DIMMs. Red Storm however has single socket nodes and custom interconnect. Micro-benchmarks and performance analysis tools help understand the causes for the observed performance differences. Control of processor and memory affinity on TLCC with the numactl utility is shown to result in significant performance gains and is essential to attenuate the detrimental impact of OS interference and cache-coherency overhead. While previous studies have investigated impact of affinity control mostly in the context of small SMP systems, the focus of this paper is on highly parallel MPI applications.

Doerfler, Douglas W.; Rajan, Mahesh R.; Vaughan, Courtenay T.

Abstract not provided.