A publication of the Advanced Simulation & Computing Division, NA-121.2, NNSA Defense Programs
June 2009
NA-ASC-500-09—Issue 11
Return to this issue’s stories
LANL Has Golden Opportunity to Prototype New Storage Archive
As archiving requirements for high performance computing (HPC) data grow while resources to maintain specialized archive systems stay the same or decline, a new approach to archiving may be needed. Newly available commercial storage technologies have opened the possibility to prototype an alternative solution to archiving data while minimizing ASC-funded code development and maintenance.
This pilot project uses the General Parallel Filesystem (GPFS) and Tivoli Storage Manager (TSM), both from IBM, with commodity hardware and operating systems. The parallel nature of GPFS and its Information Lifecycle Management feature coupled with the flexibility of the TSM data movement path provide excellent parallelism and scaling opportunities. The filesystem semantics of GPFS and the use of commodity software like MPI, MySQL, and FUSE allow us to address challenges in the solution—parallel copy, tape access efficiency, and huge file migration/recall—with a surprisingly small amount of code written.
This archive solution is being used to support the Open Science runs on the Roadrunner platform. It will eventually be moved to LANL’s Turquoise open collaboration environment. For more information, email Cody Scott at cscott@lanl.gov.
Roadrunner, the fastest supercomputer in the world, at Los Alamos National Laboratory is a cluster of 17 Connected Units. For more information, visit http://www.lanl.gov/roadrunner.
DOE Privacy Disclaimer | Sandia Privacy Disclaimer | 2009-4030 W
Developed and maintained by Sandia National Laboratories for NA-121.2