A publication of the Advanced Simulation & Computing Division, NA-121.2, NNSA Defense Programs
March 2009
NA-ASC-500-09—Issue 10
Return to this issue’s stories
High-End Computing Research Needs Investment in I/O and Storage R&D
A multiagency working group—with representation from ASC—to help manage overall government investments in high-end computing research & development predicts that in the near future, gaps and open problems in the file systems and I/O in high-end computing will be formidable. These challenges present areas in need of new and continued investment in R&D and standardization that the government should pursue.
The High-End Computing Interagency Working Group’s members are from Los Alamos, Argonne, Sandia, Pacific Northwest, and Oak Ridge national laboratories, and NASA. They point out in a paper published in the January issue of the Association for the Computing Machinery Special Interest Group on Operating Systems that sites in the near future will routinely deploy supercomputers with hundreds of thousands of processors. They anticipate that storage bandwidth requirements will go from tens of gigabytes per second to terabytes per second.
For efficient complex science, online storage requirements to support workflows will approach the exabyte range. It will require the ability to handle a more varied I/O workload ranging seven orders of magnitude in performance characteristics, extremely high metadata activities, and management of trillions of files. It will be necessary to share data using global or virtual enterprise wide-area networks with flexible and effective security. The number of storage devices needed in a single coordinated operation could be in the tens to hundreds of thousands. It will be increasingly difficult to manage enterprise-class global parallel file/storage systems due to the number of elements involved, which may approach 100,000 spinning disks with widely varying workloads.
The paper acts as a preview to the National Science Foundation’s 2009 High End Computing University Research call. To see the paper “Coordinating Government Funding of File system and I/O Research through the High-End Computing University Research Activity” go to http://institute.lanl.gov/hec-fsio/docs/HEC-FSIO-FY08-Gaps_RoadMap.pdf.
DOE Privacy Disclaimer | Sandia Privacy Disclaimer |
Developed and maintained by Sandia National Laboratories for NA-121.2