Long-awaited Red Storm supercomputer rising at Sandia
The world’s fastest supercomputer — Sandia’s Red Storm — should be one-quarter assembled by the end of September and completely up and running by January, says Director Bill Camp (9200), who heads the effort to design and assemble the machine.
Asked precisely how fast the new supercomputer would go, Bill said: "Really fast."
Early prototypes are already running Sandia and Cray software to test functionality and performance.
The $90 million, innovatively designed machine is expected to commence performance at 41.5 teraflops and in a year reach 100 teraflops.
The upward speed projection is not pie-in-the-sky but mathematically predictable. Next June, each processor will be replaced with two processors, each running 25 percent faster.
"Do the math," Bill says.
DOE requires for its investment that the new machine run seven times as fast as ASCI Red. "We think Red Storm will run ten times as fast," says Bill.
Unusually rapid assembly
Red Storm from concept to assembly also should be completed in record time.
"Cray historically has required four to seven years from concept to first product on a new supercomputer," says Bill. "Our goal was to field a brand-new supercomputer in 23 months. We’re three months behind, but we’ll still be under 30 months total when completed."
Not only is the extremely fast machine to be very rapidly assembled, it is relatively inexpensive by supercomputer standards.
Less expensive and why
Japan’s Earth Simulator, currently the world’s fastest supercomputer, cost $400 million to build. It also uses eight megawatts of power compared to Red Storm’s projected two megawatts and takes up approximately three times the space.
"Historically, Sandia has received less money than the other two [defense] labs to purchase supercomputers because in the past it wasn’t
considered a Sandia line of business," says Bill. "To keep at the leading edge, we haven’t been able to just go out and ask vendors for prices; instead we work with them to do more for less. Red Storm is a push-the-envelope machine we — Jim Tomkins (9220) and I — helped develop to meet our unique specifications."
The machine is slated to do work for the stockpile: design new components; virtually test components under hostile, abnormal, and normal conditions; and help in weapons engineering and weapons physics.
Cray was chosen because the company was "forward-looking, flexible, willing to work with us to design a new architecture, and had the lowest cost proposal."
The commercial future
The machine, because of its uniquely inexpensive design, may become the center of Cray’s future supercomputer line, says Bill. "From Cray’s point of view, the approach we’re pioneering here is so powerful they want their next supercomputers to follow suit."
A large part of the costs incurred for the machine are non-recurring engineering design costs that Cray will not face in the future.
"Cray is used to building ‘Rolls Royces’ — entirely custom-designed, water cooled, with plumbing all over it," Bill says. "We couldn’t afford that. Additionally, it’d be a nightmare for us to maintain because you’d have to disconnect the plumbing to repair a board. The way Red Storm is designed, we don’t have to shut down to replace a part. We just don’t do computations that involve that board until we decide to pull the board and fix it — all without shutting down."
The machine itself — a few facts
The machine has 96 processors in each computer cabinet, with four processors to a board. Each processor can have up to eight gigabytes of memory sitting next to it. Four Cray SeaStars — powerful networking chips — sit on a daughter board atop each board. All SeaStars talk to each other, "like a Rubik’s cube with lots of squares on each face," says Bill. "Cray SeaStars are about a factor of 10 faster than any current competing capability."
Messages encoded in MPI (the Message Passage Interface standard) move from processor to processor at a sustained speed of five gigabytes per second bidirectionally. The amount of time to get the first information bit from one processor to another is less than five microseconds across the system. Four rows of machines, with approximately 11,600 Opteron processors and a similar number of SeaStars, will be the arrangement of the major components of the machine.
The SeaStar chip includes an 800 MHz DDR Hypertransport interface to its Opteron processors, a PowerPC core for handling message-passing chores, and a six-port router. SeaStars are linked together to make up the system’s 3-D (X-Y-Z axis) mesh interconnect.
IBM will fabricate the SeaStar chips using 0.13-micron CMOS technology.
Visualization will occur inside the computer itself — a capability unique to Red Storm among supercomputers.
A proud moment
"One of the proudest achievements in my life was to help make supercomputing a line of business at Sandia," says Bill. "Look at where we are today. We’ve revolutionized the way we do business. I would claim that more than any other technology, computing affects everything we do at the Labs."