[About Sandia]
[Unique Solutions]
[Working With Us]
[Contacting Us]
[News Center]
[navigation panel]

[Sandia Lab News]

Vol. 56, No. 15           July 23, 2004
[Sandia National Laboratories]

Albuquerque, New Mexico 87185-0165    ||   Livermore, California 94550-0969
Tonopah, Nevada; Nevada Test Site; Amarillo, Texas

Back to the Lab News home page

Long-awaited Red Storm supercomputer rising at Sandia Legislation takes aim at safe, secure, sustainable water supplies through technology, national policy support Sandia research wins two R&D 100 Awards Sandia 'Working on the railroad'

Long-awaited Red Storm supercomputer rising at Sandia

Back to topBack to Lab News home page.

By Neal Singer

The world's fastest supercomputer -- Sandia's Red Storm -- should be one-quarter assembled by the end of September and completely up and running by January, says Director Bill Camp (9200), who heads the effort to design and assemble the machine.

Asked precisely how fast the new supercomputer would go, Bill said: "Really fast."

Early prototypes are already running Sandia and Cray software to test functionality and performance.

The $90 million, innovatively designed machine is expected to commence performance at 41.5 teraflops and in a year reach 100 teraflops.

The upward speed projection is not pie-in-the-sky but mathematically predictable. Next June, each processor will be replaced with two processors, each running 25 percent faster.

"Do the math," Bill says.

DOE requires for its investment that the new machine run seven times as fast as ASCI Red. "We think Red Storm will run ten times as fast," says Bill.

Unusually rapid assembly

Red Storm from concept to assembly also should be completed in record time.

"Cray historically has required four to seven years from concept to first product on a new supercomputer," says Bill. "Our goal was to field a brand-new supercomputer in 23 months. We're three months behind, but we'll still be under 30 months total when completed."

Not only is the extremely fast machine to be very rapidly assembled, it is relatively inexpensive by supercomputer standards.

Less expensive and why

Japan's Earth Simulator, currently the world's fastest supercomputer, cost $400 million to build. It also uses eight megawatts of power compared to Red Storm's projected two megawatts and takes up approximately three times the space.

"Historically, Sandia has received less money than the other two [defense] labs to purchase supercomputers because in the past it wasn't

considered a Sandia line of business," says Bill. "To keep at the leading edge, we haven't been able to just go out and ask vendors for prices; instead we work with them to do more for less. Red Storm is a push-the-envelope machine we -- Jim Tomkins (9220) and I -- helped develop to meet our unique specifications."

The machine is slated to do work for the stockpile: design new components; virtually test components under hostile, abnormal, and normal conditions; and help in weapons engineering and weapons physics.

Cray was chosen because the company was "forward-looking, flexible, willing to work with us to design a new architecture, and had the lowest cost proposal."

The commercial future

The machine, because of its uniquely inexpensive design, may become the center of Cray's future supercomputer line, says Bill. "From Cray's point of view, the approach we're pioneering here is so powerful they want their next supercomputers to follow suit."

A large part of the costs incurred for the machine are non-recurring engineering design costs that Cray will not face in the future.

"Cray is used to building 'Rolls Royces' -- entirely custom-designed, water cooled, with plumbing all over it," Bill says. "We couldn't afford that. Additionally, it'd be a nightmare for us to maintain because you'd have to disconnect the plumbing to repair a board. The way Red Storm is designed, we don't have to shut down to replace a part. We just don't do computations that involve that board until we decide to pull the board and fix it -- all without shutting down."

The machine itself -- a few facts

The machine has 96 processors in each computer cabinet, with four processors to a board. Each processor can have up to eight gigabytes of memory sitting next to it. Four Cray SeaStars -- powerful networking chips -- sit on a daughter board atop each board. All SeaStars talk to each other, "like a Rubik's cube with lots of squares on each face," says Bill. "Cray SeaStars are about a factor of 10 faster than any current competing capability."

Messages encoded in MPI (the Message Passage Interface standard) move from processor to processor at a sustained speed of five gigabytes per second bidirectionally. The amount of time to get the first information bit from one processor to another is less than five microseconds across the system. Four rows of machines, with approximately 11,600 Opteron processors and a similar number of SeaStars, will be the arrangement of the major components of the machine.

The SeaStar chip includes an 800 MHz DDR Hypertransport interface to its Opteron processors, a PowerPC core for handling message-passing chores, and a six-port router. SeaStars are linked together to make up the system's 3-D

(X-Y-Z axis) mesh interconnect.

IBM will fabricate the SeaStar chips using 0.13-micron CMOS technology.

Visualization will occur inside the computer itself -- a capability unique to Red Storm among supercomputers.

A proud moment

"One of the proudest achievements in my life was to help make supercomputing a line of business at Sandia," says Bill. "Look at where we are today. We've revolutionized the way we do business. I would claim that more than any other technology, computing affects everything we do at the Labs." -- Neal Singer

Back to topBack to Lab News home page.

Legislation takes aim at safe, secure, sustainable water supplies through technology, national policy support

Back to topBack to Lab News home page.

By Will Keener

Elected officials in both parties and from several states, including New Mexico and California, lined up behind a billion-dollar-plus bill last week that would direct scientists The bill proposes research expenditures of $200 million annually for five years to address water issues around the country. It places Sandia at the center of the effort in the role of program coordinator. Eight other national and federal laboratories would team with universities to operate regional water research centers, if the legislation is approved and funds are appropriated.

The bill also proposes $20 million annually for facility construction and $5 million for administration, bringing the five-year total to more than $1.1 billion.

Supporters say the sweeping proposal represents a major effort to revitalize water supply research and development. Federal funding of water supply R&D has been flat since the late 1960s, distributed among 17 agencies and eroded by inflation. Research to increase water supplies is currently done by only three federal agencies.

Sen. Pete Domenici, R-N.M., who introduced the bill on the Senate side, said his interests in water research reach well beyond the uncertain groundwater supplies for drinking water in Albuquerque and the dwindling flow of the Rio Grande for irrigation.

"There are water problems related to quantity and quality in the East and in the big cities," he said. "We can no longer afford to invest in water in drips and drabs when it is vividly apparent that water-related issues will create some of the most significant domestic and international dilemmas facing us this century."

A more urgent issue

Drought and population shift are two significant factors making the management and use of water a more urgent issue for the nation, said Sen. Jeff Bingaman, D-N.M., who also supports the bill. "Meeting this challenge requires an increased national commitment to water resources research. This legislation makes that commitment."

"We have been fighting over a dwindling supply of fresh water in this country for a number of years," says Rep. Richard Pombo, R-Calif. He will introduce a companion bill in the House. "We have to bring in new technologies in order to meet the demand in the future."

"This program is not about research alone. The program is structured to carry applied research through technology development and testing to full commercial implementation," says Peter Davies, Director of Geosciences and Environment Center 6100. Technology transfer and partnerships with industry leading to new commercial technologies will be critical to success.

"Right now we are talking about authorization, not appropriation. We are two or three steps along in a 10-step process," says Peter. "Given that, we are very excited about the scale of the program and the opportunities it presents."

Peter and a team of Sandia researchers got the Labs' own water initiative off the ground about four years ago and have seen it grow to a major focus of research and development activity (see "Sandia's 'Team Agua' " above right.)

Policy component

The bill also recognizes that in the world of water, regulations and policies are as critical as technologies, says Peter. "This is not purely a technical problem." A proposed policy institute would help researchers understand what regulations are actually driving technology and identify policy barriers. The University of New Mexico Law School's Utton Transboundary Resources Center is named in the bill as the lead for the policy center.

As program coordinator, Sandia would be responsible for leading the development of technology roadmaps used to define R&D pathways. Sandia would coordinate research activities at the regional centers and competitive open research efforts outside the centers. The Labs would also coordinate critical technology transfer activities.

"There is a clear intent that Sandia work with industry research foundations, other labs and that we facilitate projects that connect the research to other federal agencies," says Peter. There is also an important role set out in the legislation for an advisory panel to guide the overall program. This advisory panel would include industry, universities, federal agencies, nongovernmental organizations, international water technology institutions, and the regional centers.

"We learned from our experience with the Desalination and Water Purification Technology Roadmap (Lab News, June 25) that our success came because we had a very strong needs-driven effort, bringing together people who are in a good position to define the needs. In this case, we need to bring together water managers, industry providers, and policy makers to define the needs. Then we will work with the R&D community to define the research and development that will meet those needs."

Recognition for Sandia

Sandia's key role in the proposal is due to several factors, including the development of a broad-based Labs water initiative over the past several years. "We have enough effective engagement and impact that people are starting to recognize the kinds of contributions that Sandia can make," says Peter. Cooperative efforts with the Bureau of Reclamation to create the desalination technology roadmap and to develop security methodologies for the nation's dams and key water systems are two examples. Sandia's vulnerability assessment methodology for water infrastructure security has now been used by more than 90 percent of the large US cities, serving more than 130 million water users, he notes.

Sandia's energy expertise also comes into play, Peter adds. Sandia began working with Los Alamos and the National Energy Technology Lab in West Virginia two years ago in an effort to look at interdependencies between energy and water. "Annually in the US, we use on the order of 136 billion gallons of fresh water for agriculture and almost the same amount for electricity generation," he notes.

"As water becomes more difficult to find, water for energy competes with water for agriculture, municipalities, industry, and the environment. New energy plants can't find the water they need and existing ones operate at reduced capacity because of water shortages." US projections call for 1,300 to 1,900 new power plants by the year 2020. "Where's the additional water going to come from?" Peter asks. The original triad investigating this issue has grown to 11 labs, he notes, and the energy-water theme will be addressed by several of the regional centers under provisions of the new bill. -- Will Keener

Back to topBack to Lab News home page.

Sandia research wins two R&D 100 Awards

Back to topBack to Lab News home page.

By Chris Burroughs

Two Sandia research teams have won R&D 100 Awards in the annual competition sponsored by the Chicago-based R&D Magazine.

One award is for a new process of growing gallium nitride on an etched sapphire substrate, called cantilever epitaxy, which promises to make brighter green, blue, and white light emitting diodes (LEDs) -- solid state lighting.

The other is for the creation of the software framework and library Trilinos, which provides broad-ranging, robust, and high-performance capabilities for solving numerical systems at the heart of many complex engineering and scientific applications.

R&D Magazine annually gives the awards to the top 100 industrial innovations worldwide. This year, DOE labs -- Sandia is one --won 34 R&D 100 awards, says Jeannette Mallozzi, the magazine's managing editor. Since the award's inception, Sandia has won 70 R&D 100 awards, according to the magazine's web site.

Winners will be presented plaques at a formal banquet in October at Chicago's Navy Pier.

"The research groups winning these awards at Sandia this year are truly innovative and on the cutting edge of science," says Sandia President and Labs Director C. Paul Robinson. "The cantilever epitaxy process offers the potential for longer-lived and better performing LEDs. Trilinos has had a major impact on Sandia's engineering modeling and simulation capabilities over the past several years, and with its public licensing we are extending that to broad national impact."

Here are brief descriptions of the two winning Sandia technologies.

Cantilever epitaxy

Colored LEDs are of interest for displays and even higher-power lamps like traffic lights. A national initiative is now under way to develop solid-state sources for high-efficiency white lighting. The cantilever epitaxy process of growing LEDs may help meet those needs.

"Our new process eliminates many of the problems that have limited the optical and electronic performances of LEDs, previously grown on sapphire/gallium nitride substrates," says Carol Ashby (11500), one of the inventors.

Over the past several years LEDs have been grown with various combinations of gallium nitride (GaN) alloys on sapphire substrates. However, the atoms of the two materials do not line up perfectly due to differences in natural lengths of the bonds in their respective crystal lattices. Regions of imperfections, called dislocations, accompany this lattice mismatch. These dislocations limit LEDs' brightness and performance.

The new cantilever epitaxy process reduces the numbers of dislocations, giving the potential for longer-lived and better performing LEDs. It also means that LEDs grown on the patterned sapphire/gallium nitride substrates can produce brighter, more efficient, green, blue, and white lights than previously accomplished.

David Follstaedt (1111), another of the inventors, says that because of the reduction in dislocations, the cantilever epitaxy process shows "great promise for making a superior substrate for light-emitting devices" and has potential for applications to a wide variety of electronic devices and GaN integrated circuit technology.

Transmission electron microscopy and scanning electron microscopy were used to determine the number of dislocations eliminated through the cantilever epitaxy process.

The cantilever epitaxy program at Sandia was part of an internal three-year, $6.6 million Laboratory Directed Research and Development (LDRD) Grand Challenge. Funding also came from a grant from the DOE Office of Building Technologies for a collaborative project with Lumileds Lighting, a joint venture between Agilent Technologies and Phillips Lighting.

Inventors: Carol Ashby (11500), David Follstaedt (1111), Christine Mitchell (5932), Jung Han (now at Yale University)

Developers: Andrew Allerman (1126), Katherine Bogart (1126), Karen Cross (1126), Arthur Fischer (1123), Kristine Fullmer (1123), Leonardo Griego (17421), Daniel Koleske (1126), Nancy Missert (1112), Michael Moran (1111), Adam Norman (1111), Andrea Ongstad (1742), Gregory Peake (1742), Paula Provencio (1111), Jeanne Sergeant (1763).


Trilinos is part of a broad effort on the part of national laboratories, industry, and academia to establish high-fidelity computational modeling and simulation as an approach to engineering and scientific understanding so it becomes an equal partner with the most basic approaches of theory and experiment.

Trilinos provides a common enabling solution to one of the most difficult problems in creating these simulations: How can one solve the massive and complex systems of equations required, and do so in a way that "scales" all the way from laptop computers to the most powerful and complex parallel computers in the world?

Trilinos has become tremendously successful at addressing this "solver problem" and has become, for example, a critical enabler for the diverse simulation codes that support almost every major engineering discipline within DOE's Advanced Simulation and Computing (ASC) program.

Trilinos, led by Mike Heroux (9214), is under development at both Sandia/New Mexico and Sandia/California, with some 24 researchers involved. Trilinos offers what is probably the largest and most complete scalable solver capability in the world, and it is freely available to the public.

Meaning "string of pearls" in Greek, Trilinos has an architecture in which object-oriented packages, each of which provides a particular solver capability, are strung together like pearls on a necklace and represent more than the sum of the parts. Trilinos began as three packages, has rapidly expanded to 20, and continues to grow. Computational researchers and software developers find Trilinos attractive because they need only focus on those aspects of development that are unique to their package.

Each Trilinos package is a self-contained, independent piece of software with its own set of requirements, its own development team, and its own group of users. Because of this, Trilinos is designed to respect the autonomy of packages. It offers a variety of ways for a particular package to interact with other Trilinos packages. It also offers developers a set of tools for building on multiple platforms, generating documentation, and multi-platform regression testing.

Trilinos team members: Michael Heroux (9214), Tamara Kolda (8962), James Willenbring (9214), Roscoe Bartlett (9211), Paul Boggs (8962), Robert Heaphy (9215), Ulrich Hetmaniuk (9214), Robert Hoekstra (9233), Victoria Howle (8962), Jonathan Hu (9214), Richard Lehoucq (9214), Kevin Long (8962), Roger Pawlowski (9233), Michael Phenow, Eric Phipps (9233), Marzio Sala (9214), Andrew Salinger (9233), Paul Sexton, Kendall Stanley (9214), Heidi Thornquist (9214), Ray Tuminara (9214), Alan Williams (9143). -- Chris Burroughs

Back to topBack to Lab News home page.

Sandia 'Working on the railroad'

Back to topBack to Lab News home page.

By Will Keener

Sandia has been working on the railroad -- on the Cumbres and Toltec Scenic Railroad. That's the one that chuffs and puffs its way across the mountains along the New Mexico-Colorado state line with loads of tourists aboard.

In response to a request from Lt. Gov. Diane Denish, Ted Borek (1822) and Don Susan (1861) traveled to Chama to help with an important project this spring. "We got a call from our small business group and they said the railroad needs some help," says Ted. "We said 'sure.'"

"The railroad isn't permitted to weld on trains unless they know the composition of the metals," explains Ted. With a train yard of 10 steam locomotives and only two running, railroad officials needed to move forward with repair and maintenance schedules. Lacking a way to determine the composition of several key locomotive components, they asked New Mexico for help.

Lt. Gov. Denish's office passed the request to Sandia. "I love doing these small business projects," says Ted, who has been involved in several others.

Ted traveled to Chama in late March and returned with drill borings from several key parts and slivers of metal removed from a massive locomotive driving rod with a cold chisel. "This was a routine analysis for us, but they were thrilled we could help them out and provide a quick turnaround."

"The metallurgy was required to determine a safe and suitable welding procedure for the main driving rod on the locomotive," explains Kim Smith Flowers, long-time Chama resident and general manager of the Rio Grande Railway Preservation Corp., which operates the narrow gauge. "We had to make that determination before we could approach repairing the rod in a way that would be acceptable to the Federal Railroad Administration.

Ted's team -- Jeanne Barrera, Jeff Reich, Christine White, Polly Wilks, and Steve Meserole (all 1822) -- performed the analysis. They dissolved the metals in acid and ran them through instruments to determine iron, chromium, and nickel content. "It was routine carbon-steel, probably considered state of the art in 1925," says Ted. Don Susan returned to Chama with Ted a week after the samples were collected to test the materials for hardness.

Information on the metals was submitted as a part of a package to the railroad administration, to get needed welding permits and keep the repairs on schedule.

"We are always looking for principal investigators to work on projects like these," says Mariann Johnston (13021) of the Labs' Small Business Assistance program. "It's an opportunity for technical staff to work with small businesses outside the normal everyday projects they do and make a difference for New Mexico. In some cases, it can be technically challenging."

Funded by a special state tax credit, amounting to $1.8 million annually, Sandia helps about 300 businesses each year with a variety of problems. Any New Mexico for- profit business can apply for help not otherwise available at a reasonable cost through private industry through the program. Sandia researchers have found themselves investigating other issues such as plastic irrigation ditch liners, applying new technologies for bar scanning cattle, automating chile harvesting/cleaning, and a wide variety of consulting and testing for small businesses "We earn a tax credit to cover the costs of labor hours the principal investigator puts toward the problem. Currently, we are fully allocated for calendar year 2004 and have other businesses on a waiting list," Mariann says. (Sandia hopes to increase the $1.8 million limit with the state in the future.)

"This is another example of how we can give back to the state we live in," says Sandia's Jerry Hanks (12100), who works on special assignment as a scientific advisor in the Lt. Governor's office. "This is the tip of the iceberg, really." Jerry sees many possibilities for Sandia involvement in the Cumbres & Toltec Scenic Railroad, specifically, and for New Mexico small business in general.

Los Alamos National Laboratory is also helping the railroad operation by donating some equipment it no longer needs to the railroad's machine shop, he notes.

A related project under consideration would create a historic railroad Center of Excellence in northern New Mexico, with an annex in Chama, Jerry says. Such a center would rebuild steam engines and machine parts, providing expertise transferable to other historic railways around the country.

"You can't buy parts off the shelf for a 125-year-old railroad," says Jerry. "The train needs high-tech solutions and right now the machine shop is 1950s at best."

Such a center would teach technical skills to young people and help them find employment in the area. "We envision this to expand beyond machine shop skills to drafting, welding, and perhaps into other related industries that the hospitality and tourism industry might benefit from," Flowers says.

Some observers believe such a center could provide a significant economic impact to the Chama area, turning the railroad into a year-round operation instead of the seasonal attraction it is now. (A steam engine overhaul costs roughly $1 million.)

A first step toward this vision involves a new welding and metallurgy certification program and training center at the railroad's machine shop. This would be done in collaboration with the railroad, Northern New Mexico Community College, the two New Mexico national labs, and the Regional Development Corporation.

In all $31 million is needed for the five-year plan to bring the Cumbres & Toltec to year-round operation with modern communications, a 21st century repair shop, and other needed upgrades, rail officials estimate.

"The goal is to make the railroad self-sustaining," says Jerry. Currently, the states of New Mexico and Colorado contribute most of the narrow gauge's capital improvement and operating income. -- Will Keener

Back to topBack to Lab News home page.

Last modified: July 28, 2004

Back to the Lab News home page

Browse current and past Lab News articles

View Sandia news releases and fact sheets Back to top of page

Questions and Comments || Acknowledgment and Disclaimer