They gathered at the National Academy of Sciences building in Washington, D.C., to decide whether to adopt a new, infant sibling into the family of scientific understanding.
They heard about Sandia’s 50-year-old approach to nuclear weapons engineering and how the same principles might be applied to other high-consequence engineering challenges. They discussed ways the "surety methodology" could solve a dozen of the nation’s most difficult problems.
And they told the Sandians who developed the surety methodology just what they thought of the ideas being presented.
The conclusion: So far, so good, says Pace VanDevender (4010), technical coordinator of Sandia’s Workshop on Surety Science and Engineering in Washington Sept. 23.
Stakes are higher these days The list of invitees included current and potential Labs partners who develop or oversee high-consequence systems — from air traffic control infrastructures to US nuclear reactors — that might be made safer, more secure, or more reliable through the application of surety principles. (See "What is surety? Its four levels" on next page.)
"Increasing consequences demand a higher level of surety," workshop chair Joan Woodard (VP-6000) told participants as the day began. "The advancement of the concept and principles of surety science and engineering requires partnerships."
Joan pointed out that before 1945 a rigorous, systematic approach to making engineered systems safer, more secure, and more reliable was not really necessary. Engineered systems either worked as they were designed to work or didn’t work. If they didn’t work, engineers fixed them, tested them, and fixed them again if necessary.
With the advent of nuclear weapons, however, the stakes became higher. She described the birth and growth of surety principles in the nuclear weapons complex and their success in creating the safest, most secure, and most reliable engineered systems known to humankind. Since then, Sandia has applied the same principles to nuclear reactor safety, nuclear materials transport, and other high-consequence activities, she said.
Joan defined two types of increasing complexity that characterize modern-day society: those involving engineering complexity as technology advancements are incorporated into systems, and those involving global issues and threats made more complex by their interconnectedness.
"Keeping up with this increasing level of complexity demands advancements in surety science and engineering," she said.
Applying surety to national challenges Pace then took participants on a guided tour of the surety methodology, which he first outlined in a 17-page white paper several months ago. (See "What is surety?")
"As we talked to people in different fields [of high-consequence engineering], we found that we were really talking about the same thing, and we realized we could develop a conceptual framework for solving a wide variety of complex problems," he said.
To define this conceptual framework, he told the group, Sandia sought first to identify the "best practices" engineers subscribe to in developing high-consequence systems, including nuclear weapons. Then Pace and the other members of the Surety Strategy Team identified relevant surety expertise across the laboratory and distilled eight general approaches to engineering and four levels of surety.
All engineered systems fit into one of the four levels, he said. To increase the surety of a system incrementally, you simply adopt the approaches and best practices associated with the next higher level of surety.
Easier said than done, he added, but this formalized progression gives engineers and policy makers a systematic way of improving the level of surety of their highly complex systems and, potentially, addressing critical national concerns.
After Pace described the surety levels and approaches, participants practiced determining the surety approaches currently employed in a variety of industries and engineered systems, registering their opinions with hand-held computers. Results of the voting were presented as bar charts on an overhead projection screen.
Forum for surety issues needed
Eleven Sandians took the stage, one after the other, each describing a critical national concern, suggesting what level of surety currently exists in that industry or field, and recommending a higher level of surety for that area. (See "Surety areas discussed and their Sandia presenters" on next page.) The final Sandia speaker was Ray Bair (ret.), who proposed establishment of the Institute for Surety Science and Engineering (ISSE). The not-for-profit Institute would provide a forum for the continued development of surety principles. It would host conferences, publish a surety-related technical journal, and sponsor an annual surety awards program, perhaps similar to the Malcolm Baldrige program. Sandia has allocated funding for FY99 to help establish ISSE.
Two keynote speakers addressed the group: William Wulf, President of the National Academy of Engineering, which cosponsored the event along with the National Academy of Sciences and the Department of Energy, and Vic Reis, DOE Assistant Secretary for Defense Programs. (See "Two top-level views . . ." at left.)
Attendees grabbed a box lunch and broke up into smaller breakout session groups corresponding to the 11 specific application areas. Breakout groups spent most of the afternoon discussing whether and how the surety methodology could be applied to problems in their areas. When the larger group reconvened two hours later, it was time to hear whether Sandia’s "surety message" was heard and whether attendees thought it valid.
During the breakout sessions, each group was asked to develop two or three top-priority actions that should be accomplished to increase the level of surety in their area. Later, workshop participants voted on which of the actions should have the highest priority for each area. The resulting ideas will be used to develop the messages that will be delivered to the nation’s leadership during the Surety Expo planned for this spring.
What they thought
Breakout session summaries from the Sandians who facilitated the sessions clustered around two general themes: Many of today’s problems are complex enough that a systematic approach is necessary, and developing surety as the "common language"of high-consequence system engineers is a worthwhile endeavor.
Although some groups expressed skepticism that the surety methodology was mature enough, or that it could add value to fields such as nuclear reactor operations or nuclear materials management where safety, security, and reliability issues have been paramount for decades, participants in areas such as aviation safety, critical infrastructure protection, software reliability, and school security generally embraced the methodology.
"There are a lot of tools within Sandia and the national labs used in nuclear weapons engineering that have outside applications," said David Boyle, a nuclear engineering professor at Texas A&M University whose specialty is nuclear materials management. Although the principles behind surety aren’t new to the nuclear materials management arena, he said, "this type of interaction is useful. I’m glad to know there is this effort to push these ideas into the outside world."
Paul Jankowski, a program manager with the Federal Aviation Administration’s Tech Center, said the number of airline passengers in the US climbs every year. "If the failure rate remains the same, you’re going to have an accident every week," he said.
"This [advancement of surety] is happening at the right time," he added. "I think Sandia made a point that it is important. We have become so dependent on technology that we need a higher level of confidence that our systems will work."
Pace asked the non-Sandia participants to vote once more, this time on the degree to which they agreed with several statements. The voting showed that 45 percent of the participants agreed "very much" or "much" that attending the workshop was worth their time; 23 percent agreed "little" or "very little" that the workshop was worth attending.
On the other hand, 78 percent of participants agreed "very much" or "much" that the surety methodology is worth developing further, while only 3 percent agreed "little" or very little" that it is worth pursuing.
So what happens next?
Feedback from the workshop indicates that participants generally embraced the surety principles, says Pace, but the message might have been too much, too quickly.
"The definition and the four levels of surety were accepted and applied," he says. "The eight approaches, which are mental templates for solving surety problems, were too complicated to apply, at least for a one-day workshop. We should keep those inside Sandia for a while and write a textbook."
Jim Rice, Director of Information Systems Engineering 6500, now takes over responsibility for the surety initiative as Pace begins his new duties as Sandia’s Chief Information Officer (CIO). "While the fundamental building blocks of surety are not new, their integrated application as a systems approach to solving problems is an innovation with enormous potential to improve the performance of complex, high-consequence systems," Jim says. "The workshop helped validate the potential of this approach."
Sandia’s next steps will be to articulate and develop the science and engineering tools that form the basis of the surety methodology; to strengthen the Labs’ partnerships with agencies, companies, labs, and universities whose missions include relevant problems; and to convey the surety message to the nation’s decision makers, he adds.
"Surety is in its infancy, comparable to the quality methodology when it was in its infancy," Pace adds. "At first people said quality costs too much. Then they began to see the economic benefits of applying quality principles to their operations and they said quality was free. Now we know that quality pays. The eventual goal is to demonstrate that like quality, surety pays." "The workshop gave us a strong vote that surety science and engineering does have broad applicability beyond its original roots in nuclear weapons," says Joan. "We will continue to develop some structure to better formalize this multidisciplinary area, working extensively with university and industry representatives."