A publication of the Advanced Simulation & Computing Division, NA-121.2, NNSA Defense Programs
June 2008
NA-ASC-500-08—Issue 7
Return to this issue’s stories
Backup Internet Path Saves Los Alamos Connection
On April 15, 2008, a cut fiber cable near Santa Fe, NM, caused a complete loss of Internet connectivity at Los Alamos National Laboratory. Fortunately, LANL network engineers were able to activate a backup path within minutes, re-routing their Internet traffic across the DISCOM WAN and back onto ESNet in Livermore. The backup connection remained active for nearly 22 hours while the fiber cut was repaired. In most circumstances, this incident would have been disastrous, but thanks to the bidirectional backup path to the Internet developed by teams at LLNL and LANL and implemented just five weeks earlier, LANL was back online within minutes.
To make this emergency response possible, network engineers from LLNL and LANL had designed and configured a backup Internet connection using the ASC-funded DISCOM network. The system was designed to protect LLNL and LANL from worst-case scenarios that could disrupt their connection to ESNet, their sole Internet service provider. One such scenario, a fiber cut between LLNL and its next-hop ESNet facility in Santa Clara or between LANL and its next-hop ESNet facility in Albuquerque was deemed unlikely, but it was projected to have a major impact if it occurred.
“We did not test anything quite as drastic as disconnecting the primary link (to simulate a cable cut),” said Robin Goldstone, associate program leader at LLNL for Networks and Convergence, who is responsible for strategic planning of its unclassified network capabilities. “We did perform a real-world validation of the connectivity in late February,” she continued. “We demonstrated we could fail over to the backup link without any interruption in connectivity, perform planned maintenance, then fail back seamlessly. When the actual ‘disaster’ occurred, everything worked just as expected.”
Without the foresight, creativity, and technical skill of the joint LLNL/LANL network engineering team, last week’s outage would have caused significant impact to LANL’s ability to conduct business. LLNL stands to benefit from this experience if a similar situation occurs on its end of the network.
This diagram shows the location of the fiber cut (right) that disrupted LANL’s Internet services on April 15.
DOE Privacy Disclaimer | Sandia Privacy Disclaimer | 2008-4454 W
Developed and maintained by Sandia National Laboratories for NA-121.2