Livermore’s computing infrastructure built on a foundation of forethought and continuity of collaboration

In an era where data processing and connectivity drive innovation and national security, the infrastructure that supports this digital backbone is more critical than ever.

The Enterprise Data Center at Sandia’s Livermore site is a testament to foresight and collaborative spirit. This facility is not just a building filled with servers; it is a dynamic, evolving ecosystem designed to meet the ever-growing and unpredictable demands of the Labs’ missions.

Scalable, flexible

The California Data Center’s story begins in the mid-2010s, when the need for a modern, scalable and resilient data center became clear.

“We built the data center with intentionality — scalability, modularity and flexibility were baked into the design from the start,” said Matt Schrager, who manages the team that supports the center and its users. “We left room to grow, with space for additional air handlers, chillers and other electrical and mechanical equipment, even if they weren’t installed initially, knowing they would come when they were needed.”

Jonathan Preston, manager of the division’s digital infrastructure, was there from the beginning.

“Scalability was probably the most important design characteristic we used most regularly when speaking about it at that time,” he said.

Before the ground for the center broke on June 3, 2019, technology’s tectonic plates had already shifted. Demand for artificial intelligence was already underway and growing, forcing the team to think about power and cooling in new ways before the center was operational.

“A server rack, full of equipment the data center had in mind when it was designed, would draw 15 kilowatts. Now that same rack, loaded with AI servers, will draw closer to 100 kilowatts,” Matt said.

The flexibility and stability inherent in those designs make possible what is to come.

“What we focused on with this project was reliability,” Jonathan said. “We’ve done that job excellently. Now we want to build on that foundation to give us the capacity needed for future success with AI and high-performance workloads. We weren’t thinking about building the biggest data center stuffed with the highest compute count but about operational stability that enables the pivot to a scalable future.”

The people they needed to help accomplish that were already on the team, ready to tackle the emerging needs for new capabilities: Dave Martinez, Sandia’s engineering program project lead for Infrastructure Computing Services, and Heath Wehrman, the Critical Facilities program lead in Livermore.

“Modern compute power and cooling needs have changed dramatically since the building was designed,” Heath said. “AI systems now use liquid cooling, which the building can accommodate with some modifications. The building was initially outfitted for about 300 kilowatts but planned to grow to about 1 megawatt. We’ve exceeded the initial capacity, so ongoing investments and planning are needed to handle future scales.”

Keeping the lights on

The expansion wouldn’t be possible if the data center hadn’t been engineered for resilience and continuous, uninterrupted operation no matter the complexity.

“We have duplicate, parallel systems — multiple chillers, pumps, power panels — so we can perform maintenance without shutting down the entire building,” Heath said. “It’s a delicate dance of switching and bypassing to isolate components while keeping the rest of the facility fully functional. Our focus is preemptive. We identify risks and implement mitigation plans before they become problems.”

Jonathan agreed.

“We were thinking from the perspective of how to get to an operationally stable place that can enable where we are now — then pivot to a scalable future,” he said. “The Labs’ mandate includes responsible spending of taxpayer dollars, making every dollar go as far as possible. California’s role is not to build massive clusters that put us on the map, like top 500 systems worldwide, but to find something researchers can develop against that paves the way for future innovation.”

The pandemic tested these systems immediately after the building became operational. When thousands of employees suddenly shifted to remote work, the data center had to support a massive surge in demand.

“We had to multiply the bandwidth by many times in just a few days,” Dave said. “It was a monumental effort, but it showed the strength of our infrastructure and the collaboration between New Mexico and California teams.”

A unified mindset

That collaboration was present during the design and is a pillar of the data center’s operation. Dave, Matt and their teams see the New Mexico and California data centers as a single, geographically distributed entity.

“It is one data center with two zip codes,” Dave said. “This mindset drives how we build redundancy and resilience. Critical systems are backed up across both sites and we’re working to strengthen that relationship.”

“Most business-critical services in New Mexico are backed up in California,” Matt said. “It’s not just a backup; it’s a business continuity site that can take over quickly if needed. That’s a huge value for the Labs.”

When a challenge needs to be tackled, Matt calls upon his cohorts in New Mexico.

“We’ve been helping California with cooling and power issues,” Dave said, citing one example from a recent trip to Livermore.

Invisible, indispensable

Ascribing the California Data Center’s importance simply to data management and cutting-edge technology might be missing a big point. The collaboration of people of differing disciplines and in different geographic places not only makes this capability better, but it also speaks to Sandia as one lab.

“The data center is a complex ecosystem, and it relies on teamwork to keep it running optimally,” Matt said. “Our California Facilities partners have deep expertise in the electrical and mechanical systems, along with operations leads like Drew Portis who monitors cooling and airflow daily. With a strong partnership with New Mexico counterparts, we’re able to do this successfully.”

“Facilities is like the invisible army that keeps the site running,” Heath said. “We’re constantly adapting to shifting risk profiles and new technology demands. When we do our job well, it’s transparent — everything just works.”

It is a shared culture of continuous improvement at work.

“We’re not satisfied once a data center is built,” Dave said. “We have ongoing commissioning teams that meet biweekly to find ways to improve power, cooling and efficiency. We’re experimenting with liquid cooling and even immersion cooling, where entire computers are submerged in liquid to manage heat more effectively.”

Future-ready

One of the greatest challenges in data center design is future-proofing — building a facility that can adapt to technologies and demands that don’t yet exist.

“We plan for growth in footprint and infrastructure rigor,” Heath said. “The building was designed to accommodate future investments, even if those capabilities weren’t installed initially. The data center is still in its infancy — a toddler, if you will. As we consolidate more systems into this facility, the investment in infrastructure must scale accordingly. We plan for different scenarios, from modest growth to large-scale expansions, to ensure the site remains robust and reliable.”

For the thousands of researchers, engineers and staff who rely on the California Data Center daily, the facility’s complexity and criticality often go unnoticed. But those who work on making Sandia future-ready are focused on stewarding smart investments in a resource that can take the Labs to the next generation of computing — whatever it might be.

“We should be strategically investing in compute and critical infrastructure like power and cooling in a meaningful but modest way,” Jonathan said. “We’re not trying to build the biggest, but to deploy investments that allow adding new capabilities to enable big bets. Folks doing work that prepares for and predicts the future need places to run that compute. The more we can deploy in California, the more we can respond quickly.”

“The data center is the backbone that enables mission-critical work,” Dave said. “It’s a complex, evolving system that requires constant attention, innovation and collaboration. We’re proud of what we’ve built and where we’re going.”