The Advanced Memory Technology program kicks off round three with Cerebras
Sandia, along with Lawrence Livermore and Los Alamos National Laboratories, have entered a third round of collaborative research and development with Cerebras Systems through the Advanced Memory Technology program.
As part of NNSA’s post-exascale computing initiative, the AMT program aims to address the disparity between advancement in computing and memory capabilities. Funded through the Advanced Simulation and Computing program, the partnership will not only drive innovation in memory technology but ensure that the U.S. remains at the bleeding-edge of national security efforts in computing.
“ASC’s mission demands the highest levels of performance and scalability. We are grateful for a deep and enduring partnership with Cerebras in pushing at the boundaries of hardware performance and look forward to developing a future generation of systems that will continue American leadership in this nationally important field,” said Simon Hammond, program director of advanced computing in the Office of Advanced Simulation & Computing.
The Sandia-led, Tri-labs and Cerebras partnership produced significant results in previous rounds. In the first round the team not only achieved the original goal of the AMT program, 40X performance increase over the now deployed exascale platforms, but exceeded it by demonstrating 750X better performance over the exascale systems. In round two, this was followed by the team being selected as Gordon Bell finalists for their groundbreaking work in molecular dynamics. The successful collaboration has paved the path for the third round on this effort.
“The results of our initial work with Sandia, Los Alamos, and Lawrence Livermore were extraordinary,” said Andy Hock, senior vice president of product and strategy at Cerebras. “Exceeding exascale performance targets by such significant margins demonstrated the transformative value of wafer-scale computing and the value of our deep co-design collaboration with DOE. But this was just the beginning. Our new work under AMT Phase 3 will develop advanced memory and computing architectures to push the industry forward, accelerate mission-critical NNSA applications, and extend U.S. national leadership in AI and HPC.”
Researchers are now working to leverage what they have learned to enable a mixture of mod-sim and AI for future NNSA mission requirements, leveraging future architectural enhancements. “Our collaboration has been focused on both architectures and applications,” said Siva Rajamanickam, distinguished member of technical staff at Sandia and lead of the Cerebras collaboration. “Using applications from our existing mission space, we aim to demonstrate the performance capabilities of the existing architecture. More importantly we also focus on impacting future architectures which will be critical for converged AI/ML and mod-sim applications.”
This new phase will expand memory capacity while maintaining the world-class bandwidth and latency characteristics that enabled significant application performance improvements in the previous rounds. Increased memory capacity will allow the Cerebras Wafer Scale Engine to host complex modeling and simulation applications that are the bedrock of the Advanced Simulation and Computing stockpile stewardship mission.
The work will also study and co-design AI and HPC software applications that can uniquely leverage these emerging technologies. “In this phase we are leveraging our work into a fundamentally new architecture for how memory works with computation,” said Michael James, co-founder and Chief Architect of Advanced Technologies at Cerebras. “It has broad implications for supercomputing, AI, and national security. The steps we are taking with Sandia continues the progression of AI hardware’s transformation of what is feasible in computational science.”
This phase builds on the innovative solutions that could arise out of public-private partnerships that are important to Sandia’s traditional and emerging mission applications as the computing landscape continues to evolve. Partnerships between the national laboratories and industry foster an environment where such unique capabilities can come to fruition.
“The AMT program partnered with Cerebras Systems because we saw potential in the capabilities of the architecture,” said James H. Laros III, senior scientist and AMT program lead. “Together we have shown the potential for increased application performance and charted a roadmap for future support of our production mission, which we anticipate will be a mixture of traditional mod-sim and future AI/ML components.”