Argonne's Aurora Supercomputer Powers 2 of the Largest Cosmological Simulations to Date - HPCwire

Off the Wire Press Releases

An Argonne team used the ALCF’s Aurora exascale supercomputer to carry out massive gravity-only cosmological simulations that will help scientists investigate cosmic inflation and support the analysis of observations from DESI, SPHEREx and other major sky surveys.

Oct. 19, 2026 — Researchers at the U.S. Department of Energy’s (DOE) Argonne National Laboratory have completed two of the largest cosmological simulations ever performed. Each simulation modeled the evolution of 13.4 trillion dark matter particles across a region 28.6 billion light-years wide.

Dark matter is invisible to telescopes, but its gravitational effects drive galaxy formation and shape larger-scale structures in the universe. In the simulations, particles stand in as tracers of the dark matter density and velocity distributions. This allows researchers to trace how cosmic structure has evolved over billions of years.

Simulations at this scale require enormous computing power. The team performed the calculations on Argonne’s Aurora exascale machine. One of the world’s largest supercomputers, the system has 10,624 nodes and 63,744 graphics processing units (GPUs). Aurora is housed at the Argonne Leadership Computing Facility (ALCF), a DOE Office of Science user facility.

The team’s simulations used a significant portion of the system. Each run drew on 8,100 nodes and roughly 50,000 GPUs. The simulations and much of the data analysis were completed in about four days. Led by Argonne computational scientist Nicholas Frontiere, the project seeks to shed light on the physics of cosmic inflation.

“The theory describes a brief burst of rapid expansion in the very early universe,” Frontiere said. “Inflation is thought to have stretched microscopic quantum fluctuations to cosmological scales, leaving tiny variations in density that gravity later amplified into the cosmic web we observe today.”

But the physics behind inflation remains uncertain. By studying how matter is distributed across the cosmos, scientists can look for clues that may reveal more about inflation and the early universe.

That is where the simulations come in. They will aid the analysis of major sky surveys, including the DOE-supported Dark Energy Spectroscopic Instrument (DESI) and NASA’s Spectro-Photometer for the History of the Universe, Epoch of Reionization and Ices Explorer (SPHEREx) mission, both of which are currently collecting data. Together, the simulations and state-of-the-art surveys are providing increasingly detailed views of the cosmos to help advance studies of the origins and evolution of the universe.

Searching for Subtle Signs from the Early Universe

Standard inflationary models predict that the early fluctuations followed a Gaussian distribution, one of the simplest forms of randomness. Researchers are searching for small departures from that pattern, known as primordial non-Gaussianity.

“A measurable departure could point to additional fields, interactions or dynamics during inflation and help distinguish among competing theories of the universe’s earliest moments,” Frontiere said.

Any signal is expected to be subtle. It can also be obscured by cosmic variance — statistical uncertainty that arises because surveys sample only a finite volume of the universe.

Detecting such signals requires massive simulations that span the largest cosmic scales. That is where some of the clearest signatures of primordial non-Gaussianity may appear. At the same time, the simulations need sufficient resolution to resolve dark matter halos — the large clumps of dark matter that host galaxies. This connection is critical because the distribution of galaxies is what surveys such as DESI and SPHEREx ultimately observe.

“Achieving both enormous volume and sufficient resolution is what makes the calculation so challenging,” Frontiere said. “It’s a problem particularly well suited to exascale machines.”

Frontiere’s team uses HACC, the Hardware/Hybrid Accelerated Cosmology Code developed at Argonne, to carry out its simulations. The HACC team has a long history of using DOE supercomputers to perform extreme-scale cosmology simulations. As part of DOE’s Exascale Computing Project, the team prepared HACC to run efficiently on different exascale architectures.

Prior to the Aurora runs, the team used the Oak Ridge Leadership Computing Facility’s Frontier exascale system, located at DOE’s Oak Ridge National Laboratory, for a different type of cosmology simulation. That effort produced the largest hydrodynamic simulation of the universe to date, combining gravity with models of gas, stars, and other processes involved in galaxy formation.

The researchers took a different approach on Aurora. By focusing on gravity alone, they were able to simulate a larger volume with more particles.

Aurora’s high-capacity memory was especially important because the simulations had to keep data for trillions of particles readily available as the calculations progressed. Its Distributed Asynchronous Object Storage (DAOS) system supported the project’s immense data-writing demands. The two simulations wrote more than 85 petabytes through DAOS, mostly as restart checkpoints. This allowed the team to resume the runs in case of an interruption rather than start over. About 8 petabytes were kept for scientific analysis.

“Aurora brings together the capabilities this project needed,” said Adrian Pope, ALCF computational scientist and member of the HACC team. “Its GPUs provide the computing power to evolve trillions of particles, while its memory gives us the capacity to handle a problem this large. And with DAOS, we were able to write the massive checkpoints needed to protect the calculations.”

Creating a Realistic Testbed for Sky Surveys

The two Aurora simulations contain the same cosmological model and the same non-Gaussian signal, but their initial phases are flipped.

“The simulations act like statistical mirror images,” Frontiere explained. “When analyzed together, much of the random variation cancels, giving a cleaner measurement of the underlying physics.”

The signal strength is also blinded, so researchers using the simulations to test their analysis methods will not know the answer in advance. This allows them to evaluate their workflows without being influenced by the expected result.

“By testing whether their pipelines can recover the hidden value from the synthetic observations, they can validate their methods before applying them to real survey data,” Frontiere said.

Setting a new mark for scale, the calculations are the largest individual cosmological simulations ever performed by particle count. Their immense volume will allow scientists to build synthetic full-sky maps and realistic mock observations for DESI, SPHEREx, and other surveys. These mock datasets can help researchers find biases, improve their analysis methods and produce more reliable measurements from real survey data.

“Pushing the scale of these simulations helps us better connect theory with what we observe,” Frontiere said. “As we continue to simulate larger volumes in greater detail, we improve our ability to explore fundamental questions about the universe and make better use of increasingly powerful surveys.”

The team received computing time on Aurora and Frontier through DOE’s Innovative and Novel Computational Impact on Theory and Experiment (INCITE) program. The INCITE team includes Argonne’s Frontiere, J.D. Emberson, Michael Buehlmann and Nesar Ramachandra.

Source: Jim Collins, ALCF