FLC | DOE labs help their collaborative research teams nab two SC20 prizes

DOE labs help their collaborative research teams nab two SC20 prizes

November 20, 2020

U.S. Department of Energy (DOE) national laboratories were well represented on the two teams that were awarded distinguished prizes for research on November 19 at the 2020 International Conference for High Performance Computing, Networking, Storage and Analysis (SC20) virtual conference. Both teams conducted their research on the Oak Ridge Leadership Computing Facility’s (OLCF’s) Summit, the nation’s fastest supercomputer; one of the winning teams included Lawrence Berkeley National Laboratory and the other included Argonne National Laboratory.

The Association for Computing Machinery (ACM) Gordon Bell Prize, which recognizes outstanding achievement in high-performance computing (HPC), was presented to a team of researchers from Berkeley Lab's Computational Research Division; the University of California, Berkeley; the Institute of Applied Physics and Computational Mathematics, Peking University; and Princeton University for its successful testing of the DeePMD-kit software package, named for “deep potential molecular dynamics.”

The team refers to DeePMD-kit as a “HPC+AI+Physical model” in that it combines high-performance computing (HPC), artificial intelligence (AI), and physical principles to achieve both speed and accuracy for molecular dynamics (MD) simulations.

Simulating a block of copper atoms, the team put DeePMD-kit to the test on Summit with the goal of seeing how far they could push the simulation’s size and timescales beyond AIMD’s accepted limitations. They were able to simulate a system of 127.4 million atoms—more than 100 times larger than the current state of the art. Furthermore, the simulation achieved a time-to-solution mark of at least 1,000 times faster at 2.5 nanoseconds per day for mixed-half precision, with a peak performance of 275 petaflops (one thousand million million floating-point operations per second) for mixed-half precision.

“By combining physical principles and the representation power of deep neural networks, the Deep Potential method can achieve very good accuracy, especially for complex problems,” said Weile Jia, a postdoc in applied mathematics in Professor Lin Lin’s group at the Math Department of UC Berkeley, who co-led the project with Linfeng Zhang of Princeton. “Then we reorganize the data layout for bigger granularity on GPU and use data compression to significantly speedup the bottleneck. The neural network operators are optimized to the extreme, and most importantly, we successfully use half-precision in our code without losing accuracy.”

The second prize, the ACM Gordon Bell Special Prize for HPC-Based COVID-19 Research, was awarded to team led by Rommie Amaro, professor and endowed chair of chemistry and biochemistry at the University of California San Diego, and Arvind Ramanathan, computational biologist at Argonne National Laboratory, for work building an AI-based workflow to simulate the SARS-CoV-2 virus’ spike in numerous environments, including within the SARS-CoV-2 viral envelope comprising 305 million atoms—the most comprehensive simulation of the virus performed to date.

“We are excited to have won this prestigious award,” said Ramanathan, an Argonne computational biologist and co-principal investigator on the project. “The whole point is to push the boundaries of what we can do with AI. The ability to scale such a huge set of simulations and use AI to drive some factors was key to this work.”

The team was able to successfully optimize and scale the Nanoscale Molecular Dynamics (NAMD) code to Summit to run their massive molecular simulations. The results of the team’s initial runs on Summit have led to discoveries of one of the mechanisms that the virus uses to evade detection as well as a characterization of interactions between the spike protein and the protein that the virus takes advantage of in human cells to gain entrance into them—the ACE2 receptor.

“This is one of the first biological systems of the virus that we can learn from to drive scientific discovery,” Amaro said. “Our methods of computing allow us to get down to actually see detailed intricacies of this virus that are useful for understanding not only how it behaves but also its vulnerabilities, from a vaccine development standpoint, and a drug targeting perspective."

This research was supported by the Exascale Computing Project, a collaborative effort of the DOE Office of Science and the National Nuclear Security Administration, and the DOE’s National Virtual Biotechnology Laboratory with funding from the Coronavirus Aid, Relief, and Economic Security (CARES) Act. This work used resources, services, and support from the COVID-19 HPC Consortium.