FLC | DOE labs partner with Open Science Grid for COVID research

DOE labs partner with Open Science Grid for COVID research

April 27, 2020

In March, Fermilab Chief Information Officer Liz Sexton-Kennedy got in touch with Frank Würthwein, executive director of the Open Science Grid, to find out whether Fermilab could work with the Open Science Grid to contribute computing power to COVID-19 research.

It turned out that Würthwein was thinking along similar lines.

Shortly thereafter, members of the Open Science Grid — a network of organizations, funded by the National Science Foundation, that provides computing services for science research of all stripes — were contributing tens of thousands of core-hours to COVID-19 projects. That number quickly grew to more than a million core-hours.

As part of the unified response, scientists and engineers at the Department of Energy’s Fermilab spurred themselves into high gear, preparing computing clusters — sets of connected computers — as COVID-19-research machines. As of April 27, Fermilab had contributed 1.8 million core hours to the pandemic-fighting effort carried out by projects such as Folding@home, which is simulating how viral proteins fold to help scientists design better therapeutics. Brookhaven National Laboratory, another DOE national lab and OSG member, had contributed 2.9 million core hours.

“Understanding how the COVID-19 virus proteins fold — it’s all electrochemistry. Basically, it’s physics in the end that they’re simulating,” Sexton-Kennedy said. Like particle physics, “it’s computationally complicated science that takes lots of computing resources to figure out.”

Both Fermilab and Brookhaven’s high-throughput computing capabilities have for over a decade met the needs of experiments at the Large Hadron Collider, an enormous particle physics research facility in Europe, each analyzing data from billions of particle collisions.

During the times that select Fermilab and Brookhaven computers get a break from particle collision analysis, they’re free to crunch data outside particle physics. That’s where the Open Science Grid comes in. Among other tasks, the OSG evaluates research proposals to determine which are a good fit for its networks. In offering its resources to COVID-19 proposals, it provided the kind of vetting that the labs wouldn’t have been able to assume on their own. All the computational work is handled remotely.

Providing the world’s scientists with powerful computing capacity fuels research that would otherwise not be possible. Recognizing this, the U.S. Department of Energy supports Fermilab and Brookhaven science programs for the use and development of the OSG, and the National Science Foundation funds universities that partner with the OSG.

This effort is part of a larger COVID-19 High Performance Computing Consortium designed to provide access to the world’s most powerful high-performance computing resources in support of COVID-19 research. Consortium members include industrial partners, academic leaders, and agencies within the federal government, including the Department of Energy and many of its national laboratories. OSG prioritizes jobs to meet the needs of the consortium first.

“To maximize the chances that important COVID-19 research ideas reach us, we joined multiple national and international calls for computing requests: the COVID-19 High Performance Computing Consortium, a call with EGI for Advanced Computing Research in Europe, and the Worldwide LHC Computing Grid COVID-19 task force. We are also coordinating with all of the National Science Foundation software institutes,” said Würthwein, who is also a physics professor at the University of California, San Diego, and the lead for high-throughput computing at the university’s San Diego Supercomputer Center.

The computing power at Fermilab, Brookhaven and other OSG institutions can be used to model how virus proteins interact with receptors on cells in the human respiratory system. Simulations run on these systems can also sort through billions of potential drug molecules to narrow the search for drugs that might interfere with those interactions or disrupt the function of a virus protein in other ways.

Recent developments in machine learning accelerate this discovery process by improving the selection of leads. Modeling drug-protein interactions can be broken down into a larger number of independent calculations that can be farmed out to many computer cores. This characteristic makes the simulations well-suited to high-throughput capabilities at Fermilab, Brookhaven and the OSG infrastructure.