Harvard Researchers Harness Google Cloud to Clone a Supercomputer

Highlights:

• Koumoutsakos and the Citadel team, joined by researchers from ETH Zurich in Switzerland, illustrated their methodology by employing thousands of virtual machines on Google Cloud to create a replicated supercomputing platform.
• The research presents a promising breakthrough with potential alternatives for researchers and organizations requiring extensive compute power.

Harvard University researchers utilized Google LLC’s public cloud infrastructure to replicate a supercomputer that was used for a heart disease study.

They assert that this original utilization of cloud computing resources can aid fellow researchers grappling with limited access to robust supercomputers in completing their studies.

According to Harvard professor Petros Koumoutsakos, the study aimed to replicate a novel therapy designed to dissolve blood clots and tumor cells within the human circulatory system, as reported by Reuters. His team needed substantial computing power, usually accessible only through supercomputers.

According to Koumoutsakos, the research team successfully secured sufficient supercomputer time for a single thorough simulation. Yet, they faced a challenge as they couldn’t replicate the process for improving or optimizing the test’s elements.

It’s a prevalent issue for scientific research teams. Only a few supercomputers available to scientists in the United States are powerful enough to do the billions of computations required for a study like Koumoutsakos’. As a result, a notable waiting list is in place for individuals seeking access to these machines.

To overcome this obstacle, Koumoutsakos and his team collaborated with Citadel Enterprise Americas LLC, exploring the possibility of replicating a supercomputer within the public cloud environment. This approach eliminates the need for resource access waiting times.

However, the public cloud doesn’t offer a straightforward solution since platforms like Google Cloud aren’t inherently tailored to manage researchers’ tasks. Instead, cloud instances are optimized for numerous smaller computing tasks, like serving web pages, streaming videos, hosting applications, and facilitating database access. Conversely, the cloud is renowned for its reliability and resilience, offering uninterrupted availability and eliminating waiting lists for access.

Koumoutsakos and the Citadel team, joined by researchers from ETH Zurich in Switzerland, illustrated their methodology by employing thousands of virtual machines on Google Cloud to create a replicated supercomputing platform. By employing “extensively tuned code,” they harnessed the capabilities of distributed cloud resources to attain a notable efficiency of 80%, providing dedicated supercomputer facilities.

Bill Magro, Google Cloud’s Chief Technologist for High-Performance Computing, highlighted that the cloud has the distinctive potential to address technical, scientific, and engineering computing challenges. He elaborated that adapting cloud infrastructure to mimic supercomputer functionality necessitates alterations across software, networking, and the physical configuration of hardware components. Margo added, “Google Cloud’s high-performance computing technologies and solutions are purpose-built to both simplify and scale the largest, most complex workloads, enabling researchers to dramatically accelerate time to discovery and impact.”

The research presents a promising breakthrough with potential alternatives for researchers and organizations requiring extensive compute power. However, industry insiders find it unsurprising that Koumoutsakos and his team accomplished it, noted Holger Mueller, an analyst at Constellation Research Inc. He highlights that Google’s cloud platform has historically been exceptionally configurable due to the demands of Google’s internal workloads, which have consistently required such high levels of configurability. “An example is Google’s translation models, which have been running for many years,” Halger said. “They need high-end instances and a fast network, which are the hallmarks of supercomputers, and this is exactly what Google Cloud provides too.”

Mueller added that it’s improbable that the limited number of supercomputer providers would be overly concerned about the emergence of the public cloud as a competitor in the high-performance computing sector, considering that cloud platforms are equally sought after and in high demand. “Just about every cloud platform is seeing capacity constraints now with the interest in AI workloads, and it will remain that way for the foreseeable future.”