Harnessing the Power of Supercomputing

by Richard Blake

The explosive growth in computing power has brought computers into the heart of most scientific disciplines. While servers and workstations service this mainstream, it is parallel supercomputers capable of harnessing thousands of processors for a single calculation that are redefining the boundaries of computational science. The Science and Technology Facilities Council's (STFC) Computational Science and Engineering Department (CSED) is at the forefront of efforts to use supercomputing to tackle problems across a range of scientific disciplines.

Spread across STFC's Daresbury and Rutherford Laboratory sites, the CSED has a long pedigree of working in high-performance computing (HPC). The heart of our work is the development and application of powerful simulation codes, usually in collaboration with university research groups. Through our involvement in the Collaborative Computational Projects, we bring together leading UK expertise in key fields of computational research to tackle large-scale scientific software development. Our work spans a range of disciplines from quantum chemistry through to biology and engineering, allowing the latest techniques in HPC to be brought to bear in many different areas.

As well as developing our own codes, we also help others redesign their codes to run on the latest machines or take advantage of newly emerging technology such as FPGAs and co-processors. As supercomputing moves into the era of the petascale (systems running at around 1015 operations per second) we are actively engaged in research to understand the challenges that programming such massive machines will bring.

We are currently involved in both of the UK's major academic supercomputers, HPCx and HECToR. The 55 teraflop/s (one teraflop/s equals 1012 floating point operations per second) HECTOR machine provides the power to run the largest calculations, and we are looking at innovative ways to use the 12.9 teraflop/s HPCx as a complementary resource to Hector, taking advantage of emerging technology such as Grid computing or using it to drive large-scale visualizations.

To give a flavour of the kind of research that supercomputers enable, we here highlight two projects in which we are involved: one in climate modelling and the other in the development of quantum computing.

Modelling the Oceans
The Proudman Oceanographic Laboratory Coastal Ocean Modelling System (POLCOMS) is a simulation code that has been designed to model the global marine environment. At its core, POLCOMS consists of a 3D hydrodynamic model that simulates the effects of pressure, density etc to provide the forces that drive the flow of water. POLCOMS is an interdisciplinary code, so the core can be integrated with other models such as ecosystem (phytoplankton, zooplankton etc) or wave models. Integrating the models or increasing the resolution of any of the components can increase the computational demands by factors of ten to forty, so extensive work has been carried out on the optimization and parallelization of POLCOMS.

Figure 1: Summer mean horizontal circulation around the UK averaged at mid-depth (20-40m) as calculated with POLCOMS.

Today POLCOMS can be run on thousands of processors on machines such as HECToR. Work is currently underway to enable POLCOMS to be run concurrently on multiple supercomputers, fully utilizing the total computing power available across the machines.

Recent studies carried out with POLCOMS have investigated the northwest European shelf seas. The study has been able to separate the model currents into those driven by winds, by changes in density and by the North Atlantic. The study has demonstrated that, contrary to previous thinking, the long-term circulation is actually driven by the smaller but directional density currents and not the stronger, variable winds.

Designing the Next Generation of Supercomputers
Quantum computing promises a revolution in the capabilities of computers, harnessing the bizarre properties of matter on a quantum scale to enable calculation rates that are inconceivable with the current computer designs.

The materials group within CSED has been investigating the spin interactions in carbon peapods, a novel material that holds promise as a component of a quantum computer.

Carbon peapods are carbon nanotubes containing tiny spherical balls of carbon known as fullerenes. Inserting a metal atom such as scandium into each fullerene introduces an unpaired electron that turns each fullerene into a tiny magnet capable of interacting with its neighbour. Each magnet can hold a qubit (the fundamental unit of quantum computing), so a chain of these magnets interacting together is a potential candidate for the core of a quantum computer.

Figure 2: Charge transfer in a Sc@C82@(14,7) peapod as calculated by DFT with CRYSTAL.

The calculations to model the spin interactions require the use of the computationally demanding density functional theory (DFT). As the carbon peapod system contains more than 600 atoms, this sort of calculation is intractable with standard codes on a conventional system.

The group has therefore been using the CRYSTAL code developed jointly by CSED and the University of Turin to run the calculations on 512 processors on HPCx. It is only by using the power of such large machines with codes like Crystal that these sorts of calculations become possible.

Links:
CSED: http://www.cse.scitech.ac.uk
HECTOR: http://www.hector.ac.uk
HPCx: http://www.hpcx.ac.uk
CRYSTAL: http://www.cse.scitech.ac.uk/cmg/
CRYSTAL POLCOMS: http://www.pol.ac.uk/home/research/polcoms

Please contact:
Richard Blake (Acting Director)
Computational Science and Engineering Department
Daresbury Laboratory, UK
E-mail: r.j.blakedl.ac.uk