Mesoscale simulation of billion atom complex systems using thousands of GPGPUS's

A Use Case by

Short description

In collaboration with the UKRI STFC Daresbury Laboratory, E-CAM has developed a highly effi-cient version of DL_MESO, a software package for mesoscale simulations developed at the UKRI STFC [1]. This distributed GPU acceleration de-velopment is an extension of the DL_MESO pack-age to MPI+CUDA that exploits the computational power of the latest NVIDIA cards on hybrid CPU–GPU architectures. The need to port DL_MESO to massively parallel computing platforms arose be-cause often real systems are made of millions of particles and small clusters are usually not suffi-cient to obtain results in brief time. Moreover, with the advent of hybrid architectures, updating the code is becoming an important software engineer-ing step to allow scientist to continue their work on such systems.

Results & Achievements

The current multi-GPU version of DL_MESO scales with an 85% parallel efficiency up to 4096 GPUs (equivalent to almost 20 petaflops of raw double precisions performance) (see Fig. 1) [2]. This allows the simulation of very large systems like a phase mixture with 1.8 billion particles (Fig. 2).

For improved load balancing, E-CAM’s load balancing ALL[3] developed at the Juelich Supercomputing Centre has been implemented in the multi-GPU version of DL_MESO (DPD). The intention is to allow for better performance when modelling complex systems, like large proteins or lipid bi-layers, redistributing the workload across the GPUs. The library Kokkos [4] is also being incor-porated in DL_MESO (DPD), enabling the execu-tion of DL_MESO_DPD on NVidia GPUs as well as on other GPUs or architectures (many-core hardwarelike KNL), allowing performance porta-bility as well as separation of concern between computational science and HPC.

Objectives

The rewrite of the DL_MESO code allows the sim-ulation of billion of atoms complex systems on thousands of GPGPUs. This is necessary to simu-late surfactants, key ingredients in personal care products, dish soaps, laundry detergents, etc. At the microscopic level, surfactants are very long chains of molecules, known as polymers. Realistic poly-mers are typically very large macromolecules, and their modelling in industrial manufacturing processes (e.g. to predict complex material properties), is a very challenging task.