Molecular Dynamics

NVIDIA RTX A5000 Performance Benchmarks for RELION Cryo-EM

July 2, 2021
4 min read
blog-relion-benchmarks-rtx-a5000.jpg

NVIDIA RTX A5000 Benchmarks Overview

As a value-added supplier of scientific workstations and servers, Exxact regularly provides reference benchmarks in various GPU configurations to guide Cryogenic electron microscopy (cryo-EM) scientists looking to procure systems optimized for their research.

In this blog post we benchmark the NVIDIA's RTX A5000 performance using Relion Cryo-EM, comparing GPU runtime to Total Runtime performance.

Software Summary

RELION (REgularised LIkelihood OptimisatioN), or Relion, has revolutionized the cryo–EM field since 2012. Developed by Scheres Lab at the MRC Laboratory of Molecular Biology, this stand-alone computer program uses a Bayesian approach to refine macromolecular structures by single-particle analysis of electron cryo-microscopy data.

The development of RELION is supported through long-term funding by the UK Medical Research Council, and is distributed under a GPLv2 license. This means that anyone (including commercial users) can download, use and modify RELION without having to pay anything. They just request that if RELION is useful in your work, that you cite their papers

RELION GPU Support Summary

With advancements in automation, compute power, and visual technology, the scope and complexity of datasets used in cryo-EM have grown substantially. GPU support and acceleration are essential for the flexibility of resource management, prevention of memory limitations, and to address the most computationally intensive processes of cryo-EM such as image classification, and high-resolution refinement.

Exxact RTX A5000 System Specs:

Make / ModelSupermicro AS -4124GS-TN
Nodes1
Processor / Count2x AMD EPYC 7552
Total Logical Cores48
MemoryDDR4 512 GB
StorageNVMe 3.84 TB
OSUbuntu 18.04
CUDA Version11.2
Relion Version

3.1

GPU Benchmarks

Benchmarks below are 3D classifications performed using the Plasmodium ribosome dataset.

  • If runs are performed using the --scratch_dir option in RELION, the time to copy to scratch is subtracted from the total runtime of the command. This allows for comparison of runs independent of storage pool variability.
  • "GPU Time" is reported as the time spent on the Expectation step in each of the 25 iterations of classification, which provides a solid snapshot of GPU speed as this is the primary GPU compute step in RELION.
  • All benchmarks in this chart are performed using N+1 MPI ranks where N = Number of GPUS and six threads per process (--j 6). The Systems used in these results have eight GPU cards installed.


RELION Benchmarks – GPU Run Time Performance

Screen_Shot_2021-06-09_at_6.31.34_PM.png

Notes on System Memory

Although a minimum of 64GB of RAM is recommended to run RELION with small image sizes (eg. 200×200) on either the original or accelerated versions of RELION, 360×360 problems run best on systems with more than 128GB of RAM. 

Systems with 256GB or more RAM are recommended for the CPU-accelerated kernels on larger image sizes. Insufficient memory causes individual MPI ranks to be killed, leading to zombie RELION jobs.

MPI Settings

Where some users may want to run more than one MPI rank per GPU, sufficient GPU memory is needed. Each MPI-slave that shares a GPU increases the use of memory. In this case, however, it’s recommended running a single MPI-slave per GPU for good performance and stable execution.

Notes on Scaling

The GPUs tested were Turing/Volta-based and performed similarly. As a result, it is more beneficial to scale out than scale up. Another thing to note is the diminishing returns in scaling once you pass 4 GPUs.


Have any questions about RELION or other applications for molecular dynamics?
Contact Exxact Today


Topics

blog-relion-benchmarks-rtx-a5000.jpg
Molecular Dynamics

NVIDIA RTX A5000 Performance Benchmarks for RELION Cryo-EM

July 2, 20214 min read

NVIDIA RTX A5000 Benchmarks Overview

As a value-added supplier of scientific workstations and servers, Exxact regularly provides reference benchmarks in various GPU configurations to guide Cryogenic electron microscopy (cryo-EM) scientists looking to procure systems optimized for their research.

In this blog post we benchmark the NVIDIA's RTX A5000 performance using Relion Cryo-EM, comparing GPU runtime to Total Runtime performance.

Software Summary

RELION (REgularised LIkelihood OptimisatioN), or Relion, has revolutionized the cryo–EM field since 2012. Developed by Scheres Lab at the MRC Laboratory of Molecular Biology, this stand-alone computer program uses a Bayesian approach to refine macromolecular structures by single-particle analysis of electron cryo-microscopy data.

The development of RELION is supported through long-term funding by the UK Medical Research Council, and is distributed under a GPLv2 license. This means that anyone (including commercial users) can download, use and modify RELION without having to pay anything. They just request that if RELION is useful in your work, that you cite their papers

RELION GPU Support Summary

With advancements in automation, compute power, and visual technology, the scope and complexity of datasets used in cryo-EM have grown substantially. GPU support and acceleration are essential for the flexibility of resource management, prevention of memory limitations, and to address the most computationally intensive processes of cryo-EM such as image classification, and high-resolution refinement.

Exxact RTX A5000 System Specs:

Make / ModelSupermicro AS -4124GS-TN
Nodes1
Processor / Count2x AMD EPYC 7552
Total Logical Cores48
MemoryDDR4 512 GB
StorageNVMe 3.84 TB
OSUbuntu 18.04
CUDA Version11.2
Relion Version

3.1

GPU Benchmarks

Benchmarks below are 3D classifications performed using the Plasmodium ribosome dataset.

  • If runs are performed using the --scratch_dir option in RELION, the time to copy to scratch is subtracted from the total runtime of the command. This allows for comparison of runs independent of storage pool variability.
  • "GPU Time" is reported as the time spent on the Expectation step in each of the 25 iterations of classification, which provides a solid snapshot of GPU speed as this is the primary GPU compute step in RELION.
  • All benchmarks in this chart are performed using N+1 MPI ranks where N = Number of GPUS and six threads per process (--j 6). The Systems used in these results have eight GPU cards installed.


RELION Benchmarks – GPU Run Time Performance

Screen_Shot_2021-06-09_at_6.31.34_PM.png

Notes on System Memory

Although a minimum of 64GB of RAM is recommended to run RELION with small image sizes (eg. 200×200) on either the original or accelerated versions of RELION, 360×360 problems run best on systems with more than 128GB of RAM. 

Systems with 256GB or more RAM are recommended for the CPU-accelerated kernels on larger image sizes. Insufficient memory causes individual MPI ranks to be killed, leading to zombie RELION jobs.

MPI Settings

Where some users may want to run more than one MPI rank per GPU, sufficient GPU memory is needed. Each MPI-slave that shares a GPU increases the use of memory. In this case, however, it’s recommended running a single MPI-slave per GPU for good performance and stable execution.

Notes on Scaling

The GPUs tested were Turing/Volta-based and performed similarly. As a result, it is more beneficial to scale out than scale up. Another thing to note is the diminishing returns in scaling once you pass 4 GPUs.


Have any questions about RELION or other applications for molecular dynamics?
Contact Exxact Today


Topics