Benchmarks
RELION GPU Benchmark for 3D Classification - NVIDIA H100, RTX 6000 Ada, RTX A4500
September 1, 2023
5 min read
RELION for Cryo-EM GPU Benchmarks Overview
As a value-added supplier of scientific workstations and servers, Exxact regularly provides reference benchmarks in various GPU configurations to guide Cryogenic Electron Microscopy (cryo-EM) scientists looking to procure systems optimized for their research. In this blog, we benchmark the NVIDIA H100, NVIDIA RTX 6000 Ada, and RTX A4500 performance using Relion Cryo-EM, comparing GPU runtime to Total Runtime performance (lower is better) on an Intel Xeon Scalable 8490H platform and an Intel Xeon W9-3495X platform.
Software Summary
RELION (REgularised LIkelihood OptimisatioN), or Relion, has revolutionized the cryo-EM field since 2012. Developed by Scheres Lab at the MRC Laboratory of Molecular Biology, this stand-alone computer program uses a Bayesian approach to refine macromolecular structures by single-particle analysis of electron cryo-microscopy data.
The development of RELION is supported through long-term funding by the UK Medical Research Council and is distributed under a GPLv2 license. This means that anyone (including commercial users) can download, use and modify RELION free of cost. The MRC Laboratory just request that if RELION is useful in your work, you will cite their papers.
Exxact Testbench Specifications
| Xeon W Workstation | Xeon Scalable Server | |
|---|---|---|
| Processor | Intel Xeon W9-3495X | Dual Intel Xeon Scalable 8490H |
| Total Cores | 56 Cores | 120 Cores (60 Each) |
| Base/Max Boost Clock | 1.9GHz/4.8GHz | 1.9GHz/3.5GHz |
| Memory | 512GB DDR5 ECC | 512GB DDR5 ECC |
| Storage #1 | 1.92TB M.2 NVMe SSD | 4.09TB M.2 NVMe SSD |
| CUDA Version | 12.0 | 12.0 |
RELION GPU Benchmarks for 3D Classification - RTX 6000 Ada, RTX A4500, H100 PCIe.
In the first test, we want to visualize the performance gaps between GPUs on the workstation and server platforms. We will compare performance based on CPUs in the next section. Performance numbers are time; a lower score is better.
As expected, the top-of-the-line RTX 6000 Ada performs very well among the other GPUs. It is worth noting the H100 was not designed for HPC and RELION-type workloads; they are more suitable for deep learning training and inferencing.
The RTX A4500 is a crowd favorite in the Life Science industry due to its lower cost and well-rounded performance. However, its age shows in the Intel Xeon Scalable 8490H platform. 4x RTX A4500 performance numbers deliver, at best, 70% of the performance of RTX 6000 Ada on an unoptimized setting 5xMPI (where 9xMPI with 4x GPUs deliver to 60% of the RTX 6000 Ada performance).
With the Intel Xeon Scalable server platform, 8 and 10 GPU configurations are possible. However, 10x A4500s perform just as well as 4x RTX 6000 Ada. This puts into question, the price of 10 RTX A4500 in a 4U server enclosure, or 4x RTX 6000 Ada in a more portable desktop workstation chassis.
RELION CPU Benchmarks with NVIDIA RTX 6000 and RTX A4500
To dive a little deeper into the performance between GPU and CPU platform, we use the same numbers and format them to showcase the CPU performance when GPU is static. We test a dual CPU server configuration with Intel Xeon Scalable 8490H against a workstation Intel Xeon W9-3495X and compare GPU performance against the platforms.
Measuring performance of the RTX 6000 Ada, we can see slight performance gains on the Intel-3495X workstation configuration. We suspect that the Intel Xeon W9-3495, with its higher clock speeds, edges out versus the Intel Xeon Scalable. This means that RELION is fully capable of performing optimally without relying on the CPU too much.
For the RTX A4500 test we further support that claim that RELION works best when clock speeds are higher. The additional cores don’t lend themselves to bring performance advantages to the table. However, in a server configuration, running an 8 and 10 GPU configuration is an option whereas on an Intel Xeon W platform it is not.
GPU and CPU Hardware Recommendations for RELION
RELION, if run with other applications, is usually the most compute-intensive so optimizations should satisfy RELION workloads first. There are various optimizations that can increase or decrease performance, but we want to deliver a general idea of what to look for when configuring your next system.
For CPU, opt for at least 4 cores per GPU. More is better but prioritize high clock speeds than core count. For building a high-performance workstation, opt for an Intel Xeon W9-3495X or an AMD Threadripper PRO 5995WX. Both have ample cores, 56 and 60 respectively, and offer great clock speed performance and I/O for additional hardware. When configuring for a server configuration, dual or single configuration, opt for processors with fewer cores and higher clock speed.
For GPUs, it depends on the budget. According to our tests, 10x RTX A4500 performance is equivalent to 4x RTX 6000 Ada. When we think of the cost per GPU, buying 10 RTX A4500 is still more cost-effective, but integrating that into a server with server components, rack space, and infrastructure, the cost for the platform quickly goes up. Unless you have the data center rack space already available, opting for a 4x GPU workstation is easier to set up, has portability, and better degree of flexibility. However, if your budget constraints are not as tight, the best option would be 8 or 10 GPU configurations with RTX 6000 Ada. The RTX 5000 Ada is also a decent option that was recently released with the RTX 4500 and RTX 4000 following later this fall.
In the end, your workload may not be 100% RELION. Striking a balance between RELION optimization as well as considering your other applications is imperative in figuring out the right components for you.
If you have any questions on building your next high-performance computing solution, Exxact engineers can answer them and provide guidance in choosing the best hardware for the best price at your budget. Explore RELION optimized platforms and solutions and configure your next solution.
Benchmarks
RELION GPU Benchmark for 3D Classification - NVIDIA H100, RTX 6000 Ada, RTX A4500
September 1, 20235 min read
RELION for Cryo-EM GPU Benchmarks Overview
As a value-added supplier of scientific workstations and servers, Exxact regularly provides reference benchmarks in various GPU configurations to guide Cryogenic Electron Microscopy (cryo-EM) scientists looking to procure systems optimized for their research. In this blog, we benchmark the NVIDIA H100, NVIDIA RTX 6000 Ada, and RTX A4500 performance using Relion Cryo-EM, comparing GPU runtime to Total Runtime performance (lower is better) on an Intel Xeon Scalable 8490H platform and an Intel Xeon W9-3495X platform.
Software Summary
RELION (REgularised LIkelihood OptimisatioN), or Relion, has revolutionized the cryo-EM field since 2012. Developed by Scheres Lab at the MRC Laboratory of Molecular Biology, this stand-alone computer program uses a Bayesian approach to refine macromolecular structures by single-particle analysis of electron cryo-microscopy data.
The development of RELION is supported through long-term funding by the UK Medical Research Council and is distributed under a GPLv2 license. This means that anyone (including commercial users) can download, use and modify RELION free of cost. The MRC Laboratory just request that if RELION is useful in your work, you will cite their papers.
Exxact Testbench Specifications
| Xeon W Workstation | Xeon Scalable Server | |
|---|---|---|
| Processor | Intel Xeon W9-3495X | Dual Intel Xeon Scalable 8490H |
| Total Cores | 56 Cores | 120 Cores (60 Each) |
| Base/Max Boost Clock | 1.9GHz/4.8GHz | 1.9GHz/3.5GHz |
| Memory | 512GB DDR5 ECC | 512GB DDR5 ECC |
| Storage #1 | 1.92TB M.2 NVMe SSD | 4.09TB M.2 NVMe SSD |
| CUDA Version | 12.0 | 12.0 |
RELION GPU Benchmarks for 3D Classification - RTX 6000 Ada, RTX A4500, H100 PCIe.
In the first test, we want to visualize the performance gaps between GPUs on the workstation and server platforms. We will compare performance based on CPUs in the next section. Performance numbers are time; a lower score is better.
As expected, the top-of-the-line RTX 6000 Ada performs very well among the other GPUs. It is worth noting the H100 was not designed for HPC and RELION-type workloads; they are more suitable for deep learning training and inferencing.
The RTX A4500 is a crowd favorite in the Life Science industry due to its lower cost and well-rounded performance. However, its age shows in the Intel Xeon Scalable 8490H platform. 4x RTX A4500 performance numbers deliver, at best, 70% of the performance of RTX 6000 Ada on an unoptimized setting 5xMPI (where 9xMPI with 4x GPUs deliver to 60% of the RTX 6000 Ada performance).
With the Intel Xeon Scalable server platform, 8 and 10 GPU configurations are possible. However, 10x A4500s perform just as well as 4x RTX 6000 Ada. This puts into question, the price of 10 RTX A4500 in a 4U server enclosure, or 4x RTX 6000 Ada in a more portable desktop workstation chassis.
RELION CPU Benchmarks with NVIDIA RTX 6000 and RTX A4500
To dive a little deeper into the performance between GPU and CPU platform, we use the same numbers and format them to showcase the CPU performance when GPU is static. We test a dual CPU server configuration with Intel Xeon Scalable 8490H against a workstation Intel Xeon W9-3495X and compare GPU performance against the platforms.
Measuring performance of the RTX 6000 Ada, we can see slight performance gains on the Intel-3495X workstation configuration. We suspect that the Intel Xeon W9-3495, with its higher clock speeds, edges out versus the Intel Xeon Scalable. This means that RELION is fully capable of performing optimally without relying on the CPU too much.
For the RTX A4500 test we further support that claim that RELION works best when clock speeds are higher. The additional cores don’t lend themselves to bring performance advantages to the table. However, in a server configuration, running an 8 and 10 GPU configuration is an option whereas on an Intel Xeon W platform it is not.
GPU and CPU Hardware Recommendations for RELION
RELION, if run with other applications, is usually the most compute-intensive so optimizations should satisfy RELION workloads first. There are various optimizations that can increase or decrease performance, but we want to deliver a general idea of what to look for when configuring your next system.
For CPU, opt for at least 4 cores per GPU. More is better but prioritize high clock speeds than core count. For building a high-performance workstation, opt for an Intel Xeon W9-3495X or an AMD Threadripper PRO 5995WX. Both have ample cores, 56 and 60 respectively, and offer great clock speed performance and I/O for additional hardware. When configuring for a server configuration, dual or single configuration, opt for processors with fewer cores and higher clock speed.
For GPUs, it depends on the budget. According to our tests, 10x RTX A4500 performance is equivalent to 4x RTX 6000 Ada. When we think of the cost per GPU, buying 10 RTX A4500 is still more cost-effective, but integrating that into a server with server components, rack space, and infrastructure, the cost for the platform quickly goes up. Unless you have the data center rack space already available, opting for a 4x GPU workstation is easier to set up, has portability, and better degree of flexibility. However, if your budget constraints are not as tight, the best option would be 8 or 10 GPU configurations with RTX 6000 Ada. The RTX 5000 Ada is also a decent option that was recently released with the RTX 4500 and RTX 4000 following later this fall.
In the end, your workload may not be 100% RELION. Striking a balance between RELION optimization as well as considering your other applications is imperative in figuring out the right components for you.
If you have any questions on building your next high-performance computing solution, Exxact engineers can answer them and provide guidance in choosing the best hardware for the best price at your budget. Explore RELION optimized platforms and solutions and configure your next solution.
