SPEChpc™ 2021 Benchmark Suites

HPC systems are getting built with an increased level of heterogeneity. The numerous types of accelerators bring in tremendous extra computing power, while at the same time introduce big challenges in performance evaluation and characterization. More complications are added to the problem when multiple parallel and accelerator programming models have been developed with each only supporting a subset of the computing devices.

The SPEChpc™ 2021 Benchmark Suites address these challenges by providing a set of application benchmark suites using a comprehensive measure of real-world performance for the state-of-the-art HPC systems. They offer well-selected science and engineering codes that are representative of HPC workloads and are portable across CPU and accelerators, along with certain fair comparative performance metrics.

An incremental update to the benchmark suite, V1.1, was released on July 14, 2022. It is primarily a bugfix release. Results generated with V1.1 using pure-MPI and OpenMP are comparable to results from V1.0 and vice versa. However, a few changes do effect performance of the OpenACC and OpenMP with Target Offload versions making them non-comparable to v1.0. These performance relevant changes were made to ensure better comparability between the two offload models. More details are available in the Changes in V1.1 document.

The current version of the benchmark suite is V1.1.9, released in September 2024, which is primarily a bug fix release.

To upgrade to the latest version of the SPEChpc v1.1 benchmark, current users can contact the SPEC Office at info@spec.org or use runhpc's update feature.

Benchmark Features

The four suites, Tiny, Small, Medium, and Large, include groups of full applications or mini-apps covering a wide range of scientific domains and Fortran/C/C++ programming languages. Each suite uses increasingly larger workloads to allow for appropriate evaluation of HPC systems at different sizes, ranging from a single node to hundreds of nodes.

All codes in the suites support multiple programming models, including MPI, MPI+OpenACC, MPI+OpenMP, and MPI+OpenMP with target offload, for measuring performance on different platforms.

Most major compilers, MPI libraries, and different flavors of Linux operating systems are supported.

Results

Submitted Results
Results for all the SPEChpc™ benchmark metrics; includes all of the results submitted to SPEC from the SPEC member companies and other licensees of the benchmark.

Documentation

• Documentation overview
• System requirements
• Installation guide
• Explanation of the fields in a result disclosure
• Run and reporting rules
• Known issues
• Requesting technical support

Related Material

Publications based on the SPEChpc benchmark suites, as well as other SPEC HPG suites, can be found here.