New version of the SPECworkstation® benchmark
enables scaling beyond 64 logical processors

The SPECwpc® subcommittee has reengineered the multi-threading code in version 3.0.4 of the SPECworkstation benchmark, enabling certain computational workloads to scale beyond 64 logical processors.

The SPECwpc chair Alex Shows explains how the update came about and what it means for the future of the industry-standard SPECworkstation benchmark.

Balancing goals

One of the great strengths of a benchmark such as the SPECworkstation benchmark is that you can measure real-world performance without requiring the application to be installed on the machine being measured.

Another strength is comparability: the design, implementation and maintenance of a great benchmark ensures that results are comparable across multiple generations of hardware for different system vendors and technology providers.

Balancing these goals creates a challenge for benchmark developers such as SPEC/GWPG: We have to weigh the importance of maintaining relevance and correlation to real-world application performance with the desire to provide a long-lived benchmark with comparability across many hardware refresh cycles.

Fortunately, the SPEC/GWPG membership includes technically accomplished people to help with benchmark updates. In the case of this new version of the SPECworkstation benchmark, the bulk of the work was done by Anshu Arya, SPECwpc vice chair and director of compute research.

Enabling processor groups

With version 3.0.4 of the SPECworkstation benchmark, we maintain comparability with previous versions of the benchmark while incorporating a Microsoft feature called processor groups. Microsoft defines this as “a static set of up to 64 logical processors that is treated as a single scheduling entity.” You can read more about it on the Microsoft website.

With the processor groups patch, a configuration with more than 64 logical processors will see higher scores in most of the computational workloads due to improved partitioning and affinitization. Processor affinity enables the binding and unbinding of a process or a thread to a CPU or a range of CPUs, so that the process or thread will execute only on the designated CPU or CPUs.

When running the SPECworkstation benchmark with the 3.0.4 patch applied, you should see higher CPU utilization for these very high core count configurations, as well as better symmetry when distributing work across multiple sockets.

While the vast majority of workstations currently have fewer than 64 logical processors, ISVs are working to enable users to take advantage of systems with very high core counts. The SPECwpc subcommittee wants to be proactive in addressing this movement.

Available as patch or full download

The SPECwpc subcommittee decided to launch this update in two forms: as a minor-version patch to the benchmark for those already using version 3.0.3, and as a full benchmark installation. If your installed version is lower than 3.0.3, the full benchmark installation package should be used instead of the patch. Whether you download the full V3.0.4 benchmark or just the patch, you will retain comparability between 3.0.4 and all previous 3.0 versions.

To install the patch, extract the contents of the downloaded zip file into an existing SPECworkstation 3.0.3 installation folder, overwriting files when prompted. Users can verify their installation of the SPECworkstation benchmark  has been patched successfully by confirming that the version number shown in the application title bar and the results files matches “3.0.4”.

The SPECworkstation 3 benchmark submissions on the SPEC website results page will include the minor version information in order to disclose up front what version was used to obtain a published result.

More improvements to come

Meanwhile, the SPECwpc subcommittee is working on some minor changes to the benchmark that include UI and business logic improvements and better error handling in both configuration information gathering and workload execution. When we’ve reached a critical mass of improvements, we’ll determine whether a new version should be 3.0.5 or 3.1. If the update has an impact on the performance measurements of the vast majority of user configurations, it will be released as a 3.1 version and not be backward-comparable to 3.0 results.

As with everything we do in SPEC/GWPG, we welcome feedback from users and paid licensees of our benchmarks.

###

For more information on the SPECworkstation 3 benchmark, visit the information page or go directly to the download page to download the benchmark and/or the new V3.0.4 processor groups patch.

SPEC/GWPG benchmarks are free for the user community and available for a licensing fee to sellers of computer-related products and services. SPEC/GWPG members receive benchmark licenses as a membership benefit.