HPG Develops SPEChpc96 Benchmark Suite

by Siamak Hassanzadeh
Sun Microsystems
Palo Alto, Calif.

• The Metrics for Measurement
• Vendor Reporting
• Availability

The Standard Performance Evaluation Corp. (SPEC) announced the release of SPEChpc96, the first standard benchmark suite for measuring high-performance computing. SPEChpc96 was developed by SPEC's High Performance Group (HPG), which includes several leading high-performance computer vendors, systems integrators, and major universities and research institutes.

SPEChpc96 allows high-end computer users and vendors to compare performance objectively across different hardware platforms. "This benchmark suite is a major step in our mission to provide a level playing field for making performance comparisons based on real-world, high-performance computing applications," says Siamak Hassanzadeh, SPEC's HPG Chair.

Benchmarks within the SPEChpc96 suite represent specific industrial and scientific applications. The first two benchmarks are SPECseis96, a seismic processing application, and SPECchem96, a computational chemistry application.

SPECseis96 is based on a benchmark suite originally developed at Atlantic Richfield Corp. (ARCO). It is an industrial application representative of modern seismic processing programs used in the search for oil and gas. SPECchem96 is based on GAMESS (General Atomic and Molecular Electronic Structure System), an improved version of programs that came from the Department of Energy's National Resource for Computations in Chemistry. Many of the functions found in GAMESS are duplicated in commercial packages used in the pharmaceutical and chemical industries for drug design and bonding analysis. Both SPECseis96 and SPECchem96 incorporate a wide range of problem sizes.

As SPECseis96 and SPECchem96 can be run in serial and parallel modes, SPEChpc96 can be used for performance comparisons over a broad range of high-performance computing systems, including multiprocessor systems, workstation clusters, distributed memory parallel systems, and traditional vector and vector parallel supercomputers.

The Metrics for Measurement

The SPECseis96 and SPECchem96 suites each generate four metrics representing different problem size to characterize the scalability of the application as well as the system. The SPEChpc96 has the following metric names:

• SPECseis96_SM
• SPECseis96_MD
• SPECseis96_LG
• SPECseis96_XL
• SPECchem96_SM
• SPECchem96_MD
• SPECchem96_LG
• SPECchem96_XL.

The metrics are unitless. They are derived as follows:

metric = (86400 seconds) / (elapsed time of benchmark in seconds)

As these benchmarks are both compute-intensive and data-intensive, the above metrics reflect the performance of the entire system, including processors, memory access, I/O bandwidth, interconnect topology, etc. For example, the SPECseis96_XL requires processing of 100 GBytes of data.

Vendor Reporting

Initial results for systems from some vendor companies were presented at the Supercomputing '95 Conference, in San Diego, Calif. Additional results will be reported in the next issue of the SPEC Newsletter, scheduled for publication at the end of March 1996.

Availability

SPEChpc96 (SPECseis96 and SPECchem96) is available from SPEC's administrative office. The cost is $1,200.

SPEC/HPG was founded in January 1994 to establish, maintain and endorse a suite of benchmarks representative of real-world, high-performance computing applications. Current SPEC/HPG members include Convex Computer Corp., Cray Research, Digital Equipment Corp., Electronic Data Systems (EDS), Fujitsu America, Hewlett-Packard, International Supercomputing Technology Institute (ISTI, France), Kuck & Associates, NEC/HNSX Supercomputers, PARKBENCH (University of Tennessee), Silicon Graphics, Sun Microsystems, the University of Illinois, the University of Michigan, and the University of Minnesota.

Dr. Siamak Hassazandeh is with Sun Microsystems, Inc. in Palo Alto, Calif.

Copyright (c) 1995 Standard Performance Evaluation Corporation