SPEC Power and Performance Committee

THIS VERSION IS OUT OF DATE.   SEE WIKI FOR UPDATED VERSION!

The SPEC Power and Performance Committee is developing the first generation SPEC benchmark for evaluating the energy efficiency for server class computers.

The drive to create the power and performance benchmark comes from the recognition that the IT industry, computer manufacturers, and governments are increasingly concerned with the energy use in servers. Currently, many vendors report some energy efficiency figures, but these are often not directly comparable due to differences in workload, configuration, test environment, etc. SPEC wants to define server energy measurement standards in the same way we have done for performance. Development of this benchmark provides a means to measure energy use in conjunction with a performance metric. This should help IT managers in their acquisitions to consider power efficiency along with other selection criteria to increase the efficiency in data centers.

Some of the challenges include defining the scope we expect to address, selecting workload(s), defining test conditions, specifying a measurement protocol, and providing for extension of the metric to a wider scope later.

Benchmark Goals

The committee develops a means to fairly and consistently report system energy use under various usage levels.

The initial benchmark addresses one subset of server workloads; the performance of a server side Java. It exercises the CPUs, caches, memory hierarchy and the scalability of shared memory processors (SMPs) as well as the implementations of the JVM (Java Virtual Machine), JIT (Just-In-Time) compiler, garbage collection, threads and some aspects of the operating system.

The benchmark needs to run on a wide variety of OS and hardware architectures and should not require extensive client or storage infrastructure. Also the usability needs to be high, the setup relatively easy, and the run-time needs to be reasonable.

The energy measurements are made at the AC input to the system under test.

Current Status

We have reached agreement on the major issues, are testing benchmark prototypes, and are making good progress towards a benchmark release planned for the second half of 2007. At this time, we can not commit to any target date because if during our evaluation phase unexpected data shows a benchmark anomaly, that might risk SPEC's overriding goal of fair comparison. A hold on the release will be implied in order to correct this issue.

• Minimum power meter requirements (Draft)

SPEC and EPA's Energy Star Enterprise Server and Data Center Energy Efficiency Initiatives

• EPA Technical Workshop on Energy Efficient Servers and Datacenters in the U.S. (February 16, 2007, Santa Clara, CA)
  • EPA Study on Server and Datacenter Energy Efficiency
  • SPEC at the EPA Workshop
• Energy Efficiency Server Benchmark Technical Workshop (March 27, 2006, Santa Clara, CA)
  • Server Efficiency Benchmark Development Effort
  • SPEC at the EPA Workshop

Committee Membership

Current SPEC member companies committed to developing a power and performance measurement standard include AMD, Dell, Fujitsu Siemens Computers, HP, Intel, IBM, Sun Microsystems, SPEC Associates, and Lawrence Berkeley National Laboratory.

SPEC welcomes organizations to join and participate in our work, please contact info@spec.org for additional information.