g SPECpower_ssj2008 - Frequently Asked Questions (FAQ) at 30. May, 2012

gMany common questions are addressed in this FAQ or in the benchmark documentation on the SPECpower_ssj2008 Information (http://www.spec.org/power_ssj2008) web page.

This version of the FAQ is aligned with the 1.12 version of the SPECpower_ssj2008 benchmark.

For all other questions, send your query via e-mail to: support@spec.org with "SPECpower" in the subject line.

g About the Benchmark

g
1. Why did SPEC choose a server side Java workload as a basis for the SPECpower_ssj2008 benchmark?
   • We chose the workload that best met the following requirements:
     • Scalable, Multi-Threaded
     • Portable across a wide range of operating environments
     • Minimal Cost to Run ($$ and time)
2. Are the results of SPECpower_ssj2008 comparable to any other benchmarks?
   • No, they are not.
   • Several substantive changes have been made that make the "ssj2008" workload(s) unique and different from all other benchmarks.
3. Where can I find the documentation?
   • Documentation is provided with the benchmark release in the "\doc" directory.
   • See below (#'s 19-20) for more on documentation
4. Are updates to documents available?
   • Yes, the most recent documents are on the SPEC website on the SPECpower_ssj2008 Information (http://www.spec.org/power_ssj2008) web page.
   • Look on the left side of the page in the navigation bar.
5. What is a "compliant" run of SPECpower_ssj2008?
   • A compliant benchmark run meets all the requirements of the SPECpower_ssj2008 run rules for a valid run that could be submitted to SPEC for publication on the SPEC web site.
   • In addition to the run and report rules, several validation and tolerance checks are built-in to the benchmark.
   • If you intend to submit results to SPEC and/or publicly use the SPECpower metric "overall ssj_ops/watt", the run must be compliant.
6. What is "FDR"?
   • FDR is an acronym for "Full Disclosure Report".
   • The FDR is the document that represents a full benchmark run
7. Can I "turn off" compliance checking? My results are only non-compliant because I do not have a temperature sensor.
   • No. This is to prevent a Full Disclosure report being used that is or might be non-compliant.
8. What is a "graduated workload"?
   • A graduated workload operates at several levels, usually stepping up or down by some regular value.
9. Why did SPEC choose to implement a graduated workload?
   • Server power consumption has been shown to be significantly different at low load levels versus high load levels, and server loads vary widely from hour to hour and day to day.
   • Since no definitive surveys exist, and all data centers are different, SPEC has chosen to provide graduated loads across a set of load levels to portray performance and power consumption in several reasonable ranges.
10. How long does it take the run the benchmark?
    • The run time of the 'workload' is approximately one and one-quarter hours (~70 minutes) with default settings.
    • Here's how it breaks down:
    • There are 10 load levels, an idle measurement period, and three calibration levels; each run for 240 seconds (4 minutes).
    • Between each load level there is a 10 second period of zero activity
    • Each load level has a ramp-up and a ramp-down period, each 30 seconds.
    • (240+30+30+10)* 14 = ~75 minutes
11. How did SPEC determine the increments of the load levels?
    • Load levels are graduated by increments of 10%.
    • This choice was a compromise between too few load levels and too many.
    • Increments of 5% would require 20 levels and double the run-time.
    • Five would be too few.
12. Can I run other increments?
    • Yes. For non-compliant runs, you may run as few or as many as your needs dictate.
    • The number of load levels is set by an entry in the SPECpower_ssj.props file.
    • See the FAQ on "expert.props"
13. Is there an upper limit to the number of load levels I might run?
    • We believe that hundreds of load levels are possible but this capability has not been tested. 50 levels (2% increments) have been run successfully.
14. Can I run just one or two load levels?
    • Yes. The number of load levels is set by an entry in the SPECpower_ssj.props file.
    • See the FAQ on "expert.props"
15. What is "active idle"?
    • Idle is, generally speaking, the state when the system is not running any applications or performing any operating system management tasks. CPU utiliztion is zero.
      • We could label this state "OS idle".
    • The duration of idle states can vary from fractions of seconds to minutes.
    • Modern operating systems run many asynchronous background tasks and therefore most servers are never totally idle for long periods.
    • "active idle" is a SPEC defined state where an application is running and no transactions are incoming or in process; the system is ready to quickly respond to any incoming transactions.
16. How is active idle measured?
    • Active idle is "the eleventh load level" of the ssj_2008 graduated workload.
    • Performance is zero and power is measured at what is usually the lowest rate of consumption.
    • This load is treated the same as all others in computing the SPECpower_ssj2008 overall ssj ops / watt.
17. How long is the active idle period?
    • The duration of the active idle load level in the case of compliant (default) runs, is 240 seconds, the same as all other load levels.
    • In the SPECpower_ssj2008 Run and Reporting Rules document, active idle is documented in section 2.3.2.1.
18. Can I acquire and run this benchmark in my own facility?
    • Yes. See "Acquiring the Benchmark" below.
19. Does the SPECpower_ssj2008 benchmark support measuring systems with DC-DC power supplies? (DC=Direct Current)
• No.
• DC powered servers can not be compared to AC (Alternating Current) powered servers because the AC to DC power conversion cost (energy loss) is not present in DC powered servers.
• Although physically possible, SPEC has neither developed nor tested methods for DC system measurement.
20. Is there a "Users Guide" that walks me through set up, run and results interpretation?
    • Yes, that document is the "SPECpower_ssj2008 Users Guide". See below
21. Can I get documentation without buying the benchmark?
    • Yes, the Users Guide and other documents are available on the SPEC web site on the SPECpower_ssj2008 Information (http://www.spec.org/power_ssj2008) web page.
22. What are the key documents I should read before I get started?
    1. SPECpower Methodology
    2. SPECpower_ssj2008 Users Guide
    3. The set of Design Documents
    4. FAQ (this document)
    5. Run and Reporting Rules
       • if submitting results to SPEC or,
       • if making public comparisons using SPECpower benchmark results
23. What operating system environments are supported?
    • SPECpower_ssj2008 software elements have been successfully tested in several common operating environments (see "Software Requirements" below).
      • Find a list of tested environments at www.spec.org/specpower
    • CCS requires Linux, Solaris or Windows operating environment (see below).
24. In what language is the benchmark written?
    • Much of the code is written in Java.
    • The SPEC PTDaemon (Power Temperature Daemon) code is C++
25. Why is PTDaemon in C++ while all the other code is Java?
    • The PTD code must manage and then read and write to serial ports and therefore uses low level constructs not generally available in Java.

Hardware Requirements

g
1. What is the minimum hardware required?
   • At minimum, you will need two network connected systems and one power analyzer.
   • A temperature sensor is required for fully compliant runs.
   • One is the SUT or System Under Test, the other the CCS or Control Collect System.
   • For compliant runs, a power analyzer and a temperature sensor accepted by SPEC are required, plus full adherence to the benchmark run rules.
2. How much computer equipment do I need to run the SPECpower Benchmark?
   • The minimum equipment required is two computers and one power analyzer.
     • A temperature sensor is also required for compliant results.
   • Computer One: The SUT (System Under Test)
     • the system for which you wish to measure power and test performance.
     • As with most benchmarks, this machine runs the "workload"
   • Computer Two: A "CCS" (Control, Collect System) which collects and logs the performance and power data.
     • The CCS system can be a modern notebook or desktop computer with a minimum of 1GB memory.
     • The two systems must be networked via LAN (TCP/IP communication is used).
     • A power analyzer data transfer cable is attached (typically RS232, USB or GP-IB) to the CCS system.
   • Power Analyzer: A device that meets the current 'specifications' and with data logging capability (see "Power Analyzers" below)
3. What is the recommended physical memory size (how many MB/GB) for a test system (SUT) to run SPECpower_ssj2008 - with good performance?
• Several factors determine the amount of memory required for best performance.
• Processing capacity, usually expressed as "the total number of cores" or logical processors and the number of processor sockets - usually the number of CPUs (not cores).
• The number concurrent JVM instances is another factor.
• Good practice is to check the published SPECpower_ssj2008 reports for many common server platforms at this link: SPECpower_ssj2008 Results Reports (http://www.spec.org/power_ssj2008/results/power_ssj2008.html)
• In each report (FDR), you will find hardware and software configuration details. In the software configuration section you will find the number of JVM instances used and the JVM options used for the platform.
• Very simple memory size formulas:
• For a single socket system (one processor): Minimum 4GB:
• ~2GB for the Operating system and ~2-3GB for the JVM.
• For a two socket system (two processors):
• ~2GB for the Operating system and ~4-6GB for two JVM instances.
• More memory is usually better for performance. However, depending on your system, there is a memory size (heap size per JVM) beyond which performance is likely to increase very little.
• Consult with the vendor of your JVM.
• SPEC can not offer specific tuning recommendations.
4. How much does a power analyzer cost?
   • A quality power analyzer can cost more than $2,000(USD).
   • See Power Analyzers section below.
5. Can I run without a power analyzer?
   • Yes, for non-compliant runs. The SPEC PTDaemon (Power Temperature Daemon) has a device type "dummy" for these occasions.
   • The absence of power data makes computing a power-performance metric impossible.
6. Can I run without a temperature sensor?
   • Yes, for non-compliant runs. The SPEC PTDaemon (Power Temperature Daemon) has a device type "dummy" for these occasions.
   • For more information, see the FAQ section on Temperature Sensors.

g Software Requirements

1. What Operating Systems are supported?
   • On the SUT side, the workload will run on any OS for which a JVM is available.
   • On the CCS side, the CCS software and the PTDaemon has been tested on common Linux distribution, Microsoft Windows XP and Server 2003/2008 (32 and 64 bit), and Sun Solaris.
     • The power and temperature measurement devices you choose may or may not have drivers for your intended operating environment. Best to check.
2. What other software will I need?
   1. A Java virtual machine (JVM).
      • The workload and the CCS are written in Java to enable the highest degree of portability across software operating environments.

g Acquiring the Benchmark

1. How much does the benchmark cost?
   • The cost of the benchmark and the order form are on the web at: http://www.spec.org/order.html
2. How do I order the benchmark?
   • You will find an order form at: http://www.spec.org/order.html

g Contacting SPEC

1. I have a question or problem that isn't covered in the the documentation.
   • Send your query via e-mail to: support@spec.org with "SPECpower" in the subject line.

g Running / Troubleshooting the SPECpower_ssj2008 Benchmark

g Set Up and Starting

1. What do I have to change just to get started?
   • See the document "Quick Start Guide"
2. How many JVM instances should I use?
   • Typically, with 1 to four cores per processor socket (or package), best parallelism is achieved with one JVM per socket.
   • Each JVM instance will require a dedicated Java "heap"
   • Good practice is to look at published SPECpower_ssj2008 reports for many common server platforms at: SPECpower_ssj2008 Results Reports (http://www.spec.org/power_ssj2008/results/power_ssj2008.html)
3. What do the terms "Xmx" and "Xms" mean?
   • They are Java command line options that control Java heap size.
   • "Xms" is used to specify the minimum heap size
   • "Xmx" is used to specify the maximum heap size
   • For further information consult with the JVM vendor.
4. How important are the options used with the JVM in the JAVAOPTIONS_SSJ environment variable in the runnssj.bat (Windows) or runssj.sh (Linux) file?
   • Java options can be critical to the efficient operation of Java on a particular hardware platform.
   • We have seen performance difference of up to 50% between zero options and a full set of heap size and optimization flags.
   • Good practice is to look at published SPECpower_ssj2008 reports for many common server platforms on the SPECpower_ssj2008 Results Reports (http://www.spec.org/power_ssj2008/results/power_ssj2008.html)
   • Look for "JVM Command Line Options" in the Software Configuration section.
   • Note that options used with one brand of JVM are NOT expected to work with another brand, and sometimes not with a different version of the same JVM.
   • Options prefixed by "-X" are not standard.
   • Consult with the vendor of your JVM.
   • SPEC can not offer specific tuning recommendations.

g Controller System

1. Can I use a laptop or desktop system for the Controller System?
   • Yes; with either type of system, on Windows OS, make sure to:
     1. set your Power Scheme to "Maximum Performance" and,
     2. screen saver to "none" and,
     3. set "Power Options" (plugged in) to System Standby"Never"
2. What are the minimum system requirements?
   • Most modern machines are suitable
   • Minimum processor speed of ~1GHz,
   • Minimum 1GB memory,
   • Ethernet network capability with TCP/IP installed,
   • ~10MB free disk space for code and log files,
   • No special graphics requirements.
3. Can I change the names of properties files used by the benchmark?
• Yes. You will need to specify the particular property file name in the script files either through an environment variable, a command line parameter, or both (depends on the script construction).
4. Does CCS have command line parameters?
• Yes. One; the name of the CCS properties file.
• The default name is "ccs.props" and is not required to appear on the command line.
• If you choose to use another name, simply add that name to the command line that starts CCS (assuring there is only one filename).
5. Is the file SPECpower_ssj_EXPERT.props used in the benchmark?
• No. This file contains 'properties' that can change the default behavior of the benchmark.
• For instance, the number of load levels, duration of load levels, delay between load levels, idle period settings and many other options.
• Use these parameters with caution.
• Chosen parameters can be inserted into your controlling properties file.
• Changes from the defaults will produce warnings in the reports.
• Results from runs where defaults were changed are not compliant and should not be considered comparable to compliant runs.
6. Can I change the duration of the load levels? (from the default of 240 seconds)
• Yes.
• Add the line below to your "SPECpower_ssj.props" file to set the length of each load level to 60 seconds
• input.load_level.length_seconds=60
• Idle and Calibration load levels are controlled by other properties.
  • Load level durations of less than 30 seconds are not recommended.
• This and other changes to the defaults can be found in the file
"SPECpower_ssj_EXPERT.props" in the "\ssj" directory.
• Results from runs where defaults were changed are not compliant and should not be considered comparable to compliant runs.

g Power Analyzers

1. Is a power analyzer required to run the benchmark?
   • For compliant runs, yes.
   • For any other runs, the SPEC PTDaemon (Power Temperature Daemon) has a device type "dummy" for these occasions.
   • You may also remove the power analyzer data source from the ccs.props file.
2. Can the SPECpower_ssj2008 benchmark support multiple power analyzers?
• Yes. Multiple power analyzers and multiple temperature sensors are supported.
• Data collection devices are defined in the "ccs.props" file. The default ccs.props file contains an example of defining a second power analyzer.
• See the "CCS Design Guide"; the section on "Data Sources" provides instructions: CCS Design Document (http://www.spec.org/power/docs/SPECpower_ssj2008-Design_ccs.pdf)
• Note that an instance of the SPEC PTDaemon is required for each power analyzer or temperature sensor.
• Each PTDaemon instance must have a TCP/IP port number that matches that defined in the ccs.props file.
3. What is the difference between *tested* and *accepted* power measurement devices?
• Tested devices have operated successfully with the PTDdaemon. Not all are accepted for compliant benchmark runs. They are available for _research_ purposes.
• SPEC Accepted devices have been qualified by SPEC for use in producing compliant benchmark reports and submissions to SPEC. These devices have passed the Power Analyzer Acceptance Test.
4. What power analyzers have been *accepted* for use with the SPEC Power Benchmark?
   • SPEC Accepted power measuring devices are listed on the SPEC web site at: Accepted Measurement Devices (http://www.spec.org/power/docs/SPECpower-Device_List.html)
   • This same list will be displayed by the PTDaemon if run with no parameters; the PTDaemon will also display "help" text.
   • Under "Power Analyzer device types" are listed those power analyzers compliant with the SPEC accuracy and functionality requirements.
   • Run PTDaemon with the -u parameter (-u = unsupported) to see the complete list of power and temperature analyzers (and help text).
5. What power analyzers have been tested with the SPEC Power Benchmark?
   • A list of SPEC Accepted (supported and compliant) power measuring devices are documented on the SPEC web site at: Accepted Measurement Devices (http://www.spec.org/power/docs/SPECpower-Device_List.html)
   • The PTDaemon program can be used to display a list of power and temperature measuring devices along with their device codes;
   • For SPEC Accepted devices: Run the PTDaemon with the -h parameter, or zero parameters.
   • Listed devices (except 0/1000. Dummy) are valid for compliant runs of the benchmark.
   • To view a list of SPEC Accepted (supported) devices and unsupported devices, run the PTDaemon program with the -u parameter (-u = unsupported).
   • Listed devices annotated with an * are:
     • unsupported and not valid for compliant results
     • used with caution
6. Will my power analyzer work with the SPECpower PTDaemon even though my device does not appear on the accepted devices list shown on the SPEC web site at Accepted Measurement Devices (http://www.spec.org/power/docs/SPECpower-Device_List.html)?
   • No, only devices on the SPEC Accepted Device List are expected to produce valid results.
   • If your device is not present on the devices list, it is unlikely that PTDaemon will successfully read data.
   • Communication protocols are quite different among the various manufacturers and models of power analyzers.
7. Will the SPECpower PTDaemon (Power Temperature Daemon) connect to and read my power analyzer even though my device is not on the tested list?
   • It is unlikely that PTDaemon will successfully read a meter not on the "tested" list.
   • See the responses to the FAQ just above.
8. How can I get my power analyzer included in the SPEC accepted devices list?
   • The process for getting a new measurement device accepted for use with SPECpower_ssj2008 is described briefly in section 2.13.5 of the run rules.
   • Basically, documentation must show that the device has sufficient accuracy, code must be written to enable SPEC's PTDaemon software to read measured values from the device, and then testing must be performed to show that the device does meet SPEC's requirements in actual usage. Even if the device doesn't meet the SPEC requirements, it may still be possible to add software support such that the device could be used for internal (non-publishable) measurements.
9. I have (or plan to buy) a multi-channel power analyzer. Can I use all three channels with the SPECpower benchmark?
• No. Multiple simultaneous independent channel operation (reads from) is not yet supported due to the design of the power analyzers.
• This feature is planned for a future release.
g
10. Will a power analyzer calibration certificate for one device apply to other power analyzers of exactly the same model?
• No. The calibration certificate issued is valid only for the specific device (one serial number) calibrated.
• A recently calibrated power analyzer is required to produce and report compliant results. "Recent" means that the calibration certificate must be dated within one year of the test date.
11. Is the test certificate I received with my instrument sufficient to meet the calibration requirements of the power analyzer requirements section of the run rules?
• If the certificate is entitled "Certificate of Calibration" or "Certificate of Traceable Calibration" and includes a unique calibration number or sticker, lab accreditation to ISO or other standards, and test equipment certification with traceability to a national standards institute, it probably meets the requirements. National standards institutes include NIST (USA), PTB (Germany), and NML (ROC - Taiwan) among others.
• See the SPECpower_ssj2008 "Run and Reporting Rules" section 2.13.2
• Power Analyzer Requirements (http://www.spec.org/power/docs/SPECpower_ssj2008-Run_Reporting_Rules.html#2.13.2)
12. I can not change the voltage and current ranges once PTDaemon has been run with a Yokogawa WT210 power analyzer.
• Once PTDaemon 'connects' to the WT210, the front panel is 'locked' to avoid inadvertent changes during a SPECpower_ssj2008 run.
• To re-enable the front panel,
• Exit PTDaemon
• On the WT210 front panel, press the Stop (integrate) button then the Reset button.
• Front panel operations should be enabled.
13. Do I have to configure the power measurement device manually?
    • This depends on the device brand and/or model.
    • Some devices need to be configured manually via the front panel; for example communication settings or current and voltage ranges.
    • Other devices will be controlled by the SPEC PTDaemon software
    • For more details on device setup, see the document Measurement Setup Guide (http://www.spec.org/power/docs/SPEC-Power_Measurement_Setup_Guide.pdf)
14. No data cable came with my power analyzer (RS-232), what now?
• Each brand/model power analyzer may require a different cable. some experimentation might be required.
• Some power analyzers will work with a "null modem" cable.
• One combination that has worked with some devices is a standard "Modem" cable with the addition of a "NULL MODEM" adapter.
• The above combination can also be plugged into a RS-232 to USB converter.
• Some data cable information can be found in the Measurement Setup Guide (http://www.spec.org/power/docs/SPEC-Power_Measurement_Setup_Guide.pdf)
15. My power analyzer supports only RS-232 for data logging; my control system has no serial ports; neither DB-9 nor DB-25.
    - What do I do?
    • There are devices that convert RS-232 to USB, that cost $10-14 (US$).
    • Several brands of serial to USB converters have been found to work well.
    • With Windows, install the software per manufacturer instructions.
    • Then, use the device manager to see what "COM" port number was assigned.
    • Edit the runpower.bat file; change the default DEVICE_PORT number.
    • Note that the COM port assignment on your system may change if you disconnect and re-connect a device or change USB connections.
16. The PTDaemon software can't connect to my power analyzer.
    - What is wrong?
    • First, assure the analyzer is communicating on the data cable with the host machine.
      • The best way to verify this is to first use the vendor supplied software to read the analyzer.
    • Most problems can be resolved with this method, problems such as cable not connected, wrong cable, incorrect device settings and so on.
    • If the vendor supplied software shows the correct readings, assure that the following are correct:
      • In the ccs.props file; the IP address or hostname - usually "localhost"
      • In the ccs.props file the network (TCP/IP) port number must match that in the "runpower.bat" file.
      • The default is NETWORK_PORT=8888 (for power analyzer 1)
      • The analyzer device type used when the PTDaemon is started.
      • The default is DEVICE=0 (the dummay power analyzer)
      • If your problems persist, send e-mail to SPECpower support at this e-mail address: support@spec.org with a subject of SPECpower.
17. Why does my power analyzer display unexpected watts readings?
    • This is likely to be traced back to how the analyzer is connected or the device settings.
    • More details on device setup can be found in the Measurement Setup Guide (http://www.spec.org/power/docs/SPEC-Power_Measurement_Setup_Guide.pdf)
    • Instructions will also be found in the power analyzer documentation or on the vendor web site. You may also contact the device vendor for further technical support.
18. Why do I see -1 (or -2) in the watts column in the ssj.nnnn.ccs-log.csv log file?
    • This indicates an error acquiring data.
    • -1 indicates the data is unreadable
    • -2 indicates a communication or timing error
19. What is wrong when I see very very large numbers recorded for watts (I have a Yokogawa WT210 Power Analyzer)?
• Your analyzer is in the "wrong mode".
• Reset your analyzer to factory default settings by following these steps:
• To reset the Yokogawa WT210 to initial values:
  1. turn the device off.
  2. press and hold enter key (on front panel), while powering on the analyzer, holding the enter key down until errors display (which is normal) finishes.
  3. done.
• Next it is best to assure the RS-232 (serial) communications are set up properly .
• This is not required for GPIB devices
• Yokogawa WT210 Power Analyzer - Front Panel Setup
  1. Press and release the “local” key
  2. in display window “C” use the Up/down arrow keys to select 488.2; Press enter
  3. In display window “A” use the Up/down arrow keys to select “hand 0”; Press enter
  4. In display window “B” use the Up/down arrow keys to select “For 0”; Press enter
  5. In display window “C” use the Up/down arrow keys to select “b 9600”; Press enter
  6. In display window “C” use the Up/down arrow keys to select “CR”. Press enter
20. What is wrong when I see errors relating to uncertainty not being calculated?
    You may have autoranging enabled on your analyzer, which is not supported and may not be selected.

g Temperature Sensors

1. How do I run without a temperature sensor?
   • The PTDaemon (Power Temperature Daemon) has a sensor type of "dummy".
     • this is the best temporary fix if you will add or use a sensor in other runs.
     • The device code for "dummy" temperature sensor is 1001.
   • You may also remove the temperature sensor data source from the ccs.props file.
   • comment the lines that define the temperature sensor (# as first character)
2. What temperature sensors have been tested with the SPEC Power Benchmark?
• A list of successfully tested and compliant temperature measuring devices are listed on the SPEC web site on the Accepted Measurement Devices (http://www.spec.org/power/docs/SPECpower-Device_List.html) page.
3. How do I find the "device code" for a temperature sensor?
• Run the PTDaemon program with zero command line parameters or -h.
• A list of temperature sensors and power analyzers will be displayed, along with the 'help' information.
4. Help! My Digi Watchport/H meter worked fine last week, this week I get this set of errors from the power daemon.
   -- Serial Port Create File failed with error 2, serial port open failed with error 0, Failed to connect to meter
   -- When I look at the Ports with the device manager, I see this message in the box on the "general" tab: Windows cannot identify this hardware because it does not have a valid hardware identification number. (Code 9)
   --- FYI, Digi Watchport driver is version 4.2 for Windows XP.
   • Answer:
   • Unplug the Watchport USB connection, count to 10, plug it back in.
   • This problem sometimes occurs after the system has been re-booted.
   • At this time (27 Aug '07), we are not sure the root cause.

g General Troubleshooting

1. CCS terminates with error such as: Can't find "SCmain".
   • This problem is most likely due to an issue with resolving the path to the JVM.
   • Check the path to "java.exe"
2. What if the PTDaemon can't connect to the power analyzer?
   • See FAQ in the Power Analyzer section
   • Check COM port and the analyzer type passed to the power daemon (ptd_xxxx.exe)
   • Assure the analyzer is connected via serial or USB cable and the analyzer is powered on.
3. One of the benchmark components is "waiting for connection" - forever.
   • Check the IP address of the "director" in the ccs.props file.
   • On the SUT, check the IP address of the "director" in the runssj.bat (or .sh) script file
   • This environment variable: set DIRECTOR_HOST=192.168.1.2
   • The CCS system and the SUT system should be able to 'ping' each other.
   • Check the IP address and TCP/IP port number in the ccs.props file and assure it is the same port number that is passed to the PTDaemon (runpower.bat).
     • In most cases the CCS software and the PTDaemon will be running on the same system. There should be no changes made to the ccs.props file in this case.
   • Kill or terminate all the SPECpower software and re-start.
4. The Command window immediately closes (Windows) when I execute a batch file (by double clicking on the file name).
   - How do I debug this?
• If you execute a runxxx.bat by double clicking on the file name , a command window will appear then close upon program termination.
• If an error message is generated, it disappears with the command windows.
• Instead, first open a command window (using a script file as suggested below) then, in that window, enter runxxx.bat.
• The window will stay open and allow examination of the activity.
5. How can I quickly open a command window in the current directory?
• One easy way is to create a script file in each directory in which you need to open command windows.
• On Windows systems put the following (example) in a script file:
• start "- SSJ -" cmd
• The above will open a command window in the directory from which the script file is run, and, as a bonus, the window will be "labeled" by the text in the string within the quote marks.
6. What's wrong when my benchmark run seems to complete, but no reports are created?
   - I find a "raw.partial" file in the SPECpower_ssj2008\results directory.
• See the following FAQs on Out of Memory Errors.
7. The benchmark failed during report generation, my reports are blank.
   - What's wrong?
• Look at the messages generated by the Director - the most common cause of this issue is an Out Of Memory Error.
• This is the case where "raw.partial" file is found in the results directory
• the Director messages can be found in the log file named: ssj.nnnn.director.log
• If this is the case, the maximum heap size (JVM Options -Xmx) is likely too small.
• A quick fix is to set the JVM Options to blank in the script file that runs the director.
• Set JAVAOPTIONS_DIRECTOR=""
• This will cause the JVM to dynamically change heap size.
• If the director is running on the CCS system, benchmark performance will not be affected.
• The director is run from script files rundirector.bat / rundirector.sh or in the runssj.bat (.sh)
• For run setups where the director runs on the SUT, edit the file runssj.bat /runssj.sh.
• The memory requirements of Director are related to the number of JVMs it is directing. The exact memory requirements depend on the type of JVM, operating system, number of JVMs, number of nodes and other factors.
8. Why I am getting "Out of Memory Errors" from Java?
• A benchmark run may fail in the "reporter stage" if the JVM heap size options for the "director" are set too low and it runs short of memory.
• You may find "out of memory errors" in the command window - scroll back.
• One solution is to remove the Xmx and Xms parameters. This enables dynamic heap allocation:
• in the file ssj\runssj.bat (runssj.sh on Linux)
• find this line "set JAVAOPTIONS_DIRECTOR=-Xms64m -Xmx256m"
• Change it to: "set JAVAOPTIONS_DIRECTOR="
• this will allow the JVM to dynamically allocate heap space.
• Save the script file and run again.
• You may have too little physical memory.
• You should have ~512MB to ~1GB free memory on the CCS system before starting the benchmark components.
9. My results report shows one or more warnings referencing a discrepancy between elapsed nanoTime and elapsed currentTimeMillis.
   - Does this warning disallow publication of the result?
   - What is the recommended action?
• Discrepancies between nanoTime() and currenTimeMillis() arise from the differences in the way these calls are implemented by various JVMs and platform hardware architectures.
• The discrepancies are reported by the benchmark to alert you to a possible area of concern, especially when reviewing the disclosure (reports)
• and, for the benefit of those testing the benchmark on new hardware or software.
• As warnings, they do not invalidate the benchmark run.
• You may submit a benchmark disclosure (with these warnings) for acceptance by SPEC. It will not necessarily be found invalid due to the presence of these warnings alone.
  • If the disclosure is accepted by SPEC and published at spec.org, the warnings will not appear in the published report.
• While minor discrepancies between nanoTime() and currentTimeMillis() have not been considered an obstacle to disclosure publication in the past, each result is reviewed independently and reviewers have the right to question discrepancies if they so choose.
• If the warnings are a concern to you, you may find that less aggressive performance tuning may reduce their incidence.
10. After submitting results I received feedback notifying me that I needed to change one or more parameters and re-run the test. However, when I did so I got the same results as before – SPECpower_ssj2008 ignored my changes.
    Changing the contents of the configuration file SPECpower_ssj.props changes how the test will execute. Changing the contents of SPECpower_ssj_config.props and SPECpower_ssj_config_sut.props only changes how the final report will be generated and has no effect on the actual execution of the tests.

g SPECpower_ssj2008 Results Reports

1. Where are SPECpower_ssj2008 results cataloged on the SPEC web site?
   • Results disclosures are at this URL: http://www.spec.org/power_ssj2008/results/
2. How often are new results posted?
   • New disclosures are added approximately every two weeks.
3. Can I "turn off" compliance checking?
   - My results are only non-compliant because I do not have a temperature sensor.
   • No. This is to prevent public use of a Full Disclosure Report that is or might be non-compliant.
4. On the SPECpower report, if I compute my own performance to power ratio, why do I sometimes get different results for some load levels?
   - I've seen a difference of up to two points.
• This is due to the higher precision of the measured results used to compute the ratios that appear in the report. This is an artifact of rounding.
• The ratios in the reports are correct.

g Results Publication

1. Am I required to submit benchmark results to SPEC?
   • If the results are for your use only, then No.
   • If you intend to use the results publicly, then it is (currently) required that they follow the run and reporting rules and then be reviewed by SPEC prior to that public use.
     • This requirement is a provision of the license agreement.
2. My company is not a SPEC member or associate.
   - Can I submit benchmark results for publication by SPEC?
   • Yes, subject to the review requirements and a nominal publication fee.
   • If your company or organization becomes a member of SPEC you may submit, have your results reviewed, and potentially published on the SPEC web site.
3. Are results published on the SPEC web site audited?
   • Results are "peer reviewed" prior to posting on the SPEC web site.
   • The full veracity and accuracy of the results is ultimately the responsibility of the organization that tested and submitted the results.
4. Can I download all or selected SPECpower_ssj2008 results from the SPEC public web site?
   • Yes.
   • Use the Search Engine at this link: Search SPECpower Results (http://www.spec.org/cgi-bin/osgresults?conf=power_ssj2008)
   • To download all results to a spreadsheet, look for "Dump All Records As CSV" option.

g Disclaimer

g Product and service names mentioned herein may be the trademarks of their respective owners.
Copyright (c) 2007-2012 Standard Performance Evaluation Corporation (SPEC).

g