SERT® 3 Suite

The SERT 3 Suite is an under development standardized benchmark that welcomes industry and regulatory body engagement and feedback. Like the SERT 2 Suite it will provide a first order of approximation of server efficiency across a broad range of application environments and extending coverage for requested areas of AI/ML, HPC, and storage. It will have an updated web user interface and support for language translation.

The SERT suite will continue to be implemented and tested for a variety of 64-bit processors, operating systems, and JVMs while also introducing native-code benchmarks to better reflect the computing landscape.

The server under test (SUT) may be a single stand-alone server or a multi-node set of servers. A multi-node SUT will consist of server nodes that cannot run independently of shared infrastructure such as a backplane, power-supplies, fans, or other elements.

SERT 3 workloads will continue to utilize SPEC's concept of multiple load level, when appropriate, to reflect real-world scenarios.

SERT 3 is designed with regulators in mind. Regulatory programs seek continuous improvement, with thresholds for success rising as the industry progresses and to cover needed scope of the industry. The SERT 3 suite will be designed to match this paradigm by allowing for quick adoption process of new computing technologies.

The right balance between high repeatability of the results, high sub-system coverage, and low resource allocation is desirable and will still be a significant design factor.

The results output, including the SERT suite metric, are generated in both machine- and human-readable forms, enabling automatic submission to government-sponsored certification programs as well as both summary and detail reports for use by potential customers. This be improved to allow for regulatory agency engagement for custom format for reporting and increased visibility.

SPEC fosters cooperation across an international community. The SPEC International Standards Group(ISG) is actively engaged with the U.S. EPA and European Commission for design considerations and leverages The Green Grid's expertise for regulatory thresholds.

Workload Targets and Device/Technology Considerations

• Artificial Intellegence
• Computer Vision
• DC Power Supplies
• Distributed Ledger
• Factorization of dense matrices using partial pivoting
• Idle State Power Consumption
• Liquid Cooling
• Machine Learning
• Multi-level Memory
• Natural Language Processing (NLP)
• PC Consolidation
• Scientific Computing (GPU/APA Stress)
• Security
• Serverless
• Storage Heavy Servers
• Unpopulated Memory Channels

News:

September 17th, 2024: SPEC's SERT 3 Suite is currently under development
The development team currently consists of contributors from China, Germany, India, Malaysia, Mexico, Taiwan, and The United States. SPEC ISG welcomes members from all geographies. Click here for information on joining or contact info@spec.org for more information.

Information

Technical documentation for the SERT 3 suite
Upon release, technical and support documents can be found here.

Benefits of the SERT 3 Suite

• Open and international development team.
• Built on a robust history of SPEC developed tools and benchmarks.
• SPEC SERT is a trusted partner of international regulatory agencies, including the US EPA and the European Union.
• SPEC SERT3 leverages learnings and methods derived from SERT2, which is also ISO 21836:2020 complaint.
• Proven and open support infrastructure to welcome new members, accept new platforms, and address issues.
• Supports 30+ power analyzer models across 10+ different vendors headquartered in China, Germany, Japan, Norway, Taiwan, the United Kingdom, and the United States. The PTDaemon interface validates measurement uncertainty and also performs crest factor checking where supported.
• Supports 8+ temperature sensor models.
• Will be ISO compliant, with minor update results comparability protected by ISO designated test methods. Server Efficiency Score variance between minor releases is limted to 3%.
• The SERT Suite is independently verified for accuracy and quality by The Green Grid, US EPA, and ICF.