SPEC/RG: Exploring the frontiers of performance

Formed in 2011, the SPEC Research Group provides a platform for collaborative research efforts in the area of quantitative system evaluation and analysis, fostering the interaction between industry and academia in the field. Like seemingly everything at SPEC, it has spawned widespread growth in scope and participation over the last eight years.

Research efforts at SPEC/RG are targeted at both traditional computer benchmarking and experimental system analysis in general. They address classical performance aspects (amount of work done vs. time and resources spent) as well as system properties such as reliability, security or energy efficiency, according to SPEC/RG Chair Samuel Kounev.

Under SPEC RG co-sponsorship, the ACM/SPEC International Conference on Performance Engineering (ICPE), in its 10th year in 2019, offers a balance between academic and industrial participants exploring diverse subjects such as performance measurement, modeling, benchmark design, and run-time performance management.

The SPEC/RG umbrella now covers six working groups:

• The Big Data Working Group focuses on specifying and classifying big data systems, developing rules and tools for big data benchmarking, and fostering collaboration between benchmarking efforts.
• The Cloud Working Group aims to develop new methodologies to gain more insight into cloud performance, operation and behavior.
• The DevOps Performance Working Group conducts research in combining application performance management (APM) and model-based software performance engineering (SPE) activities for business-critical application systems.
• The Security Benchmarking Working Group addresses the increasing demand for representative and rigorous security benchmarks for computing systems, particularly for cloud platforms.
• The Power Working Group contributes research that enhances and promotes methods and tools for energy efficiency evaluation for the newest generation of servers and workstations.
• The Quality of Experience Working Group is a starting point for the release of QoE ideas, approaches, measurement tools, and assessment paradigms.

Currently the largest SPEC group with more than 50 members, SPEC/RG also offers recognition for outstanding performance research through the annual SPEC Kaivalya Dixit Distinguished Dissertation Award.

Standard Performance Evaluation Corporation