Standard Performance Evaluation Corporation

Machine Learning Committee

Welcome to the SPEC Machine Learning Committee. The ML Committee was formed in 2021 to develop practical methodologies to benchmark machine learning (ML) performance in the context of real-world platforms and environments. The Committee also works with other SPEC committees to update their benchmarks for ML environments.

The ML Committee's first benchmark, SPEC ML, will measure end-to-end performance of a system under test (SUT) handling ML training and inference tasks. The SPEC ML benchmark will better represent industry practices compared to other existing benchmarks by including major parts of the end-to-end ML/DL pipeline, including data prep and training/inference. This vendor-neutral third-party benchmark will enable ML users, such as enterprises and scientific research institutions, to better understand how solutions will perform in real-world environments, enabling them to make better purchasing decisions.

The ML Committee welcomes current SPEC members willing to join the development and management of SPEC ML, as well as new SPEC members, especially ML/DL end users and manufacturers. Visit membership or contact info@spec.org for more information.