823.llvm_s
C/C++ Language optimizing compiler
LLVM began as a research project by Chris Lattner and Vikram Adve. The full list of contributors is on the LLVM Github project page.
The LLVM project is a set of libraries and tools that is used for the development of a programming language frontend and a target specific backend. Programming languages are translated to LLVM's Intermediate Representation (IR) to which optimizations and, eventually, code generation can be applied. The LLVM toolset has been used to compile languages such as C/C++, Rust, Fortran, Julia and D. The LLVM backend supports x86_64, ARM, PowerPC among others. The LLVM framework is widely used for commercial and open source compilers.
One of the central features of LLVM is its modularity, 743.llvm_r takes advantage of that by leveraging the modular LLVM optimizer and analyzer opt. This benchmark runs opt on LLVM IR files (.ll), the outputs transformed or optimized .ll files. The order and the number of the passes is determined by the optimization level (-O0, -O2, -O3), however users can also apply specific optimization passes individually. 823.llvm_s is based on LLVM version 14.0.0.
The inputs consist of various code source files converted to IR or
bitcode format. The benchmark then runs the opt executable
at various optimization levels as specified in the control file.
The test and train data sets use files from LLVM itself.
For refrate size, the inputs consist of LLVM 14.0.0 source files, with the optimizer running at level -O3 for the PowerPC target.
The SPECspeed benchmark runs a multi-tasking batch flow, mimicking a "make -j" scenario. The inputs consist of 4000 source files that are simultaneously run using the number of jobs requested by the runcpu --threads option. The 4000 inputs come from LLVM itself, from deal.II, and from other SPEC CPU benchmark sources, namely 735.gem5, 737.gmsh, 767.nest, 721.gcc, and 823.llvm_s.
New inputs can be generated by converting C/C++ code to LLVM IR using an LLVM frontend such as clang:
clang -S -emit-llvm foo.c -o foo.llThe above command will generate a foo.ll file which can then be used by opt to generate transformed code.
The output for each input file consists of the sha512sum of the optimized IR file that would have been written. To conserve space and limit I/O operations the benchmark does not directly output the IR file but hashes the file and outputs its checksum. To debug and output the IR in full, the option --sha512 must be removed from the benchmark command line.
C++
Although std::thread is referenced, only 1 thread is
active at a time for each file that is compiled. 823.llvm_s runs one
file at time sequentally, while 823.llvm_s runs multiple files at a
time.
GNU/Linux systems implement C++ std::thread using POSIX
Threads. Although some systems automatically include the needed support,
this is not universal. Surprises have been seen when changing OS
versions, or libraries, or compilers; or when FDO is
added; or when combining C and C++ modules. Typically, it is safest to
add -pthread to all compile and link lines for all SPEC CPU
benchmarks that use std::thread. Please see the
$SPEC/config directory for Example config files that
demonstrate how to conveniently do so.
823.llvm_s is based on LLVM 14.0.0 which is released under the Apache License v2.0 with LLVM Exceptions.
src/include/llvm/Support/MD5.h and
llvm/lib/Support/MD5.cpp are listed in the public domain.The input data for refspeed uses files from LLVM itself and a variety of applications which are licensed separately:
Copyright © 2026 Standard Performance Evaluation Corporation (SPEC®)