SPECstorage(TM) Solution 2020_vda Result HPE and WEKAIO, Inc : HPE WEKA - Physical server Reference SPECstorage Solution = 14400 Streams (Overall Response Time = 1.29 msec) 2020_vda =============================================================================== Performance =========== Business Average Metric Latency Streams Streams (Streams) (msec) Ops/Sec MB/Sec ------------ ------------ ------------ ------------ 1440 0.9 14409 6636 2880 0.9 28819 13264 4320 0.9 43229 19901 5760 0.9 57639 26531 7200 1.0 72049 33173 8640 1.1 86458 39798 10080 1.3 100869 46425 11520 1.5 115278 53049 12960 2.1 129688 59696 14400 2.7 144098 66321 =============================================================================== Product and Test Information ============================ +---------------------------------------------------------------+ | HPE WEKA - Physical server Reference | +---------------------------------------------------------------+ Tested by HPE and WEKAIO, Inc Hardware Available Jan 2025 Software Available Jan 2025 Date Tested Jan 2025 License Number 4553 Licensee Locations Campbell, CA USA The WEKA Data Platform running on Intel-based, all-NVMe HPE Alletra Storage Server 4110 data storage servers delivers the fastest file performance that eliminate storage bottlenecks and unleash the full power of GPUs. HPE’s cutting-edge server technologies leveraged in the 1U 1P Alletra Storage Server 4110 provide an optimal balance of CPU clock speed, cores, memory, PCIe Gen5 I/O, network and drive performance, and storage density. Through WEKA’s zero-tuning, zero-copy highly parallelized architecture, even the most demanding, data-intensive AI, ML and HPC applications unlock extreme speed and scalability. Solution Under Test Bill of Materials ===================================== Item No Qty Type Vendor Model/Name Description ---- ---- ---------- ---------- ---------- ----------------------------------- 1 12 Weka node HPE HPE The HPE Alletra Storage Server 4110 Alletra is an all NVMe server specifically Storage designed to run the most Server performance demanding data storage- 4110 intensive workloads. Each HPE Alletra 4110 Storage Server comprised: 2 x Intel Emerald Rapid CPUs 6548N - Intel® Xeon® Gold 6548N Processor​ (32 cores/2.8GHz) 512GB RAM (16 x 32GB DDR5-5600 )​ 2 x 960GB M.2 Boot/System drives​ 16 x Samsung PM1743 NVMe SSDs -3.84TB drives​ 2 x HPE InfiniBand NDR 1-port OSFP PCIe5 x16 MCX75310AAS-NEAT Adapter 1 x 10/25Gb BCM57414 2-port (OCP) – supporting SFP28 transceivers​ 2 24 Load HPE HPE The HPE ProLiant DL385 Gen11 server Generator ProLiant is an accelerator-optimized 2U 2P DL385 solution that delivers exceptional Gen11 compute performance. Each HPE ProLiant DL385 G11 Load Generator comprised: 2x AMD EPYC 9554 (Genoa) processors (64 cores/3.1GHz)​ 768GB RAM (24 x 32GB DDR5-4800) 1 x 3.2TB NVMe disk​ 1 x HPE InfiniBand NDR 1-port OSFP PCIe5 x16 MCX75310AAS-NEAT Adapter 4 x 1Gbe​ 3 3 Switch NVIDIA HPE NVIDIA Nvidia InfiniBand (IB) NDR Switches InfiniBand are supported to work with HPE NDR InfiniBand Adapters to deliver high 64-port performance connectivity for HPC OSFP clusters using HPE servers. The 1U rack mount switches feature 64, 400Gb/s non-blocking ports with aggregate data throughput up to 51.2 Tb/s. HPE SKU# P45691-B21 Each switch contains: 1 x NVIDIA Quantum-2-based 400Gb/s InfiniBand switch 64 x 400Gb/s ports 32 x OSFP ports Note: Multiple InfiniBand switches and associated interconnect links were used purely for convenience due to the location of the Load Generators and cable runs within the HPE test lab. Configuration Diagrams ====================== 1) storage2020-20250113-00107.config1.png (see SPECstorage Solution 2020 results webpage) Component Software ================== Item Name and No Component Type Version Description ---- ------------ ------------ ------------ ----------------------------------- 1 WEKA FS Filesystem WEKA 4.4.3 WekaFS is a fully-distributed, parallel file system that was written entirely from scratch to deliver the highest-performance file services by leveraging NVMe flash. The software also includes integrated tiering that seamlessly expands the namespace to and from hard disk drive (HDD) object storage, without the need for special data migration software or complex scripts; all data resides in a single namespace for easy access and management. 2 Storage Node Storage OS Rocky Linux Operating System (OS) for 12 HPE release 8.10 Alletra Storage Server 4110s. Rocky (4.18.0-553. OS 8.10 has full support for the 30.1.el8_10. Emerald Rapid CPUs 6548N - Intel® x86_64) Xeon® Gold 6548N Processor​. 3 Load Client OS Rocky Linux Operating System (OS) for 24 HPE Generator release 8.10 ProLiant DL385 Gen11. (4.18.0-553. 30.1.el8_10. x86_64) Hardware Configuration and Tuning - Physical ============================================ +----------------------------------------------------------------------+ | Storage Server | +----------------------------------------------------------------------+ Parameter Name Value Description --------------- --------------- ---------------------------------------- Port speed 400Gb 2 x 400Gb/s InfiniBand NDR MTU 4096 MTU of the InfiniBand interface +----------------------------------------------------------------------+ | Load Generator | +----------------------------------------------------------------------+ Parameter Name Value Description --------------- --------------- ---------------------------------------- Port speed 400Gb 1 x 400Gb/s InfiniBand NDR MTU 4096 MTU of the InfiniBand interface Hardware Configuration and Tuning Notes --------------------------------------- None Software Configuration and Tuning - Physical ============================================ +----------------------------------------------------------------------+ | Load Generator | +----------------------------------------------------------------------+ Parameter Name Value Description --------------- --------------- ---------------------------------------- num_cores 12 WekaFS mount option to designate the number of frontend cores to allocate for the client net ib0 WekaFS mount option to designate client 400Gb InfiniBand network ports be used for the mount. Software Configuration and Tuning Notes --------------------------------------- The single filesystem was attached via a single mount per client. The mount string used was 'mount -t wekafs -o dentry_max_age_positive=600000,dentry_max_age_negative=600000,num_cores=12,net=ib0 192.168.0.220/default /mnt/weka' The soft and hard limits for 'nofile' was set to 99999 in /etc/security/limits.conf on each client. Service SLA Notes ----------------- None Storage and Filesystems ======================= Item Stable No Description Data Protection Storage Qty ---- ------------------------------------- ------------------ -------- ----- 1 Total of 192 x Samsung PM1743 NVMe WekaFS patented Yes 192 SSDs -3.84TB drives used for Weka distributed data File-System. protection schema (10+2 Fully protected 2 Total of 24 x Micron Technology 960GB Linux Software Yes 24 SSD used for boot/system drives. RAID1 Number of Filesystems 1 Total Capacity 502.87 TiB Filesystem Type WekaFS Filesystem Creation Notes ------------------------- Each storage server has 16x Samsung PM1743 NVMe SSDs attached to it, which are dedicated to the Weka filesystem. Each storage server has 2x Micron Technology 960GB SSD used as the boot drive. The storage cluster consisted of 12 storage servers with a single WekaFS filesystem created in the cluster. Storage and Filesystem Notes ---------------------------- WekaFS is a fully containerized storage OS, in this configuration 5 LXC containers containing WekaFS storage processing were deployed per storage server. Transport Configuration - Physical ================================== Item Number of No Transport Type Ports Used Notes ---- --------------- ---------- ----------------------------------------------- 1 400Gb 24 Each Load Generator is connected to the InfiniBand InfiniBand switch via a single 400Gb port. 2 400Gb 24 Each storage server is connected to the InfiniBand InfiniBand switch via a a pair of 400Gb ports 3 10gbe and 1Gbe 36 Each Load Generator and storage server is connected to ethernet for OS/mgmt purposes. This network is not used for benchmark load. 4 1Gbe 27 The iLO connection for each Load Generator and storage server is connected to ethernet for OOB management purposes. This network is not used for benchmark load. Transport Configuration Notes ----------------------------- None Switches - Physical =================== Total Used Item Port Port No Switch Name Switch Type Count Count Notes ---- -------------------- --------------- ------ ----- ------------------------ 1 HPE NVIDIA 400Gb 192 24 3x HPE NVIDIA InfiniBand InfiniBand NDR InfiniBand for C NDR 64-port OSFP 64-port OSFP lient switches connected s, 24 together. All 12 Storage for S servers connected to one torag switch. Clients e Ser connected to dedicated vers switches. NDR interconnect links created between the client switch and the Storage Server switch to match the number of client connections. (16 x NDR interconnect links from the first client switch and 8 NDR interconnect links from the second client switch). Note: Multiple InfiniBand switches and associated interconnect links were used purely for convenience due to the location of the Load Generators and cable runs within the HPE test lab. Processing Elements - Physical ============================== Item No Qty Type Location Description Processing Function ---- ---- -------- -------------- ------------------------- ------------------- 1 48 CPU Load Generator AMD EPYC 9554 64-Core Each client HPE Processor ProLiant DL385 G11 has a dual socket AMD EPYC 9554 64-Core Processor. 2 24 CPU Storage Server Intel® Xeon® Gold 6548N Each Weka Storage Processor Server running on HPE Alletra Storage Server 4110 has a dual socket Intel® Xeon® Gold 6548N Processor. Processing Element Notes ------------------------ None Memory - Physical ================= Size in Number of Description GiB Instances Nonvolatile Total GiB ------------------------- ---------- ---------- ------------ ------------ 24 x Load Generator 768 24 V 18432 servers HPE ProLiant DL385 G11 with 768GiB of memory 12 x storage servers HPE 512 12 V 6144 Alletra Storage Server 4110 with 512GiB of memory Grand Total Memory Gibibytes 24576 Memory Notes ------------ None Stable Storage ============== WekaFS does not use any internal memory to temporarily cache write data to the underlying storage system. All writes are committed directly to the storage disk, and protected via WekaFS Distributed Data Protection (10+2 in this case). There is no need for any RAM battery protection. In an event of a power failure a write in transit would not be acknowledged. Solution Under Test Configuration Notes ======================================= The system under test consisted of the 24 Load Generators and 12 storage servers. The 12 storage servers each have 2 single-port InfiniBand network interfaces connected to a 400Gb switch. The 24 load generating client servers each have a single-port InfiniBand network interface also connected to a 400Gb switch. Other Solution Notes ==================== None Dataflow ======== One of the 24 Load Generators acted as the benchmark's prime client to initiate the tests. The benchmark load is distributed across the 24 HPE ProLiant DL385 G11 Load Generators using a single mount point per client. Other Notes =========== None Other Report Notes ================== The solution under test was a standard WekaFS cluster in dedicated server mode. The solution will handle both large file I/O as well as small file random I/O and metadata intensive applications. No specialized tuning is required for different or mixed use workloads. SUT not found with SPECTRE and MELTDOWN vulnerabilities including CVE-2017-5754 (Meltdown), CVE-2017-5753 (Spectre variant 1), and CVE-2017-5715 (Spectre variant 2). =============================================================================== Generated on Mon Jan 27 21:08:09 2025 by SpecReport Copyright (C) 2016-2025 Standard Performance Evaluation Corporation