SPECstorage™ Solution 2020_ai_image Result
Copyright © 2016-2025 Standard Performance Evaluation Corporation
|
SPECstorage™ Solution 2020_ai_image ResultCopyright © 2016-2025 Standard Performance Evaluation Corporation |
| HPE and WEKAIO, Inc | SPECstorage Solution 2020_ai_image = 5000 AI_Jobs |
|---|---|
| HPE WEKA - Physical server Reference | Overall Response Time = 1.06 msec |
|
|
| HPE WEKA - Physical server Reference | |
|---|---|
| Tested by | HPE and WEKAIO, Inc | Hardware Available | Jan 2025 | Software Available | Jan 2025 | Date Tested | Jan 2025 | License Number | 4553 | Licensee Locations | Campbell, CA USA |
The WEKA Data Platform running on Intel-based, all-NVMe HPE Alletra Storage Server 4110 data storage servers delivers the fastest file performance that eliminate storage bottlenecks and unleash the full power of GPUs. HPE’s cutting-edge server technologies leveraged in the 1U 1P Alletra Storage Server 4110 provide an optimal balance of CPU clock speed, cores, memory, PCIe Gen5 I/O, network and drive performance, and storage density. Through WEKA’s zero-tuning, zero-copy highly parallelized architecture, even the most demanding, data-intensive AI, ML and HPC applications unlock extreme speed and scalability.
| Item No | Qty | Type | Vendor | Model/Name | Description |
|---|---|---|---|---|---|
| 1 | 12 | Weka node | HPE | HPE Alletra Storage Server 4110 | The HPE Alletra Storage Server 4110 is an all NVMe server specifically designed to run the most performance demanding data storage-intensive workloads. Each HPE Alletra 4110 Storage Server comprised: 2 x Intel Emerald Rapid CPUs 6548N - Intel® Xeon® Gold 6548N Processor (32 cores/2.8GHz) 512GB RAM (16 x 32GB DDR5-5600 ) 2 x 960GB M.2 Boot/System drives 16 x Samsung PM1743 NVMe SSDs -3.84TB drives 2 x HPE InfiniBand NDR 1-port OSFP PCIe5 x16 MCX75310AAS-NEAT Adapter 1 x 10/25Gb BCM57414 2-port (OCP) – supporting SFP28 transceivers |
| 2 | 24 | Load Generator | HPE | HPE ProLiant DL385 Gen11 | The HPE ProLiant DL385 Gen11 server is an accelerator-optimized 2U 2P solution that delivers exceptional compute performance. Each HPE ProLiant DL385 G11 Load Generator comprised: 2x AMD EPYC 9554 (Genoa) processors (64 cores/3.1GHz) 768GB RAM (24 x 32GB DDR5-4800) 1 x 3.2TB NVMe disk 1 x HPE InfiniBand NDR 1-port OSFP PCIe5 x16 MCX75310AAS-NEAT Adapter 4 x 1Gbe |
| 3 | 3 | Switch | NVIDIA | HPE NVIDIA InfiniBand NDR 64-port OSFP | Nvidia InfiniBand (IB) NDR Switches are supported to work with HPE InfiniBand Adapters to deliver high performance connectivity for HPC clusters using HPE servers. The 1U rack mount switches feature 64, 400Gb/s non-blocking ports with aggregate data throughput up to 51.2 Tb/s. HPE SKU# P45691-B21 Each switch contains: 1 x NVIDIA Quantum-2-based 400Gb/s InfiniBand switch 64 x 400Gb/s ports 32 x OSFP ports Note: Multiple InfiniBand switches and associated interconnect links were used purely for convenience due to the location of the Load Generators and cable runs within the HPE test lab. |
| Item No | Component | Type | Name and Version | Description |
|---|---|---|---|---|
| 1 | WEKA FS | Filesystem | WEKA 4.4.3 | WekaFS is a fully-distributed, parallel file system that was written entirely from scratch to deliver the highest-performance file services by leveraging NVMe flash. The software also includes integrated tiering that seamlessly expands the namespace to and from hard disk drive (HDD) object storage, without the need for special data migration software or complex scripts; all data resides in a single namespace for easy access and management. |
| 2 | Storage Node | Storage OS | Rocky Linux release 8.10 (4.18.0-553.30.1.el8_10.x86_64) | Operating System (OS) for 12 HPE Alletra Storage Server 4110s. Rocky OS 8.10 has full support for the Emerald Rapid CPUs 6548N - Intel® Xeon® Gold 6548N Processor. |
| 3 | Load Generator | Client OS | Rocky Linux release 8.10 (4.18.0-553.30.1.el8_10.x86_64) | Operating System (OS) for 24 HPE ProLiant DL385 Gen11. |
| Storage Server | Parameter Name | Value | Description |
|---|---|---|
| Port speed | 400Gb | 2 x 400Gb/s InfiniBand NDR |
| MTU | 4096 | MTU of the InfiniBand interface | Load Generator | Parameter Name | Value | Description |
| Port speed | 400Gb | 1 x 400Gb/s InfiniBand NDR |
| MTU | 4096 | MTU of the InfiniBand interface |
None
| Load Generator | Parameter Name | Value | Description |
|---|---|---|
| num_cores | 12 | WekaFS mount option to designate the number of frontend cores to allocate for the client |
| net | ib0 | WekaFS mount option to designate client 400Gb InfiniBand network ports be used for the mount. |
The single filesystem was attached via a single mount per client. The mount string used was 'mount -t wekafs -o dentry_max_age_positive=600000,dentry_max_age_negative=600000,num_cores=12,net=ib0 192.168.0.220/default /mnt/weka' The soft and hard limits for 'nofile' was set to 99999 in /etc/security/limits.conf on each client.
None
| Item No | Description | Data Protection | Stable Storage | Qty |
|---|---|---|---|---|
| 1 | Total of 192 x Samsung PM1743 NVMe SSDs -3.84TB drives used for Weka File-System. | WekaFS patented distributed data protection schema (10+2 Fully protected | Yes | 192 |
| 2 | Total of 24 x Micron Technology 960GB SSD used for boot/system drives. | Linux Software RAID1 | Yes | 24 |
| Number of Filesystems | 1 | Total Capacity | 502.87 TiB | Filesystem Type | WekaFS |
|---|
Each storage server has 16x Samsung PM1743 NVMe SSDs attached to it, which are dedicated to the Weka filesystem. Each storage server has 2x Micron Technology 960GB SSD used as the boot drive. The storage cluster consisted of 12 storage servers with a single WekaFS filesystem created in the cluster.
WekaFS is a fully containerized storage OS, in this configuration 5 LXC containers containing WekaFS storage processing were deployed per storage server.
| Item No | Transport Type | Number of Ports Used | Notes |
|---|---|---|---|
| 1 | 400Gb InfiniBand | 24 | Each Load Generator is connected to the InfiniBand switch via a single 400Gb port. |
| 2 | 400Gb InfiniBand | 24 | Each storage server is connected to the InfiniBand switch via a a pair of 400Gb ports |
| 3 | 10gbe and 1Gbe | 36 | Each Load Generator and storage server is connected to ethernet for OS/mgmt purposes. This network is not used for benchmark load. |
| 4 | 1Gbe | 27 | The iLO connection for each Load Generator and storage server is connected to ethernet for OOB management purposes. This network is not used for benchmark load. |
None
| Item No | Switch Name | Switch Type | Total Port Count | Used Port Count | Notes |
|---|---|---|---|---|---|
| 1 | HPE NVIDIA InfiniBand NDR 64-port OSFP | 400Gb InfiniBand | 192 | 24 for Clients, 24 for Storage Servers | 3x HPE NVIDIA InfiniBand NDR 64-port OSFP switches connected together. All 12 Storage servers connected to one switch. Clients connected to dedicated switches. NDR interconnect links created between the client switch and the Storage Server switch to match the number of client connections. (16 x NDR interconnect links from the first client switch and 8 NDR interconnect links from the second client switch). Note: Multiple InfiniBand switches and associated interconnect links were used purely for convenience due to the location of the Load Generators and cable runs within the HPE test lab. |
| Item No | Qty | Type | Location | Description | Processing Function |
|---|---|---|---|---|---|
| 1 | 48 | CPU | Load Generator | AMD EPYC 9554 64-Core Processor | Each client HPE ProLiant DL385 G11 has a dual socket AMD EPYC 9554 64-Core Processor. |
| 2 | 24 | CPU | Storage Server | Intel® Xeon® Gold 6548N Processor | Each Weka Storage Server running on HPE Alletra Storage Server 4110 has a dual socket Intel® Xeon® Gold 6548N Processor. |
None
| Description | Size in GiB | Number of Instances | Nonvolatile | Total GiB |
|---|---|---|---|---|
| 24 x Load Generator servers HPE ProLiant DL385 G11 with 768GiB of memory | 768 | 24 | V | 18432 |
| 12 x storage servers HPE Alletra Storage Server 4110 with 512GiB of memory | 512 | 12 | V | 6144 | Grand Total Memory Gibibytes | 24576 |
None
WekaFS does not use any internal memory to temporarily cache write data to the underlying storage system. All writes are committed directly to the storage disk, and protected via WekaFS Distributed Data Protection (10+2 in this case). There is no need for any RAM battery protection. In an event of a power failure a write in transit would not be acknowledged.
The system under test consisted of the 24 Load Generators and 12 storage servers. The 12 storage servers each have 2 single-port InfiniBand network interfaces connected to a 400Gb switch. The 24 load generating client servers each have a single-port InfiniBand network interface also connected to a 400Gb switch.
None
One of the 24 Load Generators acted as the benchmark's prime client to initiate the tests. The benchmark load is distributed across the 24 HPE ProLiant DL385 G11 Load Generators using a single mount point per client.
None
The solution under test was a standard WekaFS cluster in dedicated server mode. The solution will handle both large file I/O as well as small file random I/O and metadata intensive applications. No specialized tuning is required for different or mixed use workloads. SUT not found with any SPECTRE and MELTDOWN vulnerabilities including CVE-2017-5754 (Meltdown), CVE-2017-5753 (Spectre variant 1), and CVE-2017-5715 (Spectre variant 2).
Generated on Mon Jan 27 20:33:52 2025 by SpecReport
Copyright © 2016-2025 Standard Performance Evaluation Corporation
