application-level benchmarking with spec sfs® 2014 · benchmarking: real-life examples (env 1)...
TRANSCRIPT
![Page 1: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/1.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
Application-Level Benchmarking with SPEC SFS® 2014
Nick Principe – EMC Vernon Miller – IBM
![Page 2: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/2.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
Agenda
Why application-level benchmarking? What is application-level benchmarking? The SPEC SFS 2014 Workloads
Reporting SFS 2014 Results Testing a “Storage Solution” Ramifications of application-level benchmarking
Concepts Real-life examples
Key takeaways Q&A
2
![Page 3: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/3.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
SPEC Standard Performance Evaluation Corporation
The Standard Performance Evaluation Corporation (SPEC) is a non-profit corporation formed to establish, maintain and endorse a standardized set of relevant benchmarks that can be applied to the newest generation of high-performance computers. SPEC develops benchmark suites and also reviews and publishes submitted results from member organizations and other benchmark licensees
www.spec.org SPEC and SPEC SFS are registered trademarks of the
Standard Performance Evaluation Corporation. Additional product and service names mentioned herein may be the trademarks of their respective owners.
3
![Page 4: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/4.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
Why application-level benchmarking?
The focus of the SFS benchmark has changed in SFS 2014 Load is now generated at the application level The aim is to measure the storage
performance of the environment as a whole We call this the “Storage Solution”
Vendors now have the flexibility to configure the benchmark to match their environment Put the bottleneck where you want to show value
4
![Page 5: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/5.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
Why application-level benchmarking?
Increased flexibility addresses the market Complexity of storage solutions Diversity of architectures and protocols Fairness to all implementations
Any other approach is no longer appropriate for industry-standard benchmarking of storage solutions
5
![Page 6: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/6.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
What is Application-Level Benchmarking?
Prior to SFS 2014, benchmark generated its own NFS or SMB traffic Bypassed load generator operating system Focus: performance of monolithic NAS server
SFS 2014 uses native OS calls to generate application-level load Data and metadata ops processed by OS Focus: storage performance of the
environment as a whole (Storage Solution) 6
![Page 7: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/7.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
The SPEC SFS 2014 Workloads
DATABASE Simulates OLTP database consolidation Measured in # of concurrent DATABASES
SWBUILD Simulates large software project compilation Measured in # of concurrent BUILDS
VDA Simulates acquisition of streaming data Measured in # of concurrent STREAMS
VDI Simulates heavy steady-state VDI workload Measured in # of concurrent DESKTOPS 7
For more details, see: - SDC 2014 presentation: SPEC SFS 2014: An Under-the-Hood Review - The SPEC SFS 2014 website http://www.spec.org/sfs2014
![Page 8: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/8.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
Reporting SFS 2014 Results
Disclosure of SPEC SFS 2014 results must meet the requirements of
SPEC SFS 2014 License SPEC SFS 2014 Run and Reporting Rules SPEC Fair Use Rules
Submission to SPEC for review encouraged Certain information is required to be disclosed
Do not use this presentation as a guide for public disclosure of SFS 2014 results
Created for education under auspices of SPEC using “generic” environments 8
![Page 9: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/9.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
Testing a “Storage Solution”
More attention to benchmark configuration required You must put the bottleneck in the right place
Understanding the whole system, from load generator to the disks, is a requirement
For publication, more configuration details must be recorded and disclosed
SFS 2014 is still a storage benchmark There is no attempt to simulate compute load
9
![Page 10: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/10.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
Ramifications of application-level benchmarking: Concepts
Your load generators matter Any config detail can affect performance
Storage connectivity, OS version, patch level, memory, client count, tuning parameters
You can test anything that provides a file API to an application Traditional NAS server, block storage with a
file system on load generators, hyper-converged solutions, a single server with storage 10
![Page 11: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/11.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
Ramifications of application-level benchmarking: Concepts
With great power comes great responsibility Understand where your bottleneck is
SFS 2014 allows great flexibility in load placement as the workload scales
Getting this right is the key to getting the performance you expect Likely you want to spread load as evenly as possible
across ALL resources as the benchmark ramps up load
The key config parameter in SFS 2014? CLIENT_MOUNTPOINTS 11
![Page 12: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/12.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
Ramifications of application-level benchmarking: Concepts
Measuring performance at multiple levels of the solution under test is key to understanding your solution’s performance and bottlenecks
SFS 2014 reports application-level performance Other statistics that are helpful to collect:
Storage array statistics, NAS server statistics, Hypervisor statistics, LG OS statistics
You may see different performance at the different levels Each layer of the solution under test may change the
workload 12
![Page 13: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/13.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
Ramifications of application-level benchmarking: Real-life Examples (Env 1)
Configured an environment for testing Midrange Storage Array
FC drives, FC frontend
Large Windows Server 2012 R2 NAS Server FC backend, 48 cores, 256GB memory, 10GbE frontend
20 Windows 8.1 VMs; 10 physical servers FC or 10GbE SMB3 connectivity, 2 cores, 2 GB memory
Testing was done in two ways Via SMB3 shares from NAS server (10GbE) Via local E: drive, FC LUN via RDM to VM (FC) 13
![Page 14: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/14.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
Ramifications of application-level benchmarking: Real-life Examples (Env 1)
14
ESXi Server 1
10 GbE Switch
NAS Server
Storage Controller
SMB Server Measurements
Disk Measurements
Network Measurements
Application Measurements Win VM
Win VM
ESXi Server 2 Win VM
Win VM
ESXi Server 10
Win VM
Win VM
8 Gb FC Switch
![Page 15: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/15.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
Ramifications of application-level benchmarking: Real-life Examples (Env 2) Configured an environment for testing Midrange Storage Array
SAS drives, FC frontend
4 Node Distributed Filesystem 2 nodes with FC backend, DDR IB cluster network 2 nodes acting as NFS server, 10 GbE frontend
2 NFS Clients RHEL 6.5, 10GbE connectivity, 4 cores, 32 GB memory
Testing was done in two ways Via NFSv3 exports from NAS server (10GbE) Via local filesystem: single namespace on 2 nodes 15
![Page 16: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/16.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
Ramifications of application-level benchmarking: Real-life Examples (Env 2)
16
NFS Client NFS Client
10 GbE Switch
DDR IB Switch
Frontend Node Frontend Node
Backend Node Backend Node
Storage Controller
Load generators for NFSv3 tests
Load generators for local filesystem tests
NFS Server Measurements
Disk Measurements
Network Measurements
Application Measurements
![Page 17: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/17.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
Ramifications of application-level benchmarking: Real-life Examples (Env 2)
17
0
50,000
100,000
150,000
200,000
250,000
300,000
350,000
400,000
450,000
500,000
5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100
Kilo
byte
s/Se
cond
Streams
VDA - NFS
App Read App Write Net Recv
Net Send Disk Read Disk Write
0
50,000
100,000
150,000
200,000
250,000
300,000
350,000
400,000
450,000
500,000
5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100
Kilo
byte
s/Se
cond
Streams
VDA - Cluster FS
App Read App Write
Disk Read Disk Write
![Page 18: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/18.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
Ramifications of application-level benchmarking: Real-life Examples (Env 1)
18
0
50,000
100,000
150,000
200,000
250,000
300,000
350,000
5 10 15 20 25 30 35 40 45 50 55 60 65 70 75
Kib
ibyt
es/s
ec
Streams
VDA - SMB
App Read App Write Net Recv
Net Send Disk Read Disk Write
0
50,000
100,000
150,000
200,000
250,000
300,000
350,000
400,000
450,000
10 20 30 40 50 60 70 80 90 100
Kib
ibyt
es/s
ec
Streams
VDA - Local FS
App Read App Write
Disk Read Disk Write
![Page 19: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/19.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
Ramifications of application-level benchmarking: Real-life Examples (Env 2)
19
0
200
400
600
800
1,000
1,200
5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100
Ope
rati
ons/
Seco
nd
Streams
VDA - NFS
App NFS Total NFS Data NFS Meta Disk
0
200
400
600
800
1,000
1,200
5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100
Ope
rati
ons/
Seco
nd
Streams
VDA - Cluster FS
App Disk
![Page 20: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/20.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
Ramifications of application-level benchmarking: Real-life Examples (Env 1)
20
0
100
200
300
400
500
600
700
800
900
5 10 15 20 25 30 35 40 45 50 55 60 65 70 75
Ope
rati
ons/
sec
Streams
VDA - SMB
App SMB Total SMB Data SMB Meta Disk
0
200
400
600
800
1,000
1,200
1,400
1,600
1,800
2,000
10 20 30 40 50 60 70 80 90 100
Ope
rati
ons/
sec
Streams
VDA - Local FS
App Disk
![Page 21: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/21.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
Ramifications of application-level benchmarking: Real-life Examples
21
0
10
20
30
40
50
60
70
80
90
100
0 200 400 600 800 1,000 1,200
Ave
rage
Res
pons
e T
ime
(ms)
Achieved Ops/Sec
VDA - Env 2
NFSv3 Local
0
50
100
150
200
250
0 200 400 600 800 1,000 1,200
Ave
rage
Res
pons
e T
ime
(ms)
Achieved Ops/sec
VDA - Env 1
SMB3 Local
![Page 22: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/22.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
Ramifications of application-level benchmarking: Real-life Examples (Env 2)
22
0
20,000
40,000
60,000
80,000
100,000
120,000
2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40
Kilo
byte
s/Se
cond
Desktops
VDI - NFS
App Read App Write Net Recv
Net Send Disk Read Disk Write
0
10,000
20,000
30,000
40,000
50,000
60,000
70,000
80,000
4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40
Kilo
byte
s/Se
cond
Desktops
VDI - Cluster FS
App Read App Write
Disk Read Disk Write
![Page 23: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/23.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
Ramifications of application-level benchmarking: Real-life Examples (Env 1)
23
0
5,000
10,000
15,000
20,000
25,000
30,000
35,000
1 2 3 4 5 6 7 8 9 10 11 12
Kib
ibyt
es/s
ec
Desktops
VDI - Local FS
App Read App Write
Disk Read Disk Write
0
5,000
10,000
15,000
20,000
25,000
30,000
1 2 3 4 5 6 7 8 9 10 11
Kib
ibyt
es/s
ec
Desktops
VDI - SMB
App Read App Write Net Recv
Net Send Disk Read Disk Write
![Page 24: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/24.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
Ramifications of application-level benchmarking: Real-life Examples (Env 2)
24
0
1,000
2,000
3,000
4,000
5,000
6,000
7,000
8,000
2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40
Ope
rati
ons/
Seco
nd
Desktops
VDI - NFS
App NFS Total NFS Data NFS Meta Disk
0
2,000
4,000
6,000
8,000
10,000
12,000
14,000
2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40
Ope
rati
ons/
Seco
nd
Desktops
VDI - Cluster FS
App Disk
![Page 25: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/25.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
Ramifications of application-level benchmarking: Real-life Examples (Env 1)
25
0
500
1,000
1,500
2,000
2,500
3,000
3,500
4,000
4,500
1 2 3 4 5 6 7 8 9 10 11
Ope
rati
ons/
sec
Desktops
VDI - SMB
App SMB Total SMB Data SMB Meta Disk
0
500
1,000
1,500
2,000
2,500
3,000
3,500
4,000
4,500
1 2 3 4 5 6 7 8 9 10 11 12
Ope
rati
ons/
sec
Desktops
VDI - Local FS
App Disk
![Page 26: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/26.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
Ramifications of application-level benchmarking: Real-life Examples
26
0
2
4
6
8
10
12
14
0 2,000 4,000 6,000 8,000 10,000
Ave
rage
Res
pons
e T
ime
(ms)
Achieved Ops/Sec
VDI - Env 2
NFSv3 Local
0
1
2
3
4
5
6
0 500 1,000 1,500 2,000 2,500 3,000
Ave
rage
Res
pons
e T
ime
(ms)
Achieved Ops/sec
VDI - Env 1
SMB3 Local
![Page 27: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/27.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
Ramifications of application-level benchmarking: Real-life Examples (Env 2)
27
0
20,000
40,000
60,000
80,000
100,000
120,000
2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40
Kilo
byte
s/Se
cond
Desktops
VDI - NFS (Read Bandwidth)
App Read
Disk - Original
Disk - Tuning #1
Disk - Tuning #2
![Page 28: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/28.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
Ramifications of application-level benchmarking: Real-life Examples (Env 2)
28
0
10,000
20,000
30,000
40,000
50,000
60,000
70,000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Kilo
byte
s/Se
cond
Builds
SWBUILD - NFS
App Read App Write Net Recv
Net Send Disk Read Disk Write
0
10,000
20,000
30,000
40,000
50,000
60,000
70,000
80,000
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Kilo
byte
s/Se
cond
Builds
SWBUILD - Cluster FS
App Read App Write
Disk Read Disk Write
![Page 29: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/29.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
Ramifications of application-level benchmarking: Real-life Examples (Env 1)
29
0
10,000
20,000
30,000
40,000
50,000
60,000
70,000
2 4 6 8 10 12 14 16 18 20
Kib
ibyt
es/s
ec
Builds
SWBUILD - Local FS
App Read App Write
Disk Read Disk Write
0
10,000
20,000
30,000
40,000
50,000
60,000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Kib
ibyt
es/s
ec
Builds
SWBUILD - SMB
App Read App Write Net Recv
Net Send Disk Read Disk Write
![Page 30: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/30.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
Ramifications of application-level benchmarking: Real-life Examples (Env 2)
30
0
2,000
4,000
6,000
8,000
10,000
12,000
14,000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Ope
rati
ons/
Seco
nd
Builds
SWBUILD - NFS
App NFS Total NFS Data NFS Meta Disk
0
2,000
4,000
6,000
8,000
10,000
12,000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Ope
rati
ons/
Seco
nd
Builds
SWBUILD - Cluster FS
App Disk
![Page 31: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/31.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
Ramifications of application-level benchmarking: Real-life Examples (Env 1)
31
0
2,000
4,000
6,000
8,000
10,000
12,000
2 4 6 8 10 12 14 16 18 20
Ope
rati
ons/
sec
Builds
SWBUILD - Local FS
App Disk
0
2,000
4,000
6,000
8,000
10,000
12,000
14,000
16,000
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Ope
rati
ons/
sec
Builds
SWBUILD - SMB
App SMB Total SMB Data SMB Meta Disk
![Page 32: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/32.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
Ramifications of application-level benchmarking: Real-life Examples
32
0
2
4
6
8
10
12
0 2,000 4,000 6,000 8,000 10,000 12,000
Ave
rage
Res
pons
e T
ime
(ms)
Achieved Ops/Sec
SWBUILD - Env 2
NFSv3 Local
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
0 2,000 4,000 6,000 8,000 10,000 12,000
Ave
rage
Res
pons
e T
ime
(ms)
Achieved Ops/sec
SWBUILD - Env 1
SMB3 Local
![Page 33: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/33.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
33
Ramifications of application-level benchmarking: Real-life Examples (Env 2)
Percent of I/O
Client Cache Hit
NAS Server Cache Hit Cluster/Disk Cache Hit
Disk Access
![Page 34: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/34.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
Key takeaways
The SPEC SFS 2014 is an application-level benchmark that tests the storage performance of an entire storage solution
Understanding the storage solution under test and bottleneck placement are keys to getting “what you expect” from your storage solution
The application-level benchmarking provided by SFS 2014 allows testing of a much wider array of products and storage solutions
34
![Page 35: Application-Level Benchmarking with SPEC SFS® 2014 · benchmarking: Real-life Examples (Env 1) Configured an environment for testing Midrange Storage Array FC drives, FC frontend](https://reader031.vdocument.in/reader031/viewer/2022011811/5e143f67f46ff16d0c113e01/html5/thumbnails/35.jpg)
2015 Storage Developer Conference. © 2015 Standard Performance Evaluation Corporation. All Rights Reserved.
Q & A
Any questions?
Thank you for attending! Please remember to submit feedback!
35