I originally posted this at http://blogs.sun.com/brendan/entry/my_sun_storage_7410_perf.
As part of my role in Fishworks, I push systems to their limits to investigate and solve bottlenecks. Limits can be useful to consider as a possible upper bound of performance, as it shows what the target can do. Finding and properly analyzing these limits is a useful exercise, as I can determine the kernel code or hardware that is the bottleneck, leading to product changes and better overall performance. I thought these results would make for some interesting blog posts if I could explain the setup, describe what the tests were, and include screenshots of these results in action. (Update: I've included results from colleagues who have tested in the same manner.)
To summarize the performance limits that I found for a single Sun Storage 7410 head node (our top performing ZFS storage appliance):
|Workload||7410 (Barcelona)||7410 (Istanbul)|
|NFS streaming read from DRAM||~1.90 Gbytes/sec||~3.06 Gbytes/sec|
|NFS streaming read from disk||1.04 Gbytes/sec||2.03 Gbytes/sec|
|NFS streaming write to disk||563 Mbytes/sec||932 Mbytes/sec|
|NFS 1b read IOPS from DRAM||281,000 IOPS||506,000 IOPS|
|NFS 4k read IOPS from DRAM||191,000 IOPS||404,000 IOPS|
|NFS 512b reads from L2ARC||128 - 260 us|
|NFS 512b sync writes to SLOG||137 - 181 us|
|NFS IB/RDMA streaming read from DRAM||N/A||2.93 Gbytes/sec|
|NFS IB/RDMA streaming read from disk||N/A||2.11 Gbytes/sec|
|NFS IB/RDMA streaming write to disk||N/A||984 Mbytes/sec|
|CIFS streaming read from DRAM||~1.03 Gbytes/sec|
|CIFS streaming read from disk||849 Mbytes/sec|
|CIFS streaming write to disk||620 Mbytes/sec|
|CIFS 1b read IOPS from DRAM||203,000 IOPS|
All tests are performed on Ethernet (usually 10 GbE) unless otherwise specified ("IB" == InfiniBand).
Like many products, the 7410 will undergo software and hardware updates over time. This page currently has results for:
- 7410 Barcelona: The initial release, with four quad-core AMD Opteron CPUs (Barcelona), and the 2008.Q4 software.
- 7410 Istanbul: The latest release, with four six-core AMD Opteron CPUs (Istanbul), and the 2009.Q3 software.
I should make clear that these are provided as possible upper bounds. These aren't what to expect for any given workload, unless your workload was similar to what I used for these tests. Click on the results to see details of the workloads used.
These are also the limits that were found with a given farm of clients and JBODs. It's possible the 7410 could go faster with more clients and more JBODs.
Updated 3-Mar-2009: added CIFS results.
Updated 22-Sep-2009: added column for 7410 Istanbul. Results will be added as they are collected.
Updated 12-Nov-2009: added Cindi's InfiniBand results.