Most know Iometer as the go-to synthetic I/O measuring tool used to simulate real workload conditions. Well, somewhere, somehow, someone forgot the latter part of that sentence, which is why it ends up being so misused and abused. How many of us have seen a storage solution delivering 6 figure IOPS using Iometer, only to find that they are running a 100% read, 512 byte 100% sequential access workload simulation. Perfect for the two people on the planet that those specifications might apply to. For the rest of us, it doesn’t help much. So why would they bother running that sort of unrealistic test? Pure, unapologetic number chasing.
The unfortunate part is that sometimes this leads many to simply dismiss Iometer results. That is a shame really, as it can provide really good data if used in the correct way. Observing real world data will tell you a lot of things, but the sporadic nature of real workloads make it difficult to use for empirical measurement – hence the need for simulation.
So, what are the correct settings to use in Iometer? The answer is completely dependent on what you are trying to accomplish. The race for a million IOPS by your favorite storage vendor really means nothing if their is no correlation between their simulated workload, and your real workload. Maybe IOPS isn’t even an issue for you. Perhaps your applications are struggling with poor latency. The challenge is to emulate your environment with a synthetic workload that helps you understand how a potential upgrade, new array, or optimization might be of benefit.
The mysteries of workloads
Creating a synthetic workload representing your real workload assumes one thing; that you know what your real workload really is. This can be more challenging that one might think, as many storage monitoring tools do not help you understand the subtleties of patterns to the data that is being read or written.
Most monitoring tools tend to treat all I/O equally. By that I mean, if over a given period of time, say you have 10 million I/Os occur. Let’s say your monitoring tells you that you average 60% reads and 40% writes. What is not clear is how many of those reads are multiple reads of the same data or completely different, untouched data. It also doesn’t tell you if the writes are overwriting existing blocks (which might be read again shortly thereafter) or generating new data. As more and more tiered storage mechanisms comes into play, understanding this aspect of your workload is becoming extremely important. You may be treating your I/Os equally, but the tiered storage system using sophisticated caching algorithms certainly do not.
How can you gain more insight? Use every tool at your disposal. Get to know your applications, and the duty cycles around them. What are your peak hours? Are they in the middle of the day, or in the middle of the night when backups are running?
Suggestions on Iometer settings
You may find that the settings you choose for Iometer yields results from your shared storage that isn’t nearly as good as you thought. But does it matter? If it is an accurate representation of your real workload, not really. What matters is if are you able to deliver the payload from point a to point b to meet your acceptance criteria (such as latency, throughput, etc.). The goal would be to represent that in a synthetic workload for accurate measurement and comparison.
With that in mind, here are some suggestions for the next time you set up those Iometer runs.
1. Read/write ratio. Choose a realistic read/write ratio representing your workload. With writes, RAID penalties can hurt your effective performance by quite a bit, so if you don’t have an idea of what this ratio currently is, it’s time for you to find out.
2. Transfer request size. Is your payload the size of a ball bearing, or a bowling ball? Applications and operating systems vary on what size is used. Use your monitoring systems to best determine what your environment consists of.
3. Disk size. Use the "maximum disk size" in multiples of 1048576, which is a 1GB file. Throwing a bunch of zeros in there might fill up your disk with Iometer’s test file. Depending on your needs, a setting of 2 to 20 GB might be a good range to work with.
4. Number of outstanding I/Os. This needs to be high enough so that the test can keep sending I/O requests to it as the storage is fulfilling requests to it. A setting of 32 is pretty common.
5. Alignment of I/O. Many of the standard Iometer ICF files you find were built for physical drives. It has the "Align I/Os on:" setting to "Sector boundaries" When running tests on a storage array, this can lead to inconsistent results, so it is best to align on 4K or 512 bytes.
6. Ramp up time. Offer at least a minute of ramp up time.
7. Run time. Some might suggest running simulations long enough to exhaust all caching, so that you can see "real" throughput. While I understand the underlying reason for this statement, I believe this is missing the point. Caching is there in the first place to take advantage of a working set of warm and cold data, bursts, etc. If you have a storage solution that satisfies the duty cycles that exists in your environment, that is the most important part.
8. Number of workers. Let this spawn automatically to the number of logical processors in your VM. It might be overkill in many cases because of terrible multithreading abilities of most applications, but its a pretty conventional practice.
9. Multiple Iometer instances. Not really a setting, but more of a practice. I’ve found running multiple tests a way to better understand how a storage solution will react under load as opposed to on it’s own. It is shared storage after all.
Disclaimer
If you were looking for this to be the definitive post on Iometer, that isn’t what I was shooting for. There are many others who are much more qualified to speak to the nuances of Iometer than me. What I hope to do is to offer a little practical perspective on it’s use, and how it can help you. So next time you run Iometer, think about what you are trying to accomplish, and let go of the number chasing. Understand your workloads, and use the tool to help you improve your environment.
vmpete.com 
I’ve found the “Test Connections Rate” setting makes a huge difference but there is very little information regarding it. The VMware disk benchmarking thread has this box ticked and set to 500. Other websites and documents I’ve seen don’t check this box at all. On the 3PAR I’m currently testing this can make a difference of 100% to the IOPS I’m seeing. Strange that there isn’t more information about this setting. You omit it completely from your article
Thanks for reading. Yes, it can have an impact. It’s best to leave it off because the connection rate will scale equal to how many workers you have. Forcing it to be a static number will prevent that from happening. Its purpose is more around emulating an existing workload, and not brute force I/O measurement. The former is more challenging anyway because in many cases, the I/O is transactional; where a write may be waiting for the read behind it, which is waiting for a write, etc.
Below is the official definition of the iometer “test connection rate” in the “Disk Targets” section (not the “Network Targets” section)
The Test Connection Rate control specifies how often the worker(s) open and close their disk(s). The default is off, meaning that all the disks are opened at the beginning of the test and are not closed until the end of the test. If you turn this control on, you can specify a number of transactions to perform between opening and closing. (A transaction is an I/O request and the corresponding reply, if any; see the Reply field in the Edit Access Specification dialog for more information). If Test Connection Rate is on, the worker opens all its disks at the beginning of the test. When the specified number of transactions has been performed to a particular
disk, that disk is closed, and is re-opened again just before the next I/O to the disk. The number of transactions can be zero, in which case the worker just opens and closes the disks repeatedly. Each open + transactions + close sequence is called a connection. The time from the initiation of the open to the completion of the corresponding close is recorded for each connection, and the maximum and average connection time and the average connections per second are reported.