Redis-cluster benchmarking - redis

Question

What is the right way to benchmark the redis-cluster(released recently in 3.0 RC). AFAIK, redis-benchmark utility hits only one node of the cluster.
EDIT:(Details)
My single instance of redis without any clustering gives a throughput of ~90 k set/get operations but the cluster setup fails big time.
SETUP 1: 8 core machine running a cluster of 3 masters+ 3 slaves(all on the same machine)
I also run 3 benchmarking utilities on the same machine. The throughput drops to 25 k on each master node.
This makes me think that I am, perhaps, running one too many processes for the number of cores on my machine.
Setup 2: I update the setup to have 3 Masters and 0 slaves. Interestingly, this doesn't help the case either and the throughput is still 25 k on each machine
The benchmarking command that I am running is: redis-benchmark -p 7000 -n 10000000 -t set,get
Any help on this front would be appreciated.

Answer 1

As Josiah said, you are getting 2/3 errors. So to benchmark in the proper way, identify a key which is surely in each node (just connect with redis-cli and use GET/SET brute forcing names). Then use redis-benchmark using the key you found to be, for example, in node A, as an hash tag in order to generate only keys that will hash to that node. So you can do:
redis-benchmark -r 100000 -n 1000000 set 'key{your_hash_tag}:__rand_int__' __rand_int__
This way you'll generate different 100k random keys with names that will hash to the node you are testing. In the example above your_hash_tag is just the key name you found to be in the node you are testing.

Answer 2

First, it is not clear to me that you are actually benchmarking multiple Redis cluster masters with your description. You state, "The benchmarking command that I am running is: redis-benchmark -p 7000 -n 10000000 -t set,get". That will only actually benchmark a single Redis cluster master, specifically the one at port 7000. If you want to benchmark other servers, you have to provide different port numbers when running redis-benchmark multiple times (using the -p option).
Now, even if you do run different commands to hit the different master servers for your cluster, roughly 2/3 of the commands that you perform will result in errors instead of normal command execution simply because the redis-benchmark command is sending commands to a cluster server, and that cluster server does not necessarily hold the shards for the keys that are being operated on.
But what you have really highlighted is that there doesn't seem to be a readily-available solution for benchmarking Redis cluster. A proper benchmark would need to figure out which shards are where, calculate keys to operate on based on the servers and shards, then finally send commands to perform the benchmark, but also raise an error if a shard moves during the benchmark. As far as I know, the software to do this does not yet exist.

Answer 3

Updating this thread with my own answer so as not to leave the answer buried under the comments. When benchmarking the cluster, care must be taken to distribute the 'redis-benchmark' utility. Running them all from the same machine, and even worse from the same machine running the redis cluster, is a good way to lower the throughput. Running the 'redis-benchmark's from a different machine solved the throughput issue for me.
Also, as antirez pointed out, one should send the right keys to each node so that you are not dealing with 2/3 errors.