I've ran into a situation where using redis-cli --pipe is feasible, but I may need too much work to make redis-cli available into some container.
Thus, I'm not sure if what redis-cli --pipe does is almost what StackExchange.Redis may do with IDatabase.CreateBatch.
The concept of batch is something which does not pertain to Redis but, rather to StackExchange.Redis and its multiplexed architecture: it mainly allows to send a sequence of commands through a multiplexer with the guarantee of not having any other command external to that sequence sent in-between those ones.
On the other side, redis-cli --pipe uses Redis pipelining: StackExchange.Redis allows to use that by simply making use of the Task-based Asynchronous Pattern, as you can read in its documentation.
Related
The essence of the redis pipeline is to change the read and write order of the instructions in the pipeline. We usually say that the pipeline is a means of inter-process communication, and the redis pipeline is socket-based communication, the two are not comparable, is there a problem with this understanding?
An interprocess pipes and Redis' network pipelining are different things. One is explained at https://www.tutorialspoint.com/inter_process_communication/inter_process_communication_pipes.htm and the other at https://en.wikipedia.org/wiki/Protocol_pipelining
Both are different:
Redis pipeline is a network optimization strategy implemented on clients, where multiple commands are sent to Redis-Server at once.
More about redis-pipeline on: https://buildatscale.tech/what-is-redis-pipeline/
Unix pipelines are mechanisms for inter-process communication. https://en.wikipedia.org/wiki/Pipeline_(Unix)
I want to use redis lua to implement monitor command instead of redis-cli monitor. But I don't know how.
redis.call('monitor') doesn't work.
You can not call MONITOR from a Redis Lua script - MONITOR is a blocking command so it would block your script forever if allowed to be invoked.
I am using nutcracker in my application to fetch data from Redis nodes.
Just wanted to know whether it supports multi get or it internally fetches all keys one by one.
Yes it does.
You can find all the Redis commands that twemproxy supports here
Is there a way for a client to get notified about failover events in the Redis cluster? If so, which client library would support this? I am currently using Jedis but have the flexibility to switch to any other Java client.
There are two ways that I can think of to check this, one of them is to grep for master nodes on the cluster, keeping in mind their IDs, if the ports changed for any of them then a failover happened.
$ redis-cli -p {PORT} cluster nodes | grep master
Another way, but it is not as robust of a solution is using the consistency checker ruby script, that will start showing errors in writes as an output, which you can monitor and send notifications depending on it, since that happens when the read server is trying to take its master's role.
Sentinel (http://redis.io/topics/sentinel) has the ability to monitor the cluster member, and send a publish/subscribe notification upon failure. The link contains a more in-depth explanation and tutorial.
What is the right way to benchmark the redis-cluster(released recently in 3.0 RC). AFAIK, redis-benchmark utility hits only one node of the cluster.
EDIT:(Details)
My single instance of redis without any clustering gives a throughput of ~90 k set/get operations but the cluster setup fails big time.
SETUP 1: 8 core machine running a cluster of 3 masters+ 3 slaves(all on the same machine)
I also run 3 benchmarking utilities on the same machine. The throughput drops to 25 k on each master node.
This makes me think that I am, perhaps, running one too many processes for the number of cores on my machine.
Setup 2: I update the setup to have 3 Masters and 0 slaves. Interestingly, this doesn't help the case either and the throughput is still 25 k on each machine
The benchmarking command that I am running is: redis-benchmark -p 7000 -n 10000000 -t set,get
Any help on this front would be appreciated.
As Josiah said, you are getting 2/3 errors. So to benchmark in the proper way, identify a key which is surely in each node (just connect with redis-cli and use GET/SET brute forcing names). Then use redis-benchmark using the key you found to be, for example, in node A, as an hash tag in order to generate only keys that will hash to that node. So you can do:
redis-benchmark -r 100000 -n 1000000 set 'key{your_hash_tag}:__rand_int__' __rand_int__
This way you'll generate different 100k random keys with names that will hash to the node you are testing. In the example above your_hash_tag is just the key name you found to be in the node you are testing.
First, it is not clear to me that you are actually benchmarking multiple Redis cluster masters with your description. You state, "The benchmarking command that I am running is: redis-benchmark -p 7000 -n 10000000 -t set,get". That will only actually benchmark a single Redis cluster master, specifically the one at port 7000. If you want to benchmark other servers, you have to provide different port numbers when running redis-benchmark multiple times (using the -p option).
Now, even if you do run different commands to hit the different master servers for your cluster, roughly 2/3 of the commands that you perform will result in errors instead of normal command execution simply because the redis-benchmark command is sending commands to a cluster server, and that cluster server does not necessarily hold the shards for the keys that are being operated on.
But what you have really highlighted is that there doesn't seem to be a readily-available solution for benchmarking Redis cluster. A proper benchmark would need to figure out which shards are where, calculate keys to operate on based on the servers and shards, then finally send commands to perform the benchmark, but also raise an error if a shard moves during the benchmark. As far as I know, the software to do this does not yet exist.
Updating this thread with my own answer so as not to leave the answer buried under the comments. When benchmarking the cluster, care must be taken to distribute the 'redis-benchmark' utility. Running them all from the same machine, and even worse from the same machine running the redis cluster, is a good way to lower the throughput. Running the 'redis-benchmark's from a different machine solved the throughput issue for me.
Also, as antirez pointed out, one should send the right keys to each node so that you are not dealing with 2/3 errors.