I'm connecting to a redis cluster node using
redis-cli -c -p 7001
When I issue info command or dbsize command, I get the count of keys that only reside on that node, and not the count of all the keys across all the nodes in my cluster.
However, if I ask for a key which doesn't reside on this node, it gets me the key from that node.
What if I want keys * to yield all keys from all the available nodes?
When talking to a Redis instance of any kind, you are only talking to that specific instance. Thus any commands are only executed in and for that instance's context. If you want to aggregate key counts across the cluster you have to issue the command on every master node and sum the results.
Related
From the documentation this seems how flushall would work but in practice it is not working that way. When I use the command flushall it only flushes the keys from the db instance the cli is assigned to.
Redis flushall documentation
Delete all the keys of all the existing databases, not just the currently selected one. This command never fails.
The time-complexity for this operation is O(N), N being the number of keys in all existing databases.
For example if my cluster redis-cli has started and I search for a key and the node cli changes from 7000 to 7002 corresponding with the key that the hash is located i.e. server 7002 and then do a flush all it will delete the key per that server.
However, the other keys remain.
Is there a way to flushall meaning delete all keys across all masters and slaves?
Yes. You can use the cli's --cluster switch with the call command - it will execute the provided command on each of the cluster's master nodes (and will replicate, as FLUSHALL is a write command, to their respective slaves).
This should do it:
$ redis-cli --cluster --cluster-only-masters call <one-of-the-nodes-address>:<its-port> FLUSHALL
I know at standalone instance , it works ok .
But at redis cluster, can multi set command on different slots ?
NO. You cannot run command in Redis cluster where keys belong to multiple slots.
You can use hash tags to do multiple key operations in cluster mode.
As we known, redis cluster have 16384 hash slots.when redis client connect to redis cluster, how redis client to connect real save data redis node?
The rule is quite simple, redis use CRC16(key) % 16384 to determine which slot will a key go in. If you want to do everything by yourself, you just need to calculate every key's crc16.
But generally, you don't need to do these by yourself. At the moment, almost every well-known client, like jedis, predis,StackExchange.Redis , supports the cluster mode.
I have a cluster of 3 rabbitmq nodes spread out on 3 different servers. The second and third node joins the first node and forms the cluster. In the process of testing for failover I am finding that once the primary node is killed, I am not able to make it rejoin the cluster. The documentation does not state that I have to use join_cluster or any other command, after startup. I tried join_cluster but it is rejected since the cluster with name is the same as the node host. Is there a way to make this work?
cluster_status displays the following (not from the primary node):
Cluster status of node 'rabbit#<secondary>' ...
[{nodes,[{disc,['rabbit#<primary>','rabbit#<secondary>',
'rabbit#<tertiary>']}]},
{running_nodes,['rabbit#<secondary>','rabbit#<tertiary>']},
{cluster_name,<<"rabbit#<primary>">>},
{partitions,[]}]
On one of the nodes which are in the cluster, use the command
rabbitmqctl forget_cluster_node rabbit#rabbitmq1
To make the current cluster forget the old primary.
Now you should be able to rejoin the cluster on the old primary (rabbitmq1)
rabbitmqctl stop_app
rabbitmqctl join_cluster rabbit#rabbitmq2
rabbitmqctl start_app
See the reference cluster guide
A quote from here
Nodes that have been joined to a cluster can be stopped at any time.
It is also ok for them to crash. In both cases the rest of the cluster
continues operating unaffected, and the nodes automatically "catch up"
with the other cluster nodes when they start up again.
So you just need to start the node that you killed/stopped. Doesn't make a difference if it's "primary" or not - if it was primary and then killed, some other node becomes the primary one.
I've just tested this (with docker of course) and works as expected.
What is the right way to benchmark the redis-cluster(released recently in 3.0 RC). AFAIK, redis-benchmark utility hits only one node of the cluster.
EDIT:(Details)
My single instance of redis without any clustering gives a throughput of ~90 k set/get operations but the cluster setup fails big time.
SETUP 1: 8 core machine running a cluster of 3 masters+ 3 slaves(all on the same machine)
I also run 3 benchmarking utilities on the same machine. The throughput drops to 25 k on each master node.
This makes me think that I am, perhaps, running one too many processes for the number of cores on my machine.
Setup 2: I update the setup to have 3 Masters and 0 slaves. Interestingly, this doesn't help the case either and the throughput is still 25 k on each machine
The benchmarking command that I am running is: redis-benchmark -p 7000 -n 10000000 -t set,get
Any help on this front would be appreciated.
As Josiah said, you are getting 2/3 errors. So to benchmark in the proper way, identify a key which is surely in each node (just connect with redis-cli and use GET/SET brute forcing names). Then use redis-benchmark using the key you found to be, for example, in node A, as an hash tag in order to generate only keys that will hash to that node. So you can do:
redis-benchmark -r 100000 -n 1000000 set 'key{your_hash_tag}:__rand_int__' __rand_int__
This way you'll generate different 100k random keys with names that will hash to the node you are testing. In the example above your_hash_tag is just the key name you found to be in the node you are testing.
First, it is not clear to me that you are actually benchmarking multiple Redis cluster masters with your description. You state, "The benchmarking command that I am running is: redis-benchmark -p 7000 -n 10000000 -t set,get". That will only actually benchmark a single Redis cluster master, specifically the one at port 7000. If you want to benchmark other servers, you have to provide different port numbers when running redis-benchmark multiple times (using the -p option).
Now, even if you do run different commands to hit the different master servers for your cluster, roughly 2/3 of the commands that you perform will result in errors instead of normal command execution simply because the redis-benchmark command is sending commands to a cluster server, and that cluster server does not necessarily hold the shards for the keys that are being operated on.
But what you have really highlighted is that there doesn't seem to be a readily-available solution for benchmarking Redis cluster. A proper benchmark would need to figure out which shards are where, calculate keys to operate on based on the servers and shards, then finally send commands to perform the benchmark, but also raise an error if a shard moves during the benchmark. As far as I know, the software to do this does not yet exist.
Updating this thread with my own answer so as not to leave the answer buried under the comments. When benchmarking the cluster, care must be taken to distribute the 'redis-benchmark' utility. Running them all from the same machine, and even worse from the same machine running the redis cluster, is a good way to lower the throughput. Running the 'redis-benchmark's from a different machine solved the throughput issue for me.
Also, as antirez pointed out, one should send the right keys to each node so that you are not dealing with 2/3 errors.