I have two questions regarding the reddison client:
Does redisson support automatic synchronization of local cache with remote redis cache (when remote cache data change or invalidate)?
I understand that redisson supports data partitioning only in pro edition but isn't that feature already supported OOTB by redis cluster mode? Am I missing something here?
Answering to your questions:
RLocalCachedMap has two synchronization strategies:
INVALIDATE - Used by default. Invalidate cache entry across all RLocalCachedMap instances on map entry change.
UPDATE - Update cache entry across all LocalCachedMap instances on map entry change.
Right, all Redisson objects works also in cluster mode. Each object tied to some Redis node and its content always remain only on the same Redis node and not distributed. If your object couldn't fit in single Redis node then you need to use data partitioning feature. This feature evenly distributes content of object across multiple Redis nodes in cluster.
Re: "local cache truely local" -- I think you can just use a java Map, initially populate it with a RMap contents then from then on just serve your requests from the 'truely local' map in memory.
Related
I have 3 redis instance with redis. One is the master and the other two, are the slaves. I have connected to master node and get info by redis-cli with INFO command. I can see the parameter cluster_enabled:0 and
#Replication
role:master
connected_slaves:2
slave0:ip=xxxxx,port=6379,state=online,offset=15924636776,lag=1
slave1:ip=xxxxx,port=6379,state=online,offset=15924636776,lag=0
And the keyspace, each node has different dbs. I need to migrate all data to a single memorystore in GCP but I don't know how. Anyone can help me?
Since the two nodes are slaves and clustering is not enabled, you only need to replicate the master node. RIOT is a great tool for migrating data in and out of Redis.
However, if you say DB by node do you mean redis DB that you access by select? In that case you'll need to prefix keys as there may be overlap between the keysets of the DBs.
I think setting up another Redis cluster in a single node configuration is the least of your worries.
The real challenge for you would be migrating all your records over to the new setup. This is not a simple question to answer and would depend heavily on multiple factors:
The total size of your data being migrated
Is this is a live Database in production
Do you want to keep the two DB schemas in your new configuration separate?
Ok, I believe currently your Redis Instances are hosted on Google Compute Engine.
And you are looking to migrate to Memorystore for Redis.
As mentioned here, you can leverage Redis snapshots for this. It provides you step-wise instructions on how to achieve this, leveraging GCS buckets as transient storage.
import data into Cloud Memorystore instances using RDB (Redis Database Backup) snapshots, as well as back up data from existing Redis instances.
I have large number of key-value pairs of different types to be stored in Redis cache. Currently I use a single Redis node. When my app server starts, it reads a lot of this data in bulk (using mget) to cache it in memory.
To scale up Redis further, I want to set up a cluster. I understand that in cluster mode, I cannot use mget or mset if keys are stored on different slots.
How can I distribute data into different nodes/slots and still be able to read/write in bulk?
It's handled in redis client library. You need to find if a library exists with this feature in the language of your choice. For example, if you are using golang - per docs redis-go-cluster provides this feature.
https://redis.io/topics/cluster-tutorial
redis-go-cluster is an implementation of Redis Cluster for the Go language using the Redigo library client as the base client. Implements MGET/MSET via result aggregation.
Actually I'm using a configuration of Redis Master-Slaves with HAProxy for Wordpress to have High Avaibility. This configuration is nice and works perfect (I'm able to remove any server to maintenance without downtime). The problem of this configuration is that only one Redis server is getting all the traffic and the others are just waiting if that server dies, so in a very high load webpage can be a problem, and add more servers is not a solution because always only one will be master.
With this in mind, I'm thinking if maybe I can just use a Redis Cluster to allow to read/write on all nodes but I'm not really sure if it will works on my setup.
My setup is limited to three nodes the most of times, and I've read in some places that Redis cluster minimal setup is three nodes, but six is recommended. This is rational because this setup allow to have Slaves nodes that will become Masters if her Master dies, and then all data will be kept, but what happend if data don't cares?. I mean, on my setups the data is just cached objects, so if don't exists it just create it again so:
The data will be lost (don't care), and the other nodes will get the objects from clients again, to serve it on later requests (like happen if a Flush the data).
The nodes will answer that data doesn't exists and will reject to cache because the object would have to be on other node that is dead.
Someone know it?
Thanks!!
When a master dies, the Redis cluster goes to a down state and any command involving a key served by the failed instance will fail.
This may differ from some other distributed software because Redis Cluster is not the kind of program that every master holds all data. In fact, the key space is horizontally partitioned and each key is served by only one master.
This is mentioned in the specification:
The key space is split into 16384 slots...
a single hash slot will be served by a single node...
The base algorithm used to map keys to hash slots is the following:
HASH_SLOT = CRC16(key) mod 16384
When you setup a cluster, you certainly ask each node to serve a set of slots, and each slot can only be served by one node. If one node dies, you lose the slots on this node unless you have a slave failover to serve them, so that any command involving keys mapped to these slots will fail.
I am working on setting up a Redis Cluster.
My understanding is that if I have a 6 node cluster (3 masters, each with one slave), by default, all reads and writes will happen on the masters. If any of the masters fail, then a slave will get promoted to master.
I would like to be able to allow slaves to do some of the reading as well (it is okay if the reads are a little stale).
I saw that the cluster spec supports the READONLY flag. It sounds like I can use this to accomplish what I want. https://redis.io/commands/readonly
My question is, if I have a master and a slave, both of which can perform read operations, how is it decided which node (master or slave) is responsible for fulfilling the read operation? Is it some kind of round-robin approach? Is it up to the client to decide? If it helps, I am using ioredis as my client: https://github.com/luin/ioredis
Found the answer: it depends on the client you are using. In the case of Ioredis, the options are documented here:
https://github.com/luin/ioredis#read-write-splitting
When I add a node in a redis cluster, it has 0 hash slots. Why redis cluster doesn't automatically does a resharding operation in order to make the new node fully functional?
As you can see here, redis supports now automatic partitioning.
The process of adding a node consists of two steps:
Introduce the node to other nodes via CLUSTER MEET so that all the nodes start to communicate via cluster bus
Make the node to act as Master via CLUSTER ADDSLOTS or as a slave via CLUSTER REPLICATE
The separation helps to keep the commands simple.
Automatic resharding is part of the Redis 4.2 roadmap
regard of my expirence, automatic reshard is not i want.
the case i was dealing with is that some node has high read throughput(100k qps), so i add new nodes to reshard only these high load node for the purpose of decreasing the pressure.
you may ask why the load is different? Cause we use hash tag (eg. {user}123456 )to ensure the same kind data stored on the same node.
so automatic reshard is useless.