I use redis with my springboot application. The memory policy is lfu and would like to see the statistics of hotkeys.
One way is to connect to redis and run
./redis-cli --hotkeys
But it would be better to monitor the top x hotkeys, and present it in grafana dashboard via Prometheus. Can it be achieved in Redis server metrics? or I should use a redis client library and let it collect the key retrieved in the springboot application?
At its current (v5.0.5) version, Redis does not provide a convenient way to collect "hot keys", meaning there is no single command that returns that. What you can do is either use the cli (as noted) or rewrite its logic (scanning and calling OBJECT FREQ on each key) in your app.
Related
I have large number of key-value pairs of different types to be stored in Redis cache. Currently I use a single Redis node. When my app server starts, it reads a lot of this data in bulk (using mget) to cache it in memory.
To scale up Redis further, I want to set up a cluster. I understand that in cluster mode, I cannot use mget or mset if keys are stored on different slots.
How can I distribute data into different nodes/slots and still be able to read/write in bulk?
It's handled in redis client library. You need to find if a library exists with this feature in the language of your choice. For example, if you are using golang - per docs redis-go-cluster provides this feature.
https://redis.io/topics/cluster-tutorial
redis-go-cluster is an implementation of Redis Cluster for the Go language using the Redigo library client as the base client. Implements MGET/MSET via result aggregation.
I have two questions regarding the reddison client:
Does redisson support automatic synchronization of local cache with remote redis cache (when remote cache data change or invalidate)?
I understand that redisson supports data partitioning only in pro edition but isn't that feature already supported OOTB by redis cluster mode? Am I missing something here?
Answering to your questions:
RLocalCachedMap has two synchronization strategies:
INVALIDATE - Used by default. Invalidate cache entry across all RLocalCachedMap instances on map entry change.
UPDATE - Update cache entry across all LocalCachedMap instances on map entry change.
Right, all Redisson objects works also in cluster mode. Each object tied to some Redis node and its content always remain only on the same Redis node and not distributed. If your object couldn't fit in single Redis node then you need to use data partitioning feature. This feature evenly distributes content of object across multiple Redis nodes in cluster.
Re: "local cache truely local" -- I think you can just use a java Map, initially populate it with a RMap contents then from then on just serve your requests from the 'truely local' map in memory.
I have 3 redis servers running in docker containers. From redis-cli I can SET on specific server.
SET myValue 100
How can I do this with StackExchange.Redis client?
I don't see anything in server api that allows to do that. Bear in mind that I don't know much about Redis at all.
var connection = ConnectionMultiplexer.Connect("localhost:6379,localhost:6380,localhost:6381");
var server = connection.GetServer("localhost", 6381);
server.???
SE.Redis expects to be managing a single logical keyspace; the support for multiple nodes is intended either for master/replica setups, or for redis-cluster (although, in the case of cluster, node discovery is achieved via the redis API, so a single node would be fine if it is reachable). With that in place: the selection of servers is implicit from the operation (i.e. writes need to go to a master, and in the case of "cluster", the keyspace shard mapping should be applied).
If you want to write to separate servers as though they are separate databases, you should use a connection per server; not a single connection that spans them all. Right now, SE.Redis is probably detecting 3 master nodes and electing to use one of them arbitrarily. You can see what it thinks by passing a TextWriter to the Connect/ConnectAsync method.
I'm trying to implement cuckoo filter in Redis. What I have till now works fine except that it just inserts all the values on a single node even when working on a cluster.
In order to implement it on multiple nodes, I'm thinking of directing different elements to different nodes using some hash function. Is there any command or function call in Redis that allows forcing of elements to a particular node using its key or number, or even a particular slot?
For reference, this is the implementation of cuckoo filter I have till now.
As an aside, is there any existing implementation of Cuckoo Filter or Bloom Filter on distributed nodes in Redis that I can refer to?
This page explains how Redis cluster works and how the redis-cli works when using it in cluster mode. Other clients do a better handling of the operations in cluster mode, but the basic functionality of the redis-cli should work for simple tests.
If you check the code of other data structures (for example, hash or set) that come with Redis, you'll notice that they do not have code to deal with cluster mode. This is handled by the code in cluster.c, and should be orthogonal to your implementation. Are you sure you have correctly configured the cluster and the Redis cli?
I'm using redis hashes to store metadata associated with a single key which in my case is userId. I have a batchAPI where I pass in a batch of userIds and get the metadata back. Redis Cluster doesn't support multi key commands so What's the best way to achieve this with RedisCluster? I am using Jedis as the Java Client to connect to Redis.
If Lettuce is an option for you, then from 4.0.Final MGET on cluster is supported:
The cluster API was extended to run a command on multiple nodes and invocation to multi-key commands DEL, MGET, MSET and MSETNX perform automatic pipelining if the keys belong to different slots/masters.
(Source: https://github.com/mp911de/lettuce/releases/tag/4.0.Final)