I am thinking of using redis in my java application either through jedis or redisson client. I have a use case where I need to show all the map entries in redis on UI. I can't fetch them all at once because the data is around 100 mb which is a lot over network. How can I fetch the entries in the map by providing a range for pagination?
Related
I'm currently having problem dealing with redis-cluster.
creating a redis cluster, I'm using the "redis/redis-stack-server:latest" docker image.
I am doing a query test using RediSearch, but in standalone mode, the number of requests per second is not high than other DBs. so I'm trying to optimize the speed using Redis-Cluster.
When I first started redis-cluster, I thought I would be able to import it across nodes. However, the test results didn't. I can only access the data within the same node and cannot ft.search (RediSearch) across all the nodes.
so I looked through the document again to find a solution. The document related to RediSearch says to use RSCoordinator.
When Running RediSearch in a clustered database, you can span the index across shards using RSCoordinator. In this case the above does not apply.
https://redis.io/commands/ft.create/
but looking at github, it seems that RSCoordinator is already built-in.
https://github.com/RediSearch/RediSearch/tree/638de1dbc5e641ca4f8943732c3c468c59c5a47a
Is there a way for redis-cluster to fetch data across all nodes?
Additionally, I wonder if the reason why Redis does not have higher requests per second than other DBs (MongoDB, PostgreSQL, etc..) in large dataset(about 2,400,000) is because Redis is Single Thread? Are there other factors?
I use redis with my springboot application. The memory policy is lfu and would like to see the statistics of hotkeys.
One way is to connect to redis and run
./redis-cli --hotkeys
But it would be better to monitor the top x hotkeys, and present it in grafana dashboard via Prometheus. Can it be achieved in Redis server metrics? or I should use a redis client library and let it collect the key retrieved in the springboot application?
At its current (v5.0.5) version, Redis does not provide a convenient way to collect "hot keys", meaning there is no single command that returns that. What you can do is either use the cli (as noted) or rewrite its logic (scanning and calling OBJECT FREQ on each key) in your app.
I have large number of key-value pairs of different types to be stored in Redis cache. Currently I use a single Redis node. When my app server starts, it reads a lot of this data in bulk (using mget) to cache it in memory.
To scale up Redis further, I want to set up a cluster. I understand that in cluster mode, I cannot use mget or mset if keys are stored on different slots.
How can I distribute data into different nodes/slots and still be able to read/write in bulk?
It's handled in redis client library. You need to find if a library exists with this feature in the language of your choice. For example, if you are using golang - per docs redis-go-cluster provides this feature.
https://redis.io/topics/cluster-tutorial
redis-go-cluster is an implementation of Redis Cluster for the Go language using the Redigo library client as the base client. Implements MGET/MSET via result aggregation.
I am using Jedis to connect to Redis and push data into a list. I am using rpush for the JSON data.
These are the steps I do:
Fetch Data from Rabbitmq
Collect info from JSON data and prepare a key , value pair
Push the data into redis using the key and the value.
I dont see my code scaling more than 3000 requests per second.
Note:
I am not using pipeline , every message will result in getting jedis resource , add it to redis and close of resourse.
Options for persisting faster in Redis are
Pipelining
Jedis Connection Pooling
To Avoid:
3. No frequent opening/closing of resource, i.e open a resource and reuse it
Good link:
https://tech.trivago.com/2017/01/25/learn-redis-the-hard-way-in-production/
How I solved My Problem:
My design was perfectly fine. But I was pushing data into the same key for all my tests. When I started pushing data into different keys Performance increased hugely.
I have read that netflix uses evcache , which is a wrapper over memcache and evcache proves better than memcache
In general it is said that redis server as a better cache than memcache, was trying to find the comparisons of redis and evcache.
Does redis scale as well as evcache or memcache? I am assuming that evcache scaling is tried and tested (hence works good for netflix)
EVCache is a functionality add wrapper over memcache. It is an application that Netflix devs wrote to add functionality they need in their cache layer while using memcache as the underlying data store. You can write your own EVCache to use redis as the data store
Comparing redis to Evcache is not the correct comparison as they operate on two different layers.
Does redis scale as well as evcache or memcache?
Redis can scale to many hundreds of thousands of requests per second.
In general, redis is preferred over memcache because of its many in built data structures
Redis is single threaded so once CPU usage hits 80+% it is better to run another instance instead of giving it a bigger server