I have a question on Redis cache's behavior. Kindly clarify -
Say, for a key "xyz" if the TTL is set to 15 minutes.
And, if its eviction policy in server level is set to "allkeys-lru". Does expired items(cause of TTL) EXPIRE or WAIT until the memory is full?
The eviction policy only applies to what happens when you exceed the max memory. As long as you're within your memory limits, volatile keys will expire when they should be expired.
Once your memory is full, an LRU algorithm kicks in, evicting least recently used keys. In allkeys-lru, it doesn't matter whether a key is expired or not and what is the TTL - the least used items will be evicted. In volatile-lru only expiring keys will be evicted using this algorithm.
Related
Expiry policy on off-heap entries is not working as expected. I am seeing a linear increase in off-heap entries and after some time sudden dip in off-heap size. Is only marking of cache entry done after expiry time and collected later in bulk? I am not finding any documentation regarding the same.
We need to enable eager TTL in the cache configuration. It will create a thread in the background and removes the expired entries.
There are two modes of expiry policy in Ignite: eager and non-eager: https://www.gridgain.com/docs/latest/developers-guide/configuring-caches/expiry-policies#eager-ttl
In case of an eager TTL entries are expired periodically by a background thread. When a non-eager TTL is used, then entries are only expired upon access. So, it's possible for expired entries to be stored in off-heap for some time until they get read.
Also keep in mind that JVM can allocate more memory than is immediately required. The amount of memory allocated by the Java process is not directly related to the amount of data stored in Ignite.
Problem:
I want to set a TTL on a key (to avoid it lasting forever) but I do NOT want to have that specific key to be evicted.
When I am setting the TTL I know when it will be safe to expire that cache, but it is NOT safe to expire the cache before this time, and eviction presents the risk of having this cache expire to early.
Context:
I am using Redis to cache an object in multiple languages, if the underlying data changes however I want to remove all associated caches from Redis.
The way I went around and sorted this problem was to create a SET on Redis that contains a reference to the keys in every language. My concern is that if that SET is evicted - I loose the reference to the other keys, and risk having them persist on the cache when they shouldn't.
What I am looking for
A Redis command that looks something like
PLEASE_DO_NOT_EVICT key
while not preventing that key from expiring after the TTL runs out.
Thanks very much for taking your time to reading and answering!
While I could use wildcard matching to find all of the associated keys, this is WAY slower than SMEMBERS, and I am doing this in an environment where every MS counts, as these objects are created and deleted very frequently, so this query happens very often.
Not having a TTL in these objects means they start building up in memory which is undersirable. And they do tend to stop being referenced after a while
Having a no eviction policy seems risky, and I would very much want to
When creating:
SADD 'object:id:group', 'object:id:spanish'
SETEX 'object:id:spanish', 'Este es el object en espaniol', 100
EXPIRE 'object:id:group', 100
When expiring the group because the object changed:
SMEMBERS 'object:id:group'
=> 'object:id:spanish', 'object:id:english'
DELETE 'object:id:spanish', 'object:id:english'
DELETE 'object:id:group'
You can set the maxmemory-policy to its default value of "noeviction". In this mode, no keys are evicted.
I am experiencing a very strange case happen in production with redis, a large amount of redis keys are evicted unexpectedly even though memory not reach max configuration.
Current redis setting is max mem = 7GB, volatile-ttl.
Most of the keys are set a TTL when store to Redis.
Below graph showing a large drop in redis key eventhough memory at the time was only 3.5GB (<< 7GB)
According to my understanding, Redis would evict keys only when memory reach max-mem. And even when it does, it will only drop keys gradually according to the need for inserting new keys.
Thank you very much!
I have a redis cache in Azure with maxmemory policy set as Volatile-LRU. When writing to Redis, I am not adding an expiry time for the key. In this case, what will happen when the cache memory get filled?
Under the volatile-lru policy, redis will never evict a key without a expiry. If all of memory is used up by keys that do not have expiry set then the next time you use a command that requires allocating more memory than is available, say SET, the command will fail and you will get this error message:
OOM command not allowed when used memory > 'maxmemory'
You will still be able to use commands that don't allocate memory, like GET. If you get your database into this state, you can use the EXPIRE command to set and expiry time on keys after the fact.
Is there a way to make Redis choose a LRU (least recently used) eviction policy for only specific keys? I want a set of keys to be persistent and never be evicted if there's not enough memory. On the other hand, I want another set of keys to be freely evicted if there's low memory.
Redis has an eviction policy which might be good for your case.
You can set the maxmemory-policy to volatile-lru which causes Redis to:
remove the key with an expire set using an LRU algorithm
Which means that keys that are not set with TTL are not volatile, and therefor will not be evicted but keys that have TTL will be removed by Least-Recently-Used order.
Actually, volatile-lru is the default policy, so all you have to do is to make sure TTL is set for the keys you are willing to lose when memory is getting full.
Edit: Since version 3.0 the default eviction policy is "noeviction". (changelog)