I have a redis cache in Azure with maxmemory policy set as Volatile-LRU. When writing to Redis, I am not adding an expiry time for the key. In this case, what will happen when the cache memory get filled?
Under the volatile-lru policy, redis will never evict a key without a expiry. If all of memory is used up by keys that do not have expiry set then the next time you use a command that requires allocating more memory than is available, say SET, the command will fail and you will get this error message:
OOM command not allowed when used memory > 'maxmemory'
You will still be able to use commands that don't allocate memory, like GET. If you get your database into this state, you can use the EXPIRE command to set and expiry time on keys after the fact.
Related
When using BinaryObject for off-heap in-memory-only cache values, do we need to do anything to protect against the cache entry being evicted or expired while accessing fields via BinaryObject::field(String)?
For example, if the cache's data region has the default memory-size eviction (90% full?), or if the cache uses a creation expiry policy, and the region happens to evict entries or the cache expire entries while the code is making several calls to BinaryObject::field(String). Does Ignite automatically ensure that BinaryObject won't access invalid off-heap memory (throwing an exception perhaps), or can the developer use locking / transactions or a "touched" expiry to help prevent this?
Thanks!
BinaryObject instances returned from the Ignite API and accessed by the user code are copies. They do not reference Ignite storage memory directly.
You can work with BinaryObject even after the corresponding cache entry gets evicted.
When a cache object becomes eligible for eviction, it will be removed from memory.
Ignite has several Eviction policies:
Random-LRU
Random-2-LRU
Explained here
This means that if you recently used a cache value(or are using it at the moment that the eviction is taking place), either in BinaryObject representation or not, it will not be evicted. All Eviction algorithms use a LEAST RECENTLY USED algorithm.
Expiry policy on off-heap entries is not working as expected. I am seeing a linear increase in off-heap entries and after some time sudden dip in off-heap size. Is only marking of cache entry done after expiry time and collected later in bulk? I am not finding any documentation regarding the same.
We need to enable eager TTL in the cache configuration. It will create a thread in the background and removes the expired entries.
There are two modes of expiry policy in Ignite: eager and non-eager: https://www.gridgain.com/docs/latest/developers-guide/configuring-caches/expiry-policies#eager-ttl
In case of an eager TTL entries are expired periodically by a background thread. When a non-eager TTL is used, then entries are only expired upon access. So, it's possible for expired entries to be stored in off-heap for some time until they get read.
Also keep in mind that JVM can allocate more memory than is immediately required. The amount of memory allocated by the Java process is not directly related to the amount of data stored in Ignite.
Problem:
I want to set a TTL on a key (to avoid it lasting forever) but I do NOT want to have that specific key to be evicted.
When I am setting the TTL I know when it will be safe to expire that cache, but it is NOT safe to expire the cache before this time, and eviction presents the risk of having this cache expire to early.
Context:
I am using Redis to cache an object in multiple languages, if the underlying data changes however I want to remove all associated caches from Redis.
The way I went around and sorted this problem was to create a SET on Redis that contains a reference to the keys in every language. My concern is that if that SET is evicted - I loose the reference to the other keys, and risk having them persist on the cache when they shouldn't.
What I am looking for
A Redis command that looks something like
PLEASE_DO_NOT_EVICT key
while not preventing that key from expiring after the TTL runs out.
Thanks very much for taking your time to reading and answering!
While I could use wildcard matching to find all of the associated keys, this is WAY slower than SMEMBERS, and I am doing this in an environment where every MS counts, as these objects are created and deleted very frequently, so this query happens very often.
Not having a TTL in these objects means they start building up in memory which is undersirable. And they do tend to stop being referenced after a while
Having a no eviction policy seems risky, and I would very much want to
When creating:
SADD 'object:id:group', 'object:id:spanish'
SETEX 'object:id:spanish', 'Este es el object en espaniol', 100
EXPIRE 'object:id:group', 100
When expiring the group because the object changed:
SMEMBERS 'object:id:group'
=> 'object:id:spanish', 'object:id:english'
DELETE 'object:id:spanish', 'object:id:english'
DELETE 'object:id:group'
You can set the maxmemory-policy to its default value of "noeviction". In this mode, no keys are evicted.
I am experiencing a very strange case happen in production with redis, a large amount of redis keys are evicted unexpectedly even though memory not reach max configuration.
Current redis setting is max mem = 7GB, volatile-ttl.
Most of the keys are set a TTL when store to Redis.
Below graph showing a large drop in redis key eventhough memory at the time was only 3.5GB (<< 7GB)
According to my understanding, Redis would evict keys only when memory reach max-mem. And even when it does, it will only drop keys gradually according to the need for inserting new keys.
Thank you very much!
I want to use Redis in this way
Load the entries (which are reading/editing by user) from file db,
set to expire in a period of time
Edit & read them on Redis
Store back to file db when it is auto-deleted by Redis (while it is
evicted to free memory or it is expired )
This isn't something that Redis does OOTB but with a little effort you can achieve it. See this for more info: https://stackoverflow.com/a/25827681/3160475