I have enabled persistence in IgniteConfiguration. I have multiple CacheConfigurations. Now for some cache i want to disable persistence. How can i do it in ignite.
Can you please tell me correct way. I checked there is no api available.
While this can't be done on a per-cache basis, you can enable/disable persistence for a data region: https://apacheignite.readme.io/docs/memory-configuration#data-regions
If you want to enable persistence for only some of your caches, leaving others in-memory only, you should do the following:
Configure two data regions.
Enable persistence only for one of the regions.
Assign each cache to the appropriate region, based on whether you need persistence for that cache or not.
Related
I'm building a Spring Boot application that uses Spring Cache with a Redis backing store and needs to synchronize the updates made to the cache.
The caching is not made on the fly, but by an scheduled process that updates the cache periodically.
The algorithm I came up with is:
periodically the instances will check if the Redis cache is older than some predetermined time
if that's the case, the instance will try to acquire a lock on some Redis key
if the instance successfully locks the key, it will then proceed with the update
if some other instance already locked the key, move on
all instances can still read the cache
Everything is more or less already built, all I need is to implement the locking/releasing mechanism.
Spring Cache is using Lettuce to interact with Redis, what is the best way to get an connection to Redis and manage the locking mechanism?
As you may already be aware, Spring's Cache Abstraction provides simple coordination amongst multiple Threads in a single Spring [Boot] application process using the sync attribute on the #Cacheable annotation (see ref doc).
NOTE: Despite the comment ("... use the sync attribute to instruct the underlying cache provider to lock the cache entry while the value is being computed. As a result, only one thread is busy computing the value, while the others are blocked until the entry is updated in the cache.") in the documentation, the locking mechanics is handled by the core framework itself, and in most cases, not the provider. Anyway...
However, this "coordination" is only per-process and will not work for multiple Spring [Boot] application instances, or (OS) JVM processes. In this case, you need some form of distributed locking across your multiple Spring [Boot] application instances to coordinates access to shared cache entries stored in the single Redis server (cluster) shared by your Spring [Boot] application instances.
I am no Redis expert (I am still learning), but I am familiar with similar NoSQL stores (Apache Geode/VMware GemFire, Hazelcast, etc) and distributed locking mechanisms. I see that distributed locking is possible to achieve with Redis as well. In a quick search, I found "Distributed Locking" in Redis, and specifically, "Building a lock in Redis". This is probably the best way to go.
In addition, if you want to make this distributed locking automatically/transparently available through Spring's Cache Abstraction, then you could possibly create a custom AOP Aspect and weave this Aspect together with the framework provided Caching Aspect (Interceptor), being conscious of ordering, as 1 idea.
Alternatively, you could implement wrapper implementations for the Spring Cache and CacheManager SPI interfaces that implement distributed locking on top of the core Redis Cache and CacheManager provider implementations provided by Spring Boot/Spring Data Redis.
Of course, there are multiple ways to go about this. Just tossing out more ideas, but have a look at the distributed locking information in the book.
What I'd like to achive is to be able to scale out Azure SQL Database.
Business Critical tier has this feature to enable several read-only replicas. This is a great feature that would let me offload some traffic over to those replicas
The problem for me is that I don't understand how to manage those replicas and I don't understand how load balancing works there. Basically, I should be able to manage how many replicas there are, I probably need to have around 10 of replicas and have traffic equality balanced across them
Is this something that I could do?
If you look at the note here, it says
In Premium and Business Critical service tiers, only one of the read-only replicas is accessible at any given time. Hyperscale supports multiple read-only replicas.
This means Premium and Business critical service tiers may have multiple replicas (3-4) but only 1 of them is accessible as read only. There is no control as to which one and there is no load balancing capabilities. It is only good for use if there is a separate application which require read access only (example analytical workloads).
For Hyperscale you can refer to this.
Hyperscale allows for 1-4 secondaries(1 by default). The link states
If more than one secondary replica is present, the workload is distributed across all available secondaries.
There is no additional information and it seems the the control to load balance is abstracted away from us.
You can definitely not achieve your requirement of 10 read replicas from any of these configurations.
I'm looking for persistence configuration in IgniteQueue, but couldn't find any useful documents. In Collectionconfiguration there is no option for DataRegion. Is there any way to persists IgniteQueue to solve dataloss problem ?.
Data structures are created in the default data region, so in order to make IgniteQueue persisted you need to make the default data region persisted using DataStorageConfiguration#setDefaultDataRegionConfiguration
There is also a JIRA ticket for making it possible to specify a data region for structures: https://issues.apache.org/jira/browse/IGNITE-6820
Nobody seems to be interested in fixing it though.
I have a fairly simple Akka.NET system that tracks in-memory state, but contains only derived data. So any actor can on startup load its up-to-date state from a backend database and then start receiving messages and keep their state from there. So I can just let actors fail and restart the process whenever I want. It will rebuild itself.
But... I would like to run across multiple nodes (mostly for the memory requirements) and I'd like to increase/decrease the number of nodes according to demand. Also for releasing a new version without downtime.
What would be the most lightweight (in terms of Persistence) setup of clustering to achieve this? Can you run Clustering without Persistence?
This not a single question, so let me answer them one by one:
So I can just let actors fail and restart the process whenever I want - yes, but keep in mind, that hard reset of the process is a lot more expensive than graceful shutdown. In distributed systems if your node is going down, it's better for it to communicate that to the rest of the nodes before, than requiring them to detect the dead one - this is a part of node failure detection and can take some time (even sub minute).
I'd like to increase/decrease the number of nodes according to demand - this is a standard behavior of the cluster. In case of Akka.NET depending on which feature set are you going to use, you may sometimes need to specify an upper bound of the cluster size.
Also for releasing a new version without downtime. - most of the cluster features can be scoped to a set of particular nodes using so called roles. Each node can have it's set of roles, that can be used what services it provides and detect if other nodes have required capabilities. For that reason you can use roles for things like versioning.
Can you run Clustering without Persistence? - yes, and this is a default configuration (in Akka, cluster nodes don't need to use any form of persistent backend to work).
So here's the problem, I've created a database model. When I create the model, a = Model(args), and then perform a.put(), GAE seems to automatically update the memcache, because all the data seems up-to-date even without me hitting the database. Logging the number of elements in the cache works also shows the correct number of elements. But I'm not manually updating the cache. How do I prevent this? Cheers.
You can set policy functions:
Automatic caching is convenient for most applications but maybe your application is unusual and you want to turn off automatic caching for some or all entities. You can control the behavior of the caches by setting policy functions.
Memcache Policy
That's for NDB. You don't say what language/DB you are using but I'm sure it's all similar.