I start ignite(persistence through mysql data) with spring, after started ,the ignite cache is empty.
but How to load data from mysql to fill the ignite cache, are there any examples.
thank you
When cache store is configured, call IgniteCache.loadCache() method, which will delegate to CacheStore.loadCache() and populate the cache with data from persistent store.
See https://apacheignite.readme.io/docs/persistent-store#section-loadcache-
Related
I am new to Redis cache and this cache we are using in my node API application. On the start-up of this application, we are setting the values in the cache. Do we need to reload the values when the Redis server is restarting? Please help on this.
Thanks in advance
Redis has a configuration option that writes the content of the database to disk, and when it restarts, load the data from disk into the database. The details on this option are in the docs here: https://redis.io/topics/persistence
If you need some data to always be present in Redis, then you'll either need to implement persistence above, or do something like this in your app:
# when retrieving something from Redis cache
if (item_is_in_cache('my_key') { #inexpensive operation
retrieve_item_from_cache('my_key'); #inexpensive operation
} else {
store_important_data_in_cache(); #expensive operation
}
What this pseudo-code does is first check that the required data is in the cache, and retrieve it if it is. Checking and retrieving data from a Redis cache is an inexpensive operation, meaning the resources required are low. If the data isn't in the cache (ie, the Redis server recently started), we have to put the important data in the cache. This can be an expensive operation (more resources used than checking/retrieving data) depending on the amount of data required.
I'm using Apache Ignite as a distributed cache whose configuration I've generated based on an existing database using the Ignite Web Console--it's a writethrough cache that will periodically persist cached data to the Postgres database. However, I want to write unit tests in Java for my project, and do not have a reliable test database to use.
Part of what I'm wanting to test are the cache queries I'm occasionally running on my Ignite cache--I wanted to use sql queries to do this. However, I can't figure out how to preserve the queryEntities from my cache configuration without also having the database. I tried making a new xml file for test purposes that only configures the caches I need, and only sets the query entities (not the datastore or any db information), but when I run the test I get a "Failed to initialize DB connection" error--even though there is no DB defined in my config.
Is there a way to leverage these query entities without actually connecting the cache to a database? If not, is there a good way to spin up a postgres database as a part of a unit test?
You need to check persistence store configuration and disable that first to have everything in memory.
Next, make sure you are not initializing any DB connection while having your test cache configuration(You already said you looked after this fact).
cacheCfg.setWriteThrough(false).setReadThrough(false) should do the trick when defining a cache (note that after cache is started cfg can't be changed)
I am trying to position Ignite as Query Grid for databases such as Kudu, Hbase, etc.. Thus, all data silos will be queried over Ignite with read/write through. How this is possible? Are there any integrations with them?
The first time, SQL query runs, it will need to pull the data from such databases and create the key/value on Ignite.
Then, if one/two/three node goes down, eventually the data stored in memory will be lost. How the recovery is done or not possible?
Thanks
CK
Ignite SQL is unable to load specific data by query from external store, it's only possible on API get()/getAll() operations. To be able querying data you need load them into Ignite at first, for example, with loadCache(). Internally this function does a query to target database and transforms response into key-value manner.
BTW, if you enable persistence in Ignite, it will know the structure of data and will be able to query them, even if not all entries loaded into memory.
In case of node crash traditionally used data replication between nodes. In Ignite it's named backups. If you loose more nodes than backups set, then you'll need to preload data from store again.
Is there any easy way in Ignite to persist to disk after the Ignite servers are up and running and filled in with data?
I have seen https://apacheignite.readme.io/docs/distributed-persistent-store#section-usage but it seems you need to supply the XML property at startup of your Ignite topology in order to persist to disk.
There's no easy way I can think of. You will need to start new nodes, with persistent data region, and somehow transfer data to those nodes to newly created persistent caches. The easiest way will be to create them as a part of new cluster.
I've got a stored procedure that loads some data (about 59k items) and it takes 30 seconds. This SP must be called when the application starts. I was wondering if there's a reasonable way to invalidate the Redis cache entry via SQL ...any suggestion?
Thanks
Don't do it from your SQL, do the invalidation / (re)loading to Redis from your application.
The loading of this data into your application should be done by a separate component/service/module/part of your application. So that part should have all the responsibility of handling the needed data, including (re)loading it into the app, invalidating and reloading into Redis and so on. You should see your Redis server as an extension of your application cached data and not of your sql server data. That's why you should not link your relational database to your Redis. If you are going to change how you save this data into Redis that should not affect the SQL part, but only the application, and actually only the part of your application specialized with this.