I did some query against a cache, and I want to save the query result to a persist store, eg, mysql db.
I would ask what's the recommended way to accomplish this? Is there some code example for reference? Thanks!
If you need this data in cache as well, use write-through. Otherwise, just update your DB directly. I don't see how Ignite can participate in this process.
Related
I'm working on a system which will fetch data from a service and put pieces of the response in to a cache and/or into a SQL table.
The cache is needed for consumption by other Java services directly. These services require a more direct connection than the SQL abstraction, so we need to connect directly to the cache.
The table is needed for a JDBC SQL connection to external SQL clients e.g. SQL Workbench, DBeaver, Tableau, 3rd party systems.
My question is how Ignite works regarding caches vs tables. I know it stores its caches as maps similar to other IMDGs. What I guess I don't understand is how that gets turned into a table, or what APIs are available to set/get between the two.
So the question is, how can I take an INSERT from the JDBC/SQL side and query it via the Cache? How can I add() into the Cache and SELECT it from the JDBC/SQL side? If I have a table named "foo", does that also create a cache named "foo"?
Or am I supposed to use one or the other and not bleed between the two? I haven't found many good examples of this, so it seems to be either you use caches or you use tables.
It would be extremely advantageous to have a bridge between the two. We're migrating to Ignite from an H2 implementation where we mushed a Hazelcast cache and H2's SQL together and are hoping Ignite, being built atop H2, has done something similar already.
In particular, I was hoping to use DataStreamers but I'm not finding much in the way of how it relates to the SQL/table side of things.
Ignite cache falls under key-value type of nosql database. You can fire SQL like query from java code to ignite caches as it supports it. For example,
SELECT _KEY, _VAL from "foo".val
Here, foo is your cache name and val is the value part of key-value pair. As this is all NOSQL, relating it to RDBMS SQL is not so much rational, still we can relate all non primary columns in SQL table to the fields of your value object and primary one to the key part.
So, in datastreamer, you can construct collection of key, value objects and stream it. This internally calls nothing but put operation on cache.
To select in SQL fashon, you can fire query like below-
SqlFieldsQuery query = new SqlFieldsQuery(queryString);
FieldsQueryCursor<List<?>> cursor = cache.query(query);
There are multiple ways to do this, SqlFieldsQuery is one of that.
This was answered couple times already, basically you need to refer to Query Entities, Indexed Types or key_type/value_type parameters of CREATE TABLE to make it work. I.e. every entry in cache of correct type will be a row of table and vice versa.
Is there anyway to set pagination using configurations in select statement of in-built database connector of AnyPoint studio in mule?
I have tried reading the doc and I could not find any configurations to achieve pagination. If not should I write my own custom connector to achieve the same?
You need to implement this yourself.
Best way is to take in the query params you use for pagination, store them as variables and use those flowVariables in your select statement.
Depending on the DB you are connecting to you can then use mechanism such as LIMIT and OFFSET for MySQL as explained here
Another option would be to filter a specific range in DataWeave, but in terms of performance that is unnecessary data fetching, that may not be the nicest option because you still fetch all the data and filter it to a range later.
I have a problem to solve and at some point of it. It says this:
I decided to use a relational database to cache calculated data for the next calls.
What does the cache part means? Where is that data sorted? Is it saved in a temporal table? How can I access to that information?
Thanks in advance!!
"Cache calculated data" means the results of some resource-consuming calculations are stored in a database for faster future access without re-calculation. The data can be stored in one or several tables, it can be versioned or not, so particular implementation may vary.
In our web app, we are creating session table in database to store temporary data. So the temp table will be created and destroyed for every user. I have some 300 users for this web app. So for every user these table will be created and destroyed.
i heard that this way of design is not good due to performance issues.
I am using MS Sql server 2005. Is there any way to store a result set temporarily without creating any table.
Please suggest me some solution.
Thanks.
Either:
use a single permanent database table for all users, with a UserID column to filter on
or
just use the session handling ability of your web platform to store the info
It sounds as if you are creating and dropping permanent tables. Have you tried using real temp tables (those with table names beginning with #). OR table variables if you havea small data set. Either of these can work quite well. If you use real temp tables, you need to make sure your tempdb is sized large enough to accomodate the usual amount of users, growing tempdb can cause delays.
I think, a solution at GenerateData is what you are looking for. You can create test/sample databases their and delete them when needed.
Depending on what you're actually doing (and whether you can refactor it) it may be more appropriate to use table variables which are highly performant in general.
There is a question of whther the DB is really an appropriate place to even be trying to persist data sets if it's for your applications benefit - if the question isn't just academic perhaps it would be better to keep the object representation of the data in your app memory?
I would like to know whether is possible to add normal data to a Sesame Database using SQL queries and retrieve them through SQL queries..??
Do you mean adding new tables to the Sesame store so that you can store non-RDF data as well? If you use Sesame's RDBMS store, then it could be possible. But it is probably not such a good idea, because it would be very easy to mess up Sesame's internal schema and put the store into an inconsistent state.
If you use Sesame's native store, then it's not possible.