I have been working on ApacheIgnite in my organization.
Somewhere i have read that apacheignite is tightly coupled with h2 database .
Can I change h2 database to some other database in apache Ignite.
Ignite does not use H2 to store or index data, it just uses H2's query parser, query planner and serialization, with a lot of customization over either one.
So you can't replace H2 dependency, but if what you really want is storing data in 3rd party database, then you can certainly do that via 3rd Party Persistence.
Related
I'm working on a system which will fetch data from a service and put pieces of the response in to a cache and/or into a SQL table.
The cache is needed for consumption by other Java services directly. These services require a more direct connection than the SQL abstraction, so we need to connect directly to the cache.
The table is needed for a JDBC SQL connection to external SQL clients e.g. SQL Workbench, DBeaver, Tableau, 3rd party systems.
My question is how Ignite works regarding caches vs tables. I know it stores its caches as maps similar to other IMDGs. What I guess I don't understand is how that gets turned into a table, or what APIs are available to set/get between the two.
So the question is, how can I take an INSERT from the JDBC/SQL side and query it via the Cache? How can I add() into the Cache and SELECT it from the JDBC/SQL side? If I have a table named "foo", does that also create a cache named "foo"?
Or am I supposed to use one or the other and not bleed between the two? I haven't found many good examples of this, so it seems to be either you use caches or you use tables.
It would be extremely advantageous to have a bridge between the two. We're migrating to Ignite from an H2 implementation where we mushed a Hazelcast cache and H2's SQL together and are hoping Ignite, being built atop H2, has done something similar already.
In particular, I was hoping to use DataStreamers but I'm not finding much in the way of how it relates to the SQL/table side of things.
Ignite cache falls under key-value type of nosql database. You can fire SQL like query from java code to ignite caches as it supports it. For example,
SELECT _KEY, _VAL from "foo".val
Here, foo is your cache name and val is the value part of key-value pair. As this is all NOSQL, relating it to RDBMS SQL is not so much rational, still we can relate all non primary columns in SQL table to the fields of your value object and primary one to the key part.
So, in datastreamer, you can construct collection of key, value objects and stream it. This internally calls nothing but put operation on cache.
To select in SQL fashon, you can fire query like below-
SqlFieldsQuery query = new SqlFieldsQuery(queryString);
FieldsQueryCursor<List<?>> cursor = cache.query(query);
There are multiple ways to do this, SqlFieldsQuery is one of that.
This was answered couple times already, basically you need to refer to Query Entities, Indexed Types or key_type/value_type parameters of CREATE TABLE to make it work. I.e. every entry in cache of correct type will be a row of table and vice versa.
I need advice for design pattern how to use RabbitMQ to select data from Database.
RabbitMQ looks very good solution for inserting and updating data into Database but what about selecting data from DB?
In my case I have REST API module and Database module connected to Maria DB which communicate via queues.
REST API module -> Database module -> Maria DB
But I need to select configuration from database via database module. I can use RPC as a solution but probably there is better way?
Can you advice?
In general, some sort of RPC is the way to go.
However: The point of a queue (asynchronous tasks) is the opposite of a database select (return my data now). If the direct database select requests are performing adequately, use them, avoid the extra complexity. Or some caching system for your config. This might not work for your system architecture and load needs, but is simpler.
I have a database connected with website, data from website is inserting in that Database, i need to transfer data from that database to another Primary Database (SQL) on another server in real time (minimum latency).
I can not use transactional replication in this case. What are the other alternates to achieve this? Can i integrate DataStreams like Apache kafka etc with SQL server?
Without more detail it's hard to give a full answer. There's what's technically possible, and there's architecturally what actually makes sense :)
Yes you can stream from RDBMS to Kafka, and from Kafka to RDBMS. You can use the Kafka Connect JDBC source and sink. There are also CDC tools (e.g. Attunity, GoldenGate, etc) that support integration with MS SQL and other RDBMS)
BUT…it depends why you want the data in the second database. Do you need an exact replica of the first? If so DB-DB replication may be a better option. Kafka's a great option if you want to process the data elsewhere and/or persist it in another store. But if you just want MS SQL-MS SQL…Kafka itself may be overkill.
I am new to Magnolia CMS and the Apache Jackrabbit content repository concepts.
There is a web application which is using Magnolia CMS. Magnolia is using SQL SERVER 2012 database as persistence manager.
Here Apache Jackrabbit content repository implementation is done. There are two separate configurations of the Magnolia CMS which are used for the application, referred to as the public and author instances.
Now here we are trying to replace the existing Magnolia CMS with a custom ASP.NET MVC 5 application with all the functionalities.
I analysed the tables in the SQL SERVER database and found that data stored in format of Node_ID and Bundle_Data which is very difficult to analyse.
In short, it is not easy to interpret.
Based on the custom CMS a new database model for author instance (SQL SERVER 2012) is developed.
Hence as part of migration task ,I am trying to migrate the old data that is stored in the SQL SERVER with the Apache Jackrabbit content repository implementation to a normal SQL SERVER 2012 (as per the new database model).
Can anyone help me to know are there are any proven methods or tools available to accomplish this task.
The question is more on the jackrabbit-side, not so much on the Magnolia side, especially since you want to replace Magnolia entirely, not just the persistence layer:
Now here we are trying to replace the existing Magnolia CMS with a
custom ASP.NET MVC 5 application with all the functionalities.
although my question really is whether you really want to replace Jackrabbit entirely, or still use Jackrabbit with your ASP.NET application but with a MS SQL Server datastore (which would be my personal suggestion)? Otherwise you will be getting rid of all the benefits that Jackrabbit has.
Jackrabbit does support SQL Server and I would suggest to use it.
https://wiki.apache.org/jackrabbit/DataStore#Configuration-1:
Currently supported are: db2, derby, h2, mssql, mysql, oracle,
sqlserver.
Developing a WebCMS with just ASP.NET and SQL Server and without a content repository layer in between sounds like developing everything that a WebCMS usually comes with from scratch, especially if you want to have all the functionality that Magnolia offers (versioning, history, search, etc.).
You can check details regarding Jackrabbit data store here: http://wiki.apache.org/jackrabbit/DataStore although I am wondering why you or your customer would want to change the data store of the content repository to SQL Server. I guess you are not speaking of using MySQL for the persistence of the meta data, but really to store the binary content (a mistake that by the way OpenCms, another Java-based open source WebCMS, made in their architecture design - imho).
Note that usually large files are not stored in the database itself (with Magnolia), but on the file system.
https://wiki.magnolia-cms.com/display/WIKI/Setting+up+a+Jackrabbit+persistence+manager#SettingupaJackrabbitpersistencemanager-Datastorageandbackup:
BLOBs are not by default stored in the database when they exceed a
certain threshold definied in your Jackrabbit configuration - instead
they are saved on the file system. The default threshold used by a
Magnolia installation is 1024 bytes. All files above the defined
threshold are put onto the filesystem and not in the database.
In case you really want to get rid of Jackrabbit entirely and only use SQL Server as the persistence layer and store all binary content in it regardless of size (not recommended), I would write a custom export/import script for it, which queries the Jackrabbit repo (standard CMIS protocol) and takes the content from the file system, reading as FileInputStream and writing it to the Oracle DB (Example: http://www.java2s.com/Code/Java/Database-SQL-JDBC/StoreBLOBsdataintodatabase.htm). This would be my suggested method.
I don't think there are any out-of-the-box tools for that.
My database must be updated at arbitrary intervals, manually, with new info to put on standart tables that don't require structural modifications. one app will update the database.
Another app, the one I will distribute (to friends and family only, but doesn't require any security feature to it, like encrypting the database) will read the database and use its data on labels, listviews, etc.
The problem is, I'm the perfect definition of full-fledged n00b at programming of any sort, and still don't know what database to use.
I thought that I should use a SQL CE (*.sdf) file, and store that database thing on an FTP. then, I could download it and get data from it everytime the "client" app runs, and certain button ("connect") is clicked.
After some hard-core googling, I found out how to connect to the sdf thing, using this connection string:
Provider=Microsoft.SQLSERVER.CE.OLEDB.3.5;Data Source=D:\Documents and Settings\Camilo\JCTM.sdf
So it connects, or at least didn't show any error.
I don't know if it's a good idea to use sdf SQL CE files as databases, if it's too hard maybe I should go for XML? what do you guys suggest, what is the easiest way to implement very simple databases in VB.NET?
By simple databases I mean:
- no search needed
- no advanced features except storing strings on tables with columns and rows
- easy to access, read, edit, etc. by different VB.NET apps
Is sdf a good idea?
I would recommend Sql Server Express Its free and can be redistributed with .net applications as part of the install process.
The challenge will be syncing the changes between the different clients. If you have access to a FTP server, you may have the ability to host a website in IIS. If you can do that you can just use webservices and read against one database instead of copying one local.
Luckily for you, you can abstract away the need to be concerned with which back-end database you use to store your data.
Technologies such as ODBC (Open Database Connectivity) and OLEDB (Object Linking and Embedding, Database) allow you to limit your concern for the backend datastore to the task of crafting your connection string. In your example, the connection string says, "I'm going to connect to a SQL Server CE database via its OLEDB provider, and it's physically located over on D:/...
Within the code, you use standard OLEDB mechanisms to access and manage the database. Because of this abstraction, you can use OLEDB providers for SQL Server, Oracle, XML, Access or comma delimited text files as your backing store if you wish, and the only change you need to make to your code is the connection string. Your choice then should be to pick the database that you have the tools and know-how to set up and manage initially.
I'd start with Microsoft Access because it has its own UI, and can play well with .NET.
You can also try the ADO.Net implementation for SQLite, which I've also found very useful.