Recently, I was having a discussion with my friend over the features of SparkSQL when we came across this question. Are they ACID transactions? Does SparkSQL follow CAP theorem?
I am a little new to this field, help me out. Thanks in advance.
SparkSQL is a query language and not a storage like Hive or MYSQL. Although it can register table which can be used by others, its only temporary. SparkSQL supports what the underlying databases support.
SparkSQL follows the Relational database model.
It does not support Hive transactions ("ACID").
Following are few useful posts:
http://www.cloudera.com/documentation/enterprise/release-notes/topics/cdh_rn_spark_ki.html
http://db-engines.com/en/system/Oracle+NoSQL%3BSAP+IQ%3BSpark+SQL
Related
I come from a Teradata and Netezza background in Data Warehousing in MPP technologies.
I would like to ask how Google BigQuery distributes data by partition key on a simple table? I am really trying to understand the logic in how the BigQuery engine works if this makes sense?
Teradata and Netezza had a well documented technical page from recollection which described the processes used (like a step by step walkthrough).
Thanks,
Simon
BigQuery's partitioned tables are also very well documented here:
https://cloud.google.com/bigquery/docs/partitioned-tables
I think I don't understand what you want to know. Please rephrase your question after reading all the above.
Does Mondrian support nosql db like mongodb in the current version. I read some blogs and bugs related to the same.
Any help is appreciated.
thanks
Lokesh
please read the following Blog from Julian Hyde, creator of Mondrian
http://julianhyde.blogspot.com.es/2014/03/improvements-to-optiqs-mongodb-adapter.html
Here you can see Julian have been working on a new approach that converts even complex SQL queries into MongoDB Queries behind the scenes.
Mondrian does not directly support MongoDB at the moment. MongoDB does not have a JDBC implementation.
There are a few options. One of them can be setup if you have access to a Pentaho Data Integration server. You can use a thin JDBC implementation which will allow Mondrian to access a SQL to Mongo bridge.
There are certainly other ways to set this up, since there are a lot of data federation engines out there.
Not directly as far as I know. Maybe someone is working on a dialect, is that even possible..? Interesting question though... May be worth linking the blogs you found so far?
One solution however could be to use the kettle jdbc driver, this driver works with mondrian and then the other end can be any ETL process. So you could use a mongodb input step etc.
There is Apache Drill. You can query MongoDB through standard SQL and Drill has a JDBC driver. So maybe it is possible that Mondrian uses this driver.
Uwe
After seeing this image:
http://2.bp.blogspot.com/_T-uXeKcGTnM/TIdoKBGwk9I/AAAAAAAABcs/CLW3_cRlN78/s1600/tumblr_kxovt0VLZy1qappj8.png
I wonder is exists any tool for translating SQL querys into MongoDB map/reduce query model??
Larger version of the image: http://rickosborne.org/download/SQL-to-MongoDB.pdf
Update to the question asked in Jan 2011:
A couple of sites exist now to convert sql to mongodb.
Convert MySQL Queries to MongoDB Syntax
http://www.querymongo.com/
And
Convert sql to mongodb
http://klaus.dk/sqltomongodb/
The simple anwser? No.
The slightly more complex anwser is some people have had luck translating more complex SQL to Mapreduce functions ...
http://rickosborne.org/blog/index.php/2010/02/08/playing-around-with-mongodb-and-mapreduce-functions/
http://rickosborne.org/blog/index.php/2010/02/19/yes-virginia-thats-automated-sql-to-mongodb-mapreduce/
However, that said ... generally speaking you might as well learn mapreduce properly because if the data is in MongoDB already ... you'll really need to know how to properly query MongoDB to get anything meaningful done!
MongoDB has wonderful and helpful docs http://www.mongodb.org/display/DOCS/Advanced+Queries
As well as an easy to use online tutorial: http://try.mongodb.org/
The simple answer: Yes. Hibernate OGM - JPA for NoSQL.
JPA is Java API for mapping objects to data stores.
It includes JPQL, a query language similar to SQL which adds the OOP concepts. It's not SQL, but you don't want pure SQL - that was designed for the relational paradigm.
Hibernate OGM proposes to simplify the programming model by embracing JPA/Hibernate APIs and semantics to store data in NoSQL stores like JBoss Enterprise Data Grid instead of the traditional RDBMS. (source)
Also see this Hibernate OGM: JPA for NoSQL talk by Hardy Ferentschik
Recently I happened to see this website mongoquery.com, you can try it.
You can use free sql to mongodb converter like: https://rapidapi.com/ariefsam/api/easy-sql-to-mongodb-aggregation/
Just to add to the last comment
re:The simple answer: Yes. Hibernate OGM - JPA for NoSQL.
JPA is Java API for mapping objects to data stores.
It includes JPQL, a query language similar to SQL which adds the OOP concepts. It's not SQL, but you don't want pure SQL - that was designed for the relational paradigm.
There is a company called UnityJDBC that has released a JDBC driver for Mongo that allows you to run SQL queries against mongo in any java application that supports JDBC.
you can download this driver free at
http://www.unityjdbc.com/mongojdbc/mongo_jdbc.php
hope this helps
You can also http://teiid.org which gives full range of SQL based access to MongoDB. You can use SQL through JDBC/ODBC or use REST/ODATA based access to MongoDB. Teiid uses MongoDB's aggregation framework to provide advanced SQL MongoDB query conversation.
I am looking into replicating in real time data from Oracle to Vertica database.
So far i can not find anything that is able to do this !!!
But i have foung Tungsten replicator that seems to work well in Mysql to Vertica (havent test it yet).
My Question is :
Is there any tools or ways of doing this (oracle => vertica)???
And if so how would updates would be handled ?
WisdomForce - acquired by Informatica - does data replication between oracle and vertica. http://www.wisdomforce.com/.
Look at Tungsten Replicator. It replicates from MySQL/Oracle to Vertica and is fully open source (GPL V2). You can find the Tungsten Replicator project at code.google.com. (Disclaimer: I work for Continuent and wrote a good chunk of the replicator code.)
After seeing this image:
http://2.bp.blogspot.com/_T-uXeKcGTnM/TIdoKBGwk9I/AAAAAAAABcs/CLW3_cRlN78/s1600/tumblr_kxovt0VLZy1qappj8.png
I wonder is exists any tool for translating SQL querys into MongoDB map/reduce query model??
Larger version of the image: http://rickosborne.org/download/SQL-to-MongoDB.pdf
Update to the question asked in Jan 2011:
A couple of sites exist now to convert sql to mongodb.
Convert MySQL Queries to MongoDB Syntax
http://www.querymongo.com/
And
Convert sql to mongodb
http://klaus.dk/sqltomongodb/
The simple anwser? No.
The slightly more complex anwser is some people have had luck translating more complex SQL to Mapreduce functions ...
http://rickosborne.org/blog/index.php/2010/02/08/playing-around-with-mongodb-and-mapreduce-functions/
http://rickosborne.org/blog/index.php/2010/02/19/yes-virginia-thats-automated-sql-to-mongodb-mapreduce/
However, that said ... generally speaking you might as well learn mapreduce properly because if the data is in MongoDB already ... you'll really need to know how to properly query MongoDB to get anything meaningful done!
MongoDB has wonderful and helpful docs http://www.mongodb.org/display/DOCS/Advanced+Queries
As well as an easy to use online tutorial: http://try.mongodb.org/
The simple answer: Yes. Hibernate OGM - JPA for NoSQL.
JPA is Java API for mapping objects to data stores.
It includes JPQL, a query language similar to SQL which adds the OOP concepts. It's not SQL, but you don't want pure SQL - that was designed for the relational paradigm.
Hibernate OGM proposes to simplify the programming model by embracing JPA/Hibernate APIs and semantics to store data in NoSQL stores like JBoss Enterprise Data Grid instead of the traditional RDBMS. (source)
Also see this Hibernate OGM: JPA for NoSQL talk by Hardy Ferentschik
Recently I happened to see this website mongoquery.com, you can try it.
You can use free sql to mongodb converter like: https://rapidapi.com/ariefsam/api/easy-sql-to-mongodb-aggregation/
Just to add to the last comment
re:The simple answer: Yes. Hibernate OGM - JPA for NoSQL.
JPA is Java API for mapping objects to data stores.
It includes JPQL, a query language similar to SQL which adds the OOP concepts. It's not SQL, but you don't want pure SQL - that was designed for the relational paradigm.
There is a company called UnityJDBC that has released a JDBC driver for Mongo that allows you to run SQL queries against mongo in any java application that supports JDBC.
you can download this driver free at
http://www.unityjdbc.com/mongojdbc/mongo_jdbc.php
hope this helps
You can also http://teiid.org which gives full range of SQL based access to MongoDB. You can use SQL through JDBC/ODBC or use REST/ODATA based access to MongoDB. Teiid uses MongoDB's aggregation framework to provide advanced SQL MongoDB query conversation.