I am using CloverETL Designer Version: 4.0.0.030M2 for one of my projects. I want to read from a MySQL database and do some comparison and then write to the database. But I cant find a MySQL reader in the tool. Whereas a mongodb reader is present as well as a MySQL writer. Please help me on how to read from MySQL database in a clover graph. Thanks.
You can use DbInputTable (http://doc.cloveretl.com/documentation/UserGuide/index.jsp?topic=/com.cloveretl.gui.docs/docs/dbinputtable.html) component which is generic reader for all JDBC-enabled databases. For writing to and JDBC database you can use DbOutputTable.
Writers, like MySQLDataWriter are used for fast bulk data writing using database-specific infrastructure.
Btw. best place for asking CloverETL related questions is http://forum.cloveretl.com/
Related
Does Mondrian support nosql db like mongodb in the current version. I read some blogs and bugs related to the same.
Any help is appreciated.
thanks
Lokesh
please read the following Blog from Julian Hyde, creator of Mondrian
http://julianhyde.blogspot.com.es/2014/03/improvements-to-optiqs-mongodb-adapter.html
Here you can see Julian have been working on a new approach that converts even complex SQL queries into MongoDB Queries behind the scenes.
Mondrian does not directly support MongoDB at the moment. MongoDB does not have a JDBC implementation.
There are a few options. One of them can be setup if you have access to a Pentaho Data Integration server. You can use a thin JDBC implementation which will allow Mondrian to access a SQL to Mongo bridge.
There are certainly other ways to set this up, since there are a lot of data federation engines out there.
Not directly as far as I know. Maybe someone is working on a dialect, is that even possible..? Interesting question though... May be worth linking the blogs you found so far?
One solution however could be to use the kettle jdbc driver, this driver works with mondrian and then the other end can be any ETL process. So you could use a mongodb input step etc.
There is Apache Drill. You can query MongoDB through standard SQL and Drill has a JDBC driver. So maybe it is possible that Mondrian uses this driver.
Uwe
I have a database in a format which can be accessed via ODBC. I'm looking for a command-line tool to generate SQL file with DROP/CREATE statements from it, preferably with all the information including table/field comments and table relations. (Possibly for a tool to parse the file and import the schema too, but I guess this would be relatively easier to find). Need this to automate workflow, to be able to design the database visually but store it in SVN in code form.
Which tool should I use?
If this helps, the database in question is MS Access, but I guess there's a higher chance of finding a generic ODBC tool...
Okay, I wrote the tool to export access schema/parse SQL files myself, it's available here:
https://bitbucket.org/himselfv/jet-tool
Feel free to use if anyone needs it.
Adding this because I wanted to search an ODBC schema, and came across this post. This tool lets you dump a csv format of the schema itself:
http://sagedataobjects.blogspot.co.uk/2008/05/exploring-sage-data-schema.html
And then you can grep away..
This script may work for you with some modifications. Access (the application) is required though.
I'm currently designing a DB scheme. I have to use hsqldb.
- Normally I like creating an ER diagram, and I export the SQL source. Therefore I'm looking for a tool that automatically transforms my ER diagram into hsqldb sql code. Using MySQL Workbench or some sort of web interface (http://ondras.zarovi.cz/sql/demo/) seems to support any server, but hsqldb.
I'm beginning to do that. Surely I can export SQL code from any tool and edit it to fit. However I think this is one step I can save... with the right diagram tool :-)
Have a look at SQL PowerArchitect, it supports HSQLDB:
http://www.sqlpower.ca/page/architect
Their homepage requires a registration in order to download it (even for the community edition), but you can download the community edition without registration from the page on google code:
http://code.google.com/p/power-architect/
DB Visualizer can probably do that directly. The HSQLDB SQL dialect is ANSI SQL. Other SQL dialects that are close to ANSI are Firebird, DB2, Sybase, Informix and Postgres.
I want to start using Core Date on iPhone with pre-existing MySQL databases. What's the easiest way to transfer a MySQL database to SQLite?
I've tried using SQLite Migrator, but I don't know where to find the ODBC drivers for Mac (Snow Leopard). I found http://www.ch-werner.de/sqliteodbc/ which seems to have drivers, but they are for Power PC.
If someone could give me a walkthrough, or tell me what the best tools for this are, I'd be grateful.
Thanks.
Perhaps the simplest would be to use mysqldump to dump the raw SQL from your MySQL database into a text file and then use the sqlite3_exec() function to execute that SQL in order to populate the SQLite database.
Have you looked at this Perl script? I haven't used it - just did a quick search for mysql to sqlite migration and it popped right up.
Edit (after you replied to my comment):
The reverse direction is dealt with here.
If you are going to do it repeatedly and if data structure changes are to happen, maybe you would be better off using something like Django (albeit in a very hackish way). With it I would:
# This three lines are done once
django-admin.py startproject mymigrationproject
cd mymigrationproject
./manage.py startapp migration
# The following lines you repeat each time you want to migrate the data
edit settings.py and make the changes to connect to MySQL
./manage.py inspectdb > ./migration/models.py
edit ./migration/models.py to reorder tables (tables in which other tables depend on top)
mkdir fixtures
./manage.py dumpdata migration > ./fixtures/data.json
edit settings.py and make the changes to connect to SQLite
./manage.py syncdb
./manage.py loaddata ./fixtures.data.json
Here is a list of converters:
http://www.sqlite.org/cvstrac/wiki?p=ConverterTools
An alternative method that would work nicely but is rarely mentioned is: use a ORM class that abstracts the specific database differences away for you. e.g. you get these in PHP (RedBean), Python (Django's ORM layer, Storm, SqlAlchemy), Ruby on Rails ( ActiveRecord), Cocoa (CoreData)
i.e. you could do this:
Load data from source database using the ORM class.
Store data in memory or serialize to disk.
Store data into source database using the ORM class.
You can use a trial from http://www.sqlmaestro.com/products/sqlite/datawizard/
It is completely functional for 30 days.
You can get ODBC drivers for Mac OS X from Actual Technologies.
http://www.actualtech.com/
To connect to MySQL you need their ODBC Driver for Open Source Databases:
http://www.actualtech.com/product_opensourcedatabases.php
(Disclaimer: I am the author of SQLite Migrator)
There is a free ETL product that can be used to migrate data from one db to another. Have a look: http://www.talend.com/index.php
Good luck!
To do my conversions, I ended up using an ODBC from Actual Access. I think I used it in combination with SQLite Migrator. I never liked this way though it was always clunky. Expensive too, it ended up costing about $80 for those two pieces of software.
If I had to do this again, I'd buy SQLiteConverter by SQLabs. I use their SQLite Manager, and although it has a lot of interface problems, for database software it's not bad.
http://www.sqlabs.net/sqliteconverter.php
Does anyone know tools which look at a MySQL database, show you all the tables graphically and allow you to create complicated JOIN statements via drag-and-drop?
Before you buy anything, see if the free, official MySQL's GUI tools (specifically the MySQL Query Browser) will work for you.
Personally, I'm fairly comfortable interacting with MySQL's command line interface and haven't used their GUI tools very much, but I just downloaded Query Browser and it seems like it does exactly what you're looking for.
Also, check out "Building Queries Visually in MySQL Query Browser" for a nice tour of MySQL Query Browser.
As an update, the MySQL Tools collection is no longer supported, and has been replaced by MySQL Workbench. The documentation can be found here: http://dev.mysql.com/doc/workbench/en/index.html and you can download it here: http://dev.mysql.com/downloads/workbench/
Edit: stumbled across this today too, a good beginner tutorial for mysql workbench -> http://net.tutsplus.com/tutorials/databases/visual-database-creation-with-mysql-workbench/
EMS SQL Manager for MySQL has query constructor. Can't recall about joins, but they should be supported.