I think that XML files are more clear to read, but does it have advantage over SQL format. I found some info [here] (http://forum.liquibase.org/topic/liquibase-sql-vs-xml-command-changesets), but I would nice to hear others voices.
I prefer using xml file because it's more abstract. For example you can define column type by VARCHAR/java.sql.Types.VARCHAR and liquibase will replace that by default configuration for target database - so it is not database specific. But if you have some database specifics (like plsql) then I'm using .sql files and loading them with sqlFile change.
One of the primary reasons that Datical chose Liquibase as the core of our product was that the XML gave us the ability to look at the changelog in a programmatic way, allowing us to do things like forecasting the changes that would be made before they are applied to a database, and the ability to write rules against the changelog. If used with care, XML can give you a database-independent way of managing db changes also.
Related
With the load data option that Liquibase provides, one can specify seed data in a CSV format. Is there a way I can provide say, a JSON or XML file with data that Liquibase would understand?
The use case is we are trying to put in some sample data which is hierarchical. E.g. Category - Subcategory relation which would require putting in parent id for all related categories. If there is a way to avoid including the ids in the seed data via say, JSON.
{
"MainCat1": ["SubCat11", "SubCat12"],
"MainCat2": ["SubCat21", "SubCat22"]
}
Very likely to have this as not supported (couldn't make Google help me) but is there a way to write a plugin or something that does this? Pointer to a guide (if any) would help.
NOTE: This is not about specifying the change log in that format.
This not currently supported and supporting it robustly would be pretty difficult. The main difficultly lies in the fact that Liquibase is designed to be database-platform agnostic, combined with the design goal of being able to generate the SQL required to do an operation without actually doing the operation live.
Inserting data like you want without knowing the keys and just generating SQL that could be run later is going to be very difficult, perhaps even impossible. I would suggest approaching Nathan, who is the main developer for Liquibase, more directly. The best way to do that might be through the JIRA bug database for Liquibase.
If you want to have a crack at implementing it, you could start by looking at the code for the LoadDataChange class (source in Github), which is where the CSV support currently lives.
So we have a set o database changelogs and we want to achieve that whenever developer make a change in *hbm file, he run a diff comparing changelog commited to our codebase against edited hbm schema - so we can get new changelog with his recent modifications.
Liquibase-hibernate allows to compare hbm schema agaisnt any database, but is there a way to compare hbm schema against changelog xml file ?
No, it is not currently possible. Comparing a database (or hibernate mapping) to a changelog is a is a popular request, but it hasn't been implemented yet. The biggest problem is that to solve it for the general case we need an SQL parser in order to handle and blocks.
It can be done for a subset of all change logs where just standard etc. tags are used, but it has not been implemented yet, unfortunately.
In SQL:2008, and also previous standards, the INFORMATION_SCHEMA is described as the standard meta-schema. In principle, meta-data could be unloaded into XML for further processing and reverse-engineering of schema meta data with XSLT and other XML tools.
Has this been done before?
Is there a somewhat complete XSD available, that describes the INFORMATION_SCHEMA?
N.B: I'm asking this because I would like to implement unloading of a database schema into a SQL standard INFORMATION_SCHEMA XML structure in jooq-meta, and then in a second process to load that schema again, to generate Java source code artefacts in jOOQ. For that, I would prefer not to roll my own XSD, but use a pre-existing as-close-to-the-standard-as-possible XSD
ANSI/ISO have not defined a particular way of representing a database schema using XML.
The latest standard is SQL:2011.
Here you can see the list of official standards. But, as you can see, they're not free:
35.060: Languages used in information technology
Look for ISO/IEC 9075-x in the list. As you can see there's still only the old INFORMATION SCHEMA.
So, your only option is to look for something widely used. Altova has its own way of doing this (a function named "Create XML Schema from DB Structure"). Look at this link:
How to Convert a Database to an XML Schema
This application has also options to create the database from the exported schema.
I think this is the closest to an standard that you can find (Altova is one of the leading XML software companies. If there was some kind of standard they should know it and use it).
Oxygen XML has also its own way of doing this, and it lloks like there is some kind of "IOS draft":
Extract XML Schema From a Database Structure
I'm evaluating liquibase for a project starting today.
Has anybody used it to create procedures, functions, basically all of the plsql stuff?
If not, is it possible to write embedded sql code in the xml files?
There is a built-in createProcedure tag in liquibase for managing procedures. The best approach is usually to combine the or tags with runOnChange so liquibase will update your procedure when and only when you update the definition. That way you can do diffs between your changelog xml files over time and see how the procedure has changed.
Using the sqlFile tag to reference file per stored-proc is also popular, or, like you said, you can use the sql tag to inline custom sql.
I've encountered issues with trying the use the sql tag for stored procedures, triggers, and functions, but in my case these were provably issues with the MySQL JDBC driver, and not Liquibase itself. The practice I've settled into is to use the sqlFile refactoring as Nathan suggests, then to control the SP/trigger/function code in the same project as the changelog, versioned in the source code system along with it. This lets you manage the SP/whatever code just like it was real source code.
Setting runOnChange="true" in the changeSet containing the sqlFile refactoring is essential. It is this switch (thank you, Nathan) that enables real source control of procedural database code.
While I haven't used liquibase for stored procedures, I have some experience of Liquibase for more generic operations.
It is possible to write custom sql, either embedded in the xml file or referenced from an external file.
This is a problem that I come to on occasion and have yet to work out an answer that I'm happy with. I'm looking for a build system that works well for building a database - that is running all of the SQL files in the correct database instance as the correct user and in the correct order, and handling dependencies and the like properly.
I have a system that I hacked together using Gnu Make and it works, but it's not especially flexable and frankly can be a bit of a pain to work with in some situations. I've considered looking at things like SCons and CMake too, but I don't know how much better they are likely to be, or if there's a better system out there that already exists...
Just a shell script that runs all the create statements and imports in the proper order. You may also find migrations (comes with rails) interesting. It provides a make like infrastructure that let's you maintain a database the structure of which evolves over time.
Say you add a new column to some table. In migrations you'd write a snippet of code which describes the requirements for adding the column and also to rollback the change so you can switch to different versions of your schema automatically.
I'm not a big fan of the tight integration with rails, though, but the principles behind it are very interesting.
For SQL Server, I just use a batch file with SQLCMD.EXE and a bunch of .SQL files. It's not perfect, but it seems to work.
For my database, I use Migrator.NET
This is a .NET framework which allows you to create classes in where you define your DDL statements.
The framework comes with a command-line tool with which you can execute your 'migrations' in the correct order.
It also has a msbuild - task, so you can integrate it in a continuous integration build as well.
First export full DDL files describing all tables, views, source code
(procedures, functions, packages), sequences, and grants of a DB schema
See
Is there a tool to generate a full database DDL for SQL Server? What about Postgres and MySQL?
I created a database build system (part SQL-parser, part make file) to put these files together in a DB creation script using python.