I'm trying to execute the following changeSet in liquibase which should create an index. If the index doesn't exist, it should silently fail:
<changeSet failOnError="false" author="sys" id="1">
<createIndex unique="true" indexName="key1" tableName="Table1">
<column name="name" />
</createIndex>
</changeSet>
So far, so good. The Problem is, that this changeSet doesn't get logged into DATABASECHANGELOG table and is therefor executed every time liquibase runs. According to the liquibase documentation and e.g. this answer from Nathen Voxland i thought that the changeset should be marked as ran in the DATABASECHANGELOG table. Instead it isn't logged at all and as i said before executed every time liquibase runs (and fails everytime again).
Am i missing something?
(I'm using MySQL as DBMS)
In the answer given by Nathen Voxland, he recommended the more correct approach of using a precondition to check the state of the database, before running the changeset.
It seems to me that ignoring a failure is a bad idea.... Means you don't fully control the database configuration.... The "failOnError" parameter allows liquibase to continue. Wouldn't it be a bad idea for a build to record a changset as executed, if in fact it didn't because an error occurred?
Related
Liquibase does a pretty good job in keeping the applied changesets consistent with their source-folder.
If you modify a changeset that has already been applied to the db, Liquibase refuses to do anything, even operations that are not related with the modified changeset.
The rule enforced here is: anything that has been applied to the database must be unapplied before changing it (this is a usual workflow during development).
The problem of 'orphaned' changesets
Unfortunately this rule doesn't apply if you just delete the changeset completely.
In this case there will be what I call an 'orphaned' changeset, that is a record on the DATABASECHANGELOG table (and the database object, of course) with liquibase not complaining at all of a missing changeset in the source.
I expected an error at least when you 'bump' into the missing changeset, that is when you try to rollback it, but liquibase simply seems to ignore its presence, it skips it and rolls back the next one. This can be a problem.
The question is: can we change this liquibase behavior via settings? Is this design needed for some use case that I haven't thought of?
I think what I'm asking should be clear enough, however here is an example as demonstration.
Example
<databaseChangeLog ... >
<include relativeToChangelogFile="true" file="CS1.sql"/>
<include relativeToChangelogFile="true" file="CS2.sql"/>
</databaseChangeLog>
My changesets are SQL based files like this:
-- liquibase formatted sql
-- changeset agostinox:-1
CREATE TABLE T1 (
X INT
)
-- rollback DROP TABLE T1
And CS2.sql is the same for the T2 table.
Now i can apply my changesets like this:
PS > liquibase update
...
Liquibase Version: 4.19.0
Liquibase Open Source 4.19.0 by Liquibase
...
Running Changeset: CS1.sql::-1::agostinox
Running Changeset: CS2.sql::-1::agostinox
Liquibase command 'update' was executed successfully.
And on my db, the DATABASECHANGELOG has the following content:
ID
AUTHOR
FILENAME
DATEEXECUTED
ORDEREXECUTED
EXECTYPE
MD5SUM
-1
agostinox
CS1.sql
2023-01-18 18:52:08.476689
1
EXECUTED
8:d966f9ba2b90eaea9b917a6d93962eff
-1
agostinox
CS2.sql
2023-01-18 18:52:08.666667
2
EXECUTED
8:7f2a735fa83b196a0c72885c95362b81
So far so good. Now, now I get to the point.
Let's mess with the CS1.sql, by adding a comment:
-- liquibase formatted sql
-- changeset agostinox:-1
CREATE TABLE T1 (
X INT --Added comment, very harmless but enough to annoy liquibase :-)
)
-- rollback DROP TABLE T1
Now, I try to rollback the last changeset.
PS > liquibase rollbackcount 1
...
Unexpected error running Liquibase: Validation Failed:
1 changesets check sum
CS1.sql::-1::agostinox was: 8:d966f9ba2b90eaea9b917a6d93962eff but is now:
8:2cea5484e81eb542fa94bb67ba2ffdf5
For more information, please use the --log-level flag
You can see that liquibase complains about CS1.sql been changed even if we are not even rolling back CS1.sql but CS2.sql. Actually this change blocks any further operation, so it have to be reverted in order to break the deadlock.
However, if you just remove the changeset, liquibase doesn't complain anymore:
<databaseChangeLog ... >
<!--REMOVED <include relativeToChangelogFile="true" file="CS1.sql"/> -->
<include relativeToChangelogFile="true" file="CS2.sql"/>
</databaseChangeLog>
PS > liquibase rollbackcount 1
...
Rolling Back Changeset: CS2.sql::-1::agostinox
Liquibase command 'rollbackCount' was executed successfully.
But even if liquibase says that everything is successfull, it is not really the case, infact the table DATABASECHANGELOG now contains the 'orphaned' changeset (and the database contains the T1 table since also the rollback code is gone).
ID
AUTHOR
FILENAME
DATEEXECUTED
ORDEREXECUTED
EXECTYPE
MD5SUM
-1
agostinox
CS1.sql
2023-01-18 18:52:08.476689
1
EXECUTED
8:d966f9ba2b90eaea9b917a6d93962eff
For db people, it seems like liquibase does a left join between changesets source folder and records in DATABASECHANGELOG (that is: all the items on the left set are taken and are associated with the matching items on the right set); this way liquibase can see applied migrations and check if their checksum matches. It can also see not-yet-applied migrations (changesets in source folder without a DATABASECHANGELOG record associated) in order to apply them in the next 'update' call.
But it can't see records from DATABASECHANGELOG (the right set) that don't have an associated changeset in the source folder.
This parallel with join operation well explains the liquibase behavior, so let's use it for describing what a better behavior might be.
Liquibase should do a full-join, that is: also the items on the right side that don't have a source changeset associated are considered. Those records indicate an anomaly that is possibly just a 'major version' of the one that you have when a checksum doesn't match. Indeed you can easily think of a null file as a particolar case of checksum, the checksum of null being something necessarily different from the one in the table. Therefore the existence of those non associated record should give the same kind of error. This is what i would expected to truly ensure consistence between source an DATABASECHANGELOG.
I've been curious about this also. I'd recommend opening a github issue so a Liquibase employee can address why they don't do this check. I'm guessing this was done on purpose.
https://github.com/liquibase/liquibase/issues
I want to update very old changeset because there is a bug in that. I have updated that changesetid and After running it I got checksum error i.e checksum got modified. Now I have added validCheckSum Tag with this new checkSum. After it is getting successfully I cant not see new changes. Is there any way to update that change set which has already been executed.
Each changeSet tag is uniquely identified by the combination of the “id” tag, the “author” tag so please check whether id and author name already exists or not in DATABASECHANGELOG table. If the combination exists then change the id in your xml file then run the build
If still not works then add runOnChange property in <ChangeSet> and try it
runOnChange - Executes if any changes found it already existing changeset
Example of using runOnChange in changeset with sqlFile tag .
I had a script for Oracle 18 using IDENTITY column type and than have to change it for Oracle 10 which doesn't support IDENTITY column type
<changeSet id="2" author="auto" runOnChange="true">
<sqlFile dbms="oracle" encoding="utf8" endDelimiter=";" path="oracle/230.sql" relativeToChangelogFile="true" splitStatements="true" stripComments="true" />
</changeSet>
If it is a very old changeset as you have stated you should consider not changing the old changeset but add a new changeset instead which fixes the bug since it allows you better control over the versions you have. This is one of the main reasons of using liquibase.
I have a changeset to create an index.
<changeSet author="hilland" id="x-NC-U-y" runOnChange="true">
<createIndex indexName="NC-U-y" tableName="x" unique="true">
<column name="y"/>
</createIndex>
<rollback>
<dropIndex .../>
</rollback>
<modifySql>
<append value=" INCLUDE ( [a],[b]) WITH (DATA_COMPRESSION=page)"/>
</modifySql>
</changeSet>
The problem is that an older version of the index might exist (it will on some targets (dev server, with an older version of this index; no includes), will not on others (eg a fresh deploy to an empty database).
Is there a better way to resolve this so that the script becomes universal, than to include a prefix which says
<sql>IF select {sys.indexes.stuff} is not null {drop the index}</sql>?
Ideally there would be a precondition to drop-if-exists the existing index so that it can be recreated, but i don't think that can be done.
What is the best way to handle this situation?
an alternate solution i thought of would be to have a separate changest "if index v1 exists, drop it" then have a separate v2 changeset, but then that would make rollback hard.
I realize that the run on change condition for the changeset makes rollback hard, but the only rollback i'm concerned with in this case is to and from an empty database, although a pattern which easily accommodates rollbacks to previous version would be welcome also.
You can execute changesets dependent on a precondition. Your changeset could look like this:
<changeSet>
<preConditions onFail="MARK_RAN">
<indexExists indexName="NC-U-y" />
</preConditions>
<dropIndex indexName="NC-U-y" />
</changeSet>
See also the Liquibase Documentation.
I have been struggling with a liquibase challenge for some time and I hope somebody here can help me:
I would like to execute a simple parameterized liquibase script multiple times on the same db schema with different parameters:
<changeSet id="1" author="me" dbms="Oracle" runOnChange="false" failOnError="true">
<sql splitStatements="true">
GRANT SELECT on SOME_VIEW to ${db_user};
</sql>
</changeSet>
Now I execute liquibase one time with -Ddb_user=first_user and than with -Ddb_user=second_user. The second run fails, because liquibase calculates the checkSum after replacing the ${db_user} parameter (what makes perfect sense) and therefore the combination of id/author/filename and checkSum is already present in the DATABASECHANGELOG table.
Is there a best practice way to solve this problem?
Thanks in advance.
There is a runOnChange as an attribute for the changeSet which will run your changeset each time it was changed. Maybe this does what you are looking for?
You can always use runOnChange=true or <validCheckSum>any</validCheckSum>
There is an issue that explains this decision here: https://liquibase.jira.com/browse/CORE-2506
I want to start with a specific example where a table is created with a column having a constraint indicating this column is a foreign key:
<changeSet id="1" author="A1">
<createTable tableName="TABLE_A">
<!-- Other columns -->
<column name="FK_TABLE_B" type="BIGINT">
<!-- the 'TABLE_B' will be renamed soon ! -->
<constraints references="TABLE_B(ID_PK)"
foreignKeyName="JUST_UNIQUE_I_GUESS" nullable="false" />
</column>
</createTable>
</changeSet>
Now a first question: Assume TABLE_B would be renamed to TABLE_NEW... how would i want to change the above constraint so it points to the renamed Tables column ID_PK?
I see a few possibilities:
modify the changeSet itself (no Problem with a H2 inMemory DB... but cmon... thats not the idear right?)
Drop the whole column in my own changeSet and add it again with the new constraint which could make sence since the column name will probably change to FK_TABLE_NEW anyways... but thats not a real possibilitie for a productive environment
somehow alter that constraint -> BUT HOW!? the documentation is no
help at all...
The next question then is why to have this way of constraints if there is a
<addForeignKeyConstraint ...>
as well?
So currently i dont feel Liquibase at all just because this was my very first try to change an existing changeSet and my biggest question is: Is this a issue of not understanding the best practise or acctually a problem of the "not so verbouse" documentation?
Thanks for any help!
What i tried
assumed above constraints definition somehow leads to the same as a when done with a
<addForeignKeyConstraint ...>
tag but with less attributes. So i thought i could just use the
<dropForeignKeyConstraint ...>
tag first delete the ForeignKeyConstraint and then add a new one. But it still tells me it wont find table TABLE_B when trying to execute the first changeSet in a H2 in-memory DB.
My Changeset looked like this:
<changeSet id="1" author="A2">
<dropForeignKeyConstraint baseTableName="TABLE_A"
constraintName="JUST_UNIQUE_I_GUESS" />
<addForeignKeyConstraint
constraintName="JUST_UNIQUE_I_GUESS"
referencedTableName="TABLE_NEW" baseColumnNames="FK_TABLE_B"
baseTableName="TABLE_A" referencedColumnNames="ID_PK" />
</changeSet>
Background Information
Since were currently just building a POC using a in-memory DB only (H2) its no big deal to just change the first changeSet until we have quite of a final scheme-state... but how to deal with such things if you already have a existing DB, millions of entrys and stuff? Currently i highly doubt Liquibase is the right decision for a company with 1k+ developpers without hiring Liquibase Experts...
You should only modify your changesets as long as they (resp. the software/database that go along with the changesets) have not been released.
We keep our changeset files (along with the code) in our source code repository. During development phase everyone is allowed to change changesets defined for the version currently under development.
As soon as the version is released the changeset files are considered to be fixed and should not be changed anymore. (With the release the software is shipped and customers then have a database that reflects whatever the changesets define).
So after release you will have to create new changeSets that
drop the foreign key constraint
rename your table
add a new foreign key constraint
More or less, you need to do exactly what you would do with pure sql on the database as well.
At this point liquibase will more or less just translate your changeSets to sql and apply them to the database. So once you found a way to do your changes with slq it should also be possible to put those into liquibase changesets.