When a changeset is marked as failOnError:false, does liquibase record it as having been applied when it fails?
For example, we have a script that performs a pre-emptive drop table in one changeset and then creates the table in the next changeset. When the script is first run, the drop table statement fails as expected and then the table is created successfully. However, the changeset that attempted the drop table is not added to the databasechangelog table.
Is that expected behavior?
That is the current behavior currently. Depending on the reason for the failure, it can make sense to either continue to retry it or to not.
I created https://liquibase.jira.com/browse/CORE-1766 to add the feature to mark it as failed and not try again.
Currently, your best option would be to add a precondition to the dropTable changeSet with onFail="MARK_RAN"
Related
Liquibase does a pretty good job in keeping the applied changesets consistent with their source-folder.
If you modify a changeset that has already been applied to the db, Liquibase refuses to do anything, even operations that are not related with the modified changeset.
The rule enforced here is: anything that has been applied to the database must be unapplied before changing it (this is a usual workflow during development).
The problem of 'orphaned' changesets
Unfortunately this rule doesn't apply if you just delete the changeset completely.
In this case there will be what I call an 'orphaned' changeset, that is a record on the DATABASECHANGELOG table (and the database object, of course) with liquibase not complaining at all of a missing changeset in the source.
I expected an error at least when you 'bump' into the missing changeset, that is when you try to rollback it, but liquibase simply seems to ignore its presence, it skips it and rolls back the next one. This can be a problem.
The question is: can we change this liquibase behavior via settings? Is this design needed for some use case that I haven't thought of?
I think what I'm asking should be clear enough, however here is an example as demonstration.
Example
<databaseChangeLog ... >
<include relativeToChangelogFile="true" file="CS1.sql"/>
<include relativeToChangelogFile="true" file="CS2.sql"/>
</databaseChangeLog>
My changesets are SQL based files like this:
-- liquibase formatted sql
-- changeset agostinox:-1
CREATE TABLE T1 (
X INT
)
-- rollback DROP TABLE T1
And CS2.sql is the same for the T2 table.
Now i can apply my changesets like this:
PS > liquibase update
...
Liquibase Version: 4.19.0
Liquibase Open Source 4.19.0 by Liquibase
...
Running Changeset: CS1.sql::-1::agostinox
Running Changeset: CS2.sql::-1::agostinox
Liquibase command 'update' was executed successfully.
And on my db, the DATABASECHANGELOG has the following content:
ID
AUTHOR
FILENAME
DATEEXECUTED
ORDEREXECUTED
EXECTYPE
MD5SUM
-1
agostinox
CS1.sql
2023-01-18 18:52:08.476689
1
EXECUTED
8:d966f9ba2b90eaea9b917a6d93962eff
-1
agostinox
CS2.sql
2023-01-18 18:52:08.666667
2
EXECUTED
8:7f2a735fa83b196a0c72885c95362b81
So far so good. Now, now I get to the point.
Let's mess with the CS1.sql, by adding a comment:
-- liquibase formatted sql
-- changeset agostinox:-1
CREATE TABLE T1 (
X INT --Added comment, very harmless but enough to annoy liquibase :-)
)
-- rollback DROP TABLE T1
Now, I try to rollback the last changeset.
PS > liquibase rollbackcount 1
...
Unexpected error running Liquibase: Validation Failed:
1 changesets check sum
CS1.sql::-1::agostinox was: 8:d966f9ba2b90eaea9b917a6d93962eff but is now:
8:2cea5484e81eb542fa94bb67ba2ffdf5
For more information, please use the --log-level flag
You can see that liquibase complains about CS1.sql been changed even if we are not even rolling back CS1.sql but CS2.sql. Actually this change blocks any further operation, so it have to be reverted in order to break the deadlock.
However, if you just remove the changeset, liquibase doesn't complain anymore:
<databaseChangeLog ... >
<!--REMOVED <include relativeToChangelogFile="true" file="CS1.sql"/> -->
<include relativeToChangelogFile="true" file="CS2.sql"/>
</databaseChangeLog>
PS > liquibase rollbackcount 1
...
Rolling Back Changeset: CS2.sql::-1::agostinox
Liquibase command 'rollbackCount' was executed successfully.
But even if liquibase says that everything is successfull, it is not really the case, infact the table DATABASECHANGELOG now contains the 'orphaned' changeset (and the database contains the T1 table since also the rollback code is gone).
ID
AUTHOR
FILENAME
DATEEXECUTED
ORDEREXECUTED
EXECTYPE
MD5SUM
-1
agostinox
CS1.sql
2023-01-18 18:52:08.476689
1
EXECUTED
8:d966f9ba2b90eaea9b917a6d93962eff
For db people, it seems like liquibase does a left join between changesets source folder and records in DATABASECHANGELOG (that is: all the items on the left set are taken and are associated with the matching items on the right set); this way liquibase can see applied migrations and check if their checksum matches. It can also see not-yet-applied migrations (changesets in source folder without a DATABASECHANGELOG record associated) in order to apply them in the next 'update' call.
But it can't see records from DATABASECHANGELOG (the right set) that don't have an associated changeset in the source folder.
This parallel with join operation well explains the liquibase behavior, so let's use it for describing what a better behavior might be.
Liquibase should do a full-join, that is: also the items on the right side that don't have a source changeset associated are considered. Those records indicate an anomaly that is possibly just a 'major version' of the one that you have when a checksum doesn't match. Indeed you can easily think of a null file as a particolar case of checksum, the checksum of null being something necessarily different from the one in the table. Therefore the existence of those non associated record should give the same kind of error. This is what i would expected to truly ensure consistence between source an DATABASECHANGELOG.
I've been curious about this also. I'd recommend opening a github issue so a Liquibase employee can address why they don't do this check. I'm guessing this was done on purpose.
https://github.com/liquibase/liquibase/issues
I have introduced a changeset <dropTable table_name="foo"> that will get rid of the no longer needed table foo.
Since dropTable has no auto-rollback I can of course specify the rollback actions manually (by copying the contents of the changeset that originally created foo). It would however be more convenient and less error-prone to just re-apply the said changeset - is this possible?
Update: it is possible, the answer is hidden in the Liquibase Auto Rollback subpage of using-rollback which does not only list change types and their auto-rollback capability but also has an example of referring to an older changeSetId.
However, this feature seems not to be the most powerful: if the changeset in question is stored in another file (we have all minor versions in separate files which are put together by "include file=..." tags) the changeset is not found.
Plus it's only possible to specify one changeset, so if the table in question has been created using multiple change sets (e.g. adding foreign keys in an extra step) specifying just one set to apply will not get us to the original DB setup.
That is an interesting question. 2 conditions have to be met in order to re-apply the changesets
Have a changeste with runAlways attribute (Examplel can be found here: https://docs.liquibase.com/concepts/basic/changeset.html?Highlight=runalways)
Have a precondition that is evaluated prior to running the changeset.- https://docs.liquibase.com/concepts/advanced/preconditions.html?Highlight=pre%20condition
I wanted to change the data type of one field from string to date. So i dropped the table in db. Then modified the liquibase file and ran the application. now it complains with the following message.
liquibase.exception.ValidationFailedException: Validation Failed:
So after that I reverted the liquibase file changes and ran the application. This time no error but it is not creating the table.
Please help me how to solve this issue.
I assume the failed validation was an error about checksums. This happens when you modify a changeset which was already executed and try to execute it again.
Liquibase keeps all executed changesets in a table called databasechangelog, so it can find out which changesets can be skipped during execution.
To execute a changeset again, delete the corresponding from this table before, and run Liquibase again.
When using Liquibase, you shouldn't (in general) modify the database outside of Liquibase - the main exception being if you are a developer working on your own private development database. If you are in that state (working on your own private database), then when you modify the database outside of Liquibase (i.e. dropping a table) you will also need to delete the row in the DATABASECHANGELOG table that corresponds to the table create statement so that when you re-run liquibase update it will re-create the table.
In our case liquibase is used to update databses for existing installation. New installations are already up to date.
Assuming we have got a new installation. Starting the application will force to execute liquibase changesets (e.g. change type of a column) but as I mentioned before there is nothing to update as the column already was created with the correct type.
Does liquibase recognize that the table column is already up to date or does it try to execute the changeset as there is no entry within the databasechangelog table for it?
Liquibase uses an alternative approach that avoids a need to analyze the target database's data dictionary. This makes DB operations simpler and more cross platform.
A special table "DATABASECHANGELOG" keeps a record of the changesets applied to the target database instance. This table also contains a checksum (calculated at runtime) to determine if changsets are altered between runs of liquibase.
So if you altered the type of a table column, liquibase can detect this and can throw an error, when run against an existing database. (Obviously, on a new DB, the table would be created as expected).
Finally, the changeset documentation describes two optional attributes ("runAlways" and "runOnChange") which could tell lqiuibase to reapply a changeset more than once to a database. There is also a "clearCheckSums" command that can be used to reset the checksums on an existing database. Obviously you need to know what you're doing when using such an option :-)
Liquibase will not recognize anything automatically.
But you can use <preConditions/> in your changeSet to check if your changeSet must be applied or not.
I see an attribute runOnChange that re-runs changeset when it is changed. But is it possible to apply a rollback for this changeset before re-applying automatically?
For example, I have a script that is called from changeset. I made some changes there and want to re-apply, but before it rollback need to be called and after it new version of the script should be applied.
Thank you!
There is no feature to automatically roll back a changeSet on checksum change. Not sure if it is possible in general because if the configuration has changed you don't know what the old value was to roll back.
Depending on what you are doing in your script and your database, can roll back the changes manually in the script and use the liquibase runOnChange="true" changeSet flag.
For example, if you have a script that creates a stored procedure, you could use <changeSet runOnChange="true"> and then define your procedure as "CREATE OR REPLACE"
If you have a script that defines a view, you could add a <sql>IF EXISTS VIEW_NAME DROP VIEW VIEW_NAME</sql>