Is it OK to manually delete rows from Databasechangelog?
Btw, rollback to a TAG is not an option in my case.
Deleting rows from the table will simulate a rollback without actually performing one.... Leaving your database in an unknown state (the whole point is that liquibase manages the schema for you).
Why not just rollback a specified number of changesets instead?
liquibase rollbackCount 5
Related
Liquibase does a pretty good job in keeping the applied changesets consistent with their source-folder.
If you modify a changeset that has already been applied to the db, Liquibase refuses to do anything, even operations that are not related with the modified changeset.
The rule enforced here is: anything that has been applied to the database must be unapplied before changing it (this is a usual workflow during development).
The problem of 'orphaned' changesets
Unfortunately this rule doesn't apply if you just delete the changeset completely.
In this case there will be what I call an 'orphaned' changeset, that is a record on the DATABASECHANGELOG table (and the database object, of course) with liquibase not complaining at all of a missing changeset in the source.
I expected an error at least when you 'bump' into the missing changeset, that is when you try to rollback it, but liquibase simply seems to ignore its presence, it skips it and rolls back the next one. This can be a problem.
The question is: can we change this liquibase behavior via settings? Is this design needed for some use case that I haven't thought of?
I think what I'm asking should be clear enough, however here is an example as demonstration.
Example
<databaseChangeLog ... >
<include relativeToChangelogFile="true" file="CS1.sql"/>
<include relativeToChangelogFile="true" file="CS2.sql"/>
</databaseChangeLog>
My changesets are SQL based files like this:
-- liquibase formatted sql
-- changeset agostinox:-1
CREATE TABLE T1 (
X INT
)
-- rollback DROP TABLE T1
And CS2.sql is the same for the T2 table.
Now i can apply my changesets like this:
PS > liquibase update
...
Liquibase Version: 4.19.0
Liquibase Open Source 4.19.0 by Liquibase
...
Running Changeset: CS1.sql::-1::agostinox
Running Changeset: CS2.sql::-1::agostinox
Liquibase command 'update' was executed successfully.
And on my db, the DATABASECHANGELOG has the following content:
ID
AUTHOR
FILENAME
DATEEXECUTED
ORDEREXECUTED
EXECTYPE
MD5SUM
-1
agostinox
CS1.sql
2023-01-18 18:52:08.476689
1
EXECUTED
8:d966f9ba2b90eaea9b917a6d93962eff
-1
agostinox
CS2.sql
2023-01-18 18:52:08.666667
2
EXECUTED
8:7f2a735fa83b196a0c72885c95362b81
So far so good. Now, now I get to the point.
Let's mess with the CS1.sql, by adding a comment:
-- liquibase formatted sql
-- changeset agostinox:-1
CREATE TABLE T1 (
X INT --Added comment, very harmless but enough to annoy liquibase :-)
)
-- rollback DROP TABLE T1
Now, I try to rollback the last changeset.
PS > liquibase rollbackcount 1
...
Unexpected error running Liquibase: Validation Failed:
1 changesets check sum
CS1.sql::-1::agostinox was: 8:d966f9ba2b90eaea9b917a6d93962eff but is now:
8:2cea5484e81eb542fa94bb67ba2ffdf5
For more information, please use the --log-level flag
You can see that liquibase complains about CS1.sql been changed even if we are not even rolling back CS1.sql but CS2.sql. Actually this change blocks any further operation, so it have to be reverted in order to break the deadlock.
However, if you just remove the changeset, liquibase doesn't complain anymore:
<databaseChangeLog ... >
<!--REMOVED <include relativeToChangelogFile="true" file="CS1.sql"/> -->
<include relativeToChangelogFile="true" file="CS2.sql"/>
</databaseChangeLog>
PS > liquibase rollbackcount 1
...
Rolling Back Changeset: CS2.sql::-1::agostinox
Liquibase command 'rollbackCount' was executed successfully.
But even if liquibase says that everything is successfull, it is not really the case, infact the table DATABASECHANGELOG now contains the 'orphaned' changeset (and the database contains the T1 table since also the rollback code is gone).
ID
AUTHOR
FILENAME
DATEEXECUTED
ORDEREXECUTED
EXECTYPE
MD5SUM
-1
agostinox
CS1.sql
2023-01-18 18:52:08.476689
1
EXECUTED
8:d966f9ba2b90eaea9b917a6d93962eff
For db people, it seems like liquibase does a left join between changesets source folder and records in DATABASECHANGELOG (that is: all the items on the left set are taken and are associated with the matching items on the right set); this way liquibase can see applied migrations and check if their checksum matches. It can also see not-yet-applied migrations (changesets in source folder without a DATABASECHANGELOG record associated) in order to apply them in the next 'update' call.
But it can't see records from DATABASECHANGELOG (the right set) that don't have an associated changeset in the source folder.
This parallel with join operation well explains the liquibase behavior, so let's use it for describing what a better behavior might be.
Liquibase should do a full-join, that is: also the items on the right side that don't have a source changeset associated are considered. Those records indicate an anomaly that is possibly just a 'major version' of the one that you have when a checksum doesn't match. Indeed you can easily think of a null file as a particolar case of checksum, the checksum of null being something necessarily different from the one in the table. Therefore the existence of those non associated record should give the same kind of error. This is what i would expected to truly ensure consistence between source an DATABASECHANGELOG.
I've been curious about this also. I'd recommend opening a github issue so a Liquibase employee can address why they don't do this check. I'm guessing this was done on purpose.
https://github.com/liquibase/liquibase/issues
I perform a liquibase update command with a given ChangeLog XML file and tag it with tag1.
For example:
liquibase --driver=org.postgresql.Driver --url=... --changeLogFile=change1.xml update
liquibase --driver=org.postgresql.Driver --url=... --tag tag1
I then perform several additional updates commands with other ChangeLog XML files:
liquibase --driver=org.postgresql.Driver --url=... --changeLogFile=change2.xml update
liquibase --driver=org.postgresql.Driver --url=... --changeLogFile=change3.xml update
Now I would like to rollback to tag1:
liquibase --driver=org.postgresql.Driver --url=... --changeLogFile=??? rollback tag1
Which file should I specify in --ChangeLogFile? Is there a way to define multiple files? Is there a way Liquibase can store embed the rollback commands without the need to supply the update XMLs?
Rather than using multiple changelog files from the command line, typical usage is to just have one changelog file that is used for the whole application. It is possible to have multiple changelogs, but in that pattern you have a single 'master' changelog that then includes other changelogs. see https://www.liquibase.org/bestpractices.html for some examples of this.
When rolling back, liquibase needs to know what the actual changes are, so it can then do something else to do the rollback. The DATABASECHANGELOG table does not store the actual contents of each changeset that has been applied, it just keeps the id, author, filepath, and a checksum. It has no way of knowing that changeset id 123778 by steve in file changelog.xml was a create table or an alter column or anything. So the changelog file MUST be there to be able to roll things back.
For certain change types like 'create table', Liquibase can 'automatically' generate a rollback for you - it is just 'drop table'. But if the change was something like 'drop table', it cannot generate a rollback - drop table command only has the name of the table to drop, and doesn't know what columns were in that table, etc.
So that is why Liquibase requires that you always supply a changelog file, and if you want to be able to do rollbacks, you may also need to specify how to roll back each change.
We are using liquibase to split a column into two columns. This happens in three changes:
Add the new columns via addColumn
Insert the data from the old column into the new ones via a customChange
Delete the old column via dropColumn
This works great, but I can not find any documentation on the order of execution of changes.
I only found documentation on the order of execution of changeSets, see here.
Does liquibase guarantee, that the changes are executed sequentially in the order that they appear?
I've never saw any documentation about it, but in my experience - it does execute changes inside the changeSet sequentially in the order they appear.
Also, I don't think it's good practice to put all the above changes into one changeSet, because, as stated in the document you've provided:
Liquibase attempts to execute each changeSet in a transaction that is committed at the end, or rolled back if there is an error. Some databases will auto-commit statements which interferes with this transaction setup and could lead to an unexpected database state. Therefore, it is usually best to have just one change per changeSet unless there is a group of non-auto-committing changes that you want applied as a transaction such as inserting data.
I suggest separating your changeSet into three atomic ones with appropriate preConditions, or create a proper rollback for it.
When a changeset is marked as failOnError:false, does liquibase record it as having been applied when it fails?
For example, we have a script that performs a pre-emptive drop table in one changeset and then creates the table in the next changeset. When the script is first run, the drop table statement fails as expected and then the table is created successfully. However, the changeset that attempted the drop table is not added to the databasechangelog table.
Is that expected behavior?
That is the current behavior currently. Depending on the reason for the failure, it can make sense to either continue to retry it or to not.
I created https://liquibase.jira.com/browse/CORE-1766 to add the feature to mark it as failed and not try again.
Currently, your best option would be to add a precondition to the dropTable changeSet with onFail="MARK_RAN"
I have a situation where I must:
Disable FK constraints for some tables,
Change field values for primary keys which are referenced by foreign keys (which are also updated to match),
Re-enable FK constraints from step 1.
The problem is that re-enabling the constraints in step 3 might fail if changes from step 2 done to primary key fields are not consistent with fields referenced by foreign keys. I'd like to catch that case and rollback the changes done in step 2. As far as I know, running a DDL statement like enabling a constraint would first issue a commit, after which I cannot rollback the changes made in step 2.
Is there any way to achieve this in one standalone script? The process should either completely pass or rollback as if nothing happened. Or is there an alternative to transactions which could revert to the previous state, without doing backup/restore of the whole database?
The way I'd work around this would be to lock each table before doing the update, run queries to manually check for any violations before re-enabling the constraint. If there are any violations you can rollback the update, otherwise the constraint re-enable will do the commit and release the table lock.
You could try doing the DDL in an autonomous transaction. That way, it will be isolated from your DML. If the DDL fails, you could still rollback the DML. However, if the DDL has to "see" the uncommitted DML changes, then you're stuck. I don't think you can get where you want to go.