tagDatabase in Formatted SQL changeset - liquibase

I'm trying to tag a Formatted SQL changeset so a matching ID and tag are written to the DATABASECHANGELOG table (for rollback purposes - see changeset fragment below). The Phing liquibase task is being used to apply the 'update' command for a single changelog.
Although the 'tagDatabase' attribute isn't listed for Formatted SQL changelogs (http://www.liquibase.org/documentation/sql_format.html), neither is logicalFilePath, and that seems to be working OK!
Can someone let me know definitively if tagDatabase is not supported for a Formatted SQL changeset?
Many thanks in advance,
IR8
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
--changeset id:123 logicalFilePath:path-independent
ALTER TABLE test1
ADD COLUMN text_column
text NULL;
--rollback ALTER TABLE test1 DROP COLUMN text_column;
--changeset id:tag123 tagDatabase:123;
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Unexpected error running Liquibase: No SQL for changeset ../db/changelog/databaseChangeLog.sql::tag123::id
Execution of target "migrate" failed for the following reason: C:\data\htdocs\TestLiquibase\deploy\build.xml:49:1: Liquibase exited with code -1

I think it's not included.
The java class FormattedSqlChangeLogParser takes care of parsing the formatted sql file and has a couple of Patterns defined for this. There is a logicalFilePathPattern but nothing for tagDatabase.

So this is not implemented yet.
From the official document,
When you run the updateToTag command or the Maven update goal with the liquibase.toTag attribute, and there is a row in the DATABASECHANGELOG table corresponding to the tagDatabase changeset in the changelog, the updateToTag command or the update Maven goal deploys all changesets starting with the first changeset at the top of the changelog file and moving down to the changeset up to the tag specified by the tagDatabase Change Type.
We can do one thing that is, after you executing all the SQL format changesets, we can create a new database tag changeset and update with liquibase
<?xml version="1.0" encoding="UTF-8"?>
<databaseChangeLog
xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-4.3.xsd">
<changeSet author="sivaramr" id="7">
<tagDatabase tag="v_1.0.0"/>
</changeSet>
</databaseChangeLog>
If you are using any liquibase command tool or liquibase plugins, then you can perform tag command to add the tag to the last changelog as follow
Here It is illustrated with gradle plugin.
./gradlew tag -PliquibaseCommandValue=v1.0.0
Before executing the above tag command, you can check the tag is already exist or not by tagexists command
./gradlew tagExists -PliquibaseCommandValue=v1.0.0

Related

liquibase checks that changesets have not been changed, but doesn't check that they've not been removed: is there a setting to fix this?

Liquibase does a pretty good job in keeping the applied changesets consistent with their source-folder.
If you modify a changeset that has already been applied to the db, Liquibase refuses to do anything, even operations that are not related with the modified changeset.
The rule enforced here is: anything that has been applied to the database must be unapplied before changing it (this is a usual workflow during development).
The problem of 'orphaned' changesets
Unfortunately this rule doesn't apply if you just delete the changeset completely.
In this case there will be what I call an 'orphaned' changeset, that is a record on the DATABASECHANGELOG table (and the database object, of course) with liquibase not complaining at all of a missing changeset in the source.
I expected an error at least when you 'bump' into the missing changeset, that is when you try to rollback it, but liquibase simply seems to ignore its presence, it skips it and rolls back the next one. This can be a problem.
The question is: can we change this liquibase behavior via settings? Is this design needed for some use case that I haven't thought of?
I think what I'm asking should be clear enough, however here is an example as demonstration.
Example
<databaseChangeLog ... >
<include relativeToChangelogFile="true" file="CS1.sql"/>
<include relativeToChangelogFile="true" file="CS2.sql"/>
</databaseChangeLog>
My changesets are SQL based files like this:
-- liquibase formatted sql
-- changeset agostinox:-1
CREATE TABLE T1 (
X INT
)
-- rollback DROP TABLE T1
And CS2.sql is the same for the T2 table.
Now i can apply my changesets like this:
PS > liquibase update
...
Liquibase Version: 4.19.0
Liquibase Open Source 4.19.0 by Liquibase
...
Running Changeset: CS1.sql::-1::agostinox
Running Changeset: CS2.sql::-1::agostinox
Liquibase command 'update' was executed successfully.
And on my db, the DATABASECHANGELOG has the following content:
ID
AUTHOR
FILENAME
DATEEXECUTED
ORDEREXECUTED
EXECTYPE
MD5SUM
-1
agostinox
CS1.sql
2023-01-18 18:52:08.476689
1
EXECUTED
8:d966f9ba2b90eaea9b917a6d93962eff
-1
agostinox
CS2.sql
2023-01-18 18:52:08.666667
2
EXECUTED
8:7f2a735fa83b196a0c72885c95362b81
So far so good. Now, now I get to the point.
Let's mess with the CS1.sql, by adding a comment:
-- liquibase formatted sql
-- changeset agostinox:-1
CREATE TABLE T1 (
X INT --Added comment, very harmless but enough to annoy liquibase :-)
)
-- rollback DROP TABLE T1
Now, I try to rollback the last changeset.
PS > liquibase rollbackcount 1
...
Unexpected error running Liquibase: Validation Failed:
1 changesets check sum
CS1.sql::-1::agostinox was: 8:d966f9ba2b90eaea9b917a6d93962eff but is now:
8:2cea5484e81eb542fa94bb67ba2ffdf5
For more information, please use the --log-level flag
You can see that liquibase complains about CS1.sql been changed even if we are not even rolling back CS1.sql but CS2.sql. Actually this change blocks any further operation, so it have to be reverted in order to break the deadlock.
However, if you just remove the changeset, liquibase doesn't complain anymore:
<databaseChangeLog ... >
<!--REMOVED <include relativeToChangelogFile="true" file="CS1.sql"/> -->
<include relativeToChangelogFile="true" file="CS2.sql"/>
</databaseChangeLog>
PS > liquibase rollbackcount 1
...
Rolling Back Changeset: CS2.sql::-1::agostinox
Liquibase command 'rollbackCount' was executed successfully.
But even if liquibase says that everything is successfull, it is not really the case, infact the table DATABASECHANGELOG now contains the 'orphaned' changeset (and the database contains the T1 table since also the rollback code is gone).
ID
AUTHOR
FILENAME
DATEEXECUTED
ORDEREXECUTED
EXECTYPE
MD5SUM
-1
agostinox
CS1.sql
2023-01-18 18:52:08.476689
1
EXECUTED
8:d966f9ba2b90eaea9b917a6d93962eff
For db people, it seems like liquibase does a left join between changesets source folder and records in DATABASECHANGELOG (that is: all the items on the left set are taken and are associated with the matching items on the right set); this way liquibase can see applied migrations and check if their checksum matches. It can also see not-yet-applied migrations (changesets in source folder without a DATABASECHANGELOG record associated) in order to apply them in the next 'update' call.
But it can't see records from DATABASECHANGELOG (the right set) that don't have an associated changeset in the source folder.
This parallel with join operation well explains the liquibase behavior, so let's use it for describing what a better behavior might be.
Liquibase should do a full-join, that is: also the items on the right side that don't have a source changeset associated are considered. Those records indicate an anomaly that is possibly just a 'major version' of the one that you have when a checksum doesn't match. Indeed you can easily think of a null file as a particolar case of checksum, the checksum of null being something necessarily different from the one in the table. Therefore the existence of those non associated record should give the same kind of error. This is what i would expected to truly ensure consistence between source an DATABASECHANGELOG.
I've been curious about this also. I'd recommend opening a github issue so a Liquibase employee can address why they don't do this check. I'm guessing this was done on purpose.
https://github.com/liquibase/liquibase/issues

Update an existing trigger into Liquibase

I'm very new at Liquibase, and I need some help.
I have an existing trigger that was not capturing all the data; I made some changes to my local Oracle database. Now I need to add those changes into the Liquibase, but I'm lost how to do that.
I know you cannot breach the contract in liquibase by updating the original .xml file directly.
From my understanding, I need to create a new changelog .XML file and then include the path on the other post_migration file.
My confusion is, do I have to drop the original trigger, then create a new file or?
Thanks!
I never create triggers, procedures or even views within the XML file exactly because this makes things more complicated (I think).
I typically move the actual trigger definition into a SQL script (that I can also run separately during development and testing), then include that SQL file from within the Liquibase changelog:
<changeSet id="42" author="arthur" runOnChange="true">
<sqlFile path="triggers/some_trigger.sql"
stripComments="false"
splitStatements="true"
endDelimiter="/"
relativeToChangelogFile="true"/>
</changeSet>
The some_trigger.sql script is stored in git (svn, ...) together with the XML changelog. The runOnChange="true" is the "magical ingredient" here. You don't have to touch the XML file, you just edit the SQL script. During deployment, Liquibase will check if the (SQL) file has changed and run the script if needec.
So, I believe that you create/update/replace the SQL trigger in your local developer database and right now you want to include the liquibase script to the release distribution package of your product.
The liquibase doesn't provide special xml syntax to create triggers, so you will just add a new changeset that holds your pl/sql script inside the <SQL> tag. The script will be the same that you run on your local database.
The example code here:
<changeSet id="1" author="me">
<sql endDelimiter="/">
CREATE OR REPLACE TRIGGER trigger_name before insert
on table1_name for each row
BEGIN
select seq_myseq.nextval
into :new.myid
from dual;
END;
/
</sql>
</changeSet>
This code just compile trigger in the aimed database when you call liquibase update. In most cases, it is enough. But I strictly recommend you to ask your DBA or team led for rules that your team enforced for writing liquibase scripts. For this reason, the result may be much more complicated.

Liquibase preconditions to all changelog file in sql

I would like to run a SQL precondition checking for each changeSet in my SQL changeLogFile. It is actually a precondition on the changeLog itself
Here is an extract of it :
--liquibase formatted sql
--preconditions onFail:HALT onError:HALT
--precondition-sql-check expectedResult:"1.0" SELECT VERSION FROM VERSION_TABLE;
--changeset bob:1 failOnError:true dbms:oracle
ALTER INDEX XXX RENAME TO YYY;
--rollback YYY RENAME TO XXX;
Even if the precondition is actually not respected, liquibase still run all the changeset.
Does somebody knows if it is an error from my side or if liquibase does not allow preconditions on entire changeLog for SQL changeLog file ?
Thanks in advance !
If you go through the documentation, then its stated that we can only apply pre-conditions on a specific change set. Also, only the SQL Check precondition is supported.
Liquibase doc for sql changelog files - https://www.liquibase.org/documentation/sql_format.html
One thing you could do is have a top-level master changelog in XML/YAML/JSON format and then use <include> or <includeAll> elements to include your liquibase formatted sql changelogs. If you do that, then you can have changelog-level preconditions.

How to update existing changeset

I want to update very old changeset because there is a bug in that. I have updated that changesetid and After running it I got checksum error i.e checksum got modified. Now I have added validCheckSum Tag with this new checkSum. After it is getting successfully I cant not see new changes. Is there any way to update that change set which has already been executed.
Each changeSet tag is uniquely identified by the combination of the “id” tag, the “author” tag so please check whether id and author name already exists or not in DATABASECHANGELOG table. If the combination exists then change the id in your xml file then run the build
If still not works then add runOnChange property in <ChangeSet> and try it
runOnChange - Executes if any changes found it already existing changeset
Example of using runOnChange in changeset with sqlFile tag .
I had a script for Oracle 18 using IDENTITY column type and than have to change it for Oracle 10 which doesn't support IDENTITY column type
<changeSet id="2" author="auto" runOnChange="true">
<sqlFile dbms="oracle" encoding="utf8" endDelimiter=";" path="oracle/230.sql" relativeToChangelogFile="true" splitStatements="true" stripComments="true" />
</changeSet>
If it is a very old changeset as you have stated you should consider not changing the old changeset but add a new changeset instead which fixes the bug since it allows you better control over the versions you have. This is one of the main reasons of using liquibase.

Issue with tagDatabase operation in Liquibase

I need to add support of liquibase to existing application with existing database. I set up maven plugin and when I'm running it it creates Liquibase tables. As a next step I want to tag database at the very beginning to create a tag for rollback.
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.liquibase:liquibase-plugin:1.9.3.0:tag (tag.database) on project liquitest: Error setting up or running Liquibase: liquibase.
exception.JDBCException: Cannot tag an empty database -> [Help 1]
Is it possible to tag initial state of database when liquibase was just added and no changesets were applied?
Thanks!
Looks like it is not possible, because tag is stored in DATABASECHANGELOG table, and if there are no records there, it is not possible to store it.
The workaround is to add a dummy migration, for examle:
<changeset id="1" author="qqq">
<sql>select 1 from dual</sql>
</changeset>