I am using liquibase version-4.11 connected with snowflake and i have two pipelines for database automation using liquibase one is for DB Deploy and another one is DB Rollback. We are using this pipeline for the schema updation in snowflake. The DB changes has been versioned using tags in DB Deploy Pipeline so the rollback can be achieved using the DB Rollback Pipeline. Whenever i run the rollback pipeline i am facing the below issue for the drop statements :
"Unexpected error running Liquibase: No inverse to liquibase.change.core.DropColumnChange created"
The Liquibase is not supporting the rollback for drop statements. The only way for this is to add the rollback script manually. But in my organization is not possible for adding it manually for all the drop statements. Its there is any ways to achieve the rollback without involving any manual steps?
Related
I learned that Liquibase runs each changeSet in a transaction and commits it after inserting into the DATABASECHANGELOG table.
If something goes wrong during the changeSet, the transaction gets rolled back.
My question is, what happens if the changeSet also includes a <rollback> tag.
I know that the rollback tags are used in combination with liquibase dedicated rollback commands, but which one has precedence during a regular migration, the command from the rollback tag or the transaction abortion?
The <rollback> block is not used in when an update fails. If the update fails, Liquibase just calls "rollback" on the connection and relies on the database to correctly roll things back. The <rollback> block is only used when doing the separate rollback command where we're "undoing" the changeset that was already committed in the database.
NOTE: the fact that Liquibase relies on the connection rollback to revert a failed update is why you have to be careful if you have multiple statements in the same changeSet. If you have two createTable calls and your the database auto-commits after the first and the second fails, the rollback on the connection will not roll back the first table. So when you run update the next time, you'll get a table already exists error from the first createTable. The rules on exactly what statements auto-commit are database-specific, so the good rule of thumb is "only one statement per changeset"
What would be the rollback of droptable:
- changeSet:
id: 1
author: vikas
changes:
- sql:
sql: DROP TABLE IF EXISTS `adapter`
What should I put in rollback for the above changeset.
I am getting below error when I was trying to rollback without having rollback tag:
Error setting up or running Liquibase: liquibase.exception.RollbackImpossibleException: No inverse to liquibase.change.core.RawSQLChange created
The error
Error setting up or running Liquibase: liquibase.exception.RollbackImpossibleException: No inverse to liquibase.change.core.RawSQLChange created
happens because you use raw sql, and not the appropriate tag for it (it would be dropTable), so liquibase doesn't know what to do with it. It can't create a rollback.
Also, the documentation says that
Other refactorings such as “drop table” and “insert data” have no corresponding rollback commands that can be automatically generated. In these cases, and cases where you want to override the default generated rollback commands, you can specify the rollback commands via the tag within the changeSet tag. If you do not want anything done to undo a change in rollback mode, use an empty tag.
So you'll have to create a proper rollback for drop table operation.
I am new to Liquibase and I am trying to get the rollback feature working. I am running Liquibase in Windows. I executed an install which tagged the database at version_1.0. I ran an update which tagged the database at version_1.1. Now I am trying to rollback to version_1.0
Here is the command I am running:
liquibase rollback version_1.0
this gives an error that a --changeLogFile is needed so I ran this
liquibase --changeLogFile=v001/master.xml rollback version_1.0
I provide the name of the change log file that was executed during the update, but nothing gets. The update contains 2 create table statements and the tables were not dropped. What am I missing in the rollback process?
Liquibase does not tag the database really. It only tags to know which changessets need to be rolled back.
The rollback will be done by execute the rollback tags in the changeling.
We have couple of data schemas and we investigate the migration to Liquibase. (One of data schemas is already migrated to Liquibase).
Important question for us is if Liquibase supports dry run:
We need to run database changes on all schemas without commit to ensure we do not have problems.
In case of success all database changes run once again with commit.
(The question similar to this SQL Server query dry run but related to Liquibase)
Added after the answer
I read documentation related to updateSQL and it is not answers the requirements of “dry run”.
It just generates the SQL (in command line, in Ant task and in Maven plugin).
I will clarify my question:
Does Liquibase support control on transactions?
I want to open transaction before executing of Liquibase changelog, and to rollback the transaction after the changelog execution.
Of course, I need to verify the result of the execution.
Is it possible?
Added
Without control on transactions (or dry run) we can not migrate to Liquibase all our schemas.
Please help.
You can try "updateSQL" mode, it will connect db (check you access rights), acquire db lock, generate / print SQL sentences to be applied (based on db state and you current liquibase change sets) also it will print chageset id's missing in current state of db and release db lock.
Unfortunately, no.
By default, Liquibase commits the transaction executing all statements of a changeset. I assume that the migration paths you have in mind usually involve more than a single changeset.
The only way you can modify the transaction behavior is the runInTransaction attribute for the <changeset> tag, as documented here. By setting it to false, you effectively disable the transaction management, i.e. it enables auto-commit mode as you can see in ChangeSet.java.
I think that this feature could be a worthwhile addition to Liquibase, so I opened a feature request: CORE-1790.
I think your answer is "it does not support dry runs" but the problem is primarily with the database and not with liquibase.
Liquibase does run each changeSet in a transaction and commits it after inserting into the DATABASECHANGELOG table so in theory you could override liquibase logic to roll back that transaction instead of committing it, but you will run into the problem where most SQL ran by liquibase is auto-committing.
For example, if you had a changeSet of:
<changeSet>
<createTable name="test">
...
</createTable>
</changeSet>
What is ran is:
START TRANSACTION
CREATE TABLE NAME ...
INSERT INTO DATABASECHANGELOG...
COMMIT
but even if you changed the last command to ROLLBACK the create table call will auto-commit when it runs and the only thing that will actually roll back is the INSERT.
NOTE: there are some databases that will rollback DDL SQL such as postgresql, but the majority do not.
INSERT/UPDATE commands would run in a transaction and could be auto-rolled back at the end, but liquibase does not have a postCondition command to do the in-transaction check of the state that would be required. That would be a useful feature (https://liquibase.jira.com/browse/CORE-1793) but even it would not be usable if there are any auto-committing change tags in the changeset. If you added a postcondition to create table example above, the postcondition would fail and the update would fail, but the table would still be there.
If your Liquibase migration is sufficiently database agnostic, you can just run it on an in-memory H2 database (or some other "throwaway database") that you can spin up easily using a few lines of code.
var info = new Properties();
info.put("user", "sa");
info.put("password", "");
try (var con = new org.h2.Driver().connect("jdbc:h2:mem:db", info)) {
var accessor = new FileSystemResourceAccessor();
var jdbc = new JdbcConnection(con);
var database = DatabaseFactory.getInstance().findCorrectDatabaseImplementation(jdbc);
Liquibase liquibase = new Liquibase("/path/to/liquibase.xml", accessor, database);
liquibase.update("");
}
I've blogged about this approach more in detail here.
In HSQL to change TRANSACTION CONTROL there can't be any active transactions.
Flyway, in turn, after committing migration X and before executing SQL from migration X, sets autocommitt=false and executes some of its own statements. So if the migration contains SET DATABASE TRANSACTION CONTROL statement it will wait for those uncommitted statements forever causing application to hang.
(Side note: The statements executed by flyway before migration varies from version to version e.g. in 1.7 that were pure selects so changing from LOCK to MVCC was possible but after I had MVCC any subsequent DDL statements in further migrations hanged; in flyway 2.0 it was select for update on schema_version table so any transaction control change hanged; in 2.2 select for update was changed to explicit lock with the same effect as in 2.0)
So basically it is not possible to change transaction control in flyway migrations. On the other hand flyway discourages changes outside of its migration. Any idea then how to change transaction control in with flyway/hsql?
Update
Another observation is that when database control is set to MVCC then any DDL statement in flyway migration hangs application too. So I would just set LOCKS before each migration and restore MVCC after it. Would that be clean solution from Flyway perspective?
import com.googlecode.flyway.core.util.jdbc.JdbcUtils;
public void migrate() {
setDbTransactionControl("LOCKS");
flyway.migrate();
setDbTransactionControl("MVCC");
}
private void setDbTransactionControl(String mode) {
Connection connection = null;
try {
connection = JdbcUtils.openConnection(ds);
connection.createStatement().execute("SET DATABASE TRANSACTION CONTROL " + mode);
} catch (SQLException e) {
//log it
JdbcUtils.closeConnection(connection);
} finally {
JdbcUtils.closeConnection(connection);
}
}
This is not possible inside a Flyway migration.
Before Flyway starts a migration, it opens a transaction in a separate connection to acquire a lock on its metadata table. So you will never be able to execute a statement that absolutely must be run without any other transactions.
Your best option is probably to set it on the datasource, so it can init each connection this way upon create.
Try to use the Flyway callbacks beforeMigrate and afterMigrate. Both run apart from the migration transactions. MVCC should be used for my application so the the JDBC URL contains hsqldb.tx=mvcc. I could sucessfully change the transaction model during the Flyway migration with beforeMigrate.sql SET DATABASE TRANSACTION CONTROL LOCKS; and afterMigrate.sql SET DATABASE TRANSACTION CONTROL MVCC;. There are also Java versions of the callbacks. I'm using HSQLDB 2.3.3 and Flyway 3.2.1.