We are using liquibase to split a column into two columns. This happens in three changes:
Add the new columns via addColumn
Insert the data from the old column into the new ones via a customChange
Delete the old column via dropColumn
This works great, but I can not find any documentation on the order of execution of changes.
I only found documentation on the order of execution of changeSets, see here.
Does liquibase guarantee, that the changes are executed sequentially in the order that they appear?
I've never saw any documentation about it, but in my experience - it does execute changes inside the changeSet sequentially in the order they appear.
Also, I don't think it's good practice to put all the above changes into one changeSet, because, as stated in the document you've provided:
Liquibase attempts to execute each changeSet in a transaction that is committed at the end, or rolled back if there is an error. Some databases will auto-commit statements which interferes with this transaction setup and could lead to an unexpected database state. Therefore, it is usually best to have just one change per changeSet unless there is a group of non-auto-committing changes that you want applied as a transaction such as inserting data.
I suggest separating your changeSet into three atomic ones with appropriate preConditions, or create a proper rollback for it.
Related
I have introduced a changeset <dropTable table_name="foo"> that will get rid of the no longer needed table foo.
Since dropTable has no auto-rollback I can of course specify the rollback actions manually (by copying the contents of the changeset that originally created foo). It would however be more convenient and less error-prone to just re-apply the said changeset - is this possible?
Update: it is possible, the answer is hidden in the Liquibase Auto Rollback subpage of using-rollback which does not only list change types and their auto-rollback capability but also has an example of referring to an older changeSetId.
However, this feature seems not to be the most powerful: if the changeset in question is stored in another file (we have all minor versions in separate files which are put together by "include file=..." tags) the changeset is not found.
Plus it's only possible to specify one changeset, so if the table in question has been created using multiple change sets (e.g. adding foreign keys in an extra step) specifying just one set to apply will not get us to the original DB setup.
That is an interesting question. 2 conditions have to be met in order to re-apply the changesets
Have a changeste with runAlways attribute (Examplel can be found here: https://docs.liquibase.com/concepts/basic/changeset.html?Highlight=runalways)
Have a precondition that is evaluated prior to running the changeset.- https://docs.liquibase.com/concepts/advanced/preconditions.html?Highlight=pre%20condition
We recently adopted liquibase and it has greatly simplified the task of getting DB in sync with code.
However we might have a problem in using liquibase in production.
The issue in automating the schema application for production is the alters to big tables coming through distinct change sets at different times.
Team 1 checks in "alter table bigtable column1" , Team 2 at later point adds "alter table bigtable column2".
It would take lets say (30 mins ) to run the alters independently in production as opposed to 15 mins if they were part of a single alter stmt wrapped under a single changeset.
We cannot merge them into single change set as it would break liquibase changeset checksum validation.
I would very much appreciate any insights on how folks in general are dealing with this scenario.
Do folks use precondition to get around this issue?
Thanks!
When you recognize the issue, you could create the single changeset and use a label or a context to show that the 'combined' changeset should only be applied to environments where the two original changes have NOT been applied. You could also apply labels or contexts to the existing changes to show that they should only be applied to the environments where they are already applied.
In our case liquibase is used to update databses for existing installation. New installations are already up to date.
Assuming we have got a new installation. Starting the application will force to execute liquibase changesets (e.g. change type of a column) but as I mentioned before there is nothing to update as the column already was created with the correct type.
Does liquibase recognize that the table column is already up to date or does it try to execute the changeset as there is no entry within the databasechangelog table for it?
Liquibase uses an alternative approach that avoids a need to analyze the target database's data dictionary. This makes DB operations simpler and more cross platform.
A special table "DATABASECHANGELOG" keeps a record of the changesets applied to the target database instance. This table also contains a checksum (calculated at runtime) to determine if changsets are altered between runs of liquibase.
So if you altered the type of a table column, liquibase can detect this and can throw an error, when run against an existing database. (Obviously, on a new DB, the table would be created as expected).
Finally, the changeset documentation describes two optional attributes ("runAlways" and "runOnChange") which could tell lqiuibase to reapply a changeset more than once to a database. There is also a "clearCheckSums" command that can be used to reset the checksums on an existing database. Obviously you need to know what you're doing when using such an option :-)
Liquibase will not recognize anything automatically.
But you can use <preConditions/> in your changeSet to check if your changeSet must be applied or not.
At work we have a table to hold settings which essentially contains the following columns:
PARAMNAME
VALUE
Most of the time new settings are added but on rare occasions, settings are removed. Unfortunately this means that any scripts which might have previously updated this value will continue to do so despite the fact that the update results in "0 rows updated" and leads to unexpected behaviour.
This situation was picked up recently by a regression test failure but only after much investigation into why the data in the system was different.
So my question is: Is there a way to generate an error condition when an update results in zero rows updated?
Here are some options I have thought of, but none of them are really all that desirable:
PL/SQL wrapper which notices the failed update and throws an exception.
Not ideal as it doesn't stop anyone/a script from manually doing an update.
A trigger on the table which throws an exception.
Goes against our current policy of phasing out triggers.
Requires updating trigger every time a setting is removed and maintaining a list of obsolete settings (if doing exclusion).
Might have problems with mutating table (if doing inclusion by querying what settings currently exist).
A PL/SQL wrapper seems like the best option to me. Triggers are a great thing to phase out, with the exception of generating sequences and inserting history records.
If you're concerned about someone manually updating rather than using the PL/SQL wrapper, just restrict the user role so that it does not have UPDATE privileges on the table but has EXECUTE privileges on the procedure.
Not really a solution but a method to organize things a bit:
Create a separate table with the parameter definitions and link to that table from the parameter value table. Make the reference to the parameter definition required (nulls not allowed).
Definition table PARAMS (ID, NAME)
Actual settings table PARAM_VALUES (PARAM_ID, VALUE)
(changing your table structure is also a very effective way to evoke errors in scripts that have not been updated...)
May be you can use MERGE statement
here is a link for it
http://www.oracle-developer.net/display.php?id=203
The merge statement allows you to combine insert and update in the same query, so in case the desired row does not exist you may insert a record in a buffer table to indicate the the row does not exist or else you can update the required record
Hope it helps
My database schema has a 'locked' setting meaning that the entry can not be changed once it is set.
Before the locked flag is set we can update other attributes. So:
Would you check the locked flag in code and then update the entry
or
would it be better to combine that into a SQL query, if so, any examples?
EDIT: how would you combine the update & check into one SQL statement?
You should do both. The database should use an update trigger to decide if a row can be updated - this would prevent any one updating the row from the back tables accidentally. And the application should check to see if it should be able to update the rows and act accordingly.
"how would you combine the update & check into one SQL statement?"
update table
set ...
where key = ...
and locked ='N';
That would not raise an error, but would update 0 rows - something you should be able to test for after the update.
As for which is better, my view is that if this locked flag is important then:
you must check/enforce it in the database to ensure it is never violated by any access method
you may also check/enforce it in the application, if that is more user-friendly
I would check the locked flag in code and then (assuming the record isn't locked) run the update query, setting the locked flag in the query as well. That way it can be wrapped in a transaction and committed/rolled back all at once.
I would use a trigger if your DBMS offers this function to enforce the flag. If you set the flag, all updates fail.
Then you can create a special query which will update the flag. Your trigger can check what called the update, and allow the flag to flick back if necessary. That way whether the TSQL is nice or malicious, no one can update your row once the flag is set.
As a rule of thumb I would always prefer to check everything in code and consider writing the DB constraints after that. Having the DB perform consistency checks for you is cool and admittedly faster than doing it in your code but then you are putting some logic in the DB and have to pay some price. These constraints can become vendor-dependent and, worse, can you perform automated tests on these restrictions in your development environment?
So I'd consider putting this kind of checks in the DB as an added safety net but wouldn't rely on them alone.