I found 2 issues per se when I ran the updateSQL commandline in Liquibase
The last statement in Liquibase updateSQL output viz Insert into DBCHANGELOG table does not commit automatically when the sql is run via sqlplus commandline
As a result of this, though the changeset gets executed, the DBCHANGELOG table does not have the insert statement to record it. So when I run the updateSQL once again, the last changeset is once again created in the SQL output which is incorrect.
Liquibase does not validate / check syntax errors in SQL.
As a result of this, even if the changeset SQL fails, the insert to DBChangeLog table for the changeset succeeds which is incorrect. Is there a way that the insert statement following the changeset be stopped / failed if the changeset SQL actually failed ?
Any help is greatly appreciated... we are this close to getting Liquibase implemented... !!
To answer the question in your subject line, no, Liquibase cannot validate the SQL. Liquibase supports many different databases, and each has different SQL syntax.
If you can, stop using the SQL generated by updateSQL to actually do the updates, and use Liquibase itself to do the updates. That way Liquibase can detect errors and behave more properly. I recommend that if DBAs are scared of Liquibase touching the database that teams use the generateSQL as a pre-check to see what Liquibase will do, but let Liquibase do its job.
I also find best practice of Liquibase is not to use a SQL script but to manually write the Liquibase XML file for the change.
I've tried using the ExecuteCommand tag to lunch sqlplus or sqlcmd (as I know my target database) and it has a bug which as for now it is closed?! (but this is open source, so I can't complain :) )
Having said that, I found that working on XML to specify the changes causes many other challenges, for example:
1. Making sure that every change was included in the changelog xml file. I've heard many organizations who forget to add the file to the changelog.
2. Making sure the file for the specific change is always in sync with the file-based version control. Imagine what will happen if it doesn't - which happens to many of my customers...
3. Wasted time spent on merging changelogs between different environments (branches, UAT - critical fixes, sandboxes, etc...)
Related
We are deploying sql query's using liquibase automation tool.
liquibase version: 4.3.0
We have created DB files in SQL format and used the "liquibase validate" command to validate the syntax error. But I didn't get expected result.
Ex: dem1.sql, demo2.sql
I have found the below doc's for validation,
https://docs.liquibase.com/commands/validate.html?msclkid=72d2d27da9cf11ecae4cc3525df5294e
As per the above document, "liqubase validate" command won't support for SQL format files.
Can any one help me to validate the SQL format file with liquibase or any other ways (like shell script or jenkins pipeline stage) before deploy the sql query's to my DB.
One possible approach for syntax validation could be to utilise a Database test-container matching your runtime environment during integration coverage.
That way your LiquiBase changelogs can be executed during test-setup and syntax is implicitly validated.
See example here (https://www.baeldung.com/spring-boot-testcontainers-integration-test)
im using Liquibase to manage my database in SQLdeveloper. Now, we have a PLSQL script that runs fine by itself. It also makes the changes in the database. Now, if we want Liquibase to run this PLSQL script it gets the following error: ORA-00922: missing or invalid option.
Is it even possible for Liquibase to run external PLSQL scripts?
There are two potential issues you are running into:
You are using <sqlFile> but not specifying splitStatements="false". By default, Liquibase will split the SQL on semicolons because that is what the JDBC driver needs normally, but in a PL/SQL script you can have a single CREATE PROCEDURE or something similar which contains semicolons but is still one statement. You can also use <createProcedure> instead of sqlFile which does not split statements.
Your script has sqlplus-specific functionality in it. SQLPlus and SQLDeveloper do not simply pass the SQL strings straight through to the database, they have their own functionality to sometimes modify what is actually executed. JDBC and therefore Liquibase does not have all the same functionality and so if you are using it in your scripts they will not work. If this is the case, the best approach is to use a changeSet with <executeCommand> to make a call to sqldeveloper or sqlplus.
I use an application connected with an sql database. I found using the profiler that the application runs an update query with a syntax error. I don't have access to the application's source code. The result is that the record is not updated. Is there a way to modify the query every time it is executed with something like trigger? I can't use INSTEAD OF because there ism't any record updated or inserted.
This answer
https://stackoverflow.com/a/3319031/1359088
suggests a way to log to a text file all the errors. You could write a little utility and schedule it to run every hour or whatever, which could read through this log, find the erroneous sql statements, fix them, then run them itself.
I am trying to migrate from oracle db to mysql or postgres using Liquibas. I have generated the sql file using Liquibase but, the syntax is not right there is a lot of issue with the generated sql. If anyone has any solution please do let me know thank you.
The best approach is to use the generateChangeLog function to create an XML changeSet description of your oracle database. Go through the generated changelog to make sure everything expected is there, and make any changes to the file as needed such as data type changes.
Once the changelog is correct, you can run the XML changelog directly against your mysql or postgresql database or use updateSQL to generate the SQL liquibase would use. Liquibase will create the correct database-specific SQL when it runs a changelog against a given database.
i want to extract specific database tables & stored procedures into one master script. Do you know any software that can help me do this faster? I've tried using the SQL Database publishing tool, but it's not that efficient since its gathering tables that I didn't select.
In SQL Server 2005, right click on the database, then select Tasks, and then select Generate Scripts.
Generating SQL Scripts in SQL Server 2005
As mentioned in that link, I'm fairly sure you have to generate the DROP and CREATE statements separately.
Try DBSourceTools. http://dbsourcetools.codeplex.com
Its open source, and specifically designed to script databases - tables, views, procs to disk.
It also allows you to select which tables, views, db-objects to script.
I use Redgate SQL compare for this (by comparing to an empty DB), as well as for doing upgrades between all my DB versions (I save a copy of the DB for each released version, and then just do a compare between current and previous to get a change script for that version).
I have found the "Generate Scripts" does a bad job in some cases with dependencies - eg, it will try to create a stored procedure that uses a table before the table is created, causing the script to fail. I'll accept I'm possibly using it wrong, but SQL Compare "just works". The scripts it generates are also enclosed in a transaction -- so if something fails, the whole change is rolled back. You don't end up with a half-populated or half-upgraded database.
Downside is that this is a commercial tool, but IMHO worth the money.