Liquibase failing while attempting to drop non-existent TMP_% tables during generateChangeLog - liquibase

Running
liquibase generateChangeLog > genChgLog.txt
with the following as my liquibase.properties,
classpath=C:\Program Files (x86)\MySQL\MySQL Connector
J\mysql-connector-java-8.0.16.jar
driver=com.mysql.cj.jdbc.Driver
url=jdbc:mysql://{thisisnottheproblem,Iguarantee}
username={it'sright}
password={it'sright}
referenceUrl=jdbc:mysql://{thisisnottheproblem,Iguarantee}
referenceUsername={it'sright}
referencePassword={it'sright}
changeLogFile=databaseChangeLogSchema.mysql.sql
diffTypes=tables,columns,views,primaryKeys,indexes,foreignKeys,sequences,data
logLevel=debug
I am consistently getting the likes of this, output to the genChgLog.txt file:
Starting Liquibase at Wed, 15 May 2019 15:37:32 CDT (version 3.6.3
built at 2019-01-29 11:34:48) Unexpected error running Liquibase:
liquibase.exception.DatabaseException:
liquibase.exception.DatabaseException: Unknown table
'TMP_CTAWHBNCQVQMHSUU' [Failed SQL: DROP TABLE TMP_CTAWHBNCQVQMHSUU]
liquibase.exception.LiquibaseException:
liquibase.command.CommandExecutionException:
liquibase.exception.DatabaseException:
liquibase.exception.DatabaseException: Unknown table
'TMP_CTAWHBNCQVQMHSUU' [Failed SQL: DROP TABLE TMP_CTAWHBNCQVQMHSUU]
at
liquibase.integration.commandline.CommandLineUtils.doGenerateChangeLog(CommandLineUtils.java:279)
at liquibase.integration.commandline.Main.doMigration(Main.java:1058)
at liquibase.integration.commandline.Main.run(Main.java:199) at
liquibase.integration.commandline.Main.main(Main.java:137) Caused by:
liquibase.command.CommandExecutionException:
liquibase.exception.DatabaseException:
liquibase.exception.DatabaseException: Unknown table
'TMP_CTAWHBNCQVQMHSUU' [Failed SQL: DROP TABLE TMP_CTAWHBNCQVQMHSUU]
at liquibase.command.AbstractCommand.execute(AbstractCommand.java:24)
at
liquibase.integration.commandline.CommandLineUtils.doGenerateChangeLog(CommandLineUtils.java:277)
... 3 common frames omitted Caused by:
liquibase.exception.DatabaseException:
liquibase.exception.DatabaseException: Unknown table
'TMP_CTAWHBNCQVQMHSUU' [Failed SQL: DROP TABLE TMP_CTAWHBNCQVQMHSUU]
at
liquibase.snapshot.jvm.ForeignKeySnapshotGenerator.snapshotObject(ForeignKeySnapshotGenerator.java:223)
at
liquibase.snapshot.jvm.JdbcSnapshotGenerator.snapshot(JdbcSnapshotGenerator.java:66)
. . . at
liquibase.command.core.GenerateChangeLogCommand.run(GenerateChangeLogCommand.java:46)
at liquibase.command.AbstractCommand.execute(AbstractCommand.java:19)
... 4 common frames omitted Caused by:
liquibase.exception.DatabaseException: Unknown table
'TMP_CTAWHBNCQVQMHSUU' [Failed SQL: DROP TABLE TMP_CTAWHBNCQVQMHSUU]
at
liquibase.executor.jvm.JdbcExecutor$ExecuteStatementCallback.doInStatement(JdbcExecutor.java:356)
at liquibase.executor.jvm.JdbcExecutor.execute(JdbcExecutor.java:57)
at liquibase.executor.jvm.JdbcExecutor.execute(JdbcExecutor.java:125)
at liquibase.executor.jvm.JdbcExecutor.execute(JdbcExecutor.java:109)
at
liquibase.database.core.MySQLDatabase.hasBugJdbcConstraintsDeferrable(MySQLDatabase.java:294)
at
liquibase.snapshot.jvm.ForeignKeySnapshotGenerator.snapshotObject(ForeignKeySnapshotGenerator.java:188)
... 25 common frames omitted Caused by:
java.sql.SQLSyntaxErrorException: Unknown table 'TMP_CTAWHBNCQVQMHSUU'
at
com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120)
at
com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
at
com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
at
com.mysql.cj.jdbc.StatementImpl.executeInternal(StatementImpl.java:782)
at com.mysql.cj.jdbc.StatementImpl.execute(StatementImpl.java:666)
at
liquibase.executor.jvm.JdbcExecutor$ExecuteStatementCallback.doInStatement(JdbcExecutor.java:352)
... 30 common frames omitted
For more information, please use the --logLevel flag
Each time I run it, it changes the name of the phantom TMP_ tables (although they always begin with TMP_ and then a string of apparently random characters). I don't know where it's getting these non-existent TMP_ tables from that it wants to drop, but...is there some way to make it only attempt to drop them if they exist? Of potential note: it works fine if my only diffType is "tables", "data", or if I have both "tables" and "data" as diffTypes...otherwise, failure...

I don't have all the details, but it appears this can happen when using a database user with insufficient permissions. I talked with someone who said that using a root level database user fixed this issue for them.

Related

Can't update from a generated diff changelog

First of all, I'm using an npm library called liquibase to run Liquibase within Node.
So, I'm trying to use diffChangeLog to compare two databases and generate a changelog of their differences. The changelog is normally generated, but when I try to run an update using the generated changelog Liquibase responds with a validation error. Here's the output:
node_modules/liquibase/lib/liquibase-4.0.0/liquibase --changeLogFile=examples/common/migration.xml --url="jdbc:mysql://127.0.0.1:3306/mySchema?useJvmCharsetConverters=true" --username=migration --password=migration --classpath=lib/migrator/drivers/mysql-connector-java-8.0.22.jar --changeLogFile=examples/DatabaseDiff/diffChangeLog.xml update
Liquibase Community 4.0.0-beta1 by Datical
Starting Liquibase at 17:54:32 (version 4.0.0-beta1 #6 built at 2020-04-20 18:23+0000)
Unexpected error running Liquibase: Validation Failed:
16 changes have validation failures
columns is empty, examples/DatabaseDiff/diffChangeLog.xml::1618001652047-1::joao (generated)
columns is empty, examples/DatabaseDiff/diffChangeLog.xml::1618001652047-2::joao (generated)
columns is empty, examples/DatabaseDiff/diffChangeLog.xml::1618001652047-3::joao (generated)
columns is empty, examples/DatabaseDiff/diffChangeLog.xml::1618001652047-4::joao (generated)
columns is empty, examples/DatabaseDiff/diffChangeLog.xml::1618001652047-5::joao (generated)
columns is empty, examples/DatabaseDiff/diffChangeLog.xml::1618001652047-6::joao (generated)
columns is empty, examples/DatabaseDiff/diffChangeLog.xml::1618001652047-7::joao (generated)
columns is empty, examples/DatabaseDiff/diffChangeLog.xml::1618001652047-8::joao (generated)
columns is empty, examples/DatabaseDiff/diffChangeLog.xml::1618001652047-9::joao (generated)
columns is empty, examples/DatabaseDiff/diffChangeLog.xml::1618001652047-10::joao (generated)
columns is empty, examples/DatabaseDiff/diffChangeLog.xml::1618001652047-11::joao (generated)
columns is empty, examples/DatabaseDiff/diffChangeLog.xml::1618001652047-12::joao (generated)
columns is empty, examples/DatabaseDiff/diffChangeLog.xml::1618001652047-13::joao (generated)
columns is empty, examples/DatabaseDiff/diffChangeLog.xml::1618001652047-14::joao (generated)
columns is empty, examples/DatabaseDiff/diffChangeLog.xml::1618001652047-15::joao (generated)
columns is empty, examples/DatabaseDiff/diffChangeLog.xml::1618001652047-16::joao (generated)
For more information, please use the --logLevel flag
And here is part of the changelog that is mentioned in the output:
<changeSet author="joao (generated)" id="1618001652047-2">
<createTable tableName="FeaturedProducer"/>
</changeSet>
<changeSet author="joao (generated)" id="1618001652047-3">
<createTable tableName="PreVerification"/>
</changeSet>
<changeSet author="joao (generated)" id="1618001652047-4">
<createTable tableName="Producer"/>
</changeSet>
I also tried to use the SQL format for the generated changelog (named changeLog.mysql.sql), but it creates a bad SQL syntax, for example:
CREATE TABLE USER ();
This raises a syntax error from MySQL. Which is right, I've tried running this query manually and the error is the same.
My guess is that Liquibase is not allowing me to create empty columns, but why?
I don't understand what's going on and there are not many results on Google.
Can anyone give me a hand?!
Well, after a while we've found out the problem.
To debug the issue I raised two local databases with Docker and tried the same operation, it worked just fine.
So, it occurred to us that the reason we were getting this error could be due to user permissions. I've seen some people talking about it, that Liquibase can spit unusual errors when the user has insufficient permissions.
So I gave my user all permissions for that schema and it worked! Actually, we just went to another error, but it was definitely progress!
The error was just like this:
liquibase.exception.LiquibaseException: liquibase.command.CommandExecutionException: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.IndexOutOfBoundsException: Index 0 out of bounds for length 0
at liquibase.integration.commandline.CommandLineUtils.doDiffToChangeLog(CommandLineUtils.java:250)
at liquibase.integration.commandline.Main.doMigration(Main.java:1285)
at liquibase.integration.commandline.Main$1.lambda$run$0(Main.java:314)
at liquibase.Scope.lambda$child$0(Scope.java:149)
at liquibase.Scope.child(Scope.java:160)
at liquibase.Scope.child(Scope.java:148)
at liquibase.Scope.child(Scope.java:127)
at liquibase.Scope.child(Scope.java:173)
at liquibase.Scope.child(Scope.java:177)
at liquibase.integration.commandline.Main$1.run(Main.java:313)
at liquibase.integration.commandline.Main$1.run(Main.java:169)
at liquibase.Scope.child(Scope.java:160)
at liquibase.Scope.child(Scope.java:134)
at liquibase.integration.commandline.Main.run(Main.java:169)
at liquibase.integration.commandline.Main.main(Main.java:148)
Caused by: liquibase.command.CommandExecutionException: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.IndexOutOfBoundsException: Index 0 out of bounds for length 0
at liquibase.command.AbstractCommand.execute(AbstractCommand.java:24)
at liquibase.integration.commandline.CommandLineUtils.doDiffToChangeLog(CommandLineUtils.java:248)
... 14 more
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: java.lang.IndexOutOfBoundsException: Index 0 out of bounds for length 0
at liquibase.diff.output.changelog.DiffToChangeLog.print(DiffToChangeLog.java:193)
at liquibase.diff.output.changelog.DiffToChangeLog.print(DiffToChangeLog.java:86)
at liquibase.command.core.DiffToChangeLogCommand.run(DiffToChangeLogCommand.java:63)
at liquibase.command.AbstractCommand.execute(AbstractCommand.java:19)
... 15 more
Caused by: java.lang.RuntimeException: java.lang.IndexOutOfBoundsException: Index 0 out of bounds for length 0
at liquibase.diff.output.changelog.DiffToChangeLog$1.run(DiffToChangeLog.java:176)
at liquibase.Scope.lambda$child$0(Scope.java:149)
at liquibase.Scope.child(Scope.java:160)
at liquibase.Scope.child(Scope.java:148)
at liquibase.Scope.child(Scope.java:127)
at liquibase.diff.output.changelog.DiffToChangeLog.print(DiffToChangeLog.java:119)
... 18 more
Caused by: java.lang.IndexOutOfBoundsException: Index 0 out of bounds for length 0
at java.base/jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64)
at java.base/jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70)
at java.base/jdk.internal.util.Preconditions.checkIndex(Preconditions.java:248)
at java.base/java.util.Objects.checkIndex(Objects.java:372)
at java.base/java.util.ArrayList.get(ArrayList.java:459)
at liquibase.change.ColumnConfig.<init>(ColumnConfig.java:154)
at liquibase.change.AddColumnConfig.<init>(AddColumnConfig.java:16)
at liquibase.diff.output.changelog.core.MissingIndexChangeGenerator.fixMissing(MissingIndexChangeGenerator.java:70)
at liquibase.diff.output.changelog.ChangeGeneratorChain.fixMissing(ChangeGeneratorChain.java:48)
at liquibase.diff.output.changelog.ChangeGeneratorFactory.fixMissing(ChangeGeneratorFactory.java:95)
at liquibase.diff.output.changelog.DiffToChangeLog.generateChangeSets(DiffToChangeLog.java:279)
at liquibase.diff.output.changelog.DiffToChangeLog.printNew(DiffToChangeLog.java:203)
at liquibase.diff.output.changelog.DiffToChangeLog$1.run(DiffToChangeLog.java:125)
... 23 more
Really bad logging btw...
This MissingIndexChangeGenerator.fixMissing(MissingIndexChangeGenerator.java:70) part caught my attention. Surfing through the web I found barely anything, so we thought: Let's check the source code for that line. On their GitHub, we've found the exact line of the exception and everything became clearer.
That error meant that Liquibase failed to retrieve some table indexes.
Taking a look at our database we were reminded that some indexes were actually foreign keys to tables on another schema. And the user we were using didn't have permission on that other schema. We gave the user all permissions for that second schema too, and BAM, it finally worked!
The lesson here is: Liquibase has really bad logging... Oh, and also, never give up!

How to truncate table that my user owns in Oracle without error message

Thanks for reading this post. I am running into a strange issue in Oracle 10g. I am trying to figure out how to truncate a table without errors being generated.
What I need to do:
truncate a table that my user owns
truncate table EXAMPLE_USER.EXAMPLE_TABLE;
What is happening: rows within the table are successfully removed, but the following error is generated:
Error starting at line 1 in command:
truncate table EXAMPLE_USER.EXAMPLE_TABLE
Error report:
SQL Error: ORA-00604: error occurred at recursive SQL level 1
ORA-00942: table or view does not exist
ORA-06512: at line 50
00604. 00000 - "error occurred at recursive SQL level %s"
*Cause: An error occurred while processing a recursive SQL statement
(a statement applying to internal dictionary tables).
*Action: If the situation described in the next error on the stack
can be corrected, do so; otherwise contact Oracle Support.
What I have already done:
Ensure that EXAMPLE_USER has the "drop any table" privilege
Make sure that the rows are removed after truncate statement is ran
Additional Info:
I cannot simply drop and re-create the table because the truncation happens as part of a large database upgrade/update script which is out of my control.
The error is leading me to believe that the truncate statement within Oracle uses some sort of sys package or procedure that my user does not have privilege to access or use, based on the "at line 50" portion of the message. The actual truncate statement itself is a single line script. Does anyone have experience with this issue in Oracle?
Regards,
Dave

ora-01722 Invalid number liquibase changeSet (in Toad script works)

I'm trying to execute an update to database using liquibase. The problem is that I have some scripts that creates materialized views and I get the error:
liquibase.exception.MigrationFailedException: Migration failed for
change set
changelog/20160323192000_mviews.xml::20160323192000#1::marius: Reason:
liquibase.exception.DatabaseException: ORA-01722: invalid number
When I execute them in Toad for Oracle 11, the mviews are created without problems.
<changeSet id="20160323192000#1" author="marius">
<sql splitStatements="false"><![CDATA[
CREATE MATERIALIZED VIEW Activities_Others_M AS SELECT * FROM Activities;
]]></sql>
</changeSet>
I also tried to create table in liquibase changeSet and also is not working. Only a normal view is working.

Delete data from table that is not reorganized (in db2)

I have a table that needs to be reorganised using the reorg command in DB#, However due to large size of the table, I am unable to do so. So I plan to delete entire data from the table and then do any reorg if required..
Is there any possible way for doing so?
I already used
DELETE FROM TABLE-NAME
and
TRUNCATE TABLE SCHAME-NAME.TABLE-NAME IMMEDIATE
But both the above queries aren't working and are giving the following error which is the one for reorg.
Error: DB2 SQL Error: SQLCODE=-668, SQLSTATE=57016, SQLERRMC=7;DB2ADMIN.CERTIFICATE_TAB, DRIVER=3.50.152
SQLState: 57016
ErrorCode: -668

modifyDataType in Liquibase against SQL Server

I am trying to modify the data type on a number of columns from int to bigint using:
<modifyDataType tableName="ACCESS_HISTORY" columnName="ID" newDataType="${LongType}"/>
Where I have defined LongType to be:
<property name="LongType" value="bigint" dbms="mssql"/>
I noticed in the Liquibase JIRA (https://liquibase.jira.com/browse/CORE-1062) that there is a known issue from 2011 relating to the PKs and FKs that need to be dropped. Dropping and re-creating the keys isn't really an option for me.
The error I got was:
liquibase.exception.MigrationFailedExc
eption: Migration failed for change set ChangeColumnTypes.xml::4-4-060
-2::thof:
Reason: liquibase.exception.DatabaseException: Error executing SQL ALTER TA
BLE [dbo].[ACCESS_HISTORY] ALTER COLUMN [ID] BIGINT: The object 'PK_Access_Histo
ry' is dependent on column 'ID'.:
Caused By: Error executing SQL ALTER TABLE [dbo].[ACCESS_HISTORY] ALTE
R COLUMN [ID] BIGINT: The object 'PK_Access_History' is dependent on column 'ID'
I was wondering if anyone had got around this? Thanks in advance
I'm afraid to say that there is no way to currently do this unless you drop the PK constraint. It looks like the bug you're talking about though is scheduled to be fixed in version 3 of liquibase, however there is no release date for this version.
Sorry!
As Liquibase is open source you could always fix the bug yourself. :)