Unexpected error running Liquibase: Unknown reason Throwing NPE - nullpointerexception

Been using Liquibase a long time, and love it. However, on the project I'm currently working on, I'm encountering an issue I don't seem to be able to find the cause.
I can run every changelog/feature/*.xml file from the command line (using update) and everything succeeds.
I can run every db.changelog-*.xml file (except master) from the command line (using update) and everything succeeds.
But when I try and run the db.changelog-master file, I get:
Starting Liquibase at Tue, 14 Apr 2020 11:49:47 CDT (version 3.8.0 built at 2019-08-15T20:38:06Z)
Liquibase Community 3.8.0 by Datical
Unexpected error running Liquibase: Unknown reason
liquibase.exception.ChangeLogParseException: liquibase.exception.SetupException
at liquibase.parser.core.xml.AbstractChangeLogParser.parse(AbstractChangeLogParser.java:25) ~[liquibase.jar:na]
at liquibase.Liquibase.getDatabaseChangeLog(Liquibase.java:217) ~[liquibase.jar:na]
at liquibase.Liquibase.update(Liquibase.java:190) ~[liquibase.jar:na]
at liquibase.Liquibase.update(Liquibase.java:179) ~[liquibase.jar:na]
at liquibase.integration.commandline.Main.doMigration(Main.java:1223) [liquibase.jar:na]
at liquibase.integration.commandline.Main.run(Main.java:209) [liquibase.jar:na]
at liquibase.integration.commandline.Main.main(Main.java:132) [liquibase.jar:na]
Caused by: liquibase.exception.SetupException: null
at liquibase.changelog.DatabaseChangeLog.includeAll(DatabaseChangeLog.java:531) ~[liquibase.jar:na]
at liquibase.changelog.DatabaseChangeLog.handleChildNode(DatabaseChangeLog.java:398) ~[liquibase.jar:na]
at liquibase.changelog.DatabaseChangeLog.load(DatabaseChangeLog.java:308) ~[liquibase.jar:na]
at liquibase.parser.core.xml.AbstractChangeLogParser.parse(AbstractChangeLogParser.java:23) ~[liquibase.jar:na]
... 6 common frames omitted
Caused by: java.lang.NullPointerException: null
at java.util.AbstractCollection.addAll(AbstractCollection.java:343) ~[na:1.8.0_242]
at liquibase.integration.commandline.CommandLineResourceAccessor.list(CommandLineResourceAccessor.java:44) ~[liquibase.jar:na]
at liquibase.resource.CompositeResourceAccessor.list(CompositeResourceAccessor.java:40) ~[liquibase.jar:na]
at liquibase.changelog.DatabaseChangeLog.includeAll(DatabaseChangeLog.java:506) ~[liquibase.jar:na]
... 9 common frames omitted
For more information, please use the --logLevel flag
I did find an old view and trigger that were created during research for the project for a different schema, which were never run in this one (neither was in this DATABASECHANGELOG). They've been deleted, but the problem still exists.
Running:
macOS Catalina 10.15.3
openjdk version "1.8.0_242"
OpenJDK Runtime Environment (Zulu 8.44.0.11-CA-macosx) (build
1.8.0_242-b20)
OpenJDK 64-Bit Server VM (Zulu 8.44.0.11-CA-macosx) (build
25.242-b20, mixed mode)
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0
64bit Production PL/SQL Release 11.2.0.2.0 Production
CORE 11.2.0.2.0 Production
TNS for Linux: Version 11.2.0.2.0 Production
NLSRTL Version 11.2.0.2.0 Production
Liquibase 3.8.0
Tried everything I can think of. Anyone have any thoughts?
**UPDATE APR-29-2020**
I have a little more information - I've discovered WHAT is happening, and the plan to fix this isn't working.
The what:
Apparently Liqiubase changed the includeAll path somewhere between version 3.4.2 and 3.5.5:
See 'Upgrade from 3.4.2 to 3.5.5 or higher'
I'd copied our liquibase setup from a legacy system running 3.4.2. In the db.changelog-master.xml file, it has <includeAll path="/features"/> (leading slash). However, on the new project, I upgraded to 3.8.0. And the includeAll is what's throwing the exception.
The new value is <includeAll path="features/"/> (trailing slash). The format for the FILENAME column in DATABASECHANGELOG is the same, i.e. features/20191024-LD-1.xml
In the documentation, there's a command line maintenance command that looks perfectly designed for this fix: cleanCheckSums. Per the website:
Removes current checksums from database. On next update changesets
that have already been deployed will have their checksums recomputed,
and changesets that have not been deployed will be deployed.
Challenge is that yes, cleanCheckSums nulls the MD5SUM column, the EXECTYPE column is EXECUTED, and each row has a populated DEPLOYMENT_ID, but the next update command tries to re-run all of the changesets.
Here's the plan we created for the fix:
Steps
Change the includeAll parameter path in db.changelog-master.xml: <includeAll path="features/"/>
Run liquidate clearCheckSums:
$LIQUIBASE_HOME/liquibase \ --driver=oracle.jdbc.OracleDriver \ --classpath=/data/apps/liquibase-3.8.0/ojdbc7.jar \ --url={pathToDatabase} \ --username={dbUserName} \ --password={dbUserPass} \ clearCheckSums
SQL to fix the database column filename to the expected format
Appears to be exactly the same format:
old: features/20191024-LD-1.xml
new test: features/20200429-deleteme.xml
Run master update:
$LIQUIBASE_HOME/liquibase \ --driver=oracle.jdbc.OracleDriver \ --classpath=/data/apps/liquibase-3.8.0/ojdbc7.jar \ --url={pathToDatabase} \ --changeLogFile=db.changelog-master.xml \ --username={dbUserName} \ --password={dbUserPass} \ update
Anyone know what I'm missing?

Please try the following command:
liquibase changelogSync
This command will create corresponding rows to the db.changelog-master.xml changeSets in the DATABASECHANGELOG tracking table and consider those changeSets as deployed.

Related

Liquibase CLI cannot execute custom changes written in Kotlin

We write our custom changes in Kotlin. But we cannot execute these custom changes using the Liquibase CLI, because the kotlin-stdlib is missing from the classpath.
Here's my liquibase.properties file:
changelog-file:db/changelog.yaml
driver:org.postgresql.Driver
url:jdbc:postgresql://localhost:5432/project
username:postgres
password:postgres
classpath:build/classes/kotlin/main:src/main/resources
includeSystemClasspath:true
In src/main/resources we keep the changelog.yaml and the changesets, while in build/classes/kotlin/main we have the custom changes written in Kotlin.
Running liquibase update --log-level=debug return the following output:
[...]
Caused by: java.lang.ClassNotFoundException: kotlin.jvm.internal.Intrinsics
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
... 70 more
Caused by: liquibase.exception.ValidationFailedException: Validation Failed:
2 changes have validation errors
liquibase.exception.UnexpectedLiquibaseException: liquibase.exception.CustomChangeException: liquibase.exception.UnexpectedLiquibaseException: java.lang.reflect.InvocationTargetException
liquibase.exception.UnexpectedLiquibaseException: liquibase.exception.CustomChangeException: liquibase.exception.UnexpectedLiquibaseException: java.lang.reflect.InvocationTargetException
If I hard code the path to the kotlin-stdlib in the liquibase.properties file, then gradle update works fine.
classpath:build/classes/kotlin/main:src/main/resources:/Home/user/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib/1.6.21/11ef67f1900634fd951bad28c53ec957fabbe5b8/kotlin-stdlib-1.6.21.jar
But we cannot hard code it in liquibase.properties as the path to the kotlin-stdlib would be different for each machine. I was expecting that exporting it in the system CLASSPATH would solve this issue:
export CLASSPATH=/Home/user/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib/1.6.21
This doesn't seem to work, even when we specified includeSystemClasspath:true in the liquibase.properties file.
Using the LIQUIBASE_CLASSPATH env variable requires a Liquibase Pro or Labs license.
Liquibase Community detected and ignored the following environment variables:
- LIQUIBASE_CLASSPATH
To configure Liquibase with environment variables requires a Liquibase Pro or Liquibase Labs license. Get a free trial at https://liquibase.com/trial. Options include the liquibase.licenseKey in the defaults file, adding a flag in the CLI, and more. Learn more at https://docs.liquibase.com.

IntelliJ fails to install on Oracle Linux

OS: Oracle Linux Server release 7.8
Steps to reproduce
i) I downloaded the community edition tarball from https://www.jetbrains.com/idea/download/#section=linux - ideaIC-2022.2.2.tar.gz
ii) Extracted the tarball
on running ./idea.sh I ran into
CompileCommand: exclude com/intellij/openapi/vfs/impl/FilePartNodeRoot.trieDescend bool exclude = true
Start Failed
Internal error. Please refer to https://jb.gg/ide/critical-startup-errors
java.io.IOException: No locks available
at java.base/sun.nio.ch.FileDispatcherImpl.lock0(Native Method)
at java.base/sun.nio.ch.FileDispatcherImpl.lock(FileDispatcherImpl.java:96)
at java.base/sun.nio.ch.FileChannelImpl.lock(FileChannelImpl.java:1286)
at java.base/java.nio.channels.FileChannel.lock(FileChannel.java:1089)
at com.intellij.idea.SocketLock.lockPortFiles(SocketLock.java:205)
at com.intellij.idea.SocketLock.lockAndTryActivate(SocketLock.java:131)
at com.intellij.idea.StartupUtil.lockSystemDirs(StartupUtil.java:781)
at com.intellij.idea.StartupUtil.start(StartupUtil.java:169)
at com.intellij.idea.Main.bootstrap(Main.java:119)
at com.intellij.idea.Main.main(Main.java:81)
-----
Your JRE: 17.0.4+7-b469.53 amd64 (JetBrains s.r.o.)
None of the resolution steps mentioned in the https://intellij-support.jetbrains.com/hc/en-us/articles/360007568559 worked for me.
Answer: Well, after I cleared some space in my /scratch, it worked. I did not dig further what the issue was.

liquibase generateChangeLog Connection could not be created to jdbc:postgres

I'm trying to run liquibase to generate a changelog of my current database, but I get the following error and not sure how to get around it.
Unexpected error running Liquibase: liquibase.exception.DatabaseException: Connection could not be created to jdbc:postgres://kbjoxiifqvunac:ef1f6ba0b9633d5951c6f4a28897fe390fdf2a7f4189320569d303531899e843#ec2-54-217-235-159.eu-west-1.compute.amazonaws.com:5432/d1pdhars1kb6ep with driver org.postgresql.Driver. Possibly the wrong driver for the given database URL
liquibase.exception.DatabaseException: liquibase.exception.DatabaseException: liquibase.exception.DatabaseException: Connection could not be created to jdbc:postgres://kbjoxiifqvunac:ef1f6ba0b9633d5951c6f4a28897fe390fdf2a7f4189320569d303531899e843#ec2-54-217-235-159.eu-west-1.compute.amazonaws.com:5432/d1pdhars1kb6ep with driver org.postgresql.Driver. Possibly the wrong driver for the given database URL
at liquibase.integration.commandline.CommandLineUtils.createDatabaseObject(CommandLineUtils.java:135) ~[liquibase.jar:na]
at liquibase.integration.commandline.Main.doMigration(Main.java:977) [liquibase.jar:na]
at liquibase.integration.commandline.Main.run(Main.java:209) [liquibase.jar:na]
at liquibase.integration.commandline.Main.main(Main.java:132) [liquibase.jar:na]
Caused by: liquibase.exception.DatabaseException: liquibase.exception.DatabaseException: Connection could not be created to jdbc:postgres://kbjoxiifqvunac:ef1f6ba0b9633d5951c6f4a28897fe390fdf2a7f4189320569d303531899e843#ec2-54-217-235-159.eu-west-1.compute.amazonaws.com:5432/d1pdhars1kb6ep with driver org.postgresql.Driver. Possibly the wrong driver for the given database URL
at liquibase.database.DatabaseFactory.openConnection(DatabaseFactory.java:259) ~[liquibase.jar:na]
at liquibase.database.DatabaseFactory.openDatabase(DatabaseFactory.java:149) ~[liquibase.jar:na]
at liquibase.integration.commandline.CommandLineUtils.createDatabaseObject(CommandLineUtils.java:100) ~[liquibase.jar:na]
... 3 common frames omitted
Caused by: liquibase.exception.DatabaseException: Connection could not be created to jdbc:postgres://kbjoxiifqvunac:ef1f6ba0b9633d5951c6f4a28897fe390fdf2a7f4189320569d303531899e843#ec2-54-217-235-159.eu-west-1.compute.amazonaws.com:5432/d1pdhars1kb6ep with driver org.postgresql.Driver. Possibly the wrong driver for the given database URL
at liquibase.database.DatabaseFactory.openConnection(DatabaseFactory.java:254) ~[liquibase.jar:na]
... 5 common frames omitted
I have run the following command
liquibase --driver=org.postgresql.Driver --changeLogFile=changelog.xml --url=jdbc:postgres://user:password#ec2-54-217-235-159.eu-west-1.compute.amazonaws.com:5432/dbhost generateChangeLog
I have omitted the classpath flag since I put the postgres drivers inside the lib folder of my liquibase, and per documentation it should scan it.
I'm running on windows, liquibase version 3.8.0 and postgres-drivers 42.1.1. Any ideas? Thanks in advance for any help
I think your driver jar should be in the lib directory rather than the bin directory. If you look at these lines from liquibase.bat you can see that it adds all jar files in the bin directory that start with liquibase, and then ALL jar files that are in the lib directory.
set CP=.
for /R %LIQUIBASE_HOME% %%f in (liquibase*.jar) do set CP=!CP!;%%f
for /R %LIQUIBASE_HOME%\lib %%f in (*.jar) do set CP=!CP!;%%f

Liquibase: Error java.lang.UnsupportedClassVersionError: JVMCFRE003 bad major version

I use liquibase 3.6.1 to work with Mysql database. I use the lib "mysql-connector-java-6.0.2.jar", I got the below error when using diff command:
Exception in thread "main" java.lang.UnsupportedClassVersionError:
JVMCFRE003 bad major version; class=com/mysql/cj/jdbc/Driver, offset=6
When I tried the lib "mysql-connector-java-5.1.21.jar", I got another error:
Unexpected error running Liquibase: java.lang.RuntimeException: Cannot
find data base driver: com.mysql.cj.jdbc.Driver
My command is:
liquibase --referenceUrl=jdbc:mysql://10.53.x.x/mydb
--referenceUsername=user --referencePassword=pass
Are there anyone can help ?
I guess that you are using old version of java. Liquibase 3.6.1 requires java version 1.7. Look in this issue for more details

flyway 3.2 Unable to scan for SQL migrations in filesystem location

I am executing the oracle 11g sql scripts using flyway 3.2 from command line.
flyway -X -locations=filesystem:/flyway/ProjectA/sql -configFile=flyway.conf.ProjectA migrate
ERROR: Unexpected error org.flywaydb.core.api.FlywayException: Unable
to scan for SQL migrations in location:
filesystem:/flyway/ProjectA/sql
Caused by: org.flywaydb.core.api.FlywayException: Invalid filesystem
path: /flyway/ProjectA/sql
Any thoughts or work around to override the default /sql folder (similar to configuration files) using command line tool.