custom properties in liquibase.properties file - liquibase

I am using liquibase 2.0.5 and my properties file looks like below
#liquibase.properties
url=jdbc:oracle:thin:#//localhost:1521/orcl
username=myschama
password=myschama
masterUsername=system
masterPassword=system
I want to use system user to create myschema user. Currently, I get the following error.
Unexpected error running Liquibase: Unknown parameter: 'masterUsername'
SEVERE 12/21/17 12:59 PM:liquibase: Unexpected error running Liquibase: Unknown parameter: 'masterUsername'
liquibase.exception.CommandLineParsingException: Unknown parameter: 'masterUsername'
at liquibase.integration.commandline.Main.parsePropertiesFile(Main.java:387)
at liquibase.integration.commandline.Main.main(Main.java:122)
Is it possible to have such custom values in the properties file

If you want to use it in your changeSets as property, e.g.
<changeSet ...>
<sql>
INSERT INTO myTab(some_column) VALUES ('${masterUsername}')
</sql>
</changeSet>
then try putting this into your liquibase.properties:
parameter.masterUsername=your_desired_value
As an alternative, you can pass these on the command line as Java properties, e.g.
liquibase update ... -DmasterUsername=your_desired_value
See http://www.liquibase.org/documentation/changelog_parameters.html and http://www.liquibase.org/documentation/command_line.html for details.

Related

How to specify the searchPath when using liquibase commandline liquibase.integration.commandline.Main

(Using liquibase 4.18.0 and also tried 4.19.0)
I want to add two additional parameters to my (working) liquibase call
--hub-mode=off
--searchPath="some/resources"
Working:
java liquibase.integration.commandline.Main --logLevel=info --defaultsFile=project.properties update
Not working:
java liquibase.integration.commandline.Main --logLevel=info --searchPath="some/resources" --defaultsFile=project.properties update
I always get:
Unknown option 'searchPath'
If I remove this option I get the same for hub-mode. If I remove both the resource could not be found and liquibase tells me
"More locations can be added with the 'searchPath' parameter."
I checked the declaredFields variable and there are the following options defined and the two are missing:
runningFromNewCli
newCliChangelogParameters
outputStream
LOG
coreBundle
classLoader
driver
username
password
url
hubConnectionId
hubProjectId
hubProjectName
databaseClass
defaultSchemaName
outputDefaultSchema
outputDefaultCatalog
liquibaseCatalogName
liquibaseSchemaName
databaseChangeLogTableName
databaseChangeLogLockTableName
databaseChangeLogTablespaceName
defaultCatalogName
changeLogFile
overwriteOutputFile
classpath
contexts
labels
labelFilter
driverPropertiesFile
propertyProviderClass
changeExecListenerClass
changeExecListenerPropertiesFile
promptForNonLocalDatabase
includeSystemClasspath
defaultsFile
diffTypes
changeSetAuthor
changeSetContext
dataOutputDirectory
referenceDriver
referenceUrl
referenceUsername
referencePassword
referenceDefaultCatalogName
referenceDefaultSchemaName
currentDateTimeFunction
command
commandParams
logLevel
logFile
changeLogParameters
outputFile
excludeObjects
includeCatalog
includeObjects
includeSchema
includeTablespace
deactivate
outputSchemasAs
referenceSchemas
schemas
snapshotFormat
liquibaseProLicenseKey
liquibaseProLicenseValid
liquibaseHubApiKey
liquibaseHubUrl
managingLogConfig
outputsLogMessages
sqlFile
delimiter
rollbackScript
rollbackOnError
suspiciousCodePoints
Any idea how to specify the searchpath for the commandline executable?
I did read this post but the solution did not help.

Liquibase error when Comment at the end of sql file

When add any comments to the end of sql file liquibase raises an error.
Everything is ok in any other places.
What are we doing wrong?
version 4.1.1 #10 built at 2020-10-12 19:24+0000 Starting Liquibase at 09:43:04 (version 4.1.1 #10 built at 2020-10-12 19:24+0000) Unexpected error running Liquibase: Migration failed for change set 1000.file_name.sql::1::10000: Reason: liquibase.exception.DatabaseException: Invalid SQL type: sqlKind = UNINITIALIZED [Failed SQL: (17439) – problem comment] For more information, please use the --logLevel flag
--liquibase formatted sql
--changeset 1:10000 stripComments:false runOnChange:true endDelimiter:;
alter table SCHEMA_NAME.SOME_TABLE modify
"ANY_COLOMN varchar2(60 char)
;
commit
;
-- problem comment
UPD
The command I'm running:
liquibase.bat --driver=oracle.jdbc.OracleDriver --username=yyyy --password=xxxx --url=jdbc:oracle:thin:#(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=some_host.com)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=some_servicename))) --changeLogFile=db-changelog-test.yml update
db-changelog-test.yml
---
databaseChangeLog:
- include:
file: 1000.file_name.sql
relativeToChangelogFile: true
liquibase.bat: modified only one string
IF NOT DEFINED JAVA_OPTS set JAVA_OPTS=-Dfile.encoding=UTF-8

liquabase command generateChangeLog generate java.lang.StackOverflowError

I would like to generate csv files and loaddata changeset for some tables.
I use this command line:
$LB_HOME/liquibase --logLevel=DEBUG --changeLogFile=${TABLE}.xml \
--url=jdbc:oracle:thin:#local:1521/ORCL --username=TEST --password=TEST \
--dataOutputDirectory=csv --diffTypes=data \
--includeObjects="table:$TABLE" generateChangeLog
After a very long list of lines like this:
DEBUG [liquibase.util.DependencyUtil$DependencyGraph]:
Potential StackOverflowException. Pro-actively removing with incoming nodes
I get this error:
ERROR [liquibase.integration.commandline.Main]: Unexpected error running Liquibase: Unknown reason
java.lang.StackOverflowError: null
I put includeObjects="table:$TABLE" with only one table, why liquibase reads all object dependencies?
Any suggestion?
As per Liquibase Documentation, includeObjects is not valid parameter
here is the Link: https://docs.liquibase.com/commands/community/generatechangelog.html
Can you try running just generateChangeLog command without data and see if it works first?

Apache Pig 0.12.0 on Hue not preprocessing statements as expected

I'm using Hue for PIG scripts on amazon EMR. I am using the declare and default statements as mentioned in the documentation.
I have some %default and %declare statements and it looks like they are
not preprocessed within Hue. Therefore, although the parameters are defined
in my script, the editor keeps popping in a parameter configuration window. If I leave the parameter blank, the job fails with an error.
Sample Script
%declare OUTPUT_FOLDER 'testingOutput01';
ts = LOAD 's3://testbucket1/input/testdata-00000.gz' USING PigStorage('\t');
STORE ts INTO 's3://testbucket1/$OUTPUT_FOLDER' USING PigStorage('\t');
Upon execution, it shows the pop-up window asking for values for OUTPUT_FOLDER. If I leave it blank it fails with the following error:
2015-06-23 20:15:54,908 [main] ERROR org.apache.pig.Main - ERROR 2997:
Encountered IOException. org.apache.pig.tools.parameters.ParseException:
Encountered "<EOF>" at line 1, column 12.
Was expecting one of:
<IDENTIFIER> ...
<OTHER> ...
<LITERAL> ...
<SHELLCMD> ...
Is that the expected behavior? Is this a known issue or am I missing something?
Configuration details:
AMI version:3.7.0
Hadoop distribution:Amazon 2.4.0
Applications:Hive 0.13.1, Pig 0.12.0, Impala 1.2.4, Hue
The same behavior is seen with default instead of declare.
If you need any clarifications then please do comment on this question. I will update it as needed.
Hue does not support %declare with a default statement. It will be fixed with: https://issues.cloudera.org/browse/HUE-2508
The current temporary workaround is to put any value in the popup.

Not able to generate liquibase changelog

I am trying to execute command on my command prompt:
liquibase --driver=com.mysql.jdbc.Driver --classpath=E:\mysqljar\mysql.jar --changeLogFile=E:\1.xml --url="jdbc:mysql://localhost:3306/abc" --username=root --password=root generateChangeLog
But I am getting this error:
Liquibase Update Failed: Empty result set, expected one row
SEVERE 24/9/13 6:29 PM:liquibase: Empty result set, expected one row
liquibase.exception.DatabaseException: Error getting jdbc:mysql://localhost:3306/abc view with liquibase.statement.core.GetViewDefinitionStatement#53330681
at liquibase.snapshot.jvm.JdbcDatabaseSnapshotGenerator.readView(JdbcDatabaseSnapshotGenerator.java:168)
at liquibase.snapshot.jvm.JdbcDatabaseSnapshotGenerator.readViews(JdbcDatabaseSnapshotGenerator.java:304)
at liquibase.snapshot.jvm.JdbcDatabaseSnapshotGenerator.createSnapshot(JdbcDatabaseSnapshotGenerator.java:241)
at liquibase.snapshot.DatabaseSnapshotGeneratorFactory.createSnapshot(DatabaseSnapshotGeneratorFactory.java:69)
at liquibase.diff.Diff.compare(Diff.java:63)
at liquibase.integration.commandline.CommandLineUtils.doGenerateChangeLog(CommandLineUtils.java:145)
at liquibase.integration.commandline.Main.doMigration(Main.java:760)
at liquibase.integration.commandline.Main.main(Main.java:134)
Caused by: liquibase.exception.DatabaseException: Empty result set, expected one row
at liquibase.util.JdbcUtils.requiredSingleResult(JdbcUtils.java:124)
at liquibase.executor.jvm.JdbcExecutor.queryForObject(JdbcExecutor.java:159)
at liquibase.executor.jvm.JdbcExecutor.queryForObject(JdbcExecutor.java:167)
at liquibase.executor.jvm.JdbcExecutor.queryForObject(JdbcExecutor.java:163)
at liquibase.database.AbstractDatabase.getViewDefinition(AbstractDatabase.java:748)
at liquibase.snapshot.jvm.JdbcDatabaseSnapshotGenerator.readView(JdbcDatabaseSnapshotGenerator.java:166)
... 7 more
Could anyone help me to interpret this?
I had the same problem.
My fix:
There needs to be an entry in the database change log lock table; it needs to have id=1, locked=false and the rest of the values set to null.
I'm getting the same error also using MySQL and tried v3.0.0 and 3.0.5 of liquibase. The error was the same when I tried to do a migrate as well as a generateChangeLog.
./liquibase --logLevel=debug --changeLogFile=./db.changelog-test-v0.1.xml --username=abc --password=abc99 --url="jdbc:mysql://localhost:3306/test" migrate
FYI, here is the select statement that it had trouble executing for the 'migrate' command:
select view_definition from information_schema.views where table_name='patient_info' and table_schema='test'
The information_schema.views table is empty.