Unexpected error running Liquibase: Unexpected value [...] (options must start with a '--') && howTo? diff w/ properties file - liquibase

I'm trying to diff two databases from the command line in linux. I'm hoping that someone can tell me what I'm doing wrong... my command is
./liquibase
--driver=com.mysql.cj.jdbc.Driver \
--url=jdbc:mysql://{it'svalid}:3306/db_schema_1?useLegacyDateTimeCcxle=false&serverTimezone=UTC \
--username={it'svalid} \
--password={it'svalid} \
diff \
--referenceUrl=jdbc:mysql://{it'svalid}:3306/db_schema_2?useLegacyDateTimeCcxle=false&serverTimezone=UTC \
--referenceUsername={it'svalid}
--referencePassword={it'svalid} \
--changeLogFile=databaseChangeLog.sql
I'm getting this in response:
java -cp
".:/home/someguy/needit/liquibase/liquibase.jar:/home/someguy/needit/liquibase/lib/:/home/someguy/needit/liquibase/lib/jcl-over-slf4j-1.7.25.jar:/home/someguy/needit/liquibase/lib/jul-to-slf4j-1.7.25.jar:/home/someguy/needit/liquibase/lib/log4j-over-slf4j-1.7.25.jar:/home/someguy/needit/liquibase/lib/logback-1.1.jar:/home/someguy/needit/liquibase/lib/logback-classic-1.2.3.jar:/home/tkrueger/needit/liquibase/lib/logback-core-1.2.3.jar:/home/someguy/needit/liquibase/lib/mysql-connector-java-8.0.15.jar:/home/someguy/needit/liquibase/lib/slf4j-api-1.7.25.jar:/home/someguy/needit/liquibase/lib/snakeyaml-1.18.jar" liquibase.integration.commandline.Main
"--driver=com.mysql.cj.jdbc.Driver
--url=jdbc:mysql://{it'svalid}:3306/db_schema_1 --username={it'svalid} --password={it'svalid} diff --referenceUrl=jdbc:mysql://{it'svalid}:3306/db_schema_2 --referenceUsername={it'svalid} --referencePassword={it'svalid} --changeLogFile=databaseChangeLog.sql" Starting Liquibase at Thu, 02 May 2019 14:21:56 CDT (version 3.6.3 built at 2019-01-29 11:34:48) To
display the help, please pass the '--help' option on the command line.
Unexpected error running Liquibase: Unexpected value
'"--driver=com.mysql.cj.jdbc.Driver' (options must start with a '--')
liquibase.exception.CommandLineParsingException: Unexpected value
'"--driver=com.mysql.cj.jdbc.Driver' (options must start with a '--')
at liquibase.integration.commandline.Main.parseOptions(Main.java:778)
at liquibase.integration.commandline.Main.run(Main.java:185)
at liquibase.integration.commandline.Main.main(Main.java:137)
I've dumped most of the parameters into a liquibase.properties file, to no avail... I thought I found someone else dealing with the leading doublequote on the rejected parameter string (i.e., it appears to be parsing the parameters as "-- instead of simply --), but I'm struggling to understand how to get around that while doing a diff, by putting all my parameters into a liquibase.properties file.

You should check what end of line you have. I had same problem. Resolution was to change from windows eol to linux.

Related

liquabase command generateChangeLog generate java.lang.StackOverflowError

I would like to generate csv files and loaddata changeset for some tables.
I use this command line:
$LB_HOME/liquibase --logLevel=DEBUG --changeLogFile=${TABLE}.xml \
--url=jdbc:oracle:thin:#local:1521/ORCL --username=TEST --password=TEST \
--dataOutputDirectory=csv --diffTypes=data \
--includeObjects="table:$TABLE" generateChangeLog
After a very long list of lines like this:
DEBUG [liquibase.util.DependencyUtil$DependencyGraph]:
Potential StackOverflowException. Pro-actively removing with incoming nodes
I get this error:
ERROR [liquibase.integration.commandline.Main]: Unexpected error running Liquibase: Unknown reason
java.lang.StackOverflowError: null
I put includeObjects="table:$TABLE" with only one table, why liquibase reads all object dependencies?
Any suggestion?
As per Liquibase Documentation, includeObjects is not valid parameter
here is the Link: https://docs.liquibase.com/commands/community/generatechangelog.html
Can you try running just generateChangeLog command without data and see if it works first?

Pentaho Spoon Job Executes Fine, Endless Loop in Kitchen

Without getting too much into the weeds, I have a Pentaho PDI job with multiple sub-transformations and sub-jobs (ETL from MySQL to Postgres). This job runs exactly as expected from Spoon, no errors, but when I run the job--with the following command--I am met with an endless loop error at the first step where a parameter would need to be defined and passed from within the job (the named params from the command seem to integrate fine). The command I am using is as follows:
sudo /bin/sh kitchen.sh \
-rep=KettleFileRepo \
-dir=M2P \
-job=ETL-M2P \
-level=Rowlevel \
-param:MY.PAR.LOADTYPE=full \
-param:MY.PAR.TABLELIST=table1 \
-param:MY.PAR.TENANTS=tenant1 \
/
Has anyone run into this type of issue with a discrepancy between Spoon and Kitchen? Is there some sort of config or command line option that I am missing? I am running version 6.0.1.0-386 on OS X 10.11.4.
If you think more details would be beneficial please let me know and I can provide whatever is necessary.
I am not aware of any discrepancy between Spoon and Kitchen. Are you sure, its not something in the ETL that causing the loop. I would suggest to go through your ETL in detail.
Another thing you can try to debug is run only part of the job in kitchen and keep adding more as you see success.

Using sql within shell script

I am currently trying to integrate an sql statement into a shell script, But facing major syntax issue:
My statement in the script:
su - <sid>adm -c 'hdbsql -U SYSTEM export "'SCHEMA'"."'*'" as binary into "'Export Location'" with reconfigure'
I get the following error:
* 257: sql syntax error: incorrect syntax near "*": line 1 col 16 (at pos 16) SQLSTATE: HY000
Would really appreciate if anyone could help me with this.
Thanks and Regards,
AK
Your command line doesn't make much sense to me. It starts with
su - <sid>adm
which means that you are redirecting the contents of the file "sid" into "su" and then redirecting the result of that operation into the file "adm".
Second problem is that in the command you are giving to adm, the single quotes end right before the "" which means, that the "" will get interpreted by the shell as a file glob:
-c 'hdbsql -U SYSTEM export "'SCHEMA'"."'*'" as binary into "'Export Location'" with reconfigure'
You'll need to escape those single quotes like this: "\'".
But I think your problem solving approach is not good. Try to reduce to problem and only then start adding additional things to it. So first try to execute the SQL statement from the "hdbsql" shell. Does it work?
$ hdbsql
> YOUR SQL STATEMENT HERE
Once that works, try to execute the SQL statement from the unix shell as a user:
$ hdbsql -U SYSTEM export ...
Once that works, try to execute it via su
$ su - ...

hide error messages in dcl script

I have a test script I'm running that generates some errors,shown below, I expect these errors. Is there anyway I can prevent them from showing on the screen however? I use the
$ write sys$output
to display if there is an expected error.
I tried to use
$ DEFINE SYS$ERROR ERROR.LOG
but this then changed my entire error output log to this, if this is the correct way to handle it can I unset this at the end of my script somehow?
[error example]
%DCL-E-OPENIN, error opening TEST$DISK:[AAA]NOTTHERE.TXT; as input
-RMS-E-FNF, file not found
%DCL-E-OPENIN, error opening TEST$DISK:[AAA]NOTTHERE.TXT; as input
-RMS-E-FNF, file not found
%DCL-W-UNDFIL, file has not been opened by DCL - check logical name
DEFINE/USER creates a logical name that disappears when the next image exits.
So if you use that just before a command just to protect that command, then fine.
Otherwise I would prefer SET MESSAGE to control the output.
And of course yoy want to grab $STATUS and verify it after the command for success or for the expected error, reporting any unexpected error.
Better still... if you expect certain error conditions to occur,
then why not test for them?
For example:
$ file = F$SEARCH("TEST$DISK:[AAA]NOTTHERE.TXT")
$ IF file.NES."" THEN TYPE 'file'
Cheers,
Hein
To suppress Error message inside a script. try this command
$ DEFINE/USER SYS$ERROR NL:
NL: is a null device, so you don`t see any error messages displayed on your terminal.
good luck
This works interactively and in batch.
$ SET MESSAGE /NOTEXT /NOSEV /NOFAC /NOID
$ <DCL_Command>
$ SET MESSAGE /TEXT /SEV /FAC/ ID

127 Return code from $?

What is the meaning of return value 127 from $? in UNIX.
Value 127 is returned by /bin/sh when the given command is not found within your PATH system variable and it is not a built-in shell command. In other words, the system doesn't understand your command, because it doesn't know where to find the binary you're trying to call.
Generally it means:
127 - command not found
but it can also mean that the command is found,
but a library that is required by the command is NOT found.
127 - command not found
example: $caat
The error message will
bash:
caat: command not found
now you check using echo $?
A shell convention is that a successful executable should exit with the value 0. Anything else can be interpreted as a failure of some sort, on part of bash or the executable you that just ran. See also $PIPESTATUS and the EXIT STATUS section of the bash man page:
For the shell’s purposes, a command which exits with a zero exit status has succeeded. An exit status
of zero indicates success. A non-zero exit status indicates failure. When a command terminates on a
fatal signal N, bash uses the value of 128+N as the exit status.
If a command is not found, the child process created to execute it returns a status of 127. If a com-
mand is found but is not executable, the return status is 126.
If a command fails because of an error during expansion or redirection, the exit status is greater than
zero.
Shell builtin commands return a status of 0 (true) if successful, and non-zero (false) if an error
occurs while they execute. All builtins return an exit status of 2 to indicate incorrect usage.
Bash itself returns the exit status of the last command executed, unless a syntax error occurs, in
which case it exits with a non-zero value. See also the exit builtin command below.
It has no special meaning, other than that the last process to exit did so with an exit status of 127.
However, it is also used by bash (assuming you're using bash as a shell) to tell you that the command you tried to execute couldn't be executed (i.e. it couldn't be found). It's unfortunately not immediately deducible though, if the process exited with status 127, or if it couldn't found.
EDIT:
Not immediately deducible, except for the output on the console, but this is stack overflow, so I assume you're doing this in a script.
If you're trying to run a program using a scripting language, you may need to include the full path of the scripting language and the file to execute. For example:
exec('/usr/local/bin/node /usr/local/lib/node_modules/uglifycss/uglifycss in.css > out.css');
This error is also at times deceiving. It says file is not found even though the files is indeed present. It could be because of invalid unreadable special characters present in the files that could be caused by the editor you are using. This link might help you in such cases.
-bash: ./my_script: /bin/bash^M: bad interpreter: No such file or directory
The best way to find out if it is this issue is to simple place an echo statement in the entire file and verify if the same error is thrown.
If the IBM mainframe JCL has some extra characters or numbers at the end of the name of unix script being called then it can throw such error.
In addition to the given answers, note that running a script file with incorrect end-of-line characters could also result in 127 exit code if you use /bin/sh as your shell.
As an example, if you run a shell script with CRLF end-of-line characters in a UNIX-based system and in the /bin/sh shell, it is possible to encounter some errors like the following I've got after running my script named my_test.sh :
$ ./my_test.sh
sh: 2: ./my_test.sh: not found
$ echo $?
127
As a note, using /bin/bash, I got 126 exit code, which is in accordance with gnu.org documentation about the bash :
If a command is not found, the child process created to execute it returns a status of 127. If a command is found but is not executable, the return status is 126.
Finally, here is the result of running my script in /bin/bash :
arman#Debian-1100:~$ ./my_test.sh
-bash: ./my_test.sh: /bin/bash^M: bad interpreter: No such file or directory
arman#Debian-1100:~$ echo $?
126
go to C:\Program Files\Git\etc
open gitconfig with notepad
change
[core]
autocrlf = true
To
[core]
autocrlf = false