I'm doing some logic during the migration that requires running a custom script. To do this I'm using exexuteCommand to run a sh script which then runs my code using node. This works fine except that I can't actually make it so that the migration fails when an error occurs.
I've reduced the problem to the following example.
yaml file configuring the migration step:
databaseChangeLog:
- changeSet:
id: 1637655796
author: [me]
changes:
- executeCommand:
executable: [path to script]
sh file:
node [path to .js file]
js file:
process.exit(1)
also tried:
throw new Error()
The file runs properly but liquibase marks the migration as successfull and updates the DB change log as if everything went as planned. How do I make the migration fail when the script fails?
I figured it out, even though the node script is failing in this example, the bash script isn't. I assumed that the failure "transfers" up the chain, but it doesn't. Doing exit 1 in the bash script results in the intended behavior.
To actually transfer the error from the node script to the bash script I used:
node [path to js file]
exit_code = $?
exit $exit_code
I could have skipped the 2nd line and wrote exit $? but my actual implementation has steps in between the js file running and the sh script exiting so the value of $? would have changed by then.
Related
Hi All I am tying to setup a RestAPI pipeline in aws codebuild. I have custom Newman docker. I have a build command that will failure but I want to execute the rest of the commands as well. but shell stops executions other commands when the Newman command fails. how to execute other commands in yml file.
One simple way would be:
You can create a custom shell script (mycommand.sh) with your command that can cause error inside a try catch statement (so that it will not result in an error)
In your Code build's yml file under commands section, just execute the ./mycommand.sh
Source:
https://docs.aws.amazon.com/codebuild/latest/userguide/build-env-ref-cmd.html
I'm trying to dig into the depths of teamcity to get a better understanding of what its doing under the hood(and improve my own build knowledge). I noticed that when I run a build step it then executes its own .cmd which I presume wraps up the msbuild scripts. The problem is that whenever I look in the directory specified the file doesn't exist as I'm guessing it creates, executes then deletes almost instantly. Any suggestions on how to access the file? or what's inside?
Starting:D:\TeamCity\buildAgent\temp\agentTmp\custom_script5990675507156014131.cmd
A temporary file is created by TeamCity when you run add a Command Line Build Step with "Custom script" as runner.
The content of this file would be the Custom script you specified inside the user interface.
The produced output would be:
Step 1/1: Command Line (1s)
Starting: D:\TeamCity\buildAgent\temp\agentTmp\custom_script2362934300799611461.cmd
in directory: D:\TeamCity\buildAgent\work\c72dca7a7355b5de
Hello World
Process exited with code 0
In case anyone is wondering about this still, you can force echo back on.
Put as the first thing in the custom script
#echo on
this will undo the silent commands teamcity defaults to.
I looked around for a while but there seems to be no configuration variable in TeamCity allowing to keep generated files. Now if the commands executed take some time, e.g. more than a couple of seconds, you could just open the temp directory in explorer and start hitting F5 (refresh) from the moment a build is started until you see the .cmd file appear, then be quick and right-click it and select 'Edit' to open it in a text editor. If that is too hard you can try with the solution presented here: create a Powershell script with code like this:
$watcher = New-Object System.IO.FileSystemWatcher
$watcher.Path = "D:\TeamCity\buildAgent\temp\agentTmp"
$watcher.Filter = "*.cmd"
$watcher.IncludeSubdirectories = $false
$watcher.EnableRaisingEvents = $true
$action = { $path = $Event.SourceEventArgs.FullPath
Add-content "D:\log.txt" -value (Get-Content $path)
}
Register-ObjectEvent $watcher "Created" -Action $action
Register-ObjectEvent $watcher "Changed" -Action $action
while ($true) {sleep 1}
and run it. When the build starts and creates a cmd file, the powershell script will copy the content to d:\log.txt. This will still not work for very short-lived scripts though. In that case I'd just make the script last longer by adding something like
ping 127.0.0.1 -n 5 -w 1000 > NUL
which will make it last at least 5 seconds.
I am trying to run my .ktr file in cmd line. I have my data-integration setup in this path:
C:\Users\dhamodharan.a\Desktop\pdi-ce-4.4.0-stable\data-integration
and my .ktr file in this path:
C:\Users\dhamodharan.a\Desktop\test.krt
while am trying to run that in cmd line I am getting the following error
DEBUG: Using PENTAHO_JAVA_HOME
DEBUG: _PENTAHO_JAVA_HOME=C:\Program Files (x86)\Java\jre7
DEBUG: _PENTAHO_JAVA=C:\Program Files (x86)\Java\jre7\bin\java.exe
WARN 11-08 11:47:09,728 - Unable to load Hadoop Configuration from "file:///C:/
Users/dhamodharan.a/Desktop/pdi-ce-4.4.0-stable/data-integration/plugins/pentaho
-big-data-plugin/hadoop-configurations/mapr". For more information enable debug
logging.
INFO 11-08 11:47:09,759 - Pan - Start of run.
ERROR: No repository provided, can't load transformation.
C:\Users\dhamodharan.a\Desktop\pdi-ce-4.4.0-stable\data-integration>e:C:\Users\d
hamodharan.a\Desktop.test.ktr /level:Basic
I am trying to run an input excel file and make the output as excel. Do I also need to create repository for that?
If I try to create repository option I saw only for dbms not for excel.
Make sure the environment variable PENTAHO_JAVA_HOME is set correctly and then it'll work.
For some reason the java install is not on your path. But if spoon works you must have it somewhere.
i did the environment variable for JAVA_HOME now pan.bat and kitchen.bat works fine.
here the command:
pan.bat /file:C:\Users\dhamodharan.a\Desktop\dhamu\test.ktr /level:Basic > C:\Users\dhamodharan.a\Desktop\dhamu\test.log
thanks
Take a look at Get puppet build to fail when the contained SQL script fails execution
I was attempting to run a vagrant build which installs Oracle XE in an Ubuntu Virtualbox VM and then runs a an SQL script to initialize the Oracle Schema. The vagrant build is here : https://github.com/ajorpheus/vagrant-ubuntu-oracle-xe. The setup.sql is run as a part of the oracle module's init.pp (right at the bottom or search for 'oracle-script').
When running the SQL script as a part of the vagrant build, I see the following error:
notice: /Stage[main]/Oracle::Xe/Exec[oracle-script]/returns: Error 6 initializing SQL*Plus
notice: /Stage[main]/Oracle::Xe/Exec[oracle-script]/returns: SP2-0667: Message file sp1<lang>.msb not found
notice: /Stage[main]/Oracle::Xe/Exec[oracle-script]/returns: SP2-0750: You may need to set ORACLE_HOME to your Oracle software directory
There were two things that were instrumental in me finding a workaround for the problem:
As suggested in this answer, setting the logoutput attribute to true for the exec block under question immediately showed me the error, whereas before the exec was just failing silently.
It seemed strange that I was able to run the command (sqlplus system/manager#xe < /tmp/setup.sql) after manually logging in as the 'vagrant' user. That suggested that there was something missing in the environment. Therefore, I copied all ORACLE env. vars into the exec as seen on Line 211 here
That worked, however, setting up the env vars manually seems a bit brittle. Is there a better way to setup the ORACLE environment for the vagrant user? Or, is there a way to get puppet to setup the environment for the vagrant user similar to an interactive shell?
If some profile has been set up to give the user a working interactive shell, you should be able to pass your action through such a shell
command => 'bash -i -c "<actual command>"'
As an aside about logoutput, since you mentioned that - the documentation advises that "on_failure" is a sane default, as it will only bloat your output when there are actual errors to analyze. It is the actual default in the latests versions of Puppet.
I'm barely started to use Jenkins and this is the first problem I've had so far. Basically my jenkins job always succeed even when an error happened in some of the tests. This is what I'm running in the shell config:
bundle install
rake db:migrate:reset
rake test:units
rake spec:models
Thing is that Jenkins only reports a failure when the task which fails is the last one. For instance, if I put "rake test:units" the last task it will notify an error if something go wrong. Using this configuration I only get error reports for the rspec tests but not for the unit tests.
Anyone wondering why I don't only use rspec or unit test, we are currently migrating to rspec but this problem is still painful.
This is part of the log from Jenkinsm as you can see one of the unit test fails but jenkins still finish with success.
314 tests, 1781 assertions, 1 failures, 0 errors, 0 skips
rake aborted!
Command failed with status (1): [/var/lib/jenkins/.rvm/rubies/ruby-1.9.3-p1...]
Tasks: TOP => test:units
(See full trace by running task with --trace)
Lot of rspec tests here....
Finished in 3.84 seconds
88 examples, 0 failures, 42 pending
Pushing HEAD to branch master of origin repository
Pushing HEAD to branch master at repo origin
Finished: SUCCESS
Jenkins executes the commands you type into a Build Step box by writing them to a temporary file and then running the script using /bin/sh -xe.
Usually this produces the desired effect: Commands are executed in sequence (and printed) and the script aborts immediately when a command fails i.e. exits with non-zero exit code.
If this is not happening to you, the only reason can be that you have overridden this behavior. You can override it by starting the first line of your Build Step with these two characters: #!.
For example, if your Build Step looks like this:
#!/bin/bash
bundle install
rake db:migrate:reset
rake test:units
rake spec:models
Then it means Jenkins will write the script to a temporary file and it will be executed with /bin/bash. When invoked like that, bash will execute commands one-by-one and not care if they succeed. The exit code of the bash process will be the exit code of the last command in the script and that will be seen by Jenkins when the script ends.
So, take care in what you put on the first line of the Build Step. If you do not know how shell works, do not put a hash-bang at all and let Jenkins decide how the script should be run.
If you need more control over how the Build Step is executed, you should study the man page of the shell you use to find out how to make it behave the way you want. Jenkins doesn't have much of a role in here. It just executes the shell you wanted the way you wanted.
Jenkins can only see the result code of the last command run so it has no way of knowing what the result of rake test:units is.
The easiest thing is probably to have each command of those commands as a separate jenkins build step.
An alternative solution is change your first line to the following:
#!/bin/bash -e
This tells your script to fail if any of the commands in the script return an error.
See: Automatic exit from bash shell script on error