Take a look at Get puppet build to fail when the contained SQL script fails execution
I was attempting to run a vagrant build which installs Oracle XE in an Ubuntu Virtualbox VM and then runs a an SQL script to initialize the Oracle Schema. The vagrant build is here : https://github.com/ajorpheus/vagrant-ubuntu-oracle-xe. The setup.sql is run as a part of the oracle module's init.pp (right at the bottom or search for 'oracle-script').
When running the SQL script as a part of the vagrant build, I see the following error:
notice: /Stage[main]/Oracle::Xe/Exec[oracle-script]/returns: Error 6 initializing SQL*Plus
notice: /Stage[main]/Oracle::Xe/Exec[oracle-script]/returns: SP2-0667: Message file sp1<lang>.msb not found
notice: /Stage[main]/Oracle::Xe/Exec[oracle-script]/returns: SP2-0750: You may need to set ORACLE_HOME to your Oracle software directory
There were two things that were instrumental in me finding a workaround for the problem:
As suggested in this answer, setting the logoutput attribute to true for the exec block under question immediately showed me the error, whereas before the exec was just failing silently.
It seemed strange that I was able to run the command (sqlplus system/manager#xe < /tmp/setup.sql) after manually logging in as the 'vagrant' user. That suggested that there was something missing in the environment. Therefore, I copied all ORACLE env. vars into the exec as seen on Line 211 here
That worked, however, setting up the env vars manually seems a bit brittle. Is there a better way to setup the ORACLE environment for the vagrant user? Or, is there a way to get puppet to setup the environment for the vagrant user similar to an interactive shell?
If some profile has been set up to give the user a working interactive shell, you should be able to pass your action through such a shell
command => 'bash -i -c "<actual command>"'
As an aside about logoutput, since you mentioned that - the documentation advises that "on_failure" is a sane default, as it will only bloat your output when there are actual errors to analyze. It is the actual default in the latests versions of Puppet.
Related
I am trying to run sesu command in Unix server from Python with the help of Paramiko exec_command. However when I am running this command exec_command('sesu test'), I am getting
sh: sesu: not found
When I am running simple ls command it giving me desired output. Only with sesu command it is not working fine.
This is how my code looks like:
import paramiko
host = host
username = username
password = password
port = port
ssh=paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(ip,port,username,password)
stdin,stdout,stderr=ssh.exec_command('sesu test')
stdin.write('Password')
stdin.flush()
outlines=stdout.readlines()
resp=''.join(outlines)
print(resp)
The SSHClient.exec_command by default does not run shell in "login" mode and does not allocate a pseudo terminal for the session. As a consequence a different set of startup scripts is (might be) sourced, than in your regular interactive SSH session (particularly for non-interactive sessions, .bash_profile is not sourced). And/or different branches in the scripts are taken, based on an absence/presence of TERM environment variable.
Possible solutions (in preference order):
Fix the command not to rely on a specific environment. Use a full path to sesu in the command. E.g.:
/bin/sesu test
If you do not know the full path, on common *nix systems, you can use which sesu command in your interactive SSH session.
Fix your startup scripts to set the PATH the same for both interactive and non-interactive sessions.
Try running the script explicitly via login shell (use --login switch with common *nix shells):
bash --login -c "sesu test"
If the command itself relies on a specific environment setup and you cannot fix the startup scripts, you can change the environment in the command itself. Syntax for that depends on the remote system and/or the shell. In common *nix systems, this works:
PATH="$PATH;/path/to/sesu" && sesu test
Another (not recommended) approach is to force the pseudo terminal allocation for the "exec" channel using the get_pty parameter:
stdin,stdout,stderr = ssh.exec_command('sesu test', get_pty=True)
Using the pseudo terminal to automate a command execution can bring you nasty side effects. See for example Is there a simple way to get rid of junk values that come when you SSH using Python's Paramiko library and fetch output from CLI of a remote machine?
You may have a similar problem with LD_LIBRARY_PATH and locating shared objects.
See also:
Environment variable differences when using Paramiko
Certain Unix commands fail with "... not found", when executed through Java using JSch
I'm using singularity to run python in an environnement deprived of python. I'm also running a mysql instance as explained by the IOWA state university (running an instance of mysql, and closing it when done).
For clarity, I'm using a bash script to open mysql, then do what i have to do (a python script) and close mysql, and it works fine. But Python's only way to stop if an error occured is sys.exit([value]) and this not only stops the python script, but also the bash script that ran it. This makes it impossible for me to manage the errors and close the instance of mysql if the python script exits.
My question is : Is there a way for me to execute a 'singularity instance stop mysql' while being in the python sandbox. Something to tell singularity "hey, this command here must be used on the host !" ?
I keep searching but can't find anything.
I only tried to execute it with subprocess like any other command, but it returned an error message because I don't have this instance inside the python sandbox. I don't even have singularity in this sandbox.
For any clarifications, just ask me, I'm trying to be clear but I'm pretty sure it's not very clear.
Thanks a lot !
Generally speaking, it would be a big security issue if a process could be initiated from inside a container (docker or singularity) but run in the host OS's namespace.
If the bash script is exiting on the python failure, it sounds like you're using set -e or #!/bin/bash -e. This causes the script to abort if any command returns non-zero. It's commonly recommended for safer processing, but can cause problems like this at times. To bypass that for the python step you can modify your script:
# start mysql, do some stuff
set +x # disable abort on non-zero return
python my_script.py
set -x # re-enable abort on non-zero
# shut down mysql, do other stuff
I am trying to diagnose a few issues with ssl connectivity with Leingen. I am trying to find what SSL Key Store and Trust Store is being Used by Leingen,
I am behind a corporate firewall and we have self signed certificates deployed on all our desktops . I am running lein.bat on a windows 10.
Hence I have to start Leingen with java -Djavax.net.debug=true option.
The :jvm-opts in the project.clj wont work -- I need to make sure the Liengen's JVM is started with this option
You can set leiningen JVM options by setting LEIN_JVM_OPTS environment variable before running lein in the same terminal session.
The lein command is just a shell script which eventually invokes java with various options. You can edit this script to see what options are used and/or to modify them.
As Piotrek mentioned, the LEIN_JVM_OPTS environment variable is the canonical way of passing options to the jvm in which lein runs. You can see it used on line 372 of the source code.
For your case:
> export LEIN_JVM_OPTS='-Djavax.net.debug=true'
> lein clean
> lein run
Since you're running windows, you'll want to actually look at the lein.bat file. You'll still need to update LEIN_JVM_OPTS, but how you go about it will be a bit different. If you're using windows command terminal (cmd.exe) you will want to use the set command.
set LEIN_JVM_OPTS="-Djavax.net.debug=true"
The command is likely different if you're using powershell, and you can likely find out how to set that on this page on environment variables.
Jenkins version: 1.574
I created a simple job which performs the following:
Using "Execute shell script on remote host using SSH" as one of the BUILD steps, I'm just calling a shell script. This shell script performs stop and start operations on Tomcat to restart an application on the target machine.
I have a valid username, password, port defined for the target SSH server in Jenkins Global settings.
I saw this behavior that when I run a Jenkins job and call the restart script (which gets the application name as parameter $1), it works fine, but as soon as "Execute shell script on remote host using SSH" step completes, I see the new process dies on the remote/target application server.
If I run the script from the target/remote server itself, everything works fine and the new process/PID remains live forever, but running the same script from Jenkins, though I don't see any errors and everything works as expected, the new process dies as soon as the above mentioned SSH step is complete and control comes back to the next BUILD step in Jenkins job OR the Jenkins job is complete.
I saw a few posts/blogs and tried setting: BUILD_ID=dontKillMe in the Jenkins job (in various places i.e. Prepare Environment variables and also using Inject Environment variables...). When the job's particular build# is complete, I can see Environment Variables for that build# does say BUILD_ID=dontKillMe as its value (instead of the default Timestamp tag value).
I tried putting nohup before calling the restart script, i.e.,
nohup restart_tomcat.sh "${app}"
I also tried:
BUILD_ID=dontKillMe nohup restart_tomcat.sh "${app}"
This doesn't give any error and creates a nohup.out file on the remote server (but I'm not worried about it as the restart_tomcat.sh script itself creates its own LOG file which I'm "cat"ing after the restart_tomcat.sh script is complete. cat'ing on the log file is performed using another "Execute shell script on remote host using SSH" build step, and it successfully shows the log file created by the restart script).
I don't know what I'm missing at this point, but as soon as the restart_tomcat.sh step is complete, the new PID/process on the remote/target server dies.
How can I fix this?
I've been through this myself.
On my first iteration, before I knew about Jenkins ProcessTreeKiller, I ended up just daemonizing Tomcat. The Apache Tomcat documentation includes a section on running as a daemon.
You can also try disabling the ProcessTreeKiller for your whole Jenkins instance, if it's relatively small (read the first link for information).
The BUILD_ID=dontKillMe should be passed to the shell, and therefore it should be in your command line, not in Jenkins global configuration or job parameters.
BUILD_ID=dontKillMe restart_tomcat.sh "${app}" should have worked without problems.
You can also try nohup restart_tomcat.sh "${app}" & with the & at the end.
My solution (it worked after trying everything else) in Ubuntu 14.04 (Trusty Tahr) (Amazon AWS - Amazon EC2), Jenkins 1.601:
Exec command: (setsid COMMAND < /dev/null > /dev/null 2>&1 &);
Exec in PTY: DISABLED
// Example COMMAND=socat TCP4-LISTEN:1337,fork TCP4:127.0.0.1:1338
I created this Transfer as my last one.
#!/bin/ksh
export BUILD_ID=dontKillMe
I added the above line to the start of my script and the issue was resolved.
I'm following the instructions found here.
When I try to run $ heroku pg:psql or $ heroku pg:psql HEROKU POSTGRESQL_BROWN I recieve the following error message:
! The local psql command could not be located ! For help
installing psql, see local-postgresql
I can't find anything useful on the link it gives me (it just links to the instructions I was already using, but further down the page) nor can I find this error anywhere else.
If I've missed anything you need to know to answer this, just let me know. I'm rather new to all this and teaching myself as I go.
I had same error even after installing Postgres locally.
But after seeing this
I saw that "pqsl" was not in the PATH so I then did
PATH=%PATH%;C:\Program Files\PostgreSQL\9.2\bin
which worked for me
I have since solved this myself. When I ran heroku pg:info it says the version number is 9.1.8, I was locally running 9.2
installing 9.1.8 and ensuring Path pointed to the appropriate folder solved the problem.
After you change the path, make sure to restart the terminal!
Set the PATH. To find out the PATH of your psql script (on mac) open the sql shell script from your finder in Applications/Postgres installation. This will give you a hint as to where it is installed. That opened a window which told me it is located here: /Library/PostgreSQL/8.4/scripts/runpsql.sh
Then, I set the PATH variable from the terminal window by typing:
$ PATH="/Library/PostgreSQL/8.4/bin:$PATH"
(depends on the location of your PostgreSQL installation, find your bin path first, another exp: /usr/local/Cellar/postgresql#9.6/9.6.8/bin)
OR.....
You can also connect to the shell by opening the shell directly from your postgres installation folder. Then enter the credentials. If you don't know the credentials, here is how to find them out:
$ heroku pg:info
=== HEROKU_POSTGRESQL_RED_URL (DATABASE_URL)
$ heroku pg:credentials HEROKU_POSTGRESQL_RED_URL
Top answer wouldn't work for me oddly, my system would not add the Path via cmd with administrator access (Not sure why).
So check this > Windows key > environment variables > system variables
And add the last line (your version may differ in the path)
Make sure you've installed the toolbelt as psql is installed by default.
However you also need to ensure you've installed a local copy of PostgreSQL; if you don't the toolbelt will be unable to find the native psql client.
Assuming you have installed a local copy of PostgreSQL, make sure you can execute psql from the command line directly (i.e make sure you PATH is set correctly ). If the command does not execute, check your PATH, if it does execute see if you can connect via the PSQL connection string provided in the Heroku control panel. If you can connect reinstall the toolbelt, if you are unable to connect provision another dev database and try again.
If there are still issues, I would suggest contacting Heroku support for assistance after verifying no API issues are listed on the status page located here.
I got rid if this annoying message on Windows by adding a path element without the spaces, i.e.
C:\Progra~1\PostgreSQL\9.4\data
instead of
“C:\Program Files\PostgreSQL\9.4\data”
I followed the instructions here: http://www.computerhope.com/issues/ch000549.htm, which worked for me if you prefer to go the point-and-click configuration of the PATH variable.
This type of error usually appears in the Windows environment, because if you do not update the PATH after installing Postgresql, heroku pg:psql command does not work.
So you need to update your PATH environment variable to add the bin directory of your Postgres installation. The directory will look like this:
C:\Program Files\PostgreSQL\<VERSION>\bin.
For more information, go to the Heroku in Local setup website:
heroku-postgresql: Local setup
I had the same problem and discovered that Heroku doesn't seem to provision the latest version of PostgreSQL by default. Where the Heroku Getting Started instructions said
heroku addons:create heroku-postgresql:hobby-dev
That provisioned a v10 database for some reason (which you can check by clicking on Heroku Postgres in the Add-ons tab of your dashboard). I deleted that database and provisioned a new database using the --version flag:
heroku addons:create heroku-postgresql:hobby-dev --version 11
As of now, at least, you can find the latest version of Postgres supported by Heroku at this link: https://devcenter.heroku.com/articles/heroku-postgresql#version-support-and-legacy-infrastructure
I'm writing this in early 2019, but according to the PostgreSQL website the next version (12) is "tentatively scheduled" for third quarter of 2019 so if you're reading this in late 2019 potentially the same problem will come up for v12 instead
On Mac you can use the following:
export PATH="/Library/PostgreSQL/12/bin/:$PATH"
The only solution that I found on Windows:
go to advanced system settings
go to environment variables
select Path variable and click Edit
add a new line and enter your bin directory path (C:\Program Files\PostgreSQL<version>\bin) and click ok
restart your terminal
enter your psql command (heroku pg:psql)