run noninteractive ssh command in background immeidately suspends the job - ssh

(1) I can run the following command and get the output successfully
ssh server hostname
(2) If I run it in background (not to background hotname, but to background ssh)
ssh server hostname &
and do nothing other than wait, I can get the output
(3) However, if before it finishes I type any key to the terminal, the job immediately turns into suspended state
[ZSH] suspended (tty input) ssh server hostname
[BASH] Stopped ssh server hostname
What is the reason for this and how to solve it?
I just use hostname as an example. You can try using sleep 5 instead if the program returns too quickly. The actual program I want to run lasts for minutes.

Use ssh -T -f server hostname as the manual page states:
-f requests ssh to go to background just before command execution.
This is useful if ssh is going to ask for passwords or
passphrases, but the user wants it in the background. This
implies -n. The recommended way to start X11 programs at a
remote site is with something like ssh -f host xterm.
If the ExitOnForwardFailure configuration option is set to “yes”,
then a client started with -f will wait for all remote port for-
wards to be successfully established before placing itself in the
background.
-T Disable pseudo-tty allocation.

Related

How to do other "stuff" in the same terminal where you establish an SSH tunnel

I often use an ssh tunnel. I open up one terminal to create the tunnel (e.g. ssh -L 1111:servera:2222 user#serverb). Then I open a new terminal to do my work. Is there a way to establish the tunnel in a terminal and somehow put it in the background so I don't need to open up a new terminal? I tried putting "&" at the end, but that didn't do the trick. The tunnel went into the background before I could enter the password. Then I did fg, entered the password and I was stuck in the ssh session.
I know one possible solution would be to use screen or tmux or something like that. Is there a simple solution I'm missing?
There is the -f and -N options exactly for that:
-f Requests ssh to go to background just before command execution. This is useful if
ssh is going to ask for passwords or passphrases, but the user wants it in the
background. This implies -n. The recommended way to start X11 programs at a remote
site is with something like ssh -f host xterm.
If the ExitOnForwardFailure configuration option is set to ``yes'', then a client
started with -f will wait for all remote port forwards to be successfully established
before placing itself in the background.
-N Do not execute a remote command. This is useful for just forwarding ports
(protocol version 2 only).
So the full command would be ssh -fNL 1111:servera:2222 user#serverb.
A way to prevent ssh asking for the password would also be to use SSH public keys for authentication with an agent that either saves the password or prompts it using an external graphical program such as pinentry.
It might also be useful for you to look into autossh, which will reconnect your SSH automatically if the connection drops.

SSH to multiple hosts at once

I have a script which loops through a list of hosts, connecting to each of them with SSH using an RSA key, and then saving the output to a file on my local machine - this all works correctly. However, the commands to run on each server take a while (~30 minutes) and there are 10 servers. I would like to run the commands in parallel to save time, but can't seem to get it working. Here is the code as it is now (working):
for host in $HOSTS; do
echo "Connecting to $host"..
ssh -n -t -t $USER#$host "/data/reports/formatted_report.sh"
done
How can I speed this up?
You should add & to the end of the ssh call, it will run on the background.
for host in $HOSTS; do
echo "Connecting to $host"..
ssh -n -t -t $USER#$host "/data/reports/formatted_report.sh" &
done
I tried using & to send the SSH commands to the background, but I abandoned this because after the SSH commands are completed, the script performs some more commands on the output files, which need to have been created.
Using & made the script skip directly to those commands, which failed because the output files were not there yet. But then I learned about the wait command which waits for background commands to complete before continuing. Now this is my code which works:
for host in $HOSTS; do
echo "Connecting to $host"..
ssh -n -t -t $USER#$host "/data/reports/formatted_report.sh" &
done
wait
Try massh http://m.a.tt/er/massh/. This is a nice tool to run ssh across multiple hosts.
The Hypertable project has recently added a multi-host ssh tool. This tool is built with libssh and establishes connections and issues commands asynchronously and in parallel for maximum parallelism. See Multi-Host SSH Tool for complete documentation. To run a command on a set of hosts, you would run it as follows:
$ ht ssh host00,host01,host02 /data/reports/formatted_report.sh
You can also specify a host name or IP pattern, for example:
$ ht ssh 192.168.17.[1-99] /data/reports/formatted_report.sh
$ ht ssh host[00-99] /data/reports/formatted_report.sh
It also supports a --random-start-delay <millis> option that will delay the start of the command on each host by a random time interval between 0 and <millis> milliseconds. This option can be used to avoid thundering herd problems when the command being run accesses a central resource.

Causing SSH to Time Out (client side)

I have a little Raspberry Pi that I'm playing with. I've got it running headless, and I need to make it forward one of its ports to a remote server when certain conditions are satisfied.
However, I don't want the connection to sit indefinitely until the server closes it. Is there a way to close an SSH connection (from the client, I have no root to the server) after a certain amount of time? Ideally I'd do it directly via the ssh command, but I'm writing in Python 3, so if there's a way to do this in Python, then I'm all ears.
In your /etc/ssh/sshd_config:
ClientAliveInterval <time interval in seconds>
ClientAliveCountMax 0
So using 300 in the first directive will kick the connection after 5 minutes idle. You'll need to restart sshd to make it take effect.
try ssh -o ServerAliveInterval=10 server.org
Unless you run ssh with the "-N" option, it normally launches some kind of command or shell on the remote system (the Pi in this case). Ssh disconnects when this remote command exits.
If you're running ssh just to create some port forwards, you may be running with "-N", or you may be letting the ssh session sit at a command prompt. Instead, you could launch a command on the Pi which exits after the desired period of time. You could use the sleep command, for example:
ssh -Lwhatever -Rwhatever user#pi "sleep 3600"

SSH-add is not persistent event though ssh-agent is started

I did ssh-add. During the session it works fine, but when I exit and reconnect to the server with ssh it does not work anymore ssh-add -l says: no identities.
I do start the ssh-agent in my .bash_profile using eval $(ssh-agent).
Any ideas what I can do to keep the identities?
// EDIT
So here is my scenario. I am connecting to my web-space using ssh. I want to pull data from github using git pull. I added the connection to github and git pull works fine. But it asks for the passphrase. Doing a ssh-add adn adding the passphrase stops that, but only for the current session.
I have to start the ssh-agent using something like eval $(ssh-agent) because it does not autostart on the server.
The main problem I am having is, that I need a script to do the git pull which is invoked by a request from github, so I can not give it the passphrase.
If you're spawning the agent when your session starts, then it'll die when you disconnect. You could connect to a screen or byobu/tmux server which keeps the agent alive (you can skip an instance of bash if you connect like ssh user#hostname -t 'byobu').
Otherwise, have the agent come up when the machine boots so your session comings and goings don't affect it.
Edit: you can also forward your agent from your local machine. This works very well if you happen to have the same keys available on both machines. Try something like this in your ~/.ssh/config:
Host whatever
Hostname whatever.com
User username
ForwardAgent yes
You invoke this with ssh whatever, in case you are not familiar with that config file.

How to use ssh to run a local command after connection and quit after this local command is executed?

I wish to use SSH to establish a temporary port forward, run a local command and then quit, closing the ssh connection.
The command has to be run locally, not on the remote site.
For example consider a server in a DMZ and you need to allow an application from your machine to connect to port 8080, but you have only SSH access.
How can this be done?
Assuming you're using OpenSSH from the command line....
SSH can open a connection that will sustain the tunnel and remain active for as long as possible:
ssh -fNT -Llocalport:remotehost:remoteport targetserver
You can alternately have SSH launch something on the server that runs for some period of time. The tunnel will be open for that time. The SSH connection should remain after the remote command exits for as long as the tunnel is still in use. If you'll only use the tunnel once, then specify a short "sleep" to let the tunnel expire after use.
ssh -f -Llocalport:remotehost:remoteport targetserver sleep 10
If you want to be able to kill the tunnel from a script running on the local side, then I recommend you background it in your shell, then record the pid to kill later. Assuming you're using an operating system that includes Bourne shell....
#/bin/sh
ssh -f -Llocalport:remotehost:remoteport targetserver sleep 300 &
sshpid=$!
# Do your stuff within 300 seconds
kill $sshpid
If backgrounding your ssh using the shell is not to your liking, you can also use advanced ssh features to control a backgrounded process. As described here, the SSH features ControlMaster and ControlPath are how you make this work. For example, add the following to your ~/.ssh/config:
host targetserver
ControlMaster auto
ControlPath ~/.ssh/cm_sockets/%r#%h:%p
Now, your first connection to targetserver will set up a control, so that you can do things like this:
$ ssh -fNT -Llocalport:remoteserver:remoteport targetserver
$ ssh -O check targetserver
Master running (pid=23450)
$ <do your stuff>
$ ssh -O exit targetserver
Exit request sent.
$ ssh -O check targetserver
Control socket connect(/home/sorin/.ssh/cm_socket/sorin#192.0.2.3:22): No such file or directory
Obviously, these commands can be wrapped into your shell script as well.
You could use a script similar to this (untested):
#!/bin/bash
coproc ssh -L 8080:localhost:8080 user#server
./run-local-command
echo exit >&${COPROC[1]}
wait