Automatically (or more easily) reconnect to a screen session after network interruption - ssh

ADDED: This question is now, I believe, subsumed by this one:
Using GNU Screen completely transparently and automatically
See also this related question:
https://superuser.com/questions/147873/ssh-sessions-in-xterms-freeze-for-many-minutes-whenever-they-disconnect
Original question:
It would be nice if there were a way to ssh to a machine and immediately reconnect to a specific screen session. You can do this:
laptop> ssh server.com screen -ls
and it will show a list of screens available on server.com like so [1]:
123.pts-1
456.pts-2
And then you might try to do this:
laptop> ssh server.com screen -dr pts-2
but that fails, saying "Must be connected to a terminal."
You have to ssh in first and then do the "screen -dr pts-2" on server.com which is no good if you have a flaky connection and get disconnected a lot. You want to be able to resume with a simple "up-arrow enter" on the laptop. (Or perhaps make it even more automatic.)
I have a rihackulous solution to this problem which I'll post as an answer and hope it gets downvoted to oblivion in favor of the Right Way to deal with this.
Footnotes:
[1] Or, better, if you created the screen sessions with names like "screen -S foo" and "screen -S bar" then you'll get a friendlier list like:
123.foo
456.bar
and can reconnect with, eg, "screen -dr foo".
Mini screen tutorial, incorporating the answer to this question:
Login in to server.com and do
screen -S foo
and then never log out of that session again.
To reconnect to it from elsewhere, do
ssh -t server.com screen -dr foo
To list available screens to reconect to:
screen -ls
or, of course,
ssh server.com screen -ls
to check on server.com's available screens remotely.
I now use the following alias (tcsh), based on Jason's answer below, to connect to a named screen if it exists or create and connect otherwise:
alias ssc 'ssh -t \!:1 "screen -S \!:2 -dr || screen -S \!:2"'

Does the -t option do what you want?
-t Force pseudo-tty allocation. This can be used to execute arbi-
trary screen-based programs on a remote machine, which can be
very useful, e.g. when implementing menu services. Multiple -t
options force tty allocation, even if ssh has no local tty.
So:
laptop> ssh -t server.com screen -dr pts-2
This seems to work in my installation.

This is now subsumed by this: Using GNU Screen completely transparently and automatically
Here's a script, ssc, that works just like ssh but takes a third argument to specify the screen to reconnect to, or the name of a new screen.
I believe this script subsumes everything in the original question.
#!/usr/bin/env perl
# Use 'ssc' (this script) instead of 'ssh' to log into a remote machine.
# Without a 3rd argument it will list available screens.
# Give it a 3rd argument to attach to an existing screen or specify a new
# screen. Eg, ssc remote.com foo
# The numbers in front of the screen tag can usually be ignored.
# Screen is a little too clever though in that if there's an existing screen "bar"
# and you say "ssc remote.com b" it will reconnect you to "bar" instead of making
# a new screen "b". It's like invisible and silent tab-completion.
if(scalar(#ARGV)==0 || scalar(#ARGV) > 2) {
print "USAGE: ssc remote.com [screen name]\n";
} elsif (scalar(#ARGV) == 1) {
$machine = shift;
#screens = split("\n", `ssh $machine screen -ls`);
for(#screens) {
if(/^\s*(\d+)\.(\S+)\s+\(([^\)]*)\)/) {
($num, $tag, $status) = ($1, $2, $3);
if($status =~ /attached/i) { $att{"$num.$tag"} = 1; }
elsif($status =~ /detached/i) { $att{"$num.$tag"} = 0; }
else { print "Couldn't parse this: $_\n"; }
}
}
print "ATTACHED screens:\n";
for(keys(%att)) { print " $_\n" if $att{$_}; }
print "DETACHED screens:\n";
for(keys(%att)) { print " $_\n" unless $att{$_}; }
} else {
$machine = shift;
$tag = shift;
system("ssh -t $machine \"screen -S $tag -dr || screen -S $tag\"");
}

Use the -t option to ssh to allocate a terminal while directly running a command.
laptop> ssh -t server.com screen -dr pts-2

I've been working on something similar but not quite got there, your solutions have solved my problem so here's my suggestion:
ssh -t server.com "screen -S foo -rd || screen -S foo"
This just tries to open the existing screen named foo and if it doesnt exist, creates it.
I'll put this in a launcher on my laptop, so when the wireless network goes I can just open where I left off.
Just noticed that the default screen shell is a bit weak, so an improvement which sets up your home environment a little better is:
ssh -t server.com "screen -S foo -rd || screen -S foo bash -l"

I converted this to work on OS X .bash_profile with one addition: If no 2nd parameter is given, it will start a session "default".
function ssc() {
if [[ -z $2 ]]; then
screen="default"
else
screen=$2
fi
ssh -t $1 "screen -S $screen -dr || screen -S $screen"
}

If you like to connect to the same session always even it is active, detached or not exists yet:
ssh -t user#server screen -xR screenName
The same but create a new session if it is already active on some other pty:
ssh -t user#server screen -rR screenName

Related

how to create a GNU screen session with multi windows in it?

I alway work in a screen session with some windows, one for shell, one for mysql, one for music player, one for irc, and so on...
The problem is, when you create a screen session, it only creates one window by default. So I have to do Ctrl-a c then issue commands, again and again.
So I wrote a bash function to do this.
d(){
local i=workspace
screen -qls $i
if [ "$?" -ne 11 ];then
screen -dmS $i
screen -S $i -X screen mysql -uroot -p
screen -S $i -X screen irssi
screen -S $i -X screen nvlc $music -Z
screen -r $i -p0
else
screen -r $i
fi
}
My question is, is there a way to start a new screen session with some windows? By this question, I mean new session, NOT for existing sessions using '-X'. And, screen built-in feature, I mean, no shell scripting involved.
I didn't consider .screenrc file at first, because commands in it will be invoked every time you call screen, but sometimes I need to create a new screen session with different things.
One thing I forgot is that, I can choose configuration file.
So I think answer is:
d(){
screen -d -R -S workspace -c ~/.workspace
}
and content of ~/.workspace should be:
screen
screen mysql -uroot -p
screen irssi
screen nvlc
select 0

How to automatically start tmux on SSH session?

I have ten or so servers that I connect to with SSH on a regular basis. Each has an entry in my local computer's ~/.ssh/config file.
To avoid losing control of my running process when my Internet connection inevitably drops, I always work inside a tmux session. I would like a way to have tmux automatically connect every time an SSH connection is started, so I don't have to always type tmux attach || tmux new after I SSH in.
Unfortunately this isn't turning out to be as simple as I originally hoped.
I don't want to add any commands to the ~/.bashrc on the servers because I only want it for SSH sessions, not local sessions.
Adding tmux attach || tmux new to the ~/.ssh/rc on the servers simply results in the error not a terminal being thrown after connection, even when the RequestTTY force option is added to the line for that server in my local SSH config file.
Server-side configuration:
To automatically start tmux on your remote server when ordinarily logging in via SSH (and only SSH), edit the ~/.bashrc of your user or root (or both) on the remote server accordingly:
if [[ $- =~ i ]] && [[ -z "$TMUX" ]] && [[ -n "$SSH_TTY" ]]; then
tmux attach-session -t ssh_tmux || tmux new-session -s ssh_tmux
fi
This command creates a tmux session called ssh_tmux if none exists, or reattaches to a already existing session with that name. In case your connection dropped or when you forgot a session weeks ago, every SSH login automatically brings you back to the tmux-ssh session you left behind.
Connect from your client:
Nothing special, just ssh user#hostname.
Don't do this on the server-side!
That is potentially dangerous because you can end up being locked-out of the remote machine. And no shell hacks / aliases / etc. are required, either.
Instead...
... make use of (your client's) ~/.ssh/config like so:
tmux 3.1 or newer¹ on the remote machine
Into your local ~/.ssh/config, put²:
Host myhost
Hostname host
User user
RequestTTY yes
RemoteCommand tmux new -A -s foobar
As pointed out by #thiagowfx, this has the side effect of making it impossible to use, e.g. ssh myhost ls /tmp and should therefore not be used with Host * ... what I like to do is to have a Host myhost section with RemoteCommand tmux ... and then in addition to that I'll have a Host MYHOST section without it.
Instead of RequestTTY yes you could call ssh with the -t switch; thank you, #kyb.
Off-topic, but if you're dealing with non-ASCII characters, I'd recommend to change that into tmux -u … for explicitly enabling Unicode support even on machines that don't have the proper environment variables set.
tmux 3.0a or older on the remote machine
Almost the same as above, but change the last line to³:
RemoteCommand tmux at -t foobar || tmux new -s foobar
¹ repology.org has a list of distros and their tmux versions
² new is short for new-session.
³ at is short for attach-session.
Only if, for some reason, you really, really can't do it client-side:
Using the remote's authorized_keys file
If you would rather not have an ~/.ssh/config file for whatever reason, or want the remote machine to force the connecting machine to connect to / open the session, add this to your remote ~/.ssh/authorized_keys:
command="tmux at -t foobar || tmux new -s foobar" pubkey user#client
This will, of course, work from all clients having the corresponding private key installed, which some might consider an upside –– but: should anything go wrong, it might not be possible to connect anymore without (semi-)physical access to the machine!
One caveat!
As #thiagowfx notes in the comments, this should not be put underneath Host * as it breaks certain things, such as git push. What I personally do is to add a second entry in all-uppercase letters for where I want to automatically be connected to tmux.
Alright, I found a mostly satisfactory solution. In my local ~/.bashrc, I wrote a function:
function ssh () {/usr/bin/ssh -t "$#" "tmux attach || tmux new";}
which basically overwrites the ssh terminal function to call the built-in ssh program with the given arguments, followed by "tmux attach || tmux new".
(The $# denotes all arguments provided on the command line, so ssh -p 123 user#hostname will be expanded to ssh -t -p 123 user#hostname "tmux attach || tmux new")
(The -t argument is equivalent to RequestTTY Force and is necessary for the tmux command.)
Connect:
ssh user#host -t "tmux new-session -s user || tmux attach-session -t user"
During session:
Use Ctrl+d to finish session (tmux window closes) or Ctrl+b d to temporary detach from session and connect to it again later.
Remember! If your server restarted session lost!
When you are inside tmux anytime you can use Ctrl+b s to see sessions list and switch current to another.
Fix your .bashrc:
I recommend you to define universal function in your .bashrc:
function tmux-connect {
TERM=xterm-256color ssh -p ${3:-22} $1#$2 -t "tmux new-session -s $1 || tmux attach-session -t $1"
}
It uses 22 port by default. Define your fast-connect aliases too:
alias office-server='tmux-connect $USER 192.168.1.123'
alias cloud-server='tmux-connect root my.remote.vps.server.com 49281'
Login without password:
And if you don't want to type password everytime than generate .ssh keys to login automatically:
ssh-keygen -t rsa
eval "$(ssh-agent -s)" && ssh-add ~/.ssh/id_rsa
Put your public key to the remote host:
ssh-copy-id -p <port> user#hostname
Additional tips:
If you want to use temporary session-id which corresponds with a local bash session use as tmux id:
SID=$USER-$BASHPID
ssh user#host -t "tmux new-session -s $SID || tmux attach-session -t $SID"
I used lines from #kingmeffisto (I'm not allowed to comment that answer) and I added an exit so terminating tmux also terminates the ssh connection. This however broke SFTP sessions so I had to check for $SSH_TTY instead of $SSH_CONNECTION.
EDIT 4/2018: Added test for interactive terminal via [[ $- =~ i ]] to allow tools like Ansible to work.
if [ -z "$TMUX" ] && [ -n "$SSH_TTY" ] && [[ $- =~ i ]]; then
tmux attach-session -t ssh || tmux new-session -s ssh
exit
fi
As described in this blog post you can ssh and then attach to an existing tmux session with a single command:
ssh hostname -t tmux attach -t 0
This is the one that actually creates a great user-experience.
It automatically starts tmux whenever you open the terminal (both physically and ssh).
You can start your work on one device, exit the terminal, and resume on the other one. If it detects someone already attached to the session it will create new session.
Put it on the server, depending on your shell ~/.zshrc or ~/.bashrc.
if [[ -z "$TMUX" ]] ;then
ID="$( tmux ls | grep -vm1 attached | cut -d: -f1 )" # get the id of a deattached session
if [[ -z "$ID" ]] ;then # if not available attach to a new one
tmux new-session
else
tmux attach-session -t "$ID" # if available attach to it
fi
fi
I have the following solution that gives you two SSH hosts to connect to: one with tmux, one without:
# Common rule that 1) copies your tmux.conf 2) runs tmux on the remote host
Host *-tmux
LocalCommand scp %d/.tmux.conf %r#%n:/home/%r/
RemoteCommand tmux new -As %r
RequestTTY yes
PermitLocalCommand yes
# Just connect.
# Notice the asterisk: makes possible to re-use connection parameters
Host example.com*
HostName example.com
User login
# Connect with tmux
Host example.com-tmux
HostKeyAlias dev.dignio.com
You might find this useful - uses ssh in a loop and reconnects to or connects to an existing tmux session so you have a nice easy reliable
way to reconnect after a network outage
#!/bin/bash
#
# reconnect to or spawn a new tmux session on the remote host via ssh.
# If the network connection is lost, ssh will reconnect after a small
# delay.
#
SSH_HOSTNAME=$1
TMUX_NAME=$2
PORT=$3
if [[ "$PORT" != "" ]]
then
PORT="-p $PORT"
fi
if [ "$TMUX_NAME" = "" ]
then
SSH_UNIQUE_ID_FILE="/tmp/.ssh-UNIQUE_ID.$LOGNAME"
if [ -f $SSH_UNIQUE_ID_FILE ]
then
TMUX_NAME=`cat $SSH_UNIQUE_ID_FILE`
TMUX_NAME=`expr $TMUX_NAME + $RANDOM % 100`
else
TMUX_NAME=`expr $RANDOM % 1024`
fi
echo $TMUX_NAME > $SSH_UNIQUE_ID_FILE
TMUX_NAME="id$TMUX_NAME"
fi
echo Connecting to tmux $TMUX_NAME on hostname $SSH_HOSTNAME
SLEEP=0
while true; do
ssh $PORT -o TCPKeepAlive=no -o ServerAliveInterval=15 -Y -X -C -t -o BatchMode=yes $SSH_HOSTNAME "tmux attach-session -t $TMUX_NAME || tmux -2 -u new-session -s $TMUX_NAME"
SLEEP=10
if [ $SLEEP -gt 0 ]
then
echo Reconnecting to session $TMUX_NAME on hostname $SSH_HOSTNAME in $SLEEP seconds
sleep $SLEEP
fi
done
byobu is a nice useful wrapper for tmux/screen. Connects to an existing session if present or creates a new one.
I use it with autossh which gracefully reconnects the ssh session. Highly recommended in case of intermittent connectivity issues.
function ssh-tmux(){
if ! command -v autossh &> /dev/null; then echo "Install autossh"; fi
autossh -M 0 $* -t 'byobu || {echo "Install byobu-tmux on server..."} && bash'
}
I know I'm reviving an old thread but I've done some work on the bashrc solution and I think it has some use:
#attach to the next available tmux session that's not currently occupied
if [[ -z "$TMUX" ]] && [ "SSH_CONNECTION" != "" ];
then
for i in `seq 0 10`; do #max of 10 sessions - don't want an infinite loop until we know this works
SESH=`tmux list-clients -t "$USER-$i-tmux" 2>/dev/null` #send errors to /dev/null - if the session doesn't exist it will throw an error, but we don't care
if [ -z "$SESH" ] #if there's no clients currently connected to this session
then
tmux attach-session -t "$USER-$i-tmux" || tmux new-session -s "$USER-$i-tmux" #attach to it
break #found one and using it, don't keep looping (this will actually run after tmux exits AFAICT)
fi #otherwise, increment session counter and keep going
done
fi
There's a cap at 10 (11) sessions for now - I didn't want to kill my server with an infinite loop in bashrc. It seems to work pretty reliably, other than the error of tmux failing on list-clients if the session doesn't exist.
Thie way allows you to reconnect to an old tmux instance if your ssh session drops. The exec saves a fork of course.
if [ -z "$TMUX" ]; then
pid=$(tmux ls | grep -vm1 "(attached)" | cut -d: -f1)
if [ -z "$pid" ]; then
tmux new -d -s $pid
fi
exec tmux attach -t $pid
fi
Append to bottom of your remote server's ~/.bashrc, (or possibly its /etc/.bashrc.shared (1)):
# ======================== PUT THIS LAST IN .BASHRC ==========================
# --- If we're run by SSH, then auto start `tmux` and possibly re-attach user.
# $- interactive only via current option flags
# -z $TMUX no tmux nesting
# $SSH_TTY SSH must be running, and in a shell
#
if [[ $- == *i* ]] && [[ -z "$TMUX" ]] && [[ -n "$SSH_TTY" ]]; then
tmux attach-session -t "$USER" || tmux new-session -s "$USER" && exit
fi
Many good tips above combined here, e.g. $- and $SSH_TTY are better I think.
And I like adding a few comments to help this old guy remember what's going on without having to look it up.
And finally, I like an exit at the end to cleanly come home when I'm done.
Thanks everyone.
Note I source a shared /etc/.bashrc.shared at the end of both user and root's .bashrc's, for common stuff used in both, like colorized ls, various aliases, functions and path extensions, i.e. I don't want redundant code in my root/.bashrc nor user/.bashrc.
This guys script works great. Just copy the bashrc-tmux file to ~/.bashrc-tmux and source it from ~/.bashrc right after the PS1 check section.

GNU Screen: create or attach to a session AND source a file

Using "screen -D -R -S foo", one can attach to an existing session named "foo", or if said session doesn't exist, create it.
How does one also source a file that contains screen commands?
I thought that this would work:
screen -D -R -S foo -X source file
Unfortunately, that fails with this message:
No screen session found.
EDIT: As zebediah49 pointed out in a comment, I left out the "source" in "-X source file" by mistake. Updated now.
OK, from a close reading of the man page I note:
-X Send the specified command to a running screen session. You can
use the -d or -r option to tell screen to look only for attached
or detached screen sessions. Note that this command doesn't work
if the session is password protected.
running screen session. In other words, I don't believe you can do what you're looking for like that, with only one command. However, you can
create the window if it does not exist
send the command to the window
connect to the window:
NL=$'\n'
NAME=foo
screen -ls | grep "$NAME" || screen -d -m -S "$NAME"
screen -r "$NAME" -X stuff "source file$NL"
screen -D -R -S "$NAME"
(Clarification of how -X works, from Send commands to a GNU screen )

How to create a screen executing given command?

i'm fairly new in *nix. Is there a way to create a screen, which will immediately execute a given command sequence (with their own arguments)? Two hours of googling yields nothing - perhaps because I can't
clearly state the question.
I hope for something like
screen -dmS new_screen exec "cd /dir && java -version"
I am using screen v4.00.03 and CentOS 5.5 (kernel ver. 2.6.18-194.26.1.el5.028stab079.2)
You create a screen with a name and in detached mode:
screen -S "mylittlescreen" -d -m
Then you send the command to be executed on your screen:
screen -r "mylittlescreen" -X stuff $'ls\n'
The stuff command is to send keystrokes inside the screen. The $ before the string command is to make the shell parse the \n inside the quotes, and the newline is required to execute the command (like when you press enter).
This is working for me on this screen version:
$ screen -v
Screen version 4.00.03jw4 (FAU) 2-May-06
Please see man screen for details about the commands.
The problem is that using the 'exec' screen command does not start a shell. 'cd' is a shell builtin, so you need a shell for it. Also, you need a shell that remains running so that screen does not terminate.
You can use the -X option to screen to send commands to a running screen session, and the 'stuff' command to send keystrokes to the current window. Try this:
screen -dmS new_screen sh
screen -S new_screen -X stuff "cd /dir
"
screen -S new_screen -X stuff "java -version
"
Yes, you need to put the quotes on the next line in order for the commands to be executed.
screen -dmS screen_name bash -c 'sleep 100'
This will create new screen named screen_name. And inside the screen it will sleep for 100 seconds.
Note that if you type some command in place of sleep 100 which terminates immediately upon execution, the screen will terminate as well. So you wont be able to see the screen you just created
I wanted to launch remote screens from within a bash script with some variables defined inside the bash script and available inside screen. So what worked for me was
#!/bin/bash
SOMEVAR1="test2"
# quit existing if there is one running already, be careful
screen -D -RR test1 -X quit || true
screen -dmS test1
screen -r test1 -p 0 -X stuff $"echo ${SOMEVAR1} ^M"
Where the return character, ^M, you need to enter using vim as
i CTRL-V ENTER ESCAPE
Another approach
First line cd to the your directory.
Second line start a new screen session named new_screen with bash.
Third line executing java -version
cd /dir
screen -dmS new_screen bash
screen -S new_screen -p 0 -X exec java -version
I think that you can use this
function exec_in_screen() {
name=$1
command=$2
screen -dmS $name sh; screen -S $name -X stuff "$command\n";
}
Then...
exec_in_screen "test" "ls"
Yes, what you want is the "stuff" command
for example
screen -dmS new_screen -X stuff "cd /dir && java -version
"
the second quote is on the next line so that it executes when sent

how do you script gnu screen from within a screen session to open new windows and run commands in them?

From within a screen session, I'd like to run a shell script that opens
a few new screen windows in the same session and start running some
programs in them.
I need a script like this:
screen -t newWindow
[switch to newWindow and execute a command]
screen -t newWindow2
[switch to newWindow2 and execute a command]
I don't know how to accomplish the effect I describe in the brackets.
Any clues? Please note that this is not a script I'll be running to start a screen session. I need this script to be runnable within an existing screen session, in order to add new windows to the session.
Note: you can't launch script working following way from a screen session. And it will open in session no tabs... Its more a related tip than a real answer to the question.
There is an other solution, if you accept to have a screen session by running process...
new session script
#!/bin/sh
echo "nouvelle session screen ${sessionName}"
screen -S ${sessionName} init.sh
echo "screen session: done"
echo "go to ${AnyWhere}"
sleep 1
screenexec ${sessionName} "cd ${AnyWhere}"
init script (here "init.sh")
#!/bin/zsh
zsh -c "sleep 0.2"
screen -d #detach the initialised screen
zsh #let a prompt running
injection script (here screenexec)
#!/bin/sh
# $1 -> nom de screen cible $2 -> commande
echo "injection de «${2}» dans la session «${1}» ..."
screen -x "$1" -X stuff "$2" #inject the command
screen -x "$1" -X eval "stuff \015" #inject \n
echo "Done"
By using this way, you should inject code easily in your screens, interesting if your script act like a deamon...
For those who prefer script in python, I've made a small lib to create sessions, close sessions, inject commands: ScreenUtils.py
It's a small project, which don't handle multiwindows screen sessions.
Forgot to mention I made a real python library out of it long ago: https://github.com/Christophe31/screenutils
Running this script inside screen does what I think you want:
#!/bin/bash
screen vi
screen top