I'm trying to automate a script that copies a file from my local server to a remote server on the command line. I've done the research on scp and know how to copy the file to the remote server, but then I want to append that file to another.
This is my code:
scp ~/file.txt user#host:
ssh user#host cat file.txt >> other_file.txt
When I enter everything into the command line manually as such, everything works fine:
scp ~/file.txt user#host:
ssh user#host
cat file.txt >> other_file.txt
But when I run the script, only the file is copied, not appended to the end of other_file.txt. Help?
The second line of your code should be
ssh user#host "cat file.txt >> other_file.txt"
Three important points:
You don't want your local shell to interpret >> in any way (which it does if it's unquoted)
There is a remote shell which will interpret >> in the command correctly.
Final arguments to ssh are "joined" to form a command, not carried into an argv array as they are. It may be convenient but it also may lead to confusion or bugs: ssh cat "$MYFILE" and ssh "cat '$MYFILE'" both work in a common use case, but they both break for different values of $MYFILE.
You need to enclose the command to be run on the remote host in quotes. Otherwise, the redirection is being done locally rather than remotely. Try this instead:
scp ~/file.txt user#host:
ssh user#host 'cat file.txt >> other_file.txt'
Try this:
$ cat file.txt| ssh hostname 'cat >> other_file.txt'
Related
I have the following running one liner to delete last injected ED key from last line of authorized keys folder of a remote host.
sed -i '${/^\(ssh-ed25519\) \(.*\) \([A-Za-z][A-Za-z]*\.[A-Za-z][A-Za-z]*#hpe\.com\)$/d;}' .ssh/authorized_keys
I do a couple of operations after creating ED key and injecting it into remote host. Then I want to clear up as tear down step. However when I tried to run it by connecting to remote host it failed. I tried the following things:
ssh -T 'sh -c "sed command here"' # too many single and double quotes, so I gave up
Then I tried removing T terminal and as well as "sh -c" command running part
ssh root#my_remote_host "sed -i '${/^\(ssh-ed25519\) \(.*\) \([A-Za-z][A-Za-z]*\.[A-Za-z][A-Za-z]*#hpe\.com\)$/d;}' .ssh/authorized_keys"
had this error:
sed: couldn't open temporary file .ssh/sedWC1YAQ: Read-only file system
I referred to this link but it also didn't help : Using SED in a ssh command on a remote node
I appreciate any help, and let me know if what I'm trying is a good way to apply.
I have about a thousand files on a remote server (all in different directories). I would like to scp them to my local machine. I would not want to run scp command a thousand times in a row, so I have created a text file with a list of file locations on the remote server. It is a simple text file with a path on each line like below:
...
/iscsi/archive/aat/2005/20050801/A/RUN0010.FTS
/iscsi/archive/aat/2006/20060201/A/RUN0062.FTS
/iscsi/archive/aat/2013/20130923/B/RUN0010.FTS
/iscsi/archive/aat/2009/20090709/A/RUN1500.FTS
...
I have searched and found someone trying to do a similar but not the same thing here. The command I would like to edit is below:
cat /location/file.txt | xargs -i scp {} user#server:/location
In my case I need something like:
cat fileList.txt | xargs -i scp user#server:{} .
To download files from a remote server using the list in fileList.txt located in the same directory I run this command from.
When I run this I get an error: xargs: illegal option -- i
How can I get this command to work?
Thanks,
Aina.
You get this error xargs: illegal option -- i because -i was deprecated. Use -I {} instead (you could also use a different replace string but {} is fine).
If the list is remote, the files are remote, you can do this to retrieve it locally and use it with xargs -I {}:
ssh user#server cat fileList.txt | xargs -I {} scp user#server:{} .
But this creates N+1 connections, and more importantly this copies all remote files (scattered in different directories you said) to the same local directory. Probably not what you want.
So, in order to recreate a similar hierarchy locally, let's say everything under /iscsi/archive/aat, you can:
use cut -d/ to extract the part you want to be identical on both sides
use a subshell to create the command that creates the target directory and copies the file there
Thus:
ssh user#server cat fileList.txt \
| cut -d/ -f4- \
| xargs -I {} sh -c 'mkdir -p $(dirname {}); scp user#server:/iscsi/archive/{} ./{}'
Should work, but that's starting to look messy, and you still have N+1 connections, so now rsync looks like a better option. If you have passwordless ssh connection, this should work:
rsync -a --files-from=<(ssh user#server cat fileList.txt) user#server:/ .
The leading / is stripped by rsync and in the end you'll get everything under ./iscsi/archive/....
You can also copy the files locally first, and then:
rsync -a --files-from=localCopyOfFileList.txt user#server:/ .
You can also manipulate that file to remove for example 2 levels:
rsync -a --files-from=localCopyOfFileList2.txt user#server:/iscsi/archive .
etc.
I am trying to do something like this inside ssh session:
Script
ssh remoteservername "
col=`sed -n "8p" /tmp/temp.txt`
echo $col>>/tmp/Ankur.txt
"
This is not working and it is printing empty line instead of text what I want to store in col variable, why so, and this is working:
ssh remoteservername "
sed -n "8p" /tmp/temp.txt>>/tmp/Ankur.txt
"
This Ankur.txt file is on the remote server....The main focus is how to get the output of the command inside a variable so that i can use it further.
Please tell how to make it work.
Thanks
When you use double quotes the variable names will get expanded before passing them, so $col is getting expanded locally before running on the remote server. You can either escape the $ like \$col or use single quotes around it, which is probably better since you want to use double quotes inside the command as well
ssh remoteservername '
col=`sed -n "8p" /tmp/temp.txt`
echo $col>>/tmp/Ankur.txt
'
Without changing the quotes
ssh remoteservername 'sed -n "8p" /tmp/temp.txt >> /tmp/Ankur.txt'
as you noted, still works, by redirecting the output directly into the file. This avoids the variable expansion problem from the double quotes above.
If you're going to have many steps though, you might want to just create a script on remoteservername and invoke that in your ssh command rather than doing a lot on the same command line.
You can use a local file to execute complex commands and to use variables in remote machine via SSH as shown below.
1. Create a input file 'input_file.txt'
#-- input_file.txt
col=`sed -n "8p" /tmp/temp.txt`
echo $col>>/tmp/Ankur.txt
2. Execute the commands of input file in remote server via SSH
ssh remoteservername "sh -s" < input_file.txt
What does the ssh-copy-id command do, exactly? I've used it numerous times and it works great. However, when I try to manually cut and paste my .pub keyfile to my remote authorized_keys, it doesn't work.
I've compared the contents of my authorized_keys file where I've cut and pasted the .pub into it vs subsequently using ssh-copy-id and I'm not seeing any differences between the two, including whitespace.
Is there anything that ssh-copy-id does beyond copying the public key into authorized_keys?
This little one liner script works on sh, bash, and zsh. I use it every time there is no ssh-copy-id, for example when I'm on older version of OSX.
cat ~/.ssh/id_rsa.pub | ssh <user>#<hostname> 'cat >> ~/.ssh/authorized_keys'
How it works
I am sending the public keay to the Unix standard output (STDOUT) using the cat command. I then connect the STDOUT of cat to the standard input (STDIN) of the ssh.
The ssh executes the cat command on the server. Remember that the we have our key in the STDIN now? This key gets passed from ssh to the cat command executed on a server. The >> operator redirects the STDOUT of the cat to the end of the ~/.ssh/authorized_keys file. This way the key from public keys is appended to the authorized_keys on the server.
IMO It's better than manual copying and pasting: in this case you know exactly what content will end up in the file
I usually copy-paste keys into authorized_keys as you describe (I forget about ssh-copy-id), so it can work. Note thatchmod 600 ~/.ssh/authorized_keys is required if you're creating the file.
ssh-copy-id is a shell script so you can open it in a text editor to see what it does, this looks like the relevant bit:
printf '%s\n' "$NEW_IDS" | ssh "$#" "
umask 077 ;
mkdir -p .ssh && cat >> .ssh/authorized_keys || exit 1 ;
if type restorecon >/dev/null 2>&1 ; then restorecon -F .ssh .ssh/authorized_keys ; fi"
restorecon in the last line restores default SELinux security contexts. I haven't had to run that, but it might be necessary in your case.
I want to be able to just ssh to a server where I cannot modify profiles and set up the environment with several commands before getting the usual interactive session.
Any ideas?
I've been using an expect script with an "interact" command at the end - which works for most things but is clumsy and breaks some console apps. Also been extermienting with empty-expect and socat. Any other suggestions?
If you're able to write somewhere on the filesystem, you may be able to invoke bash with a custom rc file like this:
ssh me#example.com -t bash --rcfile /home/user/my_private_profile -i
Note that this appears to only work for interactive shell, not login shells. The -t option to ssh makes it allocate a pty even though you're specifying a command.
If you can't write to the filesystem anywhere, you could use a subshell to supply a named pipe as the rcfile:
$ ssh ares -t "bash --rcfile <(echo 'FOO=foo';echo 'BAR=bar') -i"
axa#ares:~$ echo $FOO
foo
axa#ares:~$ echo $BAR
bar