I have a script that uses SCP to pull a file from a remote Linux host on AWS. After running the same code nightly for about 6 months without issue, it started failing today with protocol error: filename does not match request. I reproduced the issue on some simpler filenames below:
$ scp -i $IDENT $HOST_AND_DIR/"foobar" .
# the file is copied successfully
$ scp -i $IDENT $HOST_AND_DIR/"'foobar'" .
protocol error: filename does not match request
# used to work, i swear...
$ scp -i $IDENT $HOST_AND_DIR/"'foobarbaz'" .
scp: /home/user_redacted/foobarbaz: No such file or directory
# less surprising...
The reason for my single quotes was that I was grabbing a file with spaces in the name originally. To deal with the spaces, I had done $HOST_AND_DIR/"'foo bar'" for many months, but starting today, it would only accept $HOST_AND_DIR/"foo\ bar". So, my issue is fixed, but I'm still curious about what's going on.
I Googled the error message, but I don't see any real mentions of it, which surprises me.
Both hosts involved have OpenSSL 1.0.2g in the output of ssh -v localhost, and bash --version says GNU bash, version 4.3.48(1)-release (x86_64-pc-linux-gnu)
Any ideas?
I ended up having a look through the source code and found the commit where this error is thrown:
GitHub Commit
remote->local directory copies satisfy the wildcard specified by the
user.
This checking provides some protection against a malicious server
sending unexpected filenames, but it comes at a risk of rejecting
wanted files due to differences between client and server wildcard
expansion rules.
For this reason, this also adds a new -T flag to disable the check.
They have added a new flag -T that will ignore this new check they've added so it is backwards compatible. However, I suppose we should look and find out why the filenames we're using are flagged as restricted.
In my case, I had [] characters in the filename that needed to be escaped using one of the options listed here. for example:
scp USERNAME#IP_ADDR:"/tmp/foo\[bar\].txt" /tmp
Related
I want to copy files remotely in a script from windows machine to Linux machine.
On the Linux machine I run the below command
scp user#remotehost:\D\mySrcCode\somefile.cpp .
I am getting an error
scp: DmySrcCodesomefile.cpp: No such file or directory
The file somefile.cpp is located at D:\mySrcCode on windows side.
Any ideas on what I am missing ?
You probably should quote or backslash the backslashes in the path.
If your interactive shell is GNU bash, read its §3.1.2 quoting chapter.
You could try:
scp user#remotehost:\\D\\mySrcCode\\somefile.cpp .
Consider also using other (more appropriate) tools, like rsync or git.
You might also use exec(3) from your C program to run /usr/bin/ssh, or look into libssh.
You could change your login shell (see chsh(1) and /etc/shells so shells(5)) to more user friendly alternatives such as zsh or fish. They could give you some warning (depending on how they are configured or used) or some autocompletion (with the tabkey).
PS. Your problem is not ssh specific. You might replace scp with echo to understand it more.
I'm running Terminal on my Mac and ssh'ing into a Linux host. I've had 2 issues around copying and pasting text into my Terminal:
When I'm just running local Terminal commands on my Mac (not though ssh), I sometimes get weird characters around text that I paste, like 00~pastedtext01~. I googled around and found out that this was apparently due to "bracketed paste". I was able to fix this problem by running the command printf '\e[?2004l' whenever it happens.
When I ssh into my Linux host, when I paste text it will often capitalize the last character of whatever I'm pasting, and the cursor will turn gray and I won't be able to type additional characters or delete characters from whatever I pasted. My only two options at that point are to either ^C to break onto the next line without running the command, or hit enter and run the messed up command. It doesn't happen 100% of the time. If I copy something and then repeatedly paste it into the shell, I'll see this issue occur about 90% of the time. I have no idea why it's apparently non-deterministic like this. I thought this might also be due to "bracketed paste" issues, but no matter how many times I run the commands printf '\e[?2004l' and set enable-bracketed-paste off, the issue persists. It even persists when I exit and re-ssh to the host, so I know it's not due to running any program like vim, since the problem still happens even immediately after I've ssh'ed into the host. Can someone please help??? This is killing my productivity!
Here's what a paste with this issue looks like:
I'm aware that other questions have been asked along these lines, like this one from Stack Exchange, but none of the answers on any of these posts have worked for me, so I think my problem may be slightly different than those...
Check your ZSH config
cat ~/.zshrc
Check if in plugins=(...) you find safe-paste. If it's there, edit the config and delete safe-paste.
if grep -q "safe-paste" "~/.zshrc"; then # checks if str in contained in file
sed -e s/safe-paste//g -i ~/.zshrc # if so, it replaces the str with nothing
fi
As a one liner:
if grep -q "safe-paste" "~/.zshrc"; then; sed -e s/safe-paste//g -i ~/.zshrc; fi
docker-machine has an scp command, but docker-cloud doesn't seem to have any way to transfer a file from my local machine to the cloud container or vice-versa.
I'm submitting an answer below that I've finally figured out (in hopes that it will help someone), but I'd love to hear better answers if there are any!
(I realize docker-cloud is going away, but perhaps this will be helpful for other cloud platforms as well)
To transfer a file from your local machine to a docker-cloud instance that is running linux with the tee command available:
docker-cloud container exec id12345 tee filename.ext < file_to_copy.ext > /dev/null
(you'll want to redirect output to /dev/null as shown unless you want the entire contents of the file to be echoed to the terminal... twice)
To transfer a file to your local machine, is somewhat easier:
docker-cloud container exec id12345 cat file_to_copy.ext > filename.ext
Note: I'm not sure this works for binary files, and it can even cause issues with linefeed characters in text files, based on terminal settings, etc. - but it's the best answer I've got short of using an external service like https://transfer.sh
I'm attempting to pipe an email to PHP with my Postfix mail server, using the technique mentioned here and have encountered the following error...
Mar 16 22:52:52 s15438530 postfix/pipe[9259]: AD1632E84C63: to=<php#[myserver].com>, relay=plesk_virtual, delay=0.61, delays=0.59/0/0/0.02, dsn=4.3.0, status=deferred (temporary failure. Command output: /bin/sh: /var/www/vhosts/[myserver].com/httpdocs/clients/emailpipe/email2php.php: Permission denied 4.2.1 Message can not be delivered at this time )
I'd really appreciate if anyone could shed some light on this issue for me. I've tried 777'ing the emailpipe directory, to no avail. Where am I going wrong?
Many thanks.
From the postfix docs...
For security reasons, deliveries to command and file destinations are performed with the rights of the alias database owner. A default userid, default_privs, is used for deliveries to commands/files in root-owned aliases.
So you have two options, either set the default_privs in main.cf to match the ownership of the email2php file.
Alternatively, there should be a way to create an alias database that is owned by the user instead of postfix/nobody. I haven't tried this before though so can't advise.
I have fixed this issue by disabling the SELINUX.
Make sure that you have
#!/usr/bin/php
<?php
(or whatever your path to php is - do "which php" on the server)
at the top of each of your php scripts and that each of the php script files is executable
chmod +x /var/.../email2php.php
Also, make sure that you can test the script from the command line:
cat some_rfc822_email.txt | /var/.../email2php.php
and get the result that you want
To fix this issue, you'll want to chown or chmod /var/www/vhosts/[myserver].com/httpdocs/clients/emailpipe/email2php.php to executable by your postfix user. Alternately, you'll want to redefine this user to execute the file successfully.
Simply changing the permissions of your directory (unless you used -R) won't be sufficient.
To illustrate why this works, consider the following toy example:
<me>#harley:~$ touch test
<me>#harley:~$ ls -al test
-rw-r--r-- 1 <me> <me> 0 2012-03-26 23:44 test
<me>#harley:~$ sh test
<me>#harley:~$
<me>#harley:~$ ./test
bash: ./test: Permission denied
<me>#harley:~$ chmod 755 test
<me>#harley:~$ ./test
<me>#harley:~$
In order to execute a file directly through the running shell, it needs to be set as executable. Other invocations (for example, sh email2php.php or php email2php.php) only require read access, because they're chaining execution off a different file entirely.
For what's likely to be causing the issue in the first place, see here.
I am trying to run a GNU make file with multiple jobs.
When I try executing ' make.exe -r -j3', the receive the following to errors:
make.exe: Do not specify -j or --jobs if sh.exe is not available.
make.exe: Resetting make for single job mode.
Do I have to add ' $(SH) -c' somewhere in the makefile? If so, where?
The error message suggests that make cannot find sh.exe. The file names indicate you are probably on CygWin. I would investigate setting the PATH to include the location of sh.exe, or defining the value of SHELL to the name (or, even, full path) of your shell.
Are you running this on Windows (more specifically, in the "windows" shell?). If you are, you might want to read this:
http://www.gnu.org/software/make/manual/make.html#Parallel
more specifically:
On MS-DOS, the ‘-j’ option has no effect, since that system doesn't support multi-processing.
Once again, assuming you're running on windows, you should get MinGW or CygWin