My question is when using lftp to download file from a sftp site with automating password :
lftp -e 'set xfer:clobber on; mget -c one_file' -u username,passwd sftp://hostname
lftp still prompt "Enter your PASSWORD:" to ask me the password. My operating system is Windows xp. Can anybody tell me what is problem ?
Background:
I want to transfer large files automatically from a sftp site when some condition is met. I find lftp is a good tool to transfer large files because it can resume the transfer in background when the link is disconnected due to the instable net connect status. I use python on Windows to control lftp to download the large files. But it seems to automating password in lftp is failed. I also try the expect-like method, such as winpexpect, but it seems can't solve this problem. However, I find others use lftp -u to automate password for sftp site when doing google searching, why it failed here ?
Thank you in advance!
(Answered in a question edit. Converted to a community wiki answer. See What is the appropriate action when the answer to a question is added to the question itself? )
The OP wrote:
I have solved it.
I find the ssh used by lftp is not the one in the directory of lftp, but another one (ssh under the directory of mingw) because the route of mingw in PATH is prior to lftp. I move the route of lftp in front of mingw in PATH variable, then the problem is solved.
Related
Hi I can login to the GCE VM with WinSCP using my own username, cannot login as root...this is by default according to Google, and can be changed.
Changed like this:
Step 1: Login SSH and Su Root
# sudo su root
Step 2: Change password Root
#passwd root
Step 3: Config SSHD allow Root login
#nano /etc/ssh/sshd_config
PermitRootLogin yes
PasswordAuthentication yes
#service sshd restart (I used ssh as I'm using ubuntu and sshd wouldn't work)
Tried to login as root via WinSCP but I get
"Received too large (1349281121 B) SFTP packet. Max supported packet
size is 1024000 B. The error is typically caused by message printed
from startup script (like .profile). The message may start with
'Plea'." Cannot initialize SFTP protocol. Is the host running a SFTP
server?"
Any ideas?
Received too large SFTP packet. Max supported packet size is 102400 B
Cause:
This problem can arise when your .bashrc file is printing data to the screen (e.g.archey, screenfetch). The .bashrc file runs every time any console shell is initialized.
Solution:
Simply move any scripts that generate output from your .bashrc file to your .bash_profile. The .bash_profile only runs when you create a physical shell session.
NOTE: Just for anyone who comes across this and simply wants to copy files and doesn't matter what file protocol they use. You can just switch file protocol from SFTP to SCP to avoid this issue. Thought it might be worth a mention.
If you used Ubuntu linux and try to connect the server then "Please login as the Ubuntu user" you should sftp as the ubuntu user, not as root.
Try that, hope it will work for you!
Thanks!
Hmmm, I added this in WinSCP in advanced settings under "protocol options":
sudo /usr/lib/openssh/sftp-servers
I can login with my own username and move files now. Although not exactly sure how this works, I think it somehow changes you to root user at login?
More info: https://winscp.net/eng/docs/faq_su
See WinSCP article on Received too large (... B) SFTP packet. Max supported packet size is 102400 B
If … (from the subject [error message]) is a very large number then the problem is typically caused by a message printed from some profile/logon script. It violates an SFTP protocol. Some of these scripts are executed even for non-interactive (no TTY) sessions, so they cannot print anything (nor ask user to type something).
To add to #ThatOneCoder's answer on the cause being too much output from .bashrc: in e.g. Ubuntu, there is also the system wide /etc/bash.bashrc that might be "too wordy" and cause the Received too large SFTP packet error.
It's a "system wide .bashrc", and if you want to execute code for all logging in, that's one location to place it. If you nixed ~/.bashrc and still get the error, check the contents of /etc/bash.bashrc.
It is happening because you haven't given shell access permission to the user.
I faced the same issue trying to login on my ubuntu 16.04 EC2 server as "root" via WinSCP. I spent a lot of time trying to fix it but in the end a simple workaround worked for me.
I ssh into the instance using PuTTY with the username "ubuntu". After this I typed
sudo -i
and with this the user was changed to root.
this has been asked again and again, but none of the solutions I found actually works for me. I'm testing a new server (Ubuntu server 14.04) and have gone through the whole installation process of the various required software. So far I can access my internal web page via
http://myInternalIP/wordpress/
I added there a dummy post and it looks ok.
Now I wanted to add a plug-in, but I'm having major trouble with that.
So here is what I have done.
I added a new user called ftps that has it's home dir in
/usr/share/wordpress/
and this is part of
~$ groups ftps
ftps : ftps www-data
When I try to add a plug-in, all goes well until I get the following message:
Downloading install package from https://downloads.wordpress.org/plugin/wordpress-importer.0.6.1.zip…
Unpacking the package…
Could not create directory.
Return to Importers
So the general answer I found in many posts, is that this is a permission issue. Fine. Well I'm fighting with the permission issues since xx hours. So here is a brief summary of what I've done:
I've tried changing ownerships and groups around (www-data, my user name, ftps). It did not work.
I've changed permissions to 777 to all the wordpress directory in /usr/share/wordpress.
I've tried the following commands:
sudo -u helder touch /usr/share/wordpress/wp-content/plugins/test.txt
sudo -u ftps touch /usr/share/wordpress/wp-content/plugins/test.txt
sudo -u www-data touch /usr/share/wordpress/wp-content/plugins/test.txt
All of these commands generated a file succesfully in the specific directory.
My feeling is that permissions are not the issue, but I might be wrong... what should I look out for?
Thanks
I had to enable write_enable=YES in the file vsftpd.conf.
If you are using vsftpd as your FTP server and have enabled passive connections, you need to add pasv_promiscuous=YES to /etc/vsftpd/vsftpd.conf.
I'm trying to edit files on a remote Amazon EC2 Linux instance. I'm currently just sshing in and using nano, but would really like a graphical text editor. I have two problems:
I have to use sudo to edit these server files when I ssh in.
I can only login with the key Amazon gave me. Ex: ssh -i Andrew.pem ec2-user#55.55.44.33
Please help! I'm not picky, just any graphical text editor since using nano is a huge pain.
For remote editing, there are lots of options here: This answer, like any other, is sure to become outdated as more options enter the field.
For vim, the netrw module meets this need, and is shipped with the editor by default.
For emacs, this is available with TRAMP.
For the ATOM editor, see the remote-files plugin.
For IntelliJ, editing files on remote hosts is supported in the commercial edition.
For Eclipse, see the Remote System Explorer from the Target Management project.
I'd suggest starting with the editor you prefer and evaluating options from there. If you set up your SSH session to be able to authenticate directly to root (password auth is best disabled for root, but if you have sudo you can install RSA keys), then you'll be able to specify root as a target user for any of the above.
By contrast, if you really do need sudo, you still have options:
See Using tramp to open files sudoed to root on the Emacs wiki. New versions also support a ssh+sudo transport, meaning this wiki entry may already be out-of-date.
To help anyone that just need a quick command line text editor:
you can use vi:
vi file-name.txt
or nano:
nano file-name.txt
optionally use sudo if editing the file, eg:
sudo nano file-name.txt
Just modify the appropriate files on your local machine and scp the file into the remote machine.
scp <local_machine_path_to_file> remoteUser#remoteHostName:<filePath>
amazon now acquire Cloud9, which is a browser-based IDE that can edit your EC2.
https://aws.amazon.com/cloud9/
Today I found two products that can use sudo, they are
MobaXterm (free version) and SmarTTY
MobaXterm has a button in the file browser that enables sudo mode. You can view, create and edit files as a sudo user. Use this switch when necessary.
Unfortunately, this only works through the SCP protocol.
SmartTTY works differently. When you try to save a file that requires sudo, SmarTTY throws an error and immediately suggests trying to save the file with sudo
Of the two products, I recommend MobaXterm.
Sudo is for root privileges for that particular command. You will need to use root privileges to edit system files. Even on a local machine. If you don't like typing sudo every time, you can type sudo -s. You will change to root user and it will show you in terminal i.e. root#ip.... The $ sign will also change to #. Honestly, I prefer not going root, because it is easier to make irreversible mistakes with root privileges. I've made some mistakes and I'm talking from experience...
As far as the second part of your question goes, you can configure various text editors to sftp into your instance such as sublime.
You will have to use the .pem key file every time you ssh using terminal. This is because AWS takes security very seriously. You can put the key file in your home directory. That way you don't have to change directories every time you open up terminal.
You can also edit a local copy of files and then use FileZilla to transfer. Setting up FileZilla to work with your EC2 instance is straightforward. You can give vim a try since it colors your code and is more advanced than nano. Use the command vi or vim from terminal.
Happy SSH'ing ;).
ssh -X user#server.
You have to make appropriate setting for forwarding.
I use SFTP Net Drive SFTP, which let you create a virtual drive on your local computer that will map the remote file system accessible via SFTP protocol. After the map is created, you can use the editor of your preference.
You can use nano, vim, vi or many others. However if you want to edit with a graphical text editor you will have to create SFTP since Amazon does not support FTP. One way is to use filezilla to upload your files. Here is a video on using filezilla https://www.youtube.com/watch?v=VawBMj29g0o I suggest SSH though. Its fast and easy here is a video on that https://www.youtube.com/watch?v=O2-3HoRjBH4
I found a weird workaround for a GUI based text editor on AWS, I used Jupyter Notebook. If you have Anaconda installed on you instance, you follow the following steps
ssh onto your instance using ssh -i <location of your private key> <username>#<public DNS>
Start jupyter notebook on your instance using jupyter notebook --no-browser --port=8888
Open a new terminal window and ssh onto your jupyter notebook using ssh -i <location of your private key> -L 8212:localhost:8888 <username>#<public DNS>
Now you can open jupyter notebook at localhost:8212
Using the jupyter notebook environment, you can not only launch and run Ipython notebooks but also create and edit any files like a text editor.
would really like a graphical text editor
You cannot have a graphic editor, you need to use any editors like nano as you said or vim,emacs. Sudo would be required when you have to edit configuration files with root as owner.
To assist others with this same question, I would suggest jEdit. It is very capable, and it has a very rich plugin environment, language parsing, etc.
http://www.jedit.org
It has "always" supported sftp read and write of files with the sshConsole plugin.
I use it now on my AWS EC2 instance with the key pair supplied by AWS.
Lastly, it is not a good idea to edit files owned by root in the "production" environment.
Do your dev work in the AWS user's home folder so that you have full control of the source files. Then use a symlink to the actual server's file tree so you can serve it to yourself for testing. There are lots of controls in nginx and apache to limit who can view your dev site.
EDIT/UPDATE:
The NppFtp plugin to Notepad++ profides sftp access to AWS. I just tested it with the .pem file that they provided for my login at AWS.
For this, i'd suggest one of:
Learn and use emacs; it's quite powerful as far as textmode editors go.
Install your favourite graphical editor on the server and use X forwarding, 'ssh -X server.com'. This will allow you to launch the editor remotely, but have it display locally.
Most elegant in my opinion, use sshfs (https://github.com/libfuse/sshfs) to mount the remote directory locally, so you can work on the files directly using your favourite text editor.
I need to copy a file from a remote machine to my local machine and I need to automate it.
I've tried SCP command and it's working, however, I could not automate the part wherein it is asking for the password of the user of the local machine and the remote machine.
Based on this article I can Perform SSH Login Without Password Using ssh-keygen & ssh-copy-id
after following all the instructions written there, I tried to access the remote machine using this
ssh lalala#XXX.XXX.XXX.XXX
it works, it doesnt ask for the password anymore. But when I tried copying a file from that machine using the command below,
scp lalala#XXX.XXX.XXX.XXX:'/a/b/c.txt' lelele#XXX.XXX.XXX.YYY:'/b/c/'
it still asks for the password of the localmachine which is the lelele#XXX.XXX.XXX.YYY
I wonder if I did something wrong? what could it be? is there something wrong with the format of the command?
BTW, im using Centos, and I'm planning to code it using python
If you are copying to your local machine why don't you just do
scp lalala#XXX.XXX.XXX.XXX:'/a/b/c.txt' /b/c/
?
I tried your line on some machine with similar setup and didn't get asked for password; I got an error instead, but this is probably due to differences in our configurations. I tried mine and it worked.
Regarding whether your connection succeeds in the remote machine you could tail this file there:
tail -f /var/log/secure
If you see no error there you can be sure (well, never say always) your layout with the generated keys is working.
In this case I bet you'll see no error there
I think you may have multiple ssh keys and set identies only as yes. If so, please check this answer: https://askubuntu.com/a/999306/398861
I am developing a project and that requires ldap validation. But, I don't have a developer/qa ldap server.
Does a small ldap server exist for windows for testing/development?.
I just want to test to validate a active account and detect if it is blocked or not, so i don't want to install a whole domain to do that.
---never mind---
I tried an compiled openldap but I was unable to understand it. Simply, I don't get how to connect to it, how to create a account and how to validate, the client ldap returned me some obfuscate error message, i tried several ways to do it and finally i give up.
Finally, i installed a domain, it was absurdly easy to install (2008 r2), restart the server and that's it.
Anyways, thanks for the advice of opendlap and aldps
If you're on Windows and use Active Directory, have a look at Active Directory Lightweight Directory Services (AD LDS) - a LDAP server you can install and use on your dev machine.
The open source LDAP server from OpenLDAP should give you what you need:
http://www.openldap.org/
Apache provide a directory server called "ApacheDS"(Apache Directory Server), and it provides a GUI management client called "Apache Directory Studio" which is based on Eclipse.
If you want to have a test only, this studio provides a built-in server for your test, easy to link.
You can also install the studio directly in Eclipse using this update site: http://directory.apache.org/studio/update/2.x/
Active Directory works fine as an LDAP server and its included in the Windows Server 2008 trial. See the answer to my question Testing LDAP Connections to Active Directory Server. I have it running in a KVM virtual machine on Linux and query it from an OpenLDAP based client.
Necromancing.
I've had the same problem.
OpenDS is very easy to get up and running, and doesn't require administrator rights.
You just need to download the ZIP file and run the installer.
The installer can populate the directory with test entries, too - if you want to see some example data.
That's exactly what you're looking for when wanting a simple dev test server.
Note:
OpenDS development has seized, and was forked into OpenDJ, a commercial project by forgerock.
While OpenDS still works on Java7, only OpenDJ will work with Java8.
However, OpenDJ is still FREE and OpenSource.
You can find the sourcecode here on Bitbucket
and you can grab it with git:
git clone https://stash.forgerock.org/scm/opendj/opendj.git
Forget OpenLDAP and AD-LDS; these are way too complicated for simple testing.
In addition, their user interface is horrible, and you need something that you can get up and running FAST, without admin rights, and have it populated with test data in a few minutes, not in a few weeks.
And ApacheDS will require administrator privileges, unfortunately (because it only works as windows service, and you can't start/stop these without being administrator).
So OpenDJ is the definite way to go.
Apache Directory Studio is a good client to browse, edit and import/export data via LDAP (LDIF).
However, despite Apache Directory Studio being written in Java, it adds a dependency to gtk, and only has binaries for x86/x64, which means it won't work on a Chromebook with ARM processor, or on a RaspberrryPI.
But with the test entries added automagically in OpenDJ/OpenDS (if you choose the option), you don't even need that.
When in doubt, use a web based interface that "talks LDAP".
Try OpenDS it is very simple and requires only Java.
You could roll your own LDAP server for testing pretty easily using godap: https://github.com/bradleypeabody/godap
It's written in Go. It's very small and simple.
You would basically need to copy the server example out of godap_test.go and wire it up however you need.
Try simple-ldap-server
I know its pretty late to answer this question. But for the reference of someone who runs into the same question.
I wrote a simple ldap server(using ldapjs on nodejs) for authentication testing purposes. Please feel free to use it. It's easy to configure. Can support both LDAP/LDAPS protocols, just require a json file including the user ids you want to add(or it comes with a pre-included users json file, which you can use if you want).
The project is on github. (I'll add a docker image too)
Feel free to visit and use
Docker image
Simple Ldap Server Git
OpenLDAP. Ships with most Unixes and Linuxes. For Windows it is available from several sources:
Cygwin
http://www.userbooster.de
as the Silver (free) edition of the CDS product http://www.symas.com/cds.shtml. This is crippled compared to the Userbooster version, which is complete.
You can use a Docker container with Samba as Domain controller, here I show how to setup one in just a few minutes
Basically you need to
Create an image with this (read the post if you want to know why)
$ git clone https://github.com/padiazg/alpine-samba-ad-container.git
$ cd alpine-samba-ad-container
# replace your-user with your username
$ docker build -t your-user/alpine-samba-ad-container .
Create some folders and files to persist the container data
mkdir /tmp/krb-conf
&& mkdir /tmp/krb-data
&& mkdir /tmp/smb-conf
&& modir /tmp/smb-data
&& touch /tmp/krb-conf/krb5.conf
Run the container
docker run -d \
-e SAMBA_ADMIN_PASSWORD=a-secure-password \
-e SAMBA_DOMAIN=local \
-e SAMBA_REALM=local.your-domain.io \
-e LDAP_ALLOW_INSECURE=true \
--mount type=bind,source=/tmp/krb-conf/krb5.conf,target=/etc/krb5.conf \
--mount type=bind,source=/tmp/krb-data,target=/var/lib/krb5kdc \
--mount type=bind,source=/tmp/smb-conf,target=/etc/samba \
--mount type=bind,source=/tmp/smb-data,target=/var/lib/samba \
-p 389:389 \
--name smb4ad \
your-user/alpine-samba-ad-container
And now you are good to go