I have a problem setting up a ipython cluster on a Windows server and connecting to this ipcluster using a ssh connection. I tried following the tutorial on https://ipython.org/ipython/doc/dev/parallel/parallel_process.html#ssh, but I have problems to understand what the options mean exactly and what parameters are to use exactly...
Could anyone help a total noob to set up an ipcluster? (Let's say the remote machine has ip 192.168.0.1 and the local machine has 192.168.0.2)
If you scroll roughly to the middle of the page https://ipython.org/ipython-doc/dev/parallel/parallel_process.html#ssh you will find this:
Current limitations of the SSH mode of ipcluster are:
Untested and unsupported on Windows. Would require a working ssh on Windows. Also, we are using shell scripts to setup and execute
commands on remote hosts.
That means, there is no easy way to build an ipcluster with ssh connection on windows (if it works at all).
Do you really need to connect the machines with an ssh connection? I guess it's possible with a ssh client on each windows machine, but if you are in a trusted local network you can also decide not to use the loopback interface and just expose the ports...
Sure you can start controller and engine separately! For further examples about ports (if you have problems with firewalls) see also How to setup ssh tunnel for ipython cluster (ipcluster)
Related
I am trying to follow the numerous tutorials and gists such as:
https://gist.github.com/creotiv/d091515703672ec0bf1a6271336806f0
https://stackoverflow.com/questions/48459804/how-can-i-ssh-to-google-colaboratory-vm/53252985#53252985
When I run the steps, it seems like everything went fine (I get the root password), but I do see this:
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of start.
but unfortunately, after all the steps when I do the following on my local machine:
ssh -p17057 root#0.tcp.ngrok.io
I get:
ssh: connect to host 0.tcp.ngrok.io port 17057: Connection timed out
I am on vanilla Debian Buster - any pointers to why this is happening would be incredibly useful to debug
thank you.
I also had trouble ssh'ing into the ngrok tcp tunnel. I was using my local laptop as the access point to the colab VM. What I did was fire up an EC2 instance on AWS and use that instead. Also, I used ssh reverse tunnel and dropped the need for ngrok altogether since the ec2 machine already had a public IP. Check my answer here: https://stackoverflow.com/a/63186681/4126114
I'm working on a project that requires me to run my code on a remote Unix server, that is not available to connect to directly (you first have to log in to the "gate" node and then to this server).
What's really bad is that they disabled key authentication, so each time I need to ssh into it, I have to type in my password twice. It's really annoying and I wonder what's the best way to transfer my local modifications of source files to this server, compile and run them without having to provide those passwords so many times.
I have no sudo access to any of those servers (neither to this "gate", nor to this target server). Any ideas on how to make the whole process more efficient?
EDIT: Martin Prikryl provided a great answer below, but it's suitable for Windows and I'm on a Mac :) I guess it might be a good thing to have it documented here also for *NIX systems.
You are looking for SSH tunneling.
WinSCP SFTP client supports one-hop SSH tunneling natively.
See the Tunnel page on WinSCP Advanced Site Settings dialog.
I assume that after you transfer the file, you need to open SSH terminal to compile the file.
You may be able to make use of WinSCP Console window for that step.
Alternatively, if you need/want to use a real SSH terminal client, make use of an existing SSH tunnel, created by WinSCP, and connect with PuTTY (or any other SSH client) over it.
In the Local tunnel port of WinSCP Tunnel page, select a fixed port number (instead of the default Autoselect). In PuTTY enter "localhost" to Host Name and the selected port in Port.
(I'm the author of WinSCP)
I want to install a number of raspberry pis at remote locations and be able to log in to them remotely. (Will begin with 30-40 boxes and hopefully grow to 1000 individual raspberry pis soon.)
I need to be able to remotely manage these boxes. Going the easier route, forwarding a port on the router and setting a DHCP reservation, requires either IT support from the company we'll be doing the install for (many of which don't have IT), or it will require one of our IT people physically installing each box.
My tentative solution is to have each box create a reverse SSH tunnel to our server. My question is: How feasible would this be? How easy would it be to manage that many connections? Would it be an issue for a small local server to have 1000+ concurrent SSH connections? Is there an easier solution to this problem?
My end goal is to be able to ship someone a box, have them plug it in, and be able to access it.
Thanks,
w
An alternate solution would be to:
Install OpenVPN server on your server machine. How to install OpenVPN Server on the PI. Additionally, add firewall rules that block everything but traffic directed for the client's ssh and other services ports (if desired), from administrating machine(s).
Run OpenVPN clients on your Raspberry PI client machines. They will connect back to your VPN server. On a side note, the VPN server and administrating machine(s) need not be the same machine if resources are limited on the VPN server. How to install OpenVPN on the client Raspberry PIs.
SSH from administrating machine(s) to each client machine. Optionally, you could use RSA authentication to simplify authentication.
Benefits include encryption for the tunnel including ssh encryption for administrating, as well as being able to monitor other services on their respective ports.
I made a WebApp to manage this exact same setting in about 60 minutes with my java web template. All I can share are some scripts that I use to list the connection and info about them. You can use those to build your own app, it is really simple to display this in some fancy way in a fast web.
Take a look at my scripts: https://unix.stackexchange.com/a/625771/332669
Those will allow you to get the listening port, as well as the public IPs they're binded from. With that you can easilly plan a system where everything is easilly identificable with a simple BBDD.
You might find this docker container useful https://hub.docker.com/r/logicethos/revssh/
I am ssh'd into a remote server would like to manipulate data on that machine without having to constantly push and pull the data around via my repository.
Can I run an IPython Notebook on that server and access/interact with it on my local browser? If so, how do I go about setting this up?
This will get you up and running if your server and your machine are on a LAN, and the server has one open port (there, 9999).
This will make it work even if you only have ssh access to the remote server. It works with as an ssh tunnel with port forwarding.
I'm using winexe to communicate with Windows running inside a virtual machine on my Linux system, to perform various test scenarios. I really don't want to have to be root to start the VMs.
When I start my Linux virtual machines, which I control with SSH, I simply map the SSH port (22) to a different, non-reserved port (>1024; say 19000). So I can start the VM without requiring root privileges. Then I use ssh -p 19000 ... when I want to ssh to the VM, and it works great.
But I cannot find a way to have winexe choose a different port than the default (I'm not sure what the default port is, actually; does it use 445 like SMB?). Is there a way to do it?
Note I cannot run an SSH server on Windows; because of my test environment requirements I can't add an SSH server to the virtual machines. Plus even if I were allowed I've had nothing but pain trying to get an SSH server to work reliably on Windows.
Winexe source code shows that the client-server communications happen over SMB in named pipes. As if you would write into unix pipes over nfs.
This results that it is very unlikely, that you can change the port. Maybe you can do that on the Linux side, but you have probably no way to do that in your Windows VM.