I came across the documentation of jupyter-server-proxy which has some useful tools but I am not sure how to make use of.
Assume I create a jupyter server on the VM on port 8888, normally I will use ssh port-forwarding to interact with it locally with ssh -L . However, I would like to automate now if I start a service such as Spark or Prometheus...etc that creates a dashboard service at 8080 (Spark) or 1234 (any other), is there an automated way that it forwards the port locally as part of jupyter so that I can access the service at, say for spark at: https://localhost:8888/port/8080 ?
Any education is much appreciated as I'm not even sure what is the correct terminology to search on.
Related
I have a problem setting up a ipython cluster on a Windows server and connecting to this ipcluster using a ssh connection. I tried following the tutorial on https://ipython.org/ipython/doc/dev/parallel/parallel_process.html#ssh, but I have problems to understand what the options mean exactly and what parameters are to use exactly...
Could anyone help a total noob to set up an ipcluster? (Let's say the remote machine has ip 192.168.0.1 and the local machine has 192.168.0.2)
If you scroll roughly to the middle of the page https://ipython.org/ipython-doc/dev/parallel/parallel_process.html#ssh you will find this:
Current limitations of the SSH mode of ipcluster are:
Untested and unsupported on Windows. Would require a working ssh on Windows. Also, we are using shell scripts to setup and execute
commands on remote hosts.
That means, there is no easy way to build an ipcluster with ssh connection on windows (if it works at all).
Do you really need to connect the machines with an ssh connection? I guess it's possible with a ssh client on each windows machine, but if you are in a trusted local network you can also decide not to use the loopback interface and just expose the ports...
Sure you can start controller and engine separately! For further examples about ports (if you have problems with firewalls) see also How to setup ssh tunnel for ipython cluster (ipcluster)
I want to install a number of raspberry pis at remote locations and be able to log in to them remotely. (Will begin with 30-40 boxes and hopefully grow to 1000 individual raspberry pis soon.)
I need to be able to remotely manage these boxes. Going the easier route, forwarding a port on the router and setting a DHCP reservation, requires either IT support from the company we'll be doing the install for (many of which don't have IT), or it will require one of our IT people physically installing each box.
My tentative solution is to have each box create a reverse SSH tunnel to our server. My question is: How feasible would this be? How easy would it be to manage that many connections? Would it be an issue for a small local server to have 1000+ concurrent SSH connections? Is there an easier solution to this problem?
My end goal is to be able to ship someone a box, have them plug it in, and be able to access it.
Thanks,
w
An alternate solution would be to:
Install OpenVPN server on your server machine. How to install OpenVPN Server on the PI. Additionally, add firewall rules that block everything but traffic directed for the client's ssh and other services ports (if desired), from administrating machine(s).
Run OpenVPN clients on your Raspberry PI client machines. They will connect back to your VPN server. On a side note, the VPN server and administrating machine(s) need not be the same machine if resources are limited on the VPN server. How to install OpenVPN on the client Raspberry PIs.
SSH from administrating machine(s) to each client machine. Optionally, you could use RSA authentication to simplify authentication.
Benefits include encryption for the tunnel including ssh encryption for administrating, as well as being able to monitor other services on their respective ports.
I made a WebApp to manage this exact same setting in about 60 minutes with my java web template. All I can share are some scripts that I use to list the connection and info about them. You can use those to build your own app, it is really simple to display this in some fancy way in a fast web.
Take a look at my scripts: https://unix.stackexchange.com/a/625771/332669
Those will allow you to get the listening port, as well as the public IPs they're binded from. With that you can easilly plan a system where everything is easilly identificable with a simple BBDD.
You might find this docker container useful https://hub.docker.com/r/logicethos/revssh/
We have an AWS EC2 server that we've configured to be only accessible (via SSH) from within our office network. Obviously this isn't ideal for remote arrangements where someone has to connect to the EC2 instance and is working remotely outside the office such as during a business trip.
I've managed to set-up a VPN through PPTP and can connect to the office network (I have two local IP's one from wlan0 and one from ppp0) regardless of anywhere I am. However, when I SSH to the EC2 instance, it's still rejecting me most likely because it sees that I'm still trying to ssh from outside the network.
I think I'm missing something very important. What do you think, what am I missing to to get the ssh to use the vpn connection rather than one in wlan/eth?
I'm starting to think this isn't possible so kindly let me know. My other option is to ssh to a machine within the office network and then use that machine to ssh to the EC2 instance but I've been hesistant to do that as it seems excessive.
Find your gateway through which you want to connect to your ec2-instance.
For Linux systems, route -n will show you the routing table. Now add the route
sudo route add -host gw
Just having a VPN to your office does not mean all traffic from your machine will go through the VPN - that depends on how you configure the VPN.
You can SSH from your machine to an office machine, and from there ssh to EC2 - that should work.
Alternatively, configure your system to route traffic for the ec2 instance through the VPN connection - doing this depends on your specific VPN configuration.
There is any way to "redirect" all traffic through HTTP local proxy?
I did that once but the software I used was not free so the time ran out and I need to use VNC and SSH to my raspberry pi at home.
But, I study on academy Networking and Programming we have 3 Networks:
1- GRSIP (this is a ADSL connection for CISCO classes (like me) to use)
-IP range: 192.168.27.0/24 proxy on .222
2- CISCO (this is for exams on the academy and training too)
3- DOM (this network is for the rest of non-networking and programming classes. I can't use any traffic except HTTP and some stuff is blocked of course :D
-IP range is all 172.34.0.0/16
So, I want to do this on DOM network because sometimes we are on some rooms with only this network. I can only use the HTTP local proxy to access the filtered WWW but I really need to bypass this.
The software I remember is "your-freedom" and "proxifier" (I don't know how to work with this one).
The network map on DOM is something like this:
the others networks I said are separated physically.
The program I use to connect to SSH is Putty because there we use windows 7 on the physical machine. VNC I use the 5800 port (via browser with java) of my PC at home.
Anybody with ideas?
int3
I want to make my site available world wide. Im using xampp server for hosting. I have no access to any kind of servers and modems. Situation is shown below:
My site server has local ip assigned by wifi router and it runs Windows 8.
Remember I have no access on any kind of servers and modems so port port-forwarding is impossible (out of my scope).
Its actually difficult, but not impossible.
One way, I would approach this is:
I would host a page on internet.
Then take request and store it in database.
One of my program will always be running from my computer.
Then check for request and curl the request to localhost. For this you may use Node.js (taking data from database using GET method and curl it to localhost).
This is the best I could think of. And I am working on it, when the code is ready I'll make it open source and notify you :)
But still, it's difficult, as you need to put user's request to sleep for 2 seconds and then transferring it.
Its slow, but may work out for you.
Disadvantages:
Program will be very slow and memory usage will be more.
Breaking may happen many times.
High bandwidth wastage
If not encrypted, MIM (Men in Middle) may possible.
Advantages:
Indirect method of hosting
Need not to worry about your code being lost.
I am looking forward for a better alternative and I would like to keep this question for bounty once again.
If you cannot open the necessary ports within your LAN you will require access to an external server. However, the external server does not need to host any code, e.g.
Create a Linux based ec2 instance using Amazon's free tier.
Install a package to redirect remote to local ports:
a. using socat:
Install socat using your distributions package manager
Connect via SSH: ssh -N -R 42500:127.0.0.1:80 -o ServerAliveInterval=60 ubuntu#xxx.xxx.xxx.xxx -N -R 8080:localhost:80 "socat TCP-LISTEN:8080,fork TCP:127.0.0.1:42500"
b. using a webserver and reverse proxy:
Install apache or nginx and any required reverse proxy modules and configure your VirtualHost to proxy requests to a local port, e.g. :8080 -> 127.0.0.1:42500
Connect via SSH: ssh -N -R 42500:127.0.0.1:80 -o ServerAliveInterval=60 ubuntu#xxx.xxx.xxx.xxx
Your machine is now reachable via the ec2 instance http://xxx.xxx.xxx.xxx:8080/.
I occasionally use this technique when debugging web service callbacks.
Update 17-02-2014
If you are a Windows user you will need to install a third-party tool to support ssh. Options include:
cygwin
git bash
PuTTY
PuTTY is the easiest choice if you are not familiar with *nix tools. To configure remote port forwarding in PuTTY expand the following setting: Connection -> SSH -> Tunnels. Given the previously described scenario, populate Source port as 42500, Desination as 127.0.0.1:80 and tick the Remote option. (You may also need to add the path to a PuTTY compatible private key in the Connection -> SSH -> Auth tab depending on your server configuration.
To test you have successfully forwarded a port, execute the command netstat -lnt on your server. You will see output similar to:
tcp 0 0 127.0.0.1:42500 0.0.0.0:* LISTEN
Finally you can test with curl http://127.0.0.1:42500. You will see the output of your own machines web root running on port 80.
if you don't have a public IP address and cannot use port forwarding it is impossible to host the site
As people have said you need a public IP address. However, even if you did you should not use xampp as a public server, as it is designed for development and therefore has some security settings disabled.
I would recommend buying some shared web hosting, and uploading it to that. (you can get cheap hosting if you google 'shared web hosting', plus free .tk domains are avaliable: http://www.dot.tk/)
Do your company has any vpn network?
If it does and you have access to the vpn network, you can include your server to the vpn network and your guest will only need to login to your company vpn network then access your site like in a local network without using port forwarding. And since your data is very confidential, I assume that using vpn will also help to increase the security of your data.
Please correct me if I'm wrong.
Thank You.
What you are asking is not possible without port forwarding.
Lets break it into steps.
To host your site locally you will need a IP that is static so that
users can access it specifically.
You will need a domain so that it can be converted into user friendly name.
A 24x7 Internet Connection is must! You added a Wifi Router in your Diagram and most of today's router are capable of port forwarding.
What i will do in your scenario is:
Instead of using XAMP, i will install WAMP because i am more familiar with it and easy to configure.(totally personal preference)
Then i would set my server "ONLINE".(Google how to set WAMP server online)
Forward port "80" from router settings to my local computer ip address.(mostly it is tagged as "Virtual Server","Firewall","Port Forwarding",etc vary router to router in settings)
Suppose you have a local ip "192.168.1.3" and global/router IP "254.232.123.232" then you would redirect all the HTTP request done towards router to your local IP.
[[[[254.232.123.232]]]] --+ :80 +-- --------->192.168.1.3
That is good for now, but then you will need to tackle dynamic IP problem of router. But don't worry, thanks to some free sites that will be easy!
Go to no-ip.org -> Setup Account -> and create a entry, just a subdomain for now to test whether everything is working fine.(subdomain like mysite.no-ip.org, later purchase a real Domain)
Input your IP address there(Router IP) and download its application which will automatically update their server if your local IP changes.
Wait for some minutes and Voila! Your site is live.