I have on the backend a Kubernetes node running on port 32656 (Kubernetes Service of type NodePort). If I create a firewall rule for the <node_ip>:32656 to allow traffic, I can open the backend in the browser on this address: http://<node_ip>:32656.
What I try to achieve now is creating an HTTP Load Balancer and link it to the above backend. I use the following script to create the infrastructure required:
#!/bin/bash
GROUP_NAME="gke-service-cluster-61155cae-group"
HEALTH_CHECK_NAME="test-health-check"
BACKEND_SERVICE_NAME="test-backend-service"
URL_MAP_NAME="test-url-map"
TARGET_PROXY_NAME="test-target-proxy"
GLOBAL_FORWARDING_RULE_NAME="test-global-rule"
NODE_PORT="32656"
PORT_NAME="http"
# instance group named ports
gcloud compute instance-groups set-named-ports "$GROUP_NAME" --named-ports "$PORT_NAME:$NODE_PORT"
# health check
gcloud compute http-health-checks create --format none "$HEALTH_CHECK_NAME" --check-interval "5m" --healthy-threshold "1" --timeout "5m" --unhealthy-threshold "10"
# backend service
gcloud compute backend-services create "$BACKEND_SERVICE_NAME" --http-health-check "$HEALTH_CHECK_NAME" --port-name "$PORT_NAME" --timeout "30"
gcloud compute backend-services add-backend "$BACKEND_SERVICE_NAME" --instance-group "$GROUP_NAME" --balancing-mode "UTILIZATION" --capacity-scaler "1" --max-utilization "1"
# URL map
gcloud compute url-maps create "$URL_MAP_NAME" --default-service "$BACKEND_SERVICE_NAME"
# target proxy
gcloud compute target-http-proxies create "$TARGET_PROXY_NAME" --url-map "$URL_MAP_NAME"
# global forwarding rule
gcloud compute forwarding-rules create "$GLOBAL_FORWARDING_RULE_NAME" --global --ip-protocol "TCP" --ports "80" --target-http-proxy "$TARGET_PROXY_NAME"
But I get the following response from the Load Balancer accessed through the public IP in the Frontend configuration:
Error: Server Error
The server encountered a temporary error and could not complete your
request. Please try again in 30 seconds.
The health check is left with default values: (/ and 80) and the backend service responds quickly with a status 200.
I have also created the firewall rule to accept any source and all ports (tcp) and no target specified (i.e. all targets).
Considering that regardless of the port I choose (in the instance group), and that I get the same result (Server Error), the problem should be somewhere in the configuration of the HTTP Load Balancer. (something with the health checks maybe?)
What am I missing from completing the linking between the frontend and the backend?
I assume you actually have instances in the instance group, and the firewall rule is not specific to a source range. Can you check your logs for a google health check? (UA will have google in it).
What version of kubernetes are you running? Fyi there's a resource in 1.2 that hooks this up for you automatically: http://kubernetes.io/docs/user-guide/ingress/, just make sure you do these: https://github.com/kubernetes/contrib/blob/master/ingress/controllers/gce/BETA_LIMITATIONS.md.
More specifically: in 1.2 you need to create a firewall rule, service of type=nodeport (both of which you already seem to have), and a health check on that service at "/" (which you don't have, this requirement is alleviated in 1.3 but 1.3 is not out yet).
Also note that you can't put the same instance into 2 loadbalanced IGs, so to use the Ingress mentioned above you will have to cleanup your existing loadbalancer (or at least, remove the instances from the IG, and free up enough quota so the Ingress controller can do its thing).
There can be a few things wrong that are mentioned:
firewall rules need to be set to all hosts, are they need to have the same network label as the machines in the instance group have
by default, the node should return 200 at / - readiness and liveness probes to configure otherwise did not work for me
It seems you try to do things that are all automated, so I can really recommend:
https://cloud.google.com/kubernetes-engine/docs/how-to/load-balance-ingress
This shows the steps that do the firewall and portforwarding for you, which also may show you what you are missing.
I noticed myself when using an app on 8080, exposed on 80 (like one of the deployments in the example) that the load balancer staid unhealthy untill I had / returning 200 (and /healthz I added to). So basically that container now exposes a webserver on port 8080, returning that and the other config wires that up to port 80.
When it comes to firewall rules, make sure they are set to all machines or make the network label match, or they won't work.
The 502 error is usually from the loadbalancer that will not pass your request if the health check does not pass.
Could you make your service type LoadBalancer (http://kubernetes.io/docs/user-guide/services/#type-loadbalancer) which would setup this all up automatically? This assumes you have the flag set for google cloud.
After you deploy, then describe the service name and should give you the endpoint which is assigned.
It's not that much of a question, rather a confirmation that what I did is right or not and if it is safe or not.
Until now what I have found googling around is that you cannot run rtorrent through a proxy. You can either put the http request through a proxy, or tsocks, in both cases either the actual transfers are done directly or not done at all. Therefore until now the only proposed viable solution is a VPN which I wanted to avoid.
What I did was use an http proxy for the http part and a port forwarding for the actual download part. For example, lets assume the following:
192.168.1.10 --> Local machine with the actual rtorrent
remote.machine.com --> The remote machine used as a proxy
Procedure:
I created 2 ssh tunnels
ssh -N -D 9090 user#remote.machine.com
ssh -R 9091:localhost:9091 user#remote.machine.com
From the local machine I installed polipo as the html proxy and configured it to use a socks proxy in the remote.machine.com.
I edited the following lines in /etc/polipo/config so that I can get the socks proxy.
socksParentProxy = "localhost:9090"
socksProxyType = socks5
I also changed the html proxy port for extra security, again in /etc/polipo/config
proxyPort = 9080
On the local machine I changed the ~/.rtorrent.rc as following:
#Proxy of the http requests through polipo
http_proxy=localhost:9080
# The ip address reported to the tracker.
#Really important, in order to get connections for downloads
ip = remote.machine.com
# The ip address the listening socket and outgoing connections is
# bound to.
bind = 192.168.1.10
# Port range to use for listening.
port_range = 9091-9091
# Start opening ports at a random position within the port range.
port_random = no
The system seems to work. I connect to the trackers and I have up and down traffic. So the questions are:
Am I safe that all the traffic concerning rtorrent is done through the remote.machine.com?
Did I miss something?
Are there any problems or concerns regarding this method?
As far as I see, you have covered inbound connections, as well as outgoing HTTP traffic, but any outbound peer-to-peer connections will be created directly, not through any tunnel. Currently, rtorrent does not appear to support passing outbound P2P connections through a tunnel or proxy of any kind, so in order to handle these, you'll need some other mechanism.
You mentioned tsocks and that it does not work – not even in addition to the rtorrent configuration you have set up above? (Although with tsocks you should be able to drop the HTTP proxy part.)
If that fails, there are alternatives to tsocks mentioned on the tsocks project page. A slightly more involved alternative would be to create a new loopback interface (lo:1 with IP 127.0.0.2), bind your rtorrent to that one and use something like sshuttle to direct all traffic originating on that interface through an SSH tunnel. Unfortunately, sshuttle doesn't let you restrict its operation to a specific interface at the moment, though, so you'd have to fiddle with the iptables rules it creates to make them match your needs. I assume a patch adding this feature to sshuttle would be welcome.
As a side note, you can create multiple port forwards and SOCKS proxies in a single SSH connection, like this:
ssh -N -D 9090 -R 0.0.0.0:9091:localhost:9091 myself#my.example.com
I cannot SSH into my instance - Operation timed out. What could be the reasons why, and what can I do to resolve it? Rebooting normally takes a long time to take effect, and might just makes things worst
UPDATE: It is not about permissions - i can log in normally just fine. I suspect it might be because of memory issues
I had the same problem, and the solution ended up being adding my
local machine's IP to the list of inbound rules in the active security
group. In the inbound dialog below, enter 22 in the port range, your local IP/32 in the source field, and leave 'custom tcp rule' in the dropdown.
Did you set an appropriate security group for the instance? I.e. one that allows access from your network to port 22 on the instance. (By default all traffic is disallowed.)
Update: Ok, not a security group issue. But does the problem persist if you launch up another instance from the same AMI and try to access that? Maybe this particular EC2 instance just randomly failed somehow – it is only matter of time that something like that happens. (Recommended reading: Architecting for the Cloud: Best Practices (PDF), a paper by Jinesh Varia who is a web services evangelist at Amazon. See especially the section titled "Design for failure and nothing will fail".)
Destroy and create anew
I had one availability zone where I could connect and another where I could not. After a few hours I got so frustrated that I deleted everything in that availability zone.
Building everything back I had to make sure to create EVERYTHING. This included:
Create VPC
CIDR: 10.0.0.0/24
Create Internet Gateway
Attach Internet Gateway to VPC
Create Routing Table
Add Route to Routing Table
Destination: 0.0.0.0/0
Target: <Internet Gateway from earlier>
Create Subnet
CIDR: 10.0.0.0/24
Routing Table: <Routing Table from earlier
It took me a LOT of fumbling to get all of this. I've ordered the steps in the way I think might be most efficient, but you may have to adjust them to get one item available for the next.
Suggestion
I'm not suggesting that you go thermo nuclear as I did. I'm offering all this information so that you can check these associations to ensure yours are appropriate.
This answer is for the silly folks (like me). Your EC2's public DNS might (will) change when it's restarted. If you don't realize this and attempt to SSH into your old public DNS, the connection will stall and time out. This may lead you to assume something is wrong with your EC2 or security group or... Nope, just SSH into the new DNS. And update your ~/.ssh/config file if you have to!
To connect use ssh like so:
ssh -i keyname.pem username#xxx.xx.xxx.xx
Where keyname.pem is the name of your private key, username is the correct username for your os distribution, and xxx.xx.xxx.xx is the public ip address.
When it times out or fails, check the following:
Security Group
Make sure to have an inbound rule for tcp port 22 and either all ips or your ip. You can find the security group through the ec2 menu, in the instance options.
Routing Table
For a new subnet in a vpc, you need to change to a routing table that points 0.0.0.0/0 to internet gateway target. When you create the subnet in your vpc, by default it assigns the default routing table, which probably does not accept incoming traffic from the internet. You can edit the routing table options in the vpc menu and then subnets.
Elastic IP
For an instance in a vpc, you need to assign a public elastic ip address, and associate it with the instance. The private ip address can't be accessed from the outside. You can get an elastic ip in the ec2 menu (not instance menu).
Username
Make sure you're using the correct username. It should be one of ec2-user or root or ubuntu. Try them all if necessary.
Private Key
Make sure you're using the correct private key (the one you download or choose when launching the instance). Seems obvious, but copy paste got me twice.
Have you looked at the console output from the instance ? You can do this via the AWS console (Instances -> Right-click on the instance -> Get System Log). I've had occasions where the network services on an EC2 instance failed to start correctly, resulting in timed out SSH connections; restarting the instance usually fixed things.
AFTER 2 HOURS I FOUND THIS
Note That ssh ip 120.138.105.251/32
IS NOT aws instance IP ADDRESS
It Is not your local ip 127.0.0.1
It Is not your local ip localhost
BUT BUT BUT
Its Your public ip address of your personal Computer from which you trying to access aws instance
Go to https://www.whatismyip.com/ whatever ip address put in ssh
IF YOU WANT TO FULLY OPEN SSH TO ALL IP ADDRESS
THIS IS HOW FULLY ACCESSIBLE ENTRIES LOOK - BASIC RECOMEENDED
THIS IS WHAT I AM USING IN PRODUCTION
The following are possible issues:
The most likely one is that the Security Group is not configured properly to provide SSH access on port 22 to your i.p. Change in security setting does not require a restart of server for it to be effective but need to wait a few minutes for it to be applicable.
The local firewall configuration does not allow SSH access to the server. ( you can try a different internet connection, your phone/dongle to try it)
The server is not started properly ( then the access checks will fail even on the amazon console), in which case you would need to stop and start the server.
Allow ssh and port 22 from ufw, then enable it and check with status command
sudo ufw allow ssh
sudo ufw allow 22
sudo ufw enable
sudo ufw status
Check out this help page on AWS docs:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/TroubleshootingInstancesConnecting.html#TroubleshootingInstancesConnectionTimeout
You will probably find your solution there. for me this part did the fix:
[EC2-VPC] Check the route table for the subnet. You need a route that
sends all traffic destined outside the VPC to the Internet gateway for
the VPC.
Open the Amazon VPC console at https://console.aws.amazon.com/vpc/.
In the navigation pane, choose Internet Gateways. Verify that there is an Internet gateway attached to your VPC. Otherwise, choose Create
Internet Gateway and follow the directions to create an Internet
gateway, select the Internet gateway, and then choose Attach to VPC
and follow the directions to attach it to your VPC.
In the navigation pane, choose Subnets, and then select your subnet.
On the Route Table tab, verify that there is a route with 0.0.0.0/0 as the destination and the Internet gateway for your VPC as the
target. Otherwise, choose the ID of the route table (rtb-xxxxxxxx) to
navigate to the Routes tab for the route table, choose Edit, Add
another route, enter 0.0.0.0/0 in Destination, select your Internet
gateway from Target, and then choose Save.
But I suggest you check out all the options the link above covers , you may find there the one or more issues that you got.
My issue - I had port 22 open for "My IP" and changed the internet connection and IP address change caused. So had to change it back.
Building off #ted.strauss's answer, you can select SSH and MyIP from the drop down menu instead of navigating to a third party site.
Just reboot the Ec2 Instance once you applied Rules
I was working on the instance and it was fine, the very next day when I tried to SSH into my instance it said - Connection timeout.
I tried to go through this post but nothing worked. So I did -
On the Edit inbound rules from source column choose MY IP and it will automatically populate your Public IP address in CIDR format (XXX.XXX.XXX.XX/32).
I tried with the #ted.strauss answer by giving local IP but it did not help in my case. So I choose MY IP and it worked.
Hope this helps someone!
In my case, Instance was reachable but suddenly it became unreachable.
I just rebooted the instance from aws console & it solved my problem.
If you still have this problem today and non of the other solutions have been helpful, what worked for me was; go to security groups, select a security group or create one, then click to add new inbound rule, click the source drop down and select "from anywhere" this will set 0.0.0.0/0 as the source ip.
You should be fine now.
Ensure you already have the private key file on your machine.
you can watch this video if you need more clarity.
https://www.youtube.com/watch?v=8UqtMcX_kg0
One more possibility. AWS security groups are setup to work only with specific incoming ip addresses. If your security group is setup in this way you (or the account holder) will need to add your ip address to the security group. Todo this open your AWS dashboard, select security groups, select a security group and click on the inbound tab. Then add your ip as appropriate.
I had the same problem, and the solution was allowing access from anywhere to the list of inbound rules in the active security group. In the inbound dialog, enter 22 in the port range, anywhere in the source field, and select 'ssh' in the dropdown.
P.S : This might not be the recommended solution as it means this instance can be ssh'ed from any machine, but I could not get it to work with my local IP.
I had similar problem, when I was using public Wifi, which didn't have password. Switching the internet connection to a secure connection did solve the problem.
If SSH access doesn't work for your EC2 instance, you need to check:
Security Group for your instance is allowing Inbound SSH access (check: view rules).
If you're using VPC instance (you've VPC ID and Subnet ID attached to your instance), check:
In VPC Dashboard, find used Subnet ID which is attached to your VPC.
Check its attached Route table which should have 0.0.0.0/0 as Destination and your Internet Gateway as Target.
On Linux, you may also check route info in System Log in Networking of the instance, e.g.:
++++++++++++++++++++++++++++++++++++++Net device info+++++++++++++++++++++++++++++++++++++++
+--------+------+------------------------------+---------------+-------+-------------------+
| Device | Up | Address | Mask | Scope | Hw-Address |
+--------+------+------------------------------+---------------+-------+-------------------+
| lo | True | 127.0.0.1 | 255.0.0.0 | . | . |
| eth0 | True | 172.30.2.226 | 255.255.255.0 | . | 0a:70:f3:2f:82:23 |
+--------+------+------------------------------+---------------+-------+-------------------+
++++++++++++++++++++++++++++Route IPv4 info+++++++++++++++++++++++++++++
+-------+-------------+------------+---------------+-----------+-------+
| Route | Destination | Gateway | Genmask | Interface | Flags |
+-------+-------------+------------+---------------+-----------+-------+
| 0 | 0.0.0.0 | 172.30.2.1 | 0.0.0.0 | eth0 | UG |
| 1 | 10.0.3.0 | 0.0.0.0 | 255.255.255.0 | lxcbr0 | U |
| 2 | 172.30.2.0 | 0.0.0.0 | 255.255.255.0 | eth0 | U |
+-------+-------------+------------+---------------+-----------+-------+
where UG flags showing you your internet gateway.
For more details, check: Troubleshooting Connecting to Your Instance at Amazon docs.
To enable ssh access from the Internet for instances in a VPC subnet do the following:
Attach an Internet gateway to your VPC.
Ensure that your subnet's route table points to the Internet gateway.
Ensure that instances in your subnet have a globally unique IP address (public IPv4 address, Elastic IP address, or IPv6 address).
Ensure that your network access control (at VPC Level) and security group rules (at ec2 level) allow the relevant traffic to flow to and from your instance. Ensure your network Public IP address is enabled for both. By default, Network AcL allow all inbound and outbound traffic except explicitly configured otherwise
For me it was the apache server hosted on a t2.micro linux EC2 instance, not the EC2 instance itself.
I fixed it by doing:
sudo su
service httpd restart
I had the same problem and I solved it by adding a rule to the security Groups
Inbound SSH 0.0.0.0/0
Or you can add your IP address only
For me, it was that I had deleted everything from the boot volume. And couldn't connect to the instance anymore.
ping the DNS first . If fails then configure your inbound/outbound rules in the launch wizard . configure ALL traffic and ALL protocol and just save with default options . Ping again with your local system and then should work
If you've just created a new instance and can't connect to it, I was able to solve the issue by terminating that one and creating a new one. Of course this will only work if it's a new instance and you haven't done any more work on it.
There could be multiple reasons for connection getting timed-out during ssh. One of the cases where i have seen this is when your route and subnet does not have associations.
If you're trying to connect from another EC2 instance, you have to make sure to only use the PRIVATE IPv4 addresses to reference them or the connections will not work.
This will be one of the reason. when you enable ufw and reboot your instance. First you have to add 22/tcp before enabling ufw. Following is the command
$ ufw allow 22/tcp
If you already made the mistake. Then follow the following guide
Start a recovery instance.
Stop the blocked instance (DON'T TERMINATE)
Detach the blocked instance volume.
Attach Blocked volume to the recovery instance.
Log to the recovery instance(Newly Launched) via ssh/putty
Type sudo lsblk to display attached volumes
Verify the name of the Blocked volume. Generally start with /dev/xvdf.
Mount blocked volume.
$ sudo mount /dev/xvdf1 /mnt
$ cd /mnt/etc/ufw
Open ufw configuration file
$ sudo vi ufw.conf
Enable insert mode by pressing i in vi editor
Update ENABLED=yes to ENABLED=no
ClickESC and type :wq to update the file.
Verify the file contents. where update to ENABLED=yes -> ENABLED=no
$ sudo cat ufw.conf
Remove the mounted blocked volume from recovery instance
$ cd ~
$ sudo umount /mnt
Now detach blocked install volume from recovery instance and re-attach it to the original instance as /dev/sda1.
Finally, Start the blocked instance. Here's you will able to access your instance. If you enable ufw again don't forget to allow 22/tcp.
sudo ufw allow ssh //this one are to be added
sudo ufw allow 22 // this one
sudo ufw enable
sudo ufw status
The solution with me was different than all the above solutions. I forgot the my VPN on. When I turned it off the problem is solved.