I have 2 services running in a single private vpc subnet (same available zone). Each service is based on a container here: https://github.com/spring-petclinic/spring-petclinic-microservices .
I've setup route53 service endpoints for both services.
When I run my tasks (each within their own service) service A times out calling service B over service B's route53 endpoint. Using localhost doesn't work because these containers are in separate services.
When I create a container for my task definition, I assign the port that my container is using (using port mapping field). However I notice in the console there is this note: "Host port mappings are not valid when the network mode for a task definition is host or awsvpc. To specify different host and container port mappings, choose the Bridge network mode."
Since I'm using Fargate I am using awsvpc mode. So is this telling my port mapping setting isnt doing anything ? Is that why my services are timing out ?
Then when I google bridge mode, this seems to tell me that awscpv networking mode support service discovery: https://aws.amazon.com/about-aws/whats-new/2018/05/amazon-ecs-service-discovery-supports-bridge-and-host-container-/
So how does "bridge mode" work here ? Why does port mapping field not work for awsvpc ?
Edit:
I read this How to communicate between Fargate services on AWS ECS? and he just says "I created a new service and things started working." That's a bit disheartening.
Edit2:
Yes my vpc has enabled dns resolution.
As it turns out the security group on my service was only allowing http on port 80. That is the inbound rules the default SG that the service wizard gives you. I updated it to allow traffic on my container ports and they seem to be talking to each other now.
Related
I try to integrate a web app to a VNET. this is a brand new subscription.
According to : Integrate your app with an Azure virtual network
In the first step I wanted to check if the web app could reach out to VNET. The second step I want to connect the web app to SQL database through a Service Endpoint
I created a VNET with 2 subnets:
Jumpbox-subnet
integration-subnet
integration-subnet setting
There is service endpoint pointing to the integration-subnet. Also I integrated the app to the VNet, It's delegated the integration subnet.
I tried to connect to VM from App using tcpping 172.16.1.0 (jumpbox VM private address) from app console, but it failed.
I also app cannot connect to the sql database.
What are the missing pieces here, is a DNS server required to make this work?
Update (Resolved):
The question above was the key, It needs away to resolve the name with some sort DNS server.
tcpping default port is 80 and nothing was listening to that port in that box.
You could use the tool tcpping to test for TCP connectivity to a host and port combination.
The syntax is: tcpping.exe hostname [optional: port]
For example, run tcpping 172.19.1.10:3389
See troubleshooting app service networking for more details.
For more references, Here is an ARM example to deploy an app service with VNet integration and enable service endpoint Microsoft.Storage.
I am newbie on Docker and Kubernetes. And now I am developing Restful APIs which later be deployed to Docker containers in a Kubernetes cluster.
How the path of the endpoints will be changed? I have heard that Docker-Swarm and Kubernetes add some ords on the endpoints.
The "path" part of the endpoint URLs themselves (for this SO question, the /questions/53008947/... part) won't change. But the rest of the URL might.
Docker publishes services at a TCP-port level (docker run -p option, Docker Compose ports: section) and doesn't look at what traffic is going over a port. If you have something like an Apache or nginx proxy as part of your stack that might change the HTTP-level path mappings, but you'd probably be aware of that in your environment.
Kubernetes works similarly, but there are more layers. A container runs in a Pod, and can publish some port out of the Pod. That's not used directly; instead, a Service refers to the Pod (by its labels) and republishes its ports, possibly on different port numbers. The Service has a DNS name service-name.namespace.svc.cluster.local that can be used within the cluster; you can also configure the Service to be reachable on a fixed TCP port on every node in the service (NodePort) or, if your Kubernetes is running on a public-cloud provider, to create a load balancer there (LoadBalancer). Again, all of this is strictly at the TCP level and doesn't affect HTTP paths.
There is one other Kubernetes piece, an Ingress controller, which acts as a declarative wrapper around the nginx proxy (or something else with similar functionality). That does operate at the HTTP level and could change paths.
The other corollary to this is that the URL to reach a service might be different in different environments: http://localhost:12345/path in a local development setup, http://other_service:8080/path in Docker Compose, http://other-service/path in Kubernetes, https://api.example.com/other/path in production. You need some way to make that configurable (often an environment variable).
I have created a redis 3.2 application from the default image catalog.
I'm trying to connect a python app that runs inside the same project with the redis db.
This is what the Python application uses to connect to redis:
REDIS_HOST = 'localhost'
REDIS_PORT = 6379
REDIS_PASSWORD = os.environ.get('REDIS_PASSWORD') or 'test'
redis = aioredis.create_redis_pool(
(REDIS_HOST, int(REDIS_PORT)),
password=REDIS_PASSWORD,
minsize=5,
maxsize=10,
loop=loop,
)
The deployment fails with an ConnectionRefusedError: [Errno 111] Connection refused.
My guess is that I need to use another value for REDIS_HOST, but I couldn't figure what to use.
Does anyone know how to fix this?
After you deployed from the image catalog a number of objects will have been created for you. One of those objects is a service, which is used to load balance requests to the Pods it fronts. Service names for a project can be retrieved using the client tools via oc get svc.
This service name should be used to connect to your redis instance. If you deploy redis before your Python application, some environment variables should already be populated which can be used, for example REDIS_SERVICE_HOST and REDIS_SERVICE_PORT.
So from your application you can connect via the service ip or service name, where service name is redis then redis.StrictRedis(host='redis', port=6379, password='secret')
The redis password may have been generated for you. In that case it is retrievable from the redis secret which could also be mounted from your python app
Databases in general do not use standard HTTP, but custom TCP protocols. This is why in Openshift we need to connect directly to the service using Openshift's Service hostname or IP address (caution: only Service hostname is predictable), instead of the usual Route, and this applies also to Redis. Bypassing the Routes in Openshift is like bypassing a reverse proxy such as nginx and directly connecting to the db backend.
There is need to use env variables, because service hostnames are auto-generated by Openshift using this predictable pattern:
container_name.project_name.svc , e.g:
redis.db.svc
More info
"When a web application is made visible outside of the OpenShift cluster a route is created. This enables a user to use a URL to access the web application from a web browser. A route is usually used for web applications which use the HTTP protocol. A route cannot be used to expose a database, as they would typically use their own distinct protocol, and routes would not be able to work with the database protocol."
[https://blog.openshift.com/openshift-connecting-database-using-port-forwarding/ ]
I would like to avoid using type: "LoadBalancer" for a certain Kubernetes Service, but still to be able to publish it on the Internet. I am using Google Cloud Platform (GCP) to run a Kubernetes cluster currently running on a single node.
I tried to us the externalIPs Service configuration and to give at turns, the IPs of:
the instance hosting the Kubernetes cluster (External IP; which also conincides with the IP address of the Kubernetes node as reported by kubernetes describe node)
the Kubernetes cluster endpoint (as reported by the Google Cloud Console in the details of the cluster)
the public/external IP of another Kubernetes Service of type LoadBalancer running on the same node.
None of the above helped me reach my application using the Kubernetes Service with an externalIPs configuration.
So, how can I publish a service on the Internet without using a LoadBalancer-type Kubernetes Service.
If you don't want to use a LoadBalancer service, other options for exposing your service publicly are:
Type NodePort
Create your service with type set to NodePort, and Kubernetes will allocate a port on all of your node VMs on which your service will be exposed (docs). E.g. if you have 2 nodes, w/ public IPs 12.34.56.78 and 23.45.67.89, and Kubernetes assigns your service port 31234, then the service will be available publicly on both 12.34.56.78:31234 & 23.45.67.89:31234
Specify externalIPs
If you have the ability to route public IPs to your nodes, you can specify externalIPs in your service to tell Kubernetes "If you see something come in destined for that IP w/ my service port, route it to me." (docs)
The cluster endpoint won't work for this because that is only the IP of your Kubernetes master. The public IP of another LoadBalancer service won't work because the LoadBalancer is only configured to route the port of that original service. I'd expect the node IP to work, but it may conflict if your service port is a privileged port.
Use the /proxy/ endpoint
The Kubernetes API includes a /proxy/ endpoint that allows you to access services on the cluster endpoint IP. E.g. if your cluster endpoint is 1.2.3.4, you could reach my-service in namespace my-ns by accessing https://1.2.3.4/api/v1/proxy/namespaces/my-ns/services/my-service with your cluster credentials. This should really only be used for testing/debugging, as it takes all traffic through your Kubernetes master on the way to the service (extra hops, SPOF, etc.).
There's another option: set the hostNetwork flag on your pod.
For example, you can use helm3 to install nginx this way:
helm install --set controller.hostNetwork=true nginx-ingress nginx-stable/nginx-ingress
The nginx is then available at port 80 & 443 on the IP address of the node that runs the pod. You can use node selectors or affinity or other tools to influence this choice.
There are a few idiomatic ways to expose a service externally in Kubernetes (see note#1):
Service.Type=LoadBalancer, as OP pointed out.
Service.Type=NodePort, this would exposing node's IP.
Service.Type=ExternalName, Maps the Service to the contents of the externalName field by returning a CNAME record (You need CoreDNS version 1.7 or higher to use the ExternalName type.)
Ingress. This is a new concept that expose eternal HTTP and/or HTTPS routes to services within the Kubernetes cluster, you can even map a route to multiple services. However, this only maps HTTP and/or HTTPS routes only. (See note#2)
I have a Google Cloud Container Engine cluster with 2 Pods, master and slave. Each of them runs RabbitMQ instance, that supposed to be joined into one cluster.
Ports exposed from Dockers aren't available from other machine, but could be accessed only through a Service. That's not a problem, I could establish a service for each instance (one-to-one, service-to-pod), and point each Pod to opposite service IP.
The problem that RabbitMQ uses more that one port for communications. That means that service IP should open all this ports from underlying Pod. But I cannot specify list of shared port for a Service, and if I create a new service for each port each of them will have own IP.
Is there any way to expose list of ports from same Docker/Pod on same internal IP address using Container Engine cluster? maybe some special routing configuration?
Your question is similar to this question, and unfortunately has the same response: Kubernetes / Google Container Engine does not currently have a way to expose a range of ports for a service at the current time. There is an open issue in GitHub to address this use case.