I have a service that exposes an API over an in-house developed TCP protocol. I am currently in the process of moving that service to a REST API. The service listens for and response to requests for both APIs simultaneously as it may be some time until all the clients move over to the REST API.
Given that the APIs are different protocols, I believe I will need both tcp frontends/backends and http frontend/backends.
I would prefer to not have to deploy separate instances of my service for each protocol. Instead, I would like to have the same set of servers for each of the backends and have HAProxy load balance (leastconn) across them.
As an example
frontend fe_custom
bind :11111
mode tcp
use_backend be_custom
frontend fe_http
bind :80
mode http
use_backend be_http
backend be_custom
mode tcp
balance leastconn
server server1 192.168.10.100:11111
server server2 192.168.10.101:11111
backend be_http
mode http
balance leastconn
server server1 192.168.10.100:80
server server2 192.168.10.101:80
So if a request is sent to my custom protocol on port 11111 and gets sent to be_custom:server1, I would like a subsequent request that comes in for my REST API on port 80 to get load balanced to be_http:server2.
Will this scenario just work if the same server is specified in different backends? If not, is this something that can be done in HAProxy?
Related
The Consul documentation shows how to register external services, but the examples shown are always targeting port 80 (and 443 only for health checks, which is even more peculiar) - example: https://learn.hashicorp.com/tutorials/consul/service-registration-external-services
If you register a service for port 443, a using service having it as upstream in fact gets a port opened from its local Envoy, but one cannot access this port - curl always complains about wrong protocol version. In contrast, if one registers a service for port 80, plain HTTP access works as expected.
Does anybody have an idea how an https based service can be provided as external service?
I would like to know how many TCP connections are created when WebSocket call is made from browser to apache http server to backend web service?
Does it create a separate TCP connection from the browser to apache http server and from apache to the web service?
When Apache is proxying websockets, there is 1 TCP connection between the client and Apache and 1 TCP connection between Apache and the backend.
Apache watches both connections for activity and forwards read from one onto the other.
This is the only way it can be in a layer 7 (Application Layer, HTTP) proxy. Something tunnelling at a much lower layer, like a NAT device or MAC forwarding IP sprayer could tunnel a single connection -- but not on the basis of anything higher up in the stack like headers.
The 2nd connection is observable with netstat.
The 2nd connection is opened when mod_proxy_wstunnel calls ap_proxy_connect_to_backend() which calls apr_socket_create() which calls the portable socket() routine. When recent releases of mod_proxy_http handle this tunneling automatically, simialr flow through ap_proxy_acquire_connection.
I've configured X_FORWARDED_FOR to capture client IP for a HTTPS request and it works as expected.
However, for MQTT, the data is sent over SSL and HTTP/S does not come into the picture.
ssl://<HOST_NAME>:<PORT>
I've tried adding the following to the backend server on HAproxy config. No luck so far.
backend TestServer
mode tcp
server TestServer01 10.6.186.24:48080 send-proxy-v2
------
server TestServer01 10.6.186.24:48080 send-proxy
------
server TestServer01 10.6.186.24:48080 send-proxy-v2-ssl
Is there a way to capture client (source) IP for an incoming MQTT request by changing HAProxy configuration?
No, there is no where in the MQTT protocol to store the original client IP address (like adding extra headers to HTTP requests).
The proxy is literally just forwarding packets that arrive on it's public port to the backend servers (with the possible exception of doing SSL termination) it doesn't change the packets at all.
If you wanted the IP address to do stick-table based abuse protection, you will need to key your stick-table with the MQTT client identifier.
For example this will reject clients if their connection rate is greater than 1 per second, over a 10s window.
tcp-request content set-var(txn.client_id) req.payload(0,0),mqtt_field_value(connect,client_identifier) if data_in_buffer
stick-table type string len 64 size 100k expire 5m store gpc0,gpc0_rate(10s)
tcp-request content track-sc0 var(txn.client_id)
tcp-request content sc-inc-gpc0(0)
tcp-request content reject if { sc0_gpc0_rate gt 10 }
AWS NLB supports TLS termination
https://aws.amazon.com/blogs/aws/new-tls-termination-for-network-load-balancers/
NLB being a Layer 4 load balancer I would expect it to work in a passthrough mode by directing the incoming packets to one of the backends without much of state maintenance (except for the flow tracking)
Are there any details available on how AWS implements the TLS termination in NLB ?
Is it possible to do it with open source tooling (like IPVS or haproxy) or AWS has some secret sauce here ?
The TLS termination itself is just what it says it is. TLS is a generic streaming protocol just like TCP one level up so you can unwrap it at the LB in a generic way. The magic is that they keep the IPs intact probably with very fancy routing magic, but it seems unlikely AWS will tell you how they did it.
In my SO question here, I have an example of how to terminate a TCP session in HAProxy and pass the unencrypted traffic to a backend.
In short, you need to use ssl in the frontend bind section and both frontend and backend configurations require use of tcp mode. Here is an example of terminating on port 443 and forwarding to port 4567.
frontend tcp-proxy
bind :443 ssl crt combined-cert-key.pem
mode tcp
default_backend bk_default
backend bk_default
mode tcp
server server1 1.2.3.4:4567
I have the following model that i drew below:
I have a number of processes running on the server. I want nginx or apache to direct the incoming clients through port 80 to one of the server processes to handle the requests. However each connection also establishes a websocket connection to the same process. This is currently initiated from the client side within javascript. At the moment for testing purposes I pass the port within the html rendered on the client. The client then takes this port and estabilishes a websocket connection to the same port that handled its request.
Moving forward to an nginx or apache envionment would it be possible not to pass the port value to the client and have nginx or apache know where it directed the incoming client and use the same port for the websocket connection?
This would have the benefit on not opening all the server ports 8000, 8001, 8002 in the diagram below to the public.