I have REST API.
For the sake of simplicity lets say I have two services:
read.request.com
write.request.com
How I can rewrite all read requests (GET method) to read.request.com and all write requests (POST, PUT, DELETE methods) to write.request.com with HAProxy?
Not quite sure which of these is applicable for your situation but hopefully one is.
One Backend
I think this is your situation.
frontend http-in
bind *:80
acl is_post method POST
acl is_get method GET
http-request set-header Host write.request.com if is_post
http-request set-header Host read.request.com if is_get
default_backend api
backend api
server one localhost:8080 check
All this does is check which method is being used and sets the Host header accordingly before passing the request on to localhost:8080.
Two Backends
In this set up you have have one instance of the code running for just read requests and another instance just for write requests. In this case the read code is running on localhost:8080 and the write code is running on localhost:8081.
frontend http-in
bind *:80
acl is_post method POST
acl is_get method GET
use_backend write if is_post
use_backend read if is_get
backend write
http-request set-header Host write.request.com #optional
server write_one localhost:8081 check
backend read
http-request set-header Host read.request.com #optional
server read_one localhost:8080 check
This option starts the same as the previous one by checking which method is being used but instead of using one HAProxy backend it splits out into two. The http-request line inside each backend is optional for this configuration.
Hope that helps.
Related
I am new to HAProxy and I am trying to do a simple redirect from <server_ip>/mq to <server_ip>:8161/admin
When I am trying to access ActiveMQ directly via <server_ip>:8161/admin everything works well.
But with the HAProxy sometimes I get 503. And when it works it doesn't load the resources at all (no css, no images..)
frontend http
bind *:80
timeout client 60s
mode http
acl app path_end -i /mq
use_backend appServer if app
default_backend all
backend appServer
timeout connect 10s
timeout server 10s
mode http
http-request set-path /admin/
server servermq <server_ip>:8161
backend all
timeout connect 10s
timeout server 10s
mode http
http-request set-path /admin/
server servermq <server_ip>:8161
In the dev tools I can see errors like this:
Refused to apply style from 'http://<server_ip>/admin/styles/site.css' because its MIME type ('text/html') is not a supported stylesheet MIME type, and strict MIME checking is enabled.
But I am not sure how to fix this.
Let's address the issues one by one.
Intermittent 503
This is most likely due to more than one haproxy processes running in the system. You can find all of them by a simple ps -ef | grep haproxy. Kill all but one and it should be fixed. Another reason might be due to two frontends binding on same port which in this case is :80. Check for all the frontend where binding port is 80. Change each one of them to be different from the other.
CSS and JS not loading which is related to the mismatched MIME type error posted
There is an issue with your haproxy configuration in the backend all section. According to your frontend http rules, all request ending with /mq should go to backend appServer where a url rewrite is happening via http-request set-path /admin/. /mq is replaced with /admin/ before sending it to servermq. This is correct but when you're loading the activemq admin in the browser, the browser is making requests for *.css and *.js files which is coming empty. This is because the browser is making request on /admin/styles/sorttable.css file. Now this doesn't match with your acl app but goes to default_backend all. Here due to this rule http-request set-path /admin/ again path is set to /admin/ which returns 200 but without any css file, that's why the error:
Refused to apply style from 'http://<server_ip>/admin/styles/site.css' because its MIME type ('text/html') is not a supported stylesheet MIME type, and strict MIME checking is enabled.
The correct config should be:
frontend http
bind *:80
timeout client 60s
mode http
acl app path_end -i /mq
use_backend appServer if app
default_backend all
backend appServer
timeout connect 10s
timeout server 10s
mode http
http-request set-path /admin/
server servermq <server_ip>:8161
backend all
timeout connect 10s
timeout server 10s
mode http
server servermq <server_ip>:8161
I am trying to get the equivalent of the nginx passtrough where a user would see my website url in the address bar but get all the content to be proxied from another website.
At the moment the code below redirects the user to example2 by returning a 304, instead of proxying the traffic.
I need it to work with http (not tcp) because I need this as part of an AB test where I need to inspect the cookies. Please check the comments on the code below for what I am trying to do.
defaults
mode http
log global
option httplog
log 127.0.0.1 local0
timeout connect 5000ms
timeout client 50000ms
timeout server 50000ms
listen http
bind 127.0.0.1:8080
acl set_cookie path_beg /set-cookie
use_backend b-backend if { req.cook(SITEID) -m beg b-backend }
use_backend b-backend if set_cookie
default_backend ab-split-backend
backend a-backend
option forwardfor
server example1 example1.com:443 check maxconn 3000 inter 30s
backend b-backend
cookie SITEID insert
option http_proxy
# how do I get example2 to passtrough and not 304 redirect?
server example2 example2.com:443 check maxconn 3000 inter 30s
backend ab-split-backend
balance roundrobin
cookie SITEID insert indirect nocache maxlife 48h
# how do I get example2 to passtrough?
server example2 example2.com:443 weight 50 cookie b-backend
server example1 example1.com:443 weight 50 cookie a-backend
HTTP 304 is not really a redirect, it is an empty response indicating Not Modified which tells the client that the server would have responded with a 200 and served the content, but the requested asset has not changed, so the client can just use what it has cached.
So I'm not entirely sure what you're seeing is incorrect behavior. That is, your requests may be being passed through just fine, and the backend server may be correctly responding with a 304.
The server makes the decision to respond with this code based on information provided in the request headers If-Modified-Since and/or If-None-Match. So if you really want to disable this caching mechanism and ensure a complete 200 response every time, you can instruct HAProxy to delete these headers from the incoming request:
listen http
bind 127.0.0.1:8080
acl set_cookie path_beg /set-cookie
# Delete headers related to caching
http-request del-header If-Modified-Since
http-request del-header If-None-Match
use_backend b-backend if { req.cook(SITEID) -m beg b-backend }
use_backend b-backend if set_cookie
default_backend ab-split-backend
it looks like what you are trying to do is keep your system from trying to pass it through via SSL and instead do clear text based assessments for testing purposes. I would recommend seeing a snippet from my config below on http-request redirects and also look into HAProxy schemes. I would also recommend seeing the additional example for instance based redirection, specifically for a dictated location, that way you dont unencrypt traffic accidentally that you want to remain encrypted.
As for the information proxied from another location, your best bet for that would be with using Cloudflare, especially if you are looking for some form of DDoS/additional layers of security. The alternative is building your own custom anti-DDoS Solution, which is a major pain.
frontend ALL
bind *:80
bind *:443 alpn h2,http/1.1 ssl crt /etc/haproxy/certs/eduarmor.com.pem
http-request redirect scheme https code 301 if !{ ssl_fc }
http-request redirect code 301 location http://www.%[hdr(host)]%[capture.req.uri] if { hdr(Host) -i eduarmor.com }
mode http
I have extremely low knowledge on haproxy. Was testing out a configuration where
S3 hosted website (route 53 alias) - content.mydomain.com
URL landing on haproxy - www.mydomain.com/getfiles/
Is it possible to redirect reroute www.mydomain.com/getfiles/ to content.mydomain.com (s3 hosted website).
I was able to redirect it to another application running on the same server with the below mentioned config
acl display-s3-content path_beg /getfiles/
use_backend my-content if display-s3-content
backend my-content
reqrep ^([^\ ]*)\ (.*) \1\ /path/\2
server test www.mycomain.com:1936
but when i try to redirect it to s3 hosted site, it does not work. Below is the backend for the not working config
reqrep ^([^\ ]*)\ (.*) \1\ /path/\2
server test1 content.mydomain.com
Thanks!
As you noted in comments, this works:
reqirep ^Host: Host:\ content.mydomain.com
The idea is to set the Host header to the value that S3 expects.
A somewhat cleaner approach is this:
http-request set-header Host content.mydomain.com
I say "better" because it uses a newer/better/safer mechanism for header manipulation, but it is fundamentally accomplishing the same purpose: changing the request header to what the destination server (S3) expects. Using the http-request approach is safer/cleaner because it is "smarter" about how it manipulates the request -- it's much easier to completely break the protocol using req[i]rep than it is with http-request.
Originally, you asked about /getfiles/test.jpg mapping to /test.jpg in the bucket. This rewrite is very easy and clean in HAProxy 1.6 and later:
http-request set-path %[path,regsub(^/getfiles,)]
...but in 1.5 you have to use reqirep since the regsub (regex substitution) converter isn't available:
reqirep ^([^\ :]+)\ +/getfiles(.*) \1\ \2
This matches the GET (HEAD, POST, etc.) line from the request and removes the extra /getfiles from the path. Including the : in the exclusion pattern prevents it from matching any other header. A similar pattern can be used to add a prefix (such as a release version, etc.) before sending the request to S3.
I posted a sample config here as well. This config will serve up:
http://<haproxy>/foo/index.html
with index.html sitting on S3.
https://gist.github.com/amslezak/6573767376860ed59a74878fbff6e61f
I have a server that listens for WebSocket connections on port 80, using Twisted and Autobahn. I want to have it also serve static HTML pages, for when the client doesn't want to use a WebSocket. Is it possible to do both things at the same time, using Twisted and Autobahn?
Sure, have a look here and here. You can run Twisted Web and add a Autobahn based WebSocket Twisted Web resource on a path. You can add any number of Twisted Web resources into your resource tree.
Briefly the technique is to start your WebSocketServerFactory manually by invoking startFactory(), then wrap it within a autobahn.twisted.resource.WebSocketResource resource, which you can then register with putChild anywhere within a Twisted Web hierarchy.
I think you have to add haproxy to the mix. If you want to just use twisted and autobahn then I don't think you can share the port. Having said that, I've got both my websockets and web server listening on the same external port. I had to use haproxy to do the trick, though...haproxy handle the inbound connection, then distributes the connection based upon things it pulls from its environment. Every environment is different. Basically, you get haproxy running, then make your web service and web socket listen on private, different ports. In my case I put my web socket server on 127.0.0.1:9000, and my web service on 127.0.0.1:8080. Then you create a haproxy.conf file for haproxy's configuration, in this example, something like:
global
maxconn 100
mode http
frontend myfrontend
bind *:80
acl is_websocket hdr(Upgrade) -i WebSocket
use_backend ws if is_websocket
default_backend mybackend
backend mybackend
server s3 127.0.0.1:8080
backend ws
timeout server 600s
reqrep ^Host:\ .* \0:9000
server ws1 127.0.0.1:9000
I had to remove a bunch of unrelated stuff from the haproxy.conf file, but this gets the idea across. It was important to me to have only one port visible from the outside instead of managing two.
haproxy is awesome !
-g
In order to test every possible solution to get Socket.io working with a parallel Apache installation, I have now installed HAproxy that listens on port 80. It proxies everything to Apache, unless the hostname equals io.server.com.
We have two IPs connected to our server: 1 is for SSL, the other for all the NON-SSL subdomains we have. I have created the io.server.com subdomain to point to that NON-SSL IP-address. However, the following this occurs:
A visit to regular_http.server.com results in Apache handling that sub domain (OK)
A visit to io.server.com results in "Welcome to Socket.io" (OK)
Next visit to regular_http.example.com results in "Welcome to Socket.io"
Why is HAproxy sending requests from a subdomain not configured to go to Socket.io, to Socket.io ?
Yes, the two sub domains share the IP, but is HAproxy really proxying the whole IP under one? What is then the point with setting up ACLs based on host name?
Here's my configuration:
global
daemon
maxconn 4096
user haproxy
group haproxy
defaults
log global
#this frontend interface receives the incoming http requests
frontend http-in
mode http
bind *:80
timeout client 86400000
#default behavior sends the requests to apache
default_backend www_backend
#when "io.test.tld" is matched, an acl I call arbitrarily
# "websocket" triggers
acl websocket hdr_end(host) -i io.server.com
use_backend node_backend if websocket
Thank you!
This problem was solved using the option http-server-close configuration value in HAproxy.