I'm puzzled why this FE doesn't seem to connect me to the BE through my HAproxy:
defaults
mode http
log global
option httplog
option dontlognull
source 0.0.0.0 usesrc clientip # transparent proxy mode
frontend fe-kb
bind :8081 ssl crt /etc/haproxy/ssl/ssl-key.pem
default_backend be-kb
backend be-kb
server afnB afnB:1080 check
I get this in HA http log:
Jan 9 17:25:04 localhost haproxy[17266]: <ip redacted>:51396 [09/Jan/2016:17:24:44.544] fe-kb~ be-kb/afnB 31/0/-1/-1/20036 503 212 - - cC-- 0/0/0/0/3 0/0 "GET / HTTP/1.1"
I can connect fine from HAproxy CLI (selinux is disabled):
[root#hapA ~]# telnet afnB 1080
Trying 10.45.69.14...
Connected to afnB.
Escape character is '^]'.
GET / HTTP/1.0
HTTP/1.1 200 OK
Server: nginx/1.9.9
Date: Sat, 09 Jan 2016 16:40:44 GMT
Content-Type: text/html
Content-Length: 612
Last-Modified: Wed, 09 Dec 2015 15:05:19 GMT
Connection: close
ETag: "5668432f-264"
Accept-Ranges: bytes
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
...
Figured it out, the default route wasn't through my HAproxy and as I was trying to do transparency of the source IPs well ;)
Thanks for watching!
Related
I installed odoo on centos 8 and use httpd as a reverse proxy. Like other Apache hardening, I use ServerTokens Proddan ServerSignature Off to hide server information.
but when I try wget the results still show server information
Spider mode enabled. Check if remote file exists.
--2020-03-12 11:57:14-- http://my.domain/
Resolving my.domain (my.domain)... 169.0.0.1
Connecting to my.domain (my.domain)|169.0.0.1|:80... connected.
HTTP request sent, awaiting response...
HTTP/1.1 301 Moved Permanently
Content-length: 0
Location: https://my.domain/
Location: https://my.domain/ [following]
Spider mode enabled. Check if remote file exists.
--2020-03-12 11:57:14-- https://my.domain/
Connecting to my.domain (my.domain)|169.0.0.1|:443... connected.
HTTP request sent, awaiting response...
HTTP/1.1 200 OK
Date: Thu, 12 Mar 2020 04:56:55 GMT
Server: Werkzeug/0.14.1 Python/3.7.5
Content-Type: text/html; charset=utf-8
Content-Length: 10589
Set-Cookie: frontend_lang=en_US; Path=/
Set-Cookie: session_id=s8487a5ec76bd455f42680c38195b5f7f0285d563; Expires=Wed, 10-Jun-2020 04:56:55 GMT; Max-Age=7776000; HttpOnly; Path=/
Vary: User-Agent
Length: 10589 (10K) [text/html]
Remote file exists and could contain further links,
but recursion is disabled -- not retrieving.
well it can use mod_header and mod_rewrite, then add
add Header set Server "value that you want" to your apache virtual host
As discussed in my other question there is no support for websockets authentication in Knox, but as a temporary solution we could handle authentication in our backend service. Our test has shown however that Knox does not pass Authorization header to the backend.
[client]$ curl -i -u '<user>:<password>' https://knox-server/gateway/default/myservice/ping
# 8090 is our backend port
[knox-server]$ ngrep -W byline port 8090
interface: eth0
filter: ( port 8090 ) and ((ip || ip6) || (vlan && (ip || ip6)))
#
T <knox-server>:59118 -> <myservice>:8090 [AP]
GET /ping?doAs=<user> HTTP/1.1.
X-Forwarded-For: <client>.
X-Forwarded-Proto: https.
X-Forwarded-Port: 443.
X-Forwarded-Host: <knox-server>.
X-Forwarded-Server: <knox-server>.
X-Forwarded-Context: /gateway/default.
User-Agent: curl/7.54.0.
Accept: */*.
Host: <myservice>:8090.
Connection: Keep-Alive.
Accept-Encoding: gzip,deflate.
.
#
T <myservice>:8090 -> <knox-server>:59118 [AP]
HTTP/1.1 200 OK.
Date: Sat, 14 Oct 2017 14:27:58 GMT.
X-Application-Context: myservice:prod:8090.
Content-Type: text/plain;charset=utf-8.
Content-Length: 4.
.
PONG
How should I configure Knox (0.12.0 from HDP 2.6.2) to make it pass Authorization header to the backend for websocket connection?
While writing this question I realised that there is a ticket KNOX-895 resolving the issue of passing cookies and headers to a backend service in Knox 0.14.0.
[EDIT]
I cloned knox git repo (commit 92b1505a), which includes KNOX-895 (2d236e78), run it locally with added websocket service to sandbox topology.
[tulinski]$ wscat -n --auth 'user:password' -c wss://localhost:8443/gateway/sandbox/echows
[tulinski]$ sudo ngrep -W byline host echo.websocket.org
#
T 192.168.0.16:59952 -> 174.129.224.73:80 [AP]
GET / HTTP/1.1.
Host: echo.websocket.org.
Upgrade: websocket.
Connection: Upgrade.
Sec-WebSocket-Key: Z4Qa9Dxwr6Qvq2QAicsT5Q==.
Sec-WebSocket-Version: 13.
Pragma: no-cache.
Cache-Control: no-cache.
Authorization: Basic dXNlcjpwYXNzd29yZA==.
.
##
T 174.129.224.73:80 -> 192.168.0.16:59952 [AP]
HTTP/1.1 101 Web Socket Protocol Handshake.
Connection: Upgrade.
Date: Mon, 16 Oct 2017 14:23:49 GMT.
Sec-WebSocket-Accept: meply+6cIyjbH+Vk2OsAqKJDWic=.
Server: Kaazing Gateway.
Upgrade: websocket.
.
Authorization header is passed to the backend service.
Below is an example HTTP 1.1 call with a single page requested :
GET /jq.js HTTP/1.1
Host: 127.0.0.1
Accept: */*
I understand with HTTP Pipelining, multiple requests can be sent without breaking the connection.
Can someone post, some text example of how this request will be sent to the server, I want to be able to do it over the command line or with PHP sockets.
Does support for pipelining need to enabled on the web-server as well?
Is pipelining supported by major Web-servers(apache, nginx) by default or does it need to be enabled
From w3c protocol details:
8.1.2.2 Pipelining
A client that supports persistent connections MAY "pipeline" its requests (i.e., send multiple requests without waiting for each response). A server MUST send its responses to those requests in the same order that the requests were received.
Clients which assume persistent connections and pipeline immediately after connection establishment SHOULD be prepared to retry their connection if the first pipelined attempt fails. If a client does such a retry, it MUST NOT pipeline before it knows the connection is persistent. Clients MUST also be prepared to resend their requests if the server closes the connection before sending all of the corresponding responses.
Clients SHOULD NOT pipeline requests using non-idempotent methods or non-idempotent sequences of methods (see section 9.1.2). Otherwise, a premature termination of the transport connection could lead to indeterminate results. A client wishing to send a non-idempotent request SHOULD wait to send that request until it has received the response status for the previous request.
So, first fact is that you should be in a KeepAlive status. So you should add Connection: keep-alive keyword in your request headers, but some webservers may still accept pipelining without this keep alive status. On the other hand, this could be rejected by the server, the server may or may not accept your connection in keepalive mode. So, at any time, being in keepalived or not, you may send 3 requests pipelined in one connection, and get only one response.
From this gist we can find a nice way to test it with telnet.
Asking for keepalive with Connection: keep-alive header:
(echo -en "GET /index.html HTTP/1.1\nHost: foo.com\nConnection: keep-alive\n\nGET /index.html HTTP/1.1\nHost: foo.com\n\n"; sleep 10) | telnet localhost 80
Trying 127.0.0.1...
Connected to localhost.lan.
Escape character is '^]'.
HTTP/1.1 200 OK
Date: Sun, 27 Oct 2013 17:51:58 GMT
Server: Apache/2.2.22 (Debian)
Last-Modified: Sun, 04 Mar 2012 15:00:29 GMT
ETag: "56176e-3e-4ba6c121c4761"
Accept-Ranges: bytes
Content-Length: 62
Vary: Accept-Encoding
Keep-Alive: timeout=5, max=100 <======= Keepalive!
Connection: Keep-Alive
Content-Type: text/html; charset=utf-8
<html>
<body>
<h1>test</h1>
</body>
</html>
HTTP/1.1 200 OK
Date: Sun, 27 Oct 2013 17:51:58 GMT
Server: Apache/2.2.22 (Debian)
Last-Modified: Sun, 04 Mar 2012 15:00:29 GMT
ETag: "56176e-3e-4ba6c121c4761"
Accept-Ranges: bytes
Content-Length: 62
Vary: Accept-Encoding
Content-Type: text/html; charset=utf-8
<html>
<body>
<h1>test</h1>
</body>
</html>
It works.
Without asking for Keepalive:
(echo -en "GET /index.html HTTP/1.1\nHost: foo.com\nConnection: keep-alive\n\nGET /index.html HTTP/1.1\nHost: foo.com\n\n"; sleep 10) | telnet localhost 80
Trying 127.0.0.1...
Connected to localhost.lan.
Escape character is '^]'.
HTTP/1.1 200 OK
Date: Sun, 27 Oct 2013 17:49:37 GMT
Server: Apache/2.2.22 (Debian)
Last-Modified: Sun, 04 Mar 2012 15:00:29 GMT
ETag: "56176e-3e-4ba6c121c4761"
Accept-Ranges: bytes
Content-Length: 62
Vary: Accept-Encoding
Content-Type: text/html; charset=utf-8
<html>
<body>
<h1>test</h1>
</body>
</html>
HTTP/1.1 200 OK
Date: Sun, 27 Oct 2013 17:49:37 GMT
Server: Apache/2.2.22 (Debian)
Last-Modified: Sun, 04 Mar 2012 15:00:29 GMT
ETag: "56176e-3e-4ba6c121c4761"
Accept-Ranges: bytes
Content-Length: 62
Vary: Accept-Encoding
Content-Type: text/html; charset=utf-8
<html>
<body>
<h1>test</h1>
</body>
</html>
Connection closed by foreign host.
Same result, I did not ask for it but it looks like a Keepalive answer (closing after 5s which is the value set in Apache). And a pipelined answer, I get my two pages.
Now if I prevent usage of any Keepalive connection in Apache by setting:
Keepalive Off
And restarting it:
(echo -en "GET /index.html HTTP/1.1\nHost: foo.com\nConnection: keep-alive\n\nGET /index.html HTTP/1.1\nHost: foo.com\n\n"; sleep 10) | telnet localhost 80
Trying 127.0.0.1...
Connected to localhost.lan.
Escape character is '^]'.
HTTP/1.1 200 OK
Date: Sun, 27 Oct 2013 18:02:41 GMT
Server: Apache/2.2.22 (Debian)
Last-Modified: Sun, 04 Mar 2012 15:00:29 GMT
ETag: "56176e-3e-4ba6c121c4761"
Accept-Ranges: bytes
Content-Length: 62
Vary: Accept-Encoding
Connection: close
Content-Type: text/html; charset=utf-8
<html>
<body>
<h1>test</h1>
</body>
</html>
Connection closed by foreign host.
Only one answer... So the server can reject my request for pipelining.
Now, for support on servers and browsers, I think your wikipedia source tells enough :-)
I'm using Nginx + uwsgi + python3
Sending any header via start_response goes well, but when I want to send more than one header, it becomes mad.
For example, if I write:
start_response('200 OK', [('Last-Modified', 'Wed, 11 Jan 2012 00:00:00 GMT'), ('Content-Type', 'text/html; charset=windows-1251')])
The headers sent are:
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Server: nginx/1.0.11
Connection: close
Date: Wed, 11 Jan 2012 04:17:22 GMT
Content-Type: text/html; charset=windows-1251
Content-Type: text/html; charset=windows-12
uwsgi sends the same header twice and even more the second one is broken.
which uWSGI and nginx version ? In both 0.9.8.x and 1.0.x i cannot reproduce your error.
You can check the real headers sent by uWSGI putting it in http mode with --http/--http-socket
I'm having a problem where items served by my Squid server are being cached by Limelight for too long, sometimes days. It happens when a piece of content has been static for a long time (weeks) and then undergoes numerous changes in a matter of hours.
Limelight gets its content from our Squid server and I'm told that if I can add 'Expires: 15m' in the HTTP header the Squid server sends, Limelight will not cache the image for more than 15 min.
Unfortunately, I can fond no setting in Squid that will allow me to add this to the header.
Here's the HTTP header as presently being sent:
HTTP/1.0 200 OK
Date: Tue, 15 Dec 2009 23:57:33 GMT
Server: nginx/0.5.26
Content-Type: image/jpeg
Content-Length: 83843
Last-Modified: Tue, 15 Dec 2009 23:52:00 GMT
Accept-Ranges: bytes
Age: 450
X-Cache: HIT from squid01.prod.mydomain
X-Cache-Lookup: HIT from squid01.prod.mydomain:3128
Via: 1.0 squid01.prod.mydomain:3128 (squid/2.6.STABLE14)
Connection: close
You need to set the header on the origin server, not on your Squid box.
See:
http://www.mnot.net/cache_docs/#IMP-SERVER