I use gwan for image generation, so I need to set correct content type, but G-WAN 3.12.26 after some load adds its own header with content type text/html and returns page with 2 http headers.
How to reproduce this:
use setheaders.c servlet from gwan package, start gwan and open this page, lets say http://localhost/?setheaders.c and you will get this (correct response):
HTTP/1.1 200 OK
Date: Sat, 29 Dec 2012 20:37:52 GMT
Last-Modified: Sat, 29 Dec 2012 20:37:52 GMT
Content-type: text/html
Content-Length: 371
Connection: close
<!DOCTYPE HTML><html lang="en"><head><title>Setting response headers</title><meta http-equiv="Content-Type" content="text/html; charset=utf-8"><link href="imgs/style.css" rel="stylesheet" type="text/css"></head><body style="margin:16px;"><h1>Setting response headers</h1><br>This reply was made with custom HTTP headers, look at the servlet source code.<br></body></html>`
now run apache bench: ab -n 1000 'http://localhost/?setheaders.c' (1000 requests were enough for my system).
DO NOT RESTART GWAN, open http://localhost/?setheaders.c again and this is what you should get (incorrect response, 2 http headers):
HTTP/1.1 200 OK
Server: G-WAN
Date: Sat, 29 Dec 2012 20:43:34 GMT
Last-Modified: Fri, 16 Jan 1970 16:53:33 GMT
ETag: "be86ada7-14b40d-16f"
Vary: Accept-Encoding
Accept-Ranges: bytes
Content-Type: text/html; charset=UTF-8
Content-Length: 367
Content-Encoding: gzip
Connection: close
HTTP/1.1 200 OK
Date: Sat, 29 Dec 2012 20:43:34 GMT
Last-Modified: Sat, 29 Dec 2012 20:43:34 GMT
Content-type: text/html
Content-Length: 371
Connection: close
<!DOCTYPE HTML><html lang="en"><head><title>Setting response headers</title><meta http-equiv="Content-Type" content="text/html; charset=utf-8"><link href="imgs/style.css" rel="stylesheet" type="text/css"></head><body style="margin:16px;"><h1>Setting response headers</h1><br>This reply was made with custom HTTP headers, look at the servlet source code.<br></body></html>
GWAN returns correct response if gzip and x-gzip are not set as acceptable encoding in request header (Accept-Encoding: gzip, x-gzip).
Is it possible to solve this modifying just servlet? If yes, then how?
Are you setting the MIME type like shown in fractal.c:
// -------------------------------------------------------------------------
// specify a MIME type so we don't have to build custom HTTP headers
// -------------------------------------------------------------------------
char *mime = (char*)get_env(argv, REPLY_MIME_TYPE);
// note that we setup the FILE EXTENTION, not the MIME type:
mime[0] = '.'; mime[1] = 'g'; mime[2] = 'i'; mime[3] = 'f'; mime[4] = 0;
If you do so then there's no way to confuse the automatic headers feature.
Other than that, v3.12 has had many instability problems (file time failures, pthread failures, signals failures, etc.) due to our direct syscalls and GLIBC wrappers - an effort initially intended to make the program run on all versions of Linux.
We have found (thanks to the many reports like yours) that rather than trying to fix those issues one by one (pointlessly fighting GLIBC, a moving target with many different releases each having its own bugs and specificities) a much safer path is to ditch GLIBC.
That's what G-WAN v4 will do, just a few days from now.
Related
I am accessing OneDrive from a C++ program using WinHttp* functions. What absolutely baffles me is REST API download command. I am downloading a small test picture with
GET /v1.0/drive/items/FF306293D40F9529!158/content
When I do this in the console, I get a response exactly as described in the docs, that is, a 302 redirect to the actual content. However when I send the same request with WinHttpSendRequest, I receive the actual file contents instead of a redirection:
HTTP/1.1 200 OK
Cache-Control: public
Date: Wed, 06 Jan 2016 10:22:36 GMT
Content-Length: 161796
Content-Type: image/jpeg
Content-Location: https://public-bn1306.files.1drv.com/blahblahbla
Expires: Tue, 05 Apr 2016 10:22:36 GMT
Last-Modified: Wed, 06 Jan 2016 10:22:35 GMT
Accept-Ranges: bytes
ETag: aRkYzMDYyOTNENDBGOTUyOSExNTguMw
P3P: CP="BUS CUR CONo FIN IVDo ONL OUR PHY SAMo TELo"
Server: Microsoft-IIS/8.5
X-MSNSERVER: BN1306____PAP099
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-SqlDataOrigin: S
CTag: aYzpGRjMwNjI5M0Q0MEY5NTI5ITE1OC4yNTc
X-PreAuthInfo: rv;poba;
Content-Disposition: attachment; filename="Test.jpg"
X-Content-Type-Options: nosniff
X-StreamOrigin: X
X-AsmVersion: UNKNOWN; 19.33.0.0
X-MSEdge-Ref: Ref A: 44F40B6FB83547EFBC895911207BDB42 Ref B: 96336B0FC1B608598B549A1CB70C8C59 Ref C: Wed Jan 06 02:22:36 2016 PST
First I thought that maybe it was because the file was small, but trying to download a 250Mb large binary file changed nothing.
I don't complain actually, this would make the program a bit simpler, but I certainly would like to know why the same request works differently with WinHttp*. I suppose I do something wrong, but what??? (Banging my head against my desk.)
There are http libraries that will follow redirects automatically. You send a request, the library receives 302, and decides on its own to go to the redirect location, which then returns a 200.
I've put this in my head section. It appears in the page source in the browser.
<meta http-equiv="Cache-Control" content="max-age=1209600">
However, when I look in the Chrome extension Live HTTP Headers, it says the following.
Cache-Control: max-age=0
Content-Encoding: gzip
Content-Length: 5849
Content-Type: text/html; charset=utf-8
Date: Sat, 05 Apr 2014 04:29:16 GMT
Expires: Sat, 05 Apr 2014 04:29:16 GMT
Last-Modified: Sat, 05 Apr 2014 03:33:19 GMT
The max-age isn't registering. I've emptied the browser cache but it makes no difference.
Any explanations? This is the site, incidentally.
UPDATES:
Firebug similarly records Cache-Control: max-age=0.
Google also makes clear here that max-age overrides the Expires header (which I don't set) and that you don't need both.
When you use tools like Live HTTP Headers, they show you the actual HTTP headers sent by the browser. What they do with meta tags used to simulate HTTP headers is a different issue. We can expect any conflict to be resolved in favor of the actual headers. (This has been normatively specified in HTML specs for Content-Type headers.)
To control cacheing, you should (at least primarily) use server configuration. See Caching Tutorial for Web Authors and Webmasters.
Below is an example HTTP 1.1 call with a single page requested :
GET /jq.js HTTP/1.1
Host: 127.0.0.1
Accept: */*
I understand with HTTP Pipelining, multiple requests can be sent without breaking the connection.
Can someone post, some text example of how this request will be sent to the server, I want to be able to do it over the command line or with PHP sockets.
Does support for pipelining need to enabled on the web-server as well?
Is pipelining supported by major Web-servers(apache, nginx) by default or does it need to be enabled
From w3c protocol details:
8.1.2.2 Pipelining
A client that supports persistent connections MAY "pipeline" its requests (i.e., send multiple requests without waiting for each response). A server MUST send its responses to those requests in the same order that the requests were received.
Clients which assume persistent connections and pipeline immediately after connection establishment SHOULD be prepared to retry their connection if the first pipelined attempt fails. If a client does such a retry, it MUST NOT pipeline before it knows the connection is persistent. Clients MUST also be prepared to resend their requests if the server closes the connection before sending all of the corresponding responses.
Clients SHOULD NOT pipeline requests using non-idempotent methods or non-idempotent sequences of methods (see section 9.1.2). Otherwise, a premature termination of the transport connection could lead to indeterminate results. A client wishing to send a non-idempotent request SHOULD wait to send that request until it has received the response status for the previous request.
So, first fact is that you should be in a KeepAlive status. So you should add Connection: keep-alive keyword in your request headers, but some webservers may still accept pipelining without this keep alive status. On the other hand, this could be rejected by the server, the server may or may not accept your connection in keepalive mode. So, at any time, being in keepalived or not, you may send 3 requests pipelined in one connection, and get only one response.
From this gist we can find a nice way to test it with telnet.
Asking for keepalive with Connection: keep-alive header:
(echo -en "GET /index.html HTTP/1.1\nHost: foo.com\nConnection: keep-alive\n\nGET /index.html HTTP/1.1\nHost: foo.com\n\n"; sleep 10) | telnet localhost 80
Trying 127.0.0.1...
Connected to localhost.lan.
Escape character is '^]'.
HTTP/1.1 200 OK
Date: Sun, 27 Oct 2013 17:51:58 GMT
Server: Apache/2.2.22 (Debian)
Last-Modified: Sun, 04 Mar 2012 15:00:29 GMT
ETag: "56176e-3e-4ba6c121c4761"
Accept-Ranges: bytes
Content-Length: 62
Vary: Accept-Encoding
Keep-Alive: timeout=5, max=100 <======= Keepalive!
Connection: Keep-Alive
Content-Type: text/html; charset=utf-8
<html>
<body>
<h1>test</h1>
</body>
</html>
HTTP/1.1 200 OK
Date: Sun, 27 Oct 2013 17:51:58 GMT
Server: Apache/2.2.22 (Debian)
Last-Modified: Sun, 04 Mar 2012 15:00:29 GMT
ETag: "56176e-3e-4ba6c121c4761"
Accept-Ranges: bytes
Content-Length: 62
Vary: Accept-Encoding
Content-Type: text/html; charset=utf-8
<html>
<body>
<h1>test</h1>
</body>
</html>
It works.
Without asking for Keepalive:
(echo -en "GET /index.html HTTP/1.1\nHost: foo.com\nConnection: keep-alive\n\nGET /index.html HTTP/1.1\nHost: foo.com\n\n"; sleep 10) | telnet localhost 80
Trying 127.0.0.1...
Connected to localhost.lan.
Escape character is '^]'.
HTTP/1.1 200 OK
Date: Sun, 27 Oct 2013 17:49:37 GMT
Server: Apache/2.2.22 (Debian)
Last-Modified: Sun, 04 Mar 2012 15:00:29 GMT
ETag: "56176e-3e-4ba6c121c4761"
Accept-Ranges: bytes
Content-Length: 62
Vary: Accept-Encoding
Content-Type: text/html; charset=utf-8
<html>
<body>
<h1>test</h1>
</body>
</html>
HTTP/1.1 200 OK
Date: Sun, 27 Oct 2013 17:49:37 GMT
Server: Apache/2.2.22 (Debian)
Last-Modified: Sun, 04 Mar 2012 15:00:29 GMT
ETag: "56176e-3e-4ba6c121c4761"
Accept-Ranges: bytes
Content-Length: 62
Vary: Accept-Encoding
Content-Type: text/html; charset=utf-8
<html>
<body>
<h1>test</h1>
</body>
</html>
Connection closed by foreign host.
Same result, I did not ask for it but it looks like a Keepalive answer (closing after 5s which is the value set in Apache). And a pipelined answer, I get my two pages.
Now if I prevent usage of any Keepalive connection in Apache by setting:
Keepalive Off
And restarting it:
(echo -en "GET /index.html HTTP/1.1\nHost: foo.com\nConnection: keep-alive\n\nGET /index.html HTTP/1.1\nHost: foo.com\n\n"; sleep 10) | telnet localhost 80
Trying 127.0.0.1...
Connected to localhost.lan.
Escape character is '^]'.
HTTP/1.1 200 OK
Date: Sun, 27 Oct 2013 18:02:41 GMT
Server: Apache/2.2.22 (Debian)
Last-Modified: Sun, 04 Mar 2012 15:00:29 GMT
ETag: "56176e-3e-4ba6c121c4761"
Accept-Ranges: bytes
Content-Length: 62
Vary: Accept-Encoding
Connection: close
Content-Type: text/html; charset=utf-8
<html>
<body>
<h1>test</h1>
</body>
</html>
Connection closed by foreign host.
Only one answer... So the server can reject my request for pipelining.
Now, for support on servers and browsers, I think your wikipedia source tells enough :-)
I'm using Nginx + uwsgi + python3
Sending any header via start_response goes well, but when I want to send more than one header, it becomes mad.
For example, if I write:
start_response('200 OK', [('Last-Modified', 'Wed, 11 Jan 2012 00:00:00 GMT'), ('Content-Type', 'text/html; charset=windows-1251')])
The headers sent are:
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Server: nginx/1.0.11
Connection: close
Date: Wed, 11 Jan 2012 04:17:22 GMT
Content-Type: text/html; charset=windows-1251
Content-Type: text/html; charset=windows-12
uwsgi sends the same header twice and even more the second one is broken.
which uWSGI and nginx version ? In both 0.9.8.x and 1.0.x i cannot reproduce your error.
You can check the real headers sent by uWSGI putting it in http mode with --http/--http-socket
I use Fiddler to debug my application. Whenever the response is compressed by server, instead of decompressed response, Fiddler shows unreadable binary data:
/* Response to my request (POST) */
HTTP/1.1 200 OK
Server: xyz.com
Date: Tue, 07 Jun 2011 22:22:21 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
X-Powered-By: PHP/5.3.3
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Content-Encoding: gzip
14
����������������
0
How can I get the response decompressed?
I use fiddler version 2.3.4.4 and just noticed that in the Inspectors tab ("Raw" sub-tab), above the response section (in case of gzip-ed response), appears "Response is encoded and may need to be decoded before inspection. Click here to transform."
If you click on that, the response becomes readable.
The settings are pretty much the default, I just installed Fiddler and did not change anything.
If you don't want to have to click per response as in the accepted answer, using the menu, click Rules -> Remove All Encodings.
From the fiddler faq
Q: I like to navigate around a site then do a "search" for a text on all the logged request/responses. I was curious if Fiddler automatically decompressed gzipped responses during search?
A: Fiddler does not decompress during searches by default, since it would need to keep both the compressed and decompressed body in memory (for data integrity reasons).
In current versions of Fiddler, you can tick the "Decode Compressed Content" checkbox on the Find dialog.
Here is a link to the site
http://www.fiddler2.com/fiddler/help/faq.asp