Per my understanding:
the Accept header is used by HTTP clients to tell the server what content types they'll accept. The server will then send back a response, which will include a Content-Type header telling the client what the content type of the returned content actually is.
With this understanding, I tried the following:
curl -X GET -H "Accept: application/xml" http://www.google.com -v
* About to connect() to www.google.com port 80 (#0)
* Trying 173.194.33.81...
* connected
* Connected to www.google.com (173.194.33.81) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8y zlib/1.2.5
> Host: www.google.com
> Accept: application/xml
>
< HTTP/1.1 200 OK
< Date: Tue, 02 Sep 2014 17:58:05 GMT
< Expires: -1
< Cache-Control: private, max-age=0
< Content-Type: text/html; charset=ISO-8859-1
< Set-Cookie: PREF=ID=5c30672b67a74789:FF=0:TM=1409680685:LM=1409680685:S=PsGclk3vR4HWjann; expires=Thu, 01-Sep-2016 17:58:05 GMT; path=/; domain=.google.com
< Set-Cookie: NID=67=rPuxpwUu5UNuapzCdbD5iwVyjjC9TzP_Ado29h3ucjEq4A_2qkSM4nQM3RO02rfyuHmrh-hvmwmgFCmOvISttFfHv06f8ay4_6Gl4pXRjqxihNhJSGbvujjDRzaSibfy; expires=Wed, 04-Mar-2015 17:58:05 GMT; path=/; domain=.google.com; HttpOnly
< P3P: CP="This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info."
< Server: gws
< X-XSS-Protection: 1; mode=block
< X-Frame-Options: SAMEORIGIN
< Alternate-Protocol: 80:quic
< Transfer-Encoding: chunked
<
<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang="en"><
As you can notice in the response, I am sent Content-Type: text/html; charset=ISO-8859-1 which is not what I asked for?
why does a different representation (HTML in this case) is sent, although I asked for xml?
Thanks
From RFC 2616:
If an Accept header field is present,
and if the server cannot send a response which is acceptable
according to the combined Accept field value, then the server SHOULD
send a 406 (not acceptable) response.
Here, "should" means that Google aren't actually obliged to throw a 406 error. But since you're receiving an HTML response, it has effectively the same meaning.
Related
I was trying to write a karate test that validated that a particular response header contained the Content-Encoding header field with a value of gzip. I tried on my api where both times the content-encoding field was missing from karate's response. Both of these endpoints returned the content-encoding field on postman and curl commands.
I then tried to hit the postman-echo service to see if it was my api endpoint that karate was having issues with and it seems that it is not only my api. Could someone take a look at my code and see if I am doing something incorrectly to get the header field to show up in the response?
Feature: test getting Content-Encoding
Background:
* url 'https://postman-echo.com/gzip'
Scenario:
Given header Accept-Encoding = 'gzip'
When method get
Then status 200
And match responseHeaders contains {'Content-Encoding':'#present'}
This is what karate request looks like
1 > GET https://postman-echo.com/gzip
1 > Accept-Encoding: gzip
1 > Connection: Keep-Alive
1 > Host: postman-echo.com
1 > User-Agent: Apache-HttpClient/4.5.12 (Java/1.8.0_252)
and the response looks like
1 < 200
1 < Connection: keep-alive
1 < Content-Type: application/json; charset=utf-8
1 < Date: Fri, 08 May 2020 16:18:42 GMT
1 < ETag: W/"ef-7kclc8pzXTvQiPUaEOf6j95iFaE"
1 < Vary: Accept-Encoding
1 < set-cookie: sails.sid=s%3A6G_FShPRZH4V1G-tVDfUEEfMwQQmolo5.T2Cb37zqYA21FTyRyIGutVWQWo9ta4EWiod36%2FkM88I; Path=/; HttpOnly
{
"gzipped": true,
"headers": {
"x-forwarded-proto": "https",
"x-forwarded-port": "443",
"host": "postman-echo.com",
"x-amzn-trace-id": "Root=1-5eb58662-c4aaeec26efd116ac0544a18",
"accept-encoding": "gzip",
"user-agent": "Apache-HttpClient/4.5.12 (Java/1.8.0_252)"
},
"method": "GET"
}
The curl response header is
curl --location --request GET 'https://postman-echo.com/gzip' \
> --header 'Accept-Encoding: gzip' -I
HTTP/2 200
date: Fri, 08 May 2020 16:21:53 GMT
content-type: application/json; charset=utf-8
content-length: 220
content-encoding: gzip
etag: W/"dc-BuD8DN1qXT7trYUQtZOuSvbq1pM"
vary: Accept-Encoding
set-cookie: sails.sid=s%3Aj86lznX3nK20fnEN4B3nbHESrfWqVJ3M.236VrsmQp7V%2F7%2BrvG%2FEtlc9yUVLTtylh1yyIAdQJSiY; Path=/; HttpOnly
This seems to be the way the Apache HttpClient works or how it is configured for Karate. I just found you get the header back with karate-jersey:
1 > GET https://postman-echo.com/gzip
1 > Accept: */*
1 > Accept-Encoding: gzip
1 > User-Agent: Jersey/2.30 (HttpUrlConnection 1.8.0_231)
22:40:48.981 [ForkJoinPool-1-worker-1] DEBUG com.intuit.karate - response time in milliseconds: 1177.37
1 < 200
1 < Connection: keep-alive
1 < Content-Encoding: gzip
1 < Content-Length: 248
1 < Content-Type: application/json; charset=utf-8
1 < Date: Fri, 08 May 2020 17:10:48 GMT
1 < ETag: W/"f8-sigbV4PuNI2Fx08AqzMEqW1WIYY"
1 < Vary: Accept-Encoding
So if Jersey is ok for the rest of your tests, maybe that's all you need. Getting this to work for karate-apache is not a priority for me so if you or anyone else is willing to investigate or fix it, that would be great.
I am not sure why i am getting this error message. i have inserted the ids
POST /oauth2/v4/token HTTP/1.1
Host: www.googleapis.com
Content-length: 322
content-type: application/x-www-form-urlencoded
user-agent: google-oauth-playground
HTTP/1.1 400 Bad Request
Content-length: 68
X-xss-protection: 1; mode=block
X-content-type-options: nosniff
Transfer-encoding: chunked
Vary: Origin, X-Origin, Referer
Server: ESF
-content-encoding: gzip
Cache-control: private
Date: Sun, 11 Nov 2018 23:59:34 GMT
X-frame-options: SAMEORIGIN
Alt-svc: quic=":443"; ma=2592000; v="44,43,39,35"
Content-type: application/json; charset=utf-8
{
"error_description": "Bad Request",
"error": "invalid_grant"
}
Bad Request error occurs when the request syntax are wrong (i.e. when a parameter or a header you are supposed to send is missing or contains an incorrect/invalid value). Best option is to check OAuth Specification are figure out what you have done wrong in the token request. Google might have a slightly different implementation from the actual specification, but I think it won't be difficult to find the issue after checking the documentation.
Access token request specification for authorization code grant can be found here.
I would like to automate chrome extension deployment and followed this guide for the most part: https://developer.chrome.com/webstore/using_webstore_api. I created a project in my developer console, activated the Chrome Web Store API. Next step is creating the right credentials. There are three different ones, when i let google help me choose it tells me that API-key is just fine to use for this API, so i did.
My question: How can i use this API-key to make API calls? I tried to include the key in the params as pointed out in the docs but that doesn't work. If this is the wrong approach i really don't understand, that these sort of authentication is suggested by the developer console.
As i try to automate things on a build server without using chrome or some other browser i cannot really use OAuth credentials.
curl \
-H "x-goog-api-version: 2" \
-H "Content-Length: 0" \
-H "Expect:" \
-X GET \
-v \
"https://www.googleapis.com/chromewebstore/v1.1/items/ITEM_ID?projection=draft&key=API_KEY"
results in
https://www.googleapis.com/chromewebstore/v1.1/items/ITEM_ID\?projection\=draft\&key\=API_KEY
* Trying 172.217.17.234...
* Connected to www.googleapis.com (172.217.17.234) port 443 (#0)
* TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate: *.googleapis.com
* Server certificate: Google Internet Authority G2
* Server certificate: GeoTrust Global CA
> GET /chromewebstore/v1.1/items/ITEM_ID?projection=draft&key=API_KEY HTTP/1.1
> Host: www.googleapis.com
> User-Agent: curl/7.43.0
> Accept: */*
> x-goog-api-version: 2
> Content-Length: 0
>
< HTTP/1.1 401 Unauthorized
< Vary: X-Origin
< WWW-Authenticate: Bearer realm="https://accounts.google.com/"
< Content-Type: application/json; charset=UTF-8
< Date: Thu, 15 Sep 2016 14:14:18 GMT
< Expires: Thu, 15 Sep 2016 14:14:18 GMT
< Cache-Control: private, max-age=0
< X-Content-Type-Options: nosniff
< X-Frame-Options: SAMEORIGIN
< X-XSS-Protection: 1; mode=block
< Server: GSE
< Alt-Svc: quic=":443"; ma=2592000; v="36,35,34,33,32"
< Accept-Ranges: none
< Vary: Origin,Accept-Encoding
< Transfer-Encoding: chunked
<
* Connection #0 to host www.googleapis.com left intact
{"error":{"errors":[{"domain":"global","reason":"required","message":"Login Required","locationType":"header","location":"Authorization"}],"code":401,"message":"Login Required"}}
I also tried to set it as header but neither of these worked.
I am verifying if my application handles file content delivered through chunked-encoding mode. I am not sure what change to make to the httpd.conf file to force chunked encoding through Apache. Is it even possible to do this with Apache server, if not what would be an easier solution? I am using Apache 2.4.2 and HTTP 1.1.
By default, keep-alive is On in Apache and I do not see the data as chunked when testing with wireshark.
EDIT: Added more info:
Only way I managed to do this was by enabling the deflate module.
Then I configured my client to send "Accept-Encoding: gzip, deflate" header and apache would compress and send the file back in chunked mode.
I had to enable the file type in the module though.
AddOutputFilterByType DEFLATE image/png
See example:
curl --raw -v --header "Accept-Encoding: gzip, deflate" http://localhost/image.png | more
* Connected to localhost (127.0.0.1) port 80 (#0)
> GET /image.png HTTP/1.1
> User-Agent: curl/7.35.0
> Host: localhost
> Accept: */*
> Accept-Encoding: gzip, deflate
>
< HTTP/1.1 200 OK
< Date: Mon, 13 Apr 2015 10:08:45 GMT
* Server Apache/2.4.7 (Ubuntu) is not blacklisted
< Server: Apache/2.4.7 (Ubuntu)
< Last-Modified: Mon, 13 Apr 2015 09:48:53 GMT
< ETag: "3b5306-5139805976dae-gzip"
< Accept-Ranges: bytes
< Vary: Accept-Encoding
< Content-Encoding: gzip
< Transfer-Encoding: chunked
< Content-Type: image/png
<
This resource produces chunked results http://www.httpwatch.com/httpgallery/chunked/
which is very useful for testing clients. You can see this by running
$ curl --raw -i http://www.httpwatch.com/httpgallery/chunked/
HTTP/1.1 200 OK
Cache-Control: private,Public
Transfer-Encoding: chunked
Content-Type: text/html
Server: Microsoft-IIS/7.5
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Mon, 22 Jul 2013 09:41:04 GMT
7b
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
2d
<html xmlns="http://www.w3.org/1999/xhtml">
....
I tried this way to get HTTP chunked encoded data in Ubuntu, it might help.
In apache server create a file index.php in your directory where index page is there ( ex : /var/www/html/) and paste below content (should have php installed):
<?php phpinfo(); ?>
Then try to curl the page as below :
root#ubuntu-16:~# curl -v http://10.11.0.230:2222/index.php
* Trying 10.11.0.230...
* Connected to 10.11.0.230 (10.11.0.230) port 2222 (#0)
> GET /index.php HTTP/1.1
> Host: 10.11.0.230:2222
> User-Agent: curl/7.47.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Wed, 01 Jul 2020 07:51:24 GMT
< Server: Apache/2.4.18 (Ubuntu)
< Vary: Accept-Encoding
< Transfer-Encoding: chunked
< Content-Type: text/html; charset=UTF-8
<
<!DOCTYPE html>
<html>
<body>
...
...
...
I'm trying to port one of my Android applications to work natively on Mac OS X.
For the initialisation of the application, it needs to connect to a server and read only the headers of the server's response. The server (3rd party server) will respond with 82274 bytes of data, but the only useful data to me are the headers; specifically I only need to read the session cookie and retrieve its value. This means that all of the other data is redundant.
Through Googling, the only working looking response is as follows:
// Create the request.
NSMutableURLRequest *theRequest = [NSMutableURLRequest requestWithURL:[NSURL URLWithString:#"http://www.grooveshark.com/"]];
[theRequest setHTTPMethod:#"HEAD"];
[theRequest setValue:#"MySpecialUserAgent/1.0" forHTTPHeaderField:#"User-Agent"];
[theRequest setTimeoutInterval:15.0];
[theRequest setCachePolicy:NSURLRequestReloadIgnoringCacheData];
However this still downloads the entire page.
Can anyone point me in the right direction?
Let's see what happens if we hit that URL.
› curl -v -X HEAD http://www.grooveshark.com
* About to connect() to www.grooveshark.com port 80 (#0)
* Trying 8.20.213.76...
* connected
* Connected to www.grooveshark.com (8.20.213.76) port 80 (#0)
> HEAD / HTTP/1.1
> User-Agent: curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8r zlib/1.2.5
> Host: www.grooveshark.com
> Accept: */*
>
< HTTP/1.1 301 Moved Permanently
< Server: richhickey
< Date: Sat, 15 Dec 2012 20:27:38 GMT
< Content-Type: text/html; charset=UTF-8
< Connection: close
< Location: http://grooveshark.com
< Vary: Accept-Encoding
< X-Hostname: rhl081
< X-Hostname: rhl081
<
* Closing connection #0
So www.grooveshark.com redirects to grooveshark.com. Let's see if that page honors HEAD requests correctly.
› curl -v -X HEAD http://grooveshark.com
* About to connect() to grooveshark.com port 80 (#0)
* Trying 8.20.213.76...
* connected
* Connected to grooveshark.com (8.20.213.76) port 80 (#0)
> HEAD / HTTP/1.1
> User-Agent: curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8r zlib/1.2.5
> Host: grooveshark.com
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: richhickey
< Date: Sat, 15 Dec 2012 20:28:06 GMT
< Content-Type: text/html; charset=UTF-8
< Connection: close
< Vary: Accept-Encoding
< Set-Cookie: PHPSESSID=844a5e6bdd6d84a97afd8f42faf4eb95; expires=Sat, 22-Dec-2012 20:28:06 GMT; path=/; domain=.grooveshark.com
< Expires: Thu, 19 Nov 1981 08:52:00 GMT
< Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
< Pragma: no-cache
< Vary: Accept-Encoding
< X-Hostname: rhl061
< Set-Cookie: ismobile=no;domain=.grooveshark.com;path=/
< X-country: US
<
* Closing connection #0
That looks good. I suspect that your request is falling back to a GET when following that redirect. It looks like Chris Suter ran into the same thing and gave an example solution: http://sutes.co.uk/2009/12/nsurlconnection-using-head-met.html
In the future you might want to try running your requests through a local proxy so that you can see them in flight. That would probably reveal that you make a HEAD request to www.grooveshark.com followed by a GET to grooveshark.com.