Having trouble getting content-encoding to show up in response header for get request - karate

I was trying to write a karate test that validated that a particular response header contained the Content-Encoding header field with a value of gzip. I tried on my api where both times the content-encoding field was missing from karate's response. Both of these endpoints returned the content-encoding field on postman and curl commands.
I then tried to hit the postman-echo service to see if it was my api endpoint that karate was having issues with and it seems that it is not only my api. Could someone take a look at my code and see if I am doing something incorrectly to get the header field to show up in the response?
Feature: test getting Content-Encoding
Background:
* url 'https://postman-echo.com/gzip'
Scenario:
Given header Accept-Encoding = 'gzip'
When method get
Then status 200
And match responseHeaders contains {'Content-Encoding':'#present'}
This is what karate request looks like
1 > GET https://postman-echo.com/gzip
1 > Accept-Encoding: gzip
1 > Connection: Keep-Alive
1 > Host: postman-echo.com
1 > User-Agent: Apache-HttpClient/4.5.12 (Java/1.8.0_252)
and the response looks like
1 < 200
1 < Connection: keep-alive
1 < Content-Type: application/json; charset=utf-8
1 < Date: Fri, 08 May 2020 16:18:42 GMT
1 < ETag: W/"ef-7kclc8pzXTvQiPUaEOf6j95iFaE"
1 < Vary: Accept-Encoding
1 < set-cookie: sails.sid=s%3A6G_FShPRZH4V1G-tVDfUEEfMwQQmolo5.T2Cb37zqYA21FTyRyIGutVWQWo9ta4EWiod36%2FkM88I; Path=/; HttpOnly
{
"gzipped": true,
"headers": {
"x-forwarded-proto": "https",
"x-forwarded-port": "443",
"host": "postman-echo.com",
"x-amzn-trace-id": "Root=1-5eb58662-c4aaeec26efd116ac0544a18",
"accept-encoding": "gzip",
"user-agent": "Apache-HttpClient/4.5.12 (Java/1.8.0_252)"
},
"method": "GET"
}
The curl response header is
curl --location --request GET 'https://postman-echo.com/gzip' \
> --header 'Accept-Encoding: gzip' -I
HTTP/2 200
date: Fri, 08 May 2020 16:21:53 GMT
content-type: application/json; charset=utf-8
content-length: 220
content-encoding: gzip
etag: W/"dc-BuD8DN1qXT7trYUQtZOuSvbq1pM"
vary: Accept-Encoding
set-cookie: sails.sid=s%3Aj86lznX3nK20fnEN4B3nbHESrfWqVJ3M.236VrsmQp7V%2F7%2BrvG%2FEtlc9yUVLTtylh1yyIAdQJSiY; Path=/; HttpOnly

This seems to be the way the Apache HttpClient works or how it is configured for Karate. I just found you get the header back with karate-jersey:
1 > GET https://postman-echo.com/gzip
1 > Accept: */*
1 > Accept-Encoding: gzip
1 > User-Agent: Jersey/2.30 (HttpUrlConnection 1.8.0_231)
22:40:48.981 [ForkJoinPool-1-worker-1] DEBUG com.intuit.karate - response time in milliseconds: 1177.37
1 < 200
1 < Connection: keep-alive
1 < Content-Encoding: gzip
1 < Content-Length: 248
1 < Content-Type: application/json; charset=utf-8
1 < Date: Fri, 08 May 2020 17:10:48 GMT
1 < ETag: W/"f8-sigbV4PuNI2Fx08AqzMEqW1WIYY"
1 < Vary: Accept-Encoding
So if Jersey is ok for the rest of your tests, maybe that's all you need. Getting this to work for karate-apache is not a priority for me so if you or anyone else is willing to investigate or fix it, that would be great.

Related

How to send a multipart request with only one json field?

I'm trying to send a multipart request with one field but I cannot find it in the logs. My code is
Feature: test
Scenario: test send
Given url 'https://apitester.com/api/send'
And multipart field myJson = {test:'send'}
When method post
Then status 200
but in the logs I find only
1 > POST https://apitester.com/api/send
1 > Accept-Encoding: gzip,deflate
1 > Connection: Keep-Alive
1 > Content-Length: 226
1 > Content-Type: multipart/mixed; boundary=57Gy9v3RTbI82kmdNf-nsWld3wK1rz6W99F
1 > Host: apitester.com
1 > User-Agent: Apache-HttpClient/4.5.12 (Java/14.0.1)
In Postman I get something like this and it's working fine:
POST /api/send HTTP/1.1
Host: apitester.com
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW
----WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="myJson"
{"test":"send"}
----WebKitFormBoundary7MA4YWxkTrZu0gW
If I add And header Content-Type = 'application/json' it gets somehow similar but it's not working either:
1 > POST https://apitester.com/api/send
1 > Accept-Encoding: gzip,deflate
1 > Connection: Keep-Alive
1 > Content-Length: 218
1 > Content-Type: application/json; boundary=bzLjH96ptD4G7HSRWI65XMnIhllxtiO
1 > Host: apitester.com
1 > User-Agent: Apache-HttpClient/4.5.12 (Java/14.0.1)
--bzLjH96ptD4G7HSRWI65XMnIhllxtiO
Content-Disposition: form-data; name="myJson"
Content-Type: application/json; charset=UTF-8
Content-Transfer-Encoding: 8bit
{"test":"send"}
--bzLjH96ptD4G7HSRWI65XMnIhllxtiO--
How can I troubleshoot this?
Yes, if the Content-Type is not "printable" (e.g. binary) Karate doesn't log it.
On troubleshooting, this can be difficult, but if you work with your server-side folks you should be able to figure it out sooner. If you post a cURL command that works, the community here can help you.
We'll improve the logging in future versions.

Karate - Cookie rejected when it contains special characters

We are struggling to pass the response cookie to the subsequent request as we are getting Cookie Rejected error. We are also not able to print the cookie response:
Here are more details: giving req and response. See the Print at the very end prints nothing and at the start of response there are warning which we think is related to not printing response cookies.
Scenario:
* configure ssl = true
Given url 'https://dvabc-qa.kdc.abc.com'
Given path 'api/dp/v1/apps/user/login'
And request {username: "test#abc.com", password: "1234"}
When method post
Then status 200
* print responseCookies
Request
1 > POST https://dvabc-qa.kdc.abc.com/api/dp/v1/apps/user/login
1 > Accept-Encoding: gzip,deflate
1 > Connection: Keep-Alive
1 > Content-Length: 52
1 > Content-Type: application/json
1 > Host: dvabc-qa.abc.com
1 > User-Agent: Apache-HttpClient/4.5.3 (Java/1.8.0_121)
{"username":"test#capone.com","password":"test1234"} Response 14:48:30.700 [main] WARN o.a.h.c.p.ResponseProcessCookies - Cookie rejected [X-AUTH-TOKEN="eyJ1c2VySW5mbyI6eyJ1c2VyRmlyc3ROYW1lIjoiVGVzdCIsInVzZXJMYXN0TmFtZSI6Ilh0ZXN0IiwidXNlcklEIjoiMTMxMTMx...", version:1, domain:.abc.com, path:/, expiry:null] Domain attribute ".abc.com" violates RFC 2109: host minus domain may not contain any dots
14:48:30.704 [main] WARN o.a.h.c.p.ResponseProcessCookies - Cookie rejected [SSOID="test#abc.com", version:1, domain:.abc.com, path:/, expiry:null] Domain attribute ".abc.com" violates RFC 2109: host minus domain may not contain any dots
14:48:30.718 [main] DEBUG com.intuit.karate -
1 < 200
1 < CUSTOMER_APP_DATA_REF_KEY: $2a$10$qnUQ.paqXvMEJSmu6G/BhL7d08oOFnSAVmOBVQjec3Umlme
1 < Cache-Control: no-cache, no-store, must-revalidate
1 < Connection: keep-alive
1 < Content-Type: application/json;charset=UTF-8
1 < Date: Mon, 23 Oct 2017 18:48:32 GMT
1 < Expires: Thu, 01 Jan 1970 00:00:00 GMT
1 < Pragma: no-cache
1 < SSOID: test#abc.com
1 < Set-Cookie: [X-AUTH-TOKEN="eyJ1c2VySW5joiVGVzdCwTaG93IjpudWxsLCJoYXNEaWFtb25kRGVhbGVyc2hpcHMiOmZhbHNlLCJub25EaWFtb25kRGVhbGVyQWNjZXNzRW5hYmxlZCI6ZmFsc2UsIm1mYVRva2VuIjpmYWxzZSwicmVzcG9uZGVkVG9BbGxUYyInVzZXJuYW1lIjoidGVzdEBjYXBvbmUuY29tIn0=.HzxYTNQGXk8n0HePKBcuk1ND5h6P8z4xvfaoJ5qah94="; Version=1; Domain=.abc.com; Path=/; HttpOnly, CUSTOMER_APP_DATA_REF_KEY=$2a$10$qnUQ.paPnsI9gqXvMEJSmu6G/BhL7d08oOFnSAVmOBVQjec3Umlme; Domain=.abc.com; Path=/; HttpOnly, SSOID="test#abc.com"; Version=1; Domain=.abc.com; Path=/; HttpOnly]
1 < Vary: Accept-Encoding,User-Agent
1 < X-AUTH-TOKEN: eyJ1c2VySW5mbyI6eyJ1c2VyRmlyc3ROYW1lIjoiVGVzdCIsInVzZXJMYXN0TmFtZSI6Ilh0ZXN0IiwidOiJ0ZXN0QGNhcG9uZS5jb20iLCJyYW5kb21UZXh0IjpudWxsLCJpbnRlcm5hbFVzZXJFbWFpbCI6bnVsbCwidXNlckRCJqb2JUaXRsZSI6IlNhbGVzIE1hbmFnZXIifSwiYWN0aXZlRGlyZWN0b3J5QWNjb3VudCI6ZmFsc2UsInRjVG9TaG93IjpudWxsLCJoYXNEaWFtb25kRGVhbGVyc2hpcHMiOmZhbHNlLCJub25EaWFtb25kRGVhbGVyQWNjZXNzRW5hYmxlZCI6ZmFsc2UsIm1mYVRva2VuIjpmYWxzZSwicmVzcG9uZGVkVG9BbGxUYyI6ZmFsc2UsInVzZXJuYW1lIjoidGVzdEBjYXBvbmUuY29tIn0=.HzxYTNk1ND5h6P8z4xvfaoJ5qah94=
1 < X-Frame-Options: SAMEORIGIN
14:48:30.719 [main] DEBUG com.intuit.karate - response time in milliseconds: 583
14:48:30.726 [main] INFO com.intuit.karate - [print]
Okay, this may need a fix or enhancement to Karate. Can you kindly file an enhancement request.
Violates RFC 2109: host > minus domain may not contain any dots
Meanwhile, can you try switching from karate-apache to karate-jersey in your pom.xml and see if that makes a difference.

Microsoft Graph API - Resumable upload returns 404 when uploading chunk [duplicate]

This question already has answers here:
Resumable upload returns Unauthorized when uploading chunk
(2 answers)
Closed 5 years ago.
Following the answer to this question I removed my Authorization header when uploading the first chunk of my resumable upload.
So creating the session:
POST https://graph.microsoft.com/V1.0/groups/a0aab702-8e58-4f25-b2dd-066e5d110f0c/drive/root:/10cb244a-d181-44df-aabc-51c30d4cf0f5.txt:/createUploadSession HTTP/1.1
Authorization: Bearer XXX
Content-Type: application/json; charset=utf-8
Host: graph.microsoft.com
Content-Length: 75
Expect: 100-continue
{
"item": {
"#microsoft.graph.conflictBehaviour": "replace"
}
}
Response:
HTTP/1.1 200 OK
Cache-Control: private
Transfer-Encoding: chunked
Content-Type: application/json;odata.metadata=minimal;odata.streaming=true;IEEE754Compatible=false;charset=utf-8
Location: https://graph.microsoft.com
request-id: 9d93ade6-8072-438b-a9fb-c648376b77ba
client-request-id: 9d93ade6-8072-438b-a9fb-c648376b77ba
x-ms-ags-diagnostic: {"ServerInfo":{"DataCenter":"North Europe","Slice":"SliceB","ScaleUnit":"002","Host":"AGSFE_IN_5","ADSiteName":"DUB"}}
OData-Version: 4.0
Duration: 1750.2813
Date: Thu, 27 Apr 2017 16:06:35 GMT
93d
{"#odata.context":"https://graph.microsoft.com/V1.0/$metadata#microsoft.graph.uploadSession","expirationDateTime":"2017-04-27T16:21:35.2923217Z","nextExpectedRanges":["0-"],"uploadUrl":"https://peterreay2.sharepoint.com/sites/SharePointTests/_api/v2.0/drive/items/01RZBHEZF6Y2GOVW7725BZO354PWSELRRZ/uploadSession?guid='e13f861f-7aa0-4f9a-b05c-26587ecfab92'&path='~tmp0E_10cb244a-d181-44df-aabc-51c30d4cf0f5.txt'&overwrite=True&rename=False&access_token=XXX&prooftoken=XXX"}
0
So I then try and upload my first chunk, however I'm getting a 404 not found response:
PUT https://peterreay2.sharepoint.com/sites/SharePointTests/_api/v2.0/drive/items/01RZBHEZF6Y2GOVW7725BZO354PWSELRRZ/uploadSession?guid='e13f861f-7aa0-4f9a-b05c-26587ecfab92'&path='~tmp0E_10cb244a-d181-44df-aabc-51c30d4cf0f5.txt'&overwrite=True&rename=False&access_token=XXX&prooftoken=XXX HTTP/1.1
Content-Range: bytes 0-10485759/11534336
Host: peterreay2.sharepoint.com
Content-Length: 10485760
Expect: 100-continue
Connection: Keep-Alive
0123456 etc
Response:
HTTP/1.1 404 Not Found
Server: Microsoft-IIS/8.5
SPRequestGuid: 8825ec9d-b0ed-3000-1791-4c35404935e9
request-id: 8825ec9d-b0ed-3000-1791-4c35404935e9
Strict-Transport-Security: max-age=31536000
X-FRAME-OPTIONS: SAMEORIGIN
X-Powered-By: ASP.NET
MicrosoftSharePointTeamServices: 16.0.0.6420
X-Content-Type-Options: nosniff
X-MS-InvokeApp: 1; RequireReadOnly
P3P: CP="ALL IND DSP COR ADM CONo CUR CUSo IVAo IVDo PSA PSD TAI TELo OUR SAMo CNT COM INT NAV ONL PHY PRE PUR UNI"
Date: Thu, 27 Apr 2017 16:08:25 GMT
Connection: close
Content-Length: 0
As in my other question, the code doing this used to work (even with a Bearer token). So I think the chunk sizes, byte ranges etc are correct.
Any thoughts on what is going wrong here?
See the answer to the linked question here. There was a bug confirmed and fixed by Microsoft. This fixed our issue with the request including a bearer token. I have not tested this scenario here (without a bearer token).

will Accept Http Headers digest any other formats other than the one specified?

Per my understanding:
the Accept header is used by HTTP clients to tell the server what content types they'll accept. The server will then send back a response, which will include a Content-Type header telling the client what the content type of the returned content actually is.
With this understanding, I tried the following:
curl -X GET -H "Accept: application/xml" http://www.google.com -v
* About to connect() to www.google.com port 80 (#0)
* Trying 173.194.33.81...
* connected
* Connected to www.google.com (173.194.33.81) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8y zlib/1.2.5
> Host: www.google.com
> Accept: application/xml
>
< HTTP/1.1 200 OK
< Date: Tue, 02 Sep 2014 17:58:05 GMT
< Expires: -1
< Cache-Control: private, max-age=0
< Content-Type: text/html; charset=ISO-8859-1
< Set-Cookie: PREF=ID=5c30672b67a74789:FF=0:TM=1409680685:LM=1409680685:S=PsGclk3vR4HWjann; expires=Thu, 01-Sep-2016 17:58:05 GMT; path=/; domain=.google.com
< Set-Cookie: NID=67=rPuxpwUu5UNuapzCdbD5iwVyjjC9TzP_Ado29h3ucjEq4A_2qkSM4nQM3RO02rfyuHmrh-hvmwmgFCmOvISttFfHv06f8ay4_6Gl4pXRjqxihNhJSGbvujjDRzaSibfy; expires=Wed, 04-Mar-2015 17:58:05 GMT; path=/; domain=.google.com; HttpOnly
< P3P: CP="This is not a P3P policy! See http://www.google.com/support/accounts/bin/answer.py?hl=en&answer=151657 for more info."
< Server: gws
< X-XSS-Protection: 1; mode=block
< X-Frame-Options: SAMEORIGIN
< Alternate-Protocol: 80:quic
< Transfer-Encoding: chunked
<
<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang="en"><
As you can notice in the response, I am sent Content-Type: text/html; charset=ISO-8859-1 which is not what I asked for?
why does a different representation (HTML in this case) is sent, although I asked for xml?
Thanks
From RFC 2616:
If an Accept header field is present,
and if the server cannot send a response which is acceptable
according to the combined Accept field value, then the server SHOULD
send a 406 (not acceptable) response.
Here, "should" means that Google aren't actually obliged to throw a 406 error. But since you're receiving an HTML response, it has effectively the same meaning.

google oauth1 to oauth2 migration invalid_token error

I have been trying to obtain new oauth2 refresh tokens using oauth1 access token but it constantly returns an "invalid_token" error. I have checked and the access token is working correctly. I have also tested the same creds/params in oauth2 playground and result is the same. Any help is appreciated...
Here is the curl verbose output:
> POST /o/oauth2/token HTTP/1.1
Host: accounts.google.com
Content-Type: application/x-www-form-urlencoded
Authorization: OAuth oauth_nonce="cb7407355fe20f509cb6bf901eae2d24", oauth_timestamp="1389169471", oauth_consumer_key="***", oauth_token="1%2FFVy....", oauth_signature_method="HMAC-SHA1", oauth_signature="0YL1hH5R571nOH1byeHxQlg%2Fa6g%3D"
Content-Length: 444
* upload completely sent off: 444 out of 444 bytes
< HTTP/1.1 400 Bad Request
< Cache-Control: no-cache, no-store, max-age=0, must-revalidate
< Pragma: no-cache
< Expires: Fri, 01 Jan 1990 00:00:00 GMT
< Date: Wed, 08 Jan 2014 08:24:31 GMT
< Content-Type: application/json
< X-Content-Type-Options: nosniff
< X-Frame-Options: SAMEORIGIN
< X-XSS-Protection: 1; mode=block
* Server GSE is not blacklisted
< Server: GSE
< Alternate-Protocol: 443:quic
< Transfer-Encoding: chunked
<
* Connection #0 to host accounts.google.com left intact
string(415) "HTTP/1.1 400 Bad Request
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: Fri, 01 Jan 1990 00:00:00 GMT
Date: Wed, 08 Jan 2014 08:24:31 GMT
Content-Type: application/json
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Server: GSE
Alternate-Protocol: 443:quic
Transfer-Encoding: chunked
{
"error" : "invalid_token"
}"
Can you check if you are putting the client_secret in {} in the POST Body?
grant_type=urn:ietf:params:oauth:grant-type:migration:oauth1&client_id=xxxxxxx.apps.googleusercontent.com&client_secret={xxxxxxx}
You will also need to put {} around the client_secret value when you are generating the oauth_signature
We have made a few changes to the validation pieces of the OAuth1->OAuth2 token migration. Would you mind checking your migration flows again and updating this thread with the results?