Karate test follows 303 dummy URL when `Expires: 0` header is missing - testing

I have a Karate test that sends a POST that then will send back a redirect to a dummy address. With a recent Spring upgrade, I am now seeing the redirect coming back as a 303 rather than a 302.
I do not want this 302 URL to be followed with a request. The test just verified that we got a 302 back, and continued with more validation. This is what I have been seeing up until the Spring upgrade. However, with the 303 returned, it is then followed by a GET request to the dummy URL, which then fails with an UnknownHostException. I think perhaps due to the presence of Expires: 0 in the headers of the 302. The 303 does not contain this header (or Pragma: no-cache but I can't see how that would affect it). Below are the reponse headers for each, with differences highlighted with *
302
5 < 302
* 5 < Cache-Control: no-cache, no-store, max-age=0, must-revalidate
5 < Connection: keep-alive
5 < Content-Language: en-US
5 < Content-Length: 0
5 < Date: Mon, 16 Mar 2020 12:04:55 GMT
* 5 < Expires: 0
5 < Location: http://anywhere?key=value&code=iXAnzC
* 5 < Pragma: no-cache
5 < Server: nginx/1.17.9
5 < X-Content-Type-Options: nosniff
5 < X-Frame-Options: DENY
5 < X-XSS-Protection: 1; mode=block
303
5 < 303
5 < Cache-Control: no-store
5 < Connection: keep-alive
5 < Content-Language: en-US
5 < Content-Length: 0
5 < Date: Fri, 13 Mar 2020 11:06:31 GMT
5 < Location: http://anywhere?key=value&code=05O7lL
5 < Server: nginx/1.17.9
5 < X-Content-Type-Options: nosniff
5 < X-Frame-Options: DENY
5 < X-XSS-Protection: 1; mode=block
We do not explicitly set these headers so it will be Tomcat/NGinx behaviour, which means it's going to be standard on many servers. Is this generally expected behaviour with 303's or is this related to the differences in the headers?

Can you look at the configure followRedirects option: https://github.com/intuit/karate#configure - but I guess you know this already and maybe that works only for 302-s.
If you have already set that to false and still see Karate make the second call automatically, it may need a fix. Any pointers to what changes we need to make in the karate-apache code will help, here is the implementation for followRedirects.
You can also try switch to karate-jersey and see if that client works the way you expect.

Related

Having trouble getting content-encoding to show up in response header for get request

I was trying to write a karate test that validated that a particular response header contained the Content-Encoding header field with a value of gzip. I tried on my api where both times the content-encoding field was missing from karate's response. Both of these endpoints returned the content-encoding field on postman and curl commands.
I then tried to hit the postman-echo service to see if it was my api endpoint that karate was having issues with and it seems that it is not only my api. Could someone take a look at my code and see if I am doing something incorrectly to get the header field to show up in the response?
Feature: test getting Content-Encoding
Background:
* url 'https://postman-echo.com/gzip'
Scenario:
Given header Accept-Encoding = 'gzip'
When method get
Then status 200
And match responseHeaders contains {'Content-Encoding':'#present'}
This is what karate request looks like
1 > GET https://postman-echo.com/gzip
1 > Accept-Encoding: gzip
1 > Connection: Keep-Alive
1 > Host: postman-echo.com
1 > User-Agent: Apache-HttpClient/4.5.12 (Java/1.8.0_252)
and the response looks like
1 < 200
1 < Connection: keep-alive
1 < Content-Type: application/json; charset=utf-8
1 < Date: Fri, 08 May 2020 16:18:42 GMT
1 < ETag: W/"ef-7kclc8pzXTvQiPUaEOf6j95iFaE"
1 < Vary: Accept-Encoding
1 < set-cookie: sails.sid=s%3A6G_FShPRZH4V1G-tVDfUEEfMwQQmolo5.T2Cb37zqYA21FTyRyIGutVWQWo9ta4EWiod36%2FkM88I; Path=/; HttpOnly
{
"gzipped": true,
"headers": {
"x-forwarded-proto": "https",
"x-forwarded-port": "443",
"host": "postman-echo.com",
"x-amzn-trace-id": "Root=1-5eb58662-c4aaeec26efd116ac0544a18",
"accept-encoding": "gzip",
"user-agent": "Apache-HttpClient/4.5.12 (Java/1.8.0_252)"
},
"method": "GET"
}
The curl response header is
curl --location --request GET 'https://postman-echo.com/gzip' \
> --header 'Accept-Encoding: gzip' -I
HTTP/2 200
date: Fri, 08 May 2020 16:21:53 GMT
content-type: application/json; charset=utf-8
content-length: 220
content-encoding: gzip
etag: W/"dc-BuD8DN1qXT7trYUQtZOuSvbq1pM"
vary: Accept-Encoding
set-cookie: sails.sid=s%3Aj86lznX3nK20fnEN4B3nbHESrfWqVJ3M.236VrsmQp7V%2F7%2BrvG%2FEtlc9yUVLTtylh1yyIAdQJSiY; Path=/; HttpOnly
This seems to be the way the Apache HttpClient works or how it is configured for Karate. I just found you get the header back with karate-jersey:
1 > GET https://postman-echo.com/gzip
1 > Accept: */*
1 > Accept-Encoding: gzip
1 > User-Agent: Jersey/2.30 (HttpUrlConnection 1.8.0_231)
22:40:48.981 [ForkJoinPool-1-worker-1] DEBUG com.intuit.karate - response time in milliseconds: 1177.37
1 < 200
1 < Connection: keep-alive
1 < Content-Encoding: gzip
1 < Content-Length: 248
1 < Content-Type: application/json; charset=utf-8
1 < Date: Fri, 08 May 2020 17:10:48 GMT
1 < ETag: W/"f8-sigbV4PuNI2Fx08AqzMEqW1WIYY"
1 < Vary: Accept-Encoding
So if Jersey is ok for the rest of your tests, maybe that's all you need. Getting this to work for karate-apache is not a priority for me so if you or anyone else is willing to investigate or fix it, that would be great.

Karate - Cookie rejected when it contains special characters

We are struggling to pass the response cookie to the subsequent request as we are getting Cookie Rejected error. We are also not able to print the cookie response:
Here are more details: giving req and response. See the Print at the very end prints nothing and at the start of response there are warning which we think is related to not printing response cookies.
Scenario:
* configure ssl = true
Given url 'https://dvabc-qa.kdc.abc.com'
Given path 'api/dp/v1/apps/user/login'
And request {username: "test#abc.com", password: "1234"}
When method post
Then status 200
* print responseCookies
Request
1 > POST https://dvabc-qa.kdc.abc.com/api/dp/v1/apps/user/login
1 > Accept-Encoding: gzip,deflate
1 > Connection: Keep-Alive
1 > Content-Length: 52
1 > Content-Type: application/json
1 > Host: dvabc-qa.abc.com
1 > User-Agent: Apache-HttpClient/4.5.3 (Java/1.8.0_121)
{"username":"test#capone.com","password":"test1234"} Response 14:48:30.700 [main] WARN o.a.h.c.p.ResponseProcessCookies - Cookie rejected [X-AUTH-TOKEN="eyJ1c2VySW5mbyI6eyJ1c2VyRmlyc3ROYW1lIjoiVGVzdCIsInVzZXJMYXN0TmFtZSI6Ilh0ZXN0IiwidXNlcklEIjoiMTMxMTMx...", version:1, domain:.abc.com, path:/, expiry:null] Domain attribute ".abc.com" violates RFC 2109: host minus domain may not contain any dots
14:48:30.704 [main] WARN o.a.h.c.p.ResponseProcessCookies - Cookie rejected [SSOID="test#abc.com", version:1, domain:.abc.com, path:/, expiry:null] Domain attribute ".abc.com" violates RFC 2109: host minus domain may not contain any dots
14:48:30.718 [main] DEBUG com.intuit.karate -
1 < 200
1 < CUSTOMER_APP_DATA_REF_KEY: $2a$10$qnUQ.paqXvMEJSmu6G/BhL7d08oOFnSAVmOBVQjec3Umlme
1 < Cache-Control: no-cache, no-store, must-revalidate
1 < Connection: keep-alive
1 < Content-Type: application/json;charset=UTF-8
1 < Date: Mon, 23 Oct 2017 18:48:32 GMT
1 < Expires: Thu, 01 Jan 1970 00:00:00 GMT
1 < Pragma: no-cache
1 < SSOID: test#abc.com
1 < Set-Cookie: [X-AUTH-TOKEN="eyJ1c2VySW5joiVGVzdCwTaG93IjpudWxsLCJoYXNEaWFtb25kRGVhbGVyc2hpcHMiOmZhbHNlLCJub25EaWFtb25kRGVhbGVyQWNjZXNzRW5hYmxlZCI6ZmFsc2UsIm1mYVRva2VuIjpmYWxzZSwicmVzcG9uZGVkVG9BbGxUYyInVzZXJuYW1lIjoidGVzdEBjYXBvbmUuY29tIn0=.HzxYTNQGXk8n0HePKBcuk1ND5h6P8z4xvfaoJ5qah94="; Version=1; Domain=.abc.com; Path=/; HttpOnly, CUSTOMER_APP_DATA_REF_KEY=$2a$10$qnUQ.paPnsI9gqXvMEJSmu6G/BhL7d08oOFnSAVmOBVQjec3Umlme; Domain=.abc.com; Path=/; HttpOnly, SSOID="test#abc.com"; Version=1; Domain=.abc.com; Path=/; HttpOnly]
1 < Vary: Accept-Encoding,User-Agent
1 < X-AUTH-TOKEN: eyJ1c2VySW5mbyI6eyJ1c2VyRmlyc3ROYW1lIjoiVGVzdCIsInVzZXJMYXN0TmFtZSI6Ilh0ZXN0IiwidOiJ0ZXN0QGNhcG9uZS5jb20iLCJyYW5kb21UZXh0IjpudWxsLCJpbnRlcm5hbFVzZXJFbWFpbCI6bnVsbCwidXNlckRCJqb2JUaXRsZSI6IlNhbGVzIE1hbmFnZXIifSwiYWN0aXZlRGlyZWN0b3J5QWNjb3VudCI6ZmFsc2UsInRjVG9TaG93IjpudWxsLCJoYXNEaWFtb25kRGVhbGVyc2hpcHMiOmZhbHNlLCJub25EaWFtb25kRGVhbGVyQWNjZXNzRW5hYmxlZCI6ZmFsc2UsIm1mYVRva2VuIjpmYWxzZSwicmVzcG9uZGVkVG9BbGxUYyI6ZmFsc2UsInVzZXJuYW1lIjoidGVzdEBjYXBvbmUuY29tIn0=.HzxYTNk1ND5h6P8z4xvfaoJ5qah94=
1 < X-Frame-Options: SAMEORIGIN
14:48:30.719 [main] DEBUG com.intuit.karate - response time in milliseconds: 583
14:48:30.726 [main] INFO com.intuit.karate - [print]
Okay, this may need a fix or enhancement to Karate. Can you kindly file an enhancement request.
Violates RFC 2109: host > minus domain may not contain any dots
Meanwhile, can you try switching from karate-apache to karate-jersey in your pom.xml and see if that makes a difference.

Disable follow redirect in Karate BDD framework?

I have a part of my application flow that reuses an old login service. This old login service uses a 302 redirect with a Location header that includes a query param I have to capture and use as an auth code.
Right now, the Karate framework is helpfully forwarding me to the location specified in the Location header. How can I disable this so that instead I can capture the values myself?
09:39:26.096 [main] DEBUG com.intuit.karate -
2 < 302
2 < Connection: keep-alive
2 < Content-Length: 0
2 < Content-Type: application/json;charset=ISO-8859-1
2 < Date: Fri, 08 Sep 2017 15:39:25 GMT
2 < Server: nginx/1.4.6 (Ubuntu)
2 < Set-Cookie: [JSESSIONID=0000UDF76JJ7ZGsFlTs9FPYcVoN:1bj4v930o; Path=/; HttpOnly, PD-S-SESSION-ID=1_2_1_upREX6Zqm0NTxJ5dNQeSyVeaiUFYnO1OdgUVYXYnekLFY-A7; Path=/; Secure; HttpOnly, PD-ID=ZugwEuE0PZmTNhZkIlog5FqfvlDKD2nfvKAZ+ebfTvVx2dzw0JAgyOMeviZx3xsxYndwQxGmURx6n32fscUXMw3X8P9FpNDssgIz355kfTeda/25qxPf+uaEvs8JpE2FxsMWkzQ25qM2KmrhAK9QTRu31oflpD6eL+klFfDahSSTQlFMXZmB4Yjsc8DOnoGAhbv19Yh9A8OuzDFXimwML2a+Yn9UYN/ZjRXF45veNQGsA9/dtP/nfZl6eAPyy7mdSKunhHuWggBtYFP9kAgmB1LJ87MFWsOhZEwcnmLsJM078tRStGLeOFo7KX+1FtMhhcV76UkTxDFj1VR68fSPdzF+84oeyxf0WhQ0UZB+oc76kQtl94B7uW4X/lp9jADlGUwrE8hPQ5asPcPblt6E0yPyi+TvouK/0XHpy7QfAeXZqyifs+yeUOKHBdHUXHI7QhnadEX0GViHPsqZ5amztnYkF5mMQLkdXYIO22oSHCtci7M08gr1usTLrHc/hIBm; Path=/; Secure; HttpOnly, BigIPWebSeal=rd3o00000000000000000000ffff0a1bf852o443; path=/]
2 < X-Application-Context: sv:dev:8080
2 < X-B3-Sampled: 1
2 < X-B3-SpanId: 3229b43a709ac873
2 < X-B3-TraceId: 3229b43a709ac873
2 < X-Backside-Transport: OK OK
2 < X-Client-IP: x.x.x.x
2 < X-Global-Transaction-ID: 1890531591
2 < cache-control: no-cache="set-cookie, set-cookie2"
2 < content-language: en-US
2 < expires: Thu, 01 Dec 1994 16:00:00 GMT
2 < iis_server_name: APPA014D
2 < location: https://a-url.com/path?code=a51769a482e5
2 < p3p: CP="NON CUR OTPi OUR NOR UNI"
2 < strict-transport-security: max-age=31536000; includeSubDomains
2 < x-powered-by: Servlet/3.0
2 < x-xss-protection: 1;mode=block
Wow, to be honest no-one has asked for this. Until now. Looks like we'll have to add this support: http://www.baeldung.com/httpclient-stop-follow-redirect
This can be implemented rather quickly, can you raise a feature request ? If this is a blocker, you could temporarily call into Java and use an instance of the Apache Http Client or something. I was wondering if you had the option to access a lastRedirectUrl instead would that have helped.
edit: this has been implemented in dev and will be available in version 0.6.0

google oauth1 to oauth2 migration invalid_token error

I have been trying to obtain new oauth2 refresh tokens using oauth1 access token but it constantly returns an "invalid_token" error. I have checked and the access token is working correctly. I have also tested the same creds/params in oauth2 playground and result is the same. Any help is appreciated...
Here is the curl verbose output:
> POST /o/oauth2/token HTTP/1.1
Host: accounts.google.com
Content-Type: application/x-www-form-urlencoded
Authorization: OAuth oauth_nonce="cb7407355fe20f509cb6bf901eae2d24", oauth_timestamp="1389169471", oauth_consumer_key="***", oauth_token="1%2FFVy....", oauth_signature_method="HMAC-SHA1", oauth_signature="0YL1hH5R571nOH1byeHxQlg%2Fa6g%3D"
Content-Length: 444
* upload completely sent off: 444 out of 444 bytes
< HTTP/1.1 400 Bad Request
< Cache-Control: no-cache, no-store, max-age=0, must-revalidate
< Pragma: no-cache
< Expires: Fri, 01 Jan 1990 00:00:00 GMT
< Date: Wed, 08 Jan 2014 08:24:31 GMT
< Content-Type: application/json
< X-Content-Type-Options: nosniff
< X-Frame-Options: SAMEORIGIN
< X-XSS-Protection: 1; mode=block
* Server GSE is not blacklisted
< Server: GSE
< Alternate-Protocol: 443:quic
< Transfer-Encoding: chunked
<
* Connection #0 to host accounts.google.com left intact
string(415) "HTTP/1.1 400 Bad Request
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: Fri, 01 Jan 1990 00:00:00 GMT
Date: Wed, 08 Jan 2014 08:24:31 GMT
Content-Type: application/json
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Server: GSE
Alternate-Protocol: 443:quic
Transfer-Encoding: chunked
{
"error" : "invalid_token"
}"
Can you check if you are putting the client_secret in {} in the POST Body?
grant_type=urn:ietf:params:oauth:grant-type:migration:oauth1&client_id=xxxxxxx.apps.googleusercontent.com&client_secret={xxxxxxx}
You will also need to put {} around the client_secret value when you are generating the oauth_signature
We have made a few changes to the validation pieces of the OAuth1->OAuth2 token migration. Would you mind checking your migration flows again and updating this thread with the results?

Apache (2.0) mod_headers not outputting environment variables?

My .htaccess file:
Header add X-Hello "time %D"
Now I check to see what it looks like, check out the X-Hello header:
$curl -v http://foo.com/bar.php
...
< HTTP/1.1 200 OK
< Date: Wed, 06 Oct 2010 20:43:39 GMT
< Server: Apache
< Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
< Expires: Thu, 19 Nov 1981 08:52:00 GMT
< Pragma: no-cache
< Set-Cookie: PHPSESSID=7d32d8fch8qrnnuvoqqopr74c0; path=/
< X-Hello: time %D
< Transfer-Encoding: chunked
< Content-Type: text/html
Which is clearly contrary to the Apache documentation below. Is there some other directive that mod_headers needs in order to interpolate environment variables?
http://httpd.apache.org/docs/2.0/mod/mod_headers.html#examples
Without note otherwise, hosting provider admitted to still using Apache 1.3 which does not interpolate values in mod_header.