When making API calls to our auth server (Keycloak, served on Wildfly) from our Angular application there is a point where requests begin to fail, apparently due to too much data being present in the bearer token. I have isolated this to the point where adding a single letter to the user's name will cause the request to fail.
Requests to our other servers with the same bearer token work as expected.
When the requests fail, the browser DevTools show them as failed with no additional information - simply a 'Failed to load response data' message and the application does not receive any data. This issue appears to be browser agnostic (have tried Chrome, Edge, and Firefox).
When I copy one of these failed requests into postman, it succeeds and gets the expected JSON data.
I have thus far experimented pretty extensively with allowed header size on all of the servers and seen no change and am a bit at a loss for where else it would make sense to continue investigating. Does anyone have recommendations?
An example of one of the requests in question, copied as cURL (bash):
curl 'OMITTED/auth/realms/OMITTED/account' \
-H 'authority: OMITTED' \
-H 'pragma: no-cache' \
-H 'cache-control: no-cache' \
-H 'sec-ch-ua: "Chromium";v="92", " Not A;Brand";v="99", "Google Chrome";v="92"' \
-H 'accept: application/json' \
-H 'authorization: bearer OMITTED' \
-H 'sec-ch-ua-mobile: ?0' \
-H 'user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/OMITTED (KHTML, like Gecko) Chrome/OMITTED Safari/OMITTED' \
-H 'origin: http://localhost:4200' \
-H 'sec-fetch-site: cross-site' \
-H 'sec-fetch-mode: cors' \
-H 'sec-fetch-dest: empty' \
-H 'referer: http://localhost:4200/' \
-H 'accept-language: en-US,en;q=0.9' \
--compressed
Related
I am trying to implement OAuth via a series of CURL commands but I am having trouble with the Authorise step. I have captured the Authorize process in Fiddler so I know what to replicate and I have defined the process to get the access code as follows:
POST login credentials to the login page in order to get the
'.ASPXAUTH' cookie in the response.
GET the authorisation page by sending the '.ASPAUTH' cookie and in the response get the '__RequestVerificationToken' from the
response cookies and also the form token (also named
__RequestVerificationToken) from the body of the webpage.
POST to the authorisation page by sending both the '.ASPAUTH' and
'__RequestVerificationToken' cookies as well as the form token
in the body.
Here are the requests I am using for each step with the username and passwords replaced:
curl -x 127.0.0.1:8866 -k -c cookie.txt 'https://oauth.sandbox.trainingpeaks.com/Account/LogOn?ReturnUrl=%2fOAuth%2fAuthorize%3fresponse_type%3dcode%26client_id%3dclientId%26scope%3dcoach%253Aathletes%2520workouts%253Aread%26redirect_uri%3dhttps%253A%252F%252Ftest_url.com%252Fcallback&response_type=code&client_id=clientId&scope=workouts%3Aread%20athlete%3Aprofile&redirect_uri=https%3A%2F%2Ftest_url.com%2Fcallback' -H 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:86.0) Gecko/20100101 Firefox/86.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Content-Type: application/x-www-form-urlencoded' -H 'Origin: https://oauth.sandbox.trainingpeaks.com' -H 'DNT: 1' -H 'Connection: keep-alive' -H 'Referer: https://oauth.sandbox.trainingpeaks.com/Account/LogOn?ReturnUrl=%2fOAuth%2fAuthorize%3fresponse_type%3dcode%26client_id%3dclientId%26scope%3dcoach%253Aathletes%2520workouts%253Aread%26redirect_uri%3dhttps%253A%252F%252Ftest_url.com%252Fcallback&response_type=code&client_id=clientId&scope=workouts%3Aread%20athlete%3Aprofile&redirect_uri=https%3A%2F%2Ftest_url.com%2Fcallback' -H 'Upgrade-Insecure-Requests: 1' --data-raw 'UserName=MY_USER_NAME&Password=MY_PASSWORD'
This returns the '.ASPXAUTH' cookie used in the next request.
curl -x 127.0.0.1:8866 -k -c cookie.txt 'https://oauth.sandbox.trainingpeaks.com/OAuth/Authorize?response_type=code&client_id=clientId&scope=coach%3Aathletes%20workouts%3Aread&redirect_uri=https%3A%2F%2Ftest_url.com%2Fcallback' -H 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:86.0) Gecko/20100101 Firefox/86.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Referer: https://oauth.sandbox.trainingpeaks.com/Account/LogOn?ReturnUrl=%2fOAuth%2fAuthorize%3fresponse_type%3dcode%26client_id%3dclientId%26scope%3dcoach%253Aathletes%2520workouts%253Aread%26redirect_uri%3dhttps%253A%252F%252Ftest_url.com%252Fcallback&response_type=code&client_id=clientId&scope=workouts%3Aread%20athlete%3Aprofile&redirect_uri=https%3A%2F%2Ftest_url.com%2Fcallback' -H 'DNT: 1' -H 'Connection: keep-alive' -H 'Cookie: .ASPXAUTH=37D88F4FF97A59398A7F3A01AEDC4ABB32FF99FCE85B346271D9F62D9CEA65B9BF0027A0304DC1E87CDE46948A9F72CC57B1479A37CB1B54F33B74E03C4D20AC44D333FE6FFAD3A4CB69336A14DCA2C46CCBD822C569C1F231383541C99D9F6715D813D1' -H 'Upgrade-Insecure-Requests: 1'
This returns the '__RequestVerificationToken' cookie and I get the form token (__RequestVerificationToken that is sent as part of the body) from the body of the response.
curl -x 127.0.0.1:8866 -k 'https://oauth.sandbox.trainingpeaks.com/OAuth/AuthorizeResponse' -H 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:86.0) Gecko/20100101 Firefox/86.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Content-Type: application/x-www-form-urlencoded' -H 'Origin: https://oauth.sandbox.trainingpeaks.com' -H 'DNT: 1' -H 'Connection: keep-alive' -H 'Referer: https://oauth.sandbox.trainingpeaks.com/OAuth/Authorize?response_type=code&client_id=clientId&scope=coach%3Aathletes%20workouts%3Aread&redirect_uri=https%3A%2F%2Ftest_url.com%2Fcallback' -H 'Cookie: .ASPXAUTH=37D88F4FF97A59398A7F3A01AEDC4ABB32FF99FCE85B346271D9F62D9CEA65B9BF0027A0304DC1E87CDE46948A9F72CC57B1479A37CB1B54F33B74E03C4D20AC44D333FE6FFAD3A4CB69336A14DCA2C46CCBD822C569C1F231383541C99D9F6715D813D1; __RequestVerificationToken=-_UjTRMCw6tv0jVe9bcA_JV7onmEAZZOPyx89_tZavaQY4U2Q4aAwgEx9Ghhp3i8Uh31FL-zHVFrSTAgqUgDPxpMDQg1' -H 'Upgrade-Insecure-Requests: 1' --data-raw '__RequestVerificationToken=O1mxbFlJ6T3AxL1ua4Vtemuhj5lzaCIxKOm49v_NdlCkitIpfUG4DSBwPwI7jb9o4BnLDA0s3R4w3i2Ftcqph0EAifWSreOQmXYcQ1YM1JlOgFzW0&IsApproved=true&client_id=clientId&redirect_uri=https%3A%2F%2Ftest_url.com%2Fcallback&state=&scope=coach%3Aathletes+workouts%3Aread&response_type=code'
This should redirect to the test URL with the access code as it does when these steps are done in the browser but instead I receive:
<html><head><title>Object moved</title></head><body>
<h2>Object moved to here.</h2>
</body></html>
This seems to indicate that the session has become invalidated and requires starting the process again.
I have compared the requests generated by CURL and those from the browser in Fiddler and I cannot determine a difference.
Here are the API docs, I am trying to implement the 'Authorise' process which results in an access code. Any help on this would be appreciated.
It is tricky to automate logins via CURL, since OAuth redirects tend to involve these actions:
Following HTTP redirects automatically
Auto posting forms with authentication results
Some complex messages, some of which may vary depending on the authentication method
For an easier way of testing - have a look at OAuth Tools, which is kind of like 'Postman for OAuth', and works with any standards based provider - here is an introductory video.
I have two curl commands below. You'll notice I'm passing in a Cookie of "usprivacy=1---". This is a cookie that seems to be added by some web browsers automatically. The issue I'm seeing -- if this cookie is present, some of my assets return HTTP error codes. As soon as the cookie is removed, it returns HTTP CODE 200. It doesn't happen with any other cookies. I suspect there's some Apache configuration variable, but I'm baffled.
This one results in the Apache webserver returning 404 not found:
curl 'https://salvagedinspirations.com/wp-content/plugins/woocommerce/packages/woocommerce-blocks/build/style.css?ver=2.4.5' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:71.0) Gecko/20100101 Firefox/71.0' -H 'Accept: text/css,*/*;q=0.1' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Connection: keep-alive' -H 'Referer: https://salvagedinspirations.com/' -H 'Cookie: usprivacy=1---;' -H 'Cache-Control: max-age=0' -D -
This one results in a "403 Forbidden" header, although HTML is still returned:
curl 'https://salvagedinspirations.com/' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:71.0) Gecko/20100101 Firefox/71.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Connection: keep-alive' -H 'Cookie: usprivacy=1---;' -H 'Upgrade-Insecure-Requests: 1' -H 'Cache-Control: max-age=0' -D -
Thank you
The trailing ; is technically invalid. Does it make a difference? You could be triggering some bad parsing or malicious request detection in a framework, mod_security, etc.
The issue was with an apache module that was loaded in. I forgot which one. Sorry
I uses the arangodb(0.9.3) to start cluster(current version 3.2.1).
there are some problemsin authentication.
query1 post to coordinator(port=8529) with basic auth, response 200
curl -X GET \
http://xx.xx.xx.xx:8529/_db/KnowledgeGraph/_api/collection/CarInfo \
-H 'authorization: Basic cm9vdDptaHh6a2hs' \
-H 'cache-control: no-cache' \
-H 'postman-token: 897691e4-c4fc-5588-b217-0da53bc368ba'
query2 post to agency(port=8531) with basic auth, response 401
curl -X GET \
http://xx.xx.xx.xx:8531/_api/agency/config \
-H 'authorization: Basic cm9vdDptaHh6a2hs' \
-H 'cache-control: no-cache' \
-H 'postman-token: 5160250f-4471-84aa-e377-bd0b99ba9926'
I want to use logger-follow to monitor the change of data in cluster. query3 post to dbserver(port=8530) with basic auth, response 401. If my cluster start without --auth.jwt-secret /my/path/to/jwt, such query has response 200.
curl -X GET \
'http://xx.xx.xx.xx:8530/_api/replication/logger-follow?from=0' \
-H 'authorization: Basic cm9vdDptaHh6a2hs' \
-H 'cache-control: no-cache' \
-H 'postman-token: 11b09c7f-85b1-4ed3-ca78-f28cf86c331b'
Is there some difference of authentication between agency/dbserver with coordinator?
Or is there something wrong with authentication of agency/dbserver?
Hey so trying to use the API to purge individual links but what do I replace with identifier in this link? https://api.cloudflare.com/client/v4/zones/:identifier/purge_cache
Anyone know where it is located exactly, cloudflare support is a waste of time.
According to the CloudFlare API documentation for Purge Cache, you can post an array of files in a HTTP DELETE request to that endpoint.
Now, that identifier is the Zone ID, for this, you can get a list using a GET request on the Zones endpoint, e.g.:
$ curl -X GET "https://api.cloudflare.com/client/v4/zones?name=example.com&status=active&page=1&per_page=20&order=status&direction=desc&match=all" \
-H "X-Auth-Email: user#example.com" \
-H "X-Auth-Key: c2547eb745079dac9320b638f5e225cf483cc5cfdda41" \
-H "Content-Type: application/json"
The ID will be in the "result" variable that comes back. Details are here: List zones.
After you've got this ID you can then pass it through as the identifier in the DELETE request.
Example with cURL:
$ curl -X DELETE "https://api.cloudflare.com/client/v4/zones/023e105f4ecef8ad9ca31a8372d0c353/purge_cache" \
-H "X-Auth-Email: user#example.com" \
-H "X-Auth-Key: c2547eb745079dac9320b638f5e225cf483cc5cfdda41" \
-H "Content-Type: application/json" \
--data '{"files":["http://www.example.com/css/styles.css"],"tags":["some-tag","another-tag"]}'
Response:
{
"success": true,
"errors": [],
"messages": [],
"result": {
"id": "023e105f4ecef8ad9ca31a8372d0c353"
}
}
mjsa is right, but you are probably running into CloudFlare's firewall.
I ran into the exact same problem with CloudFlare blocking my cURL requests. I managed to solve, or rather bypass it, by adding a User-Agent header to my requests. For example:
"User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.87 Safari/537.36"
When I submit a REST request, EG: (included auth is for admin/adminadmin)
curl -ik -X POST -H "Accept: application/json"
-H "Authorization: Basic YWRtaW46YWRtaW5hZG1pbg=="
https://localhost:4848/management/domain/applications/application/MyApp/enable
GlassFish just rejects the request:
HTTP/1.1 400 Bad Request
Content-Length: 0
Date: Wed, 17 Jul 2013 10:33:06 GMT
Connection: close
What am I doing wrong?
I've used the GET method to check the command parameters and they're all optional.
From: http://docs.oracle.com/cd/E26576_01/doc.312/e24928/general-administration.htm
REST requests that add, update, or delete objects must specify the X-Requested-By header with the value "GlassFish REST HTML interface".
So EG:
curl -ik -X POST -H "Accept: application/json"
-H "Authorization: Basic YWRtaW46YWRtaW5hZG1pbg=="
-H "X-Requested-By: GlassFish REST HTML interface"
https://localhost:4848/management/domain/applications/application/MyApp/enable
Based on the answer above, ajusted for those who try to do it with the successor of Glassfish - the Payara Server:
Enable App
curl -ik -X POST \
-H 'accept: application/json;charset=UTF-8' \
-H 'authorization: Basic YWRtaW46YWRtaW5hZG1pbg==' \
-H 'content-type: application/x-www-form-urlencoded' \
-H 'x-requested-by: GlassFish REST HTML interface' \
--data target=server \
--url https://localhost:4848/management/domain/applications/application/awesomeApp/enable
Disable App
curl -ik -X POST \
-H 'accept: application/json;charset=UTF-8' \
-H 'authorization: Basic YWRtaW46YWRtaW5hZG1pbg==' \
-H 'content-type: application/x-www-form-urlencoded' \
-H 'x-requested-by: GlassFish REST HTML interface' \
--data target=server \
--url https://localhost:4848/management/domain/applications/application/awesomeApp/disable