I have two curl commands below. You'll notice I'm passing in a Cookie of "usprivacy=1---". This is a cookie that seems to be added by some web browsers automatically. The issue I'm seeing -- if this cookie is present, some of my assets return HTTP error codes. As soon as the cookie is removed, it returns HTTP CODE 200. It doesn't happen with any other cookies. I suspect there's some Apache configuration variable, but I'm baffled.
This one results in the Apache webserver returning 404 not found:
curl 'https://salvagedinspirations.com/wp-content/plugins/woocommerce/packages/woocommerce-blocks/build/style.css?ver=2.4.5' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:71.0) Gecko/20100101 Firefox/71.0' -H 'Accept: text/css,*/*;q=0.1' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Connection: keep-alive' -H 'Referer: https://salvagedinspirations.com/' -H 'Cookie: usprivacy=1---;' -H 'Cache-Control: max-age=0' -D -
This one results in a "403 Forbidden" header, although HTML is still returned:
curl 'https://salvagedinspirations.com/' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:71.0) Gecko/20100101 Firefox/71.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Connection: keep-alive' -H 'Cookie: usprivacy=1---;' -H 'Upgrade-Insecure-Requests: 1' -H 'Cache-Control: max-age=0' -D -
Thank you
The trailing ; is technically invalid. Does it make a difference? You could be triggering some bad parsing or malicious request detection in a framework, mod_security, etc.
The issue was with an apache module that was loaded in. I forgot which one. Sorry
Related
When making API calls to our auth server (Keycloak, served on Wildfly) from our Angular application there is a point where requests begin to fail, apparently due to too much data being present in the bearer token. I have isolated this to the point where adding a single letter to the user's name will cause the request to fail.
Requests to our other servers with the same bearer token work as expected.
When the requests fail, the browser DevTools show them as failed with no additional information - simply a 'Failed to load response data' message and the application does not receive any data. This issue appears to be browser agnostic (have tried Chrome, Edge, and Firefox).
When I copy one of these failed requests into postman, it succeeds and gets the expected JSON data.
I have thus far experimented pretty extensively with allowed header size on all of the servers and seen no change and am a bit at a loss for where else it would make sense to continue investigating. Does anyone have recommendations?
An example of one of the requests in question, copied as cURL (bash):
curl 'OMITTED/auth/realms/OMITTED/account' \
-H 'authority: OMITTED' \
-H 'pragma: no-cache' \
-H 'cache-control: no-cache' \
-H 'sec-ch-ua: "Chromium";v="92", " Not A;Brand";v="99", "Google Chrome";v="92"' \
-H 'accept: application/json' \
-H 'authorization: bearer OMITTED' \
-H 'sec-ch-ua-mobile: ?0' \
-H 'user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/OMITTED (KHTML, like Gecko) Chrome/OMITTED Safari/OMITTED' \
-H 'origin: http://localhost:4200' \
-H 'sec-fetch-site: cross-site' \
-H 'sec-fetch-mode: cors' \
-H 'sec-fetch-dest: empty' \
-H 'referer: http://localhost:4200/' \
-H 'accept-language: en-US,en;q=0.9' \
--compressed
I am trying to implement OAuth via a series of CURL commands but I am having trouble with the Authorise step. I have captured the Authorize process in Fiddler so I know what to replicate and I have defined the process to get the access code as follows:
POST login credentials to the login page in order to get the
'.ASPXAUTH' cookie in the response.
GET the authorisation page by sending the '.ASPAUTH' cookie and in the response get the '__RequestVerificationToken' from the
response cookies and also the form token (also named
__RequestVerificationToken) from the body of the webpage.
POST to the authorisation page by sending both the '.ASPAUTH' and
'__RequestVerificationToken' cookies as well as the form token
in the body.
Here are the requests I am using for each step with the username and passwords replaced:
curl -x 127.0.0.1:8866 -k -c cookie.txt 'https://oauth.sandbox.trainingpeaks.com/Account/LogOn?ReturnUrl=%2fOAuth%2fAuthorize%3fresponse_type%3dcode%26client_id%3dclientId%26scope%3dcoach%253Aathletes%2520workouts%253Aread%26redirect_uri%3dhttps%253A%252F%252Ftest_url.com%252Fcallback&response_type=code&client_id=clientId&scope=workouts%3Aread%20athlete%3Aprofile&redirect_uri=https%3A%2F%2Ftest_url.com%2Fcallback' -H 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:86.0) Gecko/20100101 Firefox/86.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Content-Type: application/x-www-form-urlencoded' -H 'Origin: https://oauth.sandbox.trainingpeaks.com' -H 'DNT: 1' -H 'Connection: keep-alive' -H 'Referer: https://oauth.sandbox.trainingpeaks.com/Account/LogOn?ReturnUrl=%2fOAuth%2fAuthorize%3fresponse_type%3dcode%26client_id%3dclientId%26scope%3dcoach%253Aathletes%2520workouts%253Aread%26redirect_uri%3dhttps%253A%252F%252Ftest_url.com%252Fcallback&response_type=code&client_id=clientId&scope=workouts%3Aread%20athlete%3Aprofile&redirect_uri=https%3A%2F%2Ftest_url.com%2Fcallback' -H 'Upgrade-Insecure-Requests: 1' --data-raw 'UserName=MY_USER_NAME&Password=MY_PASSWORD'
This returns the '.ASPXAUTH' cookie used in the next request.
curl -x 127.0.0.1:8866 -k -c cookie.txt 'https://oauth.sandbox.trainingpeaks.com/OAuth/Authorize?response_type=code&client_id=clientId&scope=coach%3Aathletes%20workouts%3Aread&redirect_uri=https%3A%2F%2Ftest_url.com%2Fcallback' -H 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:86.0) Gecko/20100101 Firefox/86.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Referer: https://oauth.sandbox.trainingpeaks.com/Account/LogOn?ReturnUrl=%2fOAuth%2fAuthorize%3fresponse_type%3dcode%26client_id%3dclientId%26scope%3dcoach%253Aathletes%2520workouts%253Aread%26redirect_uri%3dhttps%253A%252F%252Ftest_url.com%252Fcallback&response_type=code&client_id=clientId&scope=workouts%3Aread%20athlete%3Aprofile&redirect_uri=https%3A%2F%2Ftest_url.com%2Fcallback' -H 'DNT: 1' -H 'Connection: keep-alive' -H 'Cookie: .ASPXAUTH=37D88F4FF97A59398A7F3A01AEDC4ABB32FF99FCE85B346271D9F62D9CEA65B9BF0027A0304DC1E87CDE46948A9F72CC57B1479A37CB1B54F33B74E03C4D20AC44D333FE6FFAD3A4CB69336A14DCA2C46CCBD822C569C1F231383541C99D9F6715D813D1' -H 'Upgrade-Insecure-Requests: 1'
This returns the '__RequestVerificationToken' cookie and I get the form token (__RequestVerificationToken that is sent as part of the body) from the body of the response.
curl -x 127.0.0.1:8866 -k 'https://oauth.sandbox.trainingpeaks.com/OAuth/AuthorizeResponse' -H 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:86.0) Gecko/20100101 Firefox/86.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Content-Type: application/x-www-form-urlencoded' -H 'Origin: https://oauth.sandbox.trainingpeaks.com' -H 'DNT: 1' -H 'Connection: keep-alive' -H 'Referer: https://oauth.sandbox.trainingpeaks.com/OAuth/Authorize?response_type=code&client_id=clientId&scope=coach%3Aathletes%20workouts%3Aread&redirect_uri=https%3A%2F%2Ftest_url.com%2Fcallback' -H 'Cookie: .ASPXAUTH=37D88F4FF97A59398A7F3A01AEDC4ABB32FF99FCE85B346271D9F62D9CEA65B9BF0027A0304DC1E87CDE46948A9F72CC57B1479A37CB1B54F33B74E03C4D20AC44D333FE6FFAD3A4CB69336A14DCA2C46CCBD822C569C1F231383541C99D9F6715D813D1; __RequestVerificationToken=-_UjTRMCw6tv0jVe9bcA_JV7onmEAZZOPyx89_tZavaQY4U2Q4aAwgEx9Ghhp3i8Uh31FL-zHVFrSTAgqUgDPxpMDQg1' -H 'Upgrade-Insecure-Requests: 1' --data-raw '__RequestVerificationToken=O1mxbFlJ6T3AxL1ua4Vtemuhj5lzaCIxKOm49v_NdlCkitIpfUG4DSBwPwI7jb9o4BnLDA0s3R4w3i2Ftcqph0EAifWSreOQmXYcQ1YM1JlOgFzW0&IsApproved=true&client_id=clientId&redirect_uri=https%3A%2F%2Ftest_url.com%2Fcallback&state=&scope=coach%3Aathletes+workouts%3Aread&response_type=code'
This should redirect to the test URL with the access code as it does when these steps are done in the browser but instead I receive:
<html><head><title>Object moved</title></head><body>
<h2>Object moved to here.</h2>
</body></html>
This seems to indicate that the session has become invalidated and requires starting the process again.
I have compared the requests generated by CURL and those from the browser in Fiddler and I cannot determine a difference.
Here are the API docs, I am trying to implement the 'Authorise' process which results in an access code. Any help on this would be appreciated.
It is tricky to automate logins via CURL, since OAuth redirects tend to involve these actions:
Following HTTP redirects automatically
Auto posting forms with authentication results
Some complex messages, some of which may vary depending on the authentication method
For an easier way of testing - have a look at OAuth Tools, which is kind of like 'Postman for OAuth', and works with any standards based provider - here is an introductory video.
I'm trying to find the way to run cURL query in Zapier webhook, but I can't seem to figure it out. How would I enter this in the Webhook?
curl 'https://api.com/graphql' -H 'Accept-Encoding: gzip, deflate, br' -H 'Content-Type: application/json' -H 'Accept: application/json' -H 'Connection: keep-alive' -H 'DNT: 1' -H 'Origin: https://api.com' -H 'access_token: <my token>' --data-binary '{"query":"{\n booking(booking_id: \"11111\"){\n user{\n name\n }\n body\n }\n}\n"}' --compressed
This is the error I'm getting:
You need to put query in the left box under Query String Params. query is the key, the value is {\n booking(book.... You don't need quotes, either.
User Session Management-
- How do we maintain the user sessions by using only SlashDB?
- Is there any other resources required for that or only SlashDb is sufficient?
SlashDB has cookie based session management. Whenever a user logs into the GUI with username/password a cookie is generated/checked. Although generally a stateless mechanism of API keys is preferred, the cookie-based method can be used from a client apps.
curl 'https://your-slashdb-host/login' -v
-H 'Pragma: no-cache'
-H 'Origin: https://demo.slashdb.com'
-H 'Accept-Encoding: gzip, deflate'
-H 'Accept-Language: en-US,en;q=0.8,pl;q=0.6,fr;q=0.4'
-H 'Content-Type: application/json;charset=UTF-8'
-H 'Accept: application/json, text/plain, */*'
-H 'Cache-Control: no-cache'
-H 'Referer: http://myslashdbhost/'
-H 'Connection: keep-alive'
--data-binary '{"login":"someuser","password":"secret-password"}'
--compressed
The response will contain cookie headers
Set-Cookie: auth_tkt=ffeef88c70bda165db830dxxb35b503559a6c5b0YWRtaW4%3D!userid_type:b64unicode; Max-Age=3600; Path=/; expires=Wed, 30-Aug-2017 15:03:28 GMT
Hey so trying to use the API to purge individual links but what do I replace with identifier in this link? https://api.cloudflare.com/client/v4/zones/:identifier/purge_cache
Anyone know where it is located exactly, cloudflare support is a waste of time.
According to the CloudFlare API documentation for Purge Cache, you can post an array of files in a HTTP DELETE request to that endpoint.
Now, that identifier is the Zone ID, for this, you can get a list using a GET request on the Zones endpoint, e.g.:
$ curl -X GET "https://api.cloudflare.com/client/v4/zones?name=example.com&status=active&page=1&per_page=20&order=status&direction=desc&match=all" \
-H "X-Auth-Email: user#example.com" \
-H "X-Auth-Key: c2547eb745079dac9320b638f5e225cf483cc5cfdda41" \
-H "Content-Type: application/json"
The ID will be in the "result" variable that comes back. Details are here: List zones.
After you've got this ID you can then pass it through as the identifier in the DELETE request.
Example with cURL:
$ curl -X DELETE "https://api.cloudflare.com/client/v4/zones/023e105f4ecef8ad9ca31a8372d0c353/purge_cache" \
-H "X-Auth-Email: user#example.com" \
-H "X-Auth-Key: c2547eb745079dac9320b638f5e225cf483cc5cfdda41" \
-H "Content-Type: application/json" \
--data '{"files":["http://www.example.com/css/styles.css"],"tags":["some-tag","another-tag"]}'
Response:
{
"success": true,
"errors": [],
"messages": [],
"result": {
"id": "023e105f4ecef8ad9ca31a8372d0c353"
}
}
mjsa is right, but you are probably running into CloudFlare's firewall.
I ran into the exact same problem with CloudFlare blocking my cURL requests. I managed to solve, or rather bypass it, by adding a User-Agent header to my requests. For example:
"User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.87 Safari/537.36"