First off, I apologize if I use any incorrect terminology. I'm familiar with calling internal apis, but this a side project and new territory for me.
I've been using chrome debugger and powershell to write some scripts against the api of a public site. I was able to retrieve the JWT token, however after a few minutes my calls will start to fail with a 403 error. To get around this I've been exporting all the request headers from chrome and storing those locally for my script to pass whenever my call to the api fails. I narrowed it down to 4 headers that must be present, the one that is most suspect seems to be "cookie".
I then removed all key value pairs in the cookie header until I found the one that makes it fail. It's named "__cf_bm". A quick google says it's some kind of cloudflare bot management data. I don't see any of the data present in the __cf_bm cookie being sent in a response during authentication, I only see it in request headers. Is it not possible to construct this value? Does this essentially prevent me from interacting with the site's api?
Related
I'm attempting to interact with the Google Sheets API and running into an inexplicable problem that I'm finally reaching out to see how anyone else may have tackled it. Put simply, I can use the in-page API Explorer tool with only the https://www.googleapis.com/auth/spreadsheets.readonly OAuth2 scope at https://developers.google.com/sheets/api/reference/rest/v4/spreadsheets/get to query my spreadsheet (just the spreadsheet ID, leaving all other fields to empty defaults) and I'll see the 200 with the response in the bottom as expected.
Of course, I can't re-use the same access token that tool uses, but if provision an access code for the same user for my own app (same scope), and make the same GET request to https://sheets.googleapis.com/v4/spreadsheets/<spreadsheetId> in Postman (again, no other fields populated), substituting the access token into the Authentication header with Bearer <accessToken>, I get a 404.
I know the file is there - I've triple checked that I'm using the same spreadsheet ID across either request and I'm consistently getting a 404 (not a 401 or 403) indicating that my access token does authenticate.
I've tried broadening my OAuth2 scopes to include the full range listed on the API Explorer:
https://www.googleapis.com/auth/drive
https://www.googleapis.com/auth/drive.file
https://www.googleapis.com/auth/drive.readonly
https://www.googleapis.com/auth/spreadsheets
https://www.googleapis.com/auth/spreadsheets.readonly
Of course, I don't want to have to use all those scopes for my purposes - I'd like to use the most narrow scope possible, but I also wanted to rule out that it wasn't failing to work for some scoping scenario. No difference - still a 404 every time I make the request in Postman. I've tried issuing multiple access tokens now, using accounts.google.com to invalidate the tokens for my app between re-issuances, but to no avail.
To be clear, the Google Sheets API has been enabled for my app.
In hopes that someone else has experienced the same inability to query Google's v4 REST API despite using valid access tokens, could you share how you managed to do it?
I appreciate it!
Update:
So I've been playing around with the OAuth 2.0 Playground shared in the comments and found that the authorization endpoint I was using was identical, but the token endpoint differed. This doesn't seem to matter since I used the custom option to use the alternate endpoint and the Playground was still able to work without issue just like the API Explorer.
Using the custom entries, I also entered my own app's client ID and client secret (after registering the playground redirect URI), minimizing the differences between what I'm doing in Postman and in the various Google tools. Again, my GET request to the spreadsheet works without issue.
Just to be clear, here's what I've been doing in the Playground:
In Step 1, I've specified the https://www.googleapis.com/auth/spreadsheets.readonly scope to authorize. I click the Authorize APIs button and log in with the user account.
It returns with the authorization code, so I exchange that code for the tokens via a POST to the token endpoint.
I then make a GET request to https://sheets.googleapis.com/v4/spreadsheets/<spreadsheetId> with no additional headers and it works without issue - 200 OK and all the data I'm expecting to see.
Here my approach in Postman:
Make a GET request to:
https://accounts.google.com/o/oauth2/v2/auth?access_type=offline&response_type=code&redirect_uri=https://<myDomain>/oauthResp&client_id=<appClientId>&scope=https://www.googleapis.com/auth/spreadsheets.readonly&state=abc123
Receive a response similar to the following in my browser on the redirect (since the domain intentionally 404s).
https:///oauthResp?state=abc123&code=zyx098&scope=https://www.googleapis.com/auth/spreadsheets.readonly
Make a POST request to: https://www.googleapis.com/oauth2/v4/token with a body of:
client_id=<appClientId>
client_secret=<appClientSecret>
redirect_uri=https://<myDomain>/oauthResp
grant_type=authorization_code
Receive a response similar to:
{
"access_token": "abc123",
"expires_in": 3599,
"refresh_token": "zyx098",
"scope": "https://www.googleapis.com/auth/spreadsheets.readonly",
"token_type": "Bearer"
}
Make a GET request to https://sheets.googleapis.com/v4/spreadsheets/<spreadsheetId> with a 'Content-Type' header of application/json and an 'Authorization' header of Bearer abc123 (per the access token above).
Unlike the API Explorer and the OAuth 2.0 Playground, this yields a 404 - exactly the issue I've been experiencing for no obvious reason.
Further, if I simply take the fresh access token from the Playground and drop that into Postman, I get the same 404.
Any other ideas?
after successfully getting past the login page using curl in Linux (bash) with two sites that use information from the HTML-form field to accomplish this, I now also want to login to another site with my credentials that is a bit more tricky. I'm new to this and it seems that it works with OAuth so I need an access token first. With my two successfull attempts it was necessary to extract a CSRF token from the HTML code so as to prove I'm not a malicious bot. But now it's completely different and the server thinks I'm a bot (see error messages below).
Can somebody please confirm whether these are the only steps involved for logging in to an OAuth protected site:
a POST request to the server that generates the token (https://name_of_site/api/v1/auth/token)
a GET request to a password-protected part of the website using the access token from step 1
When I try step 1, I get error messages as shown below*. I inspected the website before logging in and afterwards with a browser and copied everything as cURL (bash). In the header fields of the token request I can find two more fields that look like they provide dynamic information:
x-trace-request-id
x-recaptcha-v3
I did some research and found that the request ID stays the same for some interactions that go to and from a server to facilitate identification of the client(?).
And concerning the recaptcha, I know what a recaptcha is, but I did not have to solve any captchas when accessing the site with a browser. So I'm quite unsure about these bits of information.
I strongly assume that I need to send one or both of these headers in the first step to actually get an acess token. But how could I get valid values for these parameters with cURL?
Any help is aprreciated! Please point me in the right direction. I can provide more information if needed, of course.
{"status":"ERROR","statusCode":400,"data":{"message":"Request failed with status code 400","error":"invalid_grant","error_description":"invalid credential","bot-detection-error":"MissingAdditionalAuthToken"}}
This is the error I get when I do not provide x-trace-request-id and x-recaptcha-v3
{"status":"ERROR","statusCode":400,"data":{"message":"Request failed with status code 400","error":"invalid_grant","error_description":"invalid credential","bot-detection-error":"InvalidCaptchaV3Token"}}This is the error I get when I simply use the values as copied from the old request when I used the browser
I have recorded a login flow of an application and found some URIs like below:
/api/oauth2/initiate GET
/oauth2/authorize GET
/api/v1/oauth2/authorize GET
/api/v1/oauth2/authenticate POST
{"username":"${Username}","password":"${Password}","client_id":"${client_Id}","response_type":"code","redirect_uri":"${scheme}://${host}/api/oauth2/callback","server_id":"${server_Id}"}
When I am hitting above in sequence via JMeter I am getting 200 response. Just like JMeter I tried recording in Postman and it worked same, but instead of JSON it gave response in XML format.
It doesn't generate a access_token, it works via session cookies.
My question is - Do I really have API access or it is just browser record n play? If Yes, Does this mean I can get access to any API, if I am a registered user of that application? For ex: Facebook, YouTube or any startup website.
JMeter works on the protocol level. This means that whatever request you are generating. Say a simple browser request or an API call, you can do that easily.
Now the thing is replicating requests. You don't need to record the requests necessarily using the browser. You need to analyze the few things that are required. Say Postman is generating a request. You specify the things you want to send and you use the API Token there. The same things can be specified there as well. It all depends on how you are understanding the concept of request generation.
You simply need to replicate the samplers and the parameters. And the request headers in postman can be replicated here in the same way.
For each HTTP Request Sampler make sure you add a corresponding child HTTP Header Manager config element.
Headers basically tell the server that what client we are using and in what form data is being sent and then server responds accordingly with the information.
What you're recorded is OAuth2 flow and you won't be able to replay it without correlating the dynamic values.
You can have access to Google API or Facebook Graph API given you have proper access_token but I don't think you should be testing them directly, you should focus on solely your application.
I am trying to make requests to the Nike+ API to get information but I do not know where to get a long-term access token from.
I know that I am making the actual request correctly (using AJAX) because I send the same request that Nike+ send when using their test console (I take it from the Chrome network tab when the test console sends). This request works for about an hour before saying the token has expired.
How do I get a long-term token? I have looked around the API and can't find it which makes me think I am missing something pretty obvious here.
Update (12/09/2016)
I need to have a client id to start making the implicit flow requests. I took one from the network tab of the chrome browser (from when I logged in) and used it as a parameter(as well as other things) for the first api call but it said it couldn't load partner details.
Any ideas? Am i even allowed to do what I want to do?
Nike+ API uses OAuth 2 for authentication. In his API there is no long-term token, but you can use refresh_token (30 days) to get new access_token (bound with expired_in) when your authentication expires and get unauthorized response. You can read more at Nike+ API documentation.
I am developing a REST API, and have found a very interesting problem.
When I access the resources in a web browser (in my case Chrome), everything works fine. However, when I access the resources using a REST client (Google Chrome plugin, web-based client, and a Java applet), NONE of the variables pass to the API. This happens both with GET and POST methods.
I have absolutely no idea why this would be the case, and it's proving very difficult to test the methods before putting them into production (especially with POST methods).
Please help!
At first glance it sounds it could be 2 things:
You are not correctly passing API parameters via your client or
applet
A problem with authentication or cookie management. Does the API require any type of authorization?
Are you forgetting to set a necessary HTTP header?
Do you have control of the API also or is it a third party API? If so, do the params arrive at all or do they arrive empty? What's the error code? - a 403 not authorized (which would make sense if the key doesn't pass) or something else due to missing params.
Try the intermediate step of doing it with CURL form the command line - that'll give you more detail on what's coming back.