Why is my API call to TDA's API not returning a refresh token? - api

so I've followed the steps here to the letter: https://www.reddit.com/r/algotrading/comments/c81vzq/td_ameritrade_api_access_2019_guide/ in an effort to get a refresh token so that I can build a client app in C# to use TD Ameritrade's API, to conduct special stock and option screening and trading. I got to the end before reaching any trouble.
In the very last step in getting my refresh token, where you fill out the fields on https://developer.tdameritrade.com/authentication/apis/post/token-0, it fails, I just get the following response in an HTTP 401 error:
HTTP/1.1 401 Unauthorized
Access-Control-Allow-Headers: origin
Access-Control-Allow-Methods:
GET
Access-Control-Allow-Origin:
https://developer.tdameritrade.com
Access-Control-Max-Age: 3628800
Cache-Control: no-cache
Connection: keep-alive
Content-Length: 31
Content-Security-Policy: frame-ancestors 'self'
Content-Type: application/json;charset=UTF-8
Date: Sat, 04 Apr 2020 16:07:04 GMT
Strict-Transport-Security: max-age=31536000; includeSubDomains
X-Application-Context: OAUTH_SERVICE:run:8080
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Xss-Protection: 1; mode=block
{
"error": "invalid_grant"
}
What could the problem be? I've double and triple checked each step and the values in the text boxes (imaged below) to receive a refresh token, but I can't get it to work. The code and client ID are properly done, as well, based on the guide's steps - I double and triple checked.
I followed the steps in the Reddit guide, and from a linked guide by TDA Ameritrade themselves, to ensure I was putting the proper values in the proper fields, so I really have no clue what's going on.
Any help is appreciated.

"code" or "authorization code" is only a one-time pass-key that expires after 30 minutes. It allows you to get the "access token", which, although it has a similar name, seems to be different. So make sure you URL-decode the auth code reasonably rapidly, then get your first response correctly in before 30 minutes.
Also try on weekends, as TDAmeritrade's SMS gateway has been frustratingly slow recently during the GameStop raids, and it's possible their authentication handshaking could be having problems as well during trading hours.
Note that appending "#AMER.OAUTHAP" does not seem to be necessary for the client_id slot of the manual Post Access Token form.

I ran into this issue and the problem was I was url encoding the redirect uri in the console when requesting the refresh token. When I retried using a urldecoded uri it worked.

It turns out, http://localhost does not function for a redirect URI in TDA's API. You have to use https://127.0.0.1 in your app settings and then again in the many steps that follow when you try to generate your first refresh token.
So, that solves that.

Make sure that your redirect_uri matches exactly to that configured in your app.
If you're using a Windows machine, I'd suggest setting your callback URL to:
http://localhost
Note that this is http rather than https.
This will save you from having to setup IIS and configuring SSL on your workstation.
Also, something worth remembering is that the authorization code will only work once when creating your bearer and refresh token.
On any subsequent attempts at sending that same authorization code, you'll get the "invalid_grant" error.

I had similar issue and it was resolved using below steps -
Update callback URI on custom app to https://127.0.0.1
Request new access code after updating the URI.
Access token is valid only for 30 mins. Hence, the refresh token should be requested ASAP.

Related

Create session via API Token sets an expired cookie

I am trying to create a session via an API token .
The request is sent successfully, I'm getting 3 cookies in the response, but their expiry date is the same as the time at which the request was sent, which invalidates them instantly.
For example, if the request is sent successfully on 20 April 2020, 00:24:29 this is the response:
Set-Cookie: persistent=XXXX; path=/; expires=Thu, 19-Mar-2020 00:24:30 GMT; secure; HttpOnly
Set-Cookie: onelogin.com_user=XXX; domain=.onelogin.com; path=/; expires=Mon, 20-Apr-2020 00:24:30 GMT; secure; HttpOnly
Set-Cookie: sub_session_onelogin.com=XXXXXXX; path=/; secure; HttpOnly
The second cookie that creates the session for using it on Onelogin is already expired.
Is there anything that is being done wrong? Any help would be much appreciated.
I am using HTTPS.
The initial API request to get the session token (note, not cookie) is a backend call from the app to the OneLogin API and so it is not visible in user browser, but you can see that that API call returns the JWT token with a 2 minute timeout to a customer app. You can navigate to https://jwt.io/ and click Debugger for verification. There will be an expiration area on the right side and you can hover over this to see the expiration. There is always a 2 minute timeout associated with it.
It is then the responsibility of your app to direct the user's browser to send the token in a HTTP payload to OneLogin at "/session_via_api_token", so as to provide a session cookie to the browser. You can use the SAML Tracer extension in your browser to capture a SAML trace to verify, by viewing the "HTTP" tab of the POST line for "session_via_api_token". Note: 3 cookies are set, of which 2 are persistent and are immediately set to expire - so they serve no purpose. So don't stress the fact that those 2 cookies are expired as it's not relevant. The key cookie is "subsession_onelogin.com" which is a session cookie and so has no expiry time in it. It is used to track session timeout.
If the user then attempts to get access to a OneLogin resource then it includes the new "subsession_onelogin.com" cookie and so is considered authenticated. OneLogin then issues an updated version of that cookie, which you can see in the SAML Trace on the GET line for the "sub_session.onelogin.com" Set-Cookie area.
It appears that the way the CORS method works for this API is slightly different than the form post method, and the API team has been made aware, as the Dev article will likely be updated to state that the form post method is suggested.
In addition, per my testing it appears that some browsers react differently, so testing other browsers would certainly shed more light. That is browser specific and as such wouldn't be related to the API calls. With the Create Session request on Safari when the default setting of "Prevent Cross Site Cookie Tracking" is enabled I have noticed that the cookies are returned as expected but the browser refuses to accept them. There is no warning or error in the browser console. It's unknown why Safari does this and can't find any documentation about it so that's a browser specific issue out of our control.
Browsers are making it harder so our API team will probably update our best practice guidelines in the coming months to say to use the form post redirect rather than CORS. The form post method should work for any and all browsers as they handle it different than the CORS method.
The first link in the API docs is https://developers.onelogin.com/api-docs/1/login-page/create-session-via-token and the second link to use it https://developers.onelogin.com/api-docs/1/login-page/create-session-login-token. You can then use https://jwt.io/ and click Debugger for verification. It's working properly with a 2 minute expiry which is correct. The key cookie is "subsession_onelogin.com" which is a session cookie and so has no expiry time in it as it is used to track session timeout.
In my tests I have verified that there is definitely a 2-minute lifetime, which is exactly how it should work. In my example tests it shows that the API response is claiming expiry at 16:19:04, which matches the jwt.io decoder (that site is useful for this). The 2nd part of the test returned the headers DATE parameter which shows the time of issue was 16:17:04; i.e. a 2 minutes before expiry. This is correct.
I imagine the problem you are having is caused by how your app directs the user's browser to invoke the "/session_via_api_token" call - if the app uses CORS then obviously you would need to include the relevant header. However, it appears that even with that header, some browsers will not send the CORS request (presumably due to browser security settings) and so the session never gets established. At least that is what I can see when using the sample custom login page in testing. Note: The first call is a REST API call (i.e. what Postman is good for), but the 2nd request is a regular browser request.

Why AJAX sends more than one cookie?

I have an aplication running on ExpressJS handling session with "express-session" on mongo store. When i make a regular http request with the browser for loading a page getting i can see the correct cookie on the server as:
console.log(req.headers.cookie) // connect.sid=s%3ABC....
The problem raises when, on the same page, i make an AJAX call and on the server i see something like this:
console.log(req.headers.cookie) // connect.sid=s%3DEF; connect.sid=s%3GHI; connect.sid=s%3ABC
So it seems like AJAX sends old cookies with the same name on the request resulting in the server adding the data to a wrong session..
On the first request the cookie is being set as follows:
set-cookie: connect.sid=s%3ABC...; Path=/; Expires=Wed, 20 Jun 2018 19:40:59 GMT; HttpOnly
Any ideas how can this be happening?

Resumable upload returns Unauthorized when uploading chunk

Today I have a problem with the resumable upload feature of OneDrive via the Microsoft Graph API, as described here. I have integration tests which previously worked, which now fail.
I successfully call createUploadSession and get an uploadUrl to use. I've replaced actual tokens with "XXX" here.
POST https://graph.microsoft.com/V1.0/groups/273c2c33-8533-445d-ae65-4b63be296995/drive/root:/c2fa1a83-74f3-444b-9263-c9539ee3eae2.txt:/createUploadSession HTTP/1.1
Authorization: Bearer XXX
{
"item": {
"#microsoft.graph.conflictBehaviour": "replace"
}
}
Response:
{
"#odata.context": "https://graph.microsoft.com/V1.0/$metadata#microsoft.graph.uploadSession",
"expirationDateTime": "2017-04-27T11:07:50.5650598Z",
"nextExpectedRanges": ["0-"],
"uploadUrl": "https://sageglodbizp.sharepoint.com/sites/SharePointTests/_api/v2.0/drive/items/01LQXPMG56Y2GOVW7725BZO354PWSELRRZ/uploadSession?guid='9d14ed72-e532-442e-94e8-70952b365527'&path='~tmp0B_c2fa1a83-74f3-444b-9263-c9539ee3eae2.txt'&overwrite=True&rename=False&access_token=XXX"
}
So the uploadUrl is there. Then, I try to PUT a chunk to this url but get a 401 Unauthorized in response:
PUT https://sageglodbizp.sharepoint.com/sites/SharePointTests/_api/v2.0/drive/items/01LQXPMG56Y2GOVW7725BZO354PWSELRRZ/uploadSession?guid='9d14ed72-e532-442e-94e8-70952b365527'&path='~tmp0B_c2fa1a83-74f3-444b-9263-c9539ee3eae2.txt'&overwrite=True&rename=False&access_token=XXX HTTP/1.1
Authorization: Bearer XXX
Response
HTTP/1.1 401 Unauthorized
As I'm using the URL provided by the Graph API, I believe this to be a bug. I'm passing the same bearer token to the second call as the first (I've also tried it with no access token, given that one is in the URL).
I have tried this with two different Office 365 tenants, both with the same result.
This has previously worked. Any thoughts on why this has stopped working? Is it correct that the uploadUrl points to my SharePoint endpoint rather than the Graph API?
I should also add, a non-resumable upload directly to the Graph API works fine.
Any suggestions most welcome.
EDIT:
This seems to be affected by the length of the filename to which you upload:
myfile-123100000000000000000000000.txt works successfully
myfile-1231000000000000000000000000.txt fails
This was confirmed as a bug by Microsoft and fixed on Saturday 29 / Sunday 30 April 2017.
Per the documentation:
Including the Authorization header when issuing the PUT call may result in a HTTP 401 Unauthoized response. The Authoization header and bearer token should only be sent when issueing the POST during the first step. It should be not be included when issueing the PUT.
If you remove the Authorization: Bearer XXX header from your PUT call should resolve this issue.

PUT/POST request in SOAPUI giving 403 forbidden, while same request working fine in rest client Postman

There is no authentication on server side so authentication should not be issue.
URL format: PUT
https://localhost/api/v1/protections?integrationKey=111&userKey=1111&group=111&category=foo
Payload:
{"action":"BLOCK"}
This is working fine in Postman.
In SOAP UI , I am giving input as under:
EndPoint: https://localhost
Resource: /api/v1/protections
Parameters:?integrationKey=111&userKey=1111&group=111&category=foo
in Media type, I am selecting "application/json"
and entering {"action": "BLOCK"} but getting "Wed Jan 20 16:25:27 PST 2016:DEBUG:Receiving response: HTTP/1.1 403 Forbidden
"
Is there any suggestion to get the output in SOAP UI.
Depending on the server where the rest is exposed service generates an HTTP 403, you should verify that server is and thus find the fastest response.
Also try making a GET request from the browser to see if you can answer correctly because problem lock your machine to the server.
As is https, it may be that you lack some certificate set SOAPUI. possibly Postman you use already has configured. Try to check this setting.
In my case, I missed the Header "User-Agent" and "accept". I put in Soap UI and Works.
In Postman, this headers it put automatically.

Can't get past 401 Unauthorized Twitter streaming API Oauth

I am attempting to build a c# module to connect to the Twitter streaming API using OAuth (now the only option). I have got to the point where my module will successfully access api urls using GET, but everything I do to try and make a POST request fails with a 401.
I have checked my signature is correct by using the OAuth Tool tab on the page for my Twitter App, and fixing the values for nonce and timestamp in my code. I have curl for Windows set up and can verify that it works with the sample curl script generated by the OAuth tool (by the way, this needs some correction of the quotes to make it work for curl in Windows Cmd. Get rid of single quotes on values that don't need them, use double quotes on anything that needs to be quoted, and on the Authorization header, use double quotes and escape double quotes within the header with a backslash).
I have even gone to the length of running curl in trace mode and outputting the bytes I send in the post body from my c# code and I can verify that they are the same.
I am trying to access 'https://stream.twitter.com/1.1/statuses/filter.json' using 'track=twitter' as the post body. The headers are:
Accept: */*
User-Agent: curl/7.21.7(amd64-pc-win32) libcurl/7.21.7 OpenSSL/0.9.8rzlib/1.2.5
Content-Type: application/x-www-form-urlencoded
Host: stream.twitter.com
Content-Length: 13
Connection: Keep-Alive
Authorization: OAuth <the oauth stuff>
I can't inspect the packets being sent to check on the wire that the requests are identical as they are of course SSL encoded.
Any ideas?
I eventually got this to work. Things that might help you if you have this kind of problem which I discovered:
I had a problem initially because I created a new nonce every time the bit of code was accessed. This meant the nonce which was used in generating the signature key was different from the one in the header. Obviously fail.
I then ran into the above problem. What it was is that I was adding the OAuth header to my request AFTER I sent the request body. For some reason it seems to send the request as soon as you write to the request stream for a POST.
It was very useful in finding 2. that I found out how to use Fiddler to trace web requests from code. Essentially all you need to do is add this to your web.config:
<system.net>
<defaultProxy>
<proxy proxyaddress="http://127.0.0.1:8888" />
</defaultProxy>
</system.net>
As soon as I tried to read the HTTPS request, Fiddler prompted me to install bits so it could decrypt the request, which I did and then I could see the exact request going down the wire. I could compare this with what cURL was doing using
-x 127.0.0.1:8888
option.
However I then ran into a problem with my request timing out. Which bizarrely enough was caused by the fact that Fiddler was proxying the response. Once I took the above out of my web.config again it all worked. Halleluja!