I would like to upgrade a macOS application I built to allow users to directly download their transaction statements via OFX data direct connect.
So far, my research has found two good sources of info
OfxTools: https://ofxtools.readthedocs.io/en/latest/index.html
OfxHome: https://www.ofxhome.com/index.php/home/ofxget
I've downloaded and installed both (command line from OfxHome and Python from OfxTools). Then tried to download statements from two of my accounts (TD Ameritrade and Fidelity). Both failed.
TD Ameritrade fails with "Signon Invalid"
Fidelity fails with "Access Denied" cannot access from this server
I initially thought the issue is that I am overseas and running from my own computer. So I installed the same two packages on a hosted server running out of San Francisco. Yet, both failed with the same errors.
I happen to be a member of an application (Profit.ly) that allows you to download your transaction statements (they use OFX, just like Quicken, etc). I can enter my TD Ameritrade and Fidelity credentials on Profit.ly, and it can download my transaction data successfully.
So what am I missing? Below is my request from OfxHome cmd line for fidelity.
./ofxget -institution 449 -request accounts.txt
OFXHEADER:100
DATA:OFXSGML
VERSION:102
SECURITY:NONE
ENCODING:USASCII
CHARSET:1252
COMPRESSION:NONE
OLDFILEUID:NONE
NEWFILEUID:20221211035100.000
<OFX>
<SIGNONMSGSRQV1>
<SONRQ>
<DTCLIENT>20221211035100.000
<USERID>MyUserName
<USERPASS>MyPassword
<LANGUAGE>ENG
<FI>
<ORG>fidelity.com
<FID>7776
</FI>
<APPID>QBW
<APPVER>1800
</SONRQ>
</SIGNONMSGSRQV1>
<SIGNUPMSGSRQV1>
<ACCTINFOTRNRQ>
<TRNUID>20221211035100.000
<CLTCOOKIE>1
<ACCTINFORQ>
<DTACCTUP>19691231
</ACCTINFORQ>
</ACCTINFOTRNRQ>
</SIGNUPMSGSRQV1>
</OFX>
Fidelity Response
<HTML><HEAD>
<TITLE>Access Denied</TITLE>
</HEAD><BODY>
<H1>Access Denied</H1>
You don't have permission to access "http://ofx.fidelity.com/ftgw/OFX/clients/download" on this server.<P>
Reference #18.65f466ab.1670748661.20bf6f3d
</BODY>
</HTML>
TD Ameritrade OFX request
OFXHEADER:100
DATA:OFXSGML
VERSION:102
SECURITY:NONE
ENCODING:USASCII
CHARSET:1252
COMPRESSION:NONE
OLDFILEUID:NONE
NEWFILEUID:20221211040449.000
<OFX>
<SIGNONMSGSRQV1>
<SONRQ>
<DTCLIENT>20221211040449.000
<USERID>MyUserName
<USERPASS>MyPassword
<LANGUAGE>ENG
<FI>
<ORG>ameritrade.com
<FID>5024
</FI>
<APPID>QBW
<APPVER>1800
</SONRQ>
</SIGNONMSGSRQV1>
<INVSTMTMSGSRQV1>
<INVSTMTTRNRQ>
<TRNUID>20221211040449.000
<CLTCOOKIE>4
<INVSTMTRQ>
<INVACCTFROM>
<BROKERID>ameritrade.com
<ACCTID>MyAcctNumber
</INVACCTFROM>
<INCTRAN>
<DTSTART>19000101
<INCLUDE>Y
</INCTRAN>
<INCOO>Y
<INCPOS>
<DTASOF>20221211040449.000
<INCLUDE>Y
</INCPOS>
<INCBAL>Y
</INVSTMTRQ>
</INVSTMTTRNRQ>
</INVSTMTMSGSRQV1>
</OFX>
TD Ameritrade Response
OFXHEADER:100
DATA:OFXSGML
VERSION:102
SECURITY:NONE
ENCODING:USASCII
CHARSET:1252
COMPRESSION:NONE
OLDFILEUID:NONE
NEWFILEUID:20221211040449.000
<OFX><SIGNONMSGSRSV1><SONRS><STATUS><CODE>15500</CODE><SEVERITY>ERROR</SEVERITY><MESSAGE>Signon invalid</MESSAGE></STATUS><DTSERVER>20221211040450</DTSERVER><LANGUAGE>ENG</LANGUAGE><FI><ORG>ameritrade.com</ORG><FID>5024</FID></FI></SONRS></SIGNONMSGSRSV1><INVSTMTMSGSRSV1><INVSTMTTRNRS><TRNUID>20221211040449.000</TRNUID><STATUS><CODE>15500</CODE><SEVERITY>ERROR</SEVERITY><MESSAGE>pr-txlvofx-pp02-clientsys Signon invalid</MESSAGE></STATUS><CLTCOOKIE>4</CLTCOOKIE></INVSTMTTRNRS></INVSTMTMSGSRSV1></OFX>
Why can I not use my own computer with my own bank and credentials to download my data via command line using ofxtools or ofxget?
Is there some sort of authorization token, SSL cert, or something I am missing?
Is there a list somewhere of servers that banks trust to fetch OFX data?
Am I asking the wrong questions? lol
Possible deprecation: OFX merging with FDX (https://financialdataexchange.org/ofx)
Update
I got TD Ameritrade to response successfully by changing the APPID and APPVER
From
<APPID>QBW
<APPVER>1800
To
<APPID>MDNC
<APPVER>2021
Using OfxHome commands, I was able to fetch my TD Ameritrade transaction data however, still stuck on Fidelity.
This raises a bigger issue, which is probably core to the whole OFX thing. I'm trying to test OFX data fetching by using two brokers that I have. Straight off the bat I ran into problems and have spent all day just to figure out how to get one broker working. This is not scalable.
I guess the core of the problem is to find a database that contains the most update-to-date data for all the financial institutions. Then, be able to read the database, parse the data into the correct OFX format (w/ correct tags, version #, data), then send the request. This is the most challenging part.
I've reviewed 5 OFX libraries and tested 3 of them. All failed out-of-the-box to format a proper OFX request for TD and Fidelity. After hours of work I was able to piece together what is required by TD.
So the question is
Who has the most updated OFX request data for each financial institution?
Code to compile the OFX data into a proper request for each OFX version
Related
I am attempting to create a single opening balances record against an existing employee but keep getting a 400 Bad Request response with this detail...
At least one NZ opening balance item is required in the request
I am following the instructions as per this documentation...
https://developer.xero.com/documentation/api/payrollnz/employeeopeningbalances#post-opening-balances
URL : {DestinationID} is properly replaced with the employee GUIDhttps://api.xero.com/payroll.xro/2.0/employees/{DestinationID}/openingBalances
JSON Body[{"periodEndDate":"2011-01-30T00:00:00","daysPaid":5.00,"unpaidWeeks":0.00,"grossEarnings":1442.31}]
The Xero forums and support is pretty unreliable so I'm posting here in the hopes for a better response.
After some trial and error using the API Explorer that Xero provides I was able to get it working using their example....
I eventually learned that daysPaid and unpaidWeeks must both be integer whole numbers or else it fails.... The error message provided is misleading but this resolves the problem.
I'm getting the error below while connecting to Azure Table Storage,
Details:
Blockquote "AzureTables: Request failed: The remote server returned an error:
(415) Unsupported Media Type. (None of the provided media types are
supported)
The one thing I noticed is that if I fill up only the account name it will automatically add the rest of the url which is ".table.core.windows.net" where in the portal its table.cosmosdb.azure.com.
With core.windows.net Im getting err "AzureTables: Request failed: The remote name could not be resolved". But it might messing up some headers while using table.cosmosdb.azure.com
Please advise.
Thank you.
m
You should be able to connect to your azure table storage/CosmosDB account using powerBi using the following link structure: https://STORAGEACCOUNTNAME.table.core.windows.net/ , or https://yourcosmosdbname.documents.azure.com:443/ for cosmosdb
You can get the correct link by going to Portal > go to Storage accounts > Click on Tables/CosmosDB > You'll find the table link you would like to link to powerbi > remove the last table name after "/", then use it to connect in powerbi, it will later allow you to select the specific table in powerBI:
These are screenshots from testing for CosmosDB:
Errors 415:
When it comes to these errors, they can be caused by cache, which can be flushed by going to:
In Power BI Desktop: Go to "File" and select "Options". Under "Data Load" you have the option to clear the cache. After doing this you can use "Get Data" and "OData-feed" as normal and the URL won't return the 415 error
Check the following link for additional suggestions:
Not clear how you consume the table service API, but here is the solution that worked for me for React SPA and fetch api.
Request header must contain:
"Content-Type":"application/json"
It was failing for me with single quotes, and worked with double.
My reasearch need data posted from Stocktwits,Just like search :the symbol "SPX",and download microblogging messages of S&P 500 Index posted between October 2016 and October 2017.
The target page:https://stocktwits.com/symbol/SPX.
How can I download the full text?
The API does not allow querying by date. Please see the API docs on how to request streams for symbols here
https://api.stocktwits.com/developers/docs/api#streams-symbol-docs
and how to paginate through the streams
https://api.stocktwits.com/developers/docs/parameters
I am trying to test GitHub API and explore it's different endpoints. I have the different possibilities which are returned by let's say the initial get request on https://api.github.com.
The returned list is:
{
"current_user_url": "https://api.github.com/user",
"current_user_authorizations_html_url": "https://github.com/settings/connectio ns/applications{/client_id}",
"authorizations_url": "https://api.github.com/authorizations",
"code_search_url": "https://api.github.com/search/code?q={query}{&page,per_pag e,sort,order}",
"commit_search_url": "https://api.github.com/search/commits?q={query}{&page,pe r_page,sort,order}",
"emails_url": "https://api.github.com/user/emails",
"emojis_url": "https://api.github.com/emojis",
"events_url": "https://api.github.com/events",
"feeds_url": "https://api.github.com/feeds",
"followers_url": "https://api.github.com/user/followers",
"following_url": "https://api.github.com/user/following{/target}",
"gists_url": "https://api.github.com/gists{/gist_id}",
"hub_url": "https://api.github.com/hub",
"issue_search_url": "https://api.github.com/search/issues?q={query}{&page,per_ page,sort,order}",
"issues_url": "https://api.github.com/issues",
"keys_url": "https://api.github.com/user/keys",
"notifications_url": "https://api.github.com/notifications",
"organization_repositories_url": "https://api.github.com/orgs/{org}/repos{?typ e,page,per_page,sort}",
"organization_url": "https://api.github.com/orgs/{org}",
"public_gists_url": "https://api.github.com/gists/public",
"rate_limit_url": "https://api.github.com/rate_limit",
"repository_url": "https://api.github.com/repos/{owner}/{repo}",
"repository_search_url": "https://api.github.com/search/repositories?q={query} {&page,per_page,sort,order}",
"current_user_repositories_url": "https://api.github.com/user/repos{?type,page ,per_page,sort}",
"starred_url": "https://api.github.com/user/starred{/owner}{/repo}",
"starred_gists_url": "https://api.github.com/gists/starred",
"team_url": "https://api.github.com/teams",
"user_url": "https://api.github.com/users/{user}",
"user_organizations_url": "https://api.github.com/user/orgs",
"user_repositories_url": "https://api.github.com/users/{user}/repos{?type,page ,per_page,sort}",
"user_search_url": "https://api.github.com/search/users?q={query}{&page,per_pa ge,sort,order}"
}
Is there a way into Postman, or follow up facilitators to generate a collection from these?. Some scripting will be necessary for sure to follow Postman Json Collection format.
If there is a solution other than Postman completely that fits in this issue, I am all ears.
What I did:
I tried to search for some GitHub API as a Postman Collection, I
didn't find any.
I tried to understand Postman Json Collection format, which is not so easy for me to write some script to create one. Maybe someone did?
I can "find and replace : "([a-z_])*": with curl -H "Authorization: token ####". Is there a way to integrate the whole to Postman?
I can't help it unless I consume a lot of effort. But also, I may be stuck other times as I'm exploring advanced API capabilities, techniques, and choices not only for GitHub API. So this won't be the last.
I am getting the following exception whenever i try to fetch defects from rally:
com.google.gson.JsonSyntaxException:
com.google.gson.stream.MalformedJsonException: Use JsonReader.setLenient(true) to accept malformed JSON at line 1 column 12
at com.google.gson.JsonParser.parse(JsonParser.java:65)
at com.google.gson.JsonParser.parse(JsonParser.java:45)
at com.rallydev.rest.response.Response.<init>(Response.java:25)
at com.rallydev.rest.response.QueryResponse.<init>(QueryResponse.java:16)
at com.rallydev.rest.RallyRestApi.query(RallyRestApi.java:168)
at Test.main(Test.java:86)
Caused by: com.google.gson.stream.MalformedJsonException: Use JsonReader.setLenient(true) to accept malformed JSON at line 1 column 12
at com.google.gson.stream.JsonReader.syntaxError(JsonReader.java:1505)
at com.google.gson.stream.JsonReader.checkLenient(JsonReader.java:1386)
at com.google.gson.stream.JsonReader.doPeek(JsonReader.java:531)
at com.google.gson.stream.JsonReader.peek(JsonReader.java:414)
at com.google.gson.JsonParser.parse(JsonParser.java:60)
... 5
What intrigues me most is the code works perfectly fine on few machines and throws the above exception on few.
code snippet :
RallyRestApi restApi =
new RallyRestApi(new URI("http://rally1.rallydev.com"),apiKey);
QueryRequest queryRequest = new QueryRequest("defects");
queryRequest.setFetch(new Fetch("Project","FormattedID","Release"));
QueryFilter filter1 = new QueryFilter("FormattedID", "=", defetctID);
QueryResponse queryResponse1 = restApi.query(queryRequest);
Try a curl command to read the same defect using the same apiKey (in zsessionid header) on the same machine from which your java code fails.
curl --header "ZSESSIONID: _abc123" "https://rally1.rallydev.com/slm/webservice/v2.0/defect/123456789"
At least you will know if this is specific to java or not. Yes, it is strange that it fails on some machines and works on others, but the timing of those tests is not obvious from your post, and I wonder if this has anything to do with the underlying user credentials. (A user gets disabled for a period of time after a number of unsuccessful attempts). I am not positive that this is the issue you experience but I have seen when expired password caused the exact same error. API Keys are tied to a user, so when a user's password is expired, or when a user is inactivated (disabled) the same permissions(or the lack of them) is reflected in the key. For example, a user did not know that the password was expired because in the Rally UI they used SSO authentiation, but in the code they used either username/password or APIKey since the toolkit does not support SSO at this point. A 401 error would be more helpful, but instead a malformed JSON is generated.