Dear Yodlee / #apoorv / #Shreyans,
A client tried adding a Bank of America account. Bank of America is site ID 409, but when adding an account via the Site-based APIs, we get back this response:
{
"errorOccurred": "true",
"exceptionType": "com.yodlee.core.IllegalArgumentValueException",
"referenceCode": "_55558c4f-9107-4ae2-b2ef-1118b5701fca",
"message": "Invalid argument value: Site is not having any content service that is enabled and is not custom409 for: siteId"
}
I used the SiteTraversal/getAllSites API to check the list and it appears that Site 409 has been removed and there is now Site 2852 "Bank of America (US)" instead.
I have a few questions.
a) can you confirm that 409 has been removed?
b) if you just remove banks from your system and give them different ID numbers what happens to all the accounts added to that bank? what would have happened to my users with accounts under Site 409 "Bank of America (US)"? They would just stop working?
c) like all Yodlee-based apps, we need to cache the bank list within our own database to give the users a good experience. If we have to use the APIs constantly to get basic bank data, our apps would be unusably slow. So we cache them, obviously.
However, if you are deleting banks from your list regularly, how are we supposed to handle this on our side?
The SiteTraversal/getAllSites API data response is MASSIVE - really massive. This usually isn't an issue. We just run through it and create/update our internal cached bank records.
But if we have to catch deletions, that changes everything. We would have to hold the entire response JSON data in memory, even though it is enormous, extract all the site ID numbers, then grab all of our cached site ID numbers, and run them against each other to see which ID numbers are not in both lists, which means those are now removed,. The server memory required for such a large operation is substantial, and I don't want to have to do that if there is another way.
What is your guidance on this?
a) can you confirm that 409 has been removed?
Yes it has been disabled.
b) if you just remove banks from your system and give them different ID numbers what happens to all the accounts added to that bank? what would have happened to my users with accounts under Site 409 "Bank of America (US)"? They would just stop working?
If we remove and create new site for the same FI, then ideally we migrate the users from old site to the new site.
Or
We leave the users on the old site with cache refresh turned on and hence those accounts refreshes on a daily basis but no new accounts can be added to the disabled site.
c) like all Yodlee-based apps, we need to cache the bank list within our own database to give the users a good experience. If we have to use the APIs constantly to get basic bank data, our apps would be unusably slow. So we cache them, obviously.
However, if you are deleting banks from your list regularly, how are we supposed to handle this on our side?
We usually recommend calling the SiteTraversal/getAllSites once in a day and update the cached data at your end.
Thanks for you feedback at the end, we will definetely take this into consideration and see what we can do to help on this.
Related
I'm interested in downloading my transactions from Chase without using a third party such as Mint, Quicken, Yodlee, Plaid, and so on. I don't trust third parties with handling my data, which is why I want to do it myself.
Works as of 1/8/18
Summary
Chase uses OFX to support programmatic interactions with its financial data. It does so somewhat reluctantly however, as this fact isn't widely advertised nor well documented - banks would rather have you use their products directly, rather than go to third parties. That being said, support for desktop products like Quicken still exists, and so one method is to spoof yourself as a desktop product (the other options are to use a third party service or use a screen scraper). Obviously this solution is completely at the mercy of Chase's whims, and as a discouraged access pattern this is not robust. But you're still reading, so let's do this!
Solution
Set up your account to accept connections from Desktop apps
Account > Profile & Settings > Manage Account Security > Desktop apps > Click enable
Generate a Client UID. Chase will use this to verify that the OFX requests it's receiving are intended.
Go to https://www.uuidgenerator.net/ and generate a UID.
Use ofx-ba-tfb.py to POST the following to https://ofx.chase.com. Comments begin with a # sign, do not include them.
Headers:
OFXHEADER:100
DATA:OFXSGML
VERSION:102
SECURITY:NONE
ENCODING:USASCII
CHARSET:1252
COMPRESSION:NONE
OLDFILEUID:NONE
NEWFILEUID:{Insert random alphanumeric string}
Payload:
<OFX>
<SIGNONMSGSRQV1>
<SONRQ>
<DTCLIENT>20180108012004
<USERID>{Insert user id}
<USERPASS>{Insert password}
<LANGUAGE>ENG
<FI>
<ORG>B1 # Comes from ofxhome.com
<FID>10898
</FI>
<APPID>QWIN
<APPVER>1800
<CLIENTUID>{Insert random alphanumeric string}
</SONRQ>
</SIGNONMSGSRQV1>
<CREDITCARDMSGSRQV1>
<CCSTMTTRNRQ>
<TRNUID>{Insert random alphanumeric string}
<CLTCOOKIE>4
<CCSTMTRQ>
<CCACCTFROM>
<ACCTID>{Insert account id here}
</CCACCTFROM>
<INCTRAN>
<DTSTART>20171208
<INCLUDE>Y
</INCTRAN>
</CCSTMTRQ>
</CCSTMTTRNRQ>
</CREDITCARDMSGSRQV1>
</OFX>
Verify the request
You will get a response that says,
"Please verify your identity within the next 7 days. Using your desktop computer, go to your bank's website and visit the Secure Message Center for instructions." After a small delay (in minutes), you will receive a message in your Secure Message Center asking to confirm that you made this request. Confirm the request. This response and verification happens with each new Client UID you submit.
Make the OFX request again, and you should receive your transactions!
Comments
Many thanks to Harry Sit # thefinancebuff.com for doing most of the work!!
More is possible through OFX, it's a complicated standard. Look to the specification for details.
You can also use GnuCash to see other promising settings.
This is using OFX 1.02, but Chase is now at 2.2
References
https://thefinancebuff.com/replacing-microsoft-money-part-5-ofx-scripts.html#comments
http://www.ofx.net/
http://www.ofxhome.com/
http://www.ofxhome.com/ofxforum/viewtopic.php?id=47456
https://rhye.org/post/parsing-ofx-leex/
since October 5, 2022, Chase disconnected ofx.chase.com and look like you have to go aggregators like Quicken/MD+ to get your data.
As HLE indicates, Chase has killed OFX/DirectConnect as of late Sept./early Oct. 2022. Here are some related articles:
"As of October 6th, 2022, Chase will no longer be supporting Direct Connect/OFX which includes 3rd-party bill pay. Along with some of the other larger financial institutions, Chase has moved to the Open Banking connection method for transaction importing. A link to an article that explains what Open Banking is all about is listed below."
https://www.banktivity.com/support/articles/banktivity-7/ofx-direct-connect-will-no-longer-be-supported-by-chase-as-of-october-6th-2022/
"Chase Bank is changing the way it connects with personal finance management solutions like Quicken. To keep Quicken connected to your Chase account you'll need to switch you connection method on or before September 26, 2022."
https://community.quicken.com/discussion/7916266/quicken-chase-announce-killing-link-to-chase-direct-connect-bill-pay
Possible solutions for Moneydance users:
http://infinitekind.tenderapp.com/discussions/online-banking/23881-chase-bank-accounts-no-longer-can-connect
It sounds like .QXF files contain the same/similar data as .OFX files and the downloading of such files may be scriptable. I will update this answer if I find out more.
One of our users is getting an error when they attempt to make a purchase and I'm trying to identify why this is occurring.
The message returned from PayPal is:
<Errors xmlns="urn:ebay:apis:eBLBaseComponents">
<ShortMessage>Transaction refused</ShortMessage>
<LongMessage>This transaction cannot be completed because it violates the PayPal User Agreement.</LongMessage>
<ErrorCode>13122</ErrorCode>
<SeverityCode>Error</SeverityCode>
</Errors>
This product is working for other users, just not him.
Obviously it's violating the User Agreement but I'd like to identify why.
UPDATE
The users that are affected by it seem to all have one or more of the following: a non-UK email address, a non-UK PayPal account or a non-UK payment source.
We've not had a resolution yet, but have directed several users to contact PayPal directly. The feedback we've had is as follows:
"I tried with another paypal account, that failed too, despite being able to use both PayPal accounts to pay for other services."
"PayPal are aware of the error message, but they simply cannot explain why it's happening. After an hour on the phone with them today, they seem incapable of tracing the reason for this error."
Needless to say we've got some very frustrated users.
I had similar problem... Title of our item we were selling was:
Inc. Special Offer: <<Famous author>> eBook for only 10.00$
When we changed the title to something more descriptive, everything was OK
Did you open a ticket with MTS or call the Business Support line? If you did open a ticket can you please give me the ticket number? I'll take a look at it.
It may be something that you need to address with Business Support though.
I had this same error coming back for one of our customers and I had the same issue as #knagode with my product name.
For some reason "Dark Havana" in the product name set this off and produced this error. If I ever hear back from Paypal on why this phrase is not acceptable, I will circle back and edit this response.
It turns out that the value in "PaymentDetailsItem - Name" was one part of the problem. We didn't have any special characters (or so I thought) in there, just "product name: person name". When I spoke to PayPal support their response is as follows:
"I have checked it, it's a combination of many things that we cannot disclose.
However removing the ":" before the name PAYMENTREQUEST_0_NAME should work.
Can you send PAYMENTREQUEST_0_NAME=xxxx USER NAME
Instead of PAYMENTREQUEST_0_NAME=xxxx : USER NAME"
Once I had removed the ":" from that field, it started working again...bizarre!
This error still gets triggered with non-UK PayPal accounts but happens less frequently now.
For any others investigating this issue please be aware that PayPal has filters in place that looks for globally specific geographic terms. If any terms used in the product description trigger the filters, the transaction could be refused. In our case, we are online rug retailer with many of our products ranging across the traditional rug spectrum. We found that any rug with the term "Persia" in the title were being refused due to economic sanctions against nation states associated with that term.
The filters are not geographic only and other controversial terms could also trigger the filters to refuse the transaction. In one instance, we had a rug with "Confederate" in the product description that also triggered the filters.
Trying to work out what to do in regard to a redirect of a new client real estate website.
We have no access at all to the old site and the url structure on new is forcibly different due to randomly generated property IDs (our system generates a different ID from old)
The old url structure is www.mydomain.com/property/view?=1111
The new url structure is www.mydomain.com/property/street-name/2222
My instinct is to do manual 301s for every property (about 6000), matching by page title, but sadly I cant as I have no access to the structure of the old website and despite spidering it numerous times I cant get a pull of all properties off.
If any could give me any advice on what best to do to avoid bad user experience and a google frying, would really appreciate it.
Thanks in advance.
Mark
It depends on what the 1111 is. If that corresponds to an MLS ID number (some sort of UID) then you should be able to use regex to get it to work. Most of the IDX vendors offer a way to grab listings via an MLSID.
If that 1111 is instead just a GUID of the previous IDX vendor, then you might be out of luck and would need to do everything manually.
In my community, every user should only have one account.
So I need a solution to verify that the specific account is the only one the user owns. For the time being, I use email verification. But I don't really need the users' email adresses. I just try to prevent multiple accounts per person.
But this doesn't work, of course. People create temporary email addresses or they own several addresses, anyway. So they register using different email addresses and so they get more than one account - which is not allowed.
So I need a better solution than the (easy to circumvent) email verification. By the way, I do not want to use OpenID, Facebook Connect etc.
The requirements:
verification method must be accessible for all users
there should be no costs for the user (at least 1$)
the verification has to be safe (safer than the email approach)
the user should not be demanded to expose too much private details
...
Do you have ideas for good approaches? Thank you very much in advance!
Additional information:
My community is a browser game, namely a soccer manager game. The thing which makes multiple accounts attractive is that users can trade their players. So if you have two accounts, you can buy weak players for excessive prices which no "real" buyer would pay. So your "first account" gets huge amounts of money while the "second account" becomes poor. But you don't have to care: Just create another account to make the first one richer.
You should ask for something more unique than an email. But there is no way to be absolutly sure a player don't own two account.
The IP solution is not a solution, as people playing from a compagny/school/3G will have the same IP. Also, Changing IP is easy (reset the router, proxy, use your 3G vs wifi)
Some web site (job-offer, ...) ask you for an official ID number (ID, passport, social security, driver licence, visa (without the security number, so peolple will feel safe that you won't charge them), ...)
This solution got a few draw back:
minor don't always have an ID / visa
pepole don't like to give away this kind of info. (in fact, depending where you live: in spain for example, it is very common to ask for ID number)
people own more than one visa.
it is possible to generate valide ID/visa number.
Alternative way:
ask for a fee of 1$
to be allow to trade more than X players / spend more than X money.
people that pay the fee got some advantage : less ads, extra players, ...
paying a fee, will limitate creation of multiple account.
fee can be payed using taxed phone number (some compagny provide international system)
the payment medium could be use as an ID (visa number)
put some restriction in new account (like SO).
eg: "you have to play at least 1 hour before trading a player"
eg: "you have to play at least 3 hour before trading more than 3 players"
Use logic to detect multiple account
use cookie to detect multiple account
check last connection time of both player before a transaction. (if player A logout 1 minute before player B login : somethings is going on)
My recommandation :
Use a mix of all thoses methode, but keep the user experience fluide without "form to fill now to continue"
Very interesting question! The basic problem here is multi-part -
Opening an account is trivial (because creating new email IDs is trivial).
But the effect of opening an account in the game is NOT trivial. Opening a new account basically gives you a certain sum of money with which to buy players.
Transferring money to another account is trivial (by trading players).
Combining 1 & 2, you have the problem that new players have an unfair advantage (which they would not have in the real world). This is probably okay, as it drives new users to your site.
However adding 3 to the mix, you have the problem that new players are easily able to transfer their advantage to the old players. This allows old users to game the system, ruining fun for others.
The solution can be removing either 1,2,3.
Remove 1 - This is the part you are focusing on. As others have suggested, this is impossible to do with 100% accuracy. But there are ways that will be good enough, depending on how stringent your criterion for "good enough" is. I think the best compromise is to ask the user for their mobile phone numbers. It's effective and allows you to contact your users in one more way. Another way would be to make your service "invite only" - assuring that there is a well defined "trail" of invites that can uniquely identify users.
Remove 2 - No one has suggested this which is a bit surprising. Don't give new users a bunch of money just for signing up! Make them work for it, similar to raising seed capital in the real world. Does your soccer simulation have social aspects? How about only giving the users money once their "friend" count goes above a certain number (increasing the number of potential investors who will give them money)?
Remove 3 - Someone else has already posted the best solution for this. Adopt an SO like strategy where a new user has to play for 3 hours before they are allowed to transfer players. Or maybe add a "training" stage to your game which forces a new player to prove their worth by making enough money in a simulated environment before they are allowed to play with the real users.
Or any combination of the above! Combined with heuristics like matching IP addresses and looking for suspicious transactions, it is possible to make cheating on the game completely unviable.
Of course a final thing you need to keep in mind is that it is just a game. If someone goes to a lot of trouble just to gain a little bit of advantage in your simulation, they probably deserve to keep it. As long as everyone is having fun!
I know this is probably nothing you have expected, but...
My suggestion would be to discourage people from creating another account by offering some bonus values if they use the same account for a longer period, a kind of loyalty program. For some reason using a new account gives some advantages. Let's eliminate them. There are a lot of smart people here, so if you share more details on the advantages someone could come up with some idea. I am fully convinced this is on-topic on SO though.
We have implemented this by hiding the registration form. Our customers only see the login form where we use their mobile number as username and send the password by text message.
The backend systems match the mobile number to our master customer database which enforces that the mobile number is unique.
Here is an idea:
Store UUID in a cookie at clients. Each user login store the UUID from Cookie in relation to the account entity in the databse.
Do the same with the IP adresses instead of UUID.
After that write a program interface for your game masters that:
Show up different account names but same IP (within last x hours)
Show up different account names but same UUID (nevertheless how long ago)
Highlight datasets from the two point above where actions (like player transfers) happened which can be abused by using multiple accounts
I do not think you should solve that problem by preventing people having two or more accounts. This is not possible and ineffective. Make it easier to find that evil activities and (automatically temporarly) ban these people.
It's impossible to accomplish this with a program.
The closest you can do is to check the ip address. But it can change, and proxies exist.
Then you could get the computer MAC address, but a network card can be changed. And a computer too.
Then, there is one way to do this, but you need to see the people face to face. Hand them a piece of paper with a unique code. They can only subscribe if they have the code.
The most effective solution might be the use of keystroke biometrics. A person can be identified by the way the person writes a sentence.
This company provides a product which can be used to implement your requirements: http://www.psylock.com/en
I think 1 account per email address should be good enough for your needs. After all, account verification doesn't have to end right after signup.
You can publish the IP address of the computer each message was posted from to help your users detect when someone is using multiple accounts from the same computer, and you can use a ranking system to discourage people from using temporary accounts.
Do your game dynamics allow for you to require that both users be online for a trade to occur? If so, you can verify the IP addresses of both users involved in a trade, which would be the same unless the user was paying for multiple internet connections and accessing two accounts from separate machines.
Address the exact scenario that you're saying is a problem.
Keep track of the expected/fair trade value of players and prevent blatantly lope-sided trades, esp. for new accounts. Assume the vast majority of users in your system are non-cheaters.
You can also do things like trickle in funds/points for non-trading actions/automatically overtime, etc.
Have them enter their phone number and send a text message to it. Then, keep a unique of all the cell phone numbers. Most people have one cell phone, and aren't going to ask their friend to borrow it just to create a second account.
http://en.wikipedia.org/wiki/List_of_SMS_gateways
I would suggest an approach using two initiatives:
1) Don't allow brand new accounts to perform trades. Accounts must go through a waiting period and prove that the account is legitimate by performing some non-trade actions.
2) Publicize the fact that cheaters will be disqualified and punished. Periodically perform searches for accounts being used to dump bad players and investigate. Ban/disqualify cheaters and publicize the bans so that people know the rules are being enforced.
No method would be foolproof but the threat of punishment should minimize cheating.
actually you can use fingerprintjs to track every user, use js encrypt the fingerprint in browser and decrypt in server
What would be best practice for the following situation. I have an ecommerce store that pulls down inventory levels from a distributor. Should the site, for everytime a user loads a product detail page use the third party API for the most up to date data? Or, should the site using third party APIs and then store that data for a certain amount of time in it's own system and update it periodically?
To me it seems obvious that it should be updated everytime the product detail page is loaded but what about high traffic ecommerce stores? Are completely different solutions used for that case?
In this case I would definitely cache the results from the distributor's site for some period of time, rather than hitting them every time you get a request. However, I would not simply use a blanket 5 minute or 30 minute timeout for all cache entries. Instead, I would use some heuristics. If possible, for instance if your application is written in a language like Python, you could attach a simple script to every product which implements the timeout.
This way, if it is an item that is requested infrequently, or one that has a large amount in stock, you could cache for a longer time.
if product.popularityrating > 8 or product.lastqtyinstock < 20:
cache.expire(productnum)
distributor.checkstock(productnum)
This gives you flexibility that you can call on if you need it. Initially, you can set all the rules to something like:
cache.expireover("3m",productnum)
distributor.checkstock(productnum)
In actual fact, the script would probably not include the checkstock function call because that would be in the main app, but it is included here for context. If python seems too heavyweiaght to include just for this small amount of flexibilty, then have a look at TCL which was specifically designed for this type of job. Both can be embedded easily in C, C++, C# and Java applications.
Actually, there is another solution. Your distributor keeps the product catalog on their servers and gives you access to it via Open Catalog Interface. When a user wants to make an order he gets redirected in-place to the distributor's catalog, chooses items then transfers selection back to your shop.
It is widely used in SRM (Supplier Relationship Management) branch.
It depends on many factors: the traffic to your site, how often the inventory levels change, the business impact of displaing outdated data, how often the supplers allow you to call their API, their API's SLA in terms of availability and performance, and so on.
Once you have these answers, there are of course many possibilities here. For example, for a low-traffic site where getting the inventory right is important, you may want to call the 3rd-party API on every call, but revert to some alternative behavior (such as using cached data) if the API does not respond within a certain timeout.
Sometimes, well-designed APIs will include hints as to the validity period of the data. For example, some REST-over-HTTP APIs support various HTTP Cache control headers that can be used to specify a validity period, or to only retrieve data if it has changed since last request.