I use quicken, which can automatically download bank of America transactions. However, it truncates all the payees so I lose data. I'd like to work around this and I'm thinking of downloading the transaction data and generating my own QFX file with the full payee info.
Is there a way that I can download transactions programmatically, or download something like a .qif (available on their website) programmatically? For the latter, I could convert the gif to a QFX myself.
If anyone has other ideas to download all of the transaction information without losing the payee info, I would welcome those ideas as well.
Do they provide an api for this? but most probably not for 3p without a contract. since its bank , there must be check for browser etc along with standard sign in so it'll hard for curl. you can have a browser plugin to read all the data from the page and do auto scroll to get new transactions if not fitting in page. it's a hacky solution but good to get what we need as you told that data is available on the page and have to revisit with updates but changes in basic structure is rare.
A quick search for bank of america api yielded this BofA API. They even have many options for types of payment information you could query here as well as lots of individual account types that you can access it as.
It looks pretty comprehensive. If you don't see what you are looking for there I put another option below, just in case.
I don't use BofA. So I can't speak to what they have natively available. But you could always use a bot to scrape it if they present it anywhere in the User Interface.
I would agree with Meena that you should not be able to use curl. But selenium uses a browser to programmatically do just about anything that you would want to do with any website. They also have bindings for many languages. So you could just pick your favorite and go to town...
It seems the API will return a JSON so you may need to find a tool to convert that to a qif or qfx if that part is important. After digging further, I can't test this without having a CashPro account but it seems what you need to do is...
Step 1:
Get an access token from here. You'll need to send this in the header of any requests
Step 2:
Send an http request with a header in the following format:
{
"accounts": [
{
"accountNumber": "xxxxxxx",
"bankId": "xxxxxxx"
}
],
"fromDate": "yyyy-mm-dd",
"toDate": "yyyy-mm-dd"
}
to https://developer.bankofamerica.com/cashpro/reporting/v1/transaction-inquiries/previous-day
Step 3:
You should get a JSON as a response
As mentioned, I can't test this but here's the documentation of the specific API endpoint you need
I've come to the conclusion that building a truly RESTful API, one the uses HATEOAS is next to impossible.
Every content I've come across either fails to illustrate the true power of HATEOAS
or simply does not explicitly mentions the inherent pain points with the dynamic nature of HATEOAS.
What I believe HATEOAS is all about:
From my understanding, a truly HATEOAS API should have ALL the information needed to interact with the API, and while that is possible, it is a nightmare to use especially with different stacks.
For example, consider a collection of resources located at "/books":
{
"items": [
{
"self": "/book/sdgr345",
"id": "sdgr345",
"name": "Building a RESTful API - The unspoken truth",
"author": "Elad Chen ;)",
"published": 1607606637049000
}
],
// This describes every field needed to create a new book
// just like HyperText Markup Language (i.e. HTML) rendered on the server does with forms
"create-form": {
"href": "/books",
"method": "POST",
"rel": ["create-form"],
"accept": ["application/x-www-form-urlencoded"],
"fields": [
{ "name": "name", "label": "Name", "type": "text", "max-length": "255", "required": true }
{ "name": "author", "label": "Author", "type": "text", "max-length": "255", "required": true }
{ "name": "author", "label": "Publish Date", "type": "date", "format": "dd/mm/YY", "required": true }
]
}
}
Giving the above response, a client (such as a web app) can use the "create-form" property to render an actual HTML form.
What value do we get from all this work?
The same value we've been getting from HTML for years.
Think about it, this is exactly what hypertext is all about, and what HTML has been designed for.
When a browser hits "www.pizza.com" the browser has no knowledge of the other paths that a user
can visit, it does not concatenate strings to produce a link to the order page -> "www.pizza.com/order", it simply renders anchors
and navigates when a user clicks them. This is what allows web developers to change the path from "/order" to "/shut-up-and-take-my-money" without changing any client (browsers).
The above idea is also true for forms, browsers do not guess the parameters needed to order a pizza, they simply render a form and its inputs, and handle its submission.
I have seen too many lines of codes in front-ends and back-ends alike, that build strings
like -> "https://api.com" + "/order" - You don't see browsers do that, right?
The problems with HATEOAS
Giving the above example ("/books" response), in order to create a new book, clients are expected to parse the response in order to leverage the true power of this RESTful API, otherwise, they risk assuming what the names of the fields are, which of them is required, what their expected type is, etc...
Now consider having two clients within your company that are using this API,
one for the web (browsers) written in JS, and another for the mobile (say an android app) written in Java. They can be published as SDK's, hopefully making 3 party consumers have an easier integration.
As soon as the API is used by clients outside your control, say a 3rd party developer with an affinity to python, with the purpose of creating a new book.
That developer is REQUIRED to parse such a response, to figure out what the parameters are, their name, the URL to send inputs to, and so on.
In all my years of developing I have yet to come across an API such as the one I have in mind.
I have a feeling this type of API is nothing more than a pipe dream, and I was hoping to understand whether my assumptions are correct, and what downfalls it brings before starting the implementation phase.
P.S
in case it's not clear, this is exactly what HATEOAS compliant API is all about - when the fields to create a book change clients adapt without breaking.
On the Hypermedia Maturity Model (HMM), the example you give is at Level 0. At this level, you are absolutely correct about the problems with this approach. It's a lot of work and developers are probably going to ignore it and hard-code things anyway. However, with a generic hypermedia enabled media type, not only does all that extra work go away, it actually reduces the work for developers.
Let's take a step back for a moment and consider how the web works. There are three main components: the web server, the web browser, and the driver (usually a human user). The web server provides HTML, which the web browser executes to present a graphical user interface, which the driver can use to follow links and fill out forms. The browser uses the HTML to completely abstract from the driver all the details about how to present the form and how to send it over HTTP.
In the API world, this concept of the generic browser that abstracts away the media type and HTTP details hasn't taken hold yet. The only one I know about that is both active and high quality is Ketting. Using a browser like Ketting, removes all of that extra work the developer would have to put into making use of all that hypermedia. A hypermedia browser is like an SDK that API vendors often provide except that it works for any API. In theory, you can link from one API to another completely unrelated API. APIs would no longer be islands, they would become a web.
The thing that makes hypermedia browsers possible are general purpose hypermedia enabled media types. HTML is of course the most successful and famous example, but there are JSON based media types as well. Some of the more widely used examples are HAL and Siren.
The higher level a media type is on the Hypermedia Maturity Model, the more that a generic browser can do to abstract way the media-type, URIs, and HTTP details. Here's a brief explanation. Checkout the blog post linked above for more details and examples.
Level 0: At this level, hypermedia is encoded in an ad-hoc way. A browser can't do much with this because every API might encode things a little different. At best a browser can use heuristics or AI to guess that something is a link or a form and treat it as such, but generally HMM Level 0 media-types are intended for developers to read an interpret. This leads to many of the challenges you identified your question. Examples: JSON and XML.
Level 1: At this level, links are a first class feature. The media type has a well defined way to represent a link. A browser knows unambiguously what is to be interpreted as a link and can provide an interface to follow that link without the user needing to be concerned about URI or HTTP. This is sufficient for read-only APIs, but if we need the user to provide input, we don't have a way to represent a form-like hypermedia control. It's up to a human to read the documentation or an ad-hoc form representation to know how to submit data. Examples: HAL, RESTful JSON.
Level 2: At this level, forms (or form-like controls) are a first class feature. A media type has a well defined way of representing a form-like hypermedia control. A browser can use this media type to do things like build an HTML Form, validate user input, encode user input into an acceptable media type, and make an HTTP request using the appropriate HTTP method. Let's say you want to change your API to support PATCH and you prefer that application start using it over PUT. If you are using a HMM Level 2 media type, you can change method in the representation and any application that uses a hypermedia browser that knows how to construct a PATCH request will start sending PATCHs instead of PUTs without any developer intervention. Without a HMM Level 2 media type, you're stuck with those PUTs until you can get all the applications that use your API to update their code. Examples: HTML, HAL-Forms, Siren, JSON Hyper-Schema, Uber, Mason, Collection+JSON.
Level 3: At this level, in addition to hypermedia controls, data are also self-describing. Remember those three main components I mentioned? The last one, "driver", is the major difference between using hypermedia on the web and using hypermedia in an API. On the web, the driver is a human (excluding crawlers for simplicity), but with an API, the driver is an application. Humans can interpret the meaning of what they are presented with and deal with changes. Applications may act on heuristics or even AI, but usually they are following a fixed routine. If something changes about the data that the application didn't expect, the application breaks. At this level, we apply semantics to the data using something like JSON-LD. This allows us to construct drivers that are better at dealing with change and can even make decisions without human intervention. Examples: Hydra.
I think the only downside to choosing to use hypermedia in your API is that there aren't production-ready HMM Level 2 hypermedia browsers available in most languages. But, the good news is that one implementation will cover any API that uses a media type it supports. Ketting will work for any API and application that is written in JavaScript. It would only take a few more similar implementations to cover all the major languages and choosing to use hypermedia would be an easy choice.
The other reason to choose hypermedia is that it helps with the API design process. I personally use JSON Hyper-Schema to rapid-prototype APIs and use a generic web client to click through links and forms to get a feel for API workflows. Even if no one else uses the schemas, for me, it's worth having even if just for the design phase.
Implementing a HATEOAS API needs to be done both on the server and on the clients, so this point you make in the comments is very valid indeed:
changing a resource URI is risky given I don't believe clients actually "navigate" the API
Besides the World Wide Web, which is the best implementation of HATEOAS, I only know of the SunCloud API on the now decommissioned Project Kenai. Most APIs out there don't quite make use of HATEOAS, but are just a bunch of documented URLs where you can get or submit specific resources (basically, instead of being "hypermedia driven", they are "documentation driven"). With this type of API, clients don't actually navigate the API, they concatenate strings to go at specific endpoints where they know they can find specific resources.
If you expose a HATEOAS API, developers of any clients can still look at the links you return and may decide to build them on their own 'cause they figure what the API is doing, so then think they can just bypass any other navigation that might be needed and go straight for the URL, because it is always /products/categories/123, until - of course - it isn't anymore.
A HATEOAS API is more difficult to build and adds complexity to both the server and the clients, so when deciding to build one, the questions are:
do you need the flexibility HATEOAS is giving you to justify the extra complexity of the implementation?
do you want to make it easier or harder for clients to consume your API?
does the knowledge and discipline to make it all work exist on both sides (server developers and clients developers)?
Most of the times, the answer is no. In addition, many times the questions don't even get asked, instead people end up with the approach that is more familiar because they've seen the sort of APIs people build, have used some, or built some before. Also, many times, a REST API is just stuck in front of a database and doesn't do much but expose data from the database as JSON or XML, so not that much need for navigation there.
There is no one forcing you to implement HATEOAS in your API, and no one preventing you to do so either. At the end of the day, it's a matter of deciding if you want to expose your API this way or not (another example would be, do you expose your resources as JSON, XML, or let the client chose the content type?).
In the end, there is always the chance of breaking your clients when you make changes to your API (HATEOAS or no HATEOAS), because you don't control your clients and you can't control how knowledgeable the client developer are, how disciplined, or how good of a work they do in implementing someone else's API.
I have an API with endpoint GET /users/{id} which returns a User object. The User object can contain sensitive fields such as cardLast4, cardBrand, etc.
{
firstName: ...,
lastName: ...,
cardLast4: ...,
cardBrand: ...
}
If the user calls that endpoint with their own ID, all fields should be visible. However, if it is someone elses ID then cardLast4 and cardBrand should be hidden.
I want to know what are the best practices here for designing my response. I see three options:
Option 1. Two DTOs, one with all fields and one without the hidden fields:
// OtherUserDTO
{
firstName: ...,
lastName: ..., // cardLast4 and cardBrand hidden
}
I can see this becoming out of hand with DTOs based on role, what if now I have UserDTOForAdminRole, UserDTOForAccountingRole, etc... It looks like it quickly gets out of hand with the number of potential DTOs.
Option 2. One response object being the User, but null out the values that the user should not be able to see.
{
firstName: ...,
lastName: ...,
cardLast4: null, // hidden
cardBrand: null // hidden
}
Option 3. Create another endpoint such as /payment-methods?userId={userId} even though PaymentMethod is not an entity in my database. This will now require 2 api calls to get all the data. If the userId is not their own, it will return 403 forbidden.
{
cardLast4: ...,
cardBrand: ...
}
What are the best practices here?
You're gonna get different opinions about this, but I feel that doing a GET request on some endpoint, and getting a different shape of data depending on the authorization status can be confusing.
So I would be tempted, if it's reasonable to do this, to expose the privileged data via a secondary endpoint. Either by just exposing the private properties there, or by having 2 distinct endpoints, one with the unprivileged data and a second that repeats the data + the new private properties.
I tend to go for option 1 here, because an API endpoint is not just a means to get data. The URI is an identity, so I would want /users/123 to mean the same thing everywhere, and have a second /users/123/secret-properties
I have an API with endpoint GET /users/{id} which returns a User object.
In general, it may help to reframe your thinking -- resources in REST are generalizations of documents (think "web pages"), not generalizations of objects. "HTTP is an application protocol whose application domain is the transfer of documents over a network" -- Jim Webber, 2011
If the user calls that endpoint with their own ID, all fields should be visible. However, if it is someone elses ID then cardLast4 and cardBrand should be hidden.
Big picture view: in HTTP, you've got a bit of tension between privacy (only show documents with sensitive information to people allowed access) and caching (save bandwidth and server pressure by using copies of documents to satisfy more than one request).
Cache is an important architectural constraint in the REST architectural style; that's the bit that puts the "web scale" in the world wide web.
OK, good news first -- HTTP has special rules for caching web requests with Authorization headers. Unless you deliberately opt-in to allowing the responses to be re-used, you don't have to worry the caching.
Treating the two different views as two different documents, with different identifiers, makes almost everything easier -- the public documents are available to the public, the sensitive documents are locked down, operators looking at traffic in the log can distinguish the two different views because the logged identifier is different, and so on.
The thing that isn't easier: the case where someone is editing (POST/PUT/PATCH) one document and expecting to see the changes appear in the other. Cache-invalidation is one of the two hard problems in computer science. HTTP doesn't have a general purpose mechanism that allows the origin server to mark arbitrary documents for invalidation - successful unsafe requests will invalidate the effective-target-uri, the Location, the Content-Location, and that's it... and all three of those values have other important uses, making them more challenging to game.
Documents with different absolute-uri are different documents, and those documents, once copied from the origin server, can get out of sync.
This is the option I would normally choose - a client looking at cached copies of a document isn't seeing changes made by the server
OK, you decide that you don't like those trade offs. Can we do it with just one resource identifier? You immediately lose some clarity in your general purpose logs, but perhaps a bespoke logging system will get you past that.
You probably also have to dump public caching at this point. The only general purpose header that changes between the user allowed to look at the sensitive information and the user who isn't? That's the authorization header, and there's no "Vary" mechanism on authorization.
You've also got something of a challenge for the user who is making changes to the sensitive copy, but wants to now review the public copy (to make sure nothing leaked? or to make sure that the publicly visible changes took hold?)
There's no general purpose header for "show me the public version", so either you need to use a non standard header (which general purpose components will ignore), or you need to try standardizing something and then driving adoption by the implementors of general purpose components. It's doable (PATCH happened, after all) but it's a lot of work.
The other trick you can try is to play games with Content-Type and the Accept header -- perhaps clients use something normal for the public version (ex application/json), and a specialized type for the sensitive version (application/prs.example-sensitive+json).
That would allow the origin server to use the Vary header to indicate that the response is only suitable if the same accept headers are used.
Once again, general purpose components aren't going to know about your bespoke content-type, and are never going to ask for it.
The standardization route really isn't going to help you here, because the thing you really need is that clients discriminate between the two modes, where general purpose components today are trying to use that channel to advertise all of the standardized representations that they can handle.
I don't think this actually gets you anywhere that you can't fake more easily with a bespoke header.
REST leans heavily into the idea of using readily standardizable forms; if you think this is a general problem that could potentially apply to all resources in the world, then a header is the right way to go. So a reasonable approach would be to try a custom header, and get a bunch of experience with it, then try writing something up and getting everybody to buy in.
If you want something that just works with the out of the box web that we have today, use two different URI and move on to solving important problems.
I added a new type of image format 3d in my commercewebservices-spring.xml.
I want to access this new format in my product-images.component in Product Detail page, something like:
<ng-container *ngIf="mainImage$ | async as main">
<div *ngIf="main.3d as 3d">
But I get an 'unresolved variable', so I guess I need to add this new format elsewhere in spartacus or there is something I'm missing.
Could anyone help with this? Thanks a lot in advance.
there are a couple of steps you'd need to do:
Make sure that the image has been exposed in the OCC api. The OCC endpoints has a configurable fields mapping, that you can use to control the response. You need to verify if your 3d property is exposed. I'm also not sure if you add this as a new property to the backend product model and if it's a media container or a media. But I assume you figure that out.
Make the Spartacus adapter aware of the new field mapping, this is documented in https://sap.github.io/spartacus-docs/connecting-to-other-systems/#configuring-endpoints. You basically configure the required field, so that the data is requested and automatically stored inside the product state in Spartacus.
Implement the UI. I'm not sure if you like to add the 3d image as part of other thumbs/images, or that you need a dedicated UI for 3d images. If it's completely replacement of the product images, the easiest thing is to replace the current cms ProductImagesComponent mapping, i.e.:
providers: [
provideDefaultConfig(<CmsConfig>{
cmsComponents: {
ProductImagesComponent: {
component: Custom3dImagesComponent
}
}
})
]
This is documented at https://sap.github.io/spartacus-docs/customizing-cms-components/
There are alternatives for each step, but these are most straightforward.
If you look at ProductImageNormalizer (https://sap.github.io/spartacus/injectables/ProductImageNormalizer.html#source) you can see that the mapping is slightly different: either image.primary.<format> or image.gallery[n].<format>
Also, bear in mind that the format codes used in the DTO are mapped in the SAP Commerce side. There is an ImageFormatMapping concept, which maps from the actual media format used in the data model in SAP Commerce to the format strings used in the DTO.
I'm relatively experienced with Angular having written many directives, but I have a new requirement where I have to build a query-by-example form into which a user can enter different search criteria. My problem is that I do not know ahead of time what the possible criteria will be. This criteria information will be coming from the server via an ajax request and can differ per user. Thus I will need to dynamically construct a suitable user interface based on the information I get from the server.
I have built individual directives suitable for capturing the search criteria (for example a custom calendar control for date criteria) but I am unsure of the best approach to adding these directives to a form dynamically. Is this even possible in Angular?
I have built something like this before in jQuery but its not so clear to me how I would best do this in an 'Angular way'?
Any suggestions would be most appreciated!
Everything that you can express as a model-to-view projection can be implemented in AngularJS.
I think here you can make a model consisting of "query params". Each of them has name, type and data for filter builder. For example, for the "select" type a data can contain a list of all possible values to choose from.
Then you iterate through the list of "query params" with ng-repeat, rendering each control differently according to its type. That's all.
If I understood the task wrong, please provide more info.