I'm looking at using traffic mirroring with Istio to dark test releases.
The mirror'd traffic will mean write APIs like order & payment, etc are called multiple times which I don't want else I'll be charging the customer twice & sending them a duplicate product.
Is there a standard way to prevent this (stubbing seems an odd thing to do in production) or is mirroring only really applicable for read APIs?
Issue
There is diagram of mirroring setup wih traffic flows.
Although these mirror requests are mirrored as “fire and forget” and the reply from the mirror service is just dropped (by the envoy proxy sidecar) to /dev/null and not returned to the caller, it's still uses this api.
Solution
As mentioned in comments
In my opinion you should add path for your testing purposes with some custom header, so this could be tested only by you or your organization, and the customer shouldn't be involved in that.
This topic is described in detail here by Christian Posta.
When we deploy a new version of our service and mirror traffic to the test cluster, we need to be mindful of impact on the rest of the environment. Our service will typically need to collaborate with other services (query for data, update data, etc). This may not be a problem if the collaboration with other services is simply reads or GET requests and those collaborators are able to take on additional load. But if our services mutates data in our collaborators, we need to make sure those calls get directed to test doubles and not the real production traffic.
There are a few approaches you may consider, all of them are described in the link above:
Stubbing out collaborating services for certain test profiles
Synthetic transactions
Virtualizing the test-cluster’s database
Materializing the test-cluster’s database
In practice, mirroring production traffic to our test cluster (whether that cluster exists in production or in non-production environments) is a very powerful way to reduce the risk of new deployments. Big webops companies like Twitter and Amazon have been doing this for years. There are some challenges that come along with this approach, but there exist decent solutions as discussed in the patterns above.
Related
I have currently delivered a SPA (Vue.js) web application with a Java API backend. Everything is currently sitting in AWS, with the frontend being in CloudFront and the backend in ECS connecting to a RDS instance.
As part of the next phase of delivery, we are creating a B2B API. My question is that of architectural design and deployment strategy, is it commonplace to just extend the existing API with B2B functionality? Should I keep them both separate with an API gateway in front? We envisage that the B2B use eventually will outgrow the SPA use case so the initial deployment configuration needs to have the most flexibility to grow and expand.
Is there some sort of best practice here? I imagine that a lot of code would be similar between the two backends as well.
Thanks,
Terry
First off - Deciding on service boundaries is one of the most difficult problems in a service oriented architecture design and the answer strongly depends on your exact domain requirements.
Usually I would split service implementations by the domain/function as well as by organizational concerns (e.g. separate teams developing them) and not by their target audience. This usually avoids awkward situation where team responsibility is not clear, etc. If it will grow into a very large project there may also be a need for multiple layers of services and shared libraries - And at that point you would likely run into necessary re-factorings / restructurings.
So if there is a very large overlap in function between your b2b and the regular api you may not want to split the implementation.
However, you may also have to consider how the service access is provided and an API Gateway could help with providing different endpoints for the different audiences, different charging models, different auth options, etc. Depending on your exact requirements an API Gateway may not be enough and this may also require another thin service layer implementation that uses common domain services.
What considerations are needed when creating a web app that is intended to be used in an industrial plant setting for a company? My specific use case is an industrial facility with several different production plants that would each have its own device for the application interface.
How do companies enforce the usage of such apps on a monitor/tablet? For example, could I prevent them from using other stuff on the tablet?
Importantly, how would security work? They'd share a device. There may be multiple operators that use the app in a given shift. Would they all use the same authentication session (this is not preferable, as I'd like to uniquely identify the active user)? Obviously I could use standard username/passwords with token based sessions that expire, however, this leaves a lot of potential for account hijacking. Ideally, they'd be able to log on very quickly (PIN, perhaps?) and their session would end when they are done.
As long as there is internet connection, I would presume that there isn't much pro/con regarding the use of native applications versus web based or progressive web apps. Is this assumption correct?
What's the best way of identifying which device the application is being run on?
Is this a common thing to do in general? What other technologies are used to create software that obtains input from industrial operators?
--
Update - this is a good higher level consideration of the question at hand, however, it has become apparent why focused, specific questions are helpful. As such, I will follow up with questions that are specific.
Identifying the Area/Device a Web Application is Accessed On
Enforcing Specific Application Use on Tablets
Best Practices for Web App Authentication in Industrial Settings
I'm not able to answer everything in great detail but here are a few pointers. In the environment as you describe we usually see these two options. 1) you tell them what you need, internet, security, if they give you device and how it will be configured 2) they tell you exactly what you need to deliver.
I do not think you can 100% prevent them. We did it by providing the tablet( well laptops in our case) and the OS configuration took care of that, downside we had few devices to support. You seem to hint that there is always an internet connection so I guess you can collect all info about the system and send it back to you daily?
We were allowed to "tap" into their attendance SW and when you entered the facility you were able to use your 4 digit pin to log in if you were out of premisses you could not log in at all. I can imagine the following: you log in with your username and password - this does full verification, after that, you can use 4 digit pin to login for next n hours.
maybe, kinda, depends on what you are doing. Does the browser have all features you need? Our system needs multicast to perform really fast, so we have a native app
touched on this in 1. You could also use device enrolment process. You can also contractually force them that there will be only your software and it may invalidate support contract. It really depends on your creativity. My favourite( and it works - just tell them, there will only be installed my software and if not you will pay me double for support. I only saw one customer who installed some crap on the device when there were told not to
it really depends on what industry you are talking about, every industry is different. We almost always build a custom solution
The enforcement of the device/app usage depends on the customer, if the customer asked for help in the enforcement, then you can provide guide, training and workshops. If the customer serious about the enforcement then it will be a policy that's adapted by all the organization from top to down. Usually seniors will resist a workflow change more than juniors, so top management/executive should deal with that. Real life story: SAP team took 6 months to transform major newspaper workflow, during that few seniors got fired because they refuse to adapt the change.
Security shouldn't handicap the users, usually in industrial environment the network is isolated or at least restricted through VPN to connect multiple sites (plants in your case), regarding the active user: we usually provide guide/training/workshop for the users and inform them that using colleague account or device will prevent the system from tracking your accomplishment/tasks, so each user is responsible to make sure the active account/device is the one assigned to him/her.
It depends, with native you have more controls than web, but if the app is just doing monitoring then most of today apps use web for monitoring and the common way to receive input is REST APIs (even if the industrial devices doesn't support REST API, a middleware could be written to transform the output). If you need more depth about native vs web you need to ask new question with more details about the requirements.
Depends on the tech you are using (native or web), and things I mentioned in point 2: you can use whitelist of devices that's allowed to run the app. overall there are many best ways to track down the device.
How common in general? I think such information can only be achieved by survey, the world full of variations. And having something common not mean its safe or best, our industry keep changing at all levels. So to stay in the loop, we must keep learning and self-updating without reboot.
The main reason for this question is to understand/reasons behind the best practices over the usage of system APIs. If the System API itself good enough to be serve the purpose of my client application, do we still need to write an experience API to invoke the system API indirectly, or break the rule, just invoke the system-API directly from the client application. As sometimes , it is overhead/numerous API calls over the network.
System API is to unlock or expose the system asset(back end data). Now, one could write the system API in such a way that it fetches the data from system database, does the required processing, for instance convert the table rows to JSON format and then does some enrichment & trimming of fields and expose it to the customer A. This is a course grained approach. Now, another customer B requires similar data but needs some fields that were already trimmed by you to serve Customer A who wanted only few fields of the many fields that you picked from System(database). You'll have to write a separate course grained API for Customer B.
Also, in the future if the backend SYSTEM is replaced with a new SYSTEM, one would have to re-write/update both the API's for each customer A and customer B.
This course grained approach would solve your problem each time, but architecturally having a fine grained approach of breaking down a large service into multiple layers of experience, process and system API's would enable re-use, reduce work effort, increase time to market, lower total cost of ownership and allow applying the required separate policies(Security, sla's etc) for each of the clients through experience API layer. You can now better scale your integration landscape.
A fine grained approach increases usage of resources such as network, diskspace(more logging) etc but its a trade-off to all the many advantages you get. Again, the decision to go with either of the approaches should align with the current circumstances of your ecosystem, so it all depends.
We are e-commerce website with several listings. The listings can be created/updated/deleted from various apps:- i.e. desktop, mobile, apps.
We need to push all this updated information to some third party APIs i.e. any new listing created/deleted or if the existing listing updates, we need to push complete listing information through some third party API. We are using Rabbitmq for this since we expect a high volume of record update.
We have two choices:
Push from all endpoints (desk/msite/app) info like (listingId, Action on listing i.e. CREATE/UPDATE/DELETE) to rabbitmq. Rabbitmq further dequeue these messages and hit appropriate API.
Implement trigger on listings table i.e. on create, insert entry into some table with column (listingId, database action to be executed i.e. CREATE/UPDATE/DELETE). Now, create a job toread from this table after every 10 sec. and push these to rabbitmq.
Which is better appraoch?
I think an HTTP based API might be the best solution. You can implement a gateway which includes security (OAuth2/SAML), rate limiting etc. Internally you can use RabbitMQ. The gateway can publish the updates to RabbitMQ and have subscribers which write the data to your master database and other subscribers that publish the data to your third party APIs.
The added benefit of an HTTP gateway, beyond the extra security controls available, is that you could change your mind about RabbitMQ in the future without impacting your desktop and mobile apps which would be difficult to update in their entirety.
Having worked with databases for most of my career, I tend to avoid triggers. Especially if you expect large volumes of updates. I have experienced performance problems and reliability problems with triggers in the past.
It is worth noting that RabbitMQ does have message duplication problems - there is no exactly once delivery guarantee. So you will need to implement your own deduplication or preferably make all actions idempotent.
My current employer uses a 3rd party hosted CRM provider and we have a fairly sophisticated integration tier between the two systems. Amongst the capabilities of the CRM provider is for developers to author business logic in a Java like language and on events such as the user clicking a button or submitting a new account into the system, have validation and/or business logic fire off.
One of the capabilities that we make use of is for that business code running on the hosted provider to invoke web services that we host. The canonical example is a sales rep entering in a new sales lead and hitting a button to ping our systems to see if we can identify that new lead based on email address, company/first/last name, etc, and if so, return back an internal GUID that represents that individual. This all works for us fine, but we've run into a wall again and again in trying to setup a sane dev environment to work against.
So while our use case is a bit nuanced, this can generally apply to any development house that builds APIs for 3rd party consumption: what are some best practices when designing a development pipeline and environment when you're building APIs to be consumed by the outside world?
At our office, all our devs are behind a firewall, so code in progress can't be hit by the outside world, in our case the CRM provider. We could poke holes in the firewall but that's less than ideal from a security surface area standpoint. Especially if the # of devs who need to be in a DMZ like area is high. We currently are trying a single dev machine in the DMZ and then remoting into it as needed to do dev work, but that's created a resource scarcity issue if multiple devs need the box, let alone they're making potentially conflicting changes (e.g. different branches).
We've considered just mocking/faking incoming requests by building fake clients for these services, but that's a pretty major overhead in building out feature sets (though it does by nature reinforce a testability of our APIs). This also doesn't obviate the fact that sometimes we really do need to diagnose/debug issues coming from the real client itself, not some faked request payload.
What have others done in these types of scenarios? In this day and age of mashups, there have to be a lot of folks out there w/ experiences of developing APIs--what's worked (and not worked so) well for the folks out there?
In the occasions when this has been relevant to me (which, truth be told, is not often) we have tended to do a combination of hosting a dev copy of the solution in-house and mocking what we can't host.
I personally think that the more you can host on individual dev boxes the better-- if your dev's PCs are powerful enough to have the entire thing running plus whatever else they need to develop then they should be doing this. It allows them to have tonnes of flexability to develop without worrying about other people.
For dev, it would make sense to use mock objects and write good unit tests that define the task at hand. It would help to ensure that the developers understand the business requirements. The mock libraries are very sophisticated and help solve this problem.
Then perhaps a continuous build process that moves the code to the dev box in the DMZ. A robust QA process would make sense plus general UAT testing.
Also, for general debugging, you again need to have access the machine in the DMZ where you remote in.
This is probably an "ideal" situation, but you did ask for best practices :).