I am new to apache ignite and my client is having an requirement to implement it.
Basically we have an application which runs in Jboss EAP7. currently the application is using around 50 reference data tables which are static in nature. The overall record would be around 50 thousand.
This is a 3 layered application i.e. DAO, service and presentation. I am looking for best practices from design perspective that will fit this kind of applications.
Related
Current situation:
Thick client wrote in .NET
We have a very old computation software that we can't maintain anymore.
We don't really know how the kernel is working (people left, 15 years old code).
We have the code and some technical experts.
We want to migrate it to the cloud behind a public API in order to serve some SPA application or even thick client applications.
What is you recommendation about that problem?
We have thought about:
Lift-n-Shift
Lift-Adjust-n-Shift
Rearchitecting or redeveloping from the ground
Repurchasing a new cloud solution (but it doesn't seem to have any)
All options that you mentioned are possible but which one to choose really depends on your business needs time and budget.
Lift and shift (vms)
This is mostly quickest approach and you may simply use VMs to migrate to cloud. But managing VMs is your responsibility and is on going committment.
Lift adjust and shift (containers)
in my opinion you get benefits of cloud when you start using PAAS services. You may consider containerize (docker) your application and migrate it to cloud and start using paas services. your dev ops cycle will be quick and scaling is easy. Since you are not managing vms anymore it's less hassle.
rearchitect amd redevlop
this could be costly and time consuming and really depends if your business requirements allow you to do that. if you plan to expand the existing code base then you may consider this else it could be big deal when you can simply migrate your services using approaches mentioned above.
I wonder if anyone could share their thoughts on my question regarding web based APIs (we use Microsoft stacks)..
We are currently in the process of building an infrastructure to host web apis across our business.
As a organisation we have seperate business areas that provide services to our customers. These individual areas of our business generally have their own best of breed IT system. Offering APIs is something we've long thought about and we have started the design process.
The APIs we aim to offer shall be web based (.NET/webAPI/WCF etc.) and will largely (99%) be consumed within our organisation but some may be exposed externally in the future should the requirement arise (new mobile app may need to use the services etc.)
I'd love to hear your thoughts and experiences around how you architected yuor farms. I understand its quite an open question without understanding the crooks of our requirements but its more general advice/experiences I'd like to hear.
Particularly we are trying to decide whether we should design the infrastrcuture by:
1) Providing each area of the business with their own API server whereby we shall deploy each web API within a new application inside IIS.
or
2) Setup up a load balanced web api farm whereby we have say 2/3 iis web servers, all built the same, hosting the same web apis but the business areas will all share the same server effectively. Each area would have a segregated site within iis and new APIs shall be setup under new applications inside their respective web sites.
I dont foresee us having thousands of APIs but some will be business critical so I'm certainly bearing resilience in mind which is why as much as I like each business area having their own API server, I'm being swayed towards the option of having a load balanced farm which the whole business shares.
Anyone have any thoughts, experiences etc.?
Thanks!
That's a very interesting question, and i'd love to hear what others might think. I'm no big expert, but here are my two cents.
It seems to me, that the answer should be somewhere in between those two options you specified. Specifically, each critical business area, should get their own resilient, load balanced farm, while less critical services can utilize single machine deployments. Critical business area may not mean only one API, but can actually be a group of APIs, with high cohesion among themselves.
Using option 1 environment to full extent can be hard to maintain,
while utilizing option 2 fully, can be inefficient in terms of redeployment if (or better yet, when) business logic changes. Furthermore, i think it will be possible for greedy APIs to hog resources in peak traffic, making other services temporary less performant (unless you have some sort of dynamic scaling mechanism).
Background:
I'm thinking about web application organisation. I will separate front (web site for browser) from back (API): 2 apps, 2 repository, 2 hosting. Front will call API for almost everything.
So, if I have two separate domain services with my API (example: learning context and booking context) with no direct link between them, should I build 2 API (with 2 repository, 2 build process, etc.) ? Is it a good practice to build n APIs for n needs or one "big" API ? I'm speaking about a substantial web app with trafic.
(I hope this question will not be closed as not constructive... I think it's a real question for a concrete case, sorry if not. This question and some other about architecture were not closed so there is hope for mine)
It all depends on the application you are working on, its business needs, priorities you have and so on. Generally you have several options:
Stay with one monolithic application
Stay with one monolithic application but decouple domain model across separate modules/bundles/libraries
Create distributed architecture (like Service Oriented Architecture (SOA) or Event Driven Architecture (EDA))
One monolithic application
It's the easiest and the cheapest way to develop application on its beginning stage. You don't have to worry about complex architecture, complex deployment and development process. It also works better if there are no many developers around.
Once the application is growing up, this model begins to be problematic. You can't deploy modules separately, the app is more exposed to anti-patterns, spaghetti code/design (especially when a lot people working on it). QA process takes more and more time, which may make it unusable on CI basis. Introducing approaches like Continuous Integration/Delivery/Deployment is also much much harder.
Within this approach you have one repo/build process for all your APIs,
One monolithic application but decouple domain model
Within this approach you still have one big platform, but you connect logically separate modules on 3rd party basis. For example you may extract one module and create a library from it.
Thanks to that you are able to introduce separate processes (QA, dev) for different libraries but you still have to deploy whole application at once. It also helps you avoid anti-patterns, but it may be hard to keep backward compatibility across libraries within the application lifespan.
Regarding your question, in this way you have separate API, dev process and repository for each "type of actions" as long as you move its domain logic to separate library.
Distributed architecture (SOA / EDA)
SOA has a lot profits. You can introduce completely different processes for each service: dev, QA, deploying. You can deploy just one service at once. You also can use different technologies for different purposes. QA process gets more reliable as it involves smaller projects. You can version communication (API) between services which makes them even more independent. Moreover you have better ability to scale horizontally.
On the other hand complexity of the high level architecture grows. You have much more different components you have to take care: authentication / authorisation between services, security, service discovering, distributed transactions etc. If your application is data driven (separate frontend which use APIs for consuming data) and particular services don't need to communicate to each other - it may be not as much complicate (but such assumption is IMO quite risky, sooner or letter you will need to communicate them).
In that approach you have separate API, with separate repositories and separate processes for each "type of actions" (which I understand ss separate domain model / services).
As I wrote on the beginning the way you choose depends on the application and its needs. Anyway, back to your original question, my suggestion is to keep APIs as separate as you can. Even if you have one monolithic application you should be able to version APIs separately and keep their domain logic separate. Separating repositories and/or processes depends on the approach you choose (eg. among these I mentioned before).
If I missed your point, please describe in more detailed way what answer do you expect.
Best!
I'm looking at designing a commercial web site from scratch in the new year and I was planning on using NServiceBus. Other components would include RavenDB, ASP.NET MVC, Ninject, Bootstrap, etc.
My question is, if I build scalability in from the beginning, particularly if in the first 6 months I plan to run the site from a single server, would it be a foolish thing to use NServiceBus from the outset? Will I experience much of a choke-point by pushing everything through MSMQ rather than direct calls to methods in DLLs? Should NServiceBus only be added to mature systems, or systems that are intended to be deployed to more than one server?
While the fastest possible call would be a direct in-memory invocation, I wouldn't look at optimizing for latency of calls in most web scenarios. Instead, focus on what type of logic can be run asynchronously with respect to the user request. That will be most influential on your overall scalability.
NServiceBus provides a fairly clean programming model for asynchronous invocations that can later on be distributed across multiple processes and machines.
So I was listening to the latest Stackoverflow podcast (episode 19), and Jeff and Joel talked a bit about scaling server hardware as a website grows. From what Joel was saying, the first few steps are pretty standard:
One server running both the webserver and the database (the current Stackoverflow setup)
One webserver and one database server
Two load-balanced webservers and one database server
They didn't talk much about what comes next though. Do you add more webservers? Another database server? Replicate this three-machine cluster in a different datacenter for redundancy? Where does a web startup go from here in the hardware department?
A reasonable setup supporting an "average" web application might evolve as follows:
Single combined application/database server
Separate database on a different machine
Second application server with DNS round-robin (poor man's load balancing) or, e.g. Perlbal
Second, replicated database server (for read loads, requires some application logic changes so eligible database reads go to a slave)
At this point, evaluating the current state of affairs would help to determine a better scaling path. For example, if read load is high and content doesn't change too often, it might be better to emphasise caching and introduce dedicated front-end caches, e.g. Squid to avoid un-needed database reads, although you will need to consider how to maintain cache coherency, typically in the application.
On the other hand, if content changes reasonably often, then you will probably prefer a more spread-out solution; introduce a few more application servers and database slaves to help mitigate the effects, and use object caching, such as memcached to avoid hitting the database for the less volatile content.
For most sites, this is probably enough, although if you do become a global phenomenon, then you'll probably want to start considering having hardware in regional data centres, and using tricks such as geographic load balancing to direct visitors to the closest "cluster". By that point, you'll probably be in a position to hire engineers who can really fine-tune things.
Probably the most valuable scaling advice I can think of would be to avoid worrying about it all far too soon; concentrate on developing a service people are going to want to use, and making the application reasonably robust. Some easy early optimisations are to make sure your database design is fairly solid, and that indexes are set up so you're not doing anything painfully crazy; also, make sure the application emits cache-control headers that direct browsers on how to cache the data. Doing this sort of work early on in the design can yield benefits later, especially when you don't have to rework the entire thing to deal with cache coherency issues.
The second most valuable piece of advice I want to put across is that you shouldn't assume what works for some other web site will work for you; check your logs, run some analysis on your traffic and profile your application - see where your bottlenecks are and resolve them.
plenty of fish Architecture
some interesitng videos:
Youtube scalibility
Inteview with Dan Farino, System Architect at Myspace
Joel mentioned adding a second datacenter, with the same setup, and then assigning your users randomly to each. Changes to the data are logged and sent from one location to the other, so that both locations contain all the data.
The talk Scalable Web Architectures Common Patterns & Approaches from Cal Henderson (Yahoo) on Web 2.0 Expo was quite interesting. I thought there was an video, but I could not find it. But here are the slides:
http://www.slideshare.net/techdude/scalable-web-architectures-common-patterns-and-approaches
A certain next step would be a cluster of webservers (a web farm) and a clustered system of database servers (replication or Oracle RAC etc. etc.)
If your interested in caching and using .Net, look into the application caching block in enterprise library (of course use this along with the other points above).