I'm new to redis. Trying to find out if we can build a REST api for redis cache elements so that they can be consumed by different clients (node, c# etc) Is it possible to do it? If so, can I get some guidance ?
You can use webdis which will give you rest oriented client which will wrap around redis. As it is REST oriented you can connect major through all other microservices written in different languages.
It supports major feature.
Related
I have a two project linked with each other, Project A and Project B in same VS solutions.
Now the requirement is, I have to do seperate projects, it means seperate DB, web site name, servers etc.
Now I want to know how I will communicate both the project each other using web API?
Before starting separate I would suggest reading about DDD (Domain Driven Design) to make the rights choices about what functionality and classes could be in one microservice or the other.
Anyway, once you have separated the two services properly, so the communication between them would be minimum, there are mainly two ways to communicate with each other.
One is using synchronous communication using HTTP calls using some HTTP client for your language, for example, in java-spring applications is very common to use Webclient. The pros of this approach is that you don't have to deal with eventual consistency because all happens in the same thread, the main con is that you are coupling the services and if one of them fails the other will fail too.
The second approach is to use asynchronous messaging using some message broker like kafka. With this approach when something happen in one of the service, it will publish the event in the message broker and it will be consumed by the other. Ther is no direct communication so there can't be cascade failures but the problem is that you don't know when exactly the message will be consumed (eventual consistency)
As you see both options have pros and cons and it depends on your use case to choose one or the other
I am very new on RabbitMQ World and Microservices Architecture. I've watched some tutorials on youtube how to use rabbitMQ but there are some questions I would like to ask.
1.) If the client post the data to my api controller and my api controller publish the data into queue, what is the proper way to respond the post request while the data is being processed on rabbitMQ? Because from the tutorial, It just returns "Ok (Http Code 200)" even though the data is not still completed yet.
2.) Can consumer subscribe more than 1 queue? If yes, is there any configuration sample on startup.cs?
3.) Is there any sample using rabbitMQ on .NetCore for "Real World" cases? Please let me know.
Thanks
It's perfectly okey to return 200. That's just one of many tradeoffs for microservices architecture. E.g. from the performance perspective, it's efficient that you can return early with 200 and propagate all the changes asynchronously through the rest of the distributed system. On the other hand, it adds another type of complexity that you need to embrace - Eventual Consistency. This concept kind of describes what you asked about. Let say your client received 200, but if it immediately calls another microservice the client may not see changes introduced by the previous request, because there is a probability that the changes haven't propagated yet. You need to decide whether it's acceptable in your project or not. If not maybe you should redesign how you split your business domain into microservices, trying to group transactionally close to each other entities together in order to mitigate such problems. If you can't really tolerate Eventual Consistency maybe you should give up on microservices for the particular project.
Yes it can, you could for example create an implementation of IHostedService for each queue listening for messages and run them in parallel with your asp net core app by registering them in the starutp.cs
You'll find this in the repository from the below's links. They use RabbitMQ. Although, there's a bit of abstraction which can make it harder to grasp, it's a great implementation with a bonus of being documented in the free ebook.
https://github.com/dotnet-architecture/eShopOnContainers/ - I can't stress enough how this repository helped me with understanding microservices. There's also a free ebook from Microsoft docs about this repo: https://learn.microsoft.com/en-us/dotnet/architecture/microservices/ . They tackle concepts such as Eventual Consistency and asynchronous communication. It's exactly what you look for.
I am implementing a micro-service architecture for the first time.
Some of my services (.NET Core Web APIs) need to communicate with each other through HTTP requests. For that purpose, I am injecting a wrapper around HttpClient.
But I suspect that I am reinventing the wheel. Among micro-service practitioners, is there a pattern or even a third-party library to solve this problem?
In a micro-service architecture, the most important thing is a clear separation of concerns and application boundaries. Imagine a simple setup, with Product and Price micro services
An important concept is each service is master of data, and owns its own database. In this example,
a client of the 'Product' service will make an HTTP call to the Product API.
the product API will make a call to the Price API to get prices for the products
the product API therefore depends on the Price API to create a response
These are the synchronous parts of the process, generally achieved through HTTP calls across boundaries. You'll also have asynchronous parts of your solution, in this example,
the Price API publishes an event to a bus whenever a price is changed
the product API publishes an event whenever a product is created
There may be one or more subscribers to these events, that will respond and probably call an API to retrieve the changed data.
The critical parts of this are clearly defining your API and message contracts, understanding if things will be async or sync, having the right level of telemetry across the entire architecture to track and understand distributed system behaviour, and keeping everything as independently buildable/testable/deployable components.
First and foremost, if you're not using containers, start, along with orchestration (both natively supported in Visual Studio, assuming you have Docker, etc. actually installed). Among the many benefits, you can reference your services via hostname without having to worry about ports and different locations for different environments.
As far as actual communication goes. There's not really a magic solution here. HttpClient is what you use, of course, and generally, yes, you want to have a wrapper around that to abstract away the low-level HTTP communication stuff, so the rest of your code can simply call simple methods on that wrapper.
If you aren't using IHttpClientFactory, start. If you already have a wrapper class, you're halfway there, and with that, not only do you get efficient management of HttpMessageHandlers so you don't exhaust your server's connection pool, but you can also use the Polly integration to handle transient HTTP errors and even do retry policies, circuit breakers, etc. for your microservice connections.
Finally, there is the Refit library which can make things a tad more straight-forward. I find it to have more use with huge third-party APIs like Facebook, Google, etc., though. Since microservices should by design be simple, you're probably not saving much code over just having your own wrapper class. Regardless, the way it works is that you define an interface that represents the API, and then Refit uses that to actually make appropriate requests. It's kind of like a wrapper class for free, but you still need to create the interface.
I am building huge application using microservices architecture. The application will consist of multiple backend microservices (deployed on multiple cloud instances), some of which I would like to connect using rest apis in order to pass data between them.
The application will also expose public api for third parties, but the above mentioned endpoints should be restricted ONLY to other microservices within the same application creating some kind of a private network.
So, my question is:
How to achieve that restricted api access to other microservices within the same application?
If there are better ways to connect microservices than using http transport layer, please mention them.
Please keep the answers server/language agnostic if possible.
Thanks.
Yeah easy. Each client of a micro service has an API key. Micro services only accept requests from clients with a valid API Key.
Also, its good to know that REST is simply a protocol that allows communication between bounded contexts.
It doesn't have to be over HTTP. The requirement is that it has a uniform interface (this is why HTTP is used with its PUT, POST, GET, DELETE... methods) and that it is stateless (all state being transferred through a URI).
So if all your micro services run on the same box, all you need to do is something like this:
class SomeClass implements RestfulMethods {
public function get(params){ // return something}
public function post(params){ // add something}
public function put(params){ // update something}
public function delete(params){ // delete something}
}
Micro services then communicated by interacting with the RestfulMethod implementations of other services.
But if your micorservices are on different machines, its probably best to use HTTP as the transport mechanism.
One way is to use HTTPS for internal MS communication. Lock down the access (using a trust store) to only your services. You can share a certificate among the services for backend communication. Preferably a wildcard certificate. Then it should work as long as your services can be adressed to the same domain. Like *.yourcompany.com.
Once you have it all in place, it should work fine. HTTPS sessions does imply some overhead, but that's primarily in the handshake process. Using keep-alive on your sessions, there shouldn't be much overhead with encrypted channels.
Of course, you can simply add some credentials to your http headers as well. That would be less secure.
RestAPI is not only way to do it, one of the some ideas that i have seeming is about the usage of Service Registry link Eureka (Netflix), Zookeeper (Apache) and others.
Here is an example:
https://github.com/tiarebalbi/qcon2015-sao-paolo-microservices-workshop
...the above mentioned endpoints should be restricted ONLY to other
microservices within the same application...
What you are talking about in a broad sense is authorisation.
Authorisation is the granting or denying of "powers" or "abilities" within your application to authentic users.
Therefore the job of any authorisation mechanism is to validate the "claim" implicit in any inbound API request - that the user is allowed to do the thing encoded in the request.
As an example, imagine I turned up at your API with a PUT request for Widget 1234:
PUT /widgetservice/widget/1234 HTTP/1.1
This could be interpreted as me (Bob Smith, a known user) making a claim that I am allowed to make changes to a widget in your system with id 1234.
Whatever you do to validate this claim, I hope you can see this needs to be done at the application level, rather than at the API level. In fact, authorisation is an application-level concern, rather than an API-level concern (unlike authentication, which is very much an API level concern).
To demonstrate, in our example above, it's theoritically possible I'm allowed to create a new widget, but not to update an existing widget:
POST /widgetservice/widget/1234 HTTP/1.1
Or even I'm allowed to update only widget 1234 and requests to change other widgets should not be allowed
PUT /widgetservice/widget/5678 HTTP/1.1
How to achieve that restricted api access to other microservices
within the same application?
So this becomes a question about how can you build authorisation into your application so that you can validate individual requests coming from known users (in your case your other services in your ecosystem are just another kind of known user).
Well, and apologies but I'm going to be prescriptive here, you could use a claims-based authorisation service, which stores valid claims based on user identity or membership of roles.
It depends largely on how you are handling authentication, and whether or not you are supporting roles as part of that process. You could store claims against individual users but this becomes arduous as the number of users increases. OAuth, despite being pretty heavy to implement, is a leading platform for this.
I am building huge application using microservices architecture
The only thing I will say here is read this first.
The easiest way is to only enable access from the IP address that your microservices are running on.
I know i'm super late for this question :)) but for anyone who came across this thread, Kafka is a great option for operations similar to this question.
based on Kafka's own introduction
Kafka is generally used for two broad classes of applications:
Building real-time streaming data pipelines that reliably get data between systems or applications
Building real-time streaming applications that transform or react to the streams of data
Side Note: Kafka is created by LinkedIn and is being used in many huge companies so it's kindda battle tested.
you can use RabbitMQ
publish your resquests to queue and then consume tasks
I want to create something similar to the Twitter streaming API using WCF. We need to stream data to clients as quickly as possible (1/4 second). Our clients are diverse (JAVA, C++, etc. etc. etc.) and we're a .NET shop. Does anyone know if WCF can do this?
Check out the Streaming Data section here http://msdn.microsoft.com/en-us/library/ms733742.aspx
I haven't tested this but here's an example http://shevaspace.blogspot.com/2009/01/streaming-media-content-over-wcf.html