Assume that I've implemented storage & loading interfaces and concurrent requests arrived for a key that is not present in the map.
Does Hazelcast ensure that the loader will be called only once for these concurrent requests on the same key OR do I have to handle that case in my loader implementation?
No need for external locks. Map operations are atomic including mapstore operations.
Related
I am new to message queues and was wondering if people can explain usecase(s) for using an exclusive queue in RabbitMQ.
From the docs:
An exclusive queue can only be used (consumed from, purged, deleted, etc) by its declaring connection.
Exclusive queues are deleted when their declaring connection is closed or gone. They therefore are only suitable for client-specific transient state
Exclusive queues are a type of temporary queue, and as such they:
"...can be a reasonable choice for workloads with transient clients, for example, temporary WebSocket connections in user interfaces, mobile applications and devices that are expected to go offline or use switch identities. Such clients usually have inherently transient state that should be replaced when the client reconnects."
See notes on durability for more context.
Exclusive queues, as you note, have an added restriction. An exclusive queue:
"... can only be used (consumed from, purged, deleted, etc) by its declaring connection."
This makes it (potentially) suitable for use as queue contained within a single application, where the application will create and then process its own work queue items.
The following points are just my opinion, reflecting on the above notes:
I think it may be a relatively rare RabbitMQ use case, compared to "publish-and-subscribe" use cases such as topic exchanges and similar implementations which communicate across different nodes and connections.
Also, I would expect that much of the time, the core functionality of this type of queue could be provided by a language's built-in data structures (such as Java's queue implementations). But if you want a mature out-of-the-box queueing solution for your application to use internally, then maybe it can be a suitable option.
Consider a company with a large set of functionality to be exposed via web services. A subset of this functionality is used for building up some very complex and computation intensive scenarios, and requires a session to be maintained during this iterative build-up. Each scenario targets one single base structure, representing, say a single customer. That is, a scenario is a series of heavy operations on a single customer structure. The operations can be grouped by which area they target, but basically all operations in the same scenario roots in the same customer structure.
The following decision is given from the outside, and cannot be altered: an already made custom session handler must be used, which basically operates on a session given a simple GUID-token to be send to/from the client. Therefore, from a technical perspective the session need not to be limited to a single service, but can live across multiple services.
Besides the stateful operations, there is also a number of stateless operations.
Given the above decision about the custom session handler, the question is now: how should all these operations be organized? What organization is most elegant?
Some possibilities:
All stateful operations are gathered in one single stateful service, while all stateless services are grouped in an arbitrary set of services, possibly by which area they target. Possible problem: the single stateful service can become very large.
Both stateful and stateless operations are grouped into smaller services, but stateful and stateless operations are still separated so that no service contains both stateful and stateless operations. Possibly, all session estabilshment and finalization can be put in a separate thin dedicated service, say SessionService. With this a approach we have no huge single stateful service. But is the organization elegant? Why force a strict separation of the stateful and stateless operations at all?
Group all operations by their target, ignoring their statefulness. This gives a number of services with mixed stateful and stateless operations. The former can take the session GUID token as input argument, and a service behavior can take care of automatically handle the session estabilshment given some appropriate naming convention for the session key/token.
Similar to above, a separate dedicated service can take care of session establishment and finalization.
Something else, not mentioned above?
Which organizaion is the most elegant?
I have implemented what is basically your third option. We have 20+ operations, some of which check the request for a SessionID (GUID), some of which do not (like Ping() and Login(DeviceID)). All the session handling is "Custom" in the sense that we're not using WCF persistant sessions; rather, we've created a Login() function that takes a GUID, ID, Password from client requests for authentication.
On Login(), we validate the DeviceID, UserID and Pwd against the DB, creating a row on the Session table containing StartTime (session only good for <8 hrs) and DeviceID. We then send back to the client a SessionID (GUID) that he/she uses in all subsequent connections, whether uploading or downloading.
As far as organization is concerned, the subs and methods are organized by the device type (iOS, PC, Android) and the type of operation, just to keep the apples from the oranges. But each function that's Session-related always authenticates the request, validating the inbound SessionID. That may seem wasteful, checking each session with each request (again and again), but because we're using BasicHTTPBinding, we're forced to use a stateless model.
The default values are
Concurrent Calls: 16 * processor count
Concurrent Sessions: 100 * processor count
Concurrent Instances: Concurrent Calls + Concurrent Sessions
This is fine. But I am trying to understand the rationale behind the default value for Concurrent Instances. Why its sum of other two? Can somebody demystify? please
Note: Yes, we can override the values as we like.
From Wenlong Dong's old Blog: WCF 4: Higher Default Throttling Settings for WCF Services
The main purpose for the throttling settings can be classified into
the following two aspects:
Controlled resource usage: With the throttling of concurrent execution, the usage of resources such as memory or threads can be
limited to a reasonable level so that the system works well without
hitting reliability issues.
Balanced performance load: Systems always work in a balanced way when the load is controlled. If there are too much concurrent
execution happening, a lot of contention and bookkeeping would happen
and thus it would hurt the performance of the system.
There's more detail in the blog...
A service can have its SessionMode set to Allowed, NotAllowed, or Required, so the instancing behaviour of the service can depend on the incoming connections. Look at the "Allowed" column of the table at the bottom of this documentation for "Sessions, Instancing, and Concurrency".
It allows an instance per session and an instance per call, depending on the connecting channels.
So the instance limit should be the sum of the session and call limits.
What is the most sensible approach to integrate/interact NServiceBus Sagas with REST APIs?
The scenario is as follows,
We have a load balanced REST API. Depending on the load we can add more nodes.
REST API is a wrapper around a DomainServices API. This means the API can be consumed directly.
We would like to use Sagas for workflow and implement NServiceBus Distributor to scale-out.
Question is, if we use the REST API from Sagas, the actual processing happens in the API farm. This in a way defeats the purpose of implementing distributor pattern.
On the other hand, using DomainServives API directly from Sagas, allows processing locally within worker nodes. With this approach we will have to maintain API assemblies in multiple locations but the throughput could be higher.
I am trying to understand the best approach. Personally, I’d prefer to consume the API (if readily available) but this could introduce chattiness to the system and could take longer to complete as compared to to in-process.
A typical sequence could be similar to publishing an online advertisement,
Advertiser submits a new advertisement request via a web application.
Web application invokes the relevant API endpoint and sends a command
message.
Command message initiates a new publish advertisement Saga
instance.
Saga sends a command to validate caller permissions (in
process/out of process API call)
Saga sends a command to validate the
advertisement data (in process/out of process API call)
Saga sends a
command to the fraud service (third party service)
Once the content and fraud verifications are successful,
Saga sends a command to the billing system.
Saga invokes an API call to save add details. (in
process/out of process API call)
And this goes on until the advertisement is expired, there are a number of retry and failure condition paths.
After a number of design iterations we came up with the following guidelines,
Treat REST API layer as the integration platform.
Assume API endpoints are capable of abstracting fairly complex micro work-flows. Micro work-flows are operations that executes in a single burst (not interruptible) and completes with-in a short time span (<1 second).
Assume API farm is capable of serving many concurrent requests and can be easily scaled-out.
Favor synchronous invocations over asynchronous message based invocations when the target operation is fairly straightforward.
When asynchronous processing is required use a single message handler and invoke API from the handlers. This will delegate work to the API farm. This will also eliminate the need for a distributor and extra hardware resources.
Avoid Saga’s unless if the business work-flow contains multiple transactions, compensation logic and resumes. Tests reveals Sagas do not perform well under load.
Avoid consuming DomainServices directly from a message handler. This till do the work locally and also introduces a deployment hassle by distributing business logic.
Happy to hear out thoughts.
You are right on with identifying that you will need Sagas to manage workflow. I'm willing to bet that your Domain hooks up to a common database. If that is true then it will be faster to use your Domain directly and remove the serialization/network overhead. You will also lose the ability to easily manage the transactions at the database level.
Assuming your are directly calling your Domain, the performance becomes a question of how the Domain performs. You may take steps to optimize the database, drive down distributed transaction costs, sharding the data, etc. You may end up using the Distributor to have multiple Saga processing nodes, but it sounds like you have some more testing to do once a design is chosen.
Generically speaking, we use REST APIs to model the commands as resources(via POST) to allow interaction with NSB from clients who don't have direct access to messaging. This is a potential solution to get things onto NSB from your web app.
I have a web tier that forwards calls onto an application tier. The web tier uses a shared, cached channel to do so. The application tier services in question are stateless and have concurrency enabled.
But they are not being called concurrently.
If I alter the web tier to create a new channel on every call, then I do get concurrent calls onto the application tier. But I want to avoid that cost since it is functionally unnecessary for my scenario. I have no session state, and nor do I need to re-authenticate the caller each time. I understand that the creation of the channel factory is far more expensive than than the creation of the channels, but it is still a cost I'd like to avoid if possible.
I found this article on MSDN that states:
While channels and clients created by
the channels are thread-safe, they
might not support writing more than
one message to the wire concurrently.
If you are sending large messages,
particularly if streaming, the send
operation might block waiting for
another send to complete.
Firstly, I'm not sending large messages (just a lot of small ones since I'm doing load testing) but am still seeing the blocking behavior. Secondly, this is rather open-ended and unhelpful documentation. It says they "might not" support writing more than one message but doesn't explain the scenarios under which they would support concurrent messages.
Can anyone shed some light on this?
Addendum: I am also considering creating a pool of channels that the web server uses to fulfill requests. But again, I see no reason why my existing approach should block and I'd rather avoid the complexity if possible.
After much ado, this all came down to the fact that I wasn't calling Open explicitly on the channel before using it. Apparently an implicit Open can preclude concurrency in some scenarios.
You can cache the WCF proxy, but still create a channel for each service call - this will ensure concurrency, is not very expensive in comparison to creating a channel from scratch, and re-authentication for each call will not be necessary. This is explained on Wenlong Dong's blog - "Performance Improvement for WCF Client Proxy Creation in .NET 3.5 and Best Practices" (a much better source of WCF information and guidance than MSDN).
Just for completeness: Here is a blog entry explaining the observed behavior of request serialization when not opening the channel explicitly:
http://blogs.msdn.com/b/wenlong/archive/2007/10/26/best-practice-always-open-wcf-client-proxy-explicitly-when-it-is-shared.aspx