We have migrated/refactored one of our micro-service from request per thread model (sync) to Reactive (async). Development is complete and started stress testing the reactive service. We are looking for the help on how to perform the activity for reactive APIs
What we have done?
Our service makes HTTP call for an external service. During our stress test, we mocked the external service call. Instead of making a network call, we introduced delay using Thread.sleep() method and returning mock response in our service component(where we make actual HTTP call to the external service).
With this approach, we are observing our reactive service is getting crashed even with very small requests volume. Just to add, we follow similar approach while testing other sync services (request per thread model).
What can we try next?
To have infrastructure closer to what will be live environment, I would suggest to run separate mock server, preferably on separate machine, so it will not steal resources from your service.
You can do it easily with wiremock - see running as standalone page.
When you run wiremock as proxy, you can record and playback real requests from your service.
To make this mock more real, you can add delay for mock responses - it will simulate external service "processing time".
reactive services have a very small thread pool which is optimized to be utilized at 100%.
If you introduce something blocking like Thread.sleep() you are basically removing this ability. What makes reactive, "reactive" is its ability to switch execution threads if something is blocking.
Thread.sleep() holds the thread where it is, and it cant switch to doing something else, basically crippling its entire functionality.
You should never Thread.sleep() in a reactive application, unless you are using OnSubscribe and placing each call on a separate scheduler, which in turn sort of makes your reactive application "non-reactive" and instead fallbacks to default standard servlet application behavior.
Related
I have a synchronous legacy WCF service with a potentially long running operation. For some clients, this operation seems to cause network timeouts - not at the client or service end, but somewhere in the middle at some proxy server in a network topology that is opaque to me.
The question is: Could I solve this problem by using the event based asynchronous pattern on the service side (using IAsyncResult / BeginXXX(), EndXXX())?
I am puzzled about how the callback mechanism of async services actually works on the network level. Does the client do periodic polling, or some kind of long polling, or something completely different? Unless I am simply bad at using Google (I believe I am not), the MSDN documentation seems to be completely lacking in that respect.
In only keeps going on about how async programming helps you to build responsive GUIs and whatnot.
The question is: Will it prevent proxies from timing out requests that they believe to be taking too long?
EDIT: To clarify: The service in question uses basicHttpBinding.
There are no callbacks at the network level. Async IO is all about not blocking threads on a single machine. Both communication parties can independently decide to use sync or async IO as they please. Neither party can even detect what the other chose.
You are using the SOAP-based basicHttpBinding. SOAP has no notion of asynchronous invocation. Neither does any of the other bindings.
Async IO can do nothing to resolve your timeout problem. Those timeouts are caused by something on the network (as you said). The network has nothing to do with how a service is implemented. It cannot even find out and respond to it.
The timeout things are at the http stack, and have nothing to do with the async things in programming. Google "windows http stack" you may find out why.
"In only keeps going on about how async programming helps you to build responsive GUIs and whatnot." this helps you not to block the UI thread with a long running operation like a http request/response.
A WCF service is by default multi-threading -- upon one request, a new thread is created.
On the client side if you are using .NET 4.5, you may find the client proxy classes generated support async operations, however, this has nothing to do with http timeout. These functions make asynchronous/parallel programming easier on the client side.
In short, the timeout is determined by the http stacks on both server and client. If you have controls over both server and client, you may increase timeout values on both side, as quick and dirty solution.
I don't think it prevent the timeout as the asynchronous development is on the client side independent of the protocol and the network configuration. For HTTP this is likely a registry setting.
My understanding is that windows I/O is by nature asynchronous so I don't think there will be any polling, rather once the IO is complete some sort OS level interrupt will occur. You may find this article by Stephen Cleary helpful. Also the books Windows Internals might be a good reference as well.
I have a driver that I need to access via a web site that is not thread-safe. Since many people can be on the site at a given time I figured I would create a WCF service that would handle all the calls. Most of the calls would be asynchronous calls to add items to a work queue. Some would be synchronous calls to get a list of items still unprocessed or items that have been processed.
Since the driver isn't thread safe, the service must take in potentially many requests at once and either add items to the work queue, return the work queue, or return the work-completed queue. A single-threaded operation in the service needs to read from the work queue, processes the job using this non-thread-safe driver, and, when complete, update the work-completed queue.
While I conceptually have clear in my mind what to do, the specifics of implementation confuse me a little. I think I should host the service in IIS since it will have to respond to web requests and otherwise act like any other web site, but I'm not sure how to guarantee that the access of the driver will remain single-threaded without blocking web requests. Do I need a second service, perhaps a Windows service, that would process all access to the driver and use the IIS-hosted WCF service to get the next queue item and update the queue when processing is complete?
I'd consider:
Clients call your aspx pages,
Pages call to wcf service (netMsmqBinding)? - to avoid blocking and waiting (singke service, can be hosted where you want).
When server done - it's notify clients (websocket? SignalR?)
I'm writing an application which will use the Azure Service Bus. For local development I'm using Windows Server Service Bus to provide the same services (the code to use either is identical).
I want to write the application to be tolerant of transient errors when sending or receiving messages. To that end, I want to be able to test the fault-handling code can deal with the local Service Bus instance suddenly being unavailable during execution of various operations.
Ideally, I'd want to write some automated integration tests around these scenarios, but I appreciate that may not be practically achieved.
What can I do to simulate transient errors on my local Service Bus?
One easy thing would be to call the stop-sbservice (affects one node) or stop-sbfarm (affects the entire farm) cmdlets. This would let you simulate a servicebus outage locally. You can then call start-sbservice or start-sbfarm to bring the service back and validate that your code recovers properly. This approach also has the added benefit that you control when the service returns (compare to just crashing the process). This page has information on the available cmdlets.
If that's not enough, another approach that I've used in the past is to shut down the network interface, or, if the server is in another machine, put up a firewall on the ports used to communicate to service bus.
I have a WCF Web Service that has no concurrency configuration in the web.config, so I believe it is running as the default as persession. In the service, it uses a COBOL Virtual Machine to execute code that pulls data from COBOL Vision files. Per the developer of the COBOL VM, it is a singleton.
When more than one person accesses the service at a time, I'll get periodic crashes of the web service. What I believe is happening is that as one process is executing another separate process comes in at about the same time. The first process ends and closes the VM down through normal closing procedures. The second process is still executing and attempting to read/write data, but the VM was shutdown and it crashes. In the constructor for the web service, an instance of the VM is created and when a series of methods complete, the service is cleaned up and the VM closed out.
I have been reading up on Singleton concurrency in WCF web services and thinking I might need to switch to this instead. This way I can open the COBOL VM and keep it alive forever and eliminate my code shutting down the VM in my methods. The only data I need to share between requests is the status of the COBOL VM.
My alternative I'm thinking of is creating a server process that manages opening the VM and keeping it alive and allowing the web service to make read/write requests through that process instead.
Does this sound like the right path? I'm basically looking for a way to keep the Virtual Machine alive in a WCF web service situation and just keep executing code against it. The COBOL VM system sends me back locking information on the read/writes which I can use to handle retries or waits against.
Thanks,
Martin
The web service is now marked as:
[ServiceBehavior(ConcurrencyMode = ConcurrencyMode.Single)]
From what I understand, this only allows a single thread to run through the web service at a time. Other requests are queued until the first completes. This was a quick fix that works in my situation because my web service doesn't require high concurrency. There are never more than a handful of requests coming in at a time.
I have following problem:
I need to synchronize access to cache objects that are accessed by WCF-service operations. The calling clients provides a parameter that maps to which cache object(s) operation will use.
In short, requests should be queued by the parameter and then executed in synchronous manner.
What would be best way to implement this? I don't want to expose the queue to the client and I want solution that easily scales if there are multiple service instance running in different machines with shared cache.