last time I am thinking about proper using logger in our applications.
For example, I have a controller which returns a stream of users but in the log, I see the "Fetch Users" log is being logged by another thread than the thread on the processing pipeline but is it a good approach?
#Slf4j
class AwesomeController {
#GetMapping(path = "/users")
public Flux<User> getUsers() {
log.info("Fetch users..");
return Flux.just(...)..subscribeOn(Schedulers.newParallel("my-custom"));
}
}
In this case, two threads are used and from my perspective, not a good option, but I can't find good practices with loggers in reactive applications. I think below approach is better because allocation memory is from processing thread but not from spring webflux thread which potential can be blocking but logger.
#GetMapping(path = "/users")
public Flux<User> getUsers() {
return Flux.defer(() -> {
return Mono.fromCallable(() -> {
log.info("Fetch users..");
.....
})
}).subscribeOn(Schedulers.newParallel("my-custom"))
}
The normal thing to do would be to configure the logger as asynchronous (this usually has to be explicit as per the comments, but all modern logging frameworks support it) and then just include it "normally" (either as a separate line as you have there, or in a side-effect method such as doOnNext() if you want it half way through the reactive chain.)
If you want to be sure that the logger's call isn't blocking, then use BlockHound to make sure (this is never a bad idea anyway.) But in any case, I can't see a use case for your second example there - that makes the code rather difficult to follow with no real advantage.
One final thing to watch out for - remember that if you include the logging statement separately as you have above, rather than as part of the reactive chain, then it'll execute when the method at calltime rather than subscription time. That may not matter in scenarios like this where the two happen near simultaneously, but would be rather confusing if (for example) you're returning a publisher which may be subscribed to multiple times - in that case, you'd only ever see the "Fetch users..." statement once, which isn't obvious when glancing through the code.
Related
I have a bunch of network requests I want to conduct in parallel.
The following pseudo code should give a good idea of what I'm doing right now:
runBlocking {
buildList {
withContext(tracer.asContextElement()) {
items.forEach { item ->
add(
async {
// a few IO intensive operations (i.e. network requests)
}
)
}
}
}.awaitAll()
}
I have tracing tools set up and locally this seems to do the job. In my production infrastructure however the async tasks execute sequentially, i.e. the second one starts immediately after the first one finishes.
I have also tried using withContext(Dispatchers.IO.plus(tracer.asContextElement())) but I observe no difference.
The only thing I can say is that my development machine has multiple CPU cores, and my production machine will normally have 1. Regardless, due to the IO heavy nature of these processes I doubt this is the problem. I can't really explain what is causing this, but my gut feeling is that I'm fundamentally not understanding something about how Coroutines work in Kotlin.
As to the nature of the network request in question, I'm using a third party SDK that asynchronously executes the request, and seems to use ForkJoinPool.commonPool() under the hood as an executor.
If you don't switch dispatchers here, all those coroutines will run in the same thread - the one blocked by runBlocking. If the computation inside each coroutine is blocking, they will block the only thread one by one without any way to parallelize. This would explain what you're seeing (although it's strange that you don't reproduce locally).
I have also tried using withContext(Dispatchers.IO.plus(tracer.asContextElement())) but I observe no difference.
Your fix should work, unless the IO you're performing is actually managing threads itself and also confining the execution to a single thread no matter where it's called from. Maybe you should look into the actual IO then.
EDIT: you mentioned that you perform the IO operations via a third party SDK that uses the common ForkJoinPool - this one is backed by a single thread on a single-CPU machine, so this explains why the calls aren't parallelized in your single-CPU production machine. The only options to fix that would be:
check whether the SDK you're using allows to customize the backing pool of threads
customize the size of the ForkJoinPool using the JVM property java.util.concurrent.ForkJoinPool.common.parallelism
use another SDK :)
You still need to customize the dispatcher in addition to that if you're calling the library in a blocking way, but not if you're converting their async tasks into suspensions using Future.await() or similar.
Now, a few other things to note in this code:
you don't need buildList { .. }, you can just use map { thing } instead of forEach { add(thing) } and you'll get the resulting list as a return value (it also works across withContext, because it returns the lambda result)
withContext actually waits for all child coroutines to finish, so awaitAll() is misplaced here (it should rather be inside withContext)
actually, you probably don't need withContext at all, you can pass the custom context directly to runBlocking, unless you have other things in runBlocking that you don't want to run in this context
(optional) if the IO computations don't return results, you don't need awaitAll at all, and you could just use launch instead.
Assuming you do need the result, so ignoring the last point, your current code (with dispatcher fix) could be rewritten to:
val results = runBlocking(Dispatchers.IO + tracer.asContextElement()) {
items.map { item ->
async {
performIO(item)
}
}.awaitAll()
}
Otherwise:
runBlocking(Dispatchers.IO + tracer.asContextElement()) {
items.map { item ->
launch {
performIO(item)
}
}
}
Updated below...
So this is a random question that came about from a discussion over what I consider the over usage of await in projects at work...
Not sure why I never tried it until now, other than the fact that it's weird and random, but the fact it does work just makes me wonder...how?
The .Result usage was only used to get the responseMessage. I know that is blocking and no bueno and it is for demo purposes only...
So this endpoint works fine...nothing is awaited
[HttpGet("kitteh")]
public Task<string> GetCatFact()
{
var client = new HttpClient();
var res = client.GetAsync("https://catfact.ninja/fact").Result;
return res.Content.ReadAsStringAsync();
}
There's obviously something in the default ASP.NET pipleline that ultimately unwraps the task in order to return the result...but where...or how?
Is this "less efficient" than awaiting in the endpoint itself as the magic taking place behind the scenes is ultimately just blocking to get the result of the returned task?
Updated
So, I am still skeptical and don't think the answer is as easy as "yes it is blocking" or "not it is not blocking" unless there's some legit proof or something to indicate one way or the other. I tried digging through the code myself, and I still don't have a solid answer but...I do know more than I did before...
I simplified the endpoints I have been testing this with...
[HttpGet("taskstring")]
public Task<string> TaskString()
{
return Task.FromResult("Where does this block?");
}
[HttpGet("asyncstring")]
public async Task<string> AsyncString()
{
return await Task.FromResult("This definitely doesn't block");
}
I stepped through a bunch of code and ultimately landed on ObjectMethodExecutor and AwaitableObjectResultExecutor which sparked some interest.
What appears to happen for both the endpoint versions above is the same or nearly the same, at least the execution of these two endpoints and the code covered below. There's a boatload that goes on during this and even though the rider debugger is great, it's not possible to see a lot of the values while debugging due to "Evaluation is not allowed: The thread is not at a GC-safe point".
When the endpoint is called but prior to the endpoint beginning execution...
ObjectMethodExecutor.ctor is called
This does a lot of inspection of the endpoint to determine a bunch of things
One of the checks is to is determine if the method is "async" and it ultimately calls AwaitableInfo.IsTypeAwaitable to check for the required methods/properties and interfaces to ensure it is
If it is, which in the case of returning Task of string it is, makes sense given the above info
ObjectMethodExecutor.GetExecutorAsync() is called
AwaitableObjectResultExecutor.Execute is then called
this type inherits ActionMethodExecutor which overrides the Execute method
executor.ExecuteAsync(executor type of ObjectMethodExecutorAwaitable) is then called, and awaited
this returns a result of type object, boxing yeah I know
The endpoint then actually executes and returns to AwaitableObjectResultExecutor.Execute
the result is then passed to ConvertToActionResult along with the return type, the T of Task of T and the mapper implementation
the mapped ActionResult is then returned
So...I'm still not certain 100% either way but there is a lot of effort put into inspecting the endpoint that is being called and determining if it's async or not, which would make it seem logical to think this is an attempt to avoid something...maybe blocking?
So...does it block...maybe? Seeing that there is an await from the indirect caller of the endpoint, I'd lean towards, no...but it seems really difficult to say still.
What I do now know is...
It definitely does await the endpoint call, albeit indirectly
It converts the Task of string return type to an IActionResult
An async and task only version seem to follow the exact some flow shown above
A non async version does not
So, based on all of that...#Phil's answer seems to be pretty spot on with what I found...
"The framework supports asynchronous controller actions. In order to do so, it would need to inspect the return value of your methods. If the method returns a Task, it will ultimately await on the result before responding.
Even if your action itself does all the awaiting, it still has to return a Task so the caller will still wait (the alternative being some ugly blocking code)."
Thanks to all for participating in an extremely random question that doesn't hold much value to truly understand
Your assumption here is basically correct...
There's obviously something in the default ASP.NET pipleline that ultimately unwraps the task in order to return the result
The framework supports asynchronous controller actions. In order to do so, it would need to inspect the return value of your methods. If the method returns a Task, it will ultimately await on the result before responding.
Even if your action itself does all the awaiting, it still has to return a Task so the caller will still wait (the alternative being some ugly blocking code).
As has been pointed out in some other posts, there are some performance improvements to be had by not awaiting a returned Task so I would write your action as
public async Task<string> GetCatFact()
{
var client = new HttpClient();
var res = await client.GetAsync("https://catfact.ninja/fact");
return res.Content.ReadAsStringAsync(); // no await
}
Your controller handles waiting for the remote response but delegates waiting for the content stream to the caller.
There's obviously something in the default ASP.NET pipleline that ultimately unwraps the task in order to return the result...but where...or how?
ASP.NET asynchronously waits for your task to complete, and then it sends the HTTP response based on the result of the task. It's logically similar to await: an asynchronous wait.
Is this "less efficient" than awaiting in the endpoint itself as the magic taking place behind the scenes is ultimately just blocking to get the result of the returned task?
Yes. It is less efficient to block.
ASP.NET doesn't block; it asynchronously waits. Blocking ties up a thread. So when the code calls .Result, it will be using a thread just to wait for that HttpClient call to complete.
The proper solution is to keep async and await:
[HttpGet("kitteh")]
public async Task<string> GetCatFact()
{
var client = new HttpClient();
var res = await client.GetAsync("https://catfact.ninja/fact");
return await res.Content.ReadAsStringAsync();
}
This way, while the GetAsync is in progress, the thread is yielded back to the ASP.NET runtime and is available for handling other requests, instead of being blocked waiting for the GetAsync to complete.
More information: Task<string> is part of the method signature. ASP.NET has special understanding of the Task<T> type and knows to asynchronously wait for it. async is not part of the method signature. ASP.NET knows whether your method returns Task, but it has no idea whether it's async (and doesn't care). So, in some situations, it's OK to elide the keywords (as described on my blog, but only when the method implementation is trivial. If there's any logic in the method, keep the async and await.
I'm working on a project which is using Kotlin, Spring Boot, Hibernate (all on latest version) and I would like to make it reactive with WebFlux framework from Spring.
Problem is that I can't use ReactiveCrudRepository because web app have to use Oracle database and therefore Hibernate. So I couldn't figured out a way how to use non blocking access to Oracle SQL database (only free frameworks).
My question is:
Is it possible to use this like that:
Casual CrudRepository which is blocking
Service which use corountines and returns everything as Mono
Service example code:
fun getAllLanguages(): Mono<Collection<ProgrammingLanguage>> = async { repository.findAll() }.asMono()
Afterwards there will be controller with:
fun listProgrammingLanguagesReactive() = mono(Unconfined) {
service.also { logger.info { "requesting list of programming languages" } }
.getAllLanguages()
.also { logger.info { "responding with list of programming languages" } }
}
This approach works but I'm not sure whether it will work all the time and whether this is not terrible practice and so on.
The problem with synchronous blocking API is that there will be a thread blocked for each API call. There is simply no way around it, coroutines or not.
Your approach is as good as any for providing asynchronous adapter to blocking API.
However, please consider following:
You may want to confine async { repository.findAll() } and similar blocking calls to a dedicated fixed ThreadPool/Dispatcher. While coroutines are cheap, remember, that repository.findAll() blocks actual underlying thread and you don't want to exhaust all thread in the CommonPool (which is used by async by default).
This is a useful practice, as you're limiting the number of threads/simultaneous blocking calls. If your fixed pool gets exhausted at some point, then incoming requests will be suspended, without blocking threads, until there are available threads in the pool to process them.
Currently, we are using StackExchange.Redis and, as it does not provides "blocking pops", we are doing as suggested on the documentation:
db.ListLeftPush(key, newWork, flags: CommandFlags.FireAndForget);
sub.Publish(channel, "");
What is the difference from this to the following?
db.ListLeftPushAsync(key, newWork);
sub.Publish(channel, "");
We know the purpose of the commands, what we would like to know is if it has any difference internally or any risk of behaving differently? (Execution order etc.)
There's a main difference comparing fire and forget vs calling an async operation and not awaiting it.
Fire and forget means that not only you're not waiting for the result but you don't care if it works or not, while an async operation may throw an exception once it has ended if something goes wrong.
In the other hand, when you issue a fire and forget command, StackExchange.Redis doesn't try to retrieve the command result internally, which is better if you just want the so-called fire and forget behavior when issuing commands.
You may check this difference if you open ConnectionMultiplexer source code and you see how ExecuteAsyncImpl / ExecuteSyncImpl methods are implemented:
// For example, ExecuteAsyncImpl...
if (message.IsFireAndForget)
{
TryPushMessageToBridge(message, processor, null, ref server);
return CompletedTask<T>.Default(null); // F+F explicitly does not get async-state
}
else
{
var tcs = TaskSource.CreateDenyExecSync<T>(state);
var source = ResultBox<T>.Get(tcs);
if (!TryPushMessageToBridge(message, processor, source, ref server))
{
ThrowFailed(tcs, ExceptionFactory.NoConnectionAvailable(IncludeDetailInExceptions, message.Command, message, server));
}
return tcs.Task;
}
Answer to some OP comment
Hi. Thanks for your answer. We know the purpose of the commands, what
we would like to know is if it has any differrence internally or any
risk of behaving differently (execution order etc.)
Since the async operation won't be finished when you publish the message on the Redis channel, it can happen that you publish the message and the operation gets executed never. You lose a lot of control.
When you send a fire and forget command, it mightn't be executed too, but you know that the try was done before you publish the channel's message. Therefore, you shouldn't use async operations to implement fire and forget pattern when using StackExchange.Redis.
You may check this other related Q&A: Stackexchange.redis does fire and forget guarantees delivery?
In integration tests, asynchronous processes (methods, external services) make for a very tough test code. If instead, I factored out the async part and create a dependency and replace it with a synchronous one for the sake of testing, would that be a "good thing"?
By replacing the async process with a synchronous one, am I not testing in the spirit of integration testing? I guess I'm assuming that integration testing refers to testing close to the real thing.
Nice question.
In a unit test this approach would make sense but for integration testing you should be testing the real system as it will behave in real-life. This includes any asynchronous operations and any side-effects they may have - this is the most likely place for bugs to exist and is probably where you should concentrate your testing not factor it out.
I often use a "waitFor" approach where I poll to see if an answer has been received and timeout after a while if not. A good implementation of this pattern, although java-specific you can get the gist, is the JUnitConditionRunner. For example:
conditionRunner = new JUnitConditionRunner(browser, WAIT_FOR_INTERVAL, WAIT_FOR_TIMEOUT);
protected void waitForText(String text) {
try {
conditionRunner.waitFor(new Text(text));
} catch(Throwable t) {
throw new AssertionFailedError("Expecting text " + text + " failed to become true. Complete text [" + browser.getBodyText() + "]");
}
}
We have a number of automated unit tests that send off asynchronous requests and need to test the output/results. The way we handle it is to actually perform all of testing as if it were part of the actual application, in other words asynchronous requests remain asynchronous. But the test harness acts synchronously: It sends off the asynchronous request, sleeps for [up to] a period of time (the maximum in which we would expect a result to be produced), and if still no result is available, then the test has failed. There are callbacks, so in almost all cases the test is awakened and continues running before the timeout has expired, but the timeouts mean that a failure (or change in expected performance) will not stall/halt the entire test suite.
This has a few advantages:
The unit test is very close to the actual calling patters of the application
No new code/stubs are needed to make the application code (the code being tested) run synchronously
Performance is tested implicitly: If the test slept for too short a period, then some performance characteristic has changed, and that needs looking in to
The last point may need a small amount of explanation. Performance testing is important, and it is often left out of test plans. The way these unit tests are run, they end up taking a lot longer (running time) than if we had rearranged the code to do everything synchronously. However this way, performance is tested implicitly, and the tests are more faithful to their usage in the application. Plus all of our message queueing infrastructure gets tested "for free" along the way.
Edit: Added note about callbacks
What are you testing? The behaviour of your class in response to certain stimuli? In which case don't suitable mocks do the job?
Class Orchestrator implements AsynchCallback {
TheAsycnhService myDelegate; // initialised by injection
public void doSomething(Request aRequest){
myDelegate.doTheWork(aRequest, this)
}
public void tellMeTheResult(Response aResponse) {
// process response
}
}
Your test can do something like
Orchestrator orch = new Orchestrator(mockAsynchService);
orch.doSomething(request);
// assertions here that the mockAsychService received the expected request
// now either the mock really does call back
// or (probably more easily) make explicit call to the tellMeTheResult() method
// assertions here that the Orchestrator did the right thing with the response
Note that there's no true asynch processing here, and the mock itself need have no logic other than to allow verification of the receipt of the correct request. For a Unit test of the Orchestrator this is sufficient.
I used this variation on the idea when testing BPEL processes in WebSphere Process Server.